CN105930506A - Object retrieval method and object storage system - Google Patents

Object retrieval method and object storage system Download PDF

Info

Publication number
CN105930506A
CN105930506A CN201610305909.7A CN201610305909A CN105930506A CN 105930506 A CN105930506 A CN 105930506A CN 201610305909 A CN201610305909 A CN 201610305909A CN 105930506 A CN105930506 A CN 105930506A
Authority
CN
China
Prior art keywords
information
memory space
character
cloud storage
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610305909.7A
Other languages
Chinese (zh)
Inventor
吴兴义
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LeTV Holding Beijing Co Ltd
LeTV Cloud Computing Co Ltd
Original Assignee
LeTV Holding Beijing Co Ltd
LeTV Cloud Computing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LeTV Holding Beijing Co Ltd, LeTV Cloud Computing Co Ltd filed Critical LeTV Holding Beijing Co Ltd
Priority to CN201610305909.7A priority Critical patent/CN105930506A/en
Publication of CN105930506A publication Critical patent/CN105930506A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation

Abstract

An embodiment of the invention provides an object retrieval method in an object storage system. The method comprises the steps of generating a retrieval condition in a retrieval SQL according to a storage space identifier and object information in an object request; retrieving a database through the retrieval SQL to obtain storage information of objects; obtaining a plurality of the objects through the storage information and returning the objects to a client, wherein the database stores attribute information of the objects; and performing updating during object creation, modification and deletion. According to the method, the object storage information is directly obtained without traversing all the objects in the storage space, so that the overall system efficiency is improved; and wildcard setting is supported in the object information, so that multiple pieces of object data meeting the conditions can be retrieved once, the situation that the client needs to submit a plurality of object requests for obtaining the object data in certain scenes is avoided, and the overall system efficiency is further improved. Meanwhile, the invention provides the object storage system supporting and having the retrieval method.

Description

Object search method and object storage system
Technical field
It relates to cloud storage field, it is specifically related to a kind of object search method and object storage is System.
Background technology
Object is stored in the spring tide of the Internet and receives wide concern, and has obtained rapid development. Object storage is different from file storage, and object is not organized with the structure of directory hierarchy.Each right As being all located at being referred to as the same level in the flat space of storage pool, each element of every one-level exists Having unique mark in storage system, user accesses container or object by this mark. Flat data organizational structure is used to abandon the file of nesting, it is to avoid to safeguard huge directory tree. Big data and Internet era, the file/object of storage exceedes up to a million, in the unit interval Access times and concurrent visit capacity have also reached unprecedented magnitude, in this case, catalogue Tree brings the biggest expense and problems can to storage system, becomes the bottleneck of system.Other one The individual factor allowing object storage shine vitality is, object storage does not reoffer the compatible interface of POSIX (such as open, close, read, write etc.), but HTTP can be passed through easily Restful api interface and object interact, and carry out uploading download by PUT and GET, pass through DELETE deletes.
Object store in because the data of user are stored entirely in far-end, user if it is desired to from Oneself file makes a look up, and can bother very much, and user needs the mode using traversal to memory space (bucket) the whole objects in carry out a traversing operation, it is thus achieved that all of listed files, and Do primary retrieval according to this list, to obtain the object oneself wanting to search, have three to compare here Bigger expense, first be memory space (bucket) traversal time, for HTTP request each time, It is to have number to limit (typically that object storage service end at most returns to the object number of user 1000), if the data traversal operation in the memory space of user (bucket) has exceeded this Limit, then need to initiate multiple HTTP request, and therefore this action is typically the slowest, and second, Then object storage cluster service end needs first all of object name in memory space (bucket) to be sealed Dressing up an xml document and return to user, user resolves this xml document and is also required to elapsed time, Moreover user needs the xml document resolved to be directly proportional with user object data, the 3rd, resolve After xml, user also needs to go to retrieve the file required to look up from a lot of files, and this also brings along Expense.Obviously, user needs an API preferably carrying out retrieving by object name.
It addition, from the point of view of service end, service end please for memory space (bucket) traversal Ask and be also required to consume a lot of resource and go to carry out the encapsulation of all object name, and massive band width is to user Sending data, these resources actually slattern.
Summary of the invention
In view of this, the disclosure provides a kind of object search method and object storage system, with at least Solve one of the problems referred to above.
According to the first aspect of the disclosure, the disclosure provides a kind of object search method, including: build Stand from client to the communication connection of cloud storage end;Receive the object requests of described client, described Object requests comprises memory space identifier and object information;According to described memory space identifier Retrieval SQL is generated with described object information;By described retrieval SQL searching database, obtain object Storage information;The multiple object of acquisition of information is stored according to described object;And by the plurality of object Return to described client;
Wherein, described cloud storage end storage has multiple memory space and multiple object, each described right As being stored in a described memory space, described memory space identifier is a described storage Unique mark in space, described database purchase has the attribute of multiple objects of described cloud storage end to believe Breath.
Preferably, also including: when creating, revising or delete an object, correspondence increases, more New or delete the record of described data base.
Preferably, described object information is regular expression.
Preferably, described regular expression comprises the first character and the second character, described first character For metacharacter, described second character is for mating English or the numerical character of object name.
Preferably, described data base is internal memory type data base.
According to the second aspect of the disclosure, the disclosure provides a kind of object storage system, including multiple Client and cloud storage end,
Described client includes:
Communication connection is set up with described cloud storage end;Submit object requests to, described object requests is wrapped Containing memory space identifier and object information;And accept multiple objects that described cloud storage end returns;
Described cloud storage end includes:
Set up and the communication connection of described client;Receive described object requests;According to described storage Spatial identifier and described object information generate retrieval SQL;
By described retrieval SQL searching database, obtain object storage information;According to described object The storage multiple object of acquisition of information;And the plurality of object is returned to described client;
Wherein, described cloud storage end storage has multiple memory space and multiple object, each described right As being stored in a described memory space, described memory space identifier is a described storage Unique mark in space, described database purchase has the attribute of multiple objects of described cloud storage end to believe Breath.
Preferably, described cloud storage end also includes: when creating, revising or delete an object, The corresponding record increasing, update or delete described data base.
Preferably, described object information is canonical formula.
Preferably, described regular expression comprises the first character and the second character, described first character For metacharacter, described second character is for mating the English of object name or numerical character.
Preferably, described data base is internal memory type data base.
In the disclosed embodiments, it is provided that a kind of method retrieving object in object storage system, The retrieval bar in retrieval SQL is generated according to the memory space identifier in object requests and object information Part, obtains the storage information of object by retrieval SQL searching database, many by storage information acquisition Object is also returned to client by individual object, and wherein database purchase has the attribute information of object, Object Creation, update when revising and delete.The method is without traveling through all objects in memory space Directly obtain object storage information, which thereby enhance the whole efficiency of system.And due to object information Middle support asterisk wildcard sets, and can disposably retrieve multiple object data meeting condition, it is to avoid certain Under a little scenes, client need to submit multiple object requests to for obtaining object data, improves system further Whole efficiency.The disclosure provides the object storage system supporting have above-mentioned search method simultaneously.
Accompanying drawing explanation
By referring to the following drawings description to disclosure embodiment, the disclosure above-mentioned and other Objects, features and advantages will be apparent from, in the accompanying drawings:
Fig. 1 is the schematic diagram of the object storage system according to disclosure embodiment;
Fig. 2 is the flow chart of the object search method according to disclosure embodiment;
Fig. 3 is the signal that the client according to disclosure embodiment and cloud storage end set up communication connection Figure.
Detailed description of the invention
Below based on embodiment, the disclosure is described, but the disclosure is not restricted to these Embodiment.During details the most of this disclosure describes, detailed describe some specific detail portion Point.The description not having these detail sections for a person skilled in the art can also understand this completely Open.In order to avoid obscuring the essence of the disclosure, known method, process, flow process are the most in detail Narration.Additionally accompanying drawing is not necessarily drawn to scale.
In the description of the disclosure, it is to be understood that term " first ", " second " etc. are only For descriptive purposes, and it is not intended that instruction or hint relative importance.
Fig. 1 is the schematic diagram of the object storage system according to disclosure embodiment.Described objective system Including client 100 and cloud storage end 101.Here, client can be mobile phone A PP, or Video system, the front end of electricity quotient system system or rear end program.Cloud storage end then can include that software and hardware is tied The cloud service system closed.
Client 100 and cloud storage end 101 set up communication connection, send request by this communication connection, After cloud storage termination receives this request, after processing request, relevant data are returned to client 100. Here it is possible to the memory area of cloud storage end is regarded as a network SAN (Storage Area Network), i.e. the high-speed storage network of a centralized management, by multi-provider store system, Storage management software, apps server and network hardware composition.Cloud storage end uses storage sky Between (bucket) and objects on data store.Memory space (bucket) can be understood For container, each object is deposited in a vessel, as shown in Fig. 1 102, and cloud storage end Memory area is divided into several bucket (equal in magnitude), each bucket deposits multiple Object.Bucket uses memory space identifier to identify as unique, due to varying in size of object, Even if so the object number that the bucket of identical capacity deposits is the most different.
Preferably, as it is shown in figure 1, cloud storage end 101 also includes a data base 103, it is right to store As the related information with memory space, the data-base recording as shown in figure upper 1 represents that memory space identifies In the bucket of symbol 1, storage has object 1, object 2, object 3, and have recorded each object at bucket In storage position.So, when searching object by object oriented, it is not necessary to traversal bucket In all objects, direct searching database 103 can obtain object information, thus alleviate cloud storage The performance pressures of end, improves overall performance.
Fig. 2 is the flow chart of the object search method according to disclosure embodiment.Described search method Including step 201-206.
In step 201, set up from client to the communication connection of cloud storage end.
Fig. 3 shows that mobile phone A PP and cloud storage end set up the process that secure communication connects.Mark Note 30 is client, and labelling 31 is right management server, is managed by client access authority Server.When needs carry out high in the clouds storage, client passes through step 301 to rights management service Device sends a high in the clouds authorization requests, after right management server receives, forwards to cloud storage end, After cloud storage end verifies, return temporary Authorization ID or access token (Access token), Right management server receives this mandate ID or access token, returns to client.Client will This mandate ID or access token are buffered in this locality, and are deposited by this mandate ID or access token and cloud Chu Duan is communicatively coupled.In order to ensure to authorize ID or access token are not abused or steal, such Authorizing ID or access token to be generally provided with time restriction, having spent the scheduled time ceases to be in force automatically.
In step 202., receive the object requests of client, object requests comprises memory space Identifier and object information.
Object information can be and object oriented, object type, relevant each of data of object storage Category information.Preferably, object information is canonical formula.Such as, for mating the canonical of object oriented Formula, form class is similar to " myobject* ", and wherein " * " character is that (metacharacter is canonical to metacharacter Formula has special implication, such as *, ^,!,@, # etc.), " myobject " is common word Symbol." myobject* " represents the object oriented taken the lead with " myobject ", Myobject1, myobject2, myobjects1 etc. comprise wherein.
In step 203, retrieval SQL is generated according to memory space identifier and object information.
In step 204, by retrieval SQL searching database, obtain object storage information.
Here, lane database storage has the related information of memory space and object, Object Creation, When amendment or deletion, the corresponding record increasing, updating or delete in data base.Due to memory space With the relation that object shows as one-to-many, it is possible to use relational data library storage.In order to imitate Rate is higher, it is also possible to store data in internal memory by HASH table, such as, by HASH MAP Storage object information.Or use the REDIS increased income to store data.
In step 203 and 204, generate retrieval SQL according to memory space identifier and object information Search condition, similar where object like " object% " and bracketId= “bracket”.Obtained the storage information of the object meeting condition by retrieval SQL, i.e. get Object storage position in memory space.If it is not, return to user's no data.
In step 205, the multiple object of acquisition of information is stored according to object.
In step 206, multiple objects are returned to client.
In step 205 and 206, by the storage information acquisition object of object, and object is returned Back to client.
In the disclosed embodiments, it is provided that a kind of method retrieving object in object storage system, The retrieval bar in retrieval SQL is generated according to the memory space identifier in object requests and object information Part, obtains the storage information of object by retrieval SQL searching database, many by storage information acquisition Object is also returned to client by individual object, and wherein database purchase has the attribute information of object, Object Creation, update when revising and delete.The method is straight without traveling through all objects in bucket Obtain to obtain object storage information, which thereby enhance the whole efficiency of system.And due in object information Support asterisk wildcard sets, and can disposably retrieve multiple object data meeting condition, it is to avoid some Under scene, client need to submit multiple object requests to, improves the whole efficiency of system further.These public affairs Open and the object storage system supporting have above-mentioned search method is provided simultaneously.
Flow chart in accompanying drawing, block diagram illustrate the system of disclosure embodiment, method, device Possible System Framework, function and operation, the square frame on flow chart and block diagram can represent one, Program segment or only one section of code, described, program segment and code are all used to realize regulation logic The executable instruction of function.It should also be noted that the described executable instruction realizing regulation logic function Can reconfigure, thus generate new and program segment.Therefore the square frame of accompanying drawing and square frame order It is used only to preferably illustrate process and the step of embodiment, and should be in this, as to invention itself Restriction.
Each or unit of system can pass through hardware, firmware or software and realize.Software such as includes Use the encoding procedure that the various programming languages such as JAVA, C/C++/C#, SQL are formed.Although in method And provide the step of disclosure embodiment and the order of step in method legend, but described step The executable instruction of the rapid logic function realizing regulation can reconfigure, thus generates new step. The step that the order of described step also should not be restricted solely in described method and method legend is suitable Sequence, can according to function need be adjusted at any time.Such as by parallel for some step therein or Perform according to reverse order.
System and method according to the disclosure can be deployed on single or multiple server.Such as, Private server can be formed by different deployment respectively on a different server.Or, can The functional unit identical with distributed deployment on multiple servers or system, to alleviate load pressure Power.Described server is included but not limited at same LAN and is connected by Internet Multiple PCs, PC server, rolling reamer machine, supercomputer etc..
The foregoing is only preferred embodiment of the present disclosure, be not limited to the disclosure, for this For skilled person, the disclosure can have various change and change.All spirit in the disclosure Any modification, equivalent substitution and improvement etc. with being made within principle, should be included in the disclosure Within protection domain.

Claims (10)

1. an object search method, including:
Set up from client to the communication connection of cloud storage end;
Receive the object requests of described client, described object requests comprises memory space identifier And object information;
Retrieval SQL is generated according to described memory space identifier and described object information;
By described retrieval SQL searching database, obtain object storage information;
The multiple object of acquisition of information is stored according to described object;And
The plurality of object is returned to described client;
Wherein, described cloud storage end storage has multiple memory space and multiple object, each described right As being stored in a described memory space, described memory space identifier is a described storage Unique mark in space, described database purchase has the attribute of multiple objects of described cloud storage end to believe Breath.
Object search method the most according to claim 1, also includes: create, amendment or When deleting an object, the corresponding record increasing, update or delete described data base.
Object search method the most according to claim 1, wherein, described object information is just Then expression formula.
Object search method the most according to claim 3, wherein, described regular expression bag Containing the first character and the second character, described first character is metacharacter, described second character for Join English or the numerical character of object name.
Object search method the most according to claim 1, wherein, described data base is internal memory Type data base.
6. an object storage system, including multiple clients and cloud storage end,
Described client includes:
Communication connection is set up with described cloud storage end;
Submit object requests to, described object requests comprises memory space identifier and object information; And
Accept multiple objects that described cloud storage end returns;
Described cloud storage end includes:
Set up and the communication connection of described client;
Receive described object requests;
Retrieval SQL is generated according to described memory space identifier and described object information;
By described retrieval SQL searching database, obtain object storage information;
The multiple object of acquisition of information is stored according to described object;And
The plurality of object is returned to described client;
Wherein, described cloud storage end storage has multiple memory space and multiple object, each described right As being stored in a described memory space, described memory space identifier is a described storage Unique mark in space, described database purchase has the attribute of multiple objects of described cloud storage end to believe Breath.
Object retrieval system the most according to claim 6, wherein, described cloud storage end also wraps Including: when creating, revising or delete an object, correspondence increases, updates or delete described data The record in storehouse.
Object retrieval system the most according to claim 6, wherein, described object information is just Then formula.
Object retrieval system the most according to claim 8, wherein, described regular expression bag Containing the first character and the second character, described first character is metacharacter, and described second character is for being used for The English of coupling object name or numerical character.
Object retrieval system the most according to claim 6, wherein, described data base is interior Deposit type data base.
CN201610305909.7A 2016-05-10 2016-05-10 Object retrieval method and object storage system Pending CN105930506A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610305909.7A CN105930506A (en) 2016-05-10 2016-05-10 Object retrieval method and object storage system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610305909.7A CN105930506A (en) 2016-05-10 2016-05-10 Object retrieval method and object storage system

Publications (1)

Publication Number Publication Date
CN105930506A true CN105930506A (en) 2016-09-07

Family

ID=56835576

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610305909.7A Pending CN105930506A (en) 2016-05-10 2016-05-10 Object retrieval method and object storage system

Country Status (1)

Country Link
CN (1) CN105930506A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704192A (en) * 2019-09-30 2020-01-17 的卢技术有限公司 Diversified data cloud storage method and system

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102902763A (en) * 2012-09-21 2013-01-30 东软集团股份有限公司 Method and device for relating and retrieving information processing data and processing information tasks
CN103150202A (en) * 2013-03-15 2013-06-12 汉柏科技有限公司 Method for allowing CloudStack to be compatible with virtual machine existing in vCenter
CN103348344A (en) * 2010-12-30 2013-10-09 脸谱公司 Composite term index for graph data
CN103678570A (en) * 2013-12-10 2014-03-26 中国人民解放军理工大学 Multi-level storage and recovery method and system of journal file in cloud environment
CN104090901A (en) * 2013-12-31 2014-10-08 腾讯数码(天津)有限公司 Method, device and server for processing data
CN104199831A (en) * 2014-07-31 2014-12-10 深圳市腾讯计算机系统有限公司 Information processing method and device
CN105550206A (en) * 2015-12-01 2016-05-04 珠海多玩信息技术有限公司 Version control method and device for structured query language

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103348344A (en) * 2010-12-30 2013-10-09 脸谱公司 Composite term index for graph data
CN102902763A (en) * 2012-09-21 2013-01-30 东软集团股份有限公司 Method and device for relating and retrieving information processing data and processing information tasks
CN103150202A (en) * 2013-03-15 2013-06-12 汉柏科技有限公司 Method for allowing CloudStack to be compatible with virtual machine existing in vCenter
CN103678570A (en) * 2013-12-10 2014-03-26 中国人民解放军理工大学 Multi-level storage and recovery method and system of journal file in cloud environment
CN104090901A (en) * 2013-12-31 2014-10-08 腾讯数码(天津)有限公司 Method, device and server for processing data
CN104199831A (en) * 2014-07-31 2014-12-10 深圳市腾讯计算机系统有限公司 Information processing method and device
CN105550206A (en) * 2015-12-01 2016-05-04 珠海多玩信息技术有限公司 Version control method and device for structured query language

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704192A (en) * 2019-09-30 2020-01-17 的卢技术有限公司 Diversified data cloud storage method and system

Similar Documents

Publication Publication Date Title
CN103812939B (en) Big data storage system
CN106294352B (en) A kind of document handling method, device and file system
CN106030573A (en) Implementation of semi-structured data as a first-class database element
CN103338242B (en) A kind of mixed cloud storage system based on multi-level buffer and method
CN111427847B (en) Indexing and querying method and system for user-defined metadata
CN104281414B (en) Distributed file system and its small documents access method
JP2012098934A (en) Document management system, method for controlling document management system and program
CN103778255A (en) Distributed file system and data distribution method thereof
WO2012149884A1 (en) File system, and method and device for retrieving, writing, modifying or deleting file
CN109542861B (en) File management method, device and system
CN107026876A (en) A kind of file data accesses system and method
CN106294870A (en) Object-based distributed cloud storage method
JP2012531688A (en) Method for accessing file system file according to metadata, and apparatus for implementing the method
Wang et al. Retrieving and indexing spatial data in the cloud computing environment
CN104794150A (en) Cloud storage model and management method based on space knowledge cloud environment
CN106027615A (en) Object storage method and system
KR101790766B1 (en) Method, device and terminal for data search
CN106599111A (en) Data management method and storage system
CN106201351A (en) A kind of storage method based on object storage and server
Changtong An improved HDFS for small file
CN106095341A (en) Method for managing object and Object Management System
CN104391947B (en) Magnanimity GIS data real-time processing method and system
CN105930506A (en) Object retrieval method and object storage system
CN103853612A (en) Method for reading data based on digital family content under distributed storage
CN109558082B (en) Distributed file system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20160907

WD01 Invention patent application deemed withdrawn after publication