CN106873917A - A kind of object storage system based on disk - Google Patents
A kind of object storage system based on disk Download PDFInfo
- Publication number
- CN106873917A CN106873917A CN201710103993.9A CN201710103993A CN106873917A CN 106873917 A CN106873917 A CN 106873917A CN 201710103993 A CN201710103993 A CN 201710103993A CN 106873917 A CN106873917 A CN 106873917A
- Authority
- CN
- China
- Prior art keywords
- data
- disk
- storage
- module
- data server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention discloses a kind of object storage system based on disk, including：Input module, meta data server, inner nuclear layer module and disk module, wherein, input module connection meta data server, meta data server connects disk module by inner nuclear layer module；Input module exports data to be stored to meta data server；Meta data server is based on object storage technology and processes data to be stored, marks the data after the treatment for object data, the storage space information of disk is obtained by inner nuclear layer module, based on storage space information storage object data.The present invention provides a meta data server, data will be processed to realize that object is stored, and data storage directly is entered into disk based on inner nuclear layer, without other buffer structures, can significantly improve the efficiency and speed of data transfer.
Description
Technical field
The present invention relates to a kind of object storage system based on disk, belong to Computer Storage field.
Background technology
With developing rapidly for information technology, data explosion has turned into an outstanding problem, big data storage and management skill
Art has been one of study hotspot in recent years.Traditional data storage and management mode has been difficult to meet big data storage and has existed
In the requirement of the aspects such as capacity, performance, storage efficiency and security.And most of data collecting system, such as radar return
Collection of signal data acquisition, digital video signal processing and code check satellite information signal high etc., require carries out reality to data
When transmission and storage, this acquisition rate just to data acquisition and memory system, transmission speed, storage speed, memory capacity with
And the aspect such as the reliability of data storage proposes requirement higher.Therefore, the research to high speed big data memory technology is got over
Hair seems important.Big data efficient storage is following storage application trend.
Most storage scheme major part is based on purchase commercial hardware or is compared into using some in the market
Ripe distributed storage technical scheme, the advantage for buying the hardware device scheme of business is stability higher, has the disadvantage price
Costliness, typically except this industry high to data correctness requirement of bank is relatively adapted to.The scheme of distributed storage technology
Software implementation method is used, multiple hardware devices is gathered together and is used, its memory capacity can be expanded, but in read-write
Slightly inferior and business storage scheme in terms of efficiency, in addition it is also necessary to constantly perfect.
The content of the invention
In order to solve the above problems, the present invention provides a kind of object storage system based on disk.
The technical solution adopted by the present invention is a kind of object storage system based on disk, including：Input module, metadata
Server, inner nuclear layer module and disk module, wherein, input module connection meta data server, meta data server is in
Stratum nucleare module connects disk module；Input module exports data to be stored to meta data server；It is right that meta data server is based on
As memory technology treatment data to be stored, mark the data after the treatment for object data, disk is obtained by inner nuclear layer module
Storage space information, based on storage space information storage object data.
Preferably, the input module also includes web units, for obtaining data to be stored by web browser.
Preferably, the meta data server parses data to be stored, regular by deblocking, base based on default piecemeal
In the attribute object information output of block data, with reference to object information and block data, it is object data to mark it.
Preferably, the inner nuclear layer module is based on the storage space information of kernel function reading disk and feeds back to metadata
Server, meta data server is based on storage space information generation piecemeal rule.
Preferably, the storage space information includes disk size and track location.
Beneficial effects of the present invention will be processed data to realize that object is deposited to provide a meta data server
Storage, directly disk is entered based on inner nuclear layer by data storage, without other buffer structures, can significantly improve data transfer
Efficiency and speed.
Brief description of the drawings
Fig. 1 show a kind of object storage system schematic diagram based on disk based on the embodiment of the present invention.
Specific embodiment
The present invention will be described with reference to embodiments.
Based on inventive embodiment, a kind of object storage system based on disk as shown in Figure 1, including：Input module, unit
Data server, inner nuclear layer module and disk module, wherein, input module connection meta data server, meta data server leads to
Cross inner nuclear layer module connection disk module；Input module exports data to be stored to meta data server；Meta data server base
In object storage technology treatment data to be stored, mark the data after the treatment for object data, obtained by inner nuclear layer module
The storage space information of disk, based on storage space information storage object data.
Input module is the source of data, and meta data server is the center of data processing, and inner nuclear layer module is first to accept
Data server order and the module operated in terms of kernel level carries out hardware/interface；Meta data server obtain data it
Afterwards, be first that data are processed because to realize object store, it is necessary to the data to being stored carry out attribute difference and
Storage mapping based on the difference；The first step parses data, obtains the information of some base attributes of data, such as data volume
Size, the label substance of data, type of data etc., based on this partial information, generate a mark, and binding marker and data are given birth to
Into an aggregate, the i.e. primitive of object storage（Object data）, then by inner nuclear layer module it is related in
Kernel function, obtains the storage information of disk, such as track location, disk size etc.；Object data is stored into disk, such as magnetic
Head is run with some order for specifying, is then successively stored in data, while the positional information of write-in is returned, by positional information
Mapped with object data, that is, completed the process of object storage.
It is closer to the bottom of operating system that is, direct in inner nuclear layer using disc driver as object data storage
The data received from server are saved on disk, the read-write of disk is made full use of in whole system running
Can, data are read and write with the efficiency that at least doubles because needing not move through the caching of system, reduce in be stored to disk
Transfer, the maximum performance of its performance is solely dependent upon the hardware index such as parameter of disk.
The input module also includes web units, for obtaining data to be stored by web browser.
Input module is additionally provided with long-range data transmission unit in addition to the data output that will can be locally stored, and passes through
Network receives the data of web browser output.
The meta data server parses data to be stored, based on default piecemeal rule by deblocking, based on piecemeal
The attribute object information output of data, with reference to object information and block data, it is object data to mark it.
Piecemeal is carried out to data, is then marked, be the basic content of present distributed document, and the rule of piecemeal
The general type using data（Put together with category information）, size, clock information（Data in certain hour are grouped together）,
The label of data（I.e. there is data label to illustrate data content in itself, is then grouped together with label）, based on above-mentioned regular piecemeal
Data, object information output（For example belong to that type, which piecemeal, the piecemeal time etc.）, with reference to object information and
Data in itself, then form object data.
The inner nuclear layer module is based on the storage space information of kernel function reading disk and feeds back to meta data server,
Meta data server is based on storage space information generation piecemeal rule.
Inner nuclear layer module is obtained after the information of storage attribute of disk, feeds back to meta data server, server according to
Storage information is analyzed, and such as disk size is 100MB, and the data of this input are 96, then can be divided into 2 blocks each
48, then remaining 4mb then as store other suitable data blocks piecemeal because be according to object properties carry out mapping correspondence
Mode, so storage will not also cause too much influence to the continuity of data in different disk.
The storage space information includes disk size and track location.
The above, simply presently preferred embodiments of the present invention, the invention is not limited in above-mentioned implementation method, as long as
It reaches technique effect of the invention with identical means, should all belong to protection scope of the present invention.In protection model of the invention
Its technical scheme and/or implementation method can have a variety of modifications and variations in enclosing.
Claims (5)
1. a kind of object storage system based on disk, it is characterised in that including：
Input module, meta data server, inner nuclear layer module and disk module, wherein,
Input module connects meta data server, and meta data server connects disk module by inner nuclear layer module；
Input module exports data to be stored to meta data server；
Meta data server is based on object storage technology and processes data to be stored, marks the data after the treatment for object data,
The storage space information of disk is obtained by inner nuclear layer module, based on storage space information storage object data.
2. a kind of object storage system based on disk according to claim 1, it is characterised in that the input module is also
Including web units, for obtaining data to be stored by web browser.
3. a kind of object storage system based on disk according to claim 1, it is characterised in that the Metadata Service
Device parses data to be stored, based on default piecemeal rule by deblocking, the attribute object information output based on block data,
With reference to object information and block data, it is object data to mark it.
4. a kind of object storage system based on disk according to claim 1, it is characterised in that the inner nuclear layer module
Storage space information based on kernel function reading disk simultaneously feeds back to meta data server, and it is empty that meta data server is based on storage
Between information generation piecemeal rule.
5. a kind of object storage system based on disk according to claim 1 or 4, it is characterised in that the storage is empty
Between information include disk size and track location.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710103993.9A CN106873917A (en) | 2017-02-24 | 2017-02-24 | A kind of object storage system based on disk |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710103993.9A CN106873917A (en) | 2017-02-24 | 2017-02-24 | A kind of object storage system based on disk |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106873917A true CN106873917A (en) | 2017-06-20 |
Family
ID=59167705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710103993.9A Pending CN106873917A (en) | 2017-02-24 | 2017-02-24 | A kind of object storage system based on disk |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106873917A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108646981A (en) * | 2018-04-28 | 2018-10-12 | 深圳市茁壮网络股份有限公司 | A kind of data buffer storage node administration method, data cache method and caching management node |
| CN110109613A (en) * | 2019-03-14 | 2019-08-09 | 南京时沃信息科技有限公司 | A kind of object storage system based on disk |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101236530A (en) * | 2008-01-30 | 2008-08-06 | 清华大学 | High speed cache replacement policy dynamic selection method |
| CN101295310A (en) * | 2008-06-12 | 2008-10-29 | 上海交通大学 | Method for storing data and metadata on magnetic disk |
| CN101452422A (en) * | 2007-11-29 | 2009-06-10 | 大唐移动通信设备有限公司 | Chip data read-write method, corresponding apparatus and system |
| CN102567442A (en) * | 2011-10-25 | 2012-07-11 | 曙光信息产业(北京)有限公司 | Method for synchronizing metadata and disks in distributed file system |
| CN103530066A (en) * | 2013-09-16 | 2014-01-22 | 华为技术有限公司 | Data storage method, device and system |
| CN103870202A (en) * | 2012-12-10 | 2014-06-18 | 上海优刻得信息科技有限公司 | Distributed storage method and system of block device |
-
2017
- 2017-02-24 CN CN201710103993.9A patent/CN106873917A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101452422A (en) * | 2007-11-29 | 2009-06-10 | 大唐移动通信设备有限公司 | Chip data read-write method, corresponding apparatus and system |
| CN101236530A (en) * | 2008-01-30 | 2008-08-06 | 清华大学 | High speed cache replacement policy dynamic selection method |
| CN101295310A (en) * | 2008-06-12 | 2008-10-29 | 上海交通大学 | Method for storing data and metadata on magnetic disk |
| CN102567442A (en) * | 2011-10-25 | 2012-07-11 | 曙光信息产业(北京)有限公司 | Method for synchronizing metadata and disks in distributed file system |
| CN103870202A (en) * | 2012-12-10 | 2014-06-18 | 上海优刻得信息科技有限公司 | Distributed storage method and system of block device |
| CN103530066A (en) * | 2013-09-16 | 2014-01-22 | 华为技术有限公司 | Data storage method, device and system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108646981A (en) * | 2018-04-28 | 2018-10-12 | 深圳市茁壮网络股份有限公司 | A kind of data buffer storage node administration method, data cache method and caching management node |
| CN110109613A (en) * | 2019-03-14 | 2019-08-09 | 南京时沃信息科技有限公司 | A kind of object storage system based on disk |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10831734B2 (en) | Update-insert for key-value storage interface | |
| CN103635900B (en) | Time-based data partitioning | |
| CN108519862A (en) | Storage method, device, system and the storage medium of block catenary system | |
| CN110008136A (en) | The method of management flash memory module and relevant flash controller and electronic device | |
| CN101777017B (en) | Rapid recovery method of continuous data protection system | |
| US9996557B2 (en) | Database storage system based on optical disk and method using the system | |
| CN109815228A (en) | Creation method, device, computer equipment and the readable storage medium storing program for executing of database table | |
| CN102301349A (en) | Database operation-aware striping technique | |
| US11934371B2 (en) | Data processing method and apparatus, and computer-readable storage medium | |
| CN106255953A (en) | The method and system of the forward direction reference record in persistent data memory bank | |
| CN103955530A (en) | Data reconstruction and optimization method of on-line repeating data deletion system | |
| CN103425785A (en) | Data storage system and user data storage and reading method thereof | |
| WO2021073241A1 (en) | Disk storage-based data reading method and device, and apparatus | |
| CN105808163A (en) | Method for accessing to shingled magnetic recording SMR hard disk and server | |
| CN103049224A (en) | Method, device and system for importing data into physical tape | |
| US8468007B1 (en) | Emulating a peripheral mass storage device with a portable device | |
| CN101556802A (en) | RAID array transposing method and device thereof | |
| CN106873917A (en) | A kind of object storage system based on disk | |
| CN100410945C (en) | Method and system for implementing forum | |
| CN102193873B (en) | Method of quickly reading and writing mass data file | |
| CN106484763B (en) | System and method for merging data | |
| CN103309619A (en) | NAND Flash memory data storage method | |
| CN107957850A (en) | Data storage system with virtual block and disk array structure and management method thereof | |
| CN105095352B (en) | Data processing method and device applied to distributed system | |
| CN104731779A (en) | Real-time file system data organization and management method facing real-time databases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190912 Address after: 100089 Floor 1-4, No. 2 Building, No. 9 Courtyard, Dijin Road, Haidian District, Beijing Applicant after: Beijing Toyou Feiji Electronics Co., Ltd. Address before: 518000 Guangdong city of Shenzhen province Nanshan District South Road Fiyta Technology Building Room 1402 Applicant before: Shenzhen City Rui Bo deposit information technology Co. Ltd. |
|
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170620 |