CN103942262A - Method for optimal allocation of metadata in video storage system - Google Patents
Method for optimal allocation of metadata in video storage system Download PDFInfo
- Publication number
- CN103942262A CN103942262A CN201410114166.6A CN201410114166A CN103942262A CN 103942262 A CN103942262 A CN 103942262A CN 201410114166 A CN201410114166 A CN 201410114166A CN 103942262 A CN103942262 A CN 103942262A
- Authority
- CN
- China
- Prior art keywords
- index node
- node
- reserved
- index
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method for optimal allocation of metadata in a video storage system. A system structure comprises an index node reservation module and an index node allocation module, wherein the index node reservation module is used for setting whether the reservation function is enabled or not and reserving node parameters, and the index node allocation module is used for allocating index nodes according to the reservation node parameters. When a file is created under a directory, linear searching for idle and usable index nodes is conducted from the index nodes of a parent directory firstly, the index nodes are directly allocated to the file to be used if found, searching for the idle index nodes from the starting position of an index node list is not needed, and then extra magnetic disc IO expenses and time consumed for track seeking are reduced during updating of the index nodes.
Description
Technical field
The present invention relates to the optimization distribution technique of metadata in a kind of Video Storage System, this technology can be used for video information storage system in field of video monitoring.
Background technology
In video monitoring system, by the video information of collection, the form with file stores in the storage system of file server conventionally, thus file system write with a brush dipped in Chinese ink and the operation such as the allocation manager of data block all can produce significant impact to memory property.Because video file format generally can not change, and code check maintenance is constant, and therefore, the read-write operation of storage need to meet certain bandwidth, and requires bandwidth constant, otherwise image acquisition or playback just there will be frame losing phenomenon.In addition, Video Storage System need to possess unique read-write mode.Digital video code or video server are in Streaming Media mode by data write storage device, and image and the picture of control point passback are in real time kept in memory device in Streaming Media mode, and playback workstation reads the video file of having stored in Streaming Media mode.Small data piece or file-level read-write mode that in this read-write mode and file server system, storage adopts have larger difference, cause the metadata " fragmentation " that causes file under same directory when file system is multi-channel video document creation metadata.Because video information continues to write the frequent updating that causes metadata, and then brought serious IO expense and disk seek time to extend.
Therefore, while utilizing file server storage multi-channel video information, there is certain IO performance bottleneck, even if adopt specific store subsystem write fashionable or cannot meet performance requirement when processing hundreds of road video information.
In storage system, keep the relevant data block overall time that can reduce magnetic head tracking close to each other.The locality of data exerts an influence to writing of single erase block, can accelerate the speed that file rewrites, thereby to reduce as far as possible fragment be necessary.When video monitoring system adopts hundreds of road Video Monitoring Terminal, simultaneously during concurrent passback video flowing, be equivalent to simultaneously, to data writing in a hundreds of file, now can upgrade the access time to the index node of file, the while also can be upgraded the access time of corresponding parent directory.Index node is buffered in buffer zone, and is periodically write with a brush dipped in Chinese ink disk.When the index node of file is irregular distribution, mechanical disk by needs repeatedly tracking to read index node, not only increased the load of IO, the IO performance bottleneck problem that simultaneously also can cause index node to cause.
Summary of the invention
This patent is directed to the problem that prior art exists, and has designed a kind of video information metadata optimizing distribution method that is directed to, and is the optimizing distribution method of metadata in a kind of Video Storage System specifically.
The optimizing distribution method of metadata in described Video Storage System disclosed by the invention, its technical scheme that solve the technical problem employing is as follows: the architecture of the optimizing distribution method of described metadata comprises the reserved module of index node and index node distribution module; Wherein,
Index node is reserved module, and for arranging, whether reserved function enables and reserved index node number, and the maximum reserved parameter of index node is that the index number in the index node list in piece group subtracts 1;
Index node distribution module, is responsible for according to reserved parametric distribution index node, and index node distribution module receives after reserved parameter, and the index node space of more reserved files when createing directory index node, for storing the file inode under same directory.
When creating file under catalogue, first from the index node of this document parent directory, start the idle index node of linear search, if found, just directly distribute to file use, if fail to find available index node, so just adopt original linear directory node allocation strategy.And, be the reserved index node region of each catalogue, be not mandatory reservation.First new file under catalogue can attempt allocation index node from reserved area, but these index nodes also can be distributed to the file in other catalogue, also can distribute to other catalogue.
In a kind of Video Storage System disclosed by the invention, the beneficial effect of the optimizing distribution method of metadata is: adopt optimizing distribution method of the present invention, when creating file under catalogue, file system no longer starts to search for idle index node from index node list reference position, but start the idle index node of linear search from the index node of this document parent directory, thereby the time loss that when reducing index node and upgrading, extra disk I/O expense and tracking cause.
Accompanying drawing explanation
Accompanying drawing 1 is index node predistribution schematic diagram in the present invention;
Accompanying drawing 2 is index node predistribution schematic flow sheet in the present invention.
Embodiment
Below in conjunction with accompanying drawing, the optimizing distribution method of metadata in Video Storage System disclosed in this invention is described in further details.
Due to, in storage system, keep the relevant data block overall time that can reduce magnetic head tracking close to each other.The locality of data exerts an influence to writing of single erase block, can accelerate the speed that file rewrites, thereby to reduce as far as possible fragment be necessary.When video monitoring system adopts hundreds of road Video Monitoring Terminal, simultaneously during concurrent passback video flowing, be equivalent to simultaneously, to data writing in a hundreds of file, now can upgrade the access time to the index node of file, the while also can be upgraded the access time of corresponding parent directory.Index node is buffered in buffer zone, and is periodically write with a brush dipped in Chinese ink disk.When the index node of file is irregular distribution, mechanical disk by needs repeatedly tracking to read index node, not only increased the load of IO, the IO performance bottleneck problem that simultaneously also can cause index node to cause.
In order to address the above problem, the present invention is optimized the allocation flow of metadata in storing process, the architecture of this optimizing distribution method comprises the reserved module of index node and index node distribution module, wherein the reserved module of index node is used for arranging whether reserved function enables and reserved index node number, and the maximum reserved parameter of index node is that the index number in the index node list in piece group subtracts 1;
Index node distribution module is responsible for according to reserved parametric distribution index node; Index node distribution module receives after reserved parameter, and the index node space of more reserved files when createing directory index node, for storing the file inode under same directory.When creating file under catalogue, file system no longer starts to search for idle index node from index node table reference position, but starts the idle index node of linear search from the index node of this document parent directory.Thereby the time loss that when reducing index node and upgrading, extra disk I/O expense and tracking cause.
Embodiment: Fig. 1 is index node predistribution schematic diagram in the selected embodiment of the present invention, the reserved mould of described index node arranges reserved index node parameter Pre_Num=3, described index node distribution module receives reserved parameter, according to reserved parameter Pre_Num=3 allocation index node (index assignment) in tri-parent directories of A, B, C that create.
In Video Storage System during file system creation, adopt optimizing distribution method of the present invention to complete the assigning process of index node, index node predistribution flow process as shown in Figure 2:
1, reserved node parameter (index node is reserved parameter) Pre_Num is set, then this parameter passes to the reserved module of index node;
2, the inode_preload function of the reserved module of index node receives reserved node parameter Pre_Num, after relevant setting that completes reserved node parameter, relevant information is passed to index node distribution module;
3, first the inode_alloc function in index node distribution module will verify whether be catalogue file, if continue to carry out, otherwise return;
4, index node distribution module starts to join index node list for each piece component, need to obtain index node bitmap, then according to reserved node parameter, reserved index node is set in index node bitmap;
5, after being all assigned with, all index nodes restart to distribute for next piece group, until all groups are all assigned.
After completing index node distribution, while carrying out catalogue file establishment, file system no longer starts to search for idle index node from index node table reference position, but first from the index node of parent directory, start to find available index node, if find and just directly distribute to file use, if fail to find available index node, so just adopt original linear directory node allocation strategy.Thereby the time loss that when reducing index node and upgrading, extra disk I/O expense and tracking cause.
Remove outside feature technology of the present invention, be the known technology of those skilled in the art.
Claims (5)
1. an optimizing distribution method for metadata in Video Storage System, is characterized in that, in described Video Storage System, the architecture of the optimizing distribution method of metadata comprises the reserved module of index node and index node distribution module; Wherein, described index node is reserved module, and for arranging, whether reserved function enables and reserved node parameter; Described index node distribution module, is responsible for according to reserved node parameter allocation index node;
When creating file under catalogue, first from the index node of parent directory, start the idle available index node of linear search, if found, just directly distribute to file use, if fail, find available index node, adopt original linear directory node allocation strategy.
2. the optimizing distribution method of metadata in Video Storage System according to claim 1, is characterized in that, the reserved parameter of maximum of described reserved node parameter is that the index number in the index node list in piece group subtracts 1.
3. the optimizing distribution method of metadata in Video Storage System according to claim 1, it is characterized in that, described index node distribution module receives after reserved node parameter, the index node space of more reserved files when createing directory index node, for storing the file inode under same directory.
4. according to the optimizing distribution method of metadata in Video Storage System described in claim 2 or 3, it is characterized in that, in Video Storage System during file system creation, adopt the optimizing distribution method of metadata in described Video Storage System to complete the assigning process of index node, index node predistribution flow process is as follows:
Reserved node parameter Pre_Num is set, then will reserves node parameter and pass to the reserved module of index node;
The inode_preload function of the reserved module of index node receives reserved node parameter Pre_Num, after relevant setting that completes reserved node parameter, relevant information is passed to index node distribution module;
First inode_alloc function in index node distribution module will verify whether be catalogue file, if continue to carry out, otherwise return;
Index node distribution module starts to join index node list for each piece component, need to obtain index node bitmap, then according to reserved node parameter, reserved index node is set in index node bitmap;
After being all assigned with, all index nodes restart to distribute for next piece group, until all groups are all assigned.
5. the optimizing distribution method of metadata in Video Storage System according to claim 4, it is characterized in that, first new file under catalogue can attempt the index node distributing from reserved area, and these index nodes also can be distributed to the file in other catalogue or distribute to other catalogue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410114166.6A CN103942262A (en) | 2014-03-26 | 2014-03-26 | Method for optimal allocation of metadata in video storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410114166.6A CN103942262A (en) | 2014-03-26 | 2014-03-26 | Method for optimal allocation of metadata in video storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103942262A true CN103942262A (en) | 2014-07-23 |
Family
ID=51189930
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410114166.6A Pending CN103942262A (en) | 2014-03-26 | 2014-03-26 | Method for optimal allocation of metadata in video storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103942262A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112612751A (en) * | 2020-12-25 | 2021-04-06 | 北京浪潮数据技术有限公司 | Asynchronous directory operation method, device, equipment and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030126122A1 (en) * | 2001-09-18 | 2003-07-03 | Bosley Carleton J. | Systems, methods and programming for routing and indexing globally addressable objects and associated business models |
| CN101446984A (en) * | 2009-01-09 | 2009-06-03 | 成都市华为赛门铁克科技有限公司 | Method and device for storing files and method and device for deleting files |
-
2014
- 2014-03-26 CN CN201410114166.6A patent/CN103942262A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030126122A1 (en) * | 2001-09-18 | 2003-07-03 | Bosley Carleton J. | Systems, methods and programming for routing and indexing globally addressable objects and associated business models |
| CN101446984A (en) * | 2009-01-09 | 2009-06-03 | 成都市华为赛门铁克科技有限公司 | Method and device for storing files and method and device for deleting files |
Non-Patent Citations (1)
| Title |
|---|
| 李勇: "Ext4文件系统的目录索引节点预留", 《百度文库》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112612751A (en) * | 2020-12-25 | 2021-04-06 | 北京浪潮数据技术有限公司 | Asynchronous directory operation method, device, equipment and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102902709B (en) | The document storage system of a kind of fixed space distribution and implementation method | |
| CN106527969B (en) | A kind of Nand Flash memorizer reading/writing method in a balanced way of life-span | |
| US7937522B2 (en) | Method for flash memory data management | |
| CN106201355A (en) | Data processing method and device and storage system | |
| US20080288710A1 (en) | Semiconductor Memory Device and Its Control Method | |
| CN103399823B (en) | The storage means of business datum, equipment and system | |
| CN102779180A (en) | Operation processing method of data storage system and data storage system | |
| CN101576890B (en) | Novel method and system for sorting fragment | |
| CN101510332B (en) | Method and apparatus for managing memory space of smart card | |
| CN104331453A (en) | Distributed file system and constructing method thereof | |
| CN103544156A (en) | Method and device for storing files | |
| CN102541757A (en) | Write cache method, cache synchronization method and device | |
| CN103970669A (en) | Method for accelerating physical-to-logic address mapping of recycling operation in solid-state equipment | |
| CN101539884B (en) | Method and device for data storage | |
| CN103076993A (en) | Storage system and method for concentration type system | |
| CN112637616B (en) | Object storage method, system and server | |
| CN103123645A (en) | Method for reducing disk fragments | |
| CN103092849A (en) | File system cluster management method | |
| CN104123952A (en) | Cache-free disc burning method and device | |
| CN101303667B (en) | Method and device for establishing magnetic disk null cluster table and seeking magnetic disk null cluster | |
| CN103399915A (en) | Optimal reading method for index file of search engine | |
| SG126863A1 (en) | Recording apparatus | |
| CN100543748C (en) | A kind of method and system of utilizing file allocation table to carry out the file tracking | |
| CN107430546B (en) | File updating method and storage device | |
| CN104102735A (en) | File system optimizing method and file system optimizing device aiming at database files |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140723 |
|
| WD01 | Invention patent application deemed withdrawn after publication |