Background technology
Along with society, expanding economy, people are more and more higher to the demand of security protection, video monitoring is as the effective means in the safety protection field, its effect is more and more important, obtain develop rapidly in recent years, be widely used in places such as bank, warehouse, shopping, recreation center, and range of application expands constantly, as traffic control system etc.Meanwhile, on application model, technically constantly weed out the old and bring forth the new, on development trend, the video monitoring technology develops towards digitlization, networking, intelligentized direction, and the video monitoring correlation technique is had higher requirement and new challenge.Many cities and area prepare to set up large-scale video monitoring system, and number of cameras is many, and monitoring range is wide.Traditional mode and technology are difficult to satisfy user's requirement, and former specific area dedicated system develops into the video monitoring system of present digitlization, many sites, multi-field, big data quantity gradually.
The continuous expansion of network communications technology development in recent years, network size and perfect, network is almost ubiquitous, and bandwidth is more and more wideer, and the network digital monitoring technique is that growing field of video monitoring has been injected new blood.Along with reaching its maturity of embedded technology, by Network Video Surveillance equipment analog picture signal is converted into data image signal, the user can realize high-performance, the simple and direct video monitoring of configuration in the present networks environment.
Video monitoring mainly comprises video acquisition, storage and data readback.As shown in Figure 1, the basic functional principle of digital video monitor system is to utilize camera/video camera to obtain video information, is encoded by encoder, the video data digitlization again.Digitized video data also can store except being used for real-time monitoring, playback at any time after being provided with.At present in field of video monitoring, the storage mode of video data has two kinds, a kind of be with storage in the built-in hard disk of encoding device, as adopt digital video recorder (DVR, Digital Video Recorder) scheme of Shi Xianing, another kind is responsible for storage by the specific store server, as the scheme that adopts digital video frequency server (DVS, DigitalVideo Server) to realize.
Along with the raising of video monitoring image quality, the increase of picture dimension, the increase of video monitoring circuit, memory capacity will increase exponentially, and video monitoring system is bigger to the capacity requirement of storage.
Need long continuous data read-write, data traffic is big and access request quantity is relatively low;
Usually, the data retention cycle in the video monitoring system is shorter, general monitoring place data preserve certain hour (as 15 days) just after can delete.
The extensive use of network causes the IP network volume of transmitted data to increase severely, traditional data model storage, as Direct Attached Storage (DAS, Direct Attached Storage), can not satisfy the demand, adopt network storage mode, network store system mainly comprises storage area network (SAN, StorageArea Network) and network attached storage (NAS, Network Attached Storage) two big classes.
Small computer system interface SCSI host-host protocol i SCSI (internetSmall Computer System Interface) based on internet is a kind of host-host protocol based on TCP/IP, is used for transmitting scsi command and data by IP network between applied host machine and the IP storage system.And iscsi technology standardization (RFC3720, RFC3721 etc.).
At present, very fast based on the development of iSCSI protocol of I P Network storage technology, adopt the IP-based SAN of iSCSI protocol construction, overcome the problem of optical-fibre channel cost height (SAN system) and interoperability manipulation poor (NAS system), therefore in the industry cycle progressively be used widely.
In video monitoring system, adopt, digitized video data is stored in the IP storage system of far-end, can realize centralized management based on the iSCSI protocol of I P network storage.At present, the network storage of video monitoring data does not also have unified codes and standards.
As shown in Figure 2, in the video monitoring system that adopts digital video recorder DVR, coding subsystem and storage subsystem merge physically, and video data is stored in document form in the built-in hard disk of each DVR, and client is obtained video data stored from DVR and carried out playback.Video data stored generally realizes file-sharing by NFS (NFS, Network File System), and NFS adopts at the listed files of client mapping server end (DVR) and realizes that network is shared.Write when server end has data, when causing the server end partial document to upgrade, client need be carried out the Mount operation again, just can obtain the latest document tabulation of server end.This is a kind of very simple data storage scheme, still,
Stored video data in the built-in hard disk of DVR, because hard-disk capacity is limited, the remaining time of data is short, is difficult to support the image of high definition; For large-scale video monitoring system, system management is comparatively complicated simultaneously.
With the document form stored video data, the management of file depends on file system.Can't be compatible between different file system, limited the opening and the sharing of data;
With the document form stored video data, can not satisfy the concurrent access requirement of storage and playback;
Client and DVR generally realize file-sharing by NFS, and server file is updated each time, and other client all needs again Mount to DVR, influences system effectiveness;
DVR must dispose corresponding software, converts digitized video data stream to file, and is kept in the built-in hard disk.
As shown in Figure 3, in the video monitoring scheme that adopts digital video frequency server DVS, storage subsystem generally is made of video storage server, external storage system, adopts the mode of NAS to carry out the network storage of data.Realized small-scale data centralization storage, made things convenient for the user inquiring data and carry out playback.When data need be stored, encoder was sent to network with data file, and server end is stored in the mode of NAS after receiving data, and the general file system that adopts has NFS or FAT32.
The same with the DVR scheme, the DVS scheme generally still adopts the form stored video data of catalogue add file or database index add file.Video data file is divided by fixed size or is divided by the fixed storage time, and time value is generally adopted in file designation, as 200605201705.mpg.The inquiry of video data is retrieved or is retrieved by database index with the form of filename usually.
NAS adopts the file-sharing mode to store, and when encoder quantity increases, influences system effectiveness;
Terminal and storage server are shared by file system such as NFS, and the storage server file upgrades each time, and other terminal all needs Mount storage server again, influences system effectiveness.
Storage server must dispose corresponding software, converts digitized video data stream to file.
Inquire about the video data of fixed time according to filename, adopt the matching inquiry method, efficient is lower.Because single video data file generally comprises a few minutes or the long video data of dozens of minutes, therefore the time precision of inquiry is relatively poor, can not satisfy more high precision time search request.
Summary of the invention
In view of this, the technical problem to be solved in the present invention provides a kind of video monitoring data storage managing method and system, to realize the efficient access of video monitoring data in video monitoring system.
According to a kind of video monitoring data storage managing method provided by the invention, by network data processing equipment is connected with storage device, this method comprises:
Video data is divided into the plurality of data piece, and with described data block by the write direct storage device of described SAN framework of disk sector;
Directly read the described data block in the storage device of SAN framework by disk sector, to recover video data.
Preferably, described network using is carried out transfer of data based on the SCSI host-host protocol iSCSI of the Internet.
Preferably, the data that data processing unit will be stored are divided into the identical data block of some sizes, and the plurality of data piece is formed the identical data cell of size.
Preferably, set up index for each described data cell;
Read described data block in the described storage device according to the index of being set up, to recover video data.
Preferably, the described mode of setting up index comprises one or more levels indexed mode.
Preferably, this method further comprises:
In described storage device, index block is set, the corresponding memory cell of every index information in this index block, index information comprises the owner record timestamp of data cell.
Preferably, this method further comprises:
The owner record timestamp that the master index piece writes down all data cells is set in described storage device, the unit index piece is set, the timestamp and the memory location side-play amount of all data blocks in this unit index piece minute book data cell in each data cell.
A kind of video monitoring data storing and managing system comprises:
Data processing equipment is used for the video monitoring data of gathering is encoded, decoded, and by disk sector enter data directly into the storage device of SAN framework or from the storage device of SAN framework by the direct reading of data of disk sector;
Storage device is used for according to SAN framework store video monitor data;
Described data processing equipment is divided into the plurality of data piece with video data, and described transmission of data blocks is arrived described storage device, reads described data block from described storage device, to recover video data.
Preferably, this system also comprises administrative unit, is used for described data block is managed;
The data that described data processing equipment will be stored are divided into the identical data block of some sizes, and the plurality of data piece is formed the identical data cell of size;
Described administrative unit is set up index for each described data cell;
Described data processing equipment reads described data block in the described storage device according to the index of being set up, to recover video data.
Preferably, described administrative unit comprises one or more levels indexed mode for the indexed mode of each described data cell foundation.
In sum, in the solution provided by the invention, the video monitoring data processing unit directly carries out block operations to memory device, has realized the efficient access of multitude of video data in video surveillance network.
The video monitoring data processing unit is read and write by disk sector the block operations of video data, need not to divide file system, has avoided the incompatible problem of file system, realizes the open fully and shared of video data; During the storage Data Update, all encoders or client need not Mount memory device again, can satisfy the data concurrent access requirement of a large amount of encoders;
In addition, when the present invention stores by adopting two-stage or multiple index mode, when increasing new video data, only need in the secondary of correspondence or multiple index, insert corresponding information, and do not need to revise the one-level index information, avoid frequently striding sector read-write storage medium, thereby reduce loss, prolonged the life-span of storage medium; Simultaneously also reduce the addressing time, improved the video data readwrite performance.
Owing to adopt two-stage or multiple index mode, can adopt different search strategies at different search requests; When loss of data or mistake appear in the one-level index, can improve the reliability of video data and the flexibility of use by secondary index or multiple index reduction one-level index information.
Embodiment
The invention provides a kind of video monitoring data storage managing method and system, in video monitoring system, realize the efficient access of video data, by storage administration to video data, realize high efficiency, reliability and the flexibility of storage, satisfy the storage demand of large-scale video monitoring system.Core concept of the present invention is: the video monitoring data processing unit is connected by the storage device of network with network storage framework, video data is divided into the plurality of data piece, and described data block store is arrived described storage device; And network storage device directly carried out data block store and read operation, for each described data cell is set up index; Read described data block in the described storage device according to the index of being set up, to recover video data.
For making principle of the present invention, characteristic and advantage clearer, describe the present invention in conjunction with specific embodiments with reference to the accompanying drawings.
Video monitoring system data storage scheme provided by the invention comes down to a kind of digital visual frequency network storage scheme (DVNS).
At first, formation and the realization principle to concrete employing a kind of video monitoring system of the present invention describes.As shown in Figure 4, this system comprises:
Be integrated with encoder, network storage equipment IP-SAN, the client of integrated iSCSI protocol module, the video storage management server (optional) of iSCSI function, in system, play coordinated monitoring;
As shown in Figure 4, encoder, the network storage equipment, client and video storage management server link together by IP network.
Storage area network (SAN, Storage Area Network) is the storage framework that connects memory device and application server by network mode, and the storage data sharing of piece (block) level externally is provided.The exchanging visit of this network-specific between main frame and memory device, data can be by SAN high-speed transfer between a plurality of servers and a plurality of memory device.IP-SAN is based on the SAN framework of Fast Ethernet, realizes storing data high-speed transfer between server and memory device by the iSCSI agreement.
The video monitoring data storing process of native system:
1. encoder starts;
2. give the storage resources of encoder assigned ip-SAN;
3. behind the storage resources of encoder carry IP-SAN, obtain a virtual network hard disc;
4. encoder starts coding, and video data is utilized the iSCSI encapsulation, sets up into the data block of fixed size, in the virtual network hard disc that writes direct.Directly this is carried out block operations when writing data, by sector write-in block data;
5. when the storage appearance was unusual, encoder reported error message, and reporting system the keeper handle.
The video monitoring data replayed section:
1. client is to the data of the storage managing server inquiry playback of wanting;
2. storage managing server is informed the memory device at client data place;
3. this memory device of client carry obtains a virtual network hard disc;
4. client is directly carried out the block operations read block by the iSCSI agreement to virtual hard disk, to carry out playback.
With reference to Fig. 6, described virtual hard disk data block store and extraction operating principle are described.
ISCSI by data block store module operation bottom drives carry IP-SAN memory device, and after the carry success, encoder obtains a virtual network hard disc, can carry out read-write operation as local hard drive.In the process at disk read-write, encoder does not carry out file system to hard disk to be divided and format manipulation, but directly carries out the data block read-write operation by disk sector.Read-write operation can adopt the mode of read/write-sector number-data.Data block read module in client then adopts reverse read operation, read block from virtual network hard disc.
The storage administration of video data:
The video data of own coding device is divided into the identical data block of size and stores in the IP-SAN memory device by the boundary alignment mode in the future, the piece of some quantity is formed a data unit, the size of data cell is fixed, and promptly each data cell is made up of the data block of equal number.Based on the storage characteristics of video monitoring, can adopt one or more levels indexed mode that data block is carried out storage administration.
Adopt the one-level indexed mode, a zone be set in disk be used to deposit index information, every corresponding memory cell of index information, the owner record timestamp of record corresponding data unit, comprise year, month, day, hour, the concrete time numerical value that divides.Adopt the one-level indexed mode, the management of data is fairly simple, and the time granularity of inquiry is accurate to the order of magnitude of ' branch ', can satisfy common playback inquiry.
When the video data volume is very big, when the time granularity of inquiry is had relatively high expectations, can take two-stage or multiple index.
Adopt the secondary index mode, a certain zone in the disk is made as the primary index information zone, all the other are the memory space of regular length.Primary index information comprises the owner record timestamp of all data cells, every index information record year, month, day, hour, the concrete time numerical value that divides.First data block of each data cell is data cell index block (being secondary index), this index block has write down the timestamp and the memory location side-play amount of all data blocks (frame) in the notebook data unit, side-play amount is a reference point with the memory address of data cell, and the unit of side-play amount is a piece.Every index information comprise year, month, day, hour, the concrete time numerical value of minute, second, and memory location side-play amount.Adopt the secondary index mode, the time granularity of inquiry is accurate to the order of magnitude of ' second ', can satisfy the playback inquiry of requirements at the higher level.
With reference to Fig. 5, the memory management method of video monitoring data according to an embodiment of the invention, the concrete operations step is as follows:
Step S01 is divided into the plurality of data piece with video data, and described data block is sent to described storage device and storage;
Step S02, the data that data processing unit will be stored are divided into the identical data block of some sizes, and the plurality of data piece is formed the identical data cell of size;
Step S03 is for each described data cell is set up index;
Step S04 reads described data block in the described storage device according to the index of being set up, to recover video data.
Video monitoring system with 3 cameras (corresponding 3 encoders) is an example below.
With reference to Fig. 7, camera A obtains video information, carries out digital coding by encoder A.Video storage server (directly not distributing on IP-SAN when having the video storage server) is an encoder A memory allocated resource: produce uniquely identified sponsor party name a: iqn.2001-04.com.example.storage:encoder0001, produce the target designation of a corresponding stored resource: SANDISKEC0001, specify a disk space that is used to store data, such as 20GB, then encoder A is informed in the address of sponsor party name and IP-SAN.Encoder A with video data iSCSI protocol encapsulation, carries sponsor party name as sign after starting storage in the protocol massages, data are write in the allocated resource of IP-SAN.In like manner, the sponsor party name of encoder B and storage resources target designation are:
Iqn.2001-04.com.example.storage:encoder0002 and SANDISKEC0002.
The sponsor party name of encoder C and storage resources target designation are:
Iqn.2001-04.com.example.storage:encoder0003 and SANDISKEC0003.Different encoders has different sponsor party names as unique identification, distributes different storage resources.
When client reads the storage data of camera A, video storage server (directly not distributing on IP-SAN when having the video storage server) is a client memory allocated resource: produce sponsor party name a: iqn.2001-04.com.example.storage:client0001, and retrieve the storage resources that need read, produce the target designation of a corresponding stored resource: SANDISKC0001, use i SCSI agreement reading of data then, corresponding dealing with relationship is shown in Figure 7.
For the principle and advantage that explanation adopts two-stage or multiple index mode to store, be that example is described below with the secondary index.With reference to Fig. 8.
Setting the storage zero-time is 2006-07-07 00:00:00, continuously storage.If the code check of encoder is 2Mbps, I interframe was divided into 2 seconds, and then the data volume of each I frame period is 2Mx2/8=0.5MB; If the block size of storage is 16KB, every index of secondary index is 8Byte (64bit), memory cell index block (being secondary index) then, the index number that comprises is 16K/8=2048, the piece number of each I frame period is 0.5MB/16k=32, each memory cell size is 2048x0.5MB=1024MB, can write down the 2048x2=4096 video data of second.
Master index piece (being the one-level index) has write down the owner record timestamp of all memory cell, every index 4096 seconds at interval.Physical record is as follows:
2006-07-07?00:00
2006-07-07 01:08 (each memory cell 4096 seconds at interval)
2006-07-07 02:16 (each memory cell 4096 seconds at interval)
......
First blocks of data of each memory cell is memory cell index block (being secondary index), this index block has write down the timestamp and the memory location side-play amount of all I frame data in this memory cell, side-play amount is a reference point with the memory cell base address, and the unit of side-play amount is a piece.Every index information note year, month, day, hour, the concrete time numerical value of minute, second, and memory location side-play amount.Physical record is as follows:
First memory cell
2006-07-07?00:00:00?1
2006-07-07 00:00:02 33 (each I frame period 2 seconds, 32 blocks of data at interval)
2006-07-07 00:00:04 65 (each I frame period 2 seconds, 32 blocks of data at interval)
....
2006-07-07 01:08:16 65537 (writing down 4096 seconds data, last piece of this memory cell)
Second memory cell
2006-07-07?01:08:18?1
2006-07-07 01:08:20 33 (each I frame period 2 seconds, 32 blocks of data at interval)
2006-07-07 01:08:22 65 (each I frame period 2 seconds, 32 blocks of data at interval)
....
2006-07-07 02:16:32 65537 (writing down 4096 seconds data, last piece of this memory cell)
The 3rd memory cell
....
Adopt chained list that video monitoring data is managed
Chained list is a kind of common data structure, and it dynamically carries out memory allocation.Each chained list has " head pointer " variable, and it is deposited an address and points to first element.Each element is called " node " in the chained list, and each node all comprises the address of a user data and next node.Therefore, head pointer points to first element: first element points to second element again; ..., element to the last, this element no longer points to other element, is called " table tail ", and chained list leaves it at that.
In the video data storage, when adopting chained list that data are managed, every increase by one blocks of data need insert new node at the table tail; Traversal needs whole chained list during inquiry, compares one by one, finds institute's data retrieved of wanting or up to showing tail.
Adopt chained list that data are managed, advantage is a memory space utilance height, because how many its memory space dynamic allocation has what are used, can not waste.Shortcoming is when data quantity stored is bigger, and chained list is very long, adds new node at every turn and crosses certain node of inquiry and all need from head pointer, and efficient is lower, not as pressing search index efficient height.
Adopt the Hash table that video monitoring data is managed
In data structures such as linear list and tree, the position of each record is at random, but there is a key assignments in every record.In the Hash table, use the Hash function that a key assignments is mapped to and represent a record in the Hash sign indicating number.Identical key assignments is mapped to identical Hash sign indicating number, and a definite corresponding relation is arranged between the memory location of each record and the key assignments.The algorithm of Hash function has a lot, as immediately allocating method, and digital analysis method, middle square method, jackknife method, leaving remainder method etc.
In video data storage, when adopting the Hash table that data are managed, every increase by one blocks of data (i.e. record) goes out the key assignments of this record with the Hash construction of function.The input key assignments is calculated corresponding Hash sign indicating number with Ha sh function during inquiry, finds corresponding memory location.
Adopt the Hash table that data are managed, advantage be the efficient of reading and writing data and inquiry than higher, different Ha sh algorithms is distinct on efficient.Shortcoming is when data quantity stored is bigger, guarantee the uniqueness of key assignments mapping, the hash algorithm more complicated, and common hash algorithm can only be managed limited data capacity.
Also provide a kind of video monitoring data storing and managing system according to the present invention, comprising:
Data processing equipment is used for the video monitoring data of gathering is encoded, decoded, and the input data are to storage device or from the storage device reading of data;
Data storage device is used for the store video monitor data;
Described data processing equipment is divided into the plurality of data piece with video data, and described transmission of data blocks is arrived described storage device, reads described data block from described storage device, to recover video data.
Ground preferably, this system also comprises administrative unit, is used for described data block is managed;
The data that described data processing equipment will be stored are divided into the identical data block of some sizes, and the plurality of data piece is formed the identical data cell of size;
Described administrative unit is set up index for each described data cell;
Described data processing equipment reads described data block in the described storage device according to the index of being set up, to recover video data.
Described storage device adopts the memory device of storage area network SAN framework.
The indexed mode that described administrative unit is set up for each described data cell comprises one or more levels indexed mode.
The above only is an one exemplary embodiment of the present invention, does not limit protection scope of the present invention.All any modifications of being done within the spirit and principles in the present invention, be equal to replacement, improvement etc., all be included in protection scope of the present invention.