CN102012946A - High-efficiency safety monitoring video/image data storage method - Google Patents
High-efficiency safety monitoring video/image data storage method Download PDFInfo
- Publication number
- CN102012946A CN102012946A CN2010105993205A CN201010599320A CN102012946A CN 102012946 A CN102012946 A CN 102012946A CN 2010105993205 A CN2010105993205 A CN 2010105993205A CN 201010599320 A CN201010599320 A CN 201010599320A CN 102012946 A CN102012946 A CN 102012946A
- Authority
- CN
- China
- Prior art keywords
- storage
- video
- image
- key
- physical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a high-efficiency safety monitoring video/image data storage method, namely a storage method comprehensively adopting dynamic structural query language (SQL) sentences and a dynamic balanced storage module. The method comprises the following steps: separating stored objects from attribute information of the objects; on one hand, setting stored target physical equipment by the dynamic SQL sentences; and on the other hand, carrying out balanced storage on stored images or videos by a dynamic storage module so as to improve the whole storage effect of a safety monitoring storage system. Aiming at the characteristics of image and video flow storage in safety monitoring, the invention provides the storage method comprehensively adopting the dynamic SQL sentences and the dynamic balanced storage module. In the method, general hardware storage resources comprising common compatible servers or redundant array of independent disk (RAID) equipment can be combined into a distribution type storage system with strong performances. The high-efficiency safety monitoring video/image data storage method can be widely applied in the safety monitoring application fields such as 'safe city', safety monitoring of enterprises, 'safe traffic' and the like.
Description
Technical field
The invention belongs to computer memory technical field, application background is that video monitoring stream or the vehicle snapshot figure efficient storage in security monitoring and the intelligent transportation system handled, and is specifically related to the technology such as burst storage, visit of image or video.
Background technology
Along with global resident living improves constantly, awareness of safety is constantly ripe, people pay attention to home safety and quality of the life more, under the factor that individual, family, campus, community's safety guarantee market demand day by day increase, cause demand and day sharp increase to the safety product industrial market; Under information and the science and technology influence of constantly bringing forth new ideas of communicating by letter, in conjunction with information, computer photography, artificial intelligence, control many-sided technology such as electronic sensor automatically, be applied to fields such as antitheft, gate inhibition, telemonitoring, video conference, road monitoring, this shows, security monitoring market is in future development, still have sizable demand space, potential infinite commercial opportunities.National governments have the special fund subsidy to build safety monitoring system more at present.It is all the more ripe to add the idea that ensures personal safety as well as the property safety, and especially construction scheme is constantly released, and builds the merchant and also constantly introduces high performance security monitoring product.Under this opportunity, the sustainable growth of security monitoring product sales volume, industry is flourish.The security monitoring field has obtained worldwide generally acknowledged, also is one of focus of research both at home and abroad as an interdiscipline, informationization and systematized synthetic study system always.Storage in the security monitoring is the key component of total system.When work simultaneously in the information acquisition source of certain scale, how high efficient and reliable ground carries out data storage, it is the current problem that needs solution, all big enterprises have all released the specialty storage grade product of oneself on the present market, it is with high costs that yet the storage products of high standard exists again, configuration is specialty and complexity, the problem that specific aim is relatively poor too.At these problems, the present invention is directed to the characteristics of image and video flowing storage in the security monitoring, the storage means of comprehensive employing dynamic SQL statements and storage mobile equilibrium module has been proposed.This method can be used general hardware store resource, comprises common compliant server or RAID equipment, is combined into the powerful distributed memory system of performance.Can be widely used in " safe city ", security monitoring applications such as enterprise security monitoring, " safety traffic ".
Summary of the invention
Problem at current storage system existence, the purpose of this invention is to provide a kind of date storage method of security monitoring video efficiently, promptly comprehensive dynamic SQL statements and the storage means of storing the mobile equilibrium module of adopting, this method is separated the attribute information of storage object with object, by dynamic SQL statements the target physical equipment of storage is set on the one hand, by the dynamic memory module picture or the video of storing carried out the equilibrium storage on the one hand in addition, to improve the global storage effect of security monitoring storage system.
Technical scheme of the present invention is: a kind of date storage method of security monitoring video efficiently when carrying out picture or video storage, is made of following components:
(1) physical store subsystem;
(2) data base management subsystem;
(3) data acquisition subsystem;
(4) information gathering/adjustment dynamic memory subsystem;
Comprehensive dynamic SQL statements and the storage means of storing the mobile equilibrium module of adopting, the attribute information of storage object with object separated, by dynamic SQL statements the target physical equipment of storage is set on the one hand, by the dynamic memory module picture or the video of storing carried out the equilibrium storage on the one hand in addition, to improve the global storage effect of security monitoring storage system.
The present invention is directed to the characteristics of image and video flowing storage in the security monitoring, proposed the storage means of comprehensive employing dynamic SQL statements and storage mobile equilibrium module.This method can be used general hardware store resource, comprises common compliant server or RAID equipment, is combined into the powerful distributed memory system of performance.Can be widely used in " safe city ", security monitoring applications such as enterprise security monitoring, " safety traffic ".
The date storage method of security monitoring video efficiently according to above-mentioned is characterized in that comprising the steps:
(1) obtain image or video data by data acquisition software, these data should comprise time H1 except image and video data itself, filename H2, data type H3, attribute informations such as collecting location H4;
(2) generate dynamic SQL statements by storage mobile equilibrium module, table name in this SQL statement and database name are according to H1, H2, and H3 and H4 dynamically generate.The different database of native system or table name are present on the different physical disks according to the burden of each current physical storage device, are used for reducing the storage bottleneck;
(3) according to compatible/consistance hash function (consistent hash) with H1, H2 ..., Hn is mapped as a key word (KEY), and by the physical store node location P of this value decision image/video document storage and the storage relative position on P.If KEY is the array Array of n byte, then by
Obtaining the relative position of physical storage locations P. on P and can be with KEY as filename, also can be with the physical storage locations of this KEY as index;
(4) by the dynamic memory module with image or video deposits the corresponding physical storage subsystem in;
(5) deposit image or the attribute information of video in corresponding database subsystem by the dynamic memory module.
The date storage method of security monitoring video efficiently according to above-mentioned is characterized in that:
(1) this method is only handled the attribute information of video or view data and video data, and text message or other binary message native system are disregarded;
(2) method is in the time of memory image/video information, image/video data itself is separated with attribute information and is deposited, image/video data itself exists on the distributed physical storage device with the file or the form of raw data, the image/video data attribute information then deposits database in, carries out unified management;
(3) when depositing the image/video data attribute information in database, the library name of database and table name are dynamically to generate, and the principle of generation is and time H1, filename H2, and data type H3, attribute informations such as collecting location H4 are relevant;
(4) when depositing image/video data in physical equipment, according to compatible/consistance hash function (consistent hash) with H1, H2, Hn is mapped as a key word (KEY), and by the physical store node location P of this value decision image/video document storage and the storage relative position on P.If KEY is the array Array of n byte, then by
Obtaining the relative position of physical storage locations P. on P and can be with KEY as filename, also can be with the physical storage locations of this KEY as index.
The present invention is directed to the characteristics of image and video flowing storage in the security monitoring, proposed the storage means of comprehensive employing dynamic SQL statements and storage mobile equilibrium module.This method can be used general hardware store resource, comprises common compliant server or RAID equipment, is combined into the powerful distributed memory system of performance.Can be widely used in " safe city ", security monitoring applications such as enterprise security monitoring, " safety traffic ".
Description of drawings
Fig. 1 is the enforcement Organization Chart of the embodiment of the invention.
Fig. 2 is the workflow diagram of the embodiment of the invention.
Embodiment
The enforcement framework of the embodiment of the invention as shown in Figure 1.
This method implements to comprise four parts: physical store subsystem, data base management subsystem, data acquisition subsystem, information gathering/adjustment subsystem subsystem.System architecture as shown in Figure 1.
In the native system, earlier obtain various monitor datas by data acquisition server, then by database manager according to certain standard packaging, deposit different physical store servers in by the dynamic adjusting module of dynamic SQL statements and file storage, simultaneously, monitor server constantly monitors the load of each storage server, with the workload of dynamic each storage server of adjustment.
Each gordian technique and subsystem implement to take following thinking:
(a) physical store server
The minimum memory unit that the physical store server upwards provides is a memory node, and memory node is a kind of logical device, and exists with a kind of recursive structure.RAID or single this memory node of disk primordial that several disks are formed, a node can be made of other node.Memory node has shielded concrete physical equipment information to the system upper strata, by being similar to the mode of object storage device (OSD), provides unified file-level access interface to the upper strata.And fundamental node inside can be still with block-based file system organization data.For the system upper strata, except abstract indexs such as memory capacity, response speed have the difference, memory node is an indifference equipment.
(b) database server subsystem
Database server system and storage subsystem exist with separate as far as possible state.Heritage storage system is the memory allocation granularity with the data block, and our system is partition size with the file.Each memory node is in charge of the deposit data information of concrete file.
The database server subsystem adopts the file system of asymmetrical form, and promptly the mode of directory information and basic document information separated forms a kind of loose coupling relation between the file subsystem of final system and the storage subsystem.
(c) information gathering/adjustment/watchdog subsystem
The status information of each memory node is responsible for collection by self, and monitoring/adjustment server can be asked for status information to part or all of node as required.Self-adaptation is adjusted at two aspects and implements, and is respectively local and adjusts and overall the adjustment, is responsible for finishing by node self and monitoring/adjustment server respectively.The local adjustment is that node carries out oneself adjustment according to self information, or carries out internodal fine setting with adjacent node in logic.For example by file access information is analyzed, in the high node of physical property, and the low relatively deposit data of frequency of utilization is in the lower node of actual performance the high deposit data of frequency of utilization.Specifically can adopt permeating method, node is by performance queuing, and the data of using are placed on the average node of actual performance for the first time, and the data that utilization rate is high are moved to the higher node of performance, otherwise then move on the lower node of performance.When a certain node load is overweight, partial data is placed on the comparatively idle node.
Overall situation adjustment is according to all nodal informations, selects suitable adjustment strategy and enforcement.The for example selection of high-performance node; The distribution of newly added node storage area etc.Do not have the differentiation of stricti jurise between the two, for example in the new node adition process, adding the selection of implantation site and adding operation is to finish by monitoring/adjustment server, and the operations such as data migtation in the adjustment process are finished by memory node self control.
(d) data acquisition subsystem
Data acquisition subsystem is mainly used to gather all kinds of security monitoring data, comprises image, video etc.The data-interface that hardware device by different manufacturers provides is developed unified module, at last image, video is generated the needed form of storage system, carries out dynamic memory by data base management subsystem at last.
The workflow of the embodiment of the invention is seen Fig. 2.
Comprise the steps:
(1) obtain image or video data by data acquisition software, these data should comprise time H1 except image and video data itself, filename H2, data type H3, attribute informations such as collecting location H4;
(2) generate dynamic SQL statements by storage mobile equilibrium module, table name in this SQL statement and database name are according to H1, H2, and H3 and H4 dynamically generate.The different database of native system or table name are present on the different physical disks according to the burden of each current physical storage device, are used for reducing the storage bottleneck;
(3) according to compatible/consistance hash function (consistent hash) with H1, H2 ..., Hn is mapped as a key word (KEY), and by the physical store node location P of this value decision image/video document storage and the storage relative position on P.If KEY is the array Array of n byte, then by
Obtaining the relative position of physical storage locations P. on P and can be with KEY as filename, also can be with the physical storage locations of this KEY as index;
(4) by the dynamic memory module with image or video deposits the corresponding physical storage subsystem in;
(5) deposit image or the attribute information of video in corresponding database subsystem by the dynamic memory module.
The date storage method of security monitoring video efficiently according to above-mentioned is characterized in that:
(1) this method is only handled the attribute information of video or view data and video data, and text message or other binary message native system are disregarded;
(2) method is in the time of memory image/video information, image/video data itself is separated with attribute information and is deposited, image/video data itself exists on the distributed physical storage device with the file or the form of raw data, the image/video data attribute information then deposits database in, carries out unified management;
(3) when depositing the image/video data attribute information in database, the library name of database and table name are dynamically to generate, and the principle of generation is and time H1, filename H2, and data type H3, attribute informations such as collecting location H4 are relevant;
(4) when depositing image/video data in physical equipment, according to compatible/consistance hash function (consistent hash) with H1, H2, Hn is mapped as a key word (KEY), and by the physical store node location P of this value decision image/video document storage and the storage relative position on P.If KEY is the array Array of n byte, then by
Obtaining the relative position of physical storage locations P. on P and can be with KEY as filename, also can be with the physical storage locations of this KEY as index.
Claims (3)
1. date storage method of security monitoring video efficiently when carrying out picture or video storage, is made of following components:
(1) physical store subsystem;
(2) data base management subsystem;
(3) data acquisition subsystem;
(4) information gathering/adjustment dynamic memory subsystem;
Comprehensive dynamic SQL statements and the storage means of storing the mobile equilibrium module of adopting, the attribute information of storage object with object separated, by dynamic SQL statements the target physical equipment of storage is set on the one hand, by the dynamic memory module picture or the video of storing carried out the equilibrium storage on the one hand in addition, to improve the global storage effect of security monitoring storage system.
2. according to the date storage method of security monitoring video efficiently of claim 1, it is characterized in that: comprise the steps:
(1) obtain image or video data by data acquisition software, these data should comprise time H1 except image and video data itself, filename H2, data type H3, attribute informations such as collecting location H4;
(2) generate dynamic SQL statements by storage mobile equilibrium module, table name in this SQL statement and database name are according to H1, H2, and H3 and H4 dynamically generate.The different database of native system or table name are present on the different physical disks according to the burden of each current physical storage device, are used for reducing the storage bottleneck;
(3) according to compatible/consistance hash function (consistent hash) with H1, H2 ..., Hn is mapped as a key word (KEY), and by the physical store node location P of this value decision image/video document storage and the storage relative position on P.If KEY is the array Array of n byte, then by
Obtaining the relative position of physical storage locations P. on P and can be with KEY as filename, also can be with the physical storage locations of this KEY as index;
(4) by the dynamic memory module with image or video deposits the corresponding physical storage subsystem in;
(5) deposit image or the attribute information of video in corresponding database subsystem by the dynamic memory module.
3. according to the date storage method of security monitoring video efficiently of claim 1 or 2, it is characterized in that:
(1) this method is only handled the attribute information of video or view data and video data, and text message or other binary message native system are disregarded;
(2) method is in the time of memory image/video information, image/video data itself is separated with attribute information and is deposited, image/video data itself exists on the distributed physical storage device with the file or the form of raw data, the image/video data attribute information then deposits database in, carries out unified management;
(3) when depositing the image/video data attribute information in database, the library name of database and table name are dynamically to generate, and the principle of generation is and time H1, filename H2, and data type H3, attribute informations such as collecting location H4 are relevant;
(4) when depositing image/video data in physical equipment, according to compatible/consistance hash function (consistent hash) with H1, H2, Hn is mapped as a key word (KEY), and by the physical store node location P of this value decision image/video document storage and the storage relative position on P.If KEY is the array Array of n byte, then by
Obtaining the relative position of physical storage locations P. on P and can be with KEY as filename, also can be with the physical storage locations of this KEY as index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105993205A CN102012946A (en) | 2010-12-22 | 2010-12-22 | High-efficiency safety monitoring video/image data storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105993205A CN102012946A (en) | 2010-12-22 | 2010-12-22 | High-efficiency safety monitoring video/image data storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102012946A true CN102012946A (en) | 2011-04-13 |
Family
ID=43843119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105993205A Pending CN102012946A (en) | 2010-12-22 | 2010-12-22 | High-efficiency safety monitoring video/image data storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102012946A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461375A (en) * | 2013-09-24 | 2015-03-25 | 浙江大华技术股份有限公司 | Method and device for distributing magnetic disk tasks |
| CN105009010A (en) * | 2013-03-07 | 2015-10-28 | 欧姆龙株式会社 | Control system, control device, image processing device, and control method |
| CN105117171A (en) * | 2015-08-28 | 2015-12-02 | 南京国电南自美卓控制系统有限公司 | Energy SCADA massive data distributed processing system and method thereof |
| CN103618594B (en) * | 2013-11-27 | 2017-01-11 | 中国科学院软件研究所 | JPEG2000 image protecting system and method based on information attributes |
| CN106341492A (en) * | 2016-10-19 | 2017-01-18 | 杭州赛客网络科技有限公司 | Distributed memory data storing and reading method based on cloud communication service |
| CN110602563A (en) * | 2019-10-11 | 2019-12-20 | 重庆杰夫与友文化创意有限公司 | Picture and video editing processing method capable of realizing real-time preview |
| CN111556122A (en) * | 2020-04-23 | 2020-08-18 | 东电创新(北京)科技发展股份有限公司 | Video transmission and storage method based on narrow bandwidth |
| CN116521940A (en) * | 2023-06-20 | 2023-08-01 | 上海临滴科技有限公司 | Industrial video cloud storage real-time supervision method and system |
-
2010
- 2010-12-22 CN CN2010105993205A patent/CN102012946A/en active Pending
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105009010A (en) * | 2013-03-07 | 2015-10-28 | 欧姆龙株式会社 | Control system, control device, image processing device, and control method |
| CN105009010B (en) * | 2013-03-07 | 2017-12-05 | 欧姆龙株式会社 | Control system, control device, image processing apparatus and control method |
| CN104461375A (en) * | 2013-09-24 | 2015-03-25 | 浙江大华技术股份有限公司 | Method and device for distributing magnetic disk tasks |
| CN104461375B (en) * | 2013-09-24 | 2018-05-01 | 浙江大华技术股份有限公司 | A kind of distribution method and device of disk task |
| CN103618594B (en) * | 2013-11-27 | 2017-01-11 | 中国科学院软件研究所 | JPEG2000 image protecting system and method based on information attributes |
| CN105117171A (en) * | 2015-08-28 | 2015-12-02 | 南京国电南自美卓控制系统有限公司 | Energy SCADA massive data distributed processing system and method thereof |
| CN105117171B (en) * | 2015-08-28 | 2018-11-30 | 南京国电南自维美德自动化有限公司 | A kind of energy SCADA mass data distributed processing system(DPS) and its method |
| CN106341492A (en) * | 2016-10-19 | 2017-01-18 | 杭州赛客网络科技有限公司 | Distributed memory data storing and reading method based on cloud communication service |
| CN110602563A (en) * | 2019-10-11 | 2019-12-20 | 重庆杰夫与友文化创意有限公司 | Picture and video editing processing method capable of realizing real-time preview |
| CN111556122A (en) * | 2020-04-23 | 2020-08-18 | 东电创新(北京)科技发展股份有限公司 | Video transmission and storage method based on narrow bandwidth |
| CN111556122B (en) * | 2020-04-23 | 2023-04-07 | 东电创新(北京)科技发展股份有限公司 | Video transmission and storage method based on narrow bandwidth |
| CN116521940A (en) * | 2023-06-20 | 2023-08-01 | 上海临滴科技有限公司 | Industrial video cloud storage real-time supervision method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11941017B2 (en) | Event driven extract, transform, load (ETL) processing | |
| TWI698108B (en) | Blockchain-based data processing method and device | |
| CN102012946A (en) | High-efficiency safety monitoring video/image data storage method | |
| US20200301941A1 (en) | Large scale unstructured database systems | |
| US11093466B2 (en) | Incremental out-of-place updates for index structures | |
| CN102521406B (en) | Distributed query method and system for complex task of querying massive structured data | |
| US9280381B1 (en) | Execution framework for a distributed file system | |
| CN100481086C (en) | Space data clustered storage system and data searching method | |
| CN107038162A (en) | Real time data querying method and system based on database journal | |
| CN105138661A (en) | Hadoop-based k-means clustering analysis system and method of network security log | |
| CN103268336A (en) | Fast data and big data combined data processing method and system | |
| US10922285B1 (en) | Table data persistence | |
| US20150088806A1 (en) | Supporting multi-tenancy in a federated data management system | |
| US11797559B2 (en) | System and method for real time data aggregation in a virtual cube in a multidimensional database environment | |
| CN104123388A (en) | Massive-sensing-network-data-oriented high-concurrency real-time access system and method | |
| CN104270412A (en) | Three-level caching method based on Hadoop distributed file system | |
| US10095738B1 (en) | Dynamic assignment of logical partitions according to query predicate evaluations | |
| Li et al. | R-Store: A scalable distributed system for supporting real-time analytics | |
| CN112181950B (en) | Construction method of distributed object database | |
| CN107861983A (en) | Remote sensing image storage system for high-speed remote sensing image processing | |
| CN118193482A (en) | Space data storage management device | |
| CN116431635A (en) | Lake and warehouse integrated-based power distribution Internet of things data real-time processing system and method | |
| Sridhar et al. | Optimizing database load and extract for big data era | |
| Ding et al. | RDB-KV: A cloud database framework for managing massive heterogeneous sensor stream data | |
| Liu et al. | The application of Internet of things and Oracle database in the research of intelligent data management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110413 |