CN115514929A - Two-stage series storage system based on video compression data - Google Patents
Two-stage series storage system based on video compression data Download PDFInfo
- Publication number
- CN115514929A CN115514929A CN202210993218.6A CN202210993218A CN115514929A CN 115514929 A CN115514929 A CN 115514929A CN 202210993218 A CN202210993218 A CN 202210993218A CN 115514929 A CN115514929 A CN 115514929A
- Authority
- CN
- China
- Prior art keywords
- storage
- unit
- video data
- monitoring
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a two-stage series storage system based on video compression data, which belongs to the field of compression, storage and application of video monitoring data and comprises an acquisition module, a compression module and a storage module; the acquisition module is used for carrying out video monitoring on the concerned area, acquiring monitoring video data and sending the monitoring video data to the compression module; the compression module is used for compressing the received monitoring video data and sending the compressed monitoring video data to the storage module; the storage module comprises an online storage unit, an offline storage unit, a balancing unit and a migration unit; the balancing unit is used for carrying out balancing management on the online storage unit and the offline storage unit; the migration unit is used for migrating the monitoring video data stored in the online storage unit to the offline storage unit according to the migration period; the video data are stored to the offline storage unit by adopting a mode of regular migration backup, so that long-term storage can be realized, and a video-on-demand service is provided to realize retrieval of the video data.
Description
Technical Field
The invention belongs to the field of compression, storage and application of video monitoring data, relates to a compressed data storage technology, and particularly relates to a two-stage series storage system based on video compression data.
Background
With the wide construction and application of surveillance cameras and the rapid development of the internet, video data are gradually matured from front-end acquisition, network transmission, to storage of the video data, analysis and application of the data, and video-based application systems. In the process of video application, video storage is one of the key links.
Video data is composed of one frame of high-definition images, a single-channel monitoring (1080P as an example) needs a storage space of 0.62T every day, one-hundred-channel and more than one-thousand-channel video monitoring are often required to be built for monitoring requirements in real life, and storage of massive monitoring data and how to guarantee long-term effective storage of valuable video data and data loss prevention are important subjects to be solved in the field of video storage at present.
Therefore, the invention provides a two-stage series storage system based on video compression data.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a two-stage series storage system based on video compression data, which compresses real-time video data, supports frequent scheduling of the video data, adopts a mode of periodically migrating and backing up to store the video data into offline storage equipment, can realize long-term storage of the video data, and provides video on demand service to realize retrieval of the video data.
In order to achieve the above object, an embodiment according to a first aspect of the present invention provides a two-stage series storage system based on video compression data, including an acquisition module, a compression module, and a storage module;
the acquisition module is used for carrying out video monitoring on the concerned area, acquiring monitoring video data and sending the monitoring video data to the compression module;
the compression module is used for compressing the received monitoring video data and sending the compressed monitoring video data to the storage module;
the storage module comprises an online storage unit, an offline storage unit, a balancing unit and a migration unit;
the balancing unit is used for carrying out balancing management on the online storage unit and the offline storage unit; the migration unit is used for migrating the monitoring video data stored in the online storage unit to the offline storage unit according to the migration period.
Preferably, the acquisition module comprises a plurality of monitoring cameras; the compression module and the monitoring cameras are deployed in the same local area network, the compression module is communicated with all the monitoring cameras of the acquisition module, and the ONVIF or RTSP protocol is adopted to call monitoring video data from the monitoring cameras in real time.
Preferably, the compression module performs target sorting on the monitoring video data based on a deep learning algorithm, and distinguishes important targets and non-important targets in the monitoring video data;
the non-important targets are backgrounds and environments in the monitoring video data; the important target is other targets except the important target in the monitoring video data;
after the important targets and the unimportant targets are sorted, the compression module enables the important targets to form a video stream in a high-definition high-frequency mode and enables the unimportant targets to form a video stream in a high-definition low-frequency mode.
Preferably, the online storage unit comprises a plurality of storage nodes.
Preferably, the process of performing equalization management by the equalization unit includes the following steps:
the method comprises the following steps: the method comprises the steps that a balancing unit obtains information parameters of storage nodes of an online storage unit;
the information parameters of the storage nodes comprise the number of the storage nodes and the storage capacity and the available capacity of each storage node;
step two: the equalizing unit marks the storage nodes in sequence, and marks the storage nodes as i; i is a positive integer, and i =1,2 \8230, n; wherein n is the total number of the storage nodes in the online storage unit; the equalizing unit marks the storage capacity of the storage node as Nzi; marking the available capacity of the storage node as Nki;
step three: the equalizing unit acquires the data size of the compressed monitoring video data and marks the data size as By;
step four: the equalizing unit calculates a storage preference value Yxi of each storage node i by using a calculation formula, wherein the calculation mode of the storage preference value Yxi is as follows:
and sequentially arranging the storage optimal values Yxi, and selecting the storage node with the maximum storage optimal value Yxi to store the compressed monitoring video data.
Preferably, when By ≧ Nki, the storage node does not perform the calculation of the storage preference value Yxi, and the storage node is directly removed from the to-be-stored echelon.
Preferably, when the compressed monitoring video data is stored in the storage node, the equalizing unit acquires the storage time and sends the storage time to the migration unit; the migration unit is used for setting a migration period, and when the migration period is reached, the migration unit stores the corresponding compressed monitoring video data into the offline storage unit.
Preferably, in the migration period, if the compressed monitoring video data is called, the storage time is determined again, and the migration time length is calculated again.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention deploys the compression module and a plurality of monitoring cameras in the same local area network, ensures that the compression module is communicated with all the monitoring cameras of the acquisition module, and adopts an ONVIF or RTSP protocol to call monitoring video data from the monitoring cameras in real time; the compression module is used for carrying out target sorting on the monitoring video data based on a deep learning algorithm when compressing the received monitoring video data, and distinguishing important targets and non-important targets in the monitoring video data; forming video streams of important targets in a high-definition high-frequency mode, and forming video streams of unimportant targets in a high-definition low-frequency mode; 4 times of compression of monitoring video data can be realized, and a large amount of storage space is saved;
2. the invention is provided with a balancing unit and a migration unit for management; the calculation of the storage preference value Yxi is added, the storage rule is carried out according to the size of the storage preference value Yxi, the condition that a large amount of data are stored in the same storage node, the data stored in other storage nodes are few is avoided, and the storage balance can increase the data calling speed on the other hand.
Drawings
Fig. 1 is a schematic diagram of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic diagram of a two-stage tandem memory system based on video compression data. As shown in fig. 1, the two-stage series storage system based on video compression data includes an acquisition module, a compression module, a storage module, and an on-demand module;
the acquisition module is used for carrying out video monitoring on the concerned area to acquire monitoring video data, and the acquisition module sends the acquired monitoring video data to the compression module;
the acquisition module comprises a plurality of monitoring cameras;
the compression module is used for compressing the received monitoring video data and sending the compressed monitoring video data to the storage module;
the compression module and the monitoring cameras are deployed in the same local area network, the compression module is communicated with all the monitoring cameras of the acquisition module, and the ONVIF or RTSP protocol is adopted to call monitoring video data from the monitoring cameras in real time;
the storage module comprises an online storage unit and an offline storage unit which are used for storing functions;
it should be noted that, the online storage unit and the offline storage unit are both used for storing the compressed monitoring video data sent by the compression module, but the online storage unit selects a storage medium with a moderate service life, and the offline storage unit selects a storage medium with a long service life, where the storage medium with a long service life is, for example, a blu-ray disc or a magnetic disc; in view of cost, the storage medium of the online storage unit needs to be repeatedly read, and the reading frequency and frequency of the storage medium of the offline storage unit are far less than those of the storage medium of the online storage unit; the cost of the storage medium of the online storage unit is larger than that of the storage medium of the offline storage unit;
in the application, when compressing the received monitoring video data, the compression module performs target sorting on the monitoring video data based on a deep learning algorithm, and distinguishes important targets and non-important targets in the monitoring video data;
it should be explained that the non-important target is the background and environment in the monitoring video data; the important target is other targets except the important target in the monitoring video data;
after the important targets and the unimportant targets are sorted, the compression module enables the important targets to form a video stream in a high-definition high-frequency mode and enables the unimportant targets to form a video stream in a high-definition low-frequency mode; further forming compressed monitoring video data;
according to the method and the device, after the target sorting is carried out on the monitoring video data through the deep learning algorithm, the 4-time compression of the monitoring video data can be realized, and a large amount of storage space is saved;
the storage module further comprises a balancing unit and a migration unit for management; the balancing unit is configured to perform balancing management on the online storage unit, and specifically, the balancing unit performs balancing management in a process including the following steps:
the method comprises the following steps: the method comprises the steps that a balancing unit obtains information parameters of storage nodes of an online storage unit;
in this application, it should be noted that the online storage unit includes a plurality of storage nodes;
the information parameters of the storage nodes comprise the number of the storage nodes and the storage capacity and the available capacity of each storage node;
step two: the equalizing unit marks the storage nodes in sequence, and marks the storage nodes as i; i is a positive integer, and i =1,2 \8230, n; wherein n is the total number of the storage nodes in the online storage unit; the equalizing unit marks the storage capacity of the storage node as Nzi; marking the available capacity of the storage node as Nki;
step three: the equalizing unit acquires the data size of the compressed monitoring video data and marks the data size as By;
step four: the equalizing unit calculates a storage preference value Yxi of each storage node i by using a calculation formula, wherein the calculation mode of the storage preference value Yxi is as follows:
and sequentially arranging the storage optimal values Yxi, and selecting the storage node with the maximum storage optimal value Yxi to store the compressed monitoring video data.
When By is larger than or equal to Nki, the storage node does not calculate the storage preference value Yxi, and the storage node is directly removed from the to-be-stored echelon.
During storage, calculation of the storage preference value Yxi is added, and the storage rule is carried out according to the size of the storage preference value Yxi, so that the situation that a large amount of data are stored in the same storage node, the data stored in other storage nodes are few is avoided, and the balance of storage can also increase the data calling speed from the other aspect.
When the compressed monitoring video data is stored in the storage node, the equalizing unit acquires the storage time and sends the storage time to the migration unit; the migration unit is used for setting a migration period, and when the migration period is reached, the migration unit stores the corresponding compressed monitoring video data into the offline storage unit;
it should be noted that, in the migration period, if the compressed monitoring video data is called, the storage time is determined again, and the migration time length is calculated again.
The balancing unit is further configured to perform balancing management on the offline storage unit, and specifically, the balancing unit performs balancing management including the following steps:
the method comprises the following steps: the method comprises the steps that a balancing unit obtains information parameters of storage nodes of an offline storage unit;
it should be noted in this application that the offline storage unit includes a plurality of storage nodes;
the information parameters of the storage nodes comprise the number of the storage nodes and the storage capacity and the available capacity of each storage node;
step two: the balancing unit sequentially marks the storage nodes, and marks the storage nodes as j; j is a positive integer, and j =1,2 \ 8230 \ 8230, m; wherein m is the total number of the storage nodes in the online storage unit; the balancing unit marks the storage capacity of the storage node as Nzj; marking the available capacity of the storage node as Nkj;
step three: the method comprises the steps that a balancing unit obtains the data size By of compressed monitoring video data to be migrated;
step four: the equalizing unit calculates a storage preference value YXj of each storage node j by using a calculation formula, wherein the calculation mode of the storage preference value YXj is as follows:
and sequentially arranging the storage optimal values YXj, and selecting the storage node with the maximum storage optimal value YXj to store the compressed monitoring video data to be migrated.
When By is larger than or equal to Nkj, the storage node does not calculate the storage preference value YXj, and the storage node is directly removed from the to-be-stored echelon.
The video-on-demand module is used for video-on-demand, and the video-on-demand steps are as follows:
1) Inputting the name of a monitoring camera needing to request videos and the request time;
2) According to the sent video calling request, whether the video data meeting the conditions exist or not is searched from the storage module;
3) If the video data cannot be searched, the relevant video information cannot be searched in a feedback mode, if the video data cannot be searched, the video data are extracted from the storage module, and the video data are provided for the display platform after national standard transcoding.
The above formulas are all calculated by removing dimensions and taking numerical values thereof, the formula is a formula which is obtained by acquiring a large amount of data and performing software simulation to obtain the most approximate real condition, and the preset parameters and the preset threshold values in the formula are set by the technical personnel in the field according to the actual condition or obtained by simulating a large amount of data.
Although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the spirit and scope of the present invention.
Claims (8)
1. A two-stage series storage system based on video compression data is characterized by comprising an acquisition module, a compression module and a storage module;
the acquisition module is used for carrying out video monitoring on the concerned area, acquiring monitoring video data and sending the monitoring video data to the compression module;
the compression module is used for compressing the received monitoring video data and sending the compressed monitoring video data to the storage module;
the storage module comprises an online storage unit, an offline storage unit, a balancing unit and a migration unit;
the balancing unit is used for carrying out balancing management on the online storage unit and the offline storage unit; the migration unit is used for migrating the monitoring video data stored in the online storage unit to the offline storage unit according to the migration period.
2. The two-stage series storage system based on video compression data as claimed in claim 1, wherein the collection module comprises a plurality of monitoring cameras; the compression module and the monitoring cameras are deployed in the same local area network, the compression module is communicated with all the monitoring cameras of the acquisition module, and the ONVIF or RTSP protocol is adopted to call monitoring video data from the monitoring cameras in real time.
3. The two-stage series storage system based on the video compression data as claimed in claim 2, wherein the compression module performs target sorting on the surveillance video data based on a deep learning algorithm, and distinguishes important targets and unimportant targets in the surveillance video data;
the non-important targets are backgrounds and environments in the monitoring video data; the important target is other targets except the important target in the monitoring video data;
after the important targets and the unimportant targets are sorted, the compression module enables the important targets to form a video stream in a high-definition high-frequency mode and enables the unimportant targets to form a video stream in a high-definition low-frequency mode.
4. The two-stage cascade storage system for video compression data as claimed in claim 3, wherein the online storage unit comprises a plurality of storage nodes.
5. The two-stage serial storage system based on video compression data as claimed in claim 4, wherein the process of performing equalization management by the equalization unit comprises the following steps:
the method comprises the following steps: the method comprises the steps that a balancing unit obtains information parameters of storage nodes of an online storage unit;
the information parameters of the storage nodes comprise the number of the storage nodes and the storage capacity and the available capacity of each storage node;
step two: the equalizing unit marks the storage nodes in sequence, and marks the storage nodes as i; i is a positive integer, and i =1,2 \8230, 8230n; wherein n is the total number of the storage nodes in the online storage unit; the equalizing unit marks the storage capacity of the storage node as Nzi; marking the available capacity of the storage node as Nki;
step three: the equalizing unit acquires the data size of the compressed monitoring video data and marks the data size as By;
step four: the equalizing unit calculates a storage preference value Yxi of each storage node i by using a calculation formula, wherein the calculation mode of the storage preference value Yxi is as follows:
and sequentially arranging the storage preference values Yxi, and selecting the storage node with the largest storage preference value Yxi for storing the compressed monitoring video data.
6. The system of claim 5, wherein when By ≧ Nki, the storage node does not perform the calculation of the storage preference value Yxi, the storage node is directly removed from the queue to be stored.
7. The two-stage series storage system based on the video compression data as claimed in claim 6, wherein when the compressed monitoring video data is stored in the storage node, the equalizing unit obtains the storage time and sends the storage time to the migration unit; the migration unit is used for setting a migration period, and when the migration period is reached, the migration unit stores the corresponding compressed monitoring video data into the offline storage unit.
8. The two-stage series storage system according to claim 7, wherein in the migration period, if the compressed monitor video data is called, the storage time is determined again, and the migration duration is calculated again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210993218.6A CN115514929A (en) | 2022-08-18 | 2022-08-18 | Two-stage series storage system based on video compression data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210993218.6A CN115514929A (en) | 2022-08-18 | 2022-08-18 | Two-stage series storage system based on video compression data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115514929A true CN115514929A (en) | 2022-12-23 |
Family
ID=84501793
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210993218.6A Pending CN115514929A (en) | 2022-08-18 | 2022-08-18 | Two-stage series storage system based on video compression data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115514929A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117241057A (en) * | 2023-09-08 | 2023-12-15 | 黄建邦 | Data storage method, video system, device, medium, product and storage system |
-
2022
- 2022-08-18 CN CN202210993218.6A patent/CN115514929A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117241057A (en) * | 2023-09-08 | 2023-12-15 | 黄建邦 | Data storage method, video system, device, medium, product and storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108828409A (en) | A kind of fault detection system based on edge calculations | |
| CN108564052A (en) | Multi-cam dynamic human face recognition system based on MTCNN and method | |
| CN102830669B (en) | Power distribution station room monitoring system | |
| CN103826109A (en) | Video monitoring image data processing method and system | |
| CN104639393A (en) | Passive optical network flow whole-network acquisition device and passive optical network flow whole-network acquisition method | |
| CN110765134A (en) | File establishing method, equipment and storage medium | |
| CN108391092A (en) | Danger identifying system based on deep learning | |
| CN112752113B (en) | Method and device for determining abnormal factors of live broadcast server | |
| CN115514929A (en) | Two-stage series storage system based on video compression data | |
| CN110248195B (en) | Method and apparatus for outputting information | |
| CN114363565A (en) | Video polling method, device, equipment and medium | |
| CN106993164B (en) | A kind of video monitoring method based on motion image detection | |
| JP2013143655A (en) | Site condition monitoring system and site image provision method | |
| CN112883213B (en) | Picture archiving method and device and electronic equipment | |
| CN116168045B (en) | Method and system for dividing sweeping lens, storage medium and electronic equipment | |
| CN109120896B (en) | Security video monitoring guard system | |
| CN107666610A (en) | Desktop video content analysis method | |
| CN116437099A (en) | Compression storage system based on data attribute | |
| CN104767964A (en) | Video storage control and optimization method and device, and video monitoring system | |
| CN114710674B (en) | Building construction monitoring data storage method | |
| CN108073854A (en) | A kind of detection method and device of scene inspection | |
| CN113177883B (en) | Arrangement transmission system based on data queue | |
| CN114863364B (en) | Security detection method and system based on intelligent video monitoring | |
| CN109587438B (en) | Differentiated video content service method oriented to intelligent video monitoring | |
| CN115878379A (en) | Data backup method, main server, backup server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |