CN112188246B - Video cloud storage method - Google Patents
Video cloud storage method Download PDFInfo
- Publication number
- CN112188246B CN112188246B CN202011065179.0A CN202011065179A CN112188246B CN 112188246 B CN112188246 B CN 112188246B CN 202011065179 A CN202011065179 A CN 202011065179A CN 112188246 B CN112188246 B CN 112188246B
- Authority
- CN
- China
- Prior art keywords
- file
- video
- cloud storage
- data
- storage method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/27—Server based end-user applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/432—Content retrieval operation from a local storage medium, e.g. hard-disk
- H04N21/4325—Content retrieval operation from a local storage medium, e.g. hard-disk by playing back content from the storage medium
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4334—Recording operations
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention discloses a video cloud storage method which comprises the steps of encrypting and uploading video data recorded by a device end to an object storage server of a third party, storing a corresponding video file index to an internal server, enabling the internal server to generate an m3u8 file in a corresponding time period through the file index, enabling an APP (application) to access the generated m3u8 file at any time through an api interface provided by the internal server, decrypting and playing a recorded video uploaded by corresponding equipment, marking whether a ts file exists at a certain time point by one bit, and compressing the time marks by using the bits according to a certain proportion to realize rapid indexing of the ts file. According to the method and the device, the corresponding file index information is stored into the redis by day through the file uploading convention of the server and the device, so that the storage cost is saved, and meanwhile, the query efficiency is greatly improved.
Description
Technical Field
The invention relates to the field of video storage, in particular to a video cloud storage method.
Background
m3u8 is an index file, which is used to resolve the corresponding video network address on the server by using m3u8 file, so as to realize online playing, the table storage (Tablestore) is a NoSQL multi-model database of the self-research of the Aries, and the file index refers to the relative path of the file stored in the third-party resource server. In the prior art, the video file index uploaded by the equipment is stored through the Ali cloud table storage, the ts file is uploaded by the equipment every 4 seconds, and meanwhile, the server side generates a data record for the ts file. When the APP accesses the cloud playback of the corresponding period, the server generates a corresponding m3u8 play file by querying the table storage data. The prior scheme has the following defects: the device generates one record every 4 seconds, the number of the stored records is increased along with the increase of the quantity of subsequent devices, and the efficiency of accessing the video index list in the corresponding time period by the APP is reduced.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a video cloud storage method.
In order to achieve the purpose, the invention adopts the following specific scheme:
a video cloud storage method comprises the steps of encrypting and uploading video data recorded by a device end to an object storage server of a third party, storing a corresponding video file index to an internal server, enabling the internal server to generate an m3u8 file in a corresponding time period through the file index, enabling an APP (application) to access the generated m3u8 file at any time through an api interface provided by internal services, decrypting and playing recorded videos uploaded by corresponding devices, marking whether ts files exist at a certain time point by one bit, compressing time marks by using the bits according to a certain proportion, and achieving fast indexing of the ts files.
Preferably, the method specifically comprises the following steps:
s1: the cloud storage record is directly and periodically stored in the redis, and the type is String;
s2: the redis stores the data Key as the equipment ID + date;
s3: the redis storage data Val is: the fixed-length header is reserved for realizing function expansion, the original data area stores a ts flag bit with fixed length, and the compressed storage is adopted for the records which do not need to be updated;
s4: operating with GetRange/SetRange/GetButt/SetButt provided by redis itself;
s5: automatically deleting expired keys by using an expiration mechanism of the redis;
s6: inquiring data, calculating a key list according to the starting time and the equipment ID, reading a command to the local mget key list, and performing local data processing through decompression/splicing, head/tail removal;
s7: updating the pseudo code;
s8: and data compression processing, which is executed once a day, compresses the number of previous days, and refreshes data from the buffer.
Preferably, the video data length of each ts file is 4 s.
Preferably, when the device generates the ts file, the ts file name is consistent with the actual timestamp.
Preferably, the utc start time stamp of the ts file is an integer multiple of 4.
Preferably, the number of days of cloud storage is fixed.
Preferably, the object storage path remains unchanged.
Preferably, the file name and the timestamp are consistent, and the cloud file name does not need to be saved.
By adopting the technical scheme of the invention, the invention has the following beneficial effects:
the invention stores the index information of the corresponding file into the redis by day through the file uploading convention of the server and the equipment, thereby greatly improving the query efficiency while saving the storage cost and having the following advantages: the cost is low, the QPS for redis access of 100W equipment is about 50W, the data volume required each time is small, and the requirement can be met by a common redis cluster; high performance, very fast querying; technically, all operations are atomic, and the risk of data inconsistency does not exist; the network overhead can be small, and the rendering and the generation of the m3u8 file can be performed on the end side; there is no dependency on a particular cloud vendor.
Drawings
FIG. 1 is a flow chart of the present invention.
Detailed Description
The invention is further described below with reference to the following figures and specific examples.
Referring to fig. 1, the present invention provides a video cloud storage method, which includes encrypting and uploading video data recorded at a device end to an object storage server of a third party, storing a corresponding video file index to an internal server, where the internal server can generate an m3u8 file at a corresponding time interval through the file index, accessing the generated m3u8 file at any time through an api interface provided by the internal server by an APP, decrypting and playing a recorded video uploaded by the corresponding device, marking whether a ts file exists at a certain time point with one bit, and compressing the time stamp with the bit according to a certain proportion, so as to realize fast indexing of the ts file. The method specifically comprises the following steps:
s1: the cloud storage record is directly and periodically stored in the redis, and the type is String;
s2: redis stores the data Key as device ID + date, e.g., devid _ 20200408;
s3: the redis storage data Val is: a fixed-length header (for example, 8 bytes) reserved for function expansion, and an original data area stores a ts flag bit (21600 bits/2700 bytes) of a fixed length, and for records that do not need to be updated (not the current day), compressed storage can be used;
s4: operating with GetRange/SetRange/GetButt/SetButt provided by redis itself;
s5: automatically deleting expired keys by using an expiration mechanism of the redis;
s6: inquiring data, calculating a key list according to the starting time (which can span the day) and the equipment ID, reading a command to the local mget key list, and performing local data processing through decompression/splicing, head removal/tail removal;
s7: updating the pseudo code;
s8: and data compression processing, which is executed once a day, compresses the number of previous days, and refreshes data from the buffer.
Appointing and precondition: the video data length of each ts file is 4s, when the device generates the ts file, the ts file name is consistent with the actual time scale, the utc starting time scale of the ts file is integral multiple of 4, the cloud storage days are fixed (for example, 7 days), and the object storage path is kept unchanged.
The file name is consistent with the time mark, and the cloud storage file name does not need to be stored. Whether ts files exist at a certain time point is marked by one bit, the total data storage original data size of 1 day is 2.7kb, the data of 2.7kb are very regular in general, and historical data can be compressed and stored, assuming that the compression ratio is about 6:
uncompressed storage (7 days): 2.7[ kB ] 8[ day ] ═ 21.6(kB)
Uncompressed storage (30 days): 2.7[ kB ]. 31[ day ]. 83.7(kB)
Compression treatment (7 days): 2.7kB 2 days + (2.7 kB 6 days)/6 compression ratio 8.1(kB)
Compression treatment (30 days): 2.7kB 2 days + (2.7 kB 29 days)/6 compression ratio 18.45(kB)
100 ten thousand devices 7 days uncompressed storage: 21.6G
100 ten thousand devices 30 days uncompressed storage: 83.7G
The maximum storage space is compressed for 7 days by 100 ten thousand devices: 8.1G
The maximum storage space is compressed for 30 days by 100 ten thousand devices: 18.45G
The invention can use the document index uploaded by the nosql database sub-table storage device, is feasible on the scheme, but has lower query efficiency than the current scheme and increased storage space.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (7)
1. A video cloud storage method comprises the steps that video data recorded at an equipment end are encrypted and uploaded to an object storage server of a third party, a corresponding video file index is stored in an internal server, the internal server can generate an m3u8 file in a corresponding time period through the file index, an APP can access the generated m3u8 file at any time through an api interface provided by internal services, and the recorded video uploaded by corresponding equipment is decrypted and played, wherein one bit is used for marking whether a ts file exists at a certain time point or not, and the time scale is compressed according to a certain proportion by using the bit, so that the ts file is quickly indexed; the video cloud storage method specifically comprises the following steps:
s1: the cloud storage record is directly and periodically stored in the redis, and the type is String;
s2: the redis stores the data Key as the equipment ID + date;
s3: the redis storage data Val is: the fixed-length header is reserved for realizing function expansion, the original data area stores a ts flag bit with fixed length, and the compressed storage is adopted for the records which do not need to be updated;
s4: the method is operated by utilizing GetRange, SetRange, GetButt and SetButt provided by redis;
s5: automatically deleting expired keys by using an expiration mechanism of the redis;
s6: inquiring data, calculating a key list according to the starting time and the equipment ID, reading a command to the local mget key list, and performing local data processing through decompression, splicing, head and tail removal;
s7: updating the pseudo code;
s8: and data compression processing, which is executed once a day, compresses the number of previous days, and refreshes data from the buffer.
2. The video cloud storage method according to claim 1, wherein the video data length of each ts file is 4 s.
3. The video cloud storage method according to claim 2, wherein when the device generates the ts file, the ts file name is consistent with the real time stamp.
4. The video cloud storage method of claim 3, wherein the utc start time stamp of the ts file is an integer multiple of 4.
5. The video cloud storage method of claim 4, wherein the number of days of cloud storage is fixed.
6. The video cloud storage method of claim 5, wherein the object storage path remains unchanged.
7. The video cloud storage method of claim 6, wherein the file name is consistent with the timestamp without saving the cloud storage file name.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011065179.0A CN112188246B (en) | 2020-09-30 | 2020-09-30 | Video cloud storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011065179.0A CN112188246B (en) | 2020-09-30 | 2020-09-30 | Video cloud storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112188246A CN112188246A (en) | 2021-01-05 |
| CN112188246B true CN112188246B (en) | 2022-03-22 |
Family
ID=73948137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011065179.0A Active CN112188246B (en) | 2020-09-30 | 2020-09-30 | Video cloud storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112188246B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407770A (en) * | 2021-06-24 | 2021-09-17 | 深圳智多豚物联技术有限公司 | Cloud playback index service method and system based on redis |
| CN114449196B (en) * | 2021-12-21 | 2023-08-04 | 天翼云科技有限公司 | Cloud video monitoring data storage method and device and electronic equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686246A (en) * | 2013-11-26 | 2014-03-26 | 小米科技有限责任公司 | Timed playback method, device, facility and system of transport stream video |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3908691B2 (en) * | 2003-04-28 | 2007-04-25 | 株式会社東芝 | Information recording medium, pre-recording processing method, information recording method, information reproducing method, information recording apparatus, and information reproducing apparatus |
| WO2010123203A2 (en) * | 2009-04-22 | 2010-10-28 | 엘지전자 주식회사 | Reference picture list changing method of multi-view video |
| CN105808546A (en) * | 2014-12-30 | 2016-07-27 | 中国电信股份有限公司 | Video file index management platform, system and method |
| CN106777159B (en) * | 2016-12-20 | 2020-04-28 | 西安交通大学 | Video clip retrieval and positioning method based on content |
-
2020
- 2020-09-30 CN CN202011065179.0A patent/CN112188246B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686246A (en) * | 2013-11-26 | 2014-03-26 | 小米科技有限责任公司 | Timed playback method, device, facility and system of transport stream video |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112188246A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112188246B (en) | Video cloud storage method | |
| JP4598627B2 (en) | Content editing apparatus and playback apparatus thereof | |
| WO2018076952A1 (en) | Method and apparatus for storage and playback positioning of video file | |
| JP5513400B2 (en) | Hierarchical and simple index structure for multimedia files | |
| US8886896B2 (en) | Storage format for media streams | |
| EP2965189B1 (en) | Managing operations on stored data units | |
| US20170123713A1 (en) | Device and process for data storage and read/write efficiency | |
| AU2014226446B2 (en) | Managing operations on stored data units | |
| CN1224910C (en) | Enciphering method, decipher method and device and information recording medium | |
| CN111291235A (en) | Metadata storage method and device based on time sequence database | |
| CN108694331A (en) | A kind of date storage method, device, system and computer readable storage medium | |
| EP2297959B1 (en) | Linear hint video streaming | |
| CN110928899A (en) | Universal database backup method and system | |
| CN110493324A (en) | Method for down loading, downloader and computer readable storage medium | |
| CN111414341B (en) | Data normalization description method in Internet of things environment | |
| US9070403B2 (en) | Processing of scalable compressed video data formats for nonlinear video editing systems | |
| WO2022223540A1 (en) | System and method for encoding audio data | |
| CN113438503B (en) | Video file restoring method, device, computer equipment and storage medium | |
| US8988258B2 (en) | Hardware compression using common portions of data | |
| CN111813965B (en) | Encryption method and decryption method for multimedia file, storage medium and electronic device | |
| CN110262752B (en) | Method, device and storage medium for storing streaming media data | |
| CN103929447B (en) | A kind of method of the parsing HTTP chunked coded datas based on PEEK operations | |
| CN111814182A (en) | File encryption method, file decryption method, file encryption equipment and file decryption equipment and storage medium | |
| KR100896830B1 (en) | Data update method for video on demand in set-top box and the set-top box | |
| US20170308561A1 (en) | Indexing and sequentially storing variable-length data to facilitate reverse reading |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |