CN112584155B - Video data processing method and device - Google Patents
Video data processing method and device Download PDFInfo
- Publication number
- CN112584155B CN112584155B CN202011461673.9A CN202011461673A CN112584155B CN 112584155 B CN112584155 B CN 112584155B CN 202011461673 A CN202011461673 A CN 202011461673A CN 112584155 B CN112584155 B CN 112584155B
- Authority
- CN
- China
- Prior art keywords
- index file
- video stream
- video
- file
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
- H04N19/23—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
Abstract
A video data processing method and apparatus, the method comprising: acquiring video data and coding the video data; the coded video data are segmented to form a plurality of video stream segments, and a first index file is established; lossless compression is carried out on the video stream slices; and uploading the first index file and the compressed video stream to a cloud for storage, and establishing a second index file according to the first index file. The application reduces the data volume of network bandwidth transmission by performing lossless compression on the coded video stream fragments, and saves the cloud space by secondary lossless compression, thereby saving the storage cost. According to the method and the device, only the index file and the compressed video file are concerned about in cloud storage, and complexity is greatly reduced in the cloud storage, so that the stability of cloud service is improved, meanwhile, the key information of the video is processed locally, the complex service is put to a local non-cloud end, the maintenance difficulty can be reduced, and the maintenance cost is reduced.
Description
Technical Field
The application relates to the field of security and protection, in particular to a video data processing method and device.
Background
With the development trend of national public security construction and internet, the storage of videos by social public security construction is gradually changed from edge storage and regional autonomous storage into sharization, centralization and clouding, and higher requirements are made on the integrity of video storage and the security of data.
The current video cloud storage situation is as follows: the video data is huge, the internet bandwidth is expensive, and the internet cloud storage space is expensive. Both of these factors slow down the speed of video centralized cloud storage.
Video stream data is always large data transmitted in a network, and even if encoding algorithms such as H264 and H265 exist, the video stream data still occupies a large bandwidth in network transmission and consumes a large space for storage. With the development of internet cloud, the storage of videos is also gradually going to the cloud. The existing video stream data is stored in a cloud storage service in a fragmentation mode, video key attributes (I frame position, file header attribute, frame type and the like) of a video file need to be concerned during storage to be used for processing video on demand service, and complex video on demand service processing is relatively complex to realize on the cloud storage, so that the stability is difficult to guarantee.
Disclosure of Invention
The application provides a video data processing method and device.
According to a first aspect of the present application, there is provided a video data processing method, comprising:
acquiring video data and coding the video data;
the coded video data is divided into a plurality of video stream fragments, and a first index file is established;
lossless compression is carried out on the video stream slices;
and uploading the first index file and the compressed video stream to a cloud for storage, and establishing a second index file according to the first index file.
According to a second aspect of the present application, there is provided a video data processing apparatus comprising:
the encoding module is used for acquiring video data and encoding the video data;
the fragmentation module is used for fragmenting the coded video data to form a plurality of video stream fragments and establishing a first index file;
the compression module is used for carrying out lossless compression on the video stream fragments;
and the storage module is used for uploading the first index file and the compressed video stream to a cloud end for storage and establishing a second index file according to the first index file.
According to a third aspect of the present application, there is provided a video data processing apparatus comprising:
a memory for storing a program;
a processor for implementing the method of any one of claims 1-4 by executing a program stored by the memory.
According to a fourth aspect of the present application, there is provided a computer readable storage medium comprising a program executable by a processor to implement the above method.
Due to the adoption of the technical scheme, the beneficial effects of the application are as follows:
the video data processing method and device provided by the embodiment of the application comprise the following steps: acquiring video data and coding the video data; the coded video data are segmented to form a plurality of video stream segments, and a first index file is established; lossless compression is carried out on the video stream slices; and uploading the first index file and the compressed video stream to a cloud for storage, and establishing a second index file according to the first index file. The application reduces the data volume of network bandwidth transmission by performing lossless compression on the coded video stream fragments, and meanwhile, the cost of cloud space storage is higher than that of local disk storage, so that the cloud space can be saved by secondary lossless compression, and the storage cost is saved. The video file compression method and the video file compression device have the advantages that only the index file and the compressed video file are concerned about in cloud storage, attribute information of the video file is not concerned, simple file indexing and access functions are achieved, complexity is greatly reduced in cloud storage, stability of cloud service is improved, meanwhile, the video key information is processed in a compression mode, services such as compression and decompression are placed locally, complex services are placed locally but not in the cloud, maintenance difficulty can be reduced, and maintenance cost is reduced.
Drawings
FIG. 1 is a flow chart of a method in one embodiment;
FIG. 2 is a flow chart of a method in one embodiment;
FIG. 3 is a schematic diagram of program modules of an apparatus according to a second embodiment;
FIG. 4 is a schematic diagram of program modules of an apparatus according to a second embodiment of the present disclosure.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. The present application may be embodied in many different forms and is not limited to the embodiments described in the present embodiment. The following detailed description is provided to facilitate a more thorough understanding of the present disclosure, and the words used to indicate orientation, top, bottom, left, right, etc. are used solely to describe the illustrated structure in connection with the accompanying figures.
One skilled in the relevant art will recognize, however, that one or more of the specific details can be omitted, or other methods, components, or materials can be used. In some instances, some embodiments are not described or not described in detail.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning.
Furthermore, the technical features, aspects described herein may also be combined in any suitable manner in one or more embodiments. It will be readily appreciated by those of skill in the art that the order of the steps or operations of the methods associated with the embodiments provided herein may be varied. Thus, any sequence in the figures and examples is for illustrative purposes only and does not imply a requirement in a certain order unless explicitly stated to require a certain order.
The embodiment of the application provides a scheme for reducing the cost of video cloud storage in a file compression mode. According to the scheme, the streaming video is converted into the file, and then the file is compressed and stored, so that the complexity of the cloud storage service on video storage is simplified, and the stability of the service is indirectly improved. According to the embodiment of the application, the video is collected by the edge device and stored until the video is forwarded, the streaming video is firstly subjected to fragmentation processing in the process, and then the larger video file is subjected to secondary compression through a file compression algorithm, so that fragmentation indexes are not damaged. And uploading to a cloud storage service. The cloud storage service creates a storage index, and logically assembles and stores the video index file and the video fragment file. For the cloud storage service, only the video index file and the target file corresponding to the index are considered, simple logic storage is completed, and key information (such as I frames and P frames) of the video is not needed to be concerned. Thereby reducing the business complexity of the stored services. The logical file storage is completed. Meanwhile, the compressed file is equivalently encrypted once, so that the data security is enhanced. The preposed compression processing can be carried out during storage because the video storage service has low requirement on real-time performance, fragmentation can be completed by using a high-performance server, and lossless compression can ensure complete storage of data.
The first embodiment is as follows:
as shown in fig. 1, an implementation manner of a video data processing method provided in an embodiment of the present application includes the following steps:
step 102: video data is obtained and encoded. The format of the code may include H264 or H265.
Video capture typically results in image data based on RGB pixels. Before video data is transmitted, compression coding is performed according to the characteristics of video, and common coding formats of standards are H264 and H265.
Step 104: and carrying out fragmentation on the coded video data to form a plurality of video stream fragments, and establishing a first index file. The format of the video stream slices comprises an MPEG-TS format.
Step 106: and carrying out lossless compression on the video stream slices. The lossless compression specifically adopts a compression algorithm including LZ77 or LZSS.
After the streaming video data is received, the video can be sliced into small files according to summary information (such as time, frame information, etc.) of the video. Such as 5 seconds or 10 seconds per file. And simultaneously, associating by utilizing a summary index file.
The summary file is generally very small, the video file is relatively large, the video file after the video stream is segmented can be subjected to secondary lossless compression aiming at the file, and common lossless compression algorithms include LZ77, LZSS and the like. The compressed video file becomes smaller, and then internet network transmission is carried out at the moment, so that the purpose of reducing bandwidth consumption is achieved.
Step 108: and uploading the first index file and the compressed video stream to a cloud for storage, and establishing a second index file according to the first index file.
And the uploading and the cloud storage service are simply transmitted and stored. The cloud storage service only needs to perform simple logic storage according to the types and the relations of the two files, and does not need to care about the characteristics (video frames) of the videos. The storage index is simply index + time + videoIndex. And completing the cloud storage process of the video.
The first index file may include the following fields: ID, data channel, start time, end time, file name;
the second index file adds fields on the basis of the first index file: the file name after compression;
and establishing a second index file according to the first index file, wherein the method specifically comprises the step of storing the file names of the compressed video stream fragments into the compressed file names.
As shown in fig. 2, in another implementation manner of the video data processing method provided in the embodiment of the present application, the method may further include the following steps:
step 110: and downloading the corresponding second index file and the compressed video stream fragments from the cloud according to the playing request.
And matching the condition information (such as start time, end time and the like) of the received video recording request with the index time of the storage service file to find the video index file in the corresponding time period. And after the video index file is determined, the compressed fragment video file can be determined continuously according to time, and the file is determined and then the file downloading operation of cloud storage is carried out. And downloading the index file of the video and the compressed fragment video file to a local service.
Step 112: and decompressing the downloaded compressed video stream fragments, and converting the second index file into the first index file to obtain streaming video data to be played.
Decompressing the compressed fragment video file by using a decompression algorithm, and restoring the fragment file by using the video fragment index file and the decompressed fragment video file in a fluidized manner to provide service streaming video data until the playback/download process is finished.
The process downloads and restores the fragment data, and the streaming restoration can be performed as long as a complete fragment file is downloaded, without much loss in the graph efficiency.
Example two:
as shown in fig. 3, an implementation manner of the video data processing apparatus provided in the embodiment of the present application includes an encoding module 310, a slicing module 320, a compression module 330, and a storage module 340.
The encoding module 310 is configured to obtain video data and encode the video data. The format of the code may include H264 or H265.
Video capture typically results in image data based on RGB pixels. Before video data is transmitted, compression coding is performed according to the characteristics of video, and the coding formats of common standards are H264 and H265.
The fragmentation module 320 is configured to fragment the encoded video data into multiple video stream fragments, and create a first index file. The format of the video stream slices comprises an MPEG-TS format.
And the compression module 330 is configured to perform lossless compression on the video stream slices. The lossless compression specifically adopts a compression algorithm including LZ77 or LZSS.
After the streaming video data is received, the video can be sliced into small files according to summary information (such as time, frame information, etc.) of the video. Such as 5 seconds or 10 seconds per file. And simultaneously, associating by utilizing a summary index file.
The summary file is generally very small, the video file is relatively large, the video file after the video stream is segmented can be subjected to secondary lossless compression aiming at the file, and common lossless compression algorithms include LZ77, LZSS and the like. The compressed video file becomes smaller, and then internet network transmission is carried out at the moment, so that the purpose of reducing bandwidth consumption is achieved.
The storage module 340 is configured to upload the first index file and the compressed video stream to the cloud for storage, and establish a second index file according to the first index file.
The first index file may include the following fields: ID, data channel, start time, end time, file name;
the second index file adds fields on the basis of the first index file: the file name after compression;
the fragmentation module 320 is further configured to store the file name of the compressed video stream fragment into the compressed file name. The format of the code comprises H264 or H265; the format of the video stream slices comprises an MPEG-TS format; lossless compression specifically employs compression algorithms including LZ77 or LZSS.
As shown in fig. 4, an implementation manner of the video data processing apparatus provided in the embodiment of the present application further includes:
and the downloading module 350 is configured to download the corresponding second index file and the compressed video stream segments from the cloud according to the play request.
And matching the condition information (such as start time, end time and the like) of the received video recording request with the index time of the storage service file to find out the video index file in the corresponding time period. And after the video index file is determined, the compressed fragment video file can be determined continuously according to time, and the file is determined and then the file downloading operation of cloud storage is carried out. And downloading the index file of the video and the compressed fragment video file to a local service.
The processing module 360 is configured to decompress the downloaded compressed video stream segment, and convert the second index file into the first index file to obtain streaming video data to be played.
Decompressing the compressed fragment video file by using a decompression algorithm, and restoring the fragment file by using the video fragment index file and the decompressed fragment video file in a fluidized manner to provide service streaming video data until the playback/download process is finished.
In the process, the fragment data is downloaded and restored, and the fluidization restoration can be performed as long as a complete fragment file is downloaded, so that the efficiency of the graph is not lost.
Example three:
an implementation manner of a video data processing apparatus provided in an embodiment of the present application includes a memory and a processor.
A memory for storing a program.
And the processor is used for executing the program stored in the memory to realize the method in the first embodiment.
Example four:
a computer-readable storage medium comprising a program executable by a processor to perform the method of the first embodiment.
Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments may be implemented by hardware, or may be implemented by computer programs. When all or part of the functions of the above embodiments are implemented by a computer program, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to realize the above functions. For example, the program may be stored in a memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above may be implemented. In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and may be downloaded or copied to a memory of a local device, or may be version-updated in a system of the local device, and when the program in the memory is executed by a processor, all or part of the functions in the above embodiments may be implemented.
The present invention has been described in terms of specific examples, which are provided to aid understanding of the invention and are not intended to be limiting. For a person skilled in the art to which the invention pertains, several simple deductions, modifications or substitutions may be made according to the idea of the invention.
Claims (8)
1. A method of processing video data, comprising:
acquiring video data, and performing compression coding on the video data;
the video data after compression coding is sliced to form a plurality of video stream slices, and a first index file is established;
lossless compression is carried out on the video stream slices;
uploading the first index file and the compressed video stream to a cloud for storage, and establishing a second index file according to the first index file, wherein the first index file comprises the following fields: ID, data channel, start time, end time, filename; the establishing of the second index file according to the first index file comprises the following steps: the second index file adds fields on the basis of the first index file: the file name after compression; and storing the compressed file name of the video stream fragment into the compressed file name.
2. The method of claim 1, wherein the format of the compression encoding comprises H264 or H265; the format of the video stream slices comprises an MPEG-TS format; the lossless compression specifically employs a compression algorithm including LZ77 or LZSS.
3. The method of claim 1, further comprising:
downloading a corresponding second index file and the compressed video stream fragments from a cloud according to a playing request;
decompressing the downloaded compressed video stream fragments, and converting the second index file into the first index file to obtain streaming video data to be played.
4. A video data processing apparatus, comprising:
the encoding module is used for acquiring video data and performing compression encoding on the video data;
the fragmentation module is used for fragmenting the video data subjected to compression coding to form a plurality of video stream fragments and establishing a first index file;
the compression module is used for carrying out lossless compression on the video stream fragments;
the storage module is used for uploading the first index file and the compressed video stream to a cloud end for storage, and establishing a second index file according to the first index file;
wherein the first index file comprises the following fields: ID, data channel, start time, end time, filename; adding fields to the second index file on the basis of the first index file: the file name after compression; and the fragment module is also used for storing the compressed file name of the video stream fragment into the compressed file name.
5. The apparatus of claim 4, wherein the format of the compression encoding comprises H264 or H265; the format of the video stream slices comprises an MPEG-TS format; the lossless compression specifically employs a compression algorithm including LZ77 or LZSS.
6. The apparatus of claim 4, further comprising:
the downloading module is used for downloading the corresponding second index file and the compressed video stream fragments from the cloud according to the playing request;
and the processing module is used for decompressing the downloaded compressed video stream fragments and converting the second index file into the first index file to obtain streaming video data to be played.
7. A video data processing apparatus, comprising:
a memory for storing a program;
a processor for implementing the method of any one of claims 1-3 by executing a program stored by the memory.
8. A computer-readable storage medium, characterized by comprising a program executable by a processor to implement the method of any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011461673.9A CN112584155B (en) | 2020-12-11 | 2020-12-11 | Video data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011461673.9A CN112584155B (en) | 2020-12-11 | 2020-12-11 | Video data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112584155A CN112584155A (en) | 2021-03-30 |
| CN112584155B true CN112584155B (en) | 2022-11-04 |
Family
ID=75132160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011461673.9A Active CN112584155B (en) | 2020-12-11 | 2020-12-11 | Video data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112584155B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113395487A (en) * | 2021-06-01 | 2021-09-14 | 平安科技（深圳）有限公司 | Video data storage management method and device, computer equipment and storage medium |
| CN114449001A (en) * | 2021-12-30 | 2022-05-06 | 天翼云科技有限公司 | Cloud storage implementation method, device, equipment and storage medium of streaming media data |
| CN115865905B (en) * | 2023-02-21 | 2023-06-13 | 深圳依时货拉拉科技有限公司 | Method, apparatus, computer device and computer readable storage medium for downloading compressed package |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101141627A (en) * | 2007-10-23 | 2008-03-12 | 深圳市迅雷网络技术有限公司 | Storage system and method of stream media file |
| CN104954890A (en) * | 2015-06-24 | 2015-09-30 | 深圳市世文通文化传播有限公司 | Cloud storage image fragmenting and distributing method |
| CN105338424A (en) * | 2015-10-29 | 2016-02-17 | 努比亚技术有限公司 | Video processing method and system |
| CN105808546A (en) * | 2014-12-30 | 2016-07-27 | 中国电信股份有限公司 | Video file index management platform, system and method |
| WO2017035787A1 (en) * | 2015-09-01 | 2017-03-09 | 深圳好视网络科技有限公司 | Method, device, and system for storing streaming media file |
| CN108235151A (en) * | 2017-12-29 | 2018-06-29 | 北京奇虎科技有限公司 | A kind of method and apparatus of net cast |
| CN109783446A (en) * | 2019-01-04 | 2019-05-21 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN111709038A (en) * | 2020-05-07 | 2020-09-25 | 北京中科凡语科技有限公司 | File encryption and decryption method, distributed storage system, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101957836B (en) * | 2010-09-03 | 2012-07-11 | 清华大学 | Configurable real-time transparent compressing method in file system |
| CN104657362B (en) * | 2013-11-18 | 2018-07-10 | 深圳市腾讯计算机系统有限公司 | Data storage, querying method and device |
-
2020
- 2020-12-11 CN CN202011461673.9A patent/CN112584155B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101141627A (en) * | 2007-10-23 | 2008-03-12 | 深圳市迅雷网络技术有限公司 | Storage system and method of stream media file |
| CN105808546A (en) * | 2014-12-30 | 2016-07-27 | 中国电信股份有限公司 | Video file index management platform, system and method |
| CN104954890A (en) * | 2015-06-24 | 2015-09-30 | 深圳市世文通文化传播有限公司 | Cloud storage image fragmenting and distributing method |
| WO2017035787A1 (en) * | 2015-09-01 | 2017-03-09 | 深圳好视网络科技有限公司 | Method, device, and system for storing streaming media file |
| CN105338424A (en) * | 2015-10-29 | 2016-02-17 | 努比亚技术有限公司 | Video processing method and system |
| CN108235151A (en) * | 2017-12-29 | 2018-06-29 | 北京奇虎科技有限公司 | A kind of method and apparatus of net cast |
| CN109783446A (en) * | 2019-01-04 | 2019-05-21 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN111709038A (en) * | 2020-05-07 | 2020-09-25 | 北京中科凡语科技有限公司 | File encryption and decryption method, distributed storage system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112584155A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112584155B (en) | Video data processing method and device | |
| US8578058B2 (en) | Real-time multi-block lossless recompression | |
| US11736723B2 (en) | Method of coding and decoding images, coding and decoding device and computer programs corresponding thereto | |
| US8533166B1 (en) | Methods and systems for encoding/decoding files and transmission thereof | |
| US8138954B2 (en) | Method and system for efficient data transmission with server side de-duplication | |
| RU2503138C2 (en) | Embedded graphics coding for images with sparse histograms | |
| US11765390B2 (en) | Non-transform coding | |
| US20190180478A1 (en) | Lossless compression of fragmented image data | |
| EP3831065B1 (en) | Entropy coding for signal enhancement coding | |
| US8682091B2 (en) | Real-time image compression | |
| KR20170079852A (en) | Method and system for optimization of image encode quality | |
| EP2932714B1 (en) | Method of coding and decoding images, device for coding and decoding and computer programs corresponding thereto | |
| US7123656B1 (en) | Systems and methods for video compression | |
| CN115361556A (en) | High-efficiency video compression algorithm based on self-adaption and system thereof | |
| US7773815B2 (en) | System and method for compressing compressed data | |
| Reddy et al. | A novel approach of lossless image compression using hashing and Huffman coding | |
| CN105072456A (en) | Hadoop-based ciphertext video flow processing method, device, server and system | |
| KR20150060149A (en) | Storage device and method for distributed processing of multimedia data | |
| Bonde et al. | Data compression techniques for big data | |
| WO2023059689A1 (en) | Systems and methods for predictive coding | |
| CN114339196A (en) | Video coding method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |