CN107229418B

CN107229418B - Video file storage method and device

Info

Publication number: CN107229418B
Application number: CN201710302271.6A
Authority: CN
Inventors: 吕志明
Original assignee: Zhejiang Uniview Technologies Co Ltd
Current assignee: Xi'an Yu Vision Mdt Infotech Ltd
Priority date: 2017-05-02
Filing date: 2017-05-02
Publication date: 2020-07-28
Anticipated expiration: 2037-05-02
Also published as: CN107229418A

Abstract

The invention discloses a video file storage method. The video file is written into the blank fragment data segment, and the address of the fragment data segment is acquired and stored after the writing is finished. Therefore, the problem that the video file cannot be read due to the fact that the file index in the RAID storage system is damaged can be solved on the premise that the storage performance is not reduced, and the stability of the RAID storage system is improved.

Description

Video file storage method and device

Technical Field

The invention relates to the field of video data storage, in particular to a video file storage method. The invention also relates to a video file storage device.

Background

With the rapid development of network technology and the gradual maturity of digitization, networking, high-definition and intellectualization of video monitoring, the video acquisition technology of a video monitoring front-end camera is also rapidly developed, the video definition is higher and higher from 10 ten thousand pixels at the early stage to 200 ten thousand and 500 ten thousand pixels at present, and the information content in a video picture is richer and richer. As a final place for collecting and distributing security video management system information, a storage technology is more and more emphasized by people.

In the video monitoring industry, ensuring that a history video can be played as far as possible is a key index for judging whether a video monitoring system is robust. In the monitoring Storage solution, the most commonly used is IPC (IP Camera) + IPSAN (Storage Area Network) networking, and the Storage solution provides RAID (Redundant Arrays of Independent Disks) 5-level protection by using the IPSAN.

RAID is a combination of many disks into a large disk set, and the performance of the entire disk system is enhanced by the additive effect of providing data from individual disks. With this technique, a technician can cut data into a number of sections, each stored on a respective hard disk. As one of the specifications of RAID technology, RAID5 can use at least 3 (or more) hard disks to construct a RAID5 disk array, when data is written into a hard disk, the data writing is divided into 3 parts according to an algorithm, then the 3 parts are written into the 3 hard disks, check information is written into the 3 hard disks while writing, and when the written data is read, data content is read from the 3 hard disks, and then check is performed by the check information. When 1 hard disk is damaged, the data content of the 3 rd hard disk can be calculated from the data stored in the other 2 hard disks.

The inventor finds that when a check disk exists, the damage of one disk does not affect the storage of the video file, but the storage mode of the RAID only allows one hard disk to break down at the same time, and when two disks are damaged successively, the data of the whole disk array cannot be read and written because the file index is damaged. If the single disk service is provided without using the array, a file system needs to be formatted on each disk, and the file is stored on one disk independently or one file is divided into a plurality of copies to be stored on a plurality of disks.

Therefore, how to enable the video files stored in the disk array to be still read under the premise that the hard disk is damaged to cause the file index to be damaged becomes a technical problem to be urgently solved by technical personnel in the field.

Disclosure of Invention

The invention provides a video file storage method, which is used for solving the problem that the video files stored in a disk array can not be read normally according to file indexes under the condition that a disk has a problem in the prior art, and is applied to a disk array RAID storage system, and comprises the following steps:

writing the video file into a blank fragment data segment, and acquiring and storing the address of the fragment data segment after the writing is finished, wherein the fragment data segment corresponds to the video file one by one, and the length of the fragment data segment is greater than the file size of the video file;

and when the file index of the video file is abnormal, acquiring the video data of the video file in the fragment data segment according to the address.

Correspondingly, the present application further provides a video file storage apparatus, which is applied to a RAID storage system, and includes:

the writing module is used for writing the video file into a blank fragment data segment, and acquiring and storing the address of the fragment data segment after the writing is finished, wherein the fragment data segment corresponds to the video file one by one, and the length of the fragment data segment is greater than the file size of the video file;

and the acquisition module is used for acquiring the video data of the video file in the fragment data segment according to the address when the file index of the video file is abnormal.

Therefore, by applying the technical scheme of the application, the video file is written into the blank fragment data segment, and the address of the fragment data segment is obtained and stored after the writing is finished. Therefore, the problem that the video file cannot be read due to the fact that the file index in the RAID storage system is damaged can be solved on the premise that the storage performance is not reduced, and the stability of the RAID storage system is improved.

Drawings

Fig. 1 is a schematic flowchart of a video file storage method according to an embodiment of the present application;

FIG. 2 is a diagram illustrating a conventional storage format used in a RAID storage system according to the prior art;

FIG. 3 is a diagram illustrating a video index block format according to an embodiment of the present application;

FIG. 4 is a diagram illustrating a method for processing a video index block according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a video file storage device according to an embodiment of the present application.

Detailed Description

As described in the background, when a plurality of disks in a RAID storage system fail, the file index of the video files stored in the RAID storage system may be destroyed, thereby rendering the stored video files inaccessible. Even if an index of all video files in the current RAID storage system is completely copied on other disk arrays or other equipment in a redundant backup mode. However, the file system structure will occupy a large space as an index area, and both simple backup and full backup will reduce the performance of other logical volumes, thereby affecting video recording services.

In view of the problems in the prior art, the present invention provides a method for storing a video file, which is applied to a RAID storage system, as shown in fig. 1, and includes the following steps:

s101, writing the video file into a blank fragment data segment, and acquiring and storing the address of the fragment data segment after the writing is completed, wherein the fragment data segment corresponds to the video file one by one, and the length of the fragment data segment is greater than the file size of the video file.

To facilitate the explanation of the technical solution of the present invention, the following first introduces concepts involved in the solution, as shown in fig. 2, which is a schematic diagram of a common storage format adopted by a RAID storage system. The storage format divides a logical volume into the following three portions:

(1) formatting information: and the address information is used for describing the volume index, the index log, the directory index and the file index in the Slice index area.

(2) Slice index area: information for describing a plurality of data segments within the partitioned data area. Within the Slice index zone, the volume index describes the fragment segment addresses of the entire fragment data zone. The directory index and the file index record video file directory information (directory name, creation time, etc.), video file information (file name, file creation time, etc.), and occupied fragment data segment address information.

(3) A partitioned data area: the fragmented data area is divided into a plurality of fragmented data segments (default is 64MB in the prior art) according to a fixed length, and the fragmented data segments are a continuous segment of data with a fixed length.

When the video file is stored in the RAID storage system applying the file storage format, the content to be processed is divided into two parts of file description information and file data. The file description information comprises the name, the creation time and one or more occupied fragment data segment addresses of the video file. And the video data of the video file is stored in one or more fragment data segments. After the video data is stored, the file description information is used as a file index of the video file, the address of the file in the RAID storage system is returned to the user, the user saves the record to a database, and then the user can read the corresponding video data through the address.

However, in RAID, data is divided into a plurality of segments to be stored in each disk, so when more than one disk in the RAID storage system cannot be accessed due to a failure, the stripe-structured data storage method will cause the data segments at intervals to be unreadable. Taking the raid5 array using the storage format as an example, assuming that the size of the stripe block (constituting the array stripe basic unit) of the raid5 array is 64K, an array is constituted by the hard disk 4+1, when two disks are pulled out, the continuous complete data length is only 3 × 64K, and 64K data is lost in the middle interval. In this case, no matter where the file index is located in the logical volume (the logical resource divided over the array), it is inevitable that the file index data is damaged, and the storage location where the video file is obtained by the user is not accessible.

Based on the above description, in order to ensure that the recorded data of the recorded file is continuously readable under the condition that the file system index is damaged, the following three conditions are realized through the steps:

(1) one video file is only stored in one fragment data segment:

(2) on the basis of the prior art, the length of a unit fragment data segment is increased, so that a single video file can be accommodated;

(3) and (4) taking the address of the fragment data segment as another file index (the existing file index processing mode is kept unchanged) for storage.

For example, after the video file with the size of 2G is written into a certain data segment of the data segment in the RAID storage system, only the video file exists in the data segment, the video file is also stored in the data segment completely, and the length of the data segment is greater than 2G, so that the video file can be accommodated entirely.

By the step, after one video file corresponds to one fragment data segment, the video data of the single video file can be described through the address of the fragment data segment because the fragment data segment is continuous in the lower layer storage. And subsequently, when the file index is incomplete and cannot be accessed, the user can access the file through the address of the fragment data segment.

In order to achieve the above technical objective, a preferred embodiment of the present application adopts the following steps before this step to ensure that the fragmented data segments correspond to the video files one-to-one and the length of each fragmented data segment is greater than the file size of each video file:

step a) setting the length of the fragment data segment to be larger than the file size of the video file;

step b) selecting the fragmented data segments from the RAID storage system that do not store any video files.

In the existing file storage format, the size of a single file can reach 2GB or more, and the single file can simultaneously correspond to data segments of multiple slices. Since the technical scheme of the present invention requires that one file corresponds to one fragment data segment, the preferred embodiment ensures that the size of a single video file remains unchanged after being stored in the fragment data segment by increasing the length of the unit fragment data segment, thereby indirectly ensuring that the index does not increase.

It should be noted that, although the steps in the above preferred embodiment have described in a specific way how to set the lengths of the fragmented data segments and make a single fragmented data segment only store one video file, the present invention is not limited to this, and implementations that achieve the same purpose in other ways at other times before writing the video file all belong to the scope of the present application.

S102, when the file index of the video file is abnormal, the video data of the video file is obtained in the fragment data segment according to the address.

Because the technical scheme of the invention additionally maintains the address of the fragment data segment on the basis of saving the file index in the prior art, in the actual application process, a user can select to access the stored video file through the file index under the normal condition, and can read more description information (such as creation time and the like) on the basis, thereby realizing accurate retrieval. However, when the file index is abnormal (for example, the file index cannot be read due to the damage of the disk), the user can find the video data of the video file through the address of the saved fragment data segment, and continue to read the video data based on the structure of the video file.

According to the scheme, the length of the fragment data segment in the RAID storage system is increased, and the video file and the fragment data segment address are corresponding, so that the purpose that the video file can be read through the fragment data segment address is simply achieved. On this basis, in order to further enable the retrieval and playing of the video file under the condition that the video storage format index is damaged, the preferred embodiment of the present application sets a continuous space on the video file of the fragmented data segment, where a plurality of same video index blocks corresponding to the video file are stored. Based on the characteristics of a RAID storage system, these video index blocks have the following characteristics:

(1) the video index block includes verification information and playing information of the video file.

At present, video data formats are various, such as streams in PS and TS formats. To prepare for characterizing a video file, the preferred embodiment reserves a contiguous space of appropriate size (e.g., on the order of 1 MB) on the video file (e.g., header) for storing a plurality of video index blocks. The information described by the video index block comprises verification information of the video index information block (used for confirming whether the video index block is intact) and playing information of the video file, and subsequently, under the condition that the video index block is confirmed to be intact through the verification information, the data playing of the video file can be realized through the playing information.

In one embodiment of the present application, the format of the video index block is as shown in fig. 3, and the information described in the video index block includes the verification information of the video index information block, the start time, the end time, the interval time of the I frames, the relative timestamp of each I frame, and the relative storage location of the I frame. The detailed description is as follows:

the first part is the check information of the index block, the second part is the start time and the end time of the video, the third part is the interval of the I frames, the unit second is 1, the interval is 1 second, the fourth part is the relative time stamps of all the I frames in the video file, the unit second is the address information of all the I frames in the video file, and the fifth part is the relative time stamps of the I frames in the fourth part, which are in one-to-one correspondence and are arranged in sequence.

For example, taking a 2GB video file as an example, the bitrate is 6MB/s, which is about half an hour, if the inter-frame interval is 1 second, there are 1800I frames, an I frame timestamp plus an I frame offset is a total of 8 bytes, plus video index block check information (4 bytes), the total size of the video start time and end time (64 bytes), and the I frame interval description (4 bytes) is 1800 × 8+4+64+4 ≈ 14k, which are stored in a stripe block of an array, and if the stripe block size is 64KB, the extra space can be filled with 0.

It should be noted that the contents contained in the above video index block and the storage location thereof are all specifically described in the present application, and other modifications made by those skilled in the art should also fall within the scope of the present application.

(2) The size of the video index block is less than or equal to a stripe block of the RAID storage system.

(3) The number of the video index blocks is larger than the number of the disks forming the RAID storage system.

Based on the composition of the disk array and the data storage characteristics, in the reserved continuous space, the technical scheme of the invention stores the copies of a plurality of video index blocks (namely a plurality of video index blocks), the size of each video index block is smaller than or equal to the size of the array strip block, and the number of the copies of the video index blocks is required to be larger than the number of disks used for making the array, so that even if a certain video index block cannot be read due to the failure of the disks, the related information of the video file can be read by using other video index blocks, and the playing of the video data is further realized.

Taking the specific application scenario shown in fig. 4 as an example, the specific embodiment adopts an array scheme of raid5 and 4+1, the size of the array stripe block is 64KB, and the number of copies of the video index block in the continuous space of the stored video file must be greater than 5. Because the raid5 array is organized in stripes, it is guaranteed that at least one of the video index blocks is good for a contiguous stripe when multiple disks are corrupted.

Based on the above scheme, the preferred embodiment of the present application implements playing of video data by the following steps:

step a) determining the position of a key frame to be read according to the starting time, the ending time and the relative timestamp of the key frame;

step b) playing the video data from the position according to the key frame address information, and skipping incomplete key frame data according to the key frame interval when the incomplete key frame data is encountered.

When the video search is performed, the scheme of the preferred embodiment reads the information of the plurality of index blocks of the video file, and finds the correct index block according to the verification information. Then according to the video start time, the video end time and the relative timestamp of the I frame in the index block, binary search is carried out to find the position of the I frame needing to be read, then video data is played from the current position of the I frame, if the I frame data is incomplete in the playing process, the current I frame is skipped, and the next I frame data is continuously read, so that when the array is damaged, the index data of the I frame is completely readable.

By applying the technical scheme of the application, the video file is written into the blank fragment data segment, and the address of the fragment data segment is acquired and stored after the writing is finished. Therefore, the problem that the video file cannot be read due to the fact that the file index in the RAID storage system is damaged can be solved on the premise that the storage performance is not reduced, and the stability of the RAID storage system is improved.

In order to achieve the above technical object, the present application further provides a video file storage apparatus applied in a RAID storage system, as shown in fig. 5, including:

a write module 510, configured to write the video file into a blank fragmented data segment, and obtain and store an address of the fragmented data segment after the write is completed, where the fragmented data segment corresponds to the video file one to one, and the length of the fragmented data segment is greater than the file size of the video file;

the obtaining module 520 obtains the video data of the video file in the fragment data segment according to the address when the file index of the video file is abnormal.

In a specific application scenario, the method further includes:

the setting module is used for setting the length of the fragment data segment to be larger than the file size of the video file;

and the selection module selects the fragment data segment which does not store any video file from the RAID storage system.

In a specific application scenario, the setting module is further configured to:

setting a continuous space on the video file, wherein a plurality of same video index blocks corresponding to the video file are stored in the continuous space;

the video index blocks comprise verification information and playing information of the video files, the size of the video index blocks is smaller than or equal to that of the stripe blocks of the RAID storage system, and the number of the video index blocks is larger than that of the disks forming the RAID storage system.

In a specific application scenario, the method further includes:

the screening module screens out the video index blocks with correct verification information from the continuous space;

and the playing module plays the video data according to the playing information in the video index block with correct verification information.

In a specific application scenario, the playing information includes a start time, an end time, a key frame interval, a key frame relative timestamp, and key frame address information of the video file, and the playing module is specifically configured to:

determining the position of a key frame needing to be read according to the starting time, the ending time and the relative timestamp of the key frame;

playing the video data from the location according to the keyframe address information, and skipping over incomplete keyframe data according to the keyframe interval when the incomplete keyframe data is encountered.

Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by hardware, or by software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method according to the implementation scenarios of the present invention.

Those skilled in the art will appreciate that the figures are merely schematic representations of one preferred implementation scenario and that the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

Those skilled in the art will appreciate that the modules in the devices in the implementation scenario may be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the present implementation scenario with corresponding changes. The modules of the implementation scenario may be combined into one module, or may be further split into a plurality of sub-modules.

The above-mentioned invention numbers are merely for description and do not represent the merits of the implementation scenarios.

The above disclosure is only a few specific implementation scenarios of the present invention, however, the present invention is not limited thereto, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present invention.

Claims

1. A video file storage method is applied to a disk array RAID storage system and is characterized by comprising the following steps:

writing the video file into a blank fragment data segment, and acquiring and storing the address of the fragment data segment after the writing is finished, wherein the length of the fragment data segment is more than or equal to the file size of the video file, and one video file corresponds to one fragment data segment;

2. The method of claim 1, wherein prior to writing the video file to the sliced data segment, further comprising:

setting the length of the fragment data segment to be larger than or equal to the file size of the video file;

selecting the sliced data segment from the RAID storage system that does not store any video files.

3. The method of any of claims 1 or 2, further comprising:

4. The method of claim 3, wherein after obtaining the video data of the video file in the fragmented data segment according to the address, further comprising:

screening out a video index block with correct verification information from the continuous space;

and playing the video data according to the playing information in the video index block with correct verification information.

5. The method according to claim 4, wherein the playback information includes a start time, an end time, a key frame interval, a key frame relative timestamp, and key frame address information of the video file, and the playback of the video data according to the playback information in the video index block with the correct verification information includes:

6. A video file storage device is applied to a disk array RAID storage system, and is characterized by comprising:

the writing module is used for writing the video files into blank fragment data segments and acquiring and storing addresses of the fragment data segments after the writing is finished, wherein the fragment data segments correspond to the video files one by one, the length of each fragment data segment is greater than the file size of each video file, and one video file corresponds to one fragment data segment;

7. The apparatus of claim 6, further comprising:

8. The apparatus of claim 7, wherein the setup module is further to:

9. The apparatus of claim 8, further comprising:

10. The apparatus of claim 9, wherein the playback information comprises a start time, an end time, a key frame interval, a key frame relative timestamp, and key frame address information for the video recording file, and wherein the playback module is specifically configured to: