CN115955576A - Video data access method, device, storage node and storage medium - Google Patents
Video data access method, device, storage node and storage medium Download PDFInfo
- Publication number
- CN115955576A CN115955576A CN202310224521.4A CN202310224521A CN115955576A CN 115955576 A CN115955576 A CN 115955576A CN 202310224521 A CN202310224521 A CN 202310224521A CN 115955576 A CN115955576 A CN 115955576A
- Authority
- CN
- China
- Prior art keywords
- data
- video data
- frame
- read
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention relates to the technical field of distributed storage, and provides a video data access method, a device, a storage node and a storage medium, which are applied to the storage node in a distributed storage system, wherein the storage node comprises a plurality of hard disks, and the method comprises the following steps: acquiring video data; and if the video data comprise the whole frame of I frame data and the length of the video data is a preset length, storing the video data to the hard disk, wherein the whole frame of I frame data is not stored across multiple hard disks. The invention can improve the safety of the video data in the distributed storage system.
Description
Technical Field
The present invention relates to the field of distributed storage technologies, and in particular, to a video data access method, apparatus, storage node, and storage medium.
Background
Video files are mainly multimedia files containing real-time audio and video information, and the multimedia information usually comes from a video input device. The data size of video data in a video file is usually large, and therefore, the video file is usually stored in a distributed storage system.
How to improve the security of video data in a distributed storage system is a problem to be solved urgently by those skilled in the art.
Disclosure of Invention
The object of the present invention includes, for example, providing a video data access method, apparatus, storage node and storage medium, which can improve the security of video data in a distributed storage system.
Embodiments of the invention may be implemented as follows:
in a first aspect, the present invention provides a video data access method, which is applied to a storage node in a distributed storage system, where the storage node includes a plurality of hard disks, and the method includes:
acquiring video data;
and if the video data comprise whole frame I frame data and the length of the video data is a preset length, storing the video data to the hard disk, wherein the whole frame I frame data is not stored across multiple hard disks.
In an alternative embodiment, the method further comprises:
if the video data comprises partial I frame data, the partial I frame data comprises I frame header data, and the length of the video data is equal to the preset length, the video data is divided into partial I frame data and non-I frame video data;
supplementing the non-I frame video data and storing the supplemented non-I frame video data to the hard disk, wherein the length of the supplemented non-I frame video data is the preset length;
continuously acquiring video data after the video data, and supplementing the partial I frame data by using the newly acquired video data, so that the supplemented partial I frame data comprises the whole frame I frame data, and the length of the supplemented partial I frame data is the preset length;
and storing the supplemented partial I frame data to the hard disk.
In an alternative embodiment, the method further comprises:
acquiring a coding format of the video data;
determining whether I frame data is included in the video data according to the encoding format;
and if the video data comprises I frame data, judging whether the I frame data is whole frame I frame data or partial I frame data according to the head data of the I frame data.
In an alternative embodiment, the step of storing the video data to the hard disk includes:
performing hash calculation on the video data to obtain a hash value of the video data;
and storing the hash value and the video data to the hard disk.
In an optional implementation manner, the distributed storage system further includes a plurality of associated nodes having an association relationship with the storage node, each of the associated nodes and the storage node includes a plurality of data blocks, each of the data blocks in the storage node and corresponding data blocks in all of the associated nodes belong to the same erasure correction group, and the method further includes:
determining a data block to be read in a plurality of data blocks in the storage node;
if the data block to be read fails to be read, determining a substitute data block from data blocks which belong to the same erasure correcting group with the data block to be read in the associated node;
and taking the replacing data block as the data block to be read.
In an alternative embodiment, the method further comprises:
if the hard disk to which the data block to be read belongs is damaged and the data block to be read cannot be recovered through the data block in the erasure correction group to which the data block to be read belongs, determining that the data block to be read fails to be read;
and if the hash check of the data block to be read fails and the data block to be read cannot be recovered through the data block in the erasure correction group to which the data block to be read belongs, determining that the data block to be read fails to be read.
In an optional embodiment, the data block to be read includes a hash value and data to be read, and the method further includes:
performing hash operation on the data to be read to obtain a hash value of the data to be read;
and if the hash value of the data to be read is inconsistent with the hash value in the data block to be read, judging that the hash check of the data block to be read fails.
In a second aspect, the present invention provides an apparatus for accessing video data, which is applied to a storage node in a distributed storage system, where the storage node includes a plurality of hard disks, and the apparatus includes:
the acquisition module is used for acquiring video data;
the storage module is configured to store the video data to the hard disk if the video data includes a whole frame of I frame data and the length of the video data is a preset length, where the whole frame of I frame data is not stored across multiple hard disks.
In a third aspect, the present invention provides a storage node, including a processor and a memory, where the memory is used to store a program, and the processor is used to implement the video data access method described in any one of the foregoing embodiments when executing the program.
In a fourth aspect, the invention provides a computer-readable storage medium, on which a computer program is stored, which computer program, when executed by a processor, implements a video data access method as described in any one of the preceding embodiments.
Compared with the prior art, when the video data is stored, if the video data comprises the whole frame of I frame data and the length of the video data is the preset length, the video data is stored in the hard disk, and the whole frame of I frame data cannot be stored across the hard disk.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is an exemplary diagram of a distributed storage system according to an embodiment of the present invention.
Fig. 2 is a schematic block diagram of a storage node according to an embodiment of the present invention.
Fig. 3 is a first flowchart illustrating a data access method according to an embodiment of the present invention.
Fig. 4 is a diagram illustrating an exemplary video data processing method according to an embodiment of the present invention.
Fig. 5 is a diagram illustrating an example of video data storage according to an embodiment of the present invention.
Fig. 6 is a second flowchart illustrating a data access method according to an embodiment of the present invention.
Fig. 7 is a block diagram of a data access device according to an embodiment of the present invention.
Icon: 10-a storage node; 11-a processor; 12-a memory; 13-a bus; 14-a communication interface; 20-a client; 100-a data access device; 110-an acquisition module; 120-a storage module; 130-reading module.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. indicate an orientation or a positional relationship based on that shown in the drawings or that the product of the present invention is used as it is, this is only for convenience of description and simplification of the description, and it does not indicate or imply that the device or the element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus should not be construed as limiting the present invention.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.
The distributed storage system is characterized in that a large number of storage nodes are interconnected through a network and provide storage service to the outside as a whole, and the distributed storage system dispersedly stores data on a plurality of independent storage nodes. The system adopts an expandable system structure, and utilizes a plurality of storage servers to share storage load and position storage information, thereby not only improving the reliability, the availability and the access efficiency of the system, but also being easy to expand. Referring to fig. 1, fig. 1 is an exemplary diagram of a distributed storage system according to an embodiment of the present invention, in fig. 1, the distributed storage system includes a plurality of storage nodes 10, each storage node 10 includes a plurality of hard disks, a client 20 is in communication connection with the distributed storage system, the client 20 is a user of a storage service, the distributed storage system is a provider of the storage service, the client 20 sends video data to be stored to the distributed storage system, and the video data is stored in the hard disks of the plurality of storage nodes 10 in the distributed storage system in a distributed manner. In order to improve the reliability of video data, usually, erasure correction coding is performed on the video data, and then the data after erasure correction coding is stored in different storage nodes, for example, an erasure correction strategy is 12. When the damaged block of the 16 blocks does not exceed 4, the damaged block can be recovered by an erasure correction strategy through the undamaged block, thereby ensuring the reliability of the video data.
The storage node 10 may be, but is not limited to, a storage server, or a physical or virtual computer, a storage array, etc. capable of implementing data storage functions.
The client 20 may be, but is not limited to, a server, a terminal, a smart phone, a tablet computer, a notebook computer, a wearable device, and other electronic devices.
Referring to fig. 2, fig. 2 is a block schematic diagram of a storage node 10 according to an embodiment of the present invention, where the storage node 10 may be any storage node in fig. 1. The storage node 10 comprises a processor 11, a memory 12, a bus 13, a communication interface 14. The processor 11 and the memory 12 are connected by a bus 13.
The processor 11 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by instructions in the form of hardware integrated logic circuits or software in the processor 11. The Processor 11 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
The memory 12 is used for storing a program, for example, the data access apparatus 100 in the embodiment of the present invention, the data access apparatus 100 includes at least one software functional module which can be stored in the memory 12 in a form of software or firmware (firmware), and the processor 11 executes the program after receiving an execution instruction to implement the data access method in the embodiment of the present invention.
The Memory 12 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory). Alternatively, the memory 12 may be a storage device built in the processor 11, or may be a storage device independent of the processor 11.
The bus 13 may be an ISA bus, a PCI bus, an EISA bus, or the like. Fig. 2 is represented by only one double-headed arrow, but does not represent only one bus or one type of bus.
The storage node 10 communicates with other storage nodes 10 or clients 20 through a communication interface 14.
Since video files are not like other files, once the data in a file is corrupted, the entire file is completely unusable. There is also a possibility of normal play of video data in a video file that is not corrupted or lost. In addition, an I frame in the video data is a full-frame compressed encoded frame, and a complete image can be reconstructed only by using data of the I frame during decoding, so that whether the I frame is normal or not is crucial to normal playing of the video data. Which will be described in detail below.
On the basis of fig. 1 and fig. 2, the present embodiment provides a video data access method, which may be applied to any storage node 10 in fig. 1 or the storage node 10 in fig. 2, please refer to fig. 3, where fig. 3 is a flowchart illustrating a data access method according to an embodiment of the present invention, and the method includes the following steps:
step S101, video data is acquired.
In this embodiment, the video data may be from a video file or from a video capture device, and the manner of acquiring the video data may be to read the video data from the video file in a streaming manner, or to receive a video stream from the video capture device.
Step S102, if the video data comprises a whole frame of I frame data and the length of the video data is a preset length, the video data is stored in a hard disk, wherein the whole frame of I frame data is not stored across multiple hard disks.
In this embodiment, the whole frame of I frame data includes at least one complete frame of I frame data. The preset length is predetermined and is determined according to the number of bytes of data written into the single block of hard disk at a time and the number of bytes occupied by the actually written data length, taking the number of bytes of data written into the single block of hard disk at a time as 256KB for example, and the actually written data length occupies 18B, then the preset length is 256KB-18B. For a video file, the video file may be divided into a plurality of data blocks according to a preset length, and the video file may be stored or read according to the data blocks.
In this embodiment, according to the difference between the preset length and the length of the entire frame of I frame data, the video data with the preset length may include only one entire frame of I frame data, may also include one entire frame of I frame data and a part of non-I frame data, and may also include multiple entire frames of I frame data, or multiple entire frames of I frame data and a part of non-I frame data.
It should be noted that, in order to simplify the processing, the number of bytes of data stored to the single block of hard disk at a time may be set to 4MB, so as to ensure that the number of bytes of data stored to the single block of hard disk does not exceed one per single I frame. In addition, the actual length of the video data can be stored in the storage space of 8B according to actual needs. For example, the number of bytes of data stored at a time to a single block of hard disk is set to 4MB, where 8B is used to store the actual length of video data, and where the maximum of 4MB-8B is available for storing valid video data.
It can be understood that a complete I frame data is written on only one hard disk, avoiding the cross-disk storage of a single complete I frame data.
According to the method provided by the embodiment, the risk that the I frame data is damaged when the hard disk fails is reduced by avoiding the storage of the I frame data in a cross-disk mode.
In an alternative embodiment, the video data may include only non-I frame data and partial I frame data, and in order not to store the entire frame of I frame data across the disk, this embodiment further provides a processing manner:
firstly, if the video data comprises partial I frame data, the partial I frame data comprises I frame header data, and the length of the video data is equal to a preset length, the video data is divided into partial I frame data and non-I frame video data;
in this embodiment, after the position of the partial I frame data in the complete video is located in the non-I frame video data, in order to store the non-I frame video data to the hard disk according to the preset length, and store the whole frame I frame data to the hard disk in a manner of not spanning multiple hard disks, so as to avoid the storage of the whole frame I frame data across the disk, the non-I frame video data and the partial I frame data are processed separately in this embodiment.
For non-I-frame video data, the processing method is as follows: supplementing the non-I frame video data and storing the supplemented non-I frame video data to a hard disk, wherein the length of the supplemented non-I frame video data is a preset length; when supplementing, the length of the supplemented non-I frame video data can be 0 or other preset values until the length of the supplemented non-I frame video data is the preset length.
For partial I frame data, the processing method is as follows: continuously acquiring video data after the video data, and supplementing partial I frame data by using the newly acquired video data, so that the supplemented partial I frame data comprises the whole frame I frame data, and the length of the supplemented partial I frame data is a preset length; and storing the supplemented partial I frame data to a hard disk.
In addition, when the video data comprises the whole frame of I frame data and the length of the video data is smaller than the preset length, the video data after the video data is continuously acquired until the sum of the lengths of the video data and the newly acquired video data is the preset length, and the video data with the preset length is stored in the hard disk.
To more clearly illustrate the processing procedure of the video data for different situations, please refer to fig. 4, fig. 4 is an exemplary diagram of video data processing according to an embodiment of the present invention, in fig. 4, a first video data includes a whole frame of I frame data and a part of non-I frame data, the first video data is written into a hard disk a, a second video data includes non-I frame data, the second video data is written into a hard disk B, a third video data includes non-I frame data and a part of I frame data, the first video data is divided into non-I frame data and a part of I frame data, the non-I frame data and the part of I frame data are obtained after being supplemented with 0 for the non-I frame data, the non-I frame data with a preset length is written into the hard disk C, and for the part of I frame data, the video data after the part of I frame data is continuously obtained until the preset length is reached, at this time, the supplemented part of I frame data includes the whole frame data and a part of non-I frame data, and the part of I frame data is written into the hard disk D. Thus, hard disk A and hard disk D each comprise a full frame of I frame data, hard disk C comprises non-I frame data, and hard disk D comprises a portion of non-I frame data and supplemental data.
In this embodiment, since there are multiple encoding manners of video data, I-frame representations of different encoded video data are different, in order to determine whether to include I-frame data in the video data, whether the included I-frame data is whole-frame I-frame data or partial I-frame data, this embodiment further provides an implementation manner:
first, an encoding format of video data is acquired.
In this embodiment, the encoding format may be, but is not limited to, H.264, H.263, MPEG-1, MPEG-2, MPEG, etc.
Second, it is determined whether I frame data is included in the video data according to the encoding format.
In this embodiment, the frame structure of different encoding formats is different, and the frame structure usually includes fields for indicating whether the frame is an I frame, a B frame, or a P frame, for example, in MPEG-4, the beginning of each frame is fixed: 0000 b 01 b6, after which a value of 2bit characterizes which type of frame the frame is: 00: i frame; 01: a P frame; 10: b frames, whereby it can be determined whether I frame data is included in the video data.
And finally, if the video data comprises I frame data, judging whether the I frame data is the whole I frame data or partial I frame data according to the head data of the I frame data.
In this embodiment, the length of the I frame data may be stored in a preset field of the frame header, and it may be determined whether the I frame data included in the video data is the whole I frame data or the partial I frame data according to the frame start flag and the length of the I frame data, and besides, it may also be determined whether the I frame data is the whole I frame data or the partial I frame data according to the header data of the I frame data and a preset separator, for example, in the encoding format of H264, the separator is 0x0000 0001, and if the header data and the separator of the I frame data are included in the video data, it is determined that the I frame data is the whole I frame data, otherwise, it is determined that the I frame data is the partial I frame data.
In this embodiment, in order to further improve the reliability of the video data, in order to determine whether the read video data is damaged again when the video data is read, in addition to storing the video data, the hash value of the video data may also be stored at the same time, and the processing method may be: performing hash calculation on the video data to obtain a hash value of the video data; and storing the hash value and the video data to a hard disk. Referring to fig. 5, fig. 5 is a diagram illustrating an example of video data storage according to an embodiment of the present invention, in fig. 5, the first 18B of 256KB,256KB of data stored in a single block of hard disk at a time is an actual length field, the value of the field indicates the length of actual video data in 256KB, if the actual video data is 200KB, the actual length field is 200KB, and the subsequent 256KB-18B video data includes actual video data and supplementary data, such as a shaded portion in the diagram, the sum of the lengths of the actual video data and the supplementary data is 256KB, and a hash field is provided before the actual length field, the hash field being used for storing a hash value calculated by integrating the actual video data and the supplementary data.
In this embodiment, the distributed storage system further includes a plurality of associated nodes having an association relationship with the storage nodes, each associated node and the storage node includes a plurality of data blocks, each data block in the storage node and the corresponding data block in all associated nodes belong to the same erasure correction group, that is, video data in the data block of the same erasure correction group is written into the corresponding storage node after erasure correction coding, and for a video file, if the damage degree of data in the same erasure correction group in the data in the video file exceeds an erasure correction tolerance range, for example, the erasure correction ratio EC is 12:4, namely, the data blocks in the same erasure group are stored in 12 hard disks, and 4 hard disks are used for storing the verification data for recovering the data. If 5 hard disks are damaged at the same time, the video data of the erasure group cannot be recovered. For such a scene that cannot recover video data, there are generally two processing methods: 1) Hiding the video file from the user; 2) The video file is deleted from the background. These two processing methods are actually too simple and rough, and in fact, there is no damaged or lost data block in the video file and there is a possibility of normal playing, and this embodiment further provides a video data reading method for reading video data as much as possible and ensuring normal playing of the video file to the maximum extent based on the above storage method for video data, please refer to fig. 6, where fig. 6 is a second flow chart of a data access method provided by an embodiment of the present invention, and the method includes the following steps:
step S201, determining a data block to be read in a plurality of data blocks in a storage node.
In this embodiment, the data block to be read includes video data in a video file to be read, if the video data is stored in the hard disk of the storage node according to the data access method described above in this embodiment, the risk of I frame data damage is smaller, and normal playing of the video file can be ensured to the greatest extent, if the video data is stored in the hard disk of the storage node according to the prior art, by using the access method for reading the data block to be read provided in this embodiment, undamaged data can be used as much as possible, and normal playing of the video file can be ensured as much as possible, the video data of the video file to be read can be included in a plurality of data blocks, and the method of steps S201 to S203 is used for any data block to read the video data therein.
Step S202, if the data block to be read fails to be read, a substitute data block is determined from the data blocks which belong to the same erasure group with the data block to be read in the associated node.
In this embodiment, a data block may be randomly selected from data blocks in the associated node that belong to the same erasure correction group as the data block to be read as a replacement data block, and in order to obtain a better video playing effect, a data block in a video file whose position is adjacent to the data block to be read may be used as the replacement data block, for example, the data block in the same erasure correction group includes a block a, a block b, a block c, and a block d, and the positions in the video file are: a. b, c and d, if the data block to be read is the block c, the block b is used as the substitute data block.
In step S203, the substitute data block is used as a data block to be read.
In this embodiment, the video data in the substitute data block is used as the video data in the data block to be read, so as to provide the video file including the damaged video data to the maximum extent.
In this embodiment, the read failure of the read data block at least includes the following two cases:
(1) If the hard disk to which the data block to be read belongs is damaged and the data block to be read cannot be recovered through the data block in the erasure group to which the data block to be read belongs, determining that the data block to be read fails to be read;
(2) And if the hash check of the data block to be read fails and the data block to be read cannot be recovered through the data block in the erasure group to which the data block to be read belongs, determining that the data block to be read fails to be read.
In an optional embodiment, the data block to be read includes a hash value and data to be read, and one way of determining whether the hash check fails is as follows:
performing hash operation on data to be read to obtain a hash value of the data to be read;
and if the hash value of the data to be read is inconsistent with the hash value in the data block to be read, judging that the hash check of the data block to be read fails.
It should be noted that, in order to make the user sufficiently aware of the damaged video data in the scene of requesting the video file in different ways, the start positions of all damaged and lost data blocks and the data offset may be recorded. In a scene of requesting a video file in an http mode, damaged and lost data block information can be returned to a user through a header field header of a response message response; in a scene where a video file is requested in an sdk programming manner, damaged and lost data block information can be returned through sdk; in the scene of acquiring the video file in the command line mode, the damaged and lost data block information can be saved in the form of a text file to a preset path of an operating system through the command line.
In order to perform the corresponding steps in the above-described embodiments and various possible implementations, an implementation of the data access apparatus 100 is given below. Referring to fig. 7, fig. 7 is a block diagram illustrating a data access apparatus according to an embodiment of the present invention. It should be noted that the basic principle and the resulting technical effect of the data access apparatus 100 provided in this embodiment are the same as those of the foregoing embodiments, and for the sake of brief description, no reference is made to this embodiment.
The video data access apparatus 100 is applied to a storage node in a distributed storage system, and the video data access apparatus 100 includes an obtaining module 110, a storage module 120, and a reading module 130.
The obtaining module 110 is configured to obtain video data.
The storage module 120 is configured to store the video data to the hard disk if the video data includes a whole frame of I frame data and the length of the video data is a preset length, where the whole frame of I frame data is not stored across multiple hard disks.
In an alternative embodiment, the storage module 120 is further configured to: if the video data comprises partial I frame data, the partial I frame data comprises I frame header data, and the length of the video data is equal to a preset length, the video data is divided into the partial I frame data and non-I frame video data; supplementing the non-I frame video data and storing the supplemented non-I frame video data to a hard disk, wherein the length of the supplemented non-I frame video data is a preset length; continuously acquiring video data after the video data, and supplementing partial I frame data by using the newly acquired video data, so that the supplemented partial I frame data comprises the whole frame I frame data, and the length of the supplemented partial I frame data is a preset length; and storing the supplemented partial I frame data to a hard disk.
In an alternative embodiment, the storage module 120 is further configured to: acquiring a coding format of video data; determining whether I frame data is included in the video data according to the encoding format; and if the video data comprises I frame data, judging whether the I frame data is the whole I frame data or partial I frame data according to the head data of the I frame data.
In an alternative embodiment, the storage module 120 is specifically configured to, when used to store the video data to the hard disk: performing hash calculation on the video data to obtain a hash value of the video data; and storing the hash value and the video data to a hard disk.
In an optional embodiment, the distributed storage system further includes a plurality of associated nodes having an association relationship with the storage node, each associated node and the storage node includes a plurality of data blocks, each data block in the storage node and corresponding data blocks in all associated nodes belong to the same erasure correction group, and the reading module 130 is configured to: determining a data block to be read in a plurality of data blocks in a storage node; if the data block to be read fails to be read, determining a substitute data block from data blocks which belong to the same erasure group with the data block to be read in the associated node; the replacement data block is taken as the data block to be read.
In an alternative embodiment, the reading module 130 is further configured to: if the hard disk to which the data block to be read belongs is damaged and the data block to be read cannot be recovered through the data block in the erasure group to which the data block to be read belongs, determining that the data block to be read fails to be read; and if the hash check of the data block to be read fails and the data block to be read cannot be recovered through the data block in the erasure group to which the data block to be read belongs, determining that the data block to be read fails to be read.
In an alternative embodiment, the data block to be read includes the hash value and the data to be read, and the reading module 130 is further configured to: performing hash operation on data to be read to obtain a hash value of the data to be read; and if the hash value of the data to be read is inconsistent with the hash value in the data block to be read, judging that the hash check of the data block to be read fails.
The present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a video data access method as in the preceding embodiments.
In summary, embodiments of the present invention provide a video data access method, an apparatus, a storage node, and a storage medium, which are applied to a storage node in a distributed storage system, where the storage node includes a plurality of hard disks, and the method includes: acquiring video data; and if the video data comprises the whole frame of I frame data and the length of the video data is a preset length, storing the video data to a hard disk. Compared with the prior art, the method and the device have the advantages that the risk that the I frame data are damaged when the hard disk fails is reduced by avoiding the storage of the I frame data in a cross-disk mode, and because the I frame is a full-frame compressed coding frame, a complete image can be reconstructed only by using the data of the I frame during decoding, the risk that the video data are damaged is reduced, and the safety of the video data is improved.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A video data access method is applied to a storage node in a distributed storage system, wherein the storage node comprises a plurality of hard disks, and the method comprises the following steps:
acquiring video data;
and if the video data comprise whole frame I frame data and the length of the video data is a preset length, storing the video data to the hard disk, wherein the whole frame I frame data is not stored across multiple hard disks.
2. The video data access method of claim 1, wherein the method further comprises:
if the video data comprises partial I frame data, the partial I frame data comprises I frame header data, and the length of the video data is equal to the preset length, the video data is divided into partial I frame data and non-I frame video data;
supplementing the non-I frame video data and storing the supplemented non-I frame video data to the hard disk, wherein the length of the supplemented non-I frame video data is the preset length;
continuously acquiring video data after the video data, and supplementing the partial I frame data by using the newly acquired video data, so that the supplemented partial I frame data comprises the whole frame I frame data, and the length of the supplemented partial I frame data is the preset length;
and storing the supplemented partial I frame data to the hard disk.
3. The video data access method of claim 1, wherein the method further comprises:
acquiring a coding format of the video data;
determining whether I frame data is included in the video data according to the encoding format;
and if the video data comprises I frame data, judging whether the I frame data is whole frame I frame data or partial I frame data according to the head data of the I frame data.
4. The video data access method of claim 1, wherein the step of storing the video data to a hard disk comprises:
performing hash calculation on the video data to obtain a hash value of the video data;
and storing the hash value and the video data to a hard disk.
5. The video data access method according to claim 1, wherein the distributed storage system further includes a plurality of association nodes associated with the storage nodes, each of the association nodes and the storage nodes includes a plurality of data blocks, each of the data blocks in the storage nodes and corresponding data blocks in all of the association nodes belong to a same erasure group, and the method further includes:
determining a data block to be read in a plurality of data blocks in the storage node;
if the reading of the data block to be read fails, determining a substitute data block from data blocks which belong to the same erasure correction group with the data block to be read in the associated node;
and taking the replacing data block as the data block to be read.
6. The video data access method of claim 5, wherein the method further comprises:
if the hard disk to which the data block to be read belongs is damaged and the data block to be read cannot be recovered through the data block in the erasure correction group to which the data block to be read belongs, determining that the data block to be read fails to be read;
and if the hash check of the data block to be read fails and the data block to be read cannot be recovered through the data block in the erasure correction group to which the data block to be read belongs, determining that the data block to be read fails to be read.
7. The video data access method of claim 6, wherein the block of data to be read comprises a hash value and data to be read, the method further comprising:
performing hash operation on the data to be read to obtain a hash value of the data to be read;
and if the hash value of the data to be read is inconsistent with the hash value in the data block to be read, judging that the hash check of the data block to be read fails.
8. A video data access apparatus, applied to a storage node in a distributed storage system, the storage node including a plurality of hard disks, the apparatus comprising:
the acquisition module is used for acquiring video data;
the storage module is used for storing the video data into the hard disk if the video data comprises a whole frame of I frame data and the length of the video data is a preset length, wherein the whole frame of I frame data is not stored across multiple hard disks.
9. A storage node comprising a processor and a memory, the memory being arranged to store a program, the processor being arranged, when executing the program, to carry out the video data access method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a video data access method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310224521.4A CN115955576B (en) | 2023-03-10 | 2023-03-10 | Video data access method, device, storage node and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310224521.4A CN115955576B (en) | 2023-03-10 | 2023-03-10 | Video data access method, device, storage node and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115955576A true CN115955576A (en) | 2023-04-11 |
| CN115955576B CN115955576B (en) | 2023-06-09 |
Family
ID=85906899
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310224521.4A Active CN115955576B (en) | 2023-03-10 | 2023-03-10 | Video data access method, device, storage node and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115955576B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005076630A1 (en) * | 2004-02-03 | 2005-08-18 | Matsushita Electric Industrial Co., Ltd. | Decoder, encoder, interpolation frame generating system, integrated circuit device, decoding program, and encoding program |
| US20070153906A1 (en) * | 2005-12-29 | 2007-07-05 | Petrescu Mihai G | Method and apparatus for compression of a video signal |
| US20090066838A1 (en) * | 2006-02-08 | 2009-03-12 | Nec Corporation | Representative image or representative image group display system, representative image or representative image group display method, and program therefor |
| US20110032429A1 (en) * | 2009-08-06 | 2011-02-10 | Cox Communications, Inc. | Video transmission using video quality metrics |
| CN105611307A (en) * | 2015-12-17 | 2016-05-25 | 浪潮(北京)电子信息产业有限公司 | Multipath h264 video data caching method and system based on no-lock buffer queue |
| JP2016129304A (en) * | 2015-01-09 | 2016-07-14 | 株式会社東芝 | Video transmission system |
| CN105828017A (en) * | 2015-10-20 | 2016-08-03 | 广东亿迅科技有限公司 | Cloud storage access system and method for video conferences |
| CN105847722A (en) * | 2015-01-16 | 2016-08-10 | 杭州海康威视数字技术股份有限公司 | Video storage method and device, video reading method and device and video access system |
| CN106713940A (en) * | 2015-08-04 | 2017-05-24 | 杭州海康威视数字技术股份有限公司 | Video stream storage and reading methods and video stream storage and reading devices |
| CN106791908A (en) * | 2016-11-25 | 2017-05-31 | 上海熙菱信息技术有限公司 | A kind of live video stream storage method for supporting cloud platform using double buffering |
| US10009575B1 (en) * | 2016-07-06 | 2018-06-26 | Viakoo, Inc. | Storage method for loss-tolerant data stream recording |
| CN112486918A (en) * | 2019-09-11 | 2021-03-12 | 浙江宇视科技有限公司 | File processing method, device, equipment and medium |
| CN115543190A (en) * | 2022-09-16 | 2022-12-30 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Intelligent distributed storage method and system for video data |
-
2023
- 2023-03-10 CN CN202310224521.4A patent/CN115955576B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005076630A1 (en) * | 2004-02-03 | 2005-08-18 | Matsushita Electric Industrial Co., Ltd. | Decoder, encoder, interpolation frame generating system, integrated circuit device, decoding program, and encoding program |
| US20070153906A1 (en) * | 2005-12-29 | 2007-07-05 | Petrescu Mihai G | Method and apparatus for compression of a video signal |
| US20090066838A1 (en) * | 2006-02-08 | 2009-03-12 | Nec Corporation | Representative image or representative image group display system, representative image or representative image group display method, and program therefor |
| US20110032429A1 (en) * | 2009-08-06 | 2011-02-10 | Cox Communications, Inc. | Video transmission using video quality metrics |
| JP2016129304A (en) * | 2015-01-09 | 2016-07-14 | 株式会社東芝 | Video transmission system |
| CN105847722A (en) * | 2015-01-16 | 2016-08-10 | 杭州海康威视数字技术股份有限公司 | Video storage method and device, video reading method and device and video access system |
| CN106713940A (en) * | 2015-08-04 | 2017-05-24 | 杭州海康威视数字技术股份有限公司 | Video stream storage and reading methods and video stream storage and reading devices |
| EP3334165A1 (en) * | 2015-08-04 | 2018-06-13 | Hangzhou Hikvision Digital Technology Co., Ltd. | Video stream storage method, reading method and device |
| CN105828017A (en) * | 2015-10-20 | 2016-08-03 | 广东亿迅科技有限公司 | Cloud storage access system and method for video conferences |
| CN105611307A (en) * | 2015-12-17 | 2016-05-25 | 浪潮(北京)电子信息产业有限公司 | Multipath h264 video data caching method and system based on no-lock buffer queue |
| US10009575B1 (en) * | 2016-07-06 | 2018-06-26 | Viakoo, Inc. | Storage method for loss-tolerant data stream recording |
| CN106791908A (en) * | 2016-11-25 | 2017-05-31 | 上海熙菱信息技术有限公司 | A kind of live video stream storage method for supporting cloud platform using double buffering |
| CN112486918A (en) * | 2019-09-11 | 2021-03-12 | 浙江宇视科技有限公司 | File processing method, device, equipment and medium |
| CN115543190A (en) * | 2022-09-16 | 2022-12-30 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Intelligent distributed storage method and system for video data |
Non-Patent Citations (2)
| Title |
|---|
| 蒋维等: "实时流媒体系统中喷泉码的度分布设计", 小型微型计算机系统 * |
| 陈晓辉,周山,戚文芽: "嵌入式数字视频监控系统的数据存储策略", 微计算机信息 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115955576B (en) | 2023-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106131550B (en) | Method and device for playing multimedia file | |
| TWI486913B (en) | Security monitoring device with network and record function and failure detecting and repairing mehtod for storage device thereof | |
| CN109412755B (en) | Multimedia data processing method, device and storage medium | |
| CN109151505B (en) | Video transcoding method, system, device and computer readable storage medium | |
| EP3809708B1 (en) | Video data storage method and device in cloud storage system | |
| EP3334165B1 (en) | Video stream storing and video stream reading method and apparatus therefor | |
| US20160203046A1 (en) | Memory device, server device, and memory control method | |
| JP2016187122A (en) | Video data processing apparatus, video data processing system, video data processing method, and video data processing program | |
| CN109597566B (en) | Data reading and storing method and device | |
| WO2021254375A1 (en) | Video partitioning method, transfer method, server, adaptor and storage medium | |
| US8681860B2 (en) | Moving picture compression apparatus and method of controlling operation of same | |
| CN115955576B (en) | Video data access method, device, storage node and storage medium | |
| CN117834979A (en) | MP4 video recording method and system capable of realizing power-off protection | |
| CN112672154A (en) | Live video playing method and device, server and computer readable storage medium | |
| CN109032536B (en) | Data storage method, device, system and equipment based on distributed cluster system | |
| US11599570B2 (en) | Device and method to render multimedia data stream tamper-proof based on block chain recording | |
| CN116094652A (en) | Audio and video data transmission method and device, electronic equipment and readable medium | |
| CN115589506A (en) | MP4 format audio/video file generation method, system and computer storage medium | |
| CN109982067B (en) | Video processing method and device | |
| CN108769732B (en) | Image transmission method and system and electronic terminal | |
| CN114143486A (en) | Video stream synchronization method and device, computer equipment and storage medium | |
| CN109426589B (en) | File repair method and device | |
| CN111970578A (en) | Method and device for repairing audio and video data transmission | |
| CN111356003B (en) | Data writing method, system and terminal equipment | |
| US11197028B2 (en) | Recovery during video encoding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |