CN108491488B - High-speed medium storing method - Google Patents
High-speed medium storing method Download PDFInfo
- Publication number
- CN108491488B CN108491488B CN201810209643.5A CN201810209643A CN108491488B CN 108491488 B CN108491488 B CN 108491488B CN 201810209643 A CN201810209643 A CN 201810209643A CN 108491488 B CN108491488 B CN 108491488B
- Authority
- CN
- China
- Prior art keywords
- stored
- video
- video stream
- storage
- url
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a high-speed media storage method, which comprises the following steps: receiving a video storage request and judging the volume of video stream data; if the volume of the video stream data is smaller than a first threshold value, submitting the video file to a cloud disk database of a monitoring system for storage; if the volume of video stream data is greater than a second threshold, the video file is stored at the cluster of file servers. The invention provides a high-speed media storage method, which can meet the storage requirement on massive small video files, ensure the processing efficiency on ultra-large video files and improve the storage performance of intelligent traffic data.
Description
Technical Field
The invention relates to video storage, in particular to a high-speed media storage method.
Background
The intelligent traffic monitoring network can generate a large number of video files with different formats and different volumes at every moment. These data often require long term storage. Therefore, how to manage these files effectively becomes an important issue. The prior art utilizes the cloud disk database of the monitoring system related to big data which is emerged in recent years for video file management. The cloud disk database of the monitoring system connects a plurality of physically dispersed data storage units through a computer network to form a logically uniform database, and files are stored in the data storage units in the form of video blocks. When the cloud disk database of the monitoring system is stored, small video file files are combined into larger video blocks to be stored. However, in the process of storing large video files, frequent segmentation and merging of video blocks are required, and the real-time requirement of the intelligent traffic monitoring system on the storage of video files with different volumes cannot be met simultaneously.
Disclosure of Invention
In order to solve the problems existing in the prior art, the invention provides a high-speed media storage method, which comprises the following steps:
receiving a video storage request and judging the volume of video stream data;
if the volume of the video stream data is smaller than a first threshold value, submitting the video file to a cloud disk database of a monitoring system for storage;
if the volume of video stream data is greater than a second threshold, the video file is stored at the cluster of file servers.
Preferably, after receiving the video storage request, the method further comprises:
the authority of the video storage request is verified, and whether the data are allowed to be stored in the corresponding storage area is judged according to the source of the video storage request;
and when the authority verification is passed, continuing the data storage process.
Preferably, the data storage flow is ended when the rights verification fails.
Preferably, when the volume of the surveillance video stream to be stored is greater than a second threshold, the storing the surveillance video stream to be stored to a file server cluster, further includes:
storing first configuration information to the cloud disk database of the monitoring system, wherein the first configuration information comprises a first URL (uniform resource locator), and the first URL is the URL of the monitoring video stream to be stored in the file server cluster;
when a super-large video file needs to be read, firstly, a URL of the super-large video file is obtained from a block data index table of a cloud disk database of a monitoring system, and then the video file is directly read from a file directory of a file server cluster according to the URL.
Preferably, the configuration information of the video file includes: file name, video format, file description, order of video blocks, and file URL.
Compared with the prior art, the invention has the following advantages:
the invention provides a high-speed media storage method, which can meet the storage requirement on massive small video files, ensure the processing efficiency on ultra-large video files and improve the storage performance of intelligent traffic data.
Drawings
FIG. 1 is a flow chart of a media high-speed storage method according to an embodiment of the invention.
Detailed Description
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details.
One aspect of the invention provides a method for high-speed storage of media. FIG. 1 is a flow chart of a media high-speed storage method according to an embodiment of the invention.
The present invention divides the video stream data file into three levels by volume. When the volume of the video stream data is smaller than a first threshold value, submitting the video stream data as a small video file to a cloud disk database of a monitoring system for storage; when the volume of the video stream data is larger than a second threshold value, the video stream data is used as a super large video file and stored in a file server cluster; when the volume of the video stream data is between the first threshold and the second threshold, the video stream data is divided according to a predefined strategy and stored in a cloud disk database of the monitoring system in a video block mode. The steps of the method are described in detail below:
receiving a video storage request, wherein the video storage request carries a monitoring video stream to be stored; in order to ensure the safety of data storage, the authority verification is carried out after the video storage request is received, and whether the data are allowed to be stored in the corresponding storage area is judged according to the source of the video storage request. When the authority passes the verification, continuing to perform the subsequent steps; and when the authority verification is not passed, ending the data storage process.
And after the authority verification is passed, adding the to-be-stored monitoring video stream carried by the video storage request to the cache, and processing the storage request of the to-be-stored monitoring video stream in the cache according to a first-in first-out sequence.
Acquiring the volume of the monitoring video stream to be stored; and when the volume of the monitoring video stream to be stored is smaller than a first threshold value, storing the monitoring video stream to be stored into an area for storing a block data index table in a cloud disk database of the monitoring system.
When the volume of the surveillance video stream to be stored is larger than a second threshold value, storing the surveillance video stream to be stored to a file server cluster, and storing first configuration information to a cloud disk database of the surveillance system, wherein the first configuration information comprises a first URL (uniform resource locator), and the first URL is a URL of the surveillance video stream to be stored in the file server cluster;
when a super-large video file needs to be read, firstly, a URL of the super-large video file is obtained from a block data index table of a cloud disk database of a monitoring system, and then the video file is directly read from a file directory of a file server cluster according to the URL. Therefore, in subsequent operation, the super-large video file does not need to be divided and combined.
When the volume of the monitored video stream to be stored is larger than the first threshold and the volume of the monitored video stream to be stored is smaller than the second threshold, according to a predefined strategy, dividing the monitored video stream to be stored into a plurality of video blocks, respectively storing each video block into a monitoring system cloud disk database, and storing second configuration information of each video block into the monitoring system cloud disk database according to the arrangement sequence of each video block in the monitored video stream to be stored, wherein the second configuration information comprises a second URL (uniform resource locator), and the second URL is the URL of the video block in the monitoring system cloud disk database.
When a large video file needs to be read, firstly, the URL of each video block is obtained from the block data index table of the cloud disk database of the monitoring system, and then each video block is obtained from the video block table of the cloud disk database of the monitoring system according to the URL in sequence to form the large video file. The configuration information of a large video file includes its file name, video format, file description, order of video blocks, and file URL.
Since the transaction processing of the storage operation cannot simply depend on the server itself, a cache mechanism needs to be designed for storing video file directories and data information of different scenes. Namely, the method further comprises the following steps:
generating a temporary URL and a first streaming address cache, and adding the temporary URL path to the first streaming address cache; when the volume of the surveillance video stream to be stored is larger than or equal to the first threshold value, writing the surveillance video stream to be stored into the temporary URL for storage; when the stream of surveillance video to be stored is not successfully written into the temporary URL, an arbitration mechanism is performed, as described in detail below. When the subsequent specific storage step is executed, the URL of the monitoring video stream is directly obtained from the first stream address cache, and the monitoring video stream is read from the temporary URL according to the URL.
When the monitoring video stream to be stored is not successfully stored in the cloud disk database of the monitoring system, generating a second stream address cache; judging whether the monitoring video stream to be stored is successfully stored in the file server cluster; if yes, acquiring the first URL, and adding the first URL to a second stream address cache; if not, executing an arbitration mechanism;
and when the surveillance video stream to be stored is not successfully stored in the file server cluster, the subsequent storage step is not executed, and the storage process is ended after the arbitration mechanism is executed and the surveillance video stream to be stored in the cache is cleared. At this time, the server cluster is restored to the state before the storing step of the surveillance video stream to be stored is performed.
Judging whether the first configuration information is successfully stored in the cloud disk database of the monitoring system; if so, deleting the monitoring video stream to be stored in the temporary URL; if not, executing an arbitration mechanism; when the first configuration information is also successfully stored, the monitoring video stream to be stored is successfully stored in the server cluster. At this point, the corresponding temporary URL may be deleted.
And when the first configuration information is not successfully stored in the cloud disk database of the monitoring system, executing an arbitration mechanism and clearing the monitoring video stream to be stored in the cache, and then ending the storage process.
The method also comprises generating a third stream address cache; judging whether each video block is successfully stored in the cloud disk database of the monitoring system; if so, acquiring a second URL of each video block, and adding the second URL of each video block to a third stream address cache according to the arrangement sequence of each video block in the monitored video stream to be stored; if not, executing an arbitration mechanism;
and after each video block is successfully stored in the cloud disk database of the monitoring system, the second URL of each video block can be directly obtained from the third stream address cache and then sequentially stored in the block data index table of the cloud disk database of the monitoring system. And when one video block or a plurality of video blocks are not successfully stored in the cloud disk database of the monitoring system, the subsequent storage step is not executed, and the storage process is ended after an arbitration mechanism is executed and the monitoring video stream to be stored in the cache is cleared.
Judging whether the second configuration information of each video block is successfully stored in the cloud disk database of the monitoring system; if so, deleting the corresponding temporary URL; if not, an arbitration mechanism is performed.
And when the second configuration information is also successfully stored, the monitoring video stream to be stored is successfully stored in the server cluster. At this time, the to-be-stored monitoring video stream in the temporary URL may be deleted. In addition, when the second configuration information is not successfully stored in the cloud disk database of the monitoring system, the arbitration mechanism is executed, the monitoring video stream to be stored in the cache is cleared, and then the storage process is ended.
For the above arbitration scheme, the specific steps will be described in detail below:
1: checking whether the current video file is a small video file smaller than a first threshold value; if yes, executing step 2; if not, executing the step 3;
2: executing the rollback of the cloud disk database of the monitoring system, and ending an arbitration mechanism; and when the small video file fails to be stored, the storage step is cancelled, and partial data stored in the database in the small video file is deleted.
3: checking whether a temporary URL needing to be cleaned exists in the first streaming address cache; if yes, deleting the temporary URL according to the temporary URL in the first streaming address cache;
and when the large video file fails to be written into the temporary URL, deleting the temporary URL for storing the large video file so as to prevent the temporary URL from occupying the system storage space.
Wherein, when the transaction-oriented object is not a small video file, the arbitration mechanism further comprises:
4: checking whether a first URL needing to be cleaned exists in a second stream address cache; if so, deleting the monitoring video stream to be stored in the file server cluster according to the first URL in the second stream address cache;
5: checking whether a second URL needing to be cleaned exists in a third-stream address cache; and if so, deleting the video block stored in the cloud disk database of the monitoring system according to a second URL in the third stream address cache.
Through the steps, after the corresponding data in the corresponding storage areas are deleted according to the URLs in the first to third stream address caches respectively, the server can be restored to the state before the execution method, and the integrity of the video stream data storage operation is ensured.
After the monitoring video stream to be stored is divided into at least one video block according to a predefined policy, and each video block is stored in the cloud disk database of the monitoring system, the method further comprises the following steps:
and addressing the acquired video file to acquire a storage unit corresponding to the video block. The acquiring of the storage unit corresponding to the video block includes:
calculating the hash value of the video block ID through a hash function to obtain a pseudo-random value corresponding to the video block ID; and carrying out bitwise AND operation on the pseudo-random value and the mask corresponding to the obtained video block ID to obtain a storage unit identifier corresponding to the video block.
The storage units corresponding to the video blocks obtained by addressing finally ensure that the total data volume of the objects stored in each storage unit is approximately uniform, and the real-time storage efficiency and the system performance are improved.
And addressing the storage unit corresponding to the acquired video block to acquire the monitoring cloud disk node corresponding to the storage unit. And the monitoring cloud disk nodes are jointly responsible for storing and maintaining all the blocks in the corresponding storage units.
In this embodiment, each storage unit is configured with three corresponding monitoring cloud disk nodes, so that triple backup of user storage files is realized, and meanwhile, the monitoring cloud disk nodes are configured with solid state disks, so that fast read-write operation of data is realized.
And after the monitoring cloud disk node corresponding to the storage unit is obtained, writing the obtained video file into the monitoring cloud disk node corresponding to the obtained storage unit. The video file is written into and is monitored cloud disk node, include:
step 1: and writing the acquired video file into a first monitoring cloud disk node corresponding to the acquired storage unit.
Step 2: and respectively writing the video file object written by the first monitoring cloud disk node into a second monitoring cloud disk node and a third monitoring cloud disk node.
And step 3: and sending successful storage information of the video file to the video acquisition terminal.
After the three monitoring cloud disk nodes finish write operation, the successful storage information of the storage file is sent to the video acquisition terminal under control, and finally triple safe storage of the storage file of the video acquisition terminal in the three monitoring cloud disk nodes of the cloud storage system server is achieved.
After receiving the video storage request, storing video stream data to the monitoring cloud disk node according to monitoring video stream information carried in the video storage request, further comprising: the recently used data is treated as hot data, the recently unused data is treated as cold data, the hot data is stored in the high speed hardware device, and the cold data is stored in the inactive storage node. And each newly written data is regarded as hot data and is directly stored in the monitoring cloud disk node, so that preparation is made for directly finding target data in the monitoring cloud disk node next time. The video storage requests from different storage applications can be converted into standard requests based on a uniform conversion standard, so that the video storage requests from different storage applications can be identified by a caching system applying the method disclosed by the invention, and the universality of the caching method is further improved.
As a preferred embodiment, the present invention further comprises: and updating the cache management information. And calculating the ratio of the stored data quantity of the monitoring cloud disk nodes to the capacity of the monitoring cloud disk nodes. And judging whether the ratio is greater than or equal to the threshold, if so, sequentially removing the cold data in the monitored cloud disk nodes by utilizing an LRU algorithm according to the cache management information until the ratio is less than the threshold. In order not to lose data, cooler data may be backed up to inactive storage nodes before being deleted in turn from the monitoring cloud disk nodes. As a preferred embodiment, sequentially removing the cold data in the monitoring cloud disk nodes according to the cache management information specifically includes: and sequentially storing the cold data to the inactive storage nodes, and deleting the cold data which are stored to the inactive storage nodes and stored in the monitoring cloud disk nodes.
When the storage process of the cloud disk database of the monitoring system is terminated due to an error, the invention further confirms the fault position by the following method so as to correctly retry the write operation and execute data recovery. The cloud disk database of the monitoring system at least comprises m storage nodes and q error correction nodes. The storage procedure based on BMSR coding is as follows:
step 1, a storage node determines a first storage request, wherein the first storage request carries a monitored video stream to be stored and a first update mark, the first update mark is used for indicating an update mark of a video block which is stored by a video acquisition terminal and is updated by the monitored video stream to be stored, and the data volume of the monitored video stream to be stored is less than that of a BMSR coding group;
and 2, the storage node responds to the first storage request to acquire a second update mark, wherein the second update mark is used for indicating the update mark of the video block stored by the storage node.
Step 3, according to the corresponding result between the first update mark and the second update mark, processing the monitoring video stream to be stored, and further comprising:
step 31, the storage node determines whether the first update flag is the same as the second update flag, if so, the first update flag corresponds to the second update flag, and if not, the first update flag does not correspond to the second update flag. The second update mark is the latest update mark, if the second update mark is earlier than the first update mark, the update is not completed in a certain storage node and the video acquisition terminal is considered to be completed, so that the video acquisition terminal updates the update mark of the video acquisition terminal, and the storage node can send error report information to the storage system or the video acquisition terminal as long as the second update mark is later than or earlier than the first update mark and the second update mark is not corresponding to the first update mark.
And step 32, the storage node processes the monitoring video stream to be stored according to the first judgment value. In a further embodiment of the present invention, step 32 further comprises:
step 321, if the first judgment value indicates that the first update flag does not correspond to the second update flag, sending a first update response, where the first update response carries an update failure result;
step 322, discarding the to-be-stored monitoring video stream.
Step 323, if the first judgment value indicates that the first update flag corresponds to the second update flag, determining an update video block set { Δ P ] of error correction increments according to the monitored video stream to be stored and the video block1，ΔP2，...，ΔPg,...，ΔPqIn which Δ PgG is more than or equal to 1 and less than or equal to q, and q is the number of check video blocks corresponding to one BMSR code;
step 324, add the error correction delta P of the g-th error correction nodegCarried in a second storage request, sending the second storage request to a g-th error correction node, where the second storage request is used to trigger the g-th error correction node to perform error correction according to the error correction increment Δ PgAnd updating the check data stored in the error correction node per se, and updating the version vector stored in the error correction node per se.
The processing the monitoring video stream to be stored according to the first judgment value further comprises: if the first judgment value shows that the first updating mark corresponds to the second updating mark, updating the video block by using the monitoring video stream to be stored; and after the video block is updated, updating the second updating mark.
And after the video block is updated, sending a first updating response, wherein the first updating response carries an updating success result, and the updating success result is used for triggering the video acquisition terminal to update the first updating mark. In addition, a timer can be set, and if the updating is not successful within the specified time, the video acquisition terminal can reinitiate the storage request.
The storage node maintains an update mark for each initial video block, when write operation is performed, the storage node firstly judges whether the update mark of the received data corresponds to the update mark stored by the storage node, and when the update mark of the received data corresponds to the update mark stored by the storage node, the storage node updates the data in the storage request, so that the storage process is prevented from being terminated due to errors.
In the BMSR coding, assuming that a video file has a volume of M, the video file is divided into a plurality of video blocks, each k video blocks are combined into one volume, and coding operations of the BMSR coding are independent in each volume. To maximize the reliability of the data, the encoded n blocks (including k video blocks and n-k encoded blocks) are stored in n different monitoring cloud disk nodes.
In the running process of the cloud disk database of the monitoring system, when r nodes fail, other r alternative nodes are required to be searched to restore the failed data. The present invention adopts a restoration model represented by (n, k, r). And the source node completes the tasks of data segmentation, coding and distribution. X1，X2，…，Xn-rRepresenting active storage nodes, Y1，Y2，…，YrRepresenting r substitute nodes. And the target node reads any k nodes to complete the reconstruction of the initial data.
Specifically, the running process includes 3 steps, initialization, data recovery, and download. During initialization, an initial video file with a volume of M is partitioned into a plurality of video blocks, and each k video blocks form a volume. Suppose video file M is partitioned into k video blocks. The k video blocks are encoded to generate n-k encoded blocks. Let k video blocks be denoted as (d)1，d2，…，dk) Each block has a volume of M/k, k video blocks generate n-k coding blocks (p) by coding1，p2，…，pn-k)。
Wherein each coding block pi(1. ltoreq. i. ltoreq. n-k) are generated by coding k video blocks, alphai，jIs piFrom the jth video block djThe coding coefficients of (1). Finally, the process is carried out in a batch,after the coding is finished, n blocks (k video blocks and n-k coding blocks) are respectively stored in n storage nodes (X)1，X2，…，Xn) And the data amount stored by each node is M/k. I.e. the coding block piIs recovered data PiIs composed of
When r storage nodes Xn-r+1，Xn-r，…，XnAfter the fault, r replacing nodes are reselected to complete the restoration of the data Y in order to ensure the original redundancy of the data1，Y2，…，Yr. During the recovery process, one master node Y is selected from the r alternative nodesmain. The node connects the owner node with the remaining substitute nodes at the same time. Specifically, the method comprises the following steps:
firstly, selecting k owner nodes from active storage nodes, and selecting r substitute nodes from other available nodes; simultaneously, a main node Y is selected from r substitute nodesmain. Second, master node YmainRead k video blocks from k providers and recover r failed blocks from the k video blocks. Finally, YmainThe corresponding restored video block is stored locally while the remaining r-1 restored video blocks are sent to the respective r-1 replacement nodes, respectively. And the other substitute nodes store the received data locally to finish the restoration process.
After restoration is completed, n-r active nodes X are initialized1，X2，…，Xn-rAnd r substitute nodes Y1，Y2，…，YrAnd composing a new n storage node combination to provide video storage service, wherein the access request of the data can rebuild the initial data through any k nodes.
And after the alternative node is selected, selecting the alternative node with the minimum topological distance with other alternative nodes as the main node. Suppose an active node XiAnd provider YjThe topological distance between them is denoted as distancei，jThen the node with the shortest total topological distance from k to r alternative nodes is selected from the n-r active nodes as the provider. Active node XiAnd provider YjThe total topological distance between them is:
in the selection process, n-r total topological distances are arranged in an ascending mode, and finally the first k nodes with the minimum total topological distances are taken as providers X1，X2，…，Xk。
The selection of the master node needs to take into account the topological distance between the master node and the master node. Suppose a substitute node YiTo YjIs distance'i，jThen replace node YiThe sum of topological distances to other r-1 alternative nodes is
Selecting the node with the minimum total topological distance to other substitute nodes and providers as the main node Ymain。
distancemain＝min1≤i≤rdistance’i)
After the alternative node and the provider are selected, data transmission is realized between the provider and the main node and between the main node and other providers. Between provider and host node, all k providers X1，X2，…，XkWith the master node YmainConnect and transmit data to it. Each provider transmits a video block of volume M/k to the master node. And the main node completes the recovery operation of the r fault blocks through the encoding and decoding operation after receiving the data of the provider.
Between the main node and other providers, after the main node completes the construction work of r fault blocks, the corresponding blocks are stored locally, and the rest r-1 blocks are respectively sent to corresponding substitute nodes.
And other substitute nodes are stored locally after receiving the data of the substitute nodes. And after all the substitute nodes complete the data receiving and storing operation, the recovery process is finished.
Main node YmainIn the specific calculation process, different calculation operations are correspondingly performed according to the specific condition that the fault data is a video block or a coding block. Suppose the number of video blocks and the number of coding blocks in the r invalid video blocks are r respectively1And r2，r＝r1+r2。r1The failed video block is denoted as d1，d2…，dr1，r2The failing code block is denoted as p1，p2…，pr2The data read by the substitute node from the k providers is b ═ b (b)1，b2，…，bk) Then video block diAnd coding block pjIs calculated as follows:
1) read k video block vectors b ═ (b)1，b2，…，bk) The invalid video block can be directly recovered by a decoding operation. Assuming that the encoding matrix is P, the encoding process can be represented as P × D ═ C, where D is a video block; and C is a matrix of the video block corresponding to the coding block. If k video blocks extracted from C are b ═ b (b)1，b2，…，bk) If the corresponding matrix is a matrix P ', a matrix D ═ P ' can be obtained by matrix operation '-1B. While matrix D contains all video blocks, and thus, DiThe recovery operation of (a) may be expressed as:
di＝(λ1，λ2，…，λk)T*(b1，b2，…，bk)
wherein λ isiIs P'-1In diThe corresponding vector.
2) The recovery operation of the coding block is to do the coding operation of the corresponding position again on the basis of decoding. Specifically, if the k video blocks extracted from C are vectors b ═ b (b)1，b2，…，bk) The corresponding matrix is the matrix P', the decoding operationMay be represented by D ═ P'-1B. And p isjCan be expressed as pj＝βjD, wherein the vector βjIs pjThe corresponding vector, then pjThe calculation process of (a) can be expressed as:
pj＝(β1，β2，…βk)*P’-1*(b1，b2，…，bk)
after all r blocks (r) are completed1A video block and r2Individual encoded blocks), the master node sends the recovered video blocks to the corresponding replacement nodes. Reading a part of data and simultaneously constructing r video blocks through different parameters, and simultaneously, adopting a multithreading technology for computing in the substitute node to increase the parallelism of the computing process.
In conclusion, the invention provides a high-speed media storage method, which can meet the storage requirement of massive small video files, ensure the processing efficiency of ultra-large video files and improve the storage performance of intelligent traffic data.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented in a general purpose computing system, centralized on a single computing system, or distributed across a network of computing systems, and optionally implemented in program code that is executable by the computing system, such that the program code is stored in a storage system and executed by the computing system. Thus, the present invention is not limited to any specific combination of hardware and software.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explaining the principles of the invention and are not to be construed as limiting the invention. Therefore, any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. Further, it is intended that the appended claims cover all such variations and modifications as fall within the scope and boundaries of the appended claims or the equivalents of such scope and boundaries.
Claims (5)
1. A method for high-speed storage of media, comprising:
receiving a video storage request and judging the volume of video stream data;
if the volume of the video stream data is smaller than a first threshold value, submitting the video file to a cloud disk database of a monitoring system for storage;
storing the video file in a file server cluster if the volume of the video stream data is greater than a second threshold;
the method further comprises the following steps:
generating a temporary URL and a first streaming address cache, and adding the temporary URL path to the first streaming address cache; when the volume of the surveillance video stream to be stored is larger than or equal to the first threshold value, writing the surveillance video stream to be stored into the temporary URL for storage; when the monitoring video stream to be stored is not successfully written into the temporary URL, executing an arbitration mechanism; when the subsequent specific storage step is executed, directly acquiring the URL of the monitoring video stream from the first stream address cache, and reading the monitoring video stream from the temporary URL according to the URL;
when the monitoring video stream to be stored is not successfully stored in the cloud disk database of the monitoring system, generating a second stream address cache; judging whether the monitoring video stream to be stored is successfully stored in the file server cluster; if yes, acquiring a first URL, and adding the first URL to a second stream address cache; if not, executing an arbitration mechanism;
when the surveillance video stream to be stored is not successfully stored in the file server cluster, the subsequent storage step is not executed, and the storage process is ended after an arbitration mechanism is executed and the surveillance video stream to be stored in the cache is cleared; at this time, the server cluster is restored to the state before the storage step of the monitoring video stream to be stored is executed;
judging whether the first configuration information is successfully stored in the cloud disk database of the monitoring system; if so, deleting the monitoring video stream to be stored in the temporary URL; if not, executing an arbitration mechanism; when the first configuration information is also successfully stored, the monitoring video stream to be stored is successfully stored in the server cluster; at this time, the corresponding temporary URL may be deleted;
and when the first configuration information is not successfully stored in the cloud disk database of the monitoring system, executing an arbitration mechanism and clearing the monitoring video stream to be stored in the cache, and then ending the storage process.
2. The method of claim 1, further comprising, after receiving the video storage request:
the authority of the video storage request is verified, and whether the data are allowed to be stored in the corresponding storage area is judged according to the source of the video storage request;
and when the authority verification is passed, continuing the data storage process.
3. The method of claim 2, wherein the data storage process is terminated when the rights verification fails.
4. The method of claim 1, wherein when the volume of the surveillance video stream to be stored is greater than a second threshold, storing the surveillance video stream to be stored to a cluster of file servers, further comprising:
storing first configuration information to the cloud disk database of the monitoring system, wherein the first configuration information comprises a first URL (uniform resource locator), and the first URL is the URL of the monitoring video stream to be stored in the file server cluster;
when a super-large video file needs to be read, firstly, a URL of the super-large video file is obtained from a block data index table of a cloud disk database of a monitoring system, and then the video file is directly read from a file directory of a file server cluster according to the URL.
5. The method of claim 1, wherein the configuration information of the video file comprises: file name, video format, file description, order of video blocks, and file URL.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810209643.5A CN108491488B (en) | 2018-03-14 | 2018-03-14 | High-speed medium storing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810209643.5A CN108491488B (en) | 2018-03-14 | 2018-03-14 | High-speed medium storing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108491488A CN108491488A (en) | 2018-09-04 |
| CN108491488B true CN108491488B (en) | 2021-11-30 |
Family
ID=63339235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810209643.5A Active CN108491488B (en) | 2018-03-14 | 2018-03-14 | High-speed medium storing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108491488B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102546755A (en) * | 2011-12-12 | 2012-07-04 | 华中科技大学 | Data storage method of cloud storage system |
| CN104615691A (en) * | 2015-01-24 | 2015-05-13 | 刘勇 | Mobile terminal and data storage method |
| CN105183839A (en) * | 2015-09-02 | 2015-12-23 | 华中科技大学 | Hadoop-based storage optimizing method for small file hierachical indexing |
| CN106790324A (en) * | 2015-11-20 | 2017-05-31 | 华为技术有限公司 | Content distribution method, virtual server management method, cloud platform and system |
| CN106899675A (en) * | 2017-02-28 | 2017-06-27 | 福建新和兴信息技术有限公司 | A kind of file memory method and system based on Application Type |
-
2018
- 2018-03-14 CN CN201810209643.5A patent/CN108491488B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102546755A (en) * | 2011-12-12 | 2012-07-04 | 华中科技大学 | Data storage method of cloud storage system |
| CN104615691A (en) * | 2015-01-24 | 2015-05-13 | 刘勇 | Mobile terminal and data storage method |
| CN105183839A (en) * | 2015-09-02 | 2015-12-23 | 华中科技大学 | Hadoop-based storage optimizing method for small file hierachical indexing |
| CN106790324A (en) * | 2015-11-20 | 2017-05-31 | 华为技术有限公司 | Content distribution method, virtual server management method, cloud platform and system |
| CN106899675A (en) * | 2017-02-28 | 2017-06-27 | 福建新和兴信息技术有限公司 | A kind of file memory method and system based on Application Type |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108491488A (en) | 2018-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110169040B (en) | Distributed data storage method and system based on multilayer consistent hash | |
| JP6294518B2 (en) | Synchronous mirroring in non-volatile memory systems | |
| US10372537B2 (en) | Elastic metadata and multiple tray allocation | |
| US8271447B1 (en) | Mirroring metadata in a continuous data protection environment | |
| US9411685B2 (en) | Parity chunk operating method and data server apparatus for supporting the same in distributed raid system | |
| US6912669B2 (en) | Method and apparatus for maintaining cache coherency in a storage system | |
| US7882286B1 (en) | Synchronizing volumes for replication | |
| CN106776130B (en) | Log recovery method, storage device and storage node | |
| US10223007B1 (en) | Predicting IO | |
| CN114415976B (en) | Distributed data storage system and method | |
| US11003558B2 (en) | Systems and methods for sequential resilvering | |
| CN103631539A (en) | Distributed storage system and distributed storage method based on erasure coding mechanism | |
| US7849355B2 (en) | Distributed object sharing system and method thereof | |
| US8543864B2 (en) | Apparatus and method of performing error recovering process in asymmetric clustering file system | |
| CN113326006A (en) | Distributed block storage system based on erasure codes | |
| CN108174136B (en) | Cloud disk video coding storage method | |
| CN108366217B (en) | Monitoring video acquisition and storage method | |
| CN112486942A (en) | Multi-copy storage method and multi-copy storage system for file data | |
| CN108491488B (en) | High-speed medium storing method | |
| CN116166196A (en) | Method and device for recovering expansion and contraction capacity of storage pool in distributed storage system | |
| CN115470041A (en) | Data disaster recovery management method and device | |
| CN112783688B (en) | Erasure code data recovery method and device based on available partition level | |
| CN116868173A (en) | Reducing the impact of network latency during recovery operations | |
| CN117851132B (en) | Data recovery optimization method for distributed object storage | |
| CN111400302B (en) | Modification method, device and system for continuous storage data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |