CN111090617A - File management method and device in monitoring system and computer readable storage medium - Google Patents
File management method and device in monitoring system and computer readable storage medium Download PDFInfo
- Publication number
- CN111090617A CN111090617A CN201811235338.XA CN201811235338A CN111090617A CN 111090617 A CN111090617 A CN 111090617A CN 201811235338 A CN201811235338 A CN 201811235338A CN 111090617 A CN111090617 A CN 111090617A
- Authority
- CN
- China
- Prior art keywords
- file
- files
- media
- current
- subdirectory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0625—Power saving in storage systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a file management method, a file management device and a computer readable storage medium in a monitoring system, wherein the method comprises the following steps: storing the monitoring file based on three layers of directories including a channel, a date and a file size; when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory; two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
Description
Technical Field
The present invention relates to the field of mobile communications technologies, and in particular, to a method and an apparatus for managing files in a monitoring system, and a computer-readable storage medium.
Background
In an industrial monitoring system, the storage period of monitoring audio and video is generally between seven days and two years, most audio and video monitoring files are stored in a media platform according to time periods at present, and disk fragments are caused when the audio and video monitoring files are circularly deleted.
In order to reduce the occurrence of disk fragmentation caused by deletion, the related method is to combine a small file with a large file, for example, a channel name is used as a primary directory, a date is used as a secondary directory to manage the file of each channel, so as to realize dynamic creation of the file and realize dynamic expansion or reduction of the channel, but after the file is dynamically created, the writing speed of the disk can be influenced because the operation of '0' of pre-writing is not carried out; in addition, since each file is fixed in size, when each fixed-size file may not be fully written, particularly when the data to be written in the last file of each day is small, it causes a waste of storage. Moreover, creating a file when there are insufficient free files reduces the first write rate of the file.
Disclosure of Invention
In view of the above, embodiments of the present invention are intended to provide a method and an apparatus for managing files in a monitoring system, and a computer-readable storage medium.
In order to achieve the above purpose, the technical solution of the embodiment of the present invention is realized as follows:
the embodiment of the invention provides a file management method in a monitoring system, which comprises the following steps:
storing the monitoring file based on three layers of directories including a channel, a date and a file size;
when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
Optionally, the method further includes:
creating files with corresponding number and size of total capacity larger than the sum of daily storage capacity of the channels according to the estimated number of the channels;
the file is pre-written \0' and moved to the corresponding subdirectory.
Optionally, the method further includes:
checking the number of idle files under each subdirectory regularly;
and when the number of the idle files is determined to be smaller than the threshold value, creating corresponding files and preprocessing the files.
Wherein, the method also comprises the following steps:
after the video in the preset period is packaged into the MP4, judging whether the residual storage capacity of the current file can be written into the MP4 media data;
if not, closing the current file, determining the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially.
Wherein after the sequentially writing the current file, the method further comprises:
determining whether the writing is on the same day as the starting time of the current MP 4;
if the current file is in the same day, determining the residual capacity of the current file, determining whether the residual capacity can be written into the media data of the next preset period according to the media capacity of the previous preset period, and if so, returning to the writing operation; if the time length of the media file which can be written is not the same as the time length of the media file which can be written, and the operation of closing the current file or returning to the writing operation is executed based on the time length;
if not, repeating the on-demand process when the first storage period is determined not to be expired; and if the second storage period is up, renaming the sub-directories of the dates under the channel directories, and completely moving the original files in the sub-directories under the dates to the corresponding sub-directories under the free file directories.
Wherein, the method also comprises the following steps in the live broadcast process:
inquiring timeliness of media slice merging files and index files under each channel directory at intervals of preset time;
and when the last modification time of the media slice merging file or the index file is determined to be larger than a preset threshold value, recovering the related files under the corresponding channel directories.
Wherein, the method also comprises the following steps in the live broadcast process:
inquiring the number of media slice merging files and index files in a live idle file directory at intervals of preset time;
and when the number of the media slice merging files or the index files is less than a preset threshold value, creating corresponding files.
An embodiment of the present invention further provides a file management apparatus in a monitoring system, where the apparatus includes:
the storage module is used for storing the monitoring file based on three layers of directories including a channel, a date and a file size;
the processing module is used for transferring the monitoring file from the subdirectory with the corresponding size of the idle directory to the corresponding subdirectory on the current date when the monitoring file with the preset size needs to be added is determined; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
the live broadcast processing module is used for circularly using two preset media slices in the live broadcast process to merge the file storage media slices; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
An embodiment of the present invention further provides a file management apparatus in a monitoring system, where the apparatus includes: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of the above method when running the computer program.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-mentioned method.
The file management method, the file management device and the computer-readable storage medium in the monitoring system provided by the embodiment of the invention store the monitoring files based on three layers of directories including a channel, a date and a file size; when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory; two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period. The embodiment of the invention adopts three layers of directories to manage and store the files, when the storage period is up, the files do not need to be deleted, only the date subdirectory under the corresponding channel directory needs to be renamed, and the original files in each subdirectory under the date are moved to the corresponding subdirectory under the idle file directory, thereby not only realizing the rapidness, but also reducing the fragments caused by the file deletion. In addition, in the live broadcasting process, the two preset media slices are recycled to merge the file storage media slices, so that the number of small files is reduced, the maintenance cost is reduced, and disk fragments cannot be introduced due to file deletion.
In addition, according to the estimated channel quantity, files with the corresponding number and size of the total capacity larger than the sum of the daily storage capacity of the channels are created; the number of idle files under each subdirectory is checked regularly at the later stage, and when the number of the idle files is determined to be smaller than a threshold value, corresponding files are created; the method not only ensures the preprocessing speed, but also can not cause performance overhead caused by the fact that files are not used enough to create the files by the write thread, and the size of the stored files is not fixed and can be dynamically adjusted according to the needs, so that the storage loss caused by file switching can be reduced.
Drawings
Fig. 1 is a schematic flow chart of a file management method in a monitoring system according to an embodiment of the present invention;
FIG. 2 is a first schematic structural diagram of a file management apparatus in a monitoring system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a file management device in the monitoring system according to the embodiment of the present invention.
Detailed Description
The invention is described below with reference to the figures and examples.
An embodiment of the present invention provides a file management method in a monitoring system, and as shown in fig. 1, the method includes:
step 101: storing the monitoring file based on three layers of directories including a channel, a date and a file size;
step 102: when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
step 103: two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
The sequence of the above steps is not limited.
The embodiment of the invention adopts three layers of directories to manage and store the files, when the storage period is up, the files do not need to be deleted, only the date subdirectory under the corresponding channel directory needs to be renamed, and the original files in each subdirectory under the date are moved to the corresponding subdirectory under the idle file directory, thereby not only realizing the rapidness, but also reducing the fragments caused by the file deletion. In addition, in the live broadcasting process, the two preset media slices are recycled to merge the file storage media slices, so that the number of small files is reduced, the maintenance cost is reduced, and disk fragments cannot be introduced due to file deletion.
In one embodiment, the method further comprises:
creating files with corresponding number and size of total capacity larger than the sum of daily storage capacity of the channels according to the estimated number of the channels;
the file is pre-written \0' and moved to the corresponding subdirectory.
In one embodiment, the method further comprises:
checking the number of idle files under each subdirectory regularly;
and when the number of the idle files is determined to be smaller than the threshold value, creating corresponding files and preprocessing the files.
In one embodiment, the method further comprises the following steps:
after the video in the preset period is packaged into the MP4, judging whether the residual storage capacity of the current file can be written into the MP4 media data;
if not, closing the current file, determining the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially.
In one embodiment, after the sequentially writing the current file, the method further comprises:
determining whether the writing is on the same day as the starting time of the current MP 4;
if the current file is in the same day, determining the residual capacity of the current file, determining whether the residual capacity can be written into the media data of the next preset period according to the media capacity of the previous preset period, and if so, returning to the writing operation; if the time length of the media file which can be written is not the same as the time length of the media file which can be written, and the operation of closing the current file or returning to the writing operation is executed based on the time length;
if not, repeating the on-demand process when the first storage period is determined not to be expired; and if the second storage period is up, renaming the sub-directories of the dates under the channel directories, and completely moving the original files in the sub-directories under the dates to the corresponding sub-directories under the free file directories.
In one embodiment, the method further includes, during the live broadcast:
inquiring timeliness of media slice merging files and index files under each channel directory at intervals of preset time;
and when the last modification time of the media slice merging file or the index file is determined to be larger than a preset threshold value, recovering the related files under the corresponding channel directories.
In one embodiment, the method further includes, during the live broadcast:
inquiring the number of media slice merging files and index files in a live idle file directory at intervals of preset time;
and when the number of the media slice merging files or the index files is less than a preset threshold value, creating corresponding files.
An embodiment of the present invention further provides a file management apparatus in a monitoring system, and as shown in fig. 2, the apparatus includes:
the storage module 201 is used for storing the monitoring file based on three layers of directories including a channel, a date and a file size;
the processing module 202 is configured to, when determining that a monitoring file with a preset size needs to be added, transfer the monitoring file from a subdirectory with a corresponding size of an idle directory to a corresponding subdirectory on a current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
the live broadcast processing module 203 is used for cyclically using two preset media slices in the live broadcast process to merge the file storage media slices; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
In one embodiment, as shown in fig. 3, the apparatus further comprises:
a creating module 204, configured to create, according to the estimated number of channels, files with corresponding numbers and sizes, where a total capacity is greater than a sum of daily storage capacities of the channels;
the file is pre-written \0' and moved to the corresponding subdirectory.
In one embodiment, the creating module 204 is further configured to periodically check the number of free files in each sub-directory;
and when the number of the idle files is determined to be smaller than the threshold value, creating corresponding files and preprocessing the files.
In one embodiment, as shown in fig. 3, the apparatus further comprises:
the on-demand processing module 205 is configured to, after the video in the preset period is encapsulated into the MP4, determine whether the remaining storage capacity of the current file can be written into the current MP4 media data;
if the current file cannot be written, closing the current file, determining the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially.
In an embodiment, the on-demand processing module 205 is further configured to determine whether the writing is on the same day as the start time of the current MP 4;
determining the residual capacity of the current file on the same day, determining whether the residual capacity can be written into the media data of the next preset period according to the media capacity of the previous preset period, and returning to the writing operation if the residual capacity can be written into the media data of the next preset period; if the time length of the media file which can be written is not the same as the time length of the media file which can be written, and the operation of closing the current file or returning to the writing operation is executed based on the time length;
determining that the storage period is not in the same day, and repeating the on-demand process when the first storage period is determined not to be expired; and if the second storage period is up, renaming the sub-directories of the dates under the channel directories, and completely moving the original files in the sub-directories under the dates to the corresponding sub-directories under the free file directories.
In an embodiment, the live broadcast processing module 203 is further configured to query timeliness of a media slice merge file and an index file under each channel directory at intervals of preset time;
and when the last modification time of the media slice merging file or the index file is determined to be larger than a preset threshold value, recovering the related files under the corresponding channel directories.
In one embodiment, the live processing module 203 is further configured to query the number of media slice merge files and index files in a live idle file directory at intervals of preset time;
and when the number of the media slice merging files or the index files is less than a preset threshold value, creating corresponding files.
An embodiment of the present invention further provides a file management apparatus in a monitoring system, where the apparatus includes: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to execute, when running the computer program:
storing the monitoring file based on three layers of directories including a channel, a date and a file size;
when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
The processor is further configured to, when executing the computer program, perform:
creating files with corresponding number and size of total capacity larger than the sum of daily storage capacity of the channels according to the estimated number of the channels;
the file is pre-written \0' and moved to the corresponding subdirectory.
The processor is further configured to, when executing the computer program, perform:
checking the number of idle files under each subdirectory regularly;
and when the number of the idle files is determined to be smaller than the threshold value, creating corresponding files and preprocessing the files.
The processor is further configured to, when executing the computer program, perform:
after the video in the preset period is packaged into the MP4, judging whether the residual storage capacity of the current file can be written into the MP4 media data;
if not, closing the current file, determining the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially.
After the sequentially writing the current file, the processor is further configured to, when running the computer program, perform:
determining whether the writing is on the same day as the starting time of the current MP 4;
if the current file is in the same day, determining the residual capacity of the current file, determining whether the residual capacity can be written into the media data of the next preset period according to the media capacity of the previous preset period, and if so, returning to the writing operation; if the time length of the media file which can be written is not the same as the time length of the media file which can be written, and the operation of closing the current file or returning to the writing operation is executed based on the time length;
if not, repeating the on-demand process when the first storage period is determined not to be expired; and if the second storage period is up, renaming the sub-directories of the dates under the channel directories, and completely moving the original files in the sub-directories under the dates to the corresponding sub-directories under the free file directories.
In the live broadcast process, the processor is further configured to execute, when running the computer program:
inquiring timeliness of media slice merging files and index files under each channel directory at intervals of preset time;
and when the last modification time of the media slice merging file or the index file is determined to be larger than a preset threshold value, recovering the related files under the corresponding channel directories.
In the live broadcast process, the processor is further configured to execute, when running the computer program:
inquiring the number of media slice merging files and index files in a live idle file directory at intervals of preset time;
and when the number of the media slice merging files or the index files is less than a preset threshold value, creating corresponding files.
It should be noted that: in the device provided in the above embodiment, when performing file management, only the division of each program module is illustrated, and in practical applications, the above processing allocation may be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the apparatus provided in the above embodiments and the corresponding method embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
In an exemplary embodiment, the embodiment of the present invention also provides a computer-readable storage medium, which may be a Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disc, or CD-ROM; or may be a variety of devices including one or any combination of the above memories, such as a mobile phone, computer, tablet device, personal digital assistant, etc.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs:
storing the monitoring file based on three layers of directories including a channel, a date and a file size;
when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
The computer program, when executed by the processor, further performs:
creating files with corresponding number and size of total capacity larger than the sum of daily storage capacity of the channels according to the estimated number of the channels;
the file is pre-written \0' and moved to the corresponding subdirectory.
The computer program, when executed by the processor, further performs:
checking the number of idle files under each subdirectory regularly;
and when the number of the idle files is determined to be smaller than the threshold value, creating corresponding files and preprocessing the files.
The computer program, when executed by the processor, further performs:
after the video in the preset period is packaged into the MP4, judging whether the residual storage capacity of the current file can be written into the MP4 media data;
if not, closing the current file, determining the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially.
After the sequential writing to the current file, the computer program, when executed by the processor, further performs:
determining whether the writing is on the same day as the starting time of the current MP 4;
if the current file is in the same day, determining the residual capacity of the current file, determining whether the residual capacity can be written into the media data of the next preset period according to the media capacity of the previous preset period, and if so, returning to the writing operation; if the time length of the media file which can be written is not the same as the time length of the media file which can be written, and the operation of closing the current file or returning to the writing operation is executed based on the time length;
if not, repeating the on-demand process when the first storage period is determined not to be expired; and if the second storage period is up, renaming the sub-directories of the dates under the channel directories, and moving all original files in the sub-directories under the dates to the corresponding sub-directories under the free file directories.
In the live broadcast process, when the computer program is executed by the processor, the computer program further executes:
inquiring timeliness of media slice merging files and index files under each channel directory at intervals of preset time;
and when the last modification time of the media slice merging file or the index file is determined to be larger than a preset threshold value, recovering the related files under the corresponding channel directories.
In the live broadcast process, when the computer program is executed by the processor, the computer program further executes:
inquiring the number of media slice merging files and index files in a live idle file directory at intervals of preset time;
and when the number of the media slice merging files or the index files is less than a preset threshold value, creating corresponding files.
The invention is described below in conjunction with the scenario embodiments.
This embodiment provides a live, on-demand system based on trade control field, includes: the system comprises seven main modules of preprocessing, camera media access, on-demand storage, live broadcasting caching, instant review and on-demand; wherein the content of the first and second substances,
the preprocessing module is responsible for formatting the storage disk, creating a small number of files with different sizes according to the estimated number of channels, and performing '0' pre-writing operation on the files so as to improve the writing speed;
the camera access module is responsible for analyzing an H.264 bare stream from a media stream of GBT 28181;
the on-demand storage module renames a date subdirectory of the last day of the previous period at 0 point 0 min 0 sec every day, moves a video file below the date subdirectory to an idle file directory, acquires a file with a proper size from the idle file directory when the writing requirement exists, and sequentially writes the contents of the MP4 files packaged by H.264 in the later TM time period into the files;
the live broadcast module is responsible for segmenting the H.264 bare stream to generate a ts slicing file;
the live broadcast storage module is responsible for pre-distributing and initializing two media slice merging files, one is used for storing ts slices in the current TM time, and the other is used for storing ts slices in the last TM time;
the instant review module is responsible for reading the media data from the HFA and the HFB and returning the media data to the client;
and the on-demand module is responsible for reading the MP4 file in the specified time period from the storage file and sending the MP4 file to the client.
The preprocessing module of the embodiment of the invention comprises the following steps:
step one, an idle file directory is created, and subdirectories with the names of 1G, 800M, 600M, 400M, 200M and 100M are created under the idle file directory. The directory name may be: (ii)/mnt/storage/idle/1G;
step two, a plurality of files with total capacity about larger than the sum of the daily storage capacity of the channels are pre-created according to the estimated channel number, as the code rate of the monitoring video is basically between 2M and 8M, taking 2M as an example, the daily storage capacity is about 21G, and the daily storage capacity is 168G calculated by using 8M code rate, the total capacity of the pre-created storage files is Volum ═ Camsum:. 168G δ, wherein δ is an empirical parameter, the number of bytes additionally introduced by MP4 packaging is mainly considered, and δ is tentatively 1.01;
step three, determining the size and the number of files to be created according to the total daily storage capacity, creating the files in subdirectories with corresponding sizes under the free file directory, and performing pre-writing '\0' operation, wherein the file name naming rule $ size _ $ seq is as follows, for example: and/mnt/storage/idle/1G/1G _ 1. The file sizes are 1G, 800M, 600M, 400M, 200M and 100M, wherein the number of files with different sizes is as follows:
FSum1G＝$Volum/1G；
FSum800M＝FSum600M＝FSum400M＝FSum200M＝FSum100M＝$CamSum*2。
and step four, when the number of files with various sizes in the free file directory is periodically detected to be less than $ Camsum x 2, corresponding files need to be created, the number of the files with various sizes is ensured to be equal to $ Camsum x 2, and the operation of '0' pre-writing is carried out, so that the real-time property of the written files is ensured.
The implementation steps of the on-demand storage module of the embodiment of the invention are as follows:
step one, after the equipment starts to push flow, setting the sequence number $ sequence of the MP4 to be 0;
and step two, when the day in the first storage period starts, creating a channel name directory for each channel, and creating a date subdirectory under the channel name directory. Subdirectories with names 1G, 800M, 600M, 400M, 200M, 100M are created under the date subdirectories.
Example directory format 65010000001327662615/20180405/800M/.
And step three, estimating the remaining storage capacity of the current day according to the equipment code rate and the remaining time of the current day, and if the remaining storage capacity of the current day is larger than 1G, transferring a 1G file from the free file directory to a 1G subdirectory under the current date directory. Otherwise, transferring the file with the corresponding size from the free file directory to the subdirectory with the corresponding size of the current storage directory. The method for calculating the residual storage capacity of the day according to the equipment code rate comprises the following steps: volleft＝$Bitrate*Tleftγ, where γ is the storage overhead introduced by the MP4 encapsulation, tentatively 1.01.
Step four, the on-demand storage module stores the TMAfter the video in time is packaged into the MP4, whether the residual storage capacity of the current file can be written into the MP4 media data is judged. If it is notIf not, closing the current file, estimating the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially. After the file is successfully written, the file name of the MP4 is generated, the file name naming mode is $ machien ID _ $ Date _ $ sequence.MP4, and the file name of the MP4, the file length of the MP4, the starting and ending time of the MP4, the stored file name and the starting position of the MP4 section are written into the database. $ sequence is increased by 1. Turning to the step five, namely, turning to the step five,
step five, checking the current time after the MP4 is written into the time, jumping to step ten if the current time is not the same as the starting time of the MP4, otherwise jumping to step six;
step six, calculating the residual capacity of the current file, and estimating whether the residual capacity can be written into the media data of the next TM time period according to the media capacity of the previous TM period, wherein the estimation method of the media capacity of the next TM time period is VolTm’＝VolTmη, wherein VolTmMedium capacity, Vol, of last Tm periodTm"the medium capacity in the next Tm period, η is an empirical value, and is generally set to 1.01 in consideration of the fluctuation of the code rate.
Step seven, if yes, the step four is carried out;
step eight, if not, estimating the time length of the media file which can be written, such as TM', then the next MP4 package is only for TM' the data is packaged and written into the current file, and the estimation method comprises the following steps: t isM’＝(Volleft*TM)/VolTmη if TM' greater than 10s, adding TMTemporarily set to TM' go to step four;
step nine, if TM' less than 10s, close the current file. Repeating the third step to the ninth step;
step ten, if the first storage period has not expired, repeating two-nine steps, if the second storage period has expired, renaming the sub-directory for $ Date- $ Duration time under each channel directory. And repeating the third step to the ninth step.
The mechanism for realizing live broadcast caching in the embodiment of the invention is as follows:
step one, when the system is started, a plurality of media slice merging files (merge files) and index files (m3u8 files) are created in an idle file directory and are filled with '0' characters. Because the maximum code rate of video monitoring is 8192kbps, the size of merge files is set to Vol for convenient realization in consideration of the overhead of audio and TS headersmerge＝8192kps*TMη/8, wherein η is 1.2 ≈ 1280 × TMkB. In particular, when TMThe merge file size is 384M 300 s. The size of the pre-allocated space for each ts slice is then: TS (transport stream)vol＝1280*TtsKB, M3u8 index file is preset to 1M.
And step two, when a client requests the live broadcast of a certain channel, checking whether the current live broadcast exists, and if so, directly jumping to a live broadcast module for processing. If the live broadcast does not exist, checking whether a subdirectory named by the channel exists under a live broadcast directory, if not, creating a channel directory, moving a merge file from an idle file directory to the channel directory to rename the file at the current time, and naming rules: $ sequence _ $ unique _ live. merge, move two m3u8 index files to the current directory, one for storing the ts slice index and duration of the current merge file, the naming convention, $ sequence _ $ unique _ playback.m 3u8, one for storing the live ts slice index and duration, the naming convention, $ sequence _ $ unique _ live.m3u 8. The live m3u8 index file, the merge file with the earlier time of m3u8, and the review m3u8 file are renamed by the current time if there are subdirectories. Serial number ($ TS) of TS sliceseq) Initialize to 0, $ MEDIA-SEQUENCE is initialized to 0.
Step three, every TtsTime generates ts slice information, and the position of the ts slice information in the merge file is calculated firstly, and the calculation formula is as follows: TS (transport stream)loc＝TSseq*TSvol. And then sequentially writing the TS slice information into the TS of the merge filelocLocation, and generates TS slice filename, filename naming rule $ machien id $ merge filename $ TSlocLength of Media-SEQTs name, write this file name and duration to the live m3u8 index file, and update the MEDIA-SEQUENCE entry of the live m3u8 index file. Write back this filename and duration to the m3u8 index file. $ TSseqIncrease by 1, $ MEDIA-SEQUENCE by 1.
Step four, TM3u8At the time, adding Enlist to the currently processed m3u8 file indicates that the index file is ended, and closing the merge file and the m3u8 file. If there is another merge file and a review m3u8 file under the current directory, then both files are renamed at the current time. Subsequent ts slices will be written to the new merge file and slice information will be written to the new m3u8 file. If there is only one merge file and one review m3u8 file under the current directory, then one merge file and one review m3u8 file are moved from the free file directory and the two files are renamed at the current time. And setting the serial number of the Ts slice to be 0, and repeating the step two-third until the live broadcast is finished.
And step five, inquiring timeliness of merge files and m3u8 files under the channel directories at intervals of a certain time, and if the last modification time of the files is larger than a threshold value, recycling the related files under the channel directories.
And step six, inquiring the number of merge files and m3u8 files in the idle file directory at intervals, and if the number of merge files and m3u8 files in the idle file directory is less than a preset threshold, creating corresponding files.
In the embodiment of the present invention, for HLS live broadcast, the implementation process is as follows:
when the client requests for live broadcasting, the http server sends the index file information of the live m3u8 to the client, and when the client requests for a ts slice file in the live m3u8 file, the http server reads media data from a merge file according to the name of the ts file and sends the media data to the client, and the judgment process is as follows:
the method comprises the steps of obtaining the name of a merge file according to the name of the $ merge file, determining the position of the merge file according to the $ TSloc, reading the number of bytes with the length of $ and sending the number of bytes to a server side.
In the embodiment of the present invention, for the instant review request of the user, the implementation process is as follows:
instant review refers to the user wanting to see the first few seconds or minutes of video during the live broadcast. When the user requests the distance T from the current timeMAnd when reviewing within the time, the client sends a reviewing request to the current live broadcast server, the live broadcast server reads corresponding ts slice file names and durations according to the time period selected by the user and the starting time of two reviewing m3u8 index files in the channel directory, and the reviewing m3u8 file information in the corresponding time period is spliced and sent to the client.
The client requests ts files in the m3u8 index file, the http server reads media data from the merge file according to the ts file name and sends the media data to the client, and the judgment process is as follows:
obtaining the name of merge file according to $ merge file name, according to $ TSlocAnd determining the position of the terminal in the merge file, reading the byte number of the size of $ length, and sending the byte number to the server side.
In the embodiment of the present invention, for the on-demand request of the user, the implementation process is as follows:
and querying the database to obtain the MP4 file in the user request time period, and sending the MP4 file name and the start-stop time meeting the conditions to the client.
The client accesses the MP4 file, the http server obtains the MP4 file information according to the MP4 storage file name, the starting position and the file length and sends the MP4 file information to the client.
The embodiment of the invention adopts three layers of directories to manage and store the files, when the storage period is up, the files do not need to be deleted, only the date subdirectory under the corresponding channel directory needs to be renamed, and the original files in each subdirectory under the date are moved to the corresponding subdirectory under the idle file directory, thereby not only realizing the rapidness, but also reducing the fragments caused by the file deletion. In addition, in the live broadcasting process, the two preset media slices are recycled to merge the file storage media slices, so that the number of small files is reduced, the maintenance cost is reduced, and disk fragments cannot be introduced due to file deletion.
According to the estimated channel quantity, files with the corresponding number and size of the total capacity larger than the sum of the daily storage capacity of the channels are created; the number of idle files under each subdirectory is checked regularly at the later stage, and when the number of the idle files is determined to be smaller than a threshold value, corresponding files are created; the method not only ensures the preprocessing speed, but also can not cause performance overhead caused by the fact that files are not used enough to create the files by the write thread, and the size of the stored files is not fixed and can be dynamically adjusted according to the needs, so that the storage loss caused by file switching can be reduced.
In addition, the media storage in the embodiment of the invention is encapsulated by MP4, so that the storage overhead brought by TS headers is reduced, the review mode is determined according to the difference between the request time and the current time, and the video information of the latest 3s can be seen.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (10)
1. A method of file management in a monitoring system, the method comprising:
storing the monitoring file based on three layers of directories including a channel, a date and a file size;
when determining that a monitoring file with a preset size needs to be added, transferring the monitoring file from a subdirectory with the corresponding size of an idle directory to a corresponding subdirectory on the current date; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
two preset media slices are recycled in the live broadcast process, and the media slices are stored in a file merging file; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
2. The method of claim 1, further comprising:
creating files with corresponding number and size of total capacity larger than the sum of daily storage capacity of the channels according to the estimated number of the channels;
the file is pre-written \0' and moved to the corresponding subdirectory.
3. The method of claim 2, further comprising:
checking the number of idle files under each subdirectory regularly;
and when the number of the idle files is determined to be smaller than the threshold value, creating corresponding files and preprocessing the files.
4. The method of claim 1, wherein the on-demand process further comprises:
after the video in the preset period is packaged into the MP4, judging whether the residual storage capacity of the current file can be written into the MP4 media data;
if not, closing the current file, determining the sum of the MP4 capacity and the residual capacity of the current day, transferring the file with the corresponding size from the idle file directory to the corresponding subdirectory of the current channel, and writing the file; if so, the current file is written sequentially.
5. The method of claim 4, wherein after the sequentially writing the current file, the method further comprises:
determining whether the writing is on the same day as the starting time of the current MP 4;
if the current file is in the same day, determining the residual capacity of the current file, determining whether the residual capacity can be written into the media data of the next preset period according to the media capacity of the previous preset period, and if so, returning to the writing operation; if the time length of the media file which can be written is not the same as the time length of the media file which can be written, and the operation of closing the current file or returning to the writing operation is executed based on the time length;
if not, repeating the on-demand process when the first storage period is determined not to be expired; and if the second storage period is up, renaming the sub-directories of the dates under the channel directories, and completely moving the original files in the sub-directories under the dates to the corresponding sub-directories under the free file directories.
6. The method of claim 1, wherein the method further comprises, during the live broadcast:
inquiring timeliness of media slice merging files and index files under each channel directory at intervals of preset time;
and when the last modification time of the media slice merging file or the index file is determined to be larger than a preset threshold value, recovering the related files under the corresponding channel directories.
7. The method of claim 1, wherein the method further comprises, during the live broadcast:
inquiring the number of media slice merging files and index files in a live idle file directory at intervals of preset time;
and when the number of the media slice merging files or the index files is less than a preset threshold value, creating corresponding files.
8. A file management apparatus in a monitoring system, the apparatus comprising:
the storage module is used for storing the monitoring file based on three layers of directories including a channel, a date and a file size;
the processing module is used for transferring the monitoring file from the subdirectory with the corresponding size of the idle directory to the corresponding subdirectory on the current date when the monitoring file with the preset size needs to be added is determined; when the storage period is determined to be reached, renaming the date subdirectories under the corresponding channel directories, and moving original files in each subdirectory under the date to the corresponding subdirectory under the free file directory;
the live broadcast processing module is used for circularly using two preset media slices in the live broadcast process to merge the file storage media slices; one file is used for storing the media slices in the current preset period, and the other file is used for storing the media slices in the last preset period.
9. A file management apparatus in a monitoring system, the apparatus comprising: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is adapted to perform the steps of the method of any one of claims 1-7 when running the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811235338.XA CN111090617B (en) | 2018-10-23 | 2018-10-23 | File management method and device in monitoring system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811235338.XA CN111090617B (en) | 2018-10-23 | 2018-10-23 | File management method and device in monitoring system and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111090617A true CN111090617A (en) | 2020-05-01 |
| CN111090617B CN111090617B (en) | 2023-03-31 |
Family
ID=70391299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811235338.XA Active CN111090617B (en) | 2018-10-23 | 2018-10-23 | File management method and device in monitoring system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111090617B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117472297A (en) * | 2023-12-28 | 2024-01-30 | 深圳万物安全科技有限公司 | Method, device and storage medium for prolonging storage time of video file |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101539884A (en) * | 2009-04-30 | 2009-09-23 | 杭州华三通信技术有限公司 | Method and device for data storage |
| US7761497B1 (en) * | 2001-07-13 | 2010-07-20 | Vignette Software, LLC | Storage medium having a manageable file directory structure |
| CN105589662A (en) * | 2015-12-21 | 2016-05-18 | 上海华测导航技术股份有限公司 | GNSS receiver data storage and management method |
| CN107341227A (en) * | 2017-07-01 | 2017-11-10 | 中国科学院计算机网络信息中心 | Document handling method, server and computer-readable recording medium |
-
2018
- 2018-10-23 CN CN201811235338.XA patent/CN111090617B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7761497B1 (en) * | 2001-07-13 | 2010-07-20 | Vignette Software, LLC | Storage medium having a manageable file directory structure |
| CN101539884A (en) * | 2009-04-30 | 2009-09-23 | 杭州华三通信技术有限公司 | Method and device for data storage |
| CN105589662A (en) * | 2015-12-21 | 2016-05-18 | 上海华测导航技术股份有限公司 | GNSS receiver data storage and management method |
| CN107341227A (en) * | 2017-07-01 | 2017-11-10 | 中国科学院计算机网络信息中心 | Document handling method, server and computer-readable recording medium |
Non-Patent Citations (1)
| Title |
|---|
| 欧阳甸: "嵌入式数字视频监控系统中的媒体存储研究", 《电视技术》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117472297A (en) * | 2023-12-28 | 2024-01-30 | 深圳万物安全科技有限公司 | Method, device and storage medium for prolonging storage time of video file |
| CN117472297B (en) * | 2023-12-28 | 2024-04-16 | 深圳万物安全科技有限公司 | Method, device and storage medium for prolonging storage time of video file |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111090617B (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180260114A1 (en) | Predictive models of file access patterns by application and file type | |
| US20150356125A1 (en) | Method for data placement based on a file level operation | |
| CN111198856B (en) | File management method, device, computer equipment and storage medium | |
| CN100568958C (en) | A kind of based on network remote editing method for program | |
| EP3360333B1 (en) | Segmented video content storage | |
| US8060697B1 (en) | Dynamically allocated secondary browser cache | |
| CN107809684B (en) | Video segment generation method and device, cache server | |
| CN102902724A (en) | Mass raster tile map release method | |
| US20230362246A1 (en) | Systems, methods, and apparatuses for storage management | |
| CN107801051A (en) | Virtual sliced sheet information transferring method and device, video server | |
| CN113468199A (en) | Index updating method and system | |
| CN103152606B (en) | Video file processing method and device, system | |
| CN111090617B (en) | File management method and device in monitoring system and computer readable storage medium | |
| CN116166690A (en) | Mixed vector retrieval method and device for high concurrency scene | |
| CN105718569A (en) | Method, device and system for uploading compound document | |
| CN109062500B (en) | Metadata management server, data storage system and data storage method | |
| US20180203636A1 (en) | Likelihood of access based object storage in a cloud environment | |
| CN102104581A (en) | Network karaoke on-demand system and method thereof | |
| CN101848366B (en) | Filing and program broadcasting method and equipment of time shift slicing files in IPTV (Internet Protocol Television) system | |
| CN110737389A (en) | Method and device for storing data | |
| CN102722557A (en) | Self-adaption identification method for identical data blocks | |
| CN105677579A (en) | Data access method and system in cache system | |
| JP2023531751A (en) | Vehicle data storage method and system | |
| EP3293647B1 (en) | Methods of storing essence data in media file systems | |
| CN110430279B (en) | File downloading control method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |