CN113473039A - Multi-channel video storage method and system - Google Patents
Multi-channel video storage method and system Download PDFInfo
- Publication number
- CN113473039A CN113473039A CN202110677282.9A CN202110677282A CN113473039A CN 113473039 A CN113473039 A CN 113473039A CN 202110677282 A CN202110677282 A CN 202110677282A CN 113473039 A CN113473039 A CN 113473039A
- Authority
- CN
- China
- Prior art keywords
- shared memory
- video
- data
- channel
- writing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000012544 monitoring process Methods 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 11
- 230000005055 memory storage Effects 0.000 description 6
- 238000006467 substitution reaction Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/907—Television signal recording using static stores, e.g. storage tubes or semiconductor memories
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5011—Pool
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention provides a multi-channel video storage method and a multi-channel video storage system, which can be used for: for each video acquisition channel, receiving, splicing and caching video data transmitted by each video acquisition channel in real time; judging whether the data volume of each path of cached video data reaches D bytes in real time and correspondingly, respectively intercepting video data segments with the length of the front D bytes of each path of video data with the data volume reaching the D bytes, and writing the video data segments into a shared memory after marking a channel identifier; and calculating the size of the residual available space of the shared memory in real time, reading data in the shared memory based on the calculated size of the residual available space, addressing each read video data segment to the storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk. The invention is helpful to reduce the operation times of the hard disk, reduce the requirement on the storage space of the temporary memory and improve the rationality of the utilization of the hard disk space.
Description
Technical Field
The invention relates to the field of servers, in particular to a multi-channel video storage method and a multi-channel video storage system.
Background
With the development of networks, the processing scene of video information in the networks is more and more common. An increasing number of application scenarios with increasing bandwidth resources require a large number of highly concurrent video solutions. There are more and more application scenarios in which multiple videos are stored simultaneously.
At present, most of multi-channel video storage application scenes adopt multi-thread processing and are provided with a plurality of hard disks, each channel of video receives video data independently and then is stored in the corresponding hard disk, and the following problems often exist:
1. and each path of video is independently stored, namely, data hard disk read-write operation is carried out once when data sent by one path of front-end video acquisition equipment is received. The hard disk is easy to operate for too many times, and consumes more program running time.
2. Each video processing thread (used for video acquisition and receiving operations) often separately opens up a temporary memory storage space for video data, and when the number of video threads is too large, the requirement on the temporary memory storage space is often too large.
3. The hard disk for storing data cannot be systematically managed by reading and storing the data, so that the use progress of the hard disk is uneven, the situation that the space of a certain hard disk is exhausted and the use degree of other related hard disks is low is easy to occur, and the unreasonable use of the space of the hard disk is easy to cause.
Therefore, the invention provides a multi-channel video storage method and a multi-channel video storage system, which are used for solving the problems.
Disclosure of Invention
In view of the above disadvantages in the prior art, the present invention provides a method and a system for storing multiple channels of video, which are used to reduce the number of hard disk operations, reduce the requirement for the temporary memory storage space, and improve the rationality of the hard disk space utilization.
In a first aspect, the present invention provides a multi-channel video storage method, including the steps of:
for each video acquisition channel, receiving and splicing video data transmitted by each video acquisition channel in real time, and caching each spliced video data in real time;
judging whether the data volume of each path of cached video data reaches D bytes in real time and correspondingly, respectively intercepting a video data segment with the length of the front D bytes of each path of video data with the data volume reaching the D bytes, marking a channel identifier preset by a corresponding video acquisition channel, and writing the channel identifier and the marked channel identifier into a shared memory; d is the least common multiple of the capacity of the magnetic tracks of all the storage hard disks arranged in each video acquisition channel;
calculating the size of the residual available space of the shared memory in real time, reading data in the shared memory based on the calculated size of the residual available space, addressing each video data segment in the read data to a storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk; the size of the data read from the shared memory each time is n × (D + p) bytes of data, wherein D is the above D, p is the byte length of the channel identifier configured by each video acquisition channel, and the value of n is a positive integer.
Furthermore, the shared memory adopts a closed-loop data structure, and the initial read-write positions of the shared memory are the same;
after writing a video data segment and a marked channel identifier thereof in the shared memory, updating the next writing position of the shared memory to the current writing initial position of the shared memory for writing the next video data segment to be written and the marked channel identifier thereof;
in the process of writing each video data segment and the marked channel identifier into the shared memory, when a closed-loop connection point is encountered, the closed-loop connection point is written first, then the initial position of the shared memory is used as the next writing position of the shared memory, and then data is continuously written.
Further, in each process of reading data from the shared memory: when meeting the closed loop connection point of the shared memory, the data on the closed loop connection point is read first, and then the initial position of the shared memory is jumped to continue reading.
Further, the multi-channel video storage method further comprises the following steps:
monitoring the read-write speed of the shared memory in real time;
and controlling the number of threads for reading the data in the shared memory in real time according to the monitored read-write speed of the shared memory.
Further, the number of threads for reading data in the shared memory is controlled in real time according to the monitored read-write speed of the shared memory, and the specific implementation method is as follows:
judging whether the current reading speed of the shared memory is greater than the writing speed in real time;
reducing a thread for reading data in the shared memory when the current reading speed of the shared memory is judged to be greater than the writing speed each time;
and each time when the current reading speed of the shared memory is judged to be lower than the writing speed, respectively adding a thread for reading the data in the shared memory.
In a second aspect, the present invention provides a multi-channel video storage system, comprising:
sharing the memory;
the video receiving thread pool is used for receiving and splicing video data transmitted by each video acquisition channel in real time aiming at each video acquisition channel and caching the spliced video data in real time; the video acquisition device is used for judging whether the data volume of each path of cached video data reaches D bytes in real time and correspondingly, respectively intercepting a video data segment with the length of the front D bytes of each path of video data with the data volume reaching the D bytes, marking a channel identifier preset by a corresponding video acquisition channel, and then writing the channel identifier and the marked channel identifier into a shared memory; d is the least common multiple of the capacity of the magnetic tracks of all the storage hard disks arranged in each video acquisition channel;
the video storage thread pool is used for calculating the size of the residual available space of the shared memory in real time, reading data in the shared memory based on the calculated size of the residual available space, addressing each video data segment in the read data to the storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk; the size of the data read from the shared memory each time is n × (D + p) bytes of data, wherein D is the above D, p is the byte length of the channel identifier configured by each video acquisition channel, and the value of n is a positive integer.
Furthermore, the shared memory adopts a closed-loop data structure, and the initial read-write positions of the shared memory are the same;
after writing a video data segment and a marked channel identifier thereof in the shared memory, the video receiving thread pool updates the next writing position of the shared memory to the current writing initial position of the shared memory for writing the next video data segment to be written and the marked channel identifier thereof;
in the process of writing each video data segment and the marked channel identifier into the shared memory, the video receiving thread pool firstly writes the closed loop connection point each time when meeting the closed loop connection point, then uses the initial position of the shared memory as the next writing position of the shared memory, and then continuously writes data.
Further, in each process of reading data from the shared memory, the video storage thread pool: when meeting the closed loop connection point of the shared memory, the data on the closed loop connection point is read first, and then the initial position of the shared memory is jumped to continue reading.
Further, the multi-channel video storage system further comprises:
and the monitoring module is used for monitoring the read-write speed of the shared memory in real time and controlling the number of threads for reading data in the shared memory in real time according to the monitored read-write speed of the shared memory.
Further, the monitoring module includes:
the monitoring unit is used for monitoring the read-write speed of the shared memory in real time;
the first processing unit is used for reducing threads used for reading data in the shared memory in a video storage thread pool when the monitoring unit judges that the current reading speed of the shared memory is greater than the writing speed each time;
and the second processing unit is used for adding a thread used for reading data in the shared memory in the video storage thread pool when the monitoring unit judges that the current reading speed of the shared memory is less than the writing speed each time.
The beneficial effect of the invention is that,
(1) the multi-channel video storage method and the multi-channel video storage system provided by the invention have the advantages that the temporary memory space in the background technology is replaced by the shared memory, the temporary memory storage space of the video data is avoided being independently opened for each video processing thread, and the problem that the memory space is excessively required in the multi-channel video storage process is solved to a certain extent.
(2) The multi-channel video storage method and the multi-channel video storage system provided by the invention can splice and cache the video data transmitted by each video acquisition channel, sequentially intercept the video data segments with the length of D bytes when the cached data amount reaches D bytes, write the video data segments into the shared memory after marking the corresponding channel identifiers, and the data read from the shared memory each time is the data with the size of n x (D + p) bytes, and can address the storage hard disk according to the marked channel identifiers of each video data segment in the read data and then write the video data segments into the corresponding storage hard disk, thereby not only avoiding the data read-write operation of the data hard disk when receiving the data transmitted by the front-end video acquisition equipment each time, but also matching the size of the video data segments in the data read from the shared memory each time with the size of the magnetic track capacity of the corresponding storage hard disk, the invention is beneficial to reducing the track addressing times when the hard disk file is written and improving the speed of writing the video data into the hard disk to a certain extent.
(3) The multi-channel video storage method and the multi-channel video storage system provided by the invention have the advantages that each video data segment in the data read from the shared memory at each time can be addressed according to the marked channel identifier, and then written into the storage hard disk found by addressing correspondingly, so that the situation that the space of a certain hard disk is exhausted due to the fact that each channel of video data is stored in a certain storage hard disk in a centralized manner to be unfavorable for realizing the reasonable utilization of the space of the hard disk is avoided to a certain extent, and the invention is also favorable for realizing the reasonable utilization of the space of the hard disk to a certain extent.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a method of one embodiment of the invention.
FIG. 2 is a schematic block diagram of a system of one embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of a multi-channel video storage method according to an embodiment of the present invention.
The multi-channel video storage method is based on a shared memory and is used for storing video data transmitted by a plurality of video acquisition channels. The multi-channel video acquisition channels are at least two video acquisition channels. Each video acquisition channel is respectively and independently provided with a storage hard disk and a channel identifier in advance. The channel identifier is used for uniquely identifying the video acquisition channel. The video acquisition channel is associated with the storage hard disk matched with the video acquisition channel through the channel identification matched with the video acquisition channel. The storage hard disk is addressed through the channel identifier, and one storage hard disk can be uniquely found out.
For the convenience of implementation, the length of the bytes of each channel identifier involved in this embodiment is equal and is Q bytes (i.e., Q bytes).
The shared memory adopts a closed-loop data structure, is the only temporary memory storage space used in the invention, and avoids the use of a plurality of temporary memory storage spaces.
The shared memory has a start position and an end position of the memory. The end position of the shared memory is the closed loop connection point of the shared memory. The initial position of the shared memory is defaulted to a write initial position of the shared memory.
As shown in fig. 1, the method 100 includes:
and step 110, receiving and splicing the video data transmitted by the video acquisition channels in real time for each video acquisition channel, and caching the spliced video data in real time.
For example, the video acquisition channels involved in the present invention have two paths, and each of the two paths of video acquisition channels (hereinafter referred to as "video acquisition channel i") performs the following operations:
step 1101, receiving video data transmitted from a video acquisition channel i in real time;
step 1102, splicing the video data received in the step 1101 in real time;
and 1103, caching the spliced video data in the step 1102 in real time.
And 120, correspondingly judging whether the data volume of each path of cached video data reaches D bytes in real time, respectively intercepting a video data segment with the length of the previous D bytes for each path of video data with the data volume reaching the D bytes, marking a channel identifier preset by a corresponding video acquisition channel, and writing the channel identifier and the marked channel identifier into the shared memory. D is the least common multiple of the capacity (unit: byte) of the magnetic tracks of all the storage hard disks allocated to each video acquisition channel.
For example, if a video data segment a is currently intercepted, and the video data segment a corresponds to the video acquisition channel i, for the video data segment a, after a channel identifier provided by the corresponding video acquisition channel is marked, the channel identifier and the marked channel identifier are written into the shared memory together, specifically: marking a channel identifier configured by the video acquisition channel i for the video data segment A, and then writing the video data segment A and the marked channel identifier into the shared memory.
The video data segment with the length of D bytes before being cut is the video data segment (continuous video data) with the length of D bytes after being cut from the starting position of the video data after being spliced correspondingly.
After writing a video data segment and the marked channel identifier thereof in the shared memory, updating the next writing position of the shared memory to the current writing initial position of the shared memory, for writing the next video data segment to be written and the marked channel identifier thereof.
In this embodiment, the initial read/write locations of the shared memory are the same, i.e., the write start location and the read start location of the shared memory are the same. For example, the shared memory is 100M continuous memory char buf [1024 × 1024 × 100], the start position of the shared memory is buf [0], the write start position and the read start position of the shared memory are both the start position buf [0] of the shared memory by default, and the closed loop connection point of the shared memory is the end position buf [1024 × 1024 × 100] of the shared memory.
When a plurality of video data segments need to be written into the shared memory at the same time, the video data segments can be written one by one. Writing each video data segment is: writing the video data segment into the shared memory, and then continuing to write the channel identifier marked by the video data segment into the shared memory.
For example, a video data segment I and a video data segment II need to be written, a channel identifier marked by the video data segment I is a first channel identifier, a channel identifier marked by the video data segment II is a second channel identifier, the data lengths of the video data segment I and the video data segment II are both 1500 bytes, and the byte lengths of the first channel identifier and the second channel identifier are both 1 byte. When the first video data segment and the second video data segment are written into the shared memory, the first video data segment and the marked channel identifier thereof can be written into the shared memory, and then the second video data segment and the marked channel identifier thereof can be written into the shared memory. Specifically, taking the above 100M shared memory as an example, and taking the shared memory as an example before writing the first video data segment and the second video data segment, the process of writing the first video data segment and the second video data segment into the shared memory is as follows:
step one, acquiring a current write-in initial position buf [0] of the shared memory, and writing the video data segment I and a channel identifier marked by the video data segment I (namely a first channel identifier) into the shared memory.
During implementation, the first video data segment is written into the shared memory, and then the first channel identifier is written into the shared memory. Specifically, the first segment of video data is first written into the storage location buf [0] -buf [1499] of the shared memory, and then the first channel identifier is written into the storage location buf [1500] of the shared memory.
And then executing the step two.
And step two, updating the next writing position buf [1501] of the shared memory to be the current writing starting position of the shared memory, and continuously writing the video data segment two and the channel identifier (namely the second channel identifier) of the video data segment two.
And then continue writing video data segment two.
When writing the second video data segment, the second video data segment can be written into the shared memory first by referring to the first video data segment, and then the second channel identifier is written into the shared memory. Specifically, the second video data segment is first written into the storage location buf [1501] -buf [3000] of the shared memory, and then the second channel identifier is written into the storage location buf [3001] of the shared memory.
And then executing the third step.
And step three, updating the next writing position buf [3002] of the shared memory to be the current writing initial position of the shared memory for subsequent continuous writing of other video data segments.
When other video data segments need to be written into the shared memory, the writing step of the video data segment I and/or the video data segment II can be referred to for writing.
During the process of writing each video data segment and the channel identifier marked by the video data segment into the shared memory, if the tail position buf [1024 × 1024 × 100] of the shared memory is encountered, the closed-loop connection point buf [1024 × 1024 × 100] is written fully, and then the initial position buf [0] of the shared memory is used as the next writing position of the shared memory to continue writing data.
In a specific embodiment, p is Q, i.e., p ═ Q.
For the convenience of implementation, in this embodiment, n × (d + p) ≦ C-m, where C is the size of the available memory of the shared memory, and m is the predetermined threshold of the remaining available space of the shared memory.
In a specific implementation, the threshold of the remaining available space of the shared memory may be set by a person skilled in the art according to actual situations, for example, the threshold may be set to 80% of the available space of the shared memory.
The value of n is a constant, and can be specifically set by those skilled in the art according to actual situations.
In step 130, the size of the data read from the shared memory is n × (d + p) bytes. It can be seen that in the method 100, the size of the video data read from the shared memory each time is an integral multiple of the storage space of the hard disk magnetic track of the storage hard disk, which is helpful for matching the size of the video data segment written into the hard disk each time with the size of the hard disk magnetic track, and thus reduces the magnetic track addressing times when the hard disk file is written, and is helpful for increasing the speed of writing the video data into the hard disk.
Wherein, in the process of reading out data from the shared memory each time: when meeting the closed loop connection point of the shared memory, the data on the closed loop connection point is read first, and then the initial position of the shared memory is jumped to continue reading.
As an exemplary embodiment of the present invention, the implementation method of step 130 is:
step 1301: polling writable data in the shared memory in real time;
step 1302: calculating the size of the residual available space of the shared memory based on the writable data;
step 1303: comparing the size of the residual available space with the size relation of a preset shared memory residual available space critical value;
step 1304: and when the comparison result is that the size of the remaining available space is less than or equal to (namely less than or equal to) the preset critical value of the remaining available space of the shared memory, reading the data in the shared memory, addressing each video data segment in the read data to the storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk.
When data is read out from the shared memory for the first time, reading is started from the write starting position of the shared memory. And the direction of reading data from the shared memory to the outside is consistent with the writing direction of writing data into the shared memory.
The predetermined threshold of the remaining available space of the shared memory may be set by a person skilled in the art according to actual situations. In this embodiment, the predetermined threshold of the remaining available space of the shared memory is 20% of the size of the memory space of the shared memory.
As an exemplary embodiment of the present invention, the implementation method of the above step 1301 is:
acquiring the latest reading initial position and the latest writing initial position of the shared memory in real time;
and correspondingly calculating the size of the readable space between the latest read starting position and the latest write starting position of the acquired shared memory.
If the data in the shared memory is less than the read amount once, it indicates that there is no data available for writing to the storage hard disk in the shared memory.
As an exemplary embodiment of the present invention, the method 100 further comprises the steps of:
monitoring the read-write speed of the shared memory in real time;
and controlling the number of threads for reading the data in the shared memory in real time according to the monitored read-write speed of the shared memory.
Specifically, the number of threads for reading data in the shared memory is controlled in real time according to the monitored read-write speed of the shared memory, and the specific implementation method includes:
judging whether the current reading speed of the shared memory is greater than the writing speed in real time;
when the current reading speed of the shared memory is judged to be higher than the writing speed, reducing a thread for reading data in the shared memory respectively;
and each time when the current reading speed of the shared memory is judged to be lower than the writing speed, respectively adding a thread for reading the data in the shared memory.
The present invention also provides a multi-channel video storage system, as shown in fig. 2. The multi-channel video storage system is used for storing the video data transmitted by the multi-channel video acquisition channels. Each video acquisition channel is respectively and independently provided with a storage hard disk and a channel identifier in advance. The channel identifier is used for uniquely identifying the video acquisition channel. The video acquisition channel can be associated with the storage hard disk matched with the video acquisition channel through the channel identification matched with the video acquisition channel. The storage hard disk is addressed through the channel identifier, and one storage hard disk can be uniquely found out. Each video acquisition channel can adopt a video camera or a camera. As shown in fig. 2, the system 200 includes a shared memory 201, a video receive thread pool 202, and a video store thread pool 203. Wherein:
video reception thread pool 202: the video data processing device is used for receiving and splicing the video data transmitted by the video acquisition channels in real time for each video acquisition channel and caching the spliced video data in real time; the video data acquisition module is used for judging whether the data volume of each path of cached video data reaches D bytes (namely D bytes) in real time and correspondingly, respectively intercepting a video data segment with the length of the previous D bytes for each path of video data with the data volume reaching the D bytes, marking a channel identifier preset by a corresponding video acquisition channel, and then writing the channel identifier and the marked channel identifier into the shared memory 201; d is the least common multiple of the capacity of the magnetic tracks of all the storage hard disks arranged in each video acquisition channel;
the video storage thread pool 203 is used for calculating the size of the remaining available space of the shared memory 201 in real time, reading the data in the shared memory 201 each time when the calculated size of the remaining available space is less than or equal to a preset critical value of the remaining available space of the shared memory 201, addressing each video data segment in the read data to a storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk; the size of the data read from the shared memory 201 each time is n × (D + p) bytes of data, where D is the above D, p is the byte length of the channel identifier configured for each video acquisition channel, and the value of n is a positive integer.
The shared memory 201 has a closed-loop data structure and the initial read/write positions are the same.
After writing a video data segment and a channel identifier marked by the video data segment into the shared memory 201, the video receiving thread pool 202 updates the next writing position of the shared memory 201 to the current writing start position of the shared memory 201, so as to write the next video data segment to be written and the channel identifier marked by the video data segment.
In the process of writing each video data segment and the marked channel identifier thereof into the shared memory 201, the video receiving thread pool 202 finishes writing the closed loop connection point each time the closed loop connection point is encountered, then uses the initial position of the shared memory 201 as the next writing position of the shared memory 201, and then continues to write data.
As an exemplary embodiment of the present invention, the video storage thread pool 203, during each reading of data from the shared memory 201:
each time a closed loop connection point of the shared memory 201 is encountered, data on the closed loop connection point is read first, and then the read is continued by jumping to the initial position of the shared memory 201.
Further, as an exemplary embodiment of the present invention, the system 200 further comprises:
and the monitoring module is used for monitoring the read-write speed of the shared memory 201 in real time and controlling the number of threads for reading data in the shared memory 201 in real time according to the monitored read-write speed of the shared memory 201.
Further, as an exemplary embodiment of the present invention, the monitoring module includes:
the monitoring unit is used for monitoring the read-write speed of the shared memory 201 in real time;
the first processing unit is configured to respectively reduce threads used for reading data in the shared memory 201 in one video storage thread pool 203 when the monitoring unit determines that the current reading speed of the shared memory 201 is greater than the writing speed each time;
and a second processing unit, configured to add a thread in the video storage thread pool 203 for reading data in the shared memory 201 each time the monitoring unit determines that the current reading speed of the shared memory 201 is less than the writing speed.
The same and similar parts in the various embodiments in this specification may be referred to each other. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the description in the method embodiment.
Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method for multi-channel video storage, comprising the steps of:
for each video acquisition channel, receiving and splicing video data transmitted by each video acquisition channel in real time, and caching each spliced video data in real time;
judging whether the data volume of each path of cached video data reaches D bytes in real time and correspondingly, respectively intercepting a video data segment with the length of the front D bytes of each path of video data with the data volume reaching the D bytes, marking a channel identifier preset by a corresponding video acquisition channel, and writing the channel identifier and the marked channel identifier into a shared memory; d is the least common multiple of the capacity of the magnetic tracks of all the storage hard disks arranged in each video acquisition channel;
calculating the size of the residual available space of the shared memory in real time, reading data in the shared memory based on the calculated size of the residual available space, addressing each video data segment in the read data to a storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk; the size of the data read from the shared memory each time is n × (D + p) bytes of data, wherein D is the above D, p is the byte length of the channel identifier configured by each video acquisition channel, and the value of n is a positive integer.
2. The multi-channel video storage method according to claim 1, wherein the shared memory adopts a closed-loop data structure, and the initial read-write positions of the shared memory are the same;
after writing a video data segment and a marked channel identifier thereof in the shared memory, updating the next writing position of the shared memory to the current writing initial position of the shared memory for writing the next video data segment to be written and the marked channel identifier thereof;
in the process of writing each video data segment and the marked channel identifier into the shared memory, when a closed-loop connection point is encountered, the closed-loop connection point is written first, then the initial position of the shared memory is used as the next writing position of the shared memory, and then data is continuously written.
3. The multi-channel video storage method according to claim 1, wherein during each reading of data from the shared memory: when meeting the closed loop connection point of the shared memory, the data on the closed loop connection point is read first, and then the initial position of the shared memory is jumped to continue reading.
4. The multi-channel video storage method according to claim 1, further comprising the steps of:
monitoring the read-write speed of the shared memory in real time;
and controlling the number of threads for reading the data in the shared memory in real time according to the monitored read-write speed of the shared memory.
5. The multi-channel video storage method according to claim 4, wherein the number of threads for reading data in the shared memory is controlled in real time according to the monitored read-write speed of the shared memory, and the specific implementation method is as follows:
judging whether the current reading speed of the shared memory is greater than the writing speed in real time;
reducing a thread for reading data in the shared memory when the current reading speed of the shared memory is judged to be greater than the writing speed each time;
and each time when the current reading speed of the shared memory is judged to be lower than the writing speed, respectively adding a thread for reading the data in the shared memory.
6. A multi-channel video storage system, comprising:
sharing the memory;
the video receiving thread pool is used for receiving and splicing video data transmitted by each video acquisition channel in real time aiming at each video acquisition channel and caching the spliced video data in real time; the video data acquisition module is used for correspondingly judging whether the data volume of each path of cached video data reaches D bytes in real time, respectively intercepting a video data segment with the length of the front D bytes of each path of video data with the data volume reaching the D bytes, marking a channel identifier preset by a corresponding video acquisition channel, and then writing the channel identifier and the marked channel identifier into the shared memory; d is the least common multiple of the capacity of the magnetic tracks of all the storage hard disks arranged in each video acquisition channel;
the video storage thread pool is used for calculating the size of the residual available space of the shared memory in real time, reading data in the shared memory based on the calculated size of the residual available space, addressing each video data segment in the read data to the storage hard disk according to the marked channel identifier, and then writing the video data segment into the corresponding storage hard disk; the size of the data read from the shared memory each time is n × (D + p) bytes of data, wherein D is the above D, p is the byte length of the channel identifier configured by each video acquisition channel, and the value of n is a positive integer.
7. The multi-channel video storage system of claim 6, wherein the shared memory has a closed-loop data structure, and the initial read-write positions of the shared memory are the same;
after writing a video data segment and a marked channel identifier thereof in the shared memory, the video receiving thread pool updates the next writing position of the shared memory to the current writing initial position of the shared memory for writing the next video data segment to be written and the marked channel identifier thereof;
in the process of writing each video data segment and the marked channel identifier into the shared memory, the video receiving thread pool firstly writes the closed loop connection point each time when meeting the closed loop connection point, then uses the initial position of the shared memory as the next writing position of the shared memory, and then continuously writes data.
8. The multi-channel video storage system of claim 6, wherein the video storage thread pool, during each reading of data from the shared memory:
when meeting the closed loop connection point of the shared memory, the data on the closed loop connection point is read first, and then the initial position of the shared memory is jumped to continue reading.
9. The multi-channel video storage system of claim 6, further comprising:
and the monitoring module is used for monitoring the read-write speed of the shared memory in real time and controlling the number of threads for reading data in the shared memory in real time according to the monitored read-write speed of the shared memory.
10. The multi-channel video storage system of claim 9, wherein the monitor module comprises:
the monitoring unit is used for monitoring the read-write speed of the shared memory in real time;
the first processing unit is used for reducing threads used for reading data in the shared memory in a video storage thread pool when the monitoring unit judges that the current reading speed of the shared memory is greater than the writing speed each time;
and the second processing unit is used for adding a thread used for reading data in the shared memory in the video storage thread pool when the monitoring unit judges that the current reading speed of the shared memory is less than the writing speed each time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110677282.9A CN113473039B (en) | 2021-06-18 | 2021-06-18 | Multi-channel video storage method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110677282.9A CN113473039B (en) | 2021-06-18 | 2021-06-18 | Multi-channel video storage method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113473039A true CN113473039A (en) | 2021-10-01 |
| CN113473039B CN113473039B (en) | 2022-12-02 |
Family
ID=77870438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110677282.9A Active CN113473039B (en) | 2021-06-18 | 2021-06-18 | Multi-channel video storage method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113473039B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114185493A (en) * | 2022-02-15 | 2022-03-15 | 烟台市地理信息中心 | Method for dividing and storing surveying and mapping data |
| CN115484139A (en) * | 2022-09-02 | 2022-12-16 | 武汉众智数字技术有限公司 | Video strategy management decentralized method based on video network monitoring |
| CN116798144A (en) * | 2023-04-18 | 2023-09-22 | 润芯微科技(江苏)有限公司 | Collision video storage method, system, device and computer readable storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1859570A (en) * | 2005-09-07 | 2006-11-08 | 深圳市海思半导体有限公司 | Loading device and method for moving compensating data |
| CN103024363A (en) * | 2013-01-10 | 2013-04-03 | 深圳市长江力伟股份有限公司 | Method, system and electronic device for stitching multiple channels of videos |
| CN103442204A (en) * | 2013-08-08 | 2013-12-11 | 浙江工业大学 | Network video transmission system and method based on DM365 |
| CN104750858A (en) * | 2015-04-16 | 2015-07-01 | 成都影泰科技有限公司 | Network-based data storage method |
| CN104811646A (en) * | 2015-05-15 | 2015-07-29 | 电子科技大学 | Multi-video stream data concurrent modulation and buffering storage method based on continuous storage model |
| CN105306865A (en) * | 2015-10-29 | 2016-02-03 | 南京秦杜明视信息技术有限公司 | Video monitoring data system |
| CN107247561A (en) * | 2017-05-31 | 2017-10-13 | 成都华立达电力信息系统有限公司 | Buffer pool circulation storage reading/writing method |
| CN109033353A (en) * | 2018-07-25 | 2018-12-18 | 天津凯发电气股份有限公司 | A kind of Power quality management system shared drive update method |
| CN110620950A (en) * | 2019-10-10 | 2019-12-27 | 东软集团股份有限公司 | Method, device and equipment for storing audio and video files |
| CN112099970A (en) * | 2020-09-04 | 2020-12-18 | 中国第一汽车股份有限公司 | Video data processing method, device, equipment and storage medium |
-
2021
- 2021-06-18 CN CN202110677282.9A patent/CN113473039B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1859570A (en) * | 2005-09-07 | 2006-11-08 | 深圳市海思半导体有限公司 | Loading device and method for moving compensating data |
| CN103024363A (en) * | 2013-01-10 | 2013-04-03 | 深圳市长江力伟股份有限公司 | Method, system and electronic device for stitching multiple channels of videos |
| CN103442204A (en) * | 2013-08-08 | 2013-12-11 | 浙江工业大学 | Network video transmission system and method based on DM365 |
| CN104750858A (en) * | 2015-04-16 | 2015-07-01 | 成都影泰科技有限公司 | Network-based data storage method |
| CN104811646A (en) * | 2015-05-15 | 2015-07-29 | 电子科技大学 | Multi-video stream data concurrent modulation and buffering storage method based on continuous storage model |
| CN105306865A (en) * | 2015-10-29 | 2016-02-03 | 南京秦杜明视信息技术有限公司 | Video monitoring data system |
| CN107247561A (en) * | 2017-05-31 | 2017-10-13 | 成都华立达电力信息系统有限公司 | Buffer pool circulation storage reading/writing method |
| CN109033353A (en) * | 2018-07-25 | 2018-12-18 | 天津凯发电气股份有限公司 | A kind of Power quality management system shared drive update method |
| CN110620950A (en) * | 2019-10-10 | 2019-12-27 | 东软集团股份有限公司 | Method, device and equipment for storing audio and video files |
| CN112099970A (en) * | 2020-09-04 | 2020-12-18 | 中国第一汽车股份有限公司 | Video data processing method, device, equipment and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114185493A (en) * | 2022-02-15 | 2022-03-15 | 烟台市地理信息中心 | Method for dividing and storing surveying and mapping data |
| CN115484139A (en) * | 2022-09-02 | 2022-12-16 | 武汉众智数字技术有限公司 | Video strategy management decentralized method based on video network monitoring |
| CN115484139B (en) * | 2022-09-02 | 2024-03-15 | 武汉众智数字技术有限公司 | Video strategy management decentralization method based on video network monitoring |
| CN116798144A (en) * | 2023-04-18 | 2023-09-22 | 润芯微科技(江苏)有限公司 | Collision video storage method, system, device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113473039B (en) | 2022-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113473039B (en) | Multi-channel video storage method and system | |
| US8751737B2 (en) | Method and apparatus for using a shared ring buffer to provide thread synchronization in a multi-core processor system | |
| US20050086386A1 (en) | Shared running-buffer-based caching system | |
| CN110555001B (en) | Data processing method, device, terminal and medium | |
| CN109246182B (en) | Distributed lock manager and implementation method thereof | |
| CN112954244B (en) | Method, device, equipment and storage medium for realizing storage of monitoring video | |
| EP1998246A2 (en) | Management of internal operations by a storage device | |
| WO2017050064A1 (en) | Memory management method and device for shared memory database | |
| CN111416825A (en) | Inter-thread lock-free log management method and system, terminal and storage medium | |
| CN107209716A (en) | Memory management apparatus and method | |
| CN112433881A (en) | Data recovery method and device for distributed storage system | |
| CN115292266A (en) | High-reliability log storage method based on memory | |
| US8312239B2 (en) | Cache management for increasing performance of high-availability multi-core systems | |
| CN111061690A (en) | RAC-based database log file reading method and device | |
| EP4174675A1 (en) | On-board data storage method and system | |
| CN115794366A (en) | Memory prefetching method and device | |
| US20160037119A1 (en) | Video recording apparatus, and video recording method when temporary network failure occurs | |
| CN111399753B (en) | Method and device for writing pictures | |
| CN110022341A (en) | A kind of data transmission method and relevant device | |
| CN108153486A (en) | The running parameter guard method of embedded system and relevant device | |
| CN111666339B (en) | Multithreading data synchronization method | |
| CN110716923B (en) | Data processing method, data processing device, node equipment and storage medium | |
| CN109376135B (en) | Cluster file system management method and system | |
| CN111625502A (en) | Data reading method and device, storage medium and electronic device | |
| CN111405313A (en) | Method and system for storing streaming media data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |