CN108124112A - A kind of performance improvement method of multichannel stored stream media data - Google Patents
A kind of performance improvement method of multichannel stored stream media data Download PDFInfo
- Publication number
- CN108124112A CN108124112A CN201711457166.6A CN201711457166A CN108124112A CN 108124112 A CN108124112 A CN 108124112A CN 201711457166 A CN201711457166 A CN 201711457166A CN 108124112 A CN108124112 A CN 108124112A
- Authority
- CN
- China
- Prior art keywords
- file
- stream media
- data
- memory space
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/78—Television signal recording using magnetic recording
- H04N5/781—Television signal recording using magnetic recording on disks or drums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present invention provides a kind of performance improvement method of multichannel stored stream media data, when storage device initializes, removes the original fragment of storage device, and calculates the size and number of each video file according to memory capacity and the video time set and code check；Then it is to remove distribution memory space per the files in stream media for creating respective numbers all the way, and according to the size of each video file；When memory space is full an oldest files in stream media is directly selected to rename and carry out cycling covering；When writing data, in the cache pool for first copying application management to, and create a background thread and be responsible on suitable opportunity, storage device is directly write back directly around the caching of operating system with direct io modes, then improves performance by way of polishing 0 and modification file size.
Description
Technical field
The present invention relates to multi-way stream media storage field, more particularly to a kind of performance improvement of multichannel stored stream media data
Method.
Background technology
Existing multi-way stream media storage is all to store storage medium by the file interface of operating system offer, often
A file usually only stores the stream medium data of set time (adjustable), and oldest file can be deleted to release after having expired in space
Memory space is put, this way has the disadvantage that：
1. since multipath concurrence writes, write-in every time will distribute memory space, also have led to concurrently distributing storage sky
Between, and then file fragmentation is brought, and file fragmentation can reduce performance；
2. need to delete old file after memory space has been expired and carry out Free up Memory, which results in delete and write-in may
Can concurrently occur, file fragmentation can be caused, cause hydraulic performance decline；
3. concurrently when distribution and release disk space, in order to ensure data consistency, it is necessary to do mutual exclusion, mutual exclusion is not
It controllably (can not determine that current mutual exclusion is which is first carried out) and result in the uncontrollable of performance；
4. when hydraulic performance decline, the caching occupancy of file system can increase, and mean cost under embedded platform
Increase；
5. the file cache that stream medium data can be Jing Guo operating system, due to operating system only in the case where meeting specified conditions
It can go to be written back on memory, and this write-back mechanism differs and surely meets the Streaming Media requirement of multipath concurrence；
6. operating system when write-back buffer, has two class data:(mainly distribution storage is empty for file system metadata
Between when the index or bitmap that generate, such as bitmap in the fat tables in fat file system and ntfs file system) and stream matchmaker
Volume data, this usual position of two classes data from it is far, both data cross write in the case of, many storage devices are all
Being unable to reach optimum performance, (hard disk needs to seek expense；Flash classes are stored with the expense of mapping and garbage reclamation.
As disclosed in 2014-04-02, the Chinese invention of Publication No. CN103702057A discloses one kind and is adapted to multichannel
The block storage algorithm applicable of stream medium data is concurrently write, this method bypasses file system, so final files in stream media
It can only could be accessed by the software that it is specially developed, it is all inconvenient to check, play and distribute, poor compatibility；In addition this side
Method must divide the region of a fixed size on a storage device, and this block region can only record a video special, be not used to it
His purposes.
Disclosed in 2014-11-19, it is same that the Chinese invention of Publication No. CN104159058A discloses a kind of multipath media stream
When the high efficiency method recorded a video, be equally because around file system so as to cause poor compatibility.
For another example disclosed in 2010-06-16, the Chinese invention of Publication No. CN101742263A_ discloses a kind of storage and regards
The method of frequency monitoring video data, the data format of file is dedicated, also wants dedicated software that could access, incompatible
Tripartite applies；In addition the video time of each file of this method is fixed that can not switch；Last this does not account for multichannel
Concurrent situation and the controllable write-back of caching.
The content of the invention
The technical problem to be solved in the present invention is to provide a kind of performance improvement method of multichannel stored stream media data,
Mainly by avoiding the generation of file fragmentation and controlling the opportunity of write-back come improving performance, compatibility height.
What the present invention was realized in：A kind of performance improvement method of multichannel stored stream media data, it is characterised in that：Bag
Renaming after including storage device initial format flow, memory space completely covers flow and writes the process flow of data：
The storage device initial format flow is：Remove the original fragment of storage device, and according to memory capacity and
The video time and code check of setting calculate the size fsize of each video file and quantity fnum；Then created all the way to be every
Gained fnum files in stream media of quantity is calculated, each video file of each file after creating according to obtained by calculating
Size fsize removes distribution memory space；
Renaming after the memory space is full covers flow：When memory space is full, do not go to delete old stream matchmaker
Body file, as soon as but an oldest files in stream media is directly selected to rename, new stream medium data be written to rename it is later
File accomplishes Xun Huan covering；
Writing the process flow of data is：When writing data, in the cache pool for first copying application management to, and one is created
Background thread was responsible on suitable opportunity, and directly writing back storage directly around the caching of operating system with direct-io modes sets
It is standby, since the data of direct-io need to align, so the data of end-of-file can use 0 polishing, it can finally change file size
(directly changing file system metadata, performance cost very little), but do not include the 0 of polishing.
Further, the formula of the size fsize and quantity fnum for calculating each video file is as follows：
Fsize=b*t；
Fnum=c/fsize.
In formula, b is the average bit rate of estimating of the road Streaming Media, and t is video time, and c is allocated to the memory space on the road,
Fnum is the number of the road files in stream media.
Further, the control method on the suitable opportunity is：
First carry out write-back by a default time cycle t_interval, if write back data takes
t_write>For t_interval it is necessary to modification time cycle t_interval=t_write/p, wherein p is one
Constant；
If unit exception power down, the data in cache pool can lose, then by maximum allowable in upper strata setting one
The data time t_loss that loses controlled, if t_loss>0, then it can reset t_interval=min (t_loss, t_
interval)。
Further, present invention additionally comprises modification video time flows；
It is described modification video time flow be：If necessary to change video time, then oldest files in stream media is selected to close
And or split, merge and only change file system metadata when fractionation, not mobile flow medium data.
Further, present invention additionally comprises the adjustment flows of every road Streaming Media memory space；
The adjustment flow per road Streaming Media memory space is：If adjusting the memory space per Streaming Media all the way,
It is then that for reducing space is first selected the oldest files in stream media of needs all the way, merges a new file, then this
New file splits into that files in stream media all the way for increasing space.
The invention has the advantages that：
1st, allocation space when establishment file, so when video recording, it is only necessary to stream medium data is write, without
Written document system metadata (index or bitmap of memory space) is wanted, file fragmentation is avoided, prevents data from rattling；
2nd, rename when space is full, avoid file fragmentation and multichannel mutual exclusion；
3rd, video time is adjusted, memory space when need not format or delete legacy data；
4th, Piece file mergence and split when only change metadata, improving performance；
5th, on the opportunity of write-back, controlled by application layer, performance is controllable.
Description of the drawings
The present invention is further illustrated in conjunction with the embodiments with reference to the accompanying drawings.
Fig. 1 is the general flow chart of the process of multichannel stored stream media data of the present invention.
Fig. 2 is the execution flow chart of present invention initialization memory process.
Fig. 3 is that the present invention concurrently stores the execution flow chart for arriving cache pool process.
Fig. 4 is the execution flow chart of write-back process of the present invention.
Specific embodiment
Refering to Figure 1, the process of the multichannel stored stream media data of the present invention includes initialization memory, concurrently deposits
Storage, write-back.In order to avoid hydraulic performance decline it is necessary to avoid file fragmentation generation, it is necessary to avoid concurrent distribution
And Free up Memory；And in order to avoid the uncontrollable of write-back opportunity, it is necessary to the file cache for workaround system of trying every possible means.Therefore originally
The performance improvement method of the multichannel stored stream media data of invention is exactly to make some relevant processing in those steps.
It refers to shown in Fig. 2 to Fig. 4, the performance improvement method of multichannel stored stream media data of the invention, including storage
Renaming after equipment initial format flow, memory space are full covers flow and writes the process flow of data：
The storage device initial format flow is：As shown in Fig. 2, during memory is initialized, removing is deposited
The original fragment of equipment is stored up, and the size of each video file is calculated according to memory capacity and the video time set and code check
Fsize and quantity fnum；Then it is to calculate gained fnum files in stream media of quantity per establishment all the way, each file is creating
Distribution memory space is gone according to the size fsize for each video file for calculating gained later, this ensures that all stream matchmakers
The memory space of body file is all continuous, all without fragment；Wherein, the size fsize for calculating each video file and
The formula of quantity fnum is as follows：
Fsize=b*t；
Fnum=c/fsize.
In formula, b is the average bit rate of estimating of the road Streaming Media, and t is video time, and c is allocated to the memory space on the road,
Fnum is the number of the road files in stream media.
Renaming after the memory space is full covers flow：When memory space is full, do not go to delete old stream matchmaker
Body file, as soon as but an oldest files in stream media is directly selected to rename, new stream medium data be written to rename it is later
File accomplishes Xun Huan covering；Due to having lacked the process deleted ancient deed and carry out Free up Memory, so as to cause to delete and write simultaneously
It sends now, therefore the generation of file fragmentation will not be caused.
Writing the process flow of data is：As shown in Figure 3 and Figure 4, when writing data, the caching of application management is first copied to
Chi Zhong, and create an individual background thread and be responsible on suitable opportunity, with direct-io modes directly around operating system
Caching directly write back storage device, since the data of direct-io need to align, so the data of end-of-file can use 0 polishing,
File size (directly changing file system metadata, performance cost very little) can be finally changed, but does not include the 0 of polishing.Its
In, the control method on the suitable opportunity is：
(1) first write-back (such as t_interval=1s, according to memory is carried out by a default time cycle t_interval
Size goes to adjust, and the memory that smaller cache pool occupies is with regard to smaller), if write back data takes t_write>T_interval, just
Want modification time cycle t_interval=t_write/p, wherein p
It is a constant, for example can is 2,3 etc.；
(2) if unit exception power down, the data in cache pool can lose, then by upper strata a maximum being set to permit
Perhaps the data time t_loss that loses is controlled, if t_loss>0, then it can reset t_interval=min (t_loss, t_
interval)。
Caching during due to by individual background thread within suitable opportunity directly around operating system directly writes back
Storage device is allowed to the Streaming Media requirement for meeting multipath concurrence, so that storage device reaches optimum performance, wherein, modification text
Part size be for preferably compatible third party's player, due to some third party's players play some video formats when
It waits, runs into these 0 additional data, it is possible that abnormal, therefore the size directly in modification file system metadata can be with
Eliminate this exception.
It is time-consuming short when being copied out due to small documents, it does evidence and occupies little space, and the benefit of big file is to look for video
Shi Sudu is fast, therefore most of automobile data recorder all includes the function that can control the video time of each video file.For this purpose,
Present invention additionally comprises modification video time flows；
It is described modification video time flow be：If necessary to change video time, then oldest files in stream media is selected to close
And or split, merge and only change file system metadata when fractionation, not mobile flow medium data.
If there is multiple cameras, it is 720p just to have started camera a, changes the camera of 1080p into now or even does not change and takes the photograph
As head, but resolution ratio or different image quality patterns are had switched, code check is caused to become, originally 10g can record a video 2 hours, now only
1 hour can be recorded, at this time client may will think memory space of the adjustment per Streaming Media all the way.For this purpose, the method for the present invention
Further include the adjustment flow of every road Streaming Media memory space；
The adjustment flow per road Streaming Media memory space is：If adjusting the memory space per Streaming Media all the way,
It is then that for reducing space is first selected the oldest files in stream media of needs all the way, merges a new file, then this
New file splits into that files in stream media all the way for increasing space.
The invention has the advantages that：
1st, allocation space when establishment file, avoids file fragmentation, prevents data from rattling；
2nd, rename when space is full, avoid file fragmentation and multichannel mutual exclusion；
3rd, video time is adjusted, memory space when need not format or delete legacy data；
4th, Piece file mergence and split when only change metadata, improving performance；
5th, on the opportunity of write-back, controlled by application layer, performance is controllable.
Although specific embodiments of the present invention have been described above, those familiar with the art should manage
Solution, our described specific embodiments are merely exemplary rather than for the restriction to the scope of the present invention, are familiar with this
The equivalent modification and variation that the technical staff in field is made in the spirit according to the present invention, should all cover the present invention's
In scope of the claimed protection.
Claims (5)
- A kind of 1. performance improvement method of multichannel stored stream media data, it is characterised in that：Including storage device initial format Renaming after flow, memory space are full covers flow and writes the process flow of data：The storage device initial format flow is：The original fragment of storage device is removed, and according to memory capacity and setting Video time and code check calculate the size fsize of each video file and quantity fnum；Then it is to be calculated per establishment all the way Fnum files in stream media of gained quantity, the size of each video file of each file after creating according to obtained by calculating Fsize removes distribution memory space；Renaming after the memory space is full covers flow：When memory space is full, do not go to delete old Streaming Media text Part, as soon as but an oldest files in stream media is directly selected to rename, new stream medium data is written to later file of renaming, Accomplish Xun Huan covering；Writing the process flow of data is：When writing data, in the cache pool for first copying application management to, and a backstage is created Thread was responsible on suitable opportunity, and storage device is directly write back directly around the caching of operating system with direct-io modes, and The data of end-of-file can use 0 polishing, can finally change file size, but not include the 0 of polishing.
- 2. a kind of performance improvement method of multichannel stored stream media data according to claim 1, it is characterised in that：It is described The formula for calculating the size fsize and quantity fnum of each video file is as follows：Fsize=b*t；Fnum=c/fsize.In formula, b is the average bit rate of estimating of the road Streaming Media, and t is the video time of each file, and c is allocated to depositing for the road Space is stored up, fnum is the number of the road files in stream media.
- 3. a kind of performance improvement method of multichannel stored stream media data according to claim 1, it is characterised in that：It is described The control method on suitable opportunity is：First carry out write-back by a default time cycle t_interval, if write back data takes t_write>t_ For interval it is necessary to modification time cycle t_interval=t_write/p, wherein p is a constant；If unit exception power down, the data in cache pool can lose, thus by set on upper strata one it is maximum allowable It loses data time t_loss to be controlled, if t_loss>0, then it can reset t_interval=min (t_loss, t_ interval)。
- 4. a kind of performance improvement method of multichannel stored stream media data according to any one of claims 1 to 3, feature It is：Further include modification video time flow；It is described modification video time flow be：If necessary to change video time, then select oldest files in stream media do merge or It splits, merge and only changes file system metadata when fractionation, not mobile flow medium data.
- 5. a kind of performance improvement method of multichannel stored stream media data according to any one of claims 1 to 3, feature It is：Further include the adjustment flow of every road Streaming Media memory space；The adjustment flow per road Streaming Media memory space is：If adjusting the memory space per Streaming Media all the way, it is That for reducing space is first selected the oldest files in stream media of needs all the way, merges a new file, then this new text Part splits into that files in stream media all the way for increasing space.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711457166.6A CN108124112B (en) | 2017-12-28 | 2017-12-28 | Performance improving method for multi-path storage stream media data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711457166.6A CN108124112B (en) | 2017-12-28 | 2017-12-28 | Performance improving method for multi-path storage stream media data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108124112A true CN108124112A (en) | 2018-06-05 |
| CN108124112B CN108124112B (en) | 2020-04-03 |
Family
ID=62232116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711457166.6A Active CN108124112B (en) | 2017-12-28 | 2017-12-28 | Performance improving method for multi-path storage stream media data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108124112B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825697A (en) * | 2019-11-07 | 2020-02-21 | 珠海研果科技有限公司 | Method and apparatus for formatting a storage device |
| CN111506271A (en) * | 2020-06-29 | 2020-08-07 | 南京鹏云网络科技有限公司 | Data segment unit transmission differentiation management method and distributed block storage system |
| CN113805806A (en) * | 2018-12-03 | 2021-12-17 | 北京奥星贝斯科技有限公司 | Efficient data unit reusing method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102103740A (en) * | 2010-12-20 | 2011-06-22 | 福州瑞芯微电子有限公司 | Method and circuit for processing written address and width unaligned image |
| US7987384B2 (en) * | 2008-02-12 | 2011-07-26 | International Business Machines Corporation | Method, system, and computer program product for handling errors in a cache without processor core recovery |
| US8850125B2 (en) * | 2011-10-25 | 2014-09-30 | Cavium, Inc. | System and method to provide non-coherent access to a coherent memory system |
| CN104966528A (en) * | 2015-06-25 | 2015-10-07 | 广东工业大学 | Multi-path audio/video stream storage method for preventing disk fragments from formation |
| CN105867836A (en) * | 2015-01-23 | 2016-08-17 | 中兴通讯股份有限公司 | Storage management method and apparatus as well as stream media system |
-
2017
- 2017-12-28 CN CN201711457166.6A patent/CN108124112B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7987384B2 (en) * | 2008-02-12 | 2011-07-26 | International Business Machines Corporation | Method, system, and computer program product for handling errors in a cache without processor core recovery |
| CN102103740A (en) * | 2010-12-20 | 2011-06-22 | 福州瑞芯微电子有限公司 | Method and circuit for processing written address and width unaligned image |
| US8850125B2 (en) * | 2011-10-25 | 2014-09-30 | Cavium, Inc. | System and method to provide non-coherent access to a coherent memory system |
| CN105867836A (en) * | 2015-01-23 | 2016-08-17 | 中兴通讯股份有限公司 | Storage management method and apparatus as well as stream media system |
| CN104966528A (en) * | 2015-06-25 | 2015-10-07 | 广东工业大学 | Multi-path audio/video stream storage method for preventing disk fragments from formation |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113805806A (en) * | 2018-12-03 | 2021-12-17 | 北京奥星贝斯科技有限公司 | Efficient data unit reusing method and system |
| CN110825697A (en) * | 2019-11-07 | 2020-02-21 | 珠海研果科技有限公司 | Method and apparatus for formatting a storage device |
| CN111506271A (en) * | 2020-06-29 | 2020-08-07 | 南京鹏云网络科技有限公司 | Data segment unit transmission differentiation management method and distributed block storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108124112B (en) | 2020-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108124112A (en) | A kind of performance improvement method of multichannel stored stream media data | |
| JP2003513352A (en) | Virtual tape volume management device using data page atomic unit | |
| TWI381272B (en) | Flash memory apparatus and access method to flash memory | |
| WO2005109169A1 (en) | Method for accessing data, apparatus and recording medium for performing that method | |
| US20100169595A1 (en) | Storage backup | |
| JP2003525486A (en) | Disk scheduling system using finite request reordering | |
| CN102932622B (en) | The kinescope method of digital video recording equipment and device | |
| US7703012B2 (en) | Adaptable transfer and publication of digital media | |
| JP2003529146A (en) | System and method for accessing blocks on a storage medium | |
| KR20110088591A (en) | Flash based memory comprising a flash translation layer and method for storing a file therein | |
| US8625968B2 (en) | Recording apparatus and control method thereof | |
| JP2006303641A5 (en) | ||
| US20150242130A1 (en) | Multi-Threshold Storage Device and Method | |
| KR102094786B1 (en) | File system and method of storing files based on the file system | |
| EP2678769A1 (en) | A device controller for a memory device | |
| US6898610B2 (en) | Apparatus and method for space allocation of image and audio information | |
| CN108874945A (en) | A kind of data processing method and electronic equipment | |
| CN1976390A (en) | Method for realizing digital TV picture graph text information storaging | |
| CN109710189A (en) | A kind of data storage and management method and device | |
| US20080187281A1 (en) | Editing apparatus and editing method | |
| US20110194837A1 (en) | Recording apparatus, imaging and recording apparatus, recording method, and program | |
| JP2003150416A (en) | File management device and method therefor | |
| TWI840194B (en) | Driving recording file management method and system | |
| US20110187894A1 (en) | Recording apparatus and method | |
| Sparenberg et al. | Real time file system for content distribution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 350000 building, No. 89, software Avenue, Gulou District, Fujian, Fuzhou 18, China Patentee after: Ruixin Microelectronics Co., Ltd Address before: 350000 building, No. 89, software Avenue, Gulou District, Fujian, Fuzhou 18, China Patentee before: Fuzhou Rockchips Electronics Co.,Ltd. |