CN102012846A - Integrity check method for large video file - Google Patents
Integrity check method for large video file Download PDFInfo
- Publication number
- CN102012846A CN102012846A CN 201010583170 CN201010583170A CN102012846A CN 102012846 A CN102012846 A CN 102012846A CN 201010583170 CN201010583170 CN 201010583170 CN 201010583170 A CN201010583170 A CN 201010583170A CN 102012846 A CN102012846 A CN 102012846A
- Authority
- CN
- China
- Prior art keywords
- fingerprint
- file
- video file
- data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses an integrity check method for a large video file. A data file is subjected to integrity check based on a mini disk (MD) 5 algorithm. The method comprises the following steps of: extracting a target MD5 fingerprint of the large video file, comparing the target MD5 fingerprint with a source MD5 fingerprint and determining that the file integrity condition is normal if the target MD5 fingerprint and the source MD5 fingerprint are the same, and is characterized in that: the large video film target MD5 fingerprint extraction comprises the following steps of: microchannel memory mapping, intelligent data slicing, parallel extracting of MD5 fingerprints of the sliced data and MD5 fingerprint synthesizing. The integrity check method has the advantages of overcoming the defects of long consumed time and low efficiency of checking the large video file by using the conventional MD5 code, rapidly and efficiently checking whether the file has the problems such as packet dropout, tampering, virus infection and the like and ensuring the integrity and usability of the data file.
Description
Technical field
The present invention relates to a kind of method of calibration of big video file integrality.
Background technology
At present, the data integrity checking of video file is general adopts traditional MD5 sign indicating number to carry out, the problem of this technical method maximum is the bottleneck to big video file verification, for whole big video file, the file of 2GB size generates a MD5 needs about 40 seconds, and the file of 7GB size about 150 seconds (double-core 3.OGHz CPU, 2GB MEM test down), if checking, need MD5 of regeneration to compare, consuming time is once twice.For the high definition on the horizon epoch, the size of most of high definition file is all very big, and as seen the integrality of traditional MD5 verification msg is conspicuous to the influence of work efficiency.
Summary of the invention
Goal of the invention of the present invention is to overcome the deficiency of the integrality method that has the MD5 checking data now, and a kind of big video file integrity checking method is provided, and reaches the purpose of the integrality of the big video file of verification quickly and efficiently.
The objective of the invention is to be achieved through the following technical solutions: the big quick method of calibration of video file integrality, based on the MD5 algorithm data are carried out integrity verification, comprise the target MD5 fingerprint that extracts big video file, target MD5 fingerprint and source MD5 fingerprint are compared, the file integrality state is normal if the two is identical, it is characterized in that: described big video file target MD5 fingerprint extraction may further comprise the steps:
(1) microchannel memory-mapped: big video file is mapped in the virtual address space of system physical internal memory, realizes the access of file according to the access mode of internal storage data;
(2) intelligent data section: the data that will be mapped to internal memory are carried out the intelligence section, are cut into a plurality of data blocks;
(3) data block MD5 fingerprint extraction: to the data block difference executed in parallel MD5 algorithm of having cut into slices, the eap-message digest that obtains each data block respectively is the MD5 sign indicating number;
(4) the MD5 fingerprint is synthetic, is that the MD5 sign indicating number adds up with the eap-message digest of each data block, and the MD5 sign indicating number that obtains whole big video file is a finger print information.
Described microchannel memory-mapped may further comprise the steps:
(1) reflection of mapped file head;
(2) this reflection is conducted interviews;
(3) cancel this reflection;
(4) new reflection that a darker displacement from file begins of mapping;
(5) repeating step (2)～(4) are up to having visited whole file datas.
The zone of virtual address space of the present invention is in 0x800000000 ~ 0xBFFFFFFF.
The data block size of intelligent data section gained of the present invention is 64MB or 128MB.
The invention has the beneficial effects as follows: solved the big video file of traditional MD5 code check many, inefficient shortcomings consuming time, can check quickly and efficiently whether file exists and substitute, distorted and problem such as infected virus, guarantee the integrality and the availability of data file.
Description of drawings
The quick checking process figure of the big video file integrality of Fig. 1
The quick verification MD5 of the big video file integrality of Fig. 2 fingerprint parallel processing synoptic diagram.
Specific embodiment
Further describe technical scheme of the present invention below in conjunction with accompanying drawing, as Fig. 1, the big quick method of calibration of video file integrality, based on the MD5 algorithm data are carried out integrity verification, comprise the target MD5 fingerprint that extracts the big video file of target, target MD5 fingerprint and source MD5 fingerprint are compared, and the file integrality state is normal if the two is identical, it is characterized in that: described big video file target MD5 fingerprint extraction may further comprise the steps:
(1) microchannel memory-mapped: big video file is mapped in the virtual address space of system physical internal memory, realizes the access of file according to the access mode of internal storage data;
(2) intelligent data section: the data that will be mapped to internal memory are carried out the intelligence section, are cut into a plurality of data blocks;
(3) data block MD5 fingerprint extraction: to the data block difference executed in parallel MD5 algorithm of having cut into slices, the eap-message digest that obtains each data block respectively is the MD5 sign indicating number;
(4) the MD5 fingerprint is synthetic, is that the MD5 sign indicating number adds up with the eap-message digest of each data block, and the MD5 sign indicating number that obtains whole big video file is a finger print information.
Described microchannel memory-mapped may further comprise the steps:
(1) reflection of mapped file head;
(2) this reflection is conducted interviews;
(3) cancel this reflection;
(4) new reflection that a darker displacement from file begins of mapping;
(5) repeating step (2)～(4) are up to having visited whole file datas.
The zone of virtual address space of the present invention is in 0x800000000 ~ 0xBFFFFFFF.
The data block size of intelligent data section gained of the present invention is 64MB or 128MB.
With 5 thread parallels data block being carried out the MD5 algorithm below is example, further describe technical scheme of the present invention in conjunction with the embodiments, as Fig. 2, the first step: read the big video file of target from hard disk, and big video file is mapped among the regional 0x800000000 ~ 0xBFFFFFFF of being of virtual address space of system physical internal memory, the steps include: the reflection of first mapped file head, then this reflection is conducted interviews, cancel this reflection again and shine upon the new reflection that a darker displacement from file begins, so shine upon repeatedly up to having visited whole file datas.
Second step: the data that will be mapped to internal memory are carried out intelligence and are cut into slices, and are cut into a plurality of data blocks, and wherein less than the file of 4GB, the size of data block is generally 64MB, are generally 128MB greater than the size of the data block of the file of 4GB.
The 3rd step: divide the data block after 5 thread parallels are handled section, specifically being treated to walks abreast respectively carries out the MD5 algorithm to data block, and the eap-message digest that obtains each data block is MD5 sign indicating number MD51, MD52, MD53, MD54 and MD55.Whole big file is done the MD5 computing compare with disposable, accelerated treatment effeciency widely.
The 4th step: the MD5 fingerprint is synthetic, with the eap-message digest of each data block is that the MD5 sign indicating number adds up, obtain the MD5 sign indicating number finger print information of whole big video file, the MD5 code value of whole big video file is the value addition of 5 data block MD5, i.e. MD51+MD52+MD53+MD54+MD55.
Above step has obtained the target MD5 fingerprint of big video file, and target MD5 fingerprint and source MD5 fingerprint are compared, and the file integrality state is normal if the two is identical, otherwise then file is imperfect.
Claims (4)
1. the big quick method of calibration of video file integrality, based on the MD5 algorithm data are carried out integrity verification, comprise the target MD5 fingerprint that extracts big video file, target MD5 fingerprint and source MD5 fingerprint are compared, the file integrality state is normal if the two is identical, it is characterized in that: the extraction of described big video file target MD5 fingerprint may further comprise the steps:
(1) microchannel memory-mapped: big video file is mapped in the virtual address space of system physical internal memory, realizes the access of file according to the access mode of internal storage data;
(2) intelligent data section: the data that will be mapped to internal memory are carried out the intelligence section, are cut into a plurality of data blocks;
(3) data block MD5 fingerprint extraction: to the data block difference executed in parallel MD5 algorithm of having cut into slices, the eap-message digest that obtains each data block respectively is the MD5 sign indicating number;
(4) the MD5 fingerprint is synthetic, is that the MD5 sign indicating number adds up with the eap-message digest of each data block, and the MD5 sign indicating number that obtains whole big video file is a finger print information.
2. the method for the quick verification of big video file integrality according to claim 1 is characterized in that: described microchannel memory-mapped may further comprise the steps:
(1) reflection of mapped file head;
(2) this reflection is conducted interviews;
(3) cancel this reflection;
(4) new reflection that a darker displacement from file begins of mapping;
(5) repeating step (2)～(4) are up to having visited whole file datas.
3. the method for the quick verification of big video file integrality according to claim 1 is characterized in that: the zone of described virtual address space is in 0x800000000 ~ 0xBFFFFFFF.
4. the method for the quick verification of big video file integrality according to claim 1 is characterized in that: the data block size of described intelligent data section gained is 64MB or 128MB.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010583170 CN102012846A (en) | 2010-12-12 | 2010-12-12 | Integrity check method for large video file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010583170 CN102012846A (en) | 2010-12-12 | 2010-12-12 | Integrity check method for large video file |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102012846A true CN102012846A (en) | 2011-04-13 |
Family
ID=43843022
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010583170 Pending CN102012846A (en) | 2010-12-12 | 2010-12-12 | Integrity check method for large video file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102012846A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740120A (en) * | 2012-06-20 | 2012-10-17 | 北京市博汇科技有限公司 | Digital television live program broadcasting integrity judging method |
| CN103312732A (en) * | 2012-03-07 | 2013-09-18 | 腾讯科技（深圳）有限公司 | Method and device for data file transmission |
| CN103593406A (en) * | 2013-10-17 | 2014-02-19 | 北京奇虎科技有限公司 | Static resource identifier processing method and device |
| CN104680064A (en) * | 2013-12-03 | 2015-06-03 | 国际商业机器公司 | Method and system for optimizing virus scanning of files using file fingerprints |
| CN105183381A (en) * | 2015-09-08 | 2015-12-23 | 浪潮(北京)电子信息产业有限公司 | Method and device for automatically detecting LUN number in storage system under Linux system |
| CN105653648A (en) * | 2015-12-28 | 2016-06-08 | 北京金山安全软件有限公司 | File acquisition method and device and electronic equipment |
| CN106203179A (en) * | 2016-07-12 | 2016-12-07 | 何晓行 | A kind of completeness check system and method to file |
| CN106649214A (en) * | 2016-10-21 | 2017-05-10 | 天津海量信息技术股份有限公司 | Internet information content similarity definition method |
| CN106708927A (en) * | 2016-11-18 | 2017-05-24 | 北京二六三企业通信有限公司 | Duplicate removal processing method and duplicate removal processing device for files |
| CN107239505A (en) * | 2017-05-10 | 2017-10-10 | 广州杰赛科技股份有限公司 | Cluster mirror image synchronization method and system |
| CN107318041A (en) * | 2017-06-29 | 2017-11-03 | 深圳市茁壮网络股份有限公司 | The method and system that a kind of Video security is played |
| CN107612877A (en) * | 2017-07-20 | 2018-01-19 | 阿里巴巴集团控股有限公司 | Verify the methods, devices and systems of multimedia file legitimacy |
| CN108966043A (en) * | 2018-08-08 | 2018-12-07 | 福州智永信息科技有限公司 | A kind of video consistent method for authenticating and system |
| CN109657098A (en) * | 2018-12-26 | 2019-04-19 | 北京天融信网络安全技术有限公司 | A kind of method for extracting video fingerprints and device |
| CN110149529A (en) * | 2018-11-01 | 2019-08-20 | 腾讯科技（深圳）有限公司 | Processing method, server and the storage medium of media information |
| WO2021027846A1 (en) * | 2019-08-12 | 2021-02-18 | 南京中兴新软件有限责任公司 | Method, apparatus, and system for detecting video code stream |
| CN112883427A (en) * | 2019-11-29 | 2021-06-01 | 航天信息股份有限公司 | Method and device for detecting integrity of file |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521818A (en) * | 2009-04-08 | 2009-09-02 | 四川长虹电器股份有限公司 | Encryption and decryption system based on medium security protection of IPTV platform |
| CN101615411A (en) * | 2009-07-22 | 2009-12-30 | 福州星网视易信息系统有限公司 | The playlist of networked information delivery system and the encryption method of media task |
| CN101814045A (en) * | 2010-04-22 | 2010-08-25 | 华中科技大学 | Data organization method for backup services |
-
2010
- 2010-12-12 CN CN 201010583170 patent/CN102012846A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521818A (en) * | 2009-04-08 | 2009-09-02 | 四川长虹电器股份有限公司 | Encryption and decryption system based on medium security protection of IPTV platform |
| CN101615411A (en) * | 2009-07-22 | 2009-12-30 | 福州星网视易信息系统有限公司 | The playlist of networked information delivery system and the encryption method of media task |
| CN101814045A (en) * | 2010-04-22 | 2010-08-25 | 华中科技大学 | Data organization method for backup services |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103312732B (en) * | 2012-03-07 | 2017-12-26 | 腾讯科技（深圳）有限公司 | The transmission method and device of data file |
| CN103312732A (en) * | 2012-03-07 | 2013-09-18 | 腾讯科技（深圳）有限公司 | Method and device for data file transmission |
| CN102740120A (en) * | 2012-06-20 | 2012-10-17 | 北京市博汇科技有限公司 | Digital television live program broadcasting integrity judging method |
| CN103593406A (en) * | 2013-10-17 | 2014-02-19 | 北京奇虎科技有限公司 | Static resource identifier processing method and device |
| CN104680064A (en) * | 2013-12-03 | 2015-06-03 | 国际商业机器公司 | Method and system for optimizing virus scanning of files using file fingerprints |
| CN105183381A (en) * | 2015-09-08 | 2015-12-23 | 浪潮(北京)电子信息产业有限公司 | Method and device for automatically detecting LUN number in storage system under Linux system |
| CN105183381B (en) * | 2015-09-08 | 2018-01-19 | 浪潮(北京)电子信息产业有限公司 | LUN numbers automatic testing method and device in storage system under linux system |
| CN105653648A (en) * | 2015-12-28 | 2016-06-08 | 北京金山安全软件有限公司 | File acquisition method and device and electronic equipment |
| CN106203179A (en) * | 2016-07-12 | 2016-12-07 | 何晓行 | A kind of completeness check system and method to file |
| CN106203179B (en) * | 2016-07-12 | 2019-06-18 | 何晓行 | A kind of completeness check system and method for pair of file |
| CN106649214A (en) * | 2016-10-21 | 2017-05-10 | 天津海量信息技术股份有限公司 | Internet information content similarity definition method |
| CN106708927A (en) * | 2016-11-18 | 2017-05-24 | 北京二六三企业通信有限公司 | Duplicate removal processing method and duplicate removal processing device for files |
| CN106708927B (en) * | 2016-11-18 | 2021-01-05 | 北京二六三企业通信有限公司 | File deduplication processing method and device |
| CN107239505B (en) * | 2017-05-10 | 2020-09-15 | 广州杰赛科技股份有限公司 | Cluster mirror synchronization method and system |
| CN107239505A (en) * | 2017-05-10 | 2017-10-10 | 广州杰赛科技股份有限公司 | Cluster mirror image synchronization method and system |
| CN107318041A (en) * | 2017-06-29 | 2017-11-03 | 深圳市茁壮网络股份有限公司 | The method and system that a kind of Video security is played |
| CN107612877A (en) * | 2017-07-20 | 2018-01-19 | 阿里巴巴集团控股有限公司 | Verify the methods, devices and systems of multimedia file legitimacy |
| CN107612877B (en) * | 2017-07-20 | 2020-06-30 | 阿里巴巴集团控股有限公司 | Method, device and system for verifying legality of multimedia file |
| CN108966043A (en) * | 2018-08-08 | 2018-12-07 | 福州智永信息科技有限公司 | A kind of video consistent method for authenticating and system |
| CN110149529A (en) * | 2018-11-01 | 2019-08-20 | 腾讯科技（深圳）有限公司 | Processing method, server and the storage medium of media information |
| CN110149529B (en) * | 2018-11-01 | 2021-05-28 | 腾讯科技（深圳）有限公司 | Media information processing method, server and storage medium |
| CN109657098A (en) * | 2018-12-26 | 2019-04-19 | 北京天融信网络安全技术有限公司 | A kind of method for extracting video fingerprints and device |
| CN109657098B (en) * | 2018-12-26 | 2023-11-10 | 北京天融信网络安全技术有限公司 | Video fingerprint extraction method and device |
| WO2021027846A1 (en) * | 2019-08-12 | 2021-02-18 | 南京中兴新软件有限责任公司 | Method, apparatus, and system for detecting video code stream |
| US11729436B2 (en) | 2019-08-12 | 2023-08-15 | Zte Corporation | Method, apparatus, and system for detecting video code stream |
| CN112883427A (en) * | 2019-11-29 | 2021-06-01 | 航天信息股份有限公司 | Method and device for detecting integrity of file |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102012846A (en) | Integrity check method for large video file | |
| Dai et al. | A malware classification method based on memory dump grayscale image | |
| CN110059484A (en) | Deep learning is executed to be used for Hole Detection to track data | |
| CN110111113B (en) | Abnormal transaction node detection method and device | |
| JP2004318879A5 (en) | ||
| US11531873B2 (en) | Convolution acceleration with embedded vector decompression | |
| WO2022100349A1 (en) | Artificial intelligence-based dual-recording quality inspection method and apparatus, computer device and medium | |
| CN105095756A (en) | Method and device for detecting portable document format document | |
| CN102708029B (en) | Performance bottleneck diagnostic method and equipment | |
| CN110138794A (en) | A kind of counterfeit website identification method, device, equipment and readable storage medium storing program for executing | |
| EP1607823A3 (en) | Method and system for virus detection based on finite automata | |
| CN112084486A (en) | User information verification method and device, electronic equipment and storage medium | |
| CN103246877B (en) | Based on the recognition of face novel method of image outline | |
| CN111159167A (en) | Labeling quality detection device and method | |
| CN111159150A (en) | Data expansion method and device | |
| EP3971746A1 (en) | Speculative and accelerated classification based on incomplete feature sets | |
| CN106227583A (en) | A kind of multiple thread video Real-time Data Processing Method and device | |
| CN109902731A (en) | A kind of detection method and device of the performance fault based on support vector machines | |
| CN104933411A (en) | Fingerprint identification processing method and system | |
| CN117081858A (en) | Intrusion behavior detection method, system, equipment and medium based on multi-decision tree | |
| Suteja et al. | Predicting the financial distress of Indonesian manufacturing companies: an application of the multinomial logit model | |
| CN106126195B (en) | A kind of information processing method and electronic equipment | |
| CN110084067A (en) | A kind of method for secret protection and device based on privacy chain | |
| CN109032897A (en) | Data dispatching method, host and solid state hard disk | |
| CN113506164B (en) | Wind control decision method and device, electronic equipment and machine-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20110413 |