CN112866717A - Method and system capable of extracting H264 code stream stored in MP4 file - Google Patents
Method and system capable of extracting H264 code stream stored in MP4 file Download PDFInfo
- Publication number
- CN112866717A CN112866717A CN202110056158.0A CN202110056158A CN112866717A CN 112866717 A CN112866717 A CN 112866717A CN 202110056158 A CN202110056158 A CN 202110056158A CN 112866717 A CN112866717 A CN 112866717A
- Authority
- CN
- China
- Prior art keywords
- file
- extracting
- frame
- data
- pps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- AWSBQWZZLBPUQH-UHFFFAOYSA-N mdat Chemical compound C1=C2CC(N)CCC2=CC2=C1OCO2 AWSBQWZZLBPUQH-UHFFFAOYSA-N 0.000 claims abstract description 22
- 239000000284 extract Substances 0.000 claims abstract description 21
- 238000000605 extraction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to a method and a system for extracting H264 code stream stored in an MP4 file, wherein the method comprises the following steps: step S1: initializing a context of FFmpeg; step S2: calling an interface of the FFmpeg to open an MP4 file; step S3: checking whether a box identifier of an mdat field exists; step S4: extracting SPS frame information; step S5: extracting PPS frame information; step S6: extracting a video frame from the mdat field box; step S7: and repeating the steps until the analyzed frame length is greater than the remaining length of the file, and stopping. The invention reconstructs the file structure, reorganizes and analyzes according to the offset position between the boxes, and can normally extract the code stream data for decoding.
Description
Technical Field
The invention relates to the technical field of videos, in particular to a method and a system for extracting an H264 code stream stored in an MP4 file.
Background
Video decoding technology is widely used in the fields of video playing, video transcoding and video content auditing. Before decoding the compressed data of the video, it is necessary to decapsulate the video media file, that is, extract the video data stream and the audio data stream from the media file for processing by different decoders. The decapsulation operation correctly extracts the data stream corresponding to the audio/video, which is a necessary condition for the decoder to be able to correctly decode. The packaging format of the media files is various, such as MP4, AVI, FLV, MPEG, WEB M, etc. Extracting the code stream requires analyzing the description information of the file according to a data structure defined by a packaging format, so as to obtain the position information of the code stream.
All data in the MP4 file is contained in a box (atom in QuickTime), that is, the MP4 file is composed of several boxes, each having a type and a length, which can be understood as a data object block. Another box may be included in the box, which is called a container box. An M P4 file will first have and only have a box of the type "ftyp" as a marker for the MP4 format and containing some information about the file; there will be and only one "moov" type box (movie box) later, which is a container box, and the sub-box contains metadata information of the media; the media Data of the MP4 file is contained in a box (midi Data box) of the "mdat" type, which is also a container box, and there may be a plurality of boxes or none (when the media Data all refer to other files), and the structure of the media Data is described by metadata.
FFmpeg is a set of open source computer programs that can be used to record, convert digital audio, video, and convert them into streams. LGPL or GPL licenses are used. It provides a complete solution for recording, converting and streaming audio and video. It contains the very advanced audio/video codec library libavcodec, and many codes in libavcodec are developed from the beginning in order to ensure high portability and codec quality.
When the FFmpeg analyzes the MP4 file, recursive analysis is performed strictly according to the description information of each box of the MP4 file, and data loss often occurs in the network transmission process, so that the positions between the boxes are shifted, at this time, the FFmpeg cannot normally analyze the data arrangement of the MP4 file, and even if complete H264 code stream data is stored in mdat, decoding operation cannot be normally performed.
Disclosure of Invention
Problems to be solved by the invention
The invention mainly aims to provide a method and a system for extracting an H264 code stream stored in an MP4 file, so as to solve the problem that an FFmpeg cannot normally analyze an MP4 file.
Means for solving the problems
In order to achieve the above object, the present invention provides a method for extracting an H264 code stream stored in an MP4 file, comprising the following steps:
step S1: initializing a context of FFmpeg;
step S2: calling an interface of the FFmpeg to open an MP4 file;
step S3: checking whether a box identifier of an mdat field exists;
step S4: extracting SPS frame information;
step S5: extracting PPS frame information;
step S6: extracting a video frame from the mdat field box;
step S7: and repeating the steps until the analyzed frame length is greater than the remaining length of the file, and stopping.
Preferably, in step S1, the av _ register _ all () function is called to initiate an FFmpeg context.
Preferably, in step S2, the avformat _ open _ input () function is called to attempt to parse the file format initialization AVFormatContext data structure object, and an error is reported in this step for the damaged MP4 file.
Preferably, in step S3, the kmp algorithm is adopted to traverse the file according to the {'m','d', 'a','t' } character sequence, and if relevant fields are retrieved, the subsequent step is proceeded, otherwise, the mp4 file determined to be without data is stopped.
Preferably, in step S4, the file is traversed according to the { 'a', 'v', 'C' } character sequence using kmp algorithm, and if the relevant field start address a is retrieved, the subsequent data of address a is parsed.
Preferably, in step S4, the following data of address a is analyzed using the following structure, and SPS frame data is extracted from address a shifted by address B of the corresponding byte of the SPS structure by SPS _ num and nalu _ len:
preferably, in step S5, the position when the last frame of SPS information is resolved in step S4 is the starting address C of the PPS frame, and the data subsequent to the address C is resolved.
Preferably, the subsequent data of the address C is parsed by the following structure, pps _ num identifies the number of pps frames, and pps frame data is extracted from the subsequent data pointed by the data [0 ]:
preferably, in step S6, a video frame is extracted from the mdat field box;
searching from the start position of the mdat field according to a {0x0,0x0} byte sequence by adopting a KMP algorithm, checking whether the next byte is one of 0x67, 0x68, 0x65, 0x61, 0x41 and 0x06 after searching the field, and if the next byte belongs to the result of the bytes, analyzing the subsequent 4 bytes to obtain the frame length and extracting frame data according to the length information.
Another embodiment of the present invention provides a system capable of extracting an H264 bitstream stored in an MP4 file, including:
an initialize FFmpeg module that initializes a context of FFmpeg;
an open MP4 file module that calls the interface of FFmpeg to open the MP4 file;
a checking module for checking whether a box identifier of the mdat field exists;
an extract SPS frame information module that extracts SPS frame information;
a PPS frame information extracting module which extracts PPS frame information;
the video frame extracting module extracts video frames from the mdat field box;
and the repeating module is used for repeatedly executing the modules until the analyzed frame length is greater than the residual length of the file, and then stopping.
ADVANTAGEOUS EFFECTS OF INVENTION
The invention reconstructs the file structure, reorganizes and analyzes according to the offset position between the boxes, and can normally extract the code stream data for decoding.
Drawings
FIG. 1 is a flow chart of a method of the present invention for extracting H264 code stream stored in MP4 file;
FIG. 2 is a flow chart illustrating a method for extracting H264 code stream stored in MP4 file according to the present invention;
fig. 3 is a block diagram of a system of the present invention that can extract the H264 codestream stored in the MP4 file.
Description of the reference numerals
1: a system; 11: initializing an FFmpeg module; 12: opening the MP4 file module; 13: an inspection module; 14: an SPS frame information extraction module; 15: a PPS frame information extraction module; 16: a video frame extraction module; 17: and repeating the modules.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention. It should be further emphasized here that the following embodiments provide preferred embodiments, and that the various aspects (embodiments) may be used in combination or cooperation with each other.
For an MP4 file which cannot be normally analyzed by FFmpeg, a method and a system capable of extracting an H264 code stream stored in an MP4 file are provided. The general idea of the method for extracting the H264 code stream stored in the MP4 file is as follows: according to the standard of the MP4 file structure, all box identifiers can be defined. Under normal conditions, after the offset is carried out according to the length information of each box, the identifier of the next box can be analyzed, when the file is damaged, the correct identifier cannot be analyzed after the offset, at the moment, the KMP algorithm is adopted to traverse the data identified by the current box, the data is matched with the predefined box identifier, and after the corresponding box identifier is matched, the box is used as the reference to continue to be analyzed backwards.
As shown in fig. 1, a flowchart and a schematic flow chart of a method for extracting an H264 code stream stored in an MP4 file according to the present invention are shown, where the method for extracting an H264 code stream stored in an MP4 file according to the present invention includes the following steps:
step S1: initializing a context of FFmpeg;
the av _ register _ all () function is called to initiate the FFmpeg context.
Step S2: calling an interface of the FFmpeg to open an MP4 file;
invoking the avformat _ open _ input () function to attempt to parse the file format initializes the AVFormatContext data structure object, at which step an error is reported for the corrupted MP4 file.
Step S3: checking whether a box identifier of an mdat field exists;
and traversing the file according to the character sequence of {'m','d', 'a','t' } by adopting an kmp algorithm, entering a subsequent step if relevant fields are retrieved, and stopping processing of the mp4 file which is judged to be without data if the relevant fields are not retrieved.
Step S4: extracting SPS frame information;
and traversing the file according to the character sequence of { 'a', 'v', 'C' } by adopting an kmp algorithm, if a starting address A of a relevant field is searched, analyzing subsequent data of the address A by using a following structure, and extracting SPS frame data from an address B after the address A is shifted by a corresponding byte of an SPS structure through SPS _ num and nalu _ len.
Step S5: extracting PPS frame information;
the position when the last frame of SPS information is resolved in step S4 is the start address C of the PPS frame, the following data of the address C is resolved by the following structure, PPS _ num identifies the number of PPS frames, and PPS frame data can be extracted from the following data pointed by data [0 ].
Step S6: extracting a video frame from the mdat field box;
searching from the start position of the mdat field according to a {0x0,0x0} byte sequence by adopting a KMP algorithm, checking whether the next byte is one of 0x67, 0x68, 0x65, 0x61, 0x41 and 0x06 after searching the field, and if the next byte belongs to the result of the bytes, analyzing the subsequent 4 bytes to obtain the frame length and extracting frame data according to the length information.
Step S7: and repeating the steps until the analyzed frame length is greater than the remaining length of the file, and stopping.
The invention also provides a system capable of extracting the H264 code stream stored in the MP4 file, wherein the system 1 of the invention comprises: an initialize FFmpeg module 11, an open MP4 file module 12, a check module 13, an extract SPS frame information module 14, an extract PPS frame information module 15, an extract video frame module 16, and a repeat module 17.
An initialize FFmpeg module 11 that initializes the context of FFmpeg;
the av _ register _ all () function is called to initiate the FFmpeg context.
The open MP4 file module 12, which calls the interface of FFmpeg to open the MP4 file;
invoking the avformat _ open _ input () function to attempt to parse the file format initializes the AVFormatContext data structure object, at which step an error is reported for the corrupted MP4 file.
A checking module 13, which checks whether there is a box identification of the mdat field;
and traversing the file according to the character sequence of {'m','d', 'a','t' } by adopting an kmp algorithm, entering a subsequent step if relevant fields are retrieved, and stopping processing of the mp4 file which is judged to be without data if the relevant fields are not retrieved.
An extract SPS frame information module 14 that extracts SPS frame information;
and traversing the file according to the character sequence of { 'a', 'v', 'C' } by adopting an kmp algorithm, if a starting address A of a relevant field is searched, analyzing subsequent data of the address A by using a following structure, and extracting SPS frame data from an address B after the address A is shifted by a corresponding byte of an SPS structure through SPS _ num and nalu _ len.
An extraction PPS frame information module 15 which extracts PPS frame information;
the position when the last frame of SPS information is resolved in step S4 is the start address C of the PPS frame, the following data of the address C is resolved by the following structure, PPS _ num identifies the number of PPS frames, and PPS frame data can be extracted from the following data pointed by data [0 ].
An extract video frame module 16, which extracts video frames from the mdat field box;
searching from the start position of the mdat field according to a {0x0,0x0} byte sequence by adopting a KMP algorithm, checking whether the next byte is one of 0x67, 0x68, 0x65, 0x61, 0x41 and 0x06 after searching the field, and if the next byte belongs to the result of the bytes, analyzing the subsequent 4 bytes to obtain the frame length and extracting frame data according to the length information.
And a repeating module 17 for repeating the above modules until the analyzed frame length is larger than the remaining length of the file.
Compared with the prior art, the invention has the beneficial effects that:
the invention relates to a method for traversing a box to match identifiers and relocate for recursive resolution. When MP4 files with offset box positions are processed, FFmpeg is directly adopted for decoding, code stream data cannot be processed normally, after the scheme of the invention is adopted, the file structure is reconstructed, and the code stream data can be extracted normally for decoding by reorganizing and analyzing according to the offset positions among boxes.
Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.
Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for extracting H264 code stream stored in MP4 file is characterized by comprising the following steps:
step S1: initializing a context of FFmpeg;
step S2: calling an interface of the FFmpeg to open an MP4 file;
step S3: checking whether a box identifier of an mdat field exists;
step S4: extracting SPS frame information;
step S5: extracting PPS frame information;
step S6: extracting a video frame from the mdat field box;
step S7: and repeating the steps until the analyzed frame length is greater than the remaining length of the file, and stopping.
2. The method of claim 1, wherein in step S1, the av _ register _ all () function is called to initiate FFmpeg context.
3. The method of claim 1, wherein in step S2, invoking an avformat _ open _ input () function to attempt to parse a file format initialization AVFormatContext data structure object, and in step S4, an error is reported for a damaged MP4 file.
4. The method of claim 1, wherein in step S3, the file is traversed according to the {'m','d', 'a','t' } character sequence by using kmp algorithm, if relevant fields are retrieved, the subsequent step is proceeded, otherwise, the MP4 file determined to be without data is stopped.
5. The method of claim 1, wherein in step S4, the file is traversed according to the { 'a', 'v', 'C' } character sequence by using kmp algorithm, and if the relevant field start address a is retrieved, the subsequent data of address a is parsed.
6. The method of claim 5, wherein in step S4, the following structure is used to parse the data following address A, and SPS frame data is extracted from address B after address A is shifted by _ SPS structure corresponding bytes through SPS _ num and nalu _ len:
7. the method as claimed in claim 6, wherein in step S5, the position where the last frame of SPS information is analyzed in step S4 is the start address C of the PPS frame, and subsequent data of the address C is analyzed.
8. The method of claim 7, wherein the subsequent data at address C is parsed by a structure, pps _ num identifies the number of pps frames, and pps frame data is extracted from the subsequent data pointed by data [0 ]:
typedef struct_PPS{
uint8_t pps_num；
uint8_t data[0]；
}PPS。
9. the method of claim 1, wherein in step S6, extracting video frames from mdat field box;
searching from the start position of the mdat field according to a {0x0,0x0} byte sequence by adopting a KMP algorithm, checking whether the next byte is one of 0x67, 0x68, 0x65, 0x61, 0x41 and 0x06 after searching the field, and if the next byte belongs to the result of the bytes, analyzing the subsequent 4 bytes to obtain the frame length and extracting frame data according to the length information.
10. A system for extracting H264 code stream stored in MP4 file is characterized in that it includes:
an initialize FFmpeg module that initializes a context of FFmpeg;
an open MP4 file module that calls the interface of FFmpeg to open the MP4 file;
a checking module for checking whether a box identifier of the mdat field exists;
an extract SPS frame information module that extracts SPS frame information;
a PPS frame information extracting module which extracts PPS frame information;
the video frame extracting module extracts video frames from the mdat field box;
and the repeating module is used for repeatedly executing the modules until the analyzed frame length is greater than the residual length of the file, and then stopping.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110056158.0A CN112866717A (en) | 2021-01-15 | 2021-01-15 | Method and system capable of extracting H264 code stream stored in MP4 file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110056158.0A CN112866717A (en) | 2021-01-15 | 2021-01-15 | Method and system capable of extracting H264 code stream stored in MP4 file |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112866717A true CN112866717A (en) | 2021-05-28 |
Family
ID=76006935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110056158.0A Pending CN112866717A (en) | 2021-01-15 | 2021-01-15 | Method and system capable of extracting H264 code stream stored in MP4 file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112866717A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080165847A1 (en) * | 2007-01-05 | 2008-07-10 | Tae Wook Kim | Mp4 demultiplexer and method of operating the same |
| CN110662084A (en) * | 2019-10-15 | 2020-01-07 | 北京齐尔布莱特科技有限公司 | MP4 file stream live broadcasting method, mobile terminal and storage medium |
| CN110910916A (en) * | 2019-11-29 | 2020-03-24 | 四川效率源信息安全技术股份有限公司 | Carving method of monitoring video based on file structure |
| CN111683293A (en) * | 2020-05-31 | 2020-09-18 | 杭州视在数科信息技术有限公司 | Method for playing H.265 video across browsers based on HTTP-FLV protocol |
-
2021
- 2021-01-15 CN CN202110056158.0A patent/CN112866717A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080165847A1 (en) * | 2007-01-05 | 2008-07-10 | Tae Wook Kim | Mp4 demultiplexer and method of operating the same |
| CN110662084A (en) * | 2019-10-15 | 2020-01-07 | 北京齐尔布莱特科技有限公司 | MP4 file stream live broadcasting method, mobile terminal and storage medium |
| CN110910916A (en) * | 2019-11-29 | 2020-03-24 | 四川效率源信息安全技术股份有限公司 | Carving method of monitoring video based on file structure |
| CN111683293A (en) * | 2020-05-31 | 2020-09-18 | 杭州视在数科信息技术有限公司 | Method for playing H.265 video across browsers based on HTTP-FLV protocol |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2195952B1 (en) | Apparatus and method for storing and reading a file having a media data container and a metadata container | |
| CN110740391B (en) | Method for repairing MP4 damaged file | |
| KR102122628B1 (en) | Syntax-aware manipulation of media files in a container format | |
| CN103002353A (en) | Method and device for packaging multimedia documents | |
| CN104837031B (en) | A kind of method of high-speed adaptive extraction key frame of video | |
| US9883216B2 (en) | Method and apparatus for carrying transport stream | |
| US20050281289A1 (en) | System and method for embedding multimedia processing information in a multimedia bitstream | |
| US20060059509A1 (en) | System and method for embedding commercial information in a video bitstream | |
| Park et al. | Data fragment forensics for embedded DVR systems | |
| EP3941059A1 (en) | Image decoding device, image decoding method, image encoding device, and image encoding method | |
| CN112866717A (en) | Method and system capable of extracting H264 code stream stored in MP4 file | |
| KR101632689B1 (en) | The method for recovery of multimedia piece file | |
| CN113660442B (en) | Video restoration method and device, electronic equipment and readable medium | |
| US8457468B2 (en) | Method for embedding video annotation data into a coded video stream and video recording device | |
| CN109743627B (en) | Playing method of digital movie package based on AVS + video coding | |
| CN101064851A (en) | Method for displaying subtitling in video | |
| JP5409214B2 (en) | Media file repair method, media file creation method, media file playback method and apparatus | |
| RU2690163C2 (en) | Information processing device and information processing method | |
| CN116489458B (en) | MP4 repairing method, device, terminal and medium based on file byte analysis | |
| CN113055680B (en) | Distributed transcoding method | |
| EP4394602A1 (en) | Method for system verification and apparatus for implementing the same | |
| US11956294B2 (en) | Systems and methods for analyzing streamed media to determine common source encoders | |
| CN109495793B (en) | Bullet screen writing method, device, equipment and medium | |
| CN102231804B (en) | Fault tolerance method supporting PS streaming media file | |
| CN112866716A (en) | Method and system for synchronously decapsulating video file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Luo Xin Inventor after: Wang Da Inventor before: Yan Long Inventor before: Luo Xin Inventor before: Wang Da |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210528 |