KR101316579B1 - Mp4 file configuring device and restoring device, mp4 file configuring method and restoring method - Google Patents

Abstract

An apparatus for constructing an MP4 file for converting and storing video / audio data (media data) into an MP4 file format according to ISO / IEC 14496-12 standard, comprising: a media acquisition unit for acquiring media data; Necessary for dividing the media data into predetermined sample units, adding additional information indicating at least a time stamp and a size of the sample to each sample, recording the data in a mdat box as a frame, and organizing the media data into an MP4 file format. The metadata is generated and recorded in a memory, and at least a part of the metadata is recorded in a specific box included in an MP4 file in real time, and when the acquisition of the media data from the media acquisition unit is completed, the metadata is stored in the mdat box until then. An MP4 file constructing unit for constructing the moov box as the final meta data of the stored media data and recording at the end of the MP4 file; And a file storage unit.

Description

MP4 FILE CONFIGURING DEVICE AND RESTORING DEVICE, MP4 FILE CONFIGURING METHOD AND RESTORING METHOD}

The present invention relates to a device for constructing an MP4 file, an apparatus for recovering an MP4 file, and a method for constructing an MP4 file and a method for recovering an MP4 file. The present invention relates to a device for organizing and storing video / audio data in an MP4 file format, and an MP4 file stored by the device. The present invention relates to a device for recovering a damaged MP4 file if it is damaged, a method for constructing and storing an MP4 file by these devices, and a method for recovering a damaged MP4 file.

Moving Pictures Experts Group (MPEG) has developed MPEG-4, a multimedia compression standard for implementing multimedia presentations including video and sound. The MPEG-4 specification defines a set of coding tools for video / audio data and defines a syntactic description of coded acoustic-visual objects.

The file format specified for MPEG-4 is called "MP4", and the schematic configuration of the MP4 file is as illustrated in FIG. 1 or 2.

The MP4 technology is described in the ISO / IEC 14496-12 ISO base media file format. In MP4, all data is encapsulated in a structure called a "box."

Basic top level boxes include ftyp boxes, mdat boxes, and moov boxes.

ftyp box: Also called a file type box, indicates the type and compatibility of the file.

mdat box: Also called a media data box, this part contains actual video / audio frame data (media data).

-moov box: Also called movie box, this part contains meta data. Stored at the top level of the MP4 file (near the beginning or end of the file). A track of each data is included, and each track includes a data format (information indicating codec type and resolution in the case of video data, and information such as codec type and channel, sampling rate, and bit per sample in case of audio data). Stored. In addition, information related to each video / audio frame data is stored.

As such, the MP4 format separates all presentation level information (meta data) from multimedia data (media data), including video / audio data, and adds it to one embedded structure in the file, which is " moov box ".

The media data from which the meta data is separated is divided and stored as an mdat box, and the media data stored therein is interpreted with reference to the moov box.

Typically, the proportion of metadata corresponds to 5% to 20% of the total MP4 file size. In order to sequentially download and play a general track-oriented streaming file such as an MP4 file, all metadata must be transmitted in preference to the media data.

In addition, when the video is to be stored in the MP4 file format, metadata must be generated at the end of the file and included in the end of the MP4 file.

Problems caused by using the MP4 file format may be caused by a conflict between the MP4 file to be transmitted and an application program receiving the same, or insufficient file storage capacity at the receiving side while storing the received MP4 file in a storage space. If for some reason the moov box contained at the end of the file could not be saved completely, the data stored in the meantime will not be available.

Recently, image capturing / storage devices such as vehicle black boxes store captured images in the MP4 file format. At this time, the MP4 file can be reproduced only when the moov box as well as the mdat box of the media data is completely stored. Therefore, the unit of image storage (size of the MP4 file in time units) is subdivided into 30 seconds or 1 minute. Doing.

However, even if the time is subdivided, if the mdat box or the moov box is not completely stored due to the black box's power supply interruption or damage due to a vehicle accident while the image is being stored in real time, the image is stored as a new file. The problem of not being able to reproduce this video is still a problem because it cannot obtain additional information.

For example, if the unit (segmentation time) of image storage is 30 seconds, a new MP4 file is generated and near 20 seconds while the image is being stored. Since the moov box could not be stored for the MP4 file to be stored and stored, this 20-second video cannot be reproduced.

In particular, since a vehicle accident is likely to occur depending on a situation appearing about 10 seconds from the time of the accident, the video loss during 20 seconds may be the most important information for analyzing the situation of the accident. .

Therefore, there is a need to recover the previously stored image as far as possible until the image storage is stopped.

For reference, referring to Korean Patent Laid-Open Publication No. 2004-41174, the media data recorded in the mdat box of the MP4 file is divided into segments, and each segment is configured in the form of "media data-meta data pair", and each meta The data is created to specify only the media data of the segment.

The progressive MP4 file format of this document is configured to include a moov box and an mdat box for every smp4 box (atom). However, the moov box and the mdat box are top-level boxes (atoms), and none of the boxes include the moov box and the mdat box. Thus, this file format corresponds to not following the conventions of the standardized MP4 file format and cannot be played in a general MP4 file player. Moreover, the MP4 file by this file format has a problem that the overall size of the MP4 file increases when it is transmitted over a network or created using media data.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described necessity, and the recording of the moov box to be included in the file is not completed or the moov box is not completed. Although it is not complete, it is intended to provide a device and method for configuring an MP4 file that supports the recovery and playback of the video or sound stored in the meantime, and to recover the MP4 file.

The present invention for achieving the above object, MP4 file configuration device for converting and storing video / audio data (media data) to the MP4 file format according to the ISO / IEC 14496-12 standard: generated by a camera or microphone A media obtaining unit obtaining media data; Necessary for dividing the media data into predetermined sample units, adding additional information indicating at least a time stamp and a size of the sample to each sample, recording the data in a mdat box as a frame, and organizing the media data into an MP4 file format. Generate and record the metadata in a memory; and record at least a part of the metadata in a specific box included in an MP4 file, and store the metadata in the mdat box until the media data is obtained from the media acquisition unit. An MP4 file constructing unit configured to form the moov box as final metadata for media data and to record the MP4 file; And a file storage unit for storing a plurality of boxes recorded by the MP4 file configuration unit in real time as one MP4 file.

In addition, the metadata includes at least a track-specific format of the media data (at least in the case of video data, a video codec and an image resolution, and in the case of audio data, at least an audio codec, an audio channel number, a sampling rate, and a sampling bit. At least one of a number), the additional information for each frame of the media data (including at least one of a stream ID, a frame type (I-frame or P-frame), a start position offset, a time stamp, and a size), and the media The total duration of each track of data is included.

The additional information may further include a stream ID or frame type of the corresponding frame.

The specific box may be at least one of a free box, a skip box, a udta box, or a box that can be ignored when parsing an MP4 file.

The specific box is added before the mdat box in the MP4 file to be configured.

In addition, the present invention provides an MP4 file recovery apparatus for recovering video / audio data (media data) from an interrupted MP4 file generated in accordance with the ISO / IEC 14496-12 standard, wherein: the media data is divided into predetermined sample units. And additional information indicating at least a time stamp and a size of the sample is added to each sample and recorded in the mdat box as a frame, and at least a part of the metadata required to configure the media data in the MP4 file format is stored in the MP4 file. An MP4 file obtaining unit which obtains a damaged MP4 file recorded in a specific box that is included, but the mdat box is incomplete or there is no moov box or is incomplete; And identifying the last valid frame of the mdat box using the additional information recorded for each frame in the corrupted MP4 file, and using the information stored in the specific box and the time stamp and size of the identified last valid frame. Compute the total playback time of the media data, generate metadata up to the last valid frame using the additional information and the calculated total playback time, and reconstruct the moov box with the generated metadata It provides an MP4 file recovery apparatus including a file recovery unit for closing the damaged MP4 file to a normal file by adding to the MP4 file.

The specific box is at least one of a free box, a skip box, a udta box, or a box that can be ignored when parsing an MP4 file, and is located before the mdat box in the MP4 file.

The last valid frame is the last frame in which the additional information is completely recorded, and the total reproduction time is determined using the time difference between the time stamp of the last valid frame and the time stamp of the first frame.

The present invention also provides a video storage method for converting and storing video / audio data (media data) into an MP4 file format according to the ISO / IEC 14496-12 standard, the method comprising: obtaining media data generated by a camera or a microphone; ; Dividing the media data into predetermined sample units and adding additional information indicating at least a time stamp and a size of the sample for each sample to record in a mdat box as a frame; Generating and recording meta data necessary for organizing the media data into an MP4 file format in a memory, and recording at least a part of the meta data into a specific box included in the MP4 file; And when the acquisition of the media data from the media acquiring unit is completed, constructing the moov box with the final metadata for the media data stored in the mdat box and recording it in the MP4 file. To provide.

In addition, the metadata includes at least a track-specific format of the media data (at least in the case of video data, a video codec and an image resolution, and in the case of audio data, at least an audio codec, an audio channel number, a sampling rate, and a sampling bit. At least one of a number), the additional information (including at least one of a stream ID, a frame type, a start position offset, a time stamp, and a size) for each frame of media data, and a total duration of each track of the media data. Wherein the specific box is at least one of a free box, a skip box, a udta box, or a box that can be ignored when parsing an MP4 file added before the mdat box in the MP4 file. .

In addition, the present invention provides an MP4 file recovery method for recovering video / audio data (media data) from an interrupted MP4 file generated in accordance with the ISO / IEC 14496-12 standard, wherein the media data is divided into predetermined sample units. And additional information indicating at least a time stamp and a size of the sample is added to each sample and recorded in the mdat box as a frame, and at least a part of the metadata required to configure the media data in the MP4 file format is stored in the MP4 file. Acquiring a damaged MP4 file recorded in a specific box to be included, wherein the mdat box is incomplete, there is no moov box, or is incomplete; Identifying the last valid frame of the mdat box using the additional information recorded for each frame in the corrupted MP4 file, and using the information stored in the specific box and the time stamp and size of the identified last valid frame. Calculating a total play time of media data and generating metadata up to the last valid frame using the additional information and the calculated total play time; And reconstructing the moov box with the generated metadata and adding the damaged MP4 file to the damaged MP4 file, thereby closing the damaged MP4 file as a normal file.

In addition, the metadata includes at least a track-specific format of the media data (at least in the case of video data, a video codec and an image resolution, and in the case of audio data, at least an audio codec, an audio channel number, a sampling rate, and a sampling bit. The additional information (stream ID, frame type, start position offset, time stamp, size) for each frame of media data, the total duration of each track of the media data, and the specific box. Is at least one of a free box, skip box, and udta box added before the mdat box in the MP4 file, wherein the last valid frame is the last frame in which the additional information is completely recorded, and the reproduction time is Determined using the time difference between the time stamp of the last valid frame and the time stamp of the first frame The features.

According to the MP4 file construction apparatus according to the present invention having the above-described configuration, the storage of the moov box to be included in the file is completed by stopping any device while recording the image or recording the sound and storing it as an MP4 file in real time. Even if the MP4 file is not normally closed because the file is not completed or the moov box is not complete, it is possible to restore and play the video or sound saved up to that time by using only the stored data.

Further, even if the MP4 file generated by the present invention is played back using an existing MP4 file player, the same can be played back.

1 is a block diagram schematically showing the configuration of an MP4 file.
2 is a block diagram schematically showing another configuration of an MP4 file.
3 is a block diagram illustrating a configuration of an MP4 file construction apparatus and an MP4 file recovery apparatus according to an embodiment of the present invention.
4 is a view for explaining the structure of media data stored in an mdat box among MP4 files generated by the MP4 file construction apparatus according to the present invention.
Fig. 5 is a view for explaining the structure of a damaged MP4 file and an MP4 file recovered by the MP4 file recovery apparatus according to the present invention.
6 is a flowchart illustrating the operation of the apparatus for constructing an MP4 file according to the present invention.
7 is a flowchart illustrating the operation of the MP4 file recovery apparatus according to the present invention.

Hereinafter, a device for constructing an MP4 file according to the ISO / IEC 14496-12 standard according to the present invention and a device for recovering a damaged MP4 file interrupted while being generated by the device, a method for constructing an MP4 file by the devices And a method of repairing a damaged MP4 file will be described with reference to FIGS. 3 to 7.

The media data stored in the MP4 file may include video data of an image photographed by a camera and audio data of a sound recorded by a microphone. In addition, each MP4 file may include at least one video data or audio data. However, for convenience of description, only one of them is recorded as media data.

In addition, the types of respective boxes, their implementation manners, and various terms may be understood with reference to the items defined in the ISO / IEC 14496-12 ISO base media file format.

3 is a block diagram illustrating a configuration of an MP4 file construction apparatus and an MP4 file recovery apparatus according to an embodiment of the present invention. Referring to FIG. 3, the MP4 file constructing apparatus includes a media obtaining unit, an MP4 file constructing unit, and a file storing unit. In addition, the MP4 file recovery apparatus includes an MP4 file acquisition unit and a file recovery unit.

First, the media acquisition unit of the MP4 file constituting device receives video data and / or audio data generated by converting an image captured by a camera and / or a sound input through a microphone into a digital data stream. Such video / audio data may be directly input from a camera and / or a microphone, or may be obtained by being transmitted from a network communication or a connection with an external device or by extracting from a file having a format other than the MP4 file format.

The MP4 file construction unit divides the media data obtained by the media acquisition unit into arbitrary units (samples), and generates additional information indicating a time stamp and a size for a sample of each divided media data. Then, the generated additional information and the sample are combined to form a frame. The frames of media data divided into sample units and added additional information are sequentially recorded in the mdat box according to the generated order. The additional information may include a stream ID, a frame type (I-frame or P-frame), size of divided media data, and the like.

On the other hand, the MP4 file constitution unit includes at least a video codec and a video resolution in the case of a track data format (in the case of video data, and at least an audio codec and audio in the case of video data, as in the case of configuring a typical MP4 file). Information including the number of channels, the sampling rate and the number of sampling bits), additional information for each frame of the media data (stream ID, frame type (I-frame or P-frame), start position offset, time stamp, size), media Meta data including the total duration of each track of data is generated and temporarily stored in the memory.

The MP4 file constitution unit simultaneously records the meta data not only in the memory but also in a specific box defined by the MP4 file format. This particular box may be at least one of a free box, a skip box, and a udta box. These boxes are selected from the boxes that are ignored when parsing an MP4 file in a typical MP4 file player. In addition to the above-described box, if a box that can be ignored in a general situation and can record arbitrary contents is defined by the user, the metadata may be recorded in the box.

On the other hand, it is preferable that the specific box in which the meta data is recorded is arranged before the mdat box in which the media data is recorded when constituting the MP4 file. A specific box is described below as a free box.

In this way, when additional frames are added to form a frame, metadata is generated and recorded in the memory, and the recording or recording reaches an arbitrary storage time or a specified storage capacity, the MP4 file to be recorded is terminated until now. Record the metadata stored in memory in the moov box of the MP4 file. The MP4 file is closed normally by adding the moov box normally.

The file storage unit is a non-volatile storage space as a storage space capable of storing in real time the media data generated by the MP4 file configuration unit. Further, the file storage section further includes a memory for recording meta data to be recorded in the moov box.

In this way, the configuration of the mdat box in which the media data is stored by the MP4 file construction apparatus will be described with reference to FIG. In the present invention, the media data is divided into sample units (M1, M2, M3, ...), and for each divided sample, additional information (H1, H2, H3, ...) indicating the characteristics of the sample is provided. In addition, a frame is constructed (F1, F2, F3, ...) and sequentially recorded in the mdat box.

In this case, the additional information is for indicating a characteristic of a sample of media data immediately following and may include a stream ID, a frame type (I-frame, P-frame), a time stamp, a size, and other information. Here, the additional information should include at least a time stamp and a size.

The stream ID is an ID for the media data and may be divided into a stream of video data or a stream of audio data. For example, if two or more streams of video data exist, different IDs are assigned to each stream.

The frame type is used only when the media data is video data, and is used to distinguish whether an I-frame or a P-frame.

The time stamp is information for indicating the reproduction time of the media data recorded in each frame. The time stamp may be expressed in units of time (for example, ms), but is generally expressed in a predetermined time scale without a certain unit. In this case, the conversion relationship between the time scale and the actual time may be recorded in the metadata.

The size represents the size of each sample in which the media data is divided and recorded. It can be written primarily in bytes.

The other information is a space made for recording other data for recovery purposes or other purposes.

As described above, according to the structure of the mdat box according to the present invention, additional information added for each frame of media data is divided into information (time stamp) indicating at least the temporal position of the frame and the size of the frame (that is, divided into frames). Only by including the information indicating the size of the recorded media data, it is possible to recover the MP4 file (at least a part of the various information (meta data) necessary to recover the file by completing the MP4 file format). Is already stored separately in the free box, a recovery method using the same will be described later).

Fig. 5 is a view for explaining the structure of a damaged MP4 file and an MP4 file recovered by the MP4 file recovery apparatus according to the present invention. The damaged MP4 file is incompletely interrupted while being recorded by the MP4 file construction device according to the present invention. A ftyp box constituting the beginning of the MP4 file is basically present, and a free box for recovering the file is provided. The media data is then interrupted while the frames of the media data are recorded in chronological order in the mdat box.

The file interrupted in this way cannot be reproduced in a general MP4 file player because a moov box for recording information related to the media data recorded in the MP4 file and closing the file is not already recorded.

On the other hand, in the MP4 file recovered by the MP4 file recovery apparatus of the present invention, the media data recorded until the time when the recording of the mdat box is stopped is preserved as much as possible, and the moov box including the metadata for the media data up to this point is It is an added form. The MP4 file recovered thereby can be recognized as a normal playable MP4 file.

3, the configuration of the MP4 file recovery apparatus for recovering a damaged MP4 file to a normal reproducible MP4 file recorded by the MP4 file construction apparatus and then stopped recording will be described.

The MP4 file obtaining unit obtains the damaged MP4 file from the file storage unit of the MP4 file constructing device or through another database or network stream. Since the damaged MP4 file is interrupted while the media data is recorded in the mdat box, the last frame may be recorded and stopped in an incomplete form. Therefore, part of the additional information or part of the media data constituting the frame is damaged.

The file recovery unit checks the last recorded frame in the damaged file and checks whether the additional information recorded in the final frame is valid. If additional information is validly recorded in the last frame, the frame can be regarded as a valid frame.

A valid frame is additional information about a sample of media data recorded in the frame, and includes a stream ID, a frame type (I-frame, P-frame), a time stamp, a size, and other information, at least one of which is a time stamp. And size, the time stamp of the frame in the MP4 file and the size of the recorded media data of the frame can be calculated. Further, using this, it is possible to calculate the total playing time of the media data recorded in the damaged MP4 file, the offset of the corresponding frame, and the like.

In addition, the size of the mdat box may be calculated by identifying additional information included in a valid frame of the mdat box and using the additional information.

In addition, since the free box is already recorded at the beginning of the MP4 file, other information for generating the moov box can be obtained using the contents recorded in the free box.

In this way, the file recovery unit can configure information that must be included in the moov box. That is, a stts box, stsz box, stsc box, stco box, stss box, and the like can be configured.

Once the information for constructing the moov box is made, the damaged MP4 file can be normally closed by adding a moov box following the last valid frame, and the recovered MP4 file can be played normally in a general MP4 file player.

As described above, according to the MP4 file construction device and the MP4 file recovery device, an MP4 file whose recording is incompletely terminated (corrupted) even if recording is interrupted for any reason while converting and recording the media data into the MP4 file format. Can be restored to normal MP4 file. In particular, the MP4 file organizer can be mounted in, for example, a vehicle black box. If the car black box records the video / audio data shot forward in real time, even if the recording is interrupted because the MP4 file is not completely closed due to a traffic accident or other reasons, the recorded MP4 file is loaded on the computer. By being able to recover from the MP4 file recovery device, it is possible to recover the contents recorded at the time of the accident as much as possible.

Fig. 6 is a flowchart illustrating the operation of the MP4 file construction apparatus according to the present invention, and explains how the MP4 file construction apparatus configures a recoverable MP4 file.

First, a ftyp box is recorded to record / save the media data in the form of digital data of an image or sound input through a camera or a microphone (S11), and generate metadata related to the media data, and simultaneously Information necessary for the recovery of the file is constructed and recorded in the free box (S12).

Subsequently, when the stream of media data is received, a sample is generated by dividing the obtained media data by a predetermined unit, and additional information is generated for each sample of the divided media data. A frame is formed by combining the generated additional information and the sample of the separated media data. Frames are sequentially recorded in the mdat box in chronological order (S14). On the other hand, while acquiring and recording the media data, the metadata necessary to construct the MP4 file is generated and temporarily stored in the memory. Meta data that is temporarily stored can be continuously updated while frames of media data are recorded.

When the data is finally recorded while the data is recorded (S15), a moov box is created using the metadata recorded in the memory, and the generated moov box is recorded to close the MP4 file normally. (S16).

7 is a flowchart illustrating the operation of the MP4 file recovery apparatus according to the present invention. FIG. 7 shows a file recovery procedure when the MP4 file configuration device is damaged during recording of media data in real time according to the procedure of FIG. 6, the file recording is interrupted, and the mdat box is incomplete or the moov box is incomplete. to be.

First, when the MP4 file recovery apparatus acquires a damaged MP4 file due to an incomplete mdat box, a missing moov box, or an incomplete operation (S21), after checking whether a free box in which the information for recovery is stored exists in the obtained file, the stored information is stored. Acquire (S22). It is also checked whether additional information is recorded in each frame recorded in the mdat box.

Next, a frame in which additional information is completely recorded among the frames of the media data recorded in the mdat box is identified as a valid frame, and the last valid frame of the identified valid frames is selected as the last valid frame. The mdat box is set to end up to the last valid frame identified. The additional information recorded in the identified last valid frame is obtained (S23).

Using the additional information from the identified last valid frame and the information stored in the free box, generate metadata to be recorded in the moov box so that the MP4 file can be closed until the final valid frame, and the generated metadata is used to create the moov box. Create and add to the mdat box after finishing to the last valid frame and finish the MP4 file (S24).

According to the MP4 file construction device according to the present invention having the configuration as described above, while recording the image or recording the sound in real time to store as an MP4 file, any device is interrupted to be included at the end of the file of the moov box Even if the MP4 file is not completed normally, such as when the recording is not completed completely or when the mdat box is abnormally interrupted, it is possible to restore and play the image or sound saved up to that time by using only the stored data.

In addition, since the MP4 file generated by the present invention conforms to the MP4 file format standard, the MP4 file can be reproduced in the same way as a general MP4 file. That is, when using an existing MP4 file player, the free box is ignored, and the media data recorded in the mdat box is in a form that conforms to the standard, and the configuration of the moov box also conforms to the standard. It cannot be recognized, and the MP4 file constructed according to the present invention can be recognized as a general MP4 file.

In a typical MP4 file player, media data is identified using an offset for indicating the starting point of the media data sample in each frame and the size of the sample recorded in the frame.

When the MP4 file is normally closed, the moov box records the offset and time stamp of the start point of the media data sample except for the additional information portion in each frame, and the size of each sample. The existence cannot be recognized, and there is no problem with normal playback of the file.

In the following, reference is made to the references for the actual implementation of the technical contents according to the present invention by reference.

The basics

The MP4 file format is a technology that compresses data such as video and audio through encoding and stores the data in the MP4 file format, which is expressed in ISO / IEC 14496-12 ISO base media file format.

The general MP4 file format is basically divided into box units, and the basic box includes ftyp box, mdat box, and moov box. In addition, there may be a free box that the user can define and use.

ftyp box: This is called a file type box and indicates the type and compatibility of the file.

-mdat box: It is called a media data box and it is a part that contains video / audio frame data.

moov box: Called movie box, metadata is recorded and constitutes the top level of the file. In general, the moov box is near the beginning or end of the file. A track of each data is stored in the moov box. In the track, data format (information such as codec type and resolution in the case of video, and information such as codec type and channel, sampling rate (sampling rate), and bit per sample in the case of audio) is recorded. Records information about each video / audio frame (sample) data.

summary

In the damaged MP4 file, the moov box is missing or contains incorrect information. To recover the MP4 file, save the necessary information in the moov box to another location in the file.

Information representing the characteristics of the media in the moov box is defined in the free box using the free (skip, udta) box and put the necessary information. The free (skip, udta) box is a box that is ignored when parsing MP4 files in a universal player. In addition to free boxes, boxes that can be ignored by general-purpose players are also available.

The additional information of the video / audio frame data in the mdat box is recorded together when the video / audio frame data is put in the mdat box.

Since the free (skip, udta) box is a box area that is ignored when parsing in normal players, put the necessary information for recovery in this box and read the data in a predefined way in this box when you need to recover the file. Can be used as data for recovery.

free box Required information for recovery included in

end. video data Information about

-video codec (compression method): means H.264, MP4V, H.263, etc.

-video resolution: The width and height of the video.

Information needed by video codec: Can only be added if necessary. For example, in the case of H.264, a sequence parameter set and a picture parameter set can be displayed.

I. audio data Information about

-audio codec: AAC, AMR, QCELP, EVRC, etc.

-audio channel: 1 channel, 2 channel, etc.

-bit per sample: A bit for sampling sound frequency. It displays 8bit, 16bit, etc., and it can be changed according to the characteristics of audio data.

Sampling rate: The frequency used to record audio. For example, 8000 Hz, 16000 Hz, 22050 Hz, 32000 Hz, 44100 Hz, and the like.

Information needed by audio codec: Can only be added if necessary. For example, the AAC codec can display an audio specific config.

mdat box Required for recovery included in video Of audio frame data Additional information from

stream ID: ID of each stream, ID separated by video / audio.

-frame type: In case of video, it is information to distinguish I-frame and P-frame. In case of audio, it is meaningless.

time stamp: Displays the time of each frame data.

-size: The length of each frame data. Here, the size of the video / audio data recorded in the frame can be indicated.

By recording the data size in this way, even if additional additional information is included in each frame of the mdat box, the problem does not occur in the general-purpose player. That is, in a general-purpose player, frame data is obtained from a file by calculating an offset of a frame (sample) using stsc box and stco (co64) box recorded in a free box of an MP4 file, and stsc box and stco (co64) box are obtained. If the chunk number and offset of the frame data are stored in the file, the problem does not occur.

Damaged MP4  Necessary for file recovery data

video: codec type, width, height, information required for each codec, and duration (the length of the video track).

audio: codec type, number of channels, sampling-rate, bit-per-sample, information required for each codec, duration (length of audio track).

-Additional information of video / audio frame data: stream id (to distinguish video / audio), frame type (to distinguish I-frame and P-frame in video), time stamp (indicate time of each frame) Size (used to calculate the length of each frame (or the length of video / audio data recorded in each frame)).

Recovery method

end. Verify that the file is playable in the recoverable player.

I. If the file is damaged and cannot be played, check to see if a free box is present at the beginning of the file (before the mdat box) and if the data is stored in a predefined format.

All. Generates some of the information needed to create the moov box from the data stored in the free box.

la. Examine the mdat box to determine the end of valid frame data. Compute the size of the mdat box up to the valid frame data. That is, the last valid frame may be found using additional information (stream id, frame type (I-frame, P-frame), time stamp, size) of video / audio frame data included in the mdat box. At this time, the stream id, frame type, time stamp, and size must be checked first. Find the last frame using the size of the valid additional information. If the end of the frame data obtained using size is larger than the current size of the damaged file, the frame data being checked is regarded as inappropriate data and treated as the last frame of the mdat box.

hemp. Duration: The duration of each track is calculated using the difference between the time stamp of the last frame of each track and the time stamp of the first frame, identified by stream ID. The duration of each track uses the duration calculated for each track, and the duration of the MP4 file (the duration of the mvhd box) uses the larger value of the duration of each track as the duration of the MP4 file.

bar. stbl box (sample table box): This box exists for each track in the moov box. It uses std box, stsz box, stsc box, stco box, stss box by using the stream id, time stamp, size, and frame type already stored in the mdat box. And so forth. The method for creating each box follows the ISO / IEC 14496-12 ISO base media file format. Using the stream information stored in the free box, configure the rest of the data needed to close the file normally.

four. Record the moov box using the configured data, calculate the size of the mdat box and close the file so that it is a normal file.

Claims (12)

MP4 file organization and restoration system for converting, saving and restoring video / audio data (media data) to MP4 file format according to ISO / IEC 14496-12:
A media acquisition unit for acquiring media data generated by a camera or a microphone;
Necessary for dividing the media data into predetermined sample units, adding additional information indicating at least a time stamp and a size of the sample to each sample, recording the data in a mdat box as a frame, and organizing the media data into an MP4 file format. Generate and record the metadata in a memory; and record at least a part of the metadata in a specific box included in an MP4 file, and store the metadata in the mdat box until the media data is obtained from the media acquisition unit. An MP4 file constructing unit for constructing a moov box as final meta data for media data and recording it in the MP4 file;
A file storage unit for storing a plurality of boxes recorded by the MP4 file configuration unit in real time as one MP4 file;
An MP4 file obtaining unit obtaining the MP4 file stored in the file storing unit; And
Identifying the last valid frame of the mdat box using the additional information recorded for each frame in the corrupted MP4 file, and using the information stored in the specific box and the time stamp and size of the identified last valid frame. Compute the total play time of the media data, generate metadata up to the last valid frame using the additional information and the calculated total play time, and reconstruct the moov box with the generated metadata to corrupt the damaged MP4. Includes a file recovery unit for closing the damaged MP4 file to a normal file by adding to the file,
The additional information further includes a stream ID of the frame,
The last valid frame is the last frame in which the additional information is completely recorded.
And the total reproduction time is determined using a time difference between the time stamp of the last valid frame and the time stamp of the first frame. The method of claim 1,
The metadata includes at least a track-specific format of the media data (at least in the case of video data, a video codec and a video resolution, and in the case of audio data, at least audio codec, audio channel number, sampling rate and sampling bit number). At least one of the additional information (including at least one of a stream ID, a frame type (I-frame or P-frame), a start position offset, a time stamp, and a size) for each frame of the media data; MP4 file organization and restoration system comprising the total duration of each track (duration). delete The method of claim 1,
And the specific box is at least one of a free box, a skip box, a udta box, or a box that can be ignored when parsing an MP4 file. The method of claim 1,
And the specific box is added in front of the mdat box in the configured MP4 file. delete delete delete Image storage method for converting video / audio data (media data) to MP4 file format according to ISO / IEC 14496-12, saving and restoring:
Acquiring media data generated by a camera or microphone;
Dividing the media data into predetermined sample units and adding additional information indicating at least a time stamp and a size of the sample for each sample to record in a mdat box as a frame;
Generating and recording meta data necessary for organizing the media data into an MP4 file format in a memory, and recording at least a part of the meta data into a specific box included in the MP4 file;
When the acquisition of the media data from the media acquiring unit is completed, forming a moov box with the final metadata for the media data stored in the mdat box and recording it in the MP4 file;
Acquiring a corrupted MP4 file recorded in the MP4 file;
Identifying the last valid frame of the mdat box using the additional information recorded for each frame in the corrupted MP4 file, and using the information stored in the specific box and the time stamp and size of the identified last valid frame. Calculating a total play time of media data and generating metadata up to the last valid frame using the additional information and the calculated total play time; And
Reconstructing the moov box with the generated metadata and adding the damaged MP4 file to the damaged MP4 file, thereby closing the damaged MP4 file with a normal file.
The additional information further includes a stream ID of the frame,
The last valid frame is the last frame in which the additional information is completely recorded.
And the play time is determined using a time difference between the time stamp of the last valid frame and the time stamp of the first frame. 10. The method of claim 9,
The metadata includes at least a track-specific format of the media data (at least in the case of video data, a video codec and a video resolution, and in the case of audio data, at least audio codec, audio channel number, sampling rate and sampling bit number). At least one of the additional information (including at least one of a stream ID, a frame type, a start position offset, a time stamp, and a size) for each frame of media data, and a total duration of each track of the media data. MP4 file storage and recovery method comprising the. 10. The method of claim 9,
The specific box is at least one of a free box, a skip box, a udta box, or a box that can be ignored when parsing an MP4 file, which is added before the mdat box in the configured MP4 file. Way. delete

Images