JP2006072835A - Device and method for generating/recording multimedia file - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To control a recording end time after long time photographing constant and short regardless of a recording time. <P>SOLUTION: A multiplexing part 202 is composed of: a multimedia data generation control part 101 for generating meta data such as the header information of data to be inputted; a variable data file generating part 102 for generating a plurality of variable data files #1 to #N, and for storing encoded media data and meta data generated by the multimedia data generation control part 101 in those generated plurality of files; and a file connection processing part 103 for connecting the plurality of files #1 to #N generated by the variable data file generating part 102. The media data and a variable length section in the meta data are outputted in parallel as different files, and connected by the management information operation of the file management system when recording ends. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a multimedia file generation / recording apparatus and a generation / recording method thereof, and more particularly to a multimedia file generation / recording apparatus of an information processing apparatus used for multimedia processing, moving image processing, and the like, and a generation / recording method thereof. About.

  More specifically, in particular, a multimedia file in which media data (actual data portion) represented by MPEG-4 file format (hereinafter referred to as MP4) and metadata (reproduction information portion, header portion) are completely separated. The present invention relates to a device for generating / recording data.

  It has become common to store and record AV (Audio / Video) data by a recording apparatus typified by a digital video camera. As a format for saving AV data, there are a DV format stored in a DV cassette, an AVI format that can be selected when an input moving image is captured on a PC, and the like. In recent years, ITU-T H.264, represented by a TV phone. AV data transmission by H.324 is also performed.

  One of these multimedia data storage formats is MP4 (ISO / IEC 14496-1: 2001 / Amd6, “Information technology-Coding of audio-visual objects-Part 1: Systems-Amendment 6: MP4, the MPEG-4 file format”). As a feature of the MP4 format, media data (actual data portion) and metadata (reproduction information portion, header portion) are completely separated on the file.

  Because of this property, there is a drawback that the metadata size cannot be determined unless it is at the end of recording, and it takes time for the recording end processing.

  In order to solve the above drawbacks, a file conversion system as shown in Patent Document 1 below has been proposed.

  As shown in FIG. 9, the file conversion method disclosed in Patent Document 1 predicts the size of the metadata in advance between the metadata storage area 901 and the media data storage area 902, and free data storage area. The free data storage area 903 is filled with free data in advance.

When the data in the metadata section increases when the file is converted, the metadata 905 which is the file header information after conversion is recorded in the metadata storage area 901 and the free data storage area 903, so that the media data The occurrence of re-copying of media data in the storage area 902 is suppressed, and the recording processing time and memory consumption are reduced.
JP 2003-173625 A (Claim 1, FIG. 17)

  However, the method disclosed in Patent Document 1 is effective when the metadata size can be predicted, but the upper limit of the metadata size cannot be determined in long-time moving image shooting in which the maximum recording time is not set. The method disclosed in Patent Document 1 cannot be used.

  Also, as described in Patent Document 1, when an MP4 file is generated by finally determining metadata and media data, file copy / deletion processing is required, and the meta data whose file size increases with recording time is described. Data or media data is written last, and there is a problem that it takes a recording end processing time.

  The present invention has been made in view of such a point, and the recording end time (time until the next file can be shot) after long-time shooting can be kept constant and short regardless of the recording time. It is an object of the present invention to provide a user-friendly multimedia file generation / recording apparatus and a generation / recording method thereof that do not cause stress.

  The multimedia file generating / recording apparatus of the present invention includes multimedia data including media data including video data and audio data, and metadata including information necessary for completely reproducing the media data and reproducing the media data. In the multimedia file generating / recording device for generating a file, one or more first files for recording the media data and one or more second files for recording metadata are generated. A file generating means, a data recording means for recording data in parallel in the first file and the second file, and the first file and the second file are combined at the end of recording of the recorded file; And a file combination means for generating one multimedia file. That.

  Also, the multimedia file generating / recording method of the present invention includes multimedia data including media data including video and audio, and metadata including information necessary for completely reproducing the media data and reproducing the media data. A multimedia file generation / recording method for generating a file, wherein a first file for recording the media data and a plurality of variable length second files for recording metadata are respectively generated. 1 step, a second step of recording data in parallel in the first file and the second file, and at the end of recording of the recorded file, the first file and the plurality of second files And a third step of generating one multimedia file by combining the two.

  According to the present invention, if the multimedia file generating / recording apparatus and method of the present invention is used in a device having a moving image shooting function, media data and metadata are separated as a moving image recording format. For example, MP4 Even if the format is adopted, the recording end time after shooting for a long time (time until the next file can be shot) can be kept constant and short regardless of the recording time, making it less user-friendly A multimedia file generating / recording apparatus and a method for generating / recording the same can be provided.

  Embodiments of the present invention will be described below in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a multiplexing unit of a multimedia file generation / recording apparatus according to an embodiment of the present invention, and FIG. 2 shows an overall configuration of the multimedia file generation / recording apparatus of the present invention. FIG.

  In the following, the video is converted into MPEG4 Video (ISO) as multimedia file data including media data including video and audio data and metadata including information necessary for reproducing the media data completely separated from the media data. / IEC 14496 part-2: Visual), and the audio is MPEG4 AAC (Moving Picture Experts Group phase 4 Advanced Audio Coding).

  In FIG. 2, an encoding unit 200 compresses video data input in real time and encodes it into the MPEG4 Video format or the like. The encoding unit 201 compresses audio data input in real time and encodes it into MPEG4AAC or the like. The multiplexing unit 202 multiplexes the data output from the encoding unit 200 and the encoding unit 201 into the MP4 format to generate multiplexed data. The output unit 203 outputs the multiplexed data from the multiplexing unit 202 as a file to a memory or sends it to a network.

  The detailed configuration of the multiplexing unit 202 is shown in FIG. As shown in FIG. 1, the multiplexing unit 202 generates a multimedia data generation control unit 101 that generates metadata such as header information of input data, and a plurality of variable data files # 1 to #N. A variable data file generation unit 102 that stores encoded media data and metadata generated by the multimedia data generation control unit 101 in a plurality of generated files, and a plurality of data generated by the variable data file generation unit 102. The file combination processing unit 103 combines files # 1 to #N.

  Next, the file combination processing unit 103 used in the present invention will be described.

  Generally, a file management system that manages files on a computer has the following functions.

(1) Simultaneous opening of multiple files and out-of-order write processing (2) Join processing in cluster units of two files by management information control of the file management system With the function of (1) above, variable information between media data and metadata Multiple files can be opened at the same time, and data can be written in parallel to create a file. Also, the function (2) allows the files created in parallel in (1) to be combined and converted into one file in cluster units without copying the files. A cluster is the minimum size for handling files.

  Therefore, in the present invention, the files are combined according to the management information operation of the file management system that manages the multimedia files of the present invention.

  Next, the operation of the multimedia file generating / recording apparatus having the above configuration will be described with reference to FIGS. 4 to 7, MP4MUX corresponds to the multiplexing unit 202.

  Here, the data structure of MP4 will be described with reference to FIG.

  The MP4 file is composed of basic units called “Box”. A Box that stores the entire MP4 metadata is a Movie Box. In FIG. 3, 301 corresponds to the Movie Box, and in the present invention, it is expressed as moov.

  A box for storing the entire media data is a media data box. In FIG. 3, 302 corresponds to the media data box, and is indicated as mdat in the present invention. In addition to the essential boxes described above, there are also MP4 file formats that can be freely defined by the user.

  In Movie Box, for example, parameters such as stts, stsc, stsz, stco, stss for accessing desired data in mdat are used, but they are not directly related to the present invention. Detailed description will be omitted.

  First, the recording start preparation process will be described (FIG. 4).

  (A) As a preparation process for MP4 recording, the multimedia data generation control unit 101 determines items to be variable data based on characteristics of a recording medium or the like (in this example, nine items are necessary). .

  (B) In accordance with the number of variable data items, necessary functions are assigned to the variable data file generation unit 102 to create files (File # 1 to File 9). Here, media data is stored in File # 1, and metadata is stored in File # 2-9.

  (C) Write fixed value information (recording media information, etc.) known before starting recording, and finish the preparation process for starting recording.

  Next, recording will be described (FIG. 5).

  During recording, the following processing is performed on the video and audio data input from the encoding units 200 and 201 via the variable data file generation unit 102 in real time.

  (A) The encoded video / audio data (VOP, frame) is written in mdat (File # 1).

  (B) Stts, stsc, stsz, stco, stss information (File # 2-9) is additionally written from the time stamp, size, file position, and key flag of each video / audio data.

  (C) At this time, the unit of writing to each file is written in a fixed amount unit (for example, 16 KB unit) in order to increase the writing efficiency.

  Next, the end of recording will be described (FIG. 6).

  At the end of recording, input video or audio data is determined, and the following processing is performed via the variable data file generation unit 102.

  (A) The information of the unprocessed variable length file (File # 1 to 9) is determined.

  (B) Update information (box size information, playback time) that is not known unless it is last.

  (C) Write uuidBox to File # 1-8 files (excluding the file to be combined at the end) so that each is a multiple of 16KB (uuid is defined as a box that can be determined by the user in the MP4 standard) ing).

  In this embodiment, it is assumed that the file system is handled in a cluster of 16 KB.

  (D) If there is udta information (such as a title) for File # 9, it is written. Since File # 9 is the end of the file, 16KB cluster alignment is not necessary.

  Finally, recording file confirmation will be described (FIG. 7).

  When the recording file is confirmed, processing is performed via the file combination processing unit 103.

  (A) The files # 1 to # 9 are combined in order by changing the management information of the file management system, and determined as an MP4 file 701.

  In the present invention, when files are combined, the files are combined only by file system management information operations, and the existing information in the variable-length file is updated according to the size of free data and the combination order. .

  FIG. 8 is a flowchart showing the operation of the multimedia file generation / recording apparatus according to the present embodiment. As shown in FIG. 8, the operation of the multimedia file generation / recording apparatus according to the present embodiment includes a first file # 1 for recording media data and a plurality of variable lengths for recording metadata. A first step (ST801) for generating the second files # 2 to # 9, respectively, and a second step for recording data in parallel in the first file # 1 and the second files # 2 to # 9 (ST802) and at the end of recording of the recorded file, the first file # 1 and the plurality of second files # 2-9 are combined to generate one multimedia file (MP4 file 701). 3 process (ST803).

  Next, when the above processing is performed, the processing time required for file confirmation from the end of recording is estimated as follows.

  As an estimation condition, the performance of the file management system was calculated assuming that a write operation requires 20 ms for one write within 16 KB, and a join operation requires 150 ms regardless of the file size.

(*1)再生時間情報は、mvhd, tkhd(V), mdhd(V), tkhd(A), mdhd(A)の５つである。
(*2)uudiBox内情報は、無視されるためSize+Typeの８バイトのみ書き込む。

(* 1) There are five playback time information: mvhd, tkhd (V), mdhd (V), tkhd (A), mdhd (A).
(* 2) Since the information in uudiBox is ignored, only 8 bytes of Size + Type are written.

  As described above, the recording end process is completed in less than 2 seconds regardless of the recording time. However, when recording is performed for a long time using the same file management system, for example, the metadata is finally transferred to the media data. Assuming that 10M bytes of metadata are generated after writing, the writing time (msec) is 10240/16 × 20 = 1900, which takes a little over 13 seconds, and 10 MB of metadata is once filed. If it is expanded above, the Read operation and the Write operation to the file system occur alternately, and the processing time becomes longer.

  As described above, according to the embodiment of the present invention, even when recording a multimedia file for a long time without defining a maximum recording time, the data copy (Write) process is reduced during the recording end process, At the same time, the recording end processing time is constant in a short time, and recording conforming to the MP4 standard can be performed.

  In addition, even when recording is performed using a file management system, file copying does not occur, and it does not depend on the size of the variable-length file, and the joining process time is constant. The processing time can be shortened within a certain time.

  In addition, even if each variable length file is handled in cluster units of the file management system, the file can be played back on an existing multimedia file playback device in accordance with the multimedia file format standard.

  The above embodiment has been described on the assumption that MPEG4 Video + MPEG4AAC MP4 recording is used. However, when a different codec is used, or a file format in which the same metadata and media data as MP4 are separated. The above can also be applied in the same manner as described above.

  In addition, when the multimedia file generating / recording device of the present invention is installed in a communication terminal device having a moving image shooting function or a mobile communication terminal device, the shot moving image can be transmitted to more devices, and more It becomes possible to view with the device, and usability is improved.

  The present invention is particularly suitable for use in data conversion / distribution of multimedia files.

The block diagram which shows the structure of the multiplexing part of the multimedia file production | generation / recording apparatus based on one embodiment of this invention The block diagram which shows the structure of the multimedia file production | generation / recording apparatus based on the said embodiment of this invention The figure which shows the data structure of MP4 which concerns on the said embodiment of this invention The figure which shows the image at the time of the MP4 recording start which concerns on the said embodiment of this invention The figure which shows the image in MP4 recording which concerns on the said embodiment of this invention The figure which shows the image at the time of the end of MP4 recording which concerns on the said embodiment of this invention The figure which shows the MP4 file image after completion | finish of MP4 recording based on the said embodiment of this invention The flowchart which shows operation | movement of the multimedia file production | generation / recording apparatus based on the said embodiment of this invention. Diagram explaining the prior art

Explanation of symbols

101 multimedia data generation control unit 102 variable data file generation unit 103 file combination processing unit 200, 201 encoding unit 202 multiplexing unit 203 output unit

Claims (9)

In a multimedia file generation / recording apparatus for generating a multimedia file including media data including video and audio data and metadata including information necessary for completely reproducing the media data and reproducing the media data,
File generating means for generating one or more first files for recording the media data and one or more second files for recording metadata;
Data recording means for recording data in parallel in the first file and the second file;
File combination means for combining the first file and the second file to generate one multimedia file at the end of recording of the recorded file;
A multimedia file generating / recording apparatus comprising:   The file combining means converts the first and second files output in parallel at the end of recording into a file of a cluster or a multiple of a cluster of a file management system, 2. The multimedia file generating / recording apparatus according to claim 1, wherein the files are combined in accordance with a management information operation of a file management system that manages the multimedia files.   3. The multimedia file according to claim 2, wherein when the files are combined, the files are combined after inserting data defined in the format of the multimedia file or free data defined uniquely. Generator / recorder.   When combining the files, the files are combined only by a management information operation of the file system, and the existing information in the first and second files is updated according to the size and order of the free data. The multimedia file generating / recording apparatus according to claim 2. A multimedia file generation / recording method for generating a multimedia file including media data including video data and audio data, and metadata including information necessary for completely reproducing the media data and reproducing the media data There,
A first step of generating a first file for recording the media data and a plurality of variable length second files for recording metadata;
A second step of recording data in parallel in the first file and the second file;
A third step of combining the first file and the plurality of second files to generate one multimedia file at the end of recording of the recorded file;
A multimedia file generating / recording method comprising:   In the third step, at the end of recording, the second file that is the variable-length file is changed to a file of a cluster or a multiple of a cluster of the file management system, and the file of the cluster or a multiple of the cluster is 6. The multimedia file generating / recording method according to claim 5, wherein the files are combined according to management information operation of a file management system that manages the media files.   7. The multimedia file according to claim 6, wherein when the files are combined, the files are combined after inserting data defined in the format of the multimedia file or free data defined uniquely. Generation / recording method.   When combining the files, the files are combined only by a management information operation of the file system, and the existing information in the first and second files is updated according to the size and order of the free data. The multimedia file generating / recording method according to claim 6.   An information processing apparatus having a moving image shooting function, comprising the multimedia file generation / recording apparatus according to claim 1.

Images