JP2007159058A - Recording apparatus and recording method, and reproduction apparatus and reproduction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize techniques for increasing data access speed, while reducing the memory capacity necessary for reproducing or searching for image data or audio data recorded as a file. <P>SOLUTION: An apparatus recording image data and audio data as a file includes an image data encoding means, an image data code amount control means, an audio data encoding means, an audio data code amount control means, an image stream generating means, an audio stream generating means, a multiplexing means for multiplexing the generated image stream and audio stream, and a recording means for recording a stream multiplexed by the multiplexing means on a recording medium, wherein the image data code amount control means and the audio data code amount control means control the image data encoding means and the audio data encoding means so as to fix the lengths of the image stream and the audio stream for each unit to be multiplexed by the multiplexing means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for recording and reproducing moving image data using interframe coding and audio data using block coding.

  In recent years, devices such as digital video cameras and digital cameras that record and reproduce moving image data by encoding using inter-frame predictive encoding such as MPEG have become widespread.

  In addition, MPEG (MPEG2 / MPEG4) moving image data and audio data shot by a digital video camera, a digital camera, or the like are recorded in a general-purpose file format called MP4. By recording these data as MP4 files, compatibility that can be reproduced by other devices is guaranteed. MP4 is an abbreviation for MPEG audio Layer-4.

  The MP4 file includes an mdat box containing encoded image data and audio data streams and a moov box containing information about each stream. The mdat box is further composed of chunks, for example, a moving image chunk and an audio chunk.

  The moov box stores the number of offset bytes from each head of the file to each chunk, the size, the number of samples, and the like.

  When playing back an MP4 file, the moov box of the MP4 file is read from the recording medium, and the chunk of moving image data and the chunk of audio data can be accessed based on the moov box.

  However, the size of moov box data increases in proportion to the number of chunks. In other words, the longer the recording time, the larger the size, and in order to read all the moov box data, which is the access information to the stream, into a portable small photographic device such as a digital video camera or digital camera. Requires large memory.

  In order to deal with such a problem, a technique has been proposed in which information for accessing a stream is selected separately from the data in the moov box and is recorded as separate data and recorded as a separate file together with the information linked to the MP4 file. . According to this technology, the MP4 stream data is accessed from the separate file at the time of reproduction, thereby reducing a memory for developing access information to the stream at the time of reproduction (for example, see Patent Document 1).

However, even if the access information to the stream data in the mdat box is selected as described above and the access information is reduced, it does not change as the recording time increases. Further, since the access data is recorded as a separate file, it cannot be used when the MP4 file is copied to another recording medium.
JP 2004-128938 A

  As described above, when an MP4 file is reproduced, there is a problem that a large memory capacity is required to store access information to the data stream.

  Further, when access information to a data stream in the moov box cannot be taken in from a single MP4 file recorded on the recording medium at a time, the following measures must be taken. In other words, as the MP4 file data is played back, when the break of the read access information approaches, the access information read first is discarded, and the access information following the break is sought at the head of the file and read from the moov box. Must get. However, there is a problem that seamless playback in real time becomes impossible when a file with a high recording rate or a file with a long recording time is played back, or when a special playback (search) is performed.

  The present invention has been made in view of the above problems, and an object of the present invention is to realize a technique capable of reducing the memory capacity required for reproducing or searching image data or audio data recorded as a file and speeding up data access. It is.

  In order to solve the above-mentioned problems and achieve the object, the present invention provides an image data encoding means for encoding image data and an image data encoding means in an apparatus for recording image data and audio data as a file. Image data code amount control means for controlling, audio data encoding means for encoding audio data, audio data code amount control means for controlling the audio data encoding means, and image data encoding means An image stream generating means for streaming the image data, an audio stream generating means for streaming the audio data encoded by the audio data encoding means, and the image data and audio data for a predetermined period, respectively. The generated image in units of A multiplexing unit that multiplexes the stream and the audio stream; and a recording unit that records the stream multiplexed by the multiplexing unit on a recording medium. The image data code amount control unit and the audio data code amount control unit include: The image data encoding means and the audio data encoding means are controlled so that the image stream and the audio stream are each fixed-length for each unit multiplexed by the multiplexing means.

  The present invention also provides a reproducing means for reproducing image data and audio data recorded as a file on a recording medium, a reproduction control means for controlling the reproducing means, and analyzing header information recorded on the recording medium. Header information analysis means, and when the header information analysis means detects unique user data in the header recorded on the recording medium, the access information of the stream recorded in the header Reading is prohibited, and the reproduction control unit controls the reproduction unit based on the parameter information of the stream recorded in the file analyzed by the header information analysis unit, and is recorded as a file on the recording medium. Play back image data and audio data.

  The present invention relates to a method for recording image data and audio data as a file, an image data encoding control step for controlling encoding of image data, and an audio data encoding control step for controlling encoding of audio data. An image stream generation step for streaming the encoded image data, an audio stream generation step for streaming the encoded audio data, and the image data and audio data for a predetermined period, respectively. The unit includes a multiplexing step of multiplexing the generated image stream and audio stream, and a recording step of recording the multiplexed stream on a recording medium, and the image data encoding control step and the audio data code In the multiplexing control process, multiplexing is performed in the multiplexing process. The encoding said image data and the audio data to each fixed length image and audio streams for each that unit.

  The present invention also provides a playback method for playing back image data and audio data recorded as files on a recording medium, the header information analyzing step for analyzing header information recorded on the recording medium, and the header When unique user data is detected in the header recorded in the recording medium by the information analysis process, reading of access data to the stream recorded in the header is prohibited and analyzed in the header information analysis process. A reproduction step of reproducing the image data and the audio data recorded on the recording medium based on the parameter information of the stream recorded in the recorded file.

  According to the present invention, it is possible to reduce the memory capacity required for reproduction or search of image data and audio data recorded as a file, and to speed up data access.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

[Description of recording device]
FIG. 3 is a block diagram showing the configuration of the recording apparatus according to the embodiment of the present invention.

  In FIG. 3, reference numeral 301 denotes an image data input terminal. Reference numeral 302 denotes an image encoding circuit. A multiplexing circuit 303 multiplexes streams of images, data, and audio data. Reference numeral 304 denotes a recording circuit. Reference numeral 305 denotes an audio data input terminal. Reference numeral 306 denotes an audio encoding circuit. Reference numeral 307 denotes a file header addition circuit. Reference numeral 308 denotes an input terminal for parameters such as image quality during recording. Reference numeral 309 denotes a code amount setting circuit. Reference numeral 310 denotes a chunk configuration setting circuit. Reference numeral 311 denotes a recording medium.

  In this embodiment, moving image data and audio data are recorded and reproduced in the MP4 file format.

  Here, the configuration of the MP4 file will be described.

  The MP4 file is based on the Quicktime (registered trademark) file format. As shown in FIG. 9, the Quicktime (registered trademark) file format is composed of basic units called boxes (or atoms).

  As shown in FIG. 9, the box includes a 4-byte size field indicating the size of the box, a 4-byte type field indicating the type of the box, and a subsequent data field.

  As shown in FIG. 7A, the MP4 file basically includes an mdat box containing encoded image data and audio data streams, and a moov box containing information about each stream.

  The mdat box is further composed of chunks as shown in FIG. 7B. In this example, the video chunk v (n) and the audio chunk a (n) It consists of.

  Each chunk of moving image and audio is further composed of samples as shown in FIGS. 7 (d) and 7 (f). FIG. 7D shows that chunk v (1) of the moving image is composed of samples sv (1), sv (2),..., Sv (K). FIG. 7F shows that audio chunk a (1) is composed of samples sa (1), sa (2),..., Sa (M).

For example, in the case of a moving image, each sample is I _{0 with} respect to samples sv (1), sv (2), sv (3), s (4),... As shown in FIG. , B ₋₂ , B ₋₁ , P ₃ ,... Encoded MPEG streams correspond. Where I _n is the frame image data that has been intra-coded (intraframe coding). B _n is frame image data which is encoded (inter-frame encoding) with reference from both directions. P _n is frame image data that is encoded (inter-frame encoded) with reference from one direction (forward direction). The image data is variable length data.

  Each sample of audio data is associated with a frame, which is a unit for encoding audio data, as shown in FIG.

  As shown in FIG. 7C, the moov box in FIG. 7A has an mvhd box made up of header information into which the creation date and time is entered. Also, the moov box has a trak box (video) and a trak box (audio) for moving images and audio into which information of stream data stored in the mdat box is stored.

  The box stored in the trak box is a box as shown in FIG. 8A. Here, only the stsc, stsz, and stco boxes related to the present invention will be described.

  As shown in FIG. 8B, the stco box stores an offset address value for each chunk of the stream. FIGS. 8A and 8B show only the stco box of the moving image, but one stco box exists for each track of the moving image and audio.

  As shown in FIG. 8C, the stsz box stores information indicating the size of each sample of the stream. FIGS. 8A and 8C show only the stsz box of the moving image, but one stsz box exists for each track of the moving image and the audio.

  As shown in FIG. 8D, the stsc box stores information indicating the number of samples of each chunk. FIGS. 8A and 8D show only the stsc box of the moving image, but there is one stsc box for each track of the moving image and the audio.

  Note that the data stored in the boxes of stco, stsz, and stsc increase with the recording time.

  For example, assuming that an image of 30 frames / second per second is stored in one chunk every 15 frames, and recorded for one hour, the total number of chunks of moving images is approximately 2 × 60 × 60 = 7200.

  The stco box that stores offset address information for each chunk is 2 × 60 × 60 × 4 = 28800 (bytes), and requires a data area of about 28800 bytes (about 28 kbytes).

  The stsz box for storing information indicating the size of each sample is 30 × 60 × 60 × 4 = 432000 (bytes), and requires a data area of about 432000 bytes (about 422 kbytes).

  The stsc box that stores information indicating the number of samples of each chunk is 2 × 60 × 60 × 12 = 86400 (bytes), and requires a data area of approximately 86400 bytes (approximately 84 kbytes).

  As for audio data, for example, audio data is compressed using AAC. The sampling frequency is 48 kHz, and the frame size (sample size) is 1024 audio samples. Assuming 46 frames (samples) per chunk (about 1 second), the total number of audio chunks when recording for 1 hour is approximately 48000 × 60 × 60 / (1024 × 46) ≈3668.

stco box is 3668 × 4 = 14672 (byte)
stsz box is 46 × 3668 × 4 = 674912 (byte)
The stsc box is 3668 × 12 = 44016 (byte), which is about 14 kbytes, 659 kbytes, and 42 kbytes, respectively.

  When recording as an MP4 file from the above, the capacity of a moov box of 1.2 Mbytes or more is required in one hour.

[Description of recording and playback operations]
First, a specific recording operation will be described.

  Depending on the setting of the operator, a recording parameter for image quality or a parameter for a desired recording time to the recording medium 311 is input to the code amount setting circuit 309 from the input terminal 308.

  Further, information on the chunk configuration of the image and audio data is sent from the chunk configuration setting circuit 310 to the code amount setting circuit 309, the image encoding circuit 302, the audio encoding circuit 306, and the header addition circuit 307.

  Here, for example, as shown in FIGS. 7D and 7F, the chunk configuration setting circuit 310 compresses the moving image data with MPEG2 or MPEG4, and uses the K frame as one chunk of the moving image data. The audio data is compressed by MPEG2-AAC, and setting information is sent to the code amount setting circuit 309, the image encoding circuit 302, the audio encoding circuit 306, and the header addition circuit 307 so as to form one chunk with M frames. In the present embodiment, one chunk of moving image data has a frame number that is an integral multiple of GOP (Group of Picture) in MPEG encoding.

  The code amount setting circuit 309 determines the code amount per chunk of image and audio data from the parameters input from the input terminal 308 and the chunk configuration information supplied from the chunk configuration setting circuit 310.

  For example, as shown in FIG. 5A, the code amount of one chunk of image data is VL, the code amount of one chunk of audio data is AL, and the total code amount VL + AL = Lfix of one chunk of image data and one chunk of audio data Decide. In FIG. 5A, 1 GOP is 15 frames, and 1 chunk is 2 GOPs, that is, K = 30 frames.

  Lfix is set as a fixed value at least from recording start to recording stop.

  The code amount setting circuit 309 sets the determined image data code amount VL and audio data code amount AL per chunk in the image encoding circuit 302, the audio encoding circuit 306, and the header addition circuit 307.

  Next, when recording is started, the image encoding circuit 302 and the audio encoding circuit 306 start encoding image data and audio data.

  Here, the details of the image encoding circuit 302 will be described with reference to FIG.

  In FIG. 1, reference numeral 101 denotes an image data input terminal. Reference numeral 102 denotes a camera signal processing circuit. Reference numeral 103 denotes a rearrangement circuit that rearranges the frames in order for encoding. Reference numeral 104 denotes an adder circuit. Reference numeral 105 denotes a switch circuit. Reference numeral 106 denotes a DCT circuit. Reference numeral 107 denotes a quantization circuit. Reference numeral 108 denotes an inverse quantization circuit. Reference numeral 109 denotes an inverse DCT circuit. Reference numeral 110 denotes an adder circuit. Reference numeral 111 denotes a memory. Reference numeral 112 denotes a motion compensation circuit. Reference numeral 113 denotes a switch circuit. Reference numeral 114 denotes a variable length coding circuit. Reference numeral 115 denotes a stream generation circuit. Reference numeral 116 denotes a buffer. Reference numeral 117 denotes a code amount control circuit. Reference numeral 118 denotes an output terminal for stream data. Reference numeral 119 denotes a stream parameter addition circuit. Reference numeral 120 denotes a header information generation circuit. Reference numeral 121 denotes an output terminal for information to be added to the header. Reference numeral 122 denotes a code amount setting circuit. Reference numeral 123 denotes a recording control circuit. Reference numeral 124 denotes a control data input terminal to the recording control circuit.

  The code amount setting circuit 309 and the code amount VL per chunk, which are parameters related to recording, and the parameter K indicating the chunk configuration are input from the input terminal 124. The recording control circuit 123 supplies the input code amount VL per chunk and the parameter K indicating the chunk configuration to the code amount setting circuit 122. The code amount setting circuit 122 calculates the number of GOPs gn per chunk, sets it to the code amount control circuit 117 so that the number of GOPs is gn and the code amount VL, and supplies it to the stream parameter addition circuit 119 as well.

  When recording is started, image data captured by an image sensor (not shown) is input from the input terminal 101, processed by the camera signal processing circuit 102, and output to the rearrangement circuit 103 as a luminance signal / color difference signal. The

The rearrangement circuit 103, and FIG. 5 (b) camera signals with respect to the time axis t as shown in the processing circuit 102 each frame image from _{F -2, F -1, F 0} , F 1, F 2, F 3, F 4 , F ₅ , F ₆ ,... Are output. Then, the rearrangement circuit 103 rearranges these image frames in the order of F ₀ , F ₋₂ , F ₋₁ , F ₃ , F ₁ , F ₂ , F ₆ , F ₄ , F ₅ ,. The rearrangement circuit 103 performs a process of dividing the image data in the frame into small blocks of units to be encoded, and outputs the result to the addition circuit 104 and the switch circuit 105.

When the recording control circuit 123 starts recording the terminal of the switch circuit 105, the recording control circuit 123 first instructs the terminal a to select the terminal a. The switching circuit 105 selects the terminal a by the recording control circuit 123 and sends the frame image F ₀ to the DCT circuit 106. To do. The DCT circuit 106 performs a discrete cosine transform (hereinafter referred to as “DCT”) on the frame image F ₀ and sends it to the quantization circuit 107.

  The quantization circuit 107 quantizes the DCT-processed data supplied from the DCT circuit 106 by the quantization process set by the code amount control circuit 117, and sends it to the variable-length encoding circuit 114 and the inverse quantization circuit 108. .

The variable length coding circuit 114 performs variable length coding on the quantized image data supplied from the quantization circuit 107 using a Huffman code or the like. Then, as shown in FIG. 5B, the variable length encoding circuit 114 sends the frame image F ₀ to the stream generation circuit 115 as intra-frame encoded data I ₀ .

  The inverse quantization circuit 108 inversely quantizes the quantized image data supplied from the quantization circuit 107 and sends it to the inverse DCT circuit 109. The inverse DCT circuit 109 performs inverse discrete cosine transform (inverse DCT) on the image data inversely quantized by the inverse quantization circuit 108 and sends it to the adder circuit 110.

  The adder circuit 110 adds the data supplied from the switch circuit 113 to the image data subjected to the inverse DCT supplied from the inverse DCT circuit 109 and supplies it to the memory 111. Here, the switch circuit 113 selects the terminal a in accordance with an instruction from the recording control circuit 123, and supplies the “0” data to the adder circuit 110. Therefore, the adder circuit 110 supplies the same value as the data supplied from the inverse DCT circuit 109 to the memory 111, and the memory 111 stores the supplied image data.

When the processing of the frame image F ₀ is completed, the image data of the frame image F _-2 is continuously input from the rearrangement circuit 103 to the addition circuit 104 and the motion compensation circuit 112. At this time, the switch circuit 105 selects the terminal b in accordance with an instruction from the recording control circuit 123. Further, the recording control circuit 123 controls the motion compensation circuit 112. Since the motion compensation circuit 112 is at the start of recording, referring to only the frame image F ₀ as shown in FIG. 5, the block of the frame image F _{0 with} the smallest prediction error is read from the memory 111 and supplied to the adder circuit 104. .

The adder circuit 104 subtracts the image data of the frame image F- ₂ supplied from the rearrangement circuit 103 and the data supplied from the motion compensation circuit 112, and supplies the result to the switch circuit 105 as prediction error data.

  The switch circuit 105 sends the subtracted prediction error data supplied from the adder circuit 104 to the DCT circuit 106. The DCT circuit 106 performs DCT processing on the prediction error data supplied from the adder circuit 104 and supplies it to the quantization circuit 107.

  The quantization circuit 107 quantizes the DCT-processed prediction error data supplied from the DCT circuit 106 by the quantization process set by the code amount control circuit 117, and sends it to the variable-length coding circuit 114.

The variable length coding circuit 114 performs variable length coding on the quantized prediction error data supplied from the quantization circuit 107. Then, as shown in FIG. 5, the variable length encoding circuit 114 sends the frame image F _-2 to the stream generation circuit 115 as interframe encoded data B _-2 .

The next frame image F- ₁ is processed in the same manner as the frame image F- ₂ . The processed frame image is sent from the variable length coding circuit 114 to the stream generation circuit 115 as interframe coding (bidirectional predictive coding) data B- ₁ .

Next, the frame image F ₃ is input from the rearrangement circuit 103 to the addition circuit 104 and the motion compensation circuit 112. At this time, the switch circuit 105 selects the terminal b in accordance with an instruction from the recording control circuit 123. The motion compensation circuit 112 reads the block of the frame image F ₀ having the smallest prediction error with reference to the frame image F ₀ from the memory 111 and supplies the read block to the addition circuit 104.

The adder circuit 104 subtracts the image data of the frame image F ₃ supplied from the rearrangement circuit 103 and the data of the frame image F ₀ subjected to motion compensation supplied from the motion compensation circuit 112. Further, the adder circuit 104 supplies the switch circuit 105 as prediction error data. The switch circuit 105 sends the subtracted prediction error data supplied from the adder circuit 104 to the DCT circuit 106. The DCT circuit 106 performs DCT processing on the prediction error data supplied from the adder circuit 104 and supplies it to the quantization circuit 107.

  The quantization circuit 107 quantizes the DCT-processed prediction error data supplied from the DCT circuit 106 by the quantization process set by the code amount control circuit 117, and supplies the quantized circuit to the variable-length encoding circuit 114 and the inverse quantization circuit 108. Send it out.

The variable length coding circuit 114 performs variable length coding on the quantized prediction error data supplied from the quantization circuit 107. The variable-length coding circuit 114, as shown in FIG. 5 (b), and sends the stream generating circuit 115 a frame image F3 as inter-frame coding (unidirectional predictive encoding) data P _3.

  The inverse quantization circuit 108 inversely quantizes the quantized prediction error data supplied from the quantization circuit 107 and sends it to the inverse DCT circuit 109. The inverse DCT circuit 109 performs inverse DCT processing on the prediction error data inversely quantized by the inverse quantization circuit 108 and sends it to the adder circuit 110.

The adder circuit 110 adds the data supplied from the switch circuit 113 to the prediction error data subjected to the inverse DCT process supplied from the inverse DCT circuit 109 and supplies the result to the memory 111. Here, the switch circuit 113 selects the terminal b according to the instruction of the recording control circuit 123 and supplies the motion compensated frame image F ₀ data supplied from the motion compensation circuit 112 to the adder circuit 110. Therefore, the adder circuit 110 adds the motion compensated data supplied from the motion compensation circuit 112 to the prediction error data supplied from the inverse DCT circuit 109 and supplies it to the memory 111. The memory 111 stores the supplied image data. To do.

Next, the frame images F ₁ and F ₂ are processed. In other words, the processing is the same as the processing of F _-2 and F _-1 except that the motion compensation circuit 112 predicts locally decoded frame images F ₀ and F ₃ from both directions. Variable length coding. Then, as shown in FIG. 5B, the frame images F ₁ and F ₂ are sent to the stream generation circuit 115 as interframe encoded data B ₁ and B ₂ .

Thereafter, the frame images F ₆ , F ₄ , F ₅ , F ₉ , F ₇ , F ₈ , F ₁₂ , F ₁₀ , F ₁₁ are processed, and P ₆ , B ₄ , B ₁₁ are processed as shown in FIG. ₅ , P ₉ , B ₇ , B ₈ , P ₁₂ , B ₁₀ , B ₁₁ are obtained and supplied to the stream generation circuit 115.

As shown in FIG. 5C, the stream generation circuit 115 generates a stream by adding a sequence header, a GOP header, a picture header, etc., and supplies the stream to the buffer 116. Then, as shown in FIG. 5 (b), the stream generation circuit 115 generates I ₀ , B ₋₂ , B ₋₁ , P ₃ , B ₁ ,..., P ₁₂ , B ₁₀ ,. forming a 1GOP _{(GOP (1)))} at B _11.

As shown in FIG. 5C, the stream parameter addition circuit 119 sends the stream data generation circuit 115 the image data code amount VL and the intra-frame encoded data I ₀ encoded size I ₀ (size) as user data. Append. Further, as shown in FIG. 5C, the stream parameter addition circuit 119 adds an offset value I ₀ (offset) up to the I frame of the next GOP as user data to the stream generation circuit 115.

  Under the control of the recording control circuit 123, the buffer 116 sends image stream data from the output terminal 118 to the multiplexing circuit 303 of FIG.

  The code amount control circuit 117 monitors the code amount of the buffer 116 according to the number of GOPs per chunk gn set by the code amount setting circuit 122 and the code amount VL per chunk so that the number of GOPs is gn and the code amount is VL. Then, the quantization processing in the quantization circuit 107 is controlled.

  The header information generation circuit 120 generates information such as the offset value to the chunk, the number of samples per chunk, and the sample size required for the header of the MP4 file from the stream generation circuit 115. Then, the header information generation circuit 120 outputs the generated information from the output terminal 121 and supplies it to the header addition circuit 307 in FIG.

  Next, audio encoding will be described with reference to FIG.

  In FIG. 2, reference numeral 201 denotes an audio data input terminal. Reference numeral 202 denotes an MDCT circuit. Reference numeral 203 denotes a TNS circuit. Reference numeral 204 denotes an intensity stereo circuit. Reference numeral 205 denotes an M / S stereo circuit. Reference numeral 206 denotes a scale factor calculation circuit. Reference numeral 207 denotes a quantization circuit. Reference numeral 208 denotes a variable length coding circuit. Reference numeral 209 denotes a stream generation circuit. 210 is an output terminal. Reference numeral 211 denotes a header information generation circuit. Reference numeral 212 denotes an output terminal. Reference numeral 213 denotes an auditory psychological model circuit. Reference numeral 214 denotes a bit allocation circuit. Reference numeral 215 denotes a recording control circuit. Reference numeral 216 denotes an input terminal.

  A code amount AL per chunk, which is a parameter relating to recording from the code amount setting circuit 309 and the chunk configuration setting circuit 310, and a parameter M indicating the chunk configuration are input to the recording control circuit 215 from the input terminal 216.

  When recording is started, audio data is input to the input terminal 201 from an A / D conversion circuit (not shown) or the like, and is supplied to the MDCT circuit 202 and the psychoacoustic model circuit 213.

  In the MDCT circuit 202, audio data input from the input terminal 201 is subjected to modified discrete cosine transform (hereinafter “MDCT”) with 1024 samples as one frame, and is output to the TNS circuit 203. The modified discrete cosine transform is an abbreviation for Modified Discrete Cosine Transform. The TNS circuit 203 performs TNS (Temporal Noise Shaping) processing and outputs it to the intensity stereo circuit 204. TNS is a process for improving sound quality by shaping quantization noise according to the amplitude value of a signal waveform. Here, a part of MDCT coefficients is regarded as a time series signal and linear prediction is performed. As a result, the quantization noise is concentrated where the amplitude of the signal waveform is large.

  Intensity stereo circuit 204 uses the auditory property of feeling the position of the sound source due to the difference in signal power between the left and right channels at high frequencies, and uses the sum signal and power ratio of the left and right channels instead of the original two-channel data. The audio data output from the intensity stereo circuit 204 is sent to the M / S stereo circuit 205.

  The M / S stereo circuit 205 uses the sum and difference signals of the left and right channels instead of the original two-channel data. Used mainly for low-frequency signal processing. The audio data output from the M / S stereo circuit 205 is supplied to the scale factor calculation circuit 206.

  The scale factor calculation circuit 206 calculates a representative value (scale factor) for each band divided into a plurality of bands (scale factor bands) in accordance with the auditory characteristics, and performs a normalization process using the scale factor for each band. . Then, the scale factor calculation circuit 206 sends the normalized data to the quantization circuit 207.

  The quantization circuit 207 quantizes the data supplied from the scale factor calculation circuit 206 according to the control from the bit allocation circuit 214 and sends the data to the variable length coding circuit 208.

  Here, the audio data input from the input terminal 201 is also input to the psychoacoustic model circuit 213. The psychoacoustic model circuit 213 performs FFT processing on the input audio data, calculates a corrected audible threshold value by adding an absolute audible threshold value and a masking effect for each scale factor band, and sends the corrected audible threshold value to the bit allocation circuit 214.

  The bit allocation circuit 214 sets the AL bit in M frames (one chunk) for components exceeding the threshold from the corrected audible threshold for each scale factor band supplied from the psychoacoustic model circuit 213 by the recording control circuit 215. Assign bits.

  The variable length coding circuit 208 is supplied with the quantized data from the quantization circuit 207, performs Huffman coding, and sends the coded data to the stream generation circuit 209 and the bit allocation circuit 214.

  The stream generation circuit 209 generates an audio stream using the encoded data supplied from the variable length encoding circuit 208 and the encoding parameters such as the processing band range and scale factor from the circuits 202 to 207. Then, the stream generation circuit 209 outputs the generated audio stream from the output terminal 210 and supplies it to the multiplexing circuit 303 in FIG.

  Also, the header information generation circuit 211 generates information such as an offset value to the chunk, the number of samples per chunk, and the size of the samples required from the stream generation circuit 209 for the header of the MP4 file. Then, the header information generation circuit 211 outputs the generated information from the output terminal 212 and supplies it to the header addition circuit 307 in FIG.

  The multiplexing circuit 303 interleaves and multiplexes the video and audio encoded data supplied from the image encoding circuit 302 and the audio encoding circuit 306 for each video and audio chunk as shown in FIG. . Then, the multiplexing circuit 303 generates mdat box data so that the code amount is VL + AL = Lfix in one video chunk and one audio chunk, and records the data in the recording medium 311.

  Further, the header addition circuit 307 is supplied with information such as the offset value to the chunk, the number of samples per chunk, and the sample size from the image encoding circuit 302 and the audio encoding circuit 306. Then, the header addition circuit 307 creates stco, stsc, and stsz box data of the video track and the audio track from these pieces of information, and supplies them to the multiplexing circuit 303.

  The header addition circuit 307 generates a free box that records the total code amount Lfix of video chunks and audio chunks VL, AL, video chunks, and audio chunks as user data, and supplies it to the multiplexing circuit 303. The header addition circuit 307 generates a free box that records the number of frames K per chunk indicating the configuration of the video chunk as user data, and supplies the free box to the multiplexing circuit 303. Furthermore, the header addition circuit 307 generates a free box for recording the GOP number gn, the number M of frames per chunk indicating the configuration of the audio chunk, the frame rate of recorded image data, etc. as user data, and a multiplexing circuit 303.

  The multiplexing circuit 303 generates a free box F in the moov box and records the moov box on the recording medium 311 as shown in FIG. In addition, identification information indicating that the data is in the above format is put in the user data.

  In FIG. 5D, the free box F is provided in the moov box. However, the free box F may be provided not in the moov box but in the same position as the moov box and mdat box.

[Description of playback device]
Next, an apparatus for reproducing the MP4 file generated as described above will be described with reference to FIG.

  In FIG. 4, 401 is a recording medium. Reference numeral 402 denotes a reproduction circuit for reproducing data recorded on the recording medium 401. Reference numeral 403 denotes a buffer. Reference numeral 404 denotes a separation circuit. Reference numeral 405 denotes a variable length decoding circuit. Reference numeral 406 denotes an inverse quantization circuit. Reference numeral 407 denotes an inverse DCT circuit. Reference numeral 408 denotes an adder circuit. Reference numeral 409 denotes a memory. Reference numeral 410 denotes a motion compensation circuit. Reference numeral 411 denotes a switch circuit. Reference numeral 412 denotes a rearrangement circuit. Reference numeral 413 denotes an output terminal. Reference numeral 414 denotes a variable length decoding circuit. Reference numeral 415 denotes an inverse quantization circuit. Reference numeral 416 denotes a scale factor circuit. Reference numeral 417 denotes an M / S stereo circuit. Reference numeral 418 denotes an intensity stereo circuit. Reference numeral 419 denotes a TNS circuit. Reference numeral 420 denotes an inverse MDCT circuit. Reference numeral 421 denotes an output terminal. Reference numeral 422 denotes a header information analysis circuit. Reference numeral 423 denotes a stream user data analysis circuit. Reference numeral 424 denotes a reproduction control circuit. Reference numeral 425 denotes an input terminal.

  FIG. 6 is a flowchart showing the reproduction process.

  First, when a reproduction instruction is received from the input terminal 425, the reproduction control circuit 424 checks whether or not the designated file is a file recorded in the above format in order to reproduce the designated file.

  The reproduction control circuit 424 controls the reproduction circuit 402 to open the file recorded on the recording medium 401 as shown in FIG. 6 (S602), reproduce the file data through the reproduction circuit 402, and supply it to the buffer 403.

  The header information analysis circuit 422 analyzes the reproduction data reproduced and stored in the buffer 403, and detects the moov box. When the moov box is detected, basic encoding parameters such as the size of the recorded image and audio encoding method are acquired (S604).

  Next, it is detected from the free box whether there is identification information indicating that there is user-defined data recorded in the above format (S605).

  When the free box F is detected, the number K of frames per chunk indicating the configuration of the video chunk, the number of GOPs gn, and the number M of frames per chunk indicating the configuration of the audio chunk are acquired (S606). Further, the code amount VL, AL of the video chunk and the audio chunk, and the total code amount Lfix of the video chunk and the audio chunk are acquired (S607). Then, stsc, stsz, and stco box are not read, and an mdat box with stream data is detected (S608).

  If there is no free box F in S605, the playback control circuit 424 detects stsc, stsz, and stco box as normal files (S610). In addition, the chunk offset value, the number of samples, and the sample size of the chunk configuration parameters are read out as much as possible to the buffer 403 via the reproduction circuit 402 (S611).

  In the present embodiment, when the header information analysis circuit 422 detects user data indicating the configuration and size of the chunk, the playback control circuit 424 uses the stream data in the mdat box without using the parameters of the stsc box, stsz box, and stco box. To access.

  That is, when user data is detected, the MP4 file is recorded with a fixed length for each chunk, so that a desired chunk can be detected without using an offset value.

  At the time of reproduction, the offset position from the head of each video chunk is obtained by (Lfix × n) bytes (n = 0, 1, 2,...). Also, the offset position from the beginning of the audio chunk is obtained by (Lfix × n + VL) bytes (n = 0, 1, 2,...).

  The playback circuit 402 plays back the stream data of the mdat box of the file recorded on the recording medium 401 under the control of the playback control circuit 424, and supplies it to the buffer 403. The stream data stored in the buffer 403 starts to be read in view of the buffer occupation state and the like, and is supplied to the separation circuit 404.

The separation circuit 404 separates the video chunk data and the audio chunk data, supplies the video chunk data to the variable length decoding circuit 405 and the stream user data analysis circuit 423, and the audio chunk data to the variable length decoding circuit 414. To supply.
The variable length decoding circuit 405 performs variable length decoding on the reproduced stream data supplied from the separation circuit 404 and supplies the decoded stream data to the inverse quantization circuit 406.

  The inverse quantization circuit 406 performs inverse quantization on the variable length decoded data supplied from the variable length decoding circuit 405 and sends the data to the inverse DCT circuit 407. The inverse DCT circuit 407 performs inverse DCT processing on the inversely quantized data supplied from the inverse quantization circuit 406 and sends it to the addition circuit 408. The adder circuit 408 adds the inverse DCT processed data supplied from the inverse DCT circuit 407 and the data supplied from the switch circuit 411.

Here, in the stream data reproduced from the recording medium 401, first, the intra-frame encoded I _{0 of} GOP ₍₁₎ is reproduced. The reproduction control circuit 424 controls to select the terminal a of the switch circuit 411, and the switch circuit 411 supplies data “0” to the adder circuit 408. The adder circuit 408 adds the “0” data supplied from the switch circuit 411 and the data subjected to the inverse DCT process supplied from the inverse DCT circuit 407, and the memory 409 and the rearrangement circuit 412 as the reproduced frame F _0. To supply. The memory 409 stores the addition data supplied from the addition circuit 408.

Next to the intraframe encoded data I ₀ of GOP ₍₁₎ , the picture data B ₋₂ and B ₋₁ that have been bidirectionally predicted encoded are reproduced. The reproduction procedure up to the inverse DCT circuit 407 is the same as the reproduction process described for the intra-frame encoded data I ₀ , and is therefore omitted.

  The inverse DCT circuit 407 supplies the inverse DCT-processed picture data that has been subjected to bidirectional prediction encoding to the addition circuit 408. At this time, the reproduction control circuit 424 controls the switch circuit 411 to select the terminal b of the switch circuit 411, and the switch circuit 411 selects the terminal b and supplies data from the motion compensation circuit 410 to the adder circuit 408.

The motion compensation circuit 410 detects a motion vector generated at the time of encoding and recorded in the stream from the reproduced stream. Then, the motion compensation circuit 410 reads the data of the reference block (in this case, since it is the recording start time, only the data from the reproduced intraframe encoded data F ₀ ) from the memory 409, and outputs the terminal b of the switch circuit 411. To supply.

The adder circuit 408 adds the inverse DCT processed data supplied from the inverse DCT circuit 407 and the motion compensated data supplied from the switch circuit 411. Further, the adding circuit 408 supplies the regenerated frames F ₋₂ and F ₋₁ to the rearrangement circuit 412.

Next, the picture data P _{3 that} has been unidirectionally predictive encoded is reproduced, and the reproduction process up to the inverse DCT circuit 407 is the same as the reproduction process described for the intra-frame encoded data I ₀ , and is therefore omitted.

  The inverse DCT circuit 407 supplies the picture data subjected to the inverse DCT process and subjected to the one-way prediction encoding to the addition circuit 408. The reproduction control circuit 424 controls the switch circuit 411 so as to select the terminal b of the switch circuit 411, and the switch circuit 411 selects the terminal b and supplies the data from the motion compensation circuit 410 to the adder circuit 408.

The motion compensation circuit 410 detects a motion vector that is generated during encoding from the reproduced stream and recorded in the stream. Then, the motion compensation circuit 410 reads the reference block data (reproduced data from the intra-frame encoded data F ₀ ) from the memory 409 and supplies it to the terminal b of the switch circuit 411.

The adder circuit 408 adds the inverse DCT-processed data supplied from the inverse DCT circuit 407 and the motion compensated data supplied from the switch circuit 411, and outputs the resulting frame F ₃ to the memory 409 and the rearrangement circuit 412. Supply. The memory 409 stores the addition data supplied from the addition circuit 408.

Next, pictures B ₁ and B ₂ are reproduced. The difference from B _-2 and B _-1 is that it is not a frame at the start of recording, and is reproduced from frames F ₀ and F ₃ as bidirectional prediction. Other than that, it is reproduced by the same processing as the above B- ₂ and B- ₁ .

As described above, P ₆ , B ₄ , B ₅ ,... Are sequentially reproduced.

The rearrangement circuit 412 rearranges the frames F ₀ , F ₋₂ , F ₋₁ , F ₃ , F ₁ , F ₂ , F ₆ , F ₄ , F ₅ ,. That is, the rearrangement circuit 412 rearranges the reproduced frames into F _-2 , F _-1 , F ₀ , F ₁ , F ₂ , F ₃ , F ₄ , F ₅ , F ₆ ,. It outputs to 413.

  The audio chunk playback data supplied to the variable length decoding circuit 414 is variable length decoded and supplied to the inverse quantization circuit 415. The inverse quantization circuit 415 inversely quantizes the decoded reproduction data and sends it to the scale factor circuit 416.

  The scale factor circuit 416 restores the normalized component from the scale factor value and sends it to the M / S stereo circuit 417. The M / S stereo circuit 417 reproduces the left and right channel signals from the sum signal and the difference signal and sends them to the intensity stereo circuit 418.

  The intensity stereo circuit 418 reproduces the left and right channel signals from the right channel sum signal and the power ratio, and sends them to the TNS circuit 419.

  The TNS circuit 419 reproduces data using a filter having a characteristic opposite to that of the prediction filter used at the time of encoding, and sends the data to the inverse MDCT circuit 420.

  The inverse MDCT circuit 420 performs inverse MDCT on the reproduced data to obtain reproduced audio data and outputs it from the output terminal 421.

<Special playback>
The normal reproduction has been described above. Next, the special reproduction will be described.

  The reproduction control circuit 424 acquires and holds an offset value St (offset) from the beginning of the file of the mdat box stream at the start of reproduction of the file.

  Here, it is assumed that a special reproduction instruction signal in the forward direction is input from the input terminal 425 in FIG. 4 during reproduction of the nth image frame in the stream. For example, assume that a time of s seconds is specified.

The number of frames for s seconds is s × frame_rate = u (1)
The frame_rate is acquired from the parameters recorded as user data of the free box F of the MP4 file at the start of playback.

  The frame to be reproduced is the (n + u) th frame. However, when the stream data compressed by MPEG is specially reproduced, it must be reproduced from the data I encoded in the frame. Therefore, the intra-frame encoded image data (intra frame) immediately before the n + u-th frame is reproduced.

First, the number r of chunks up to a chunk including the n + u-th frame is calculated.
r = (n + u) / K (2)
However, the decimal part of r is rounded down to the smaller integer.

The offset value Pr up to the next chunk to be reproduced is Pr = St (offset) + Lfix × r (3)
Is calculated by

Next, the number q in the GOP chunk having the intra frame to be reproduced is calculated.
q = (n + ur) / (K / gn) (4)
However, the decimal part of q is rounded down to the smaller integer.

  The reproduction control circuit 424 calculates each value by the above formulas (1) to (4), initializes the decoding circuit such as the buffer 403 and the variable length coding circuit 405, and mutes the output to the audio output terminal 421. .

  The reproducing circuit 402 is controlled to seek the recording medium 401 up to GOP (Pr) in the Pr byte, reproduce data from GOP (Pr), and supply the data to the buffer 403. The reproduction data stored in the buffer 403 is immediately supplied to the stream user data analysis circuit 423 via the separation circuit 404.

  The stream user data analysis circuit 423 analyzes GOP (Pr) user data, acquires an offset value I (offset) up to the next intra frame, and supplies it to the reproduction control circuit 424.

  As soon as the offset value I (offset) up to the next intra frame is acquired, the playback control circuit 424 seeks the file on the recording medium 401 through the playback circuit 402 by the offset value I (offset), and the next GOP ( Play Pr + 1). Then, an offset value from the user data of GOP (Pr + 1) to the next intra frame is acquired.

  The above process is repeated to arrive at GOP (Pr + q) which is the purpose of special reproduction. The reproduction control circuit 424 reproduces GOP (Pr + q) through the reproduction circuit 402 and stores it in the buffer 403. The separation circuit 404 supplies the reproduction data of GOP (Pr + q) stored in the buffer 403 to the stream user data analysis circuit 423. The stream user data analysis circuit 423 obtains the size I (size) of the intra frame from the user data, and informs the reproduction control circuit 424 of the size (I (size)). Based on the size information I (size), the playback control circuit 424 can send only an intra frame to the variable length decoding circuit 405 and below to obtain a playback image for special playback. Then, the calculation of the next special reproduction image is started using the equations (1) to (4).

  As described above, at the time of recording, the chunk configuration of image data and audio data is determined, and when the code amount VL of the chunk of image data and the code amount AL of the chunk of audio data are used, the code amount Lfix of both chunks LVL = VL + AL Encoding is performed so that becomes constant.

  Further, for each GOP, the image data stream includes the code amount VL of the image data, the code amount I (size) of the intra-frame encoded data (intra frame), and the offset value I (offset) to the intra frame of the next GOP. Record as user data.

  Further, information indicating the configuration of each chunk such as the number of frames K and M per chunk and the number of encoding units gn is stored in the free box and recorded as user data. In addition, the encoding frame rate, the code amounts VL and AL of each chunk, the total code amount Lfix, identification information indicating the above configuration, and the like are stored and recorded in the free box as user data.

  When recognizing the stream from the free box at the time of reproduction, the stream data of each chunk of video and audio is accessed from the information of the user data recorded in the free box and the stream. Thus, normal reproduction and special reproduction are performed, and reproduction is possible without reading header information such as stco, stsc, stsz box, etc., so that a memory for storing a header that requires a large capacity becomes unnecessary.

  In addition, since it is not necessary to read and analyze the above-mentioned parameters such as stco, stsc, stsz box, etc. related to encoding in the header, access to stream data can be performed quickly, playback waiting time can be reduced, and seamless playback is possible. Is possible.

  Conventionally, when there is too much parameter information such as stco, stsc, stsz box and so on, it is necessary to update the parameter information of the memory by rereading the parameter from the recording medium while reproducing. Met. However, this is not necessary, and the processing load on the device body can be reduced.

  Also, since the file is a normal MP4 file, it can be played back on other devices.

  Furthermore, since it can be realized by one file, even if data copied between recording media is reproduced by this apparatus, it can be reproduced in the same manner as the above reproduction processing.

  In the above example, the audio encoding method has been described as MPEG-AAC. However, other encoding methods such as MPEGI, II, and III can be similarly recorded and reproduced, and the same effect can be obtained.

[Other Embodiments]
The embodiment according to the present invention has been described in detail using specific examples. However, the present invention can take an embodiment as a system, apparatus, method, program, storage medium (recording medium), or the like. is there. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  It goes without saying that the object of the present invention can be achieved even if any part of the illustrated functional blocks and operations is realized by a hardware circuit or by software processing using a computer.

  Note that the present invention includes a case where the present invention is achieved by supplying a software program for realizing the functions of the above-described embodiments directly or remotely to a system or apparatus. In that case, a computer such as a system reads and executes the program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the recording medium (storage medium) for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. In addition, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser can be used to connect to a homepage on the Internet, and the computer program itself of the present invention can be downloaded from the homepage. It can also be supplied by downloading a compressed file including an automatic installation function to a recording medium such as a hard disk. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and a user who satisfies predetermined conditions downloads key information for decryption from a homepage via the Internet. You can also. In this case, the downloaded key information is used to execute the encrypted program and install it on the computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instructions of the program may be part of the actual processing. Alternatively, it can be realized by performing all of them.

  Further, after the program read from the recording medium is written in the memory of a function expansion board inserted into the computer or a function expansion unit connected to the computer, the CPU of the board or the like performs a part of the actual processing. Alternatively, it can be realized by performing all of them.

FIG. 4 is a block diagram illustrating a detailed configuration of an image encoding circuit 302 in FIG. 3. FIG. 4 is a block diagram showing a detailed configuration of an audio encoding circuit 305 in FIG. 3. 1 is a block diagram illustrating a recording apparatus according to an embodiment of the present invention. It is a block diagram which shows the reproducing | regenerating apparatus of embodiment which concerns on this invention. (A) is a diagram for explaining the relationship between image streams and audio streams and chunks, (b) is a diagram for explaining the frame order of encoding, (c) is a diagram for explaining the data structure of an image stream, (d) These are the figures explaining the structure of the free box of this embodiment. It is a flowchart which shows the reproduction | regeneration processing of this embodiment. It is a figure explaining the structure of the image data recorded on an MP4 file, and audio data. It is a figure explaining stsc, stsz, stco box in moov box of MP4 file. It is a figure explaining the structure of a Quicktime (trademark) format.

Explanation of symbols

101 Image data input terminal 102 Camera signal processing circuit 103 Rearrangement circuit 104 Addition circuit 105 Switch circuit 106 DCT circuit 107 Quantization circuit 108 Inverse quantization circuit 109 Inverse DCT circuit 110 Addition circuit 111 Memory 112 Motion compensation circuit 113 Switch circuit 114 Variable Long encoding circuit 115 Stream generation circuit 116 Buffer 117 Code amount control circuit 118 Stream data output terminal 119 Stream parameter addition circuit 120 Header information generation circuit 121 Header information output terminal 122 Code amount setting circuit 123 Recording control circuit 124 Control data input terminal 201 Audio data input terminal 202 MDCT circuit 203 TNS circuit 204 Intensity stereo circuit 205 M / S stereo circuit 206 Scale factor calculation circuit 207 Quantization times Path 208 Variable length encoding circuit 209 Stream generation circuit 210 Output terminal 211 Header information generation circuit 212 Output terminal 213 Auditory psychology model circuit 214 Bit allocation circuit 215 Recording control circuit 216 Input terminal 301 Image data input terminal 302 Image encoding circuit 303 Multiplexing Conversion circuit 304 recording circuit 305 audio data input terminal 306 audio encoding circuit 307 header addition circuit 308 parameter input terminal 309 code amount setting circuit 310 chunk configuration setting circuit 311 recording medium 401 recording medium 402 reproduction circuit 403 buffer 404 separation circuit 405 variable length Decoding circuit 406 Inverse quantization circuit 407 Inverse DCT circuit 408 Adder circuit 409 Memory 410 Motion compensation circuit 411 Switch circuit 412 Rearrangement circuit 413 Output terminal 414 Variable length decoding circuit 415 Inverse quantization circuit 416 Scale factor circuit 417 M / S stereo circuit 418 Intensity stereo circuit 419 TNS circuit 420 Inverse MDCT circuit 421 Output terminal 422 Header information analysis circuit 423 Stream user data analysis circuit 424 Playback control circuit 425 Input terminal

Claims (17)

In an apparatus for recording image data and audio data as files,
Image data encoding means for encoding image data;
Image data code amount control means for controlling the image data encoding means;
Audio data encoding means for encoding audio data;
Audio data code amount control means for controlling the audio data encoding means;
Image stream generation means for streaming the image data encoded by the image data encoding means;
Audio stream generating means for streaming the audio data encoded by the audio data encoding means;
Multiplexing means for multiplexing the generated image stream and audio stream in units of the image data and audio data for a predetermined period, respectively.
Recording means for recording the stream multiplexed by the multiplexing means on a recording medium,
The image data code amount control means and the audio data code amount control means have the image data encoding means and the audio data so that the image stream and the audio stream are fixed-length for each unit multiplexed by the multiplexing means. A recording apparatus for controlling data encoding means.   The recording means adds and records additional information including the size of a multiplexed unit of the encoded image stream and the number of frames of image data of the multiplexed unit to the image stream for recording. The recording apparatus according to 1.   The recording means records the stream multiplexed by the multiplexing means on the recording medium as a file, and also includes an offset value from the beginning of the file and each multiplexing for each multiplexing unit of the image stream and audio stream. 2. The recording apparatus according to claim 1, wherein information on the number of samples per unit is recorded as a header of the file.   The additional information includes information on the number of coding units included in one of the multiplexing units of the image stream and audio stream, and the total code amount of the multiplexing units corresponding to each other in the image data and audio data. The recording apparatus according to claim 2, further comprising:   2. The recording apparatus according to claim 1, wherein the encoding unit of the image data matches an image data group when the image data is MPEG-encoded.   2. The recording / reproducing apparatus according to claim 1, wherein the audio data encoding unit matches a frame which is an encoding unit of the audio encoding means.   2. The recording / reproducing apparatus according to claim 1, wherein the file recorded by the recording means is in an MPEG audio Layer-4 format. Playback means for playing back image data and audio data recorded as files on a recording medium;
Reproduction control means for controlling the reproduction means;
Header information analysis means for analyzing header information recorded in the recording medium,
When the header information analysis means detects unique user data in the header recorded in the recording medium, it prohibits reading of access data to the stream recorded in the header,
The reproduction control means controls the reproduction means based on the parameter information of the stream recorded in the file analyzed by the header information analysis means, and the image data and audio data recorded as a file on the recording medium A playback device characterized by playing back video. It has stream user data analysis means for detecting user data recorded in a stream of image data and analyzing encoding information of the stream,
The reproduction control unit controls the reproduction unit based on the stream parameter information detected by the header information analysis unit and the stream encoding information analyzed by the stream user data analysis unit to control the recording medium. 9. The reproducing apparatus according to claim 8, wherein the image data and audio data recorded as files are reproduced.   The stream user data analyzing means detects the encoding size of the minimum reproduction unit of the image data from the user data recorded in the image stream and the offset value to the encoding unit recorded next on the recording medium. The reproducing apparatus according to claim 9.   The header information analysis means detects the user data and corresponds to the number of encoding units of the multiplexed image data and audio data and the number of encoding units of the image data as parameter information of the stream to be analyzed. A code amount corresponding to the number of encoding units of the audio data, and a code amount obtained by summing up the code amounts corresponding to the number of encoding units of each of the image data and the audio data. Item 9. The playback device according to Item 8.   12. The reproducing apparatus according to claim 11, wherein the encoding unit of the image data matches an image data group when the image data is MPEG-encoded.   9. The reproducing apparatus according to claim 8, wherein the file recorded on the recording medium is recorded in the MPEG audio Layer-4 format. A method of recording image data and audio data as files,
An image data encoding control step for controlling the encoding of the image data;
An audio data encoding control step for controlling encoding of audio data;
An image stream generating step for streaming the encoded image data;
An audio stream generating step for streaming the encoded audio data;
A multiplexing step of multiplexing the generated image stream and audio stream, with the image data and audio data for a predetermined period as a unit,
A recording step of recording the multiplexed stream on a recording medium,
In the image data encoding control step and the audio data encoding control step, the image data and the audio data are set to have a fixed length for each of the units to be multiplexed in the multiplexing step. A recording method characterized by encoding. A reproduction method for reproducing image data and audio data recorded as files on a recording medium,
A header information analysis step of analyzing header information recorded in the recording medium;
When unique user data is detected in the header recorded on the recording medium by the header information analysis step, reading of access data to the stream recorded in the header is prohibited,
A reproduction step of reproducing image data and audio data recorded on the recording medium based on the parameter information of the stream recorded in the file analyzed in the header information analysis step .   The program for making a computer perform the method of Claim 14 or 15.   A computer-readable storage medium storing the program according to claim 16.

Images