JP5048944B2 - Moving image processing device - Google Patents

Description

  The present invention relates to a moving image processing apparatus.

  In recent years, with the development of information technology, technology for transmitting and reproducing moving image data such as digitized moving images with high quality and high compression rate has been developed. One of them is, for example, H.264. H.264 exists.

  H. above. H.264 is a video codec standard jointly standardized by ITU-T (International Telecommunication Union / Telecommunication Standardization Sector) and ISO (International Organization for Standardization) in 2004, and is also called MPEG4 AVC. .

  H. above. H.264 can compress the size to about half that of MPEG-2 with a high data compression rate that is more than twice that of MPEG-2 and 1.5 times that of MPEG-4. Also, even with the same bit rate, higher image quality can be achieved than with conventional video formats, so compact, high-quality video from large-scale high-quality content such as movies, concerts, and terrestrial digital TV broadcasts. Widespread use is expected to the required moving image distribution format for mobile phones.

  H. In H.264, parameter sets (or simply parameters) called SPS (Sequence Parameter Set) and PPS (Picture Parameter Set) are used. An example of parameters included in both parameter sets is as shown in FIG.

  As an example of the parameters included in the SPS, as shown in FIG. 20, “profile_idc” indicating a bitstream profile, “constraint_set0_flag” indicating that the bitstream can be decoded with a baseline profile, and the bitstream being a main profile. "Constraint_set1_flag" indicating that decoding is possible, "constraint_set2_flag" indicating that the bitstream can be decoded with the extended profile, and "level_idc" indicating the level of the bitstream.

  As an example of parameters included in the PPS, as shown in FIG. 20, “pic_parameter_set_id” indicating the ID (identifier) of the PPS, “seq_parameter_set_id” indicating the ID of the SPS referred to by the PPS, and an entropy code ( “Entropy_coding_mode_flag” indicating a mode of (arithmetic coding) as a flag is included.

  H. above. According to the H.264 standard, SPS and PPS need only be included at the head of the stream. However, even if playback is performed from a frame other than the head, that is, random access is attempted, SPS and PPS for decoding the frame to start playback are There is a problem that it cannot be played back because it does not exist.

  As such a solution, a moving picture coding apparatus for decoding and reproducing in units of access units to which the SPS and PPS are added during random access by adding and recording the SPS and PPS for each predetermined access unit. (For example, refer to Patent Document 1).

JP 2004-248255 A

  However, since the moving picture coding apparatus described in Patent Document 1 adds a plurality of SPSs and PPSs for each access unit, the data amount corresponding to the addition of the SPSs and PPSs at the time of moving picture recording corresponds to the bit rate. The processing data amount allocated to the moving image is pressed. For this reason, there is a problem that the compression efficiency of the moving picture is deteriorated, and as a result, the quality of the compressed moving image is deteriorated.

  In addition, since it is limited to reproduction / editing for each access unit in which the SPS and PPS are inserted, a random access and editable position is already determined at the time of recording, and the convenience for the user may be lowered. was there.

  The present invention has been made in view of the above-mentioned problems, and an object of the present invention is a novel and capable of random access and editing without reducing the amount of moving image processing data at the bit rate at the time of recording. An object is to provide an improved moving image processing apparatus.

In order to solve the above problems, according to a first aspect of the present invention, there is provided a moving image processing apparatus that encodes the moving image data and generates a stream. The moving image processing apparatus calculates decoding parameter set generation means for generating a decoding parameter set used for decoding the stream by an encoder , and calculates a total data amount of the decoding parameter set from the moving image stream encoded by the encoder , Based on the comparison result between the total data amount of the decoding parameter set and the counter value, the extraction means for selectively outputting the decoding parameter set and the moving image, and the decoding parameter set generated by the decoding parameter set generation means. And a first file creation means for creating a decoding parameter set file from the decoding parameter set output from the extraction means, and a moving picture output from the extraction means as a file based on the encoded stream the first to create a moving image file from the image File creation means, and a recording medium recording means for recording the decoding parameter set file and the moving image file created by the first file creation means and the second file creation means on a recording medium. It is characterized by that. According to such a configuration, by recording the decoding parameter set file and the moving image file as separate files on the recording medium, it is possible to minimize deterioration in image quality without reducing the bit rate for the moving image.

  The first file creation means further generates, based on the decryption parameter set file, decryption parameter set data amount information indicating the data amount of the decryption parameter set file; the recording medium recording means comprises the decryption parameter set data The data amount information and the moving image file may be recorded on a recording medium.

  You may comprise so that the said decoding parameter set may not be provided to the head of the moving image stream of the moving image file recorded on the said recording medium.

In order to solve the above problems, according to another aspect of the present invention, a recording medium reading means for reading recorded on the recording medium the moving image file, the moving image read out by the recording medium reading means and moving image stream extracting means for extracting the video stream is a payload part from a file, a set of parameters leading Ru granted during decoding of the video stream, decoding parameter sets used for decoding of the moving picture stream a decoding parameter set extracting means for extracting, the decoding parameter set, and inserting means for inserting the front of the picture corresponding to the decoding start position of the video stream, and reads the video stream, the video stream Based on the decoding parameter set inserted into the video stream, the video stream is decoded. Means, and a network communication means capable of outputting data to the outside, said insertion means, after confirming that the external from the predetermined device is connected to the network, the decoding parameter set, the moving image processing apparatus according to claim also be output from the said network communication means is provided by inserting before the picture corresponding to the decoding start position of the video stream. According to such a configuration, the external device can decode and reproduce a moving image from an arbitrary position. The insertion destination by the insertion unit is, for example, immediately before an IDR (Instantaneous Decoder Refresh) picture configured in a moving image stream.

  The recording medium stores a decoding parameter set file in which a decoding parameter set used for decoding a moving image stream is filed, and the recording medium reading means reads the moving image file and the decoding parameter from the recording medium. The set file may be read as a separate file, and the decoding parameter set extraction unit may be configured to extract the decoding parameter set based on the decoding parameter set file read from the recording medium.

  You may comprise so that the said decoding parameter set may not be provided to the head of the moving image stream of the moving image file recorded on the said recording medium.

  The decoding parameter set may be repeatedly inserted by the inserting means at a predetermined interval. A moving image can be reproduced from an arbitrary position instead of from the beginning of the moving image stream.

  The decoding parameter set inserted by the inserting means may be before the IDR picture configured in the moving image stream.

  The decoding start position may correspond to a position selected by the user as a reproduction start position for reproducing the moving image stream.

  The insertion unit may insert the decoding parameter set into the moving image stream at a predetermined interval. With this configuration, the external device can decode and reproduce a moving image from an arbitrary position.

  As described above, according to the present invention, random playback and random editing can be performed without reducing the amount of moving image data at the recording bit rate.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, components having substantially the same functions and configurations are denoted by the same reference numerals, and redundant description is omitted.

(About the first embodiment)
First, the moving image recording apparatus (or moving image processing apparatus) according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating an example of a schematic configuration of a moving image recording apparatus according to the first embodiment.

  As shown in FIG. 1, the moving image recording apparatus 101 includes an imaging unit 111, an encoder 112, a control unit 113, an extraction unit 114 that extracts SPS and PPS as parameter sets, and a parameter file creation unit (or , First file creation means) 115, moving image file creation section (or second file creation means) 116, and recording medium writing section 128.

  The imaging unit 111 has zoom and wide shooting functions, and outputs moving image data obtained by imaging a subject to the encoder 112. The imaging unit 111 includes at least an optical component and an imaging element.

  The optical component is a lens or an optical filter that removes unnecessary wavelengths, and examples thereof include an ND filter. Light incident from the subject and transmitted through the optical component is captured by the image sensor and converted into an electrical signal.

  The imaging device (imaging device) photoelectrically converts an optical image received from a subject by a plurality of pixels including photoelectric conversion elements provided two-dimensionally on a light receiving surface, and outputs the result as electrical moving image data. Is possible. For example, examples of the imaging element include various types of solid-state imaging devices such as a CCD.

  Note that the imaging unit 111 according to the first embodiment has been described by taking an example in which moving image data is output by photographing a subject. However, the imaging unit 111 is not limited to this example. In preparation, a moving image and sound may be output.

  The control unit 113 controls processing of the encoder 112, the extraction unit 114, the parameter file creation unit 115, the moving image file creation unit 116, and the recording medium writing unit 128 of the moving image recording apparatus 101.

  The moving image data input from the imaging unit 111 is H.264. The data is encoded by an encoder 112 that encodes using the H.264 system.

  The moving image stream encoded by the encoder 112 is input to the extraction unit 114.

  The extraction unit 114 extracts parameter sets (decoding parameter sets) of SPS (Sequence Parameter Set) and PPS (Picture Parameter Set) from the input video stream. The extraction of the SPS and PPS will be described later.

  The extraction unit 114 outputs the moving image stream input from the encoder 112 to the moving image file creation unit 116.

  Next, when the SPS and PPS extracted by the extraction unit 114 are input, the parameter file creation unit 115 converts the SPS and PPS into a file and creates an SPS and PPS file (decoding parameter set file).

  The moving image file creation unit 116 converts the moving image input from the extraction unit 114 into a file, and creates a moving image file.

  The parameter file creation unit 115 and the moving image file creation unit 116 file the data so as to correspond to the format of the recording medium 131 (for example, FAT16, FAT32, etc.).

  The recording medium writing unit 128 writes the SPS and PPS files filed by the parameter file creation unit 115 and the moving image file creation unit 116 and the moving image file in the recording medium 131 and records them. That is, the SPS, PPS file, and moving image file are recorded as separate files on the recording medium 131.

  In addition, the recording medium writing unit 128 writes the SPS and PPS files to the recording medium 131 first, writes the moving image files to the recording medium 131 in order, etc. Bit rate reduction can be prevented. That is, in the writing of a moving image file, the processing data amount of header information such as SPS and PPS other than the moving image can be reduced, so that the processing data amount of the bit rate for the moving image file can be allocated to the maximum. Can be kept to a minimum.

  Next, the extraction unit 114 shown in FIG. 1 will be described in more detail with reference to FIG. FIG. 2 is a block diagram showing a schematic configuration of the extraction unit shown in FIG.

  As shown in FIG. 2, the extraction unit 114 includes a stream analysis unit 201, a counter 203, an SPS / PPS total data amount setting unit 205, a comparator 207, and an output selection unit 209.

  When the stream analysis unit 201 sends an encoding start signal from the control unit 113, the moving image stream input from the encoder 112 is first analyzed.

  When SPS is detected by the stream analysis unit 201, the counter 203 starts counting the amount of data of SPS and PPS.

  In addition, the stream analysis unit 201 transmits the analysis result of the moving image stream to the SPS and PPS total data amount setting unit 205.

  The SPS and PPS total data amount setting unit 205 calculates the total data amount of SPS and PPS based on the analysis result received from the stream analysis unit 201 and transmits it to the comparator 207.

  The comparator 207 compares the calculated total data amount of SPS and PPS with the counter value from the counter 203 and outputs the comparison result to the output selection unit 209.

  In accordance with the comparison result from the comparator 207, the output selection unit 209 switches the data output destination to the parameter file creation unit 115 in the case of the following expression 1, and outputs the SPS and PPS data to the parameter file creation unit 115. In the case of Equation 2 below, the data output destination is switched to the moving image file creation unit 116 side, and the moving image data is output to the moving image file creation unit 116.

Counter value <Setting value (total data amount) (Equation 1)
Counter value ≥ Setting value (total data amount) ... (Formula 2)

  Here, a specific example of the SPS and PPS extraction processing by the extraction unit 114 will be shown below.

  The stream analysis unit 201 performs parameter data such as profile_idc (8bit), constraint_set1_flag (1bit), constraint_set0_flag (1bit), etc., and parameters and loops expressed by exponential Golomb (ExponentialGolomb) codes such as seq_parameter_set_id and log2_max_frame_num_minus4 The data amount obtained by adding the rbsp_trailing_bit to the data amount obtained by adding the data amount obtained by analyzing the structure and conditional branching is defined as the SPS data amount.

  Similarly, the amount of data obtained by analyzing parameters, loop structures, conditional branches such as pic_parameter_set_id and seq_parameter_set_id, exponential Golomb codes, and the number of fixed-length bits such as entropy_coding_mode_flag (1bit) and pic_order_present_flag (1bit) The data amount obtained by adding the rbsp_trailing_bit to the data amount obtained by adding the data amounts of the parameters in (2) is defined as the PPS data amount.

Above SPS
And the PPS data amount are transferred to the SPS and PPS total data amount setting unit 205. The comparator 207 compares the value of the SPS and PPS total data amount setting unit 205 with the value of the counter 203. If the counter value is smaller than the set value, it determines that SPS and PPS, and outputs the data to the parameter file creation unit 115. If the counter value ≧ the set value, it is determined as moving image data, and the data is output to the moving image file creation unit 116. For details, refer to the H.264 standard (ITU-T recommendation H.264 or ISO / IEC international standard 14496-10) (ITU-T recommendation H.264 03/2005 editions p50 to p53, etc.).

  Next, the data structure written to the recording medium 131 by the recording medium writing unit 128 will be described. As shown in FIG. 3, when the SPS and PPS are added to the head of the moving image file, they are not added. There are cases.

  As shown in FIG. 3, the moving image file includes one or more pictures 300 (300-1 to 300-n). The pictures include, for example, types of I picture (intra-frame encoded picture), P picture (inter-frame forward prediction encoded image), and B picture (bidirectional predictive encoded image).

  FIG. 3A shows a case where SPS and PPS are added to the beginning of the moving image file, and FIG. 3B shows a data structure when SPS and PPS are not added to the beginning of the moving image file. Show.

  For example, if the SPS and PPS at the beginning of the moving image file are removed by the moving image file creation unit 116 as shown in FIG. However, the present invention is not limited to this example.

  As shown in FIG. 4, the recording medium 131 includes, for example, move-1. avi (moving image file) and head-1. In addition to the above files, a plurality of sets of files having a set of dat (SPS, PPS file) are recorded in a hierarchical structure. avi (moving image file) and head-1. The present invention is not limited to such an example as long as the relationship between dat (SPS, PPS file) is known. Further, in FIG. 4, SPS and PPS are handled as one file (head-1.avi), and different files (head_SPS-1.avi, head_PPS-1.avi) may be used.

  Also, as shown in FIG. 4, on the recording medium 131, for example, SPS, PPS files and moving image files are managed by assigning the same numbers such as 1, 2, 3,. As described above, the present invention is not limited to this example as long as a signature or the like that indicates the relationship between the moving image file and the SPS or PPS file is added.

(About the second embodiment)
Next, a moving image recording apparatus (or moving image processing apparatus) according to the second embodiment will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of a schematic configuration of a moving image recording apparatus according to the second embodiment.

  In the description of the moving image recording apparatus 101a according to the second embodiment, the moving image recording apparatus 101a according to the second embodiment is compared with the moving image recording apparatus 101 according to the first embodiment. In particular, the differences will be described. Others are substantially the same unless otherwise specified, and will not be described in detail.

  In the moving picture recording apparatus 101 according to the first embodiment, as described above, the SPS and PPS extracted from the moving picture stream are filed and the SPS and PPS files are written to the recording medium 131, whereas the second The moving image recording apparatus 101a according to the embodiment is different in that the total data amount of SPS and PPS is analyzed from the moving image stream and the total data amount is written to the recording medium 131a.

  As shown in FIG. 5, the moving image recording apparatus 101 includes an imaging unit 111, an encoder 112, a control unit 113a, a parameter file creating unit (first file creating unit) 115a, and a moving image file creating unit ( (Second file creation means) 116a, a data amount extraction unit 121 for extracting data amounts of SPS and PPS as parameter sets, and a recording medium writing unit 128 are provided.

  The moving image recording apparatus 101 according to the first embodiment shown in FIG. 1 and the moving image recording apparatus 101a according to the second embodiment shown in FIG. 114 is different from the moving image recording apparatus 101a in that a data amount extraction unit 121 is provided.

  Next, the data amount extraction unit 121 will be described in more detail with reference to FIG. FIG. 6 is a block diagram illustrating an example of a schematic configuration of the data amount extraction unit 121 according to the second embodiment.

  As illustrated in FIG. 6, the data amount extraction unit 121 includes a stream analysis unit 201 a and an SPS / PPS total data amount setting unit 210.

  When an encoding start signal is sent from the control unit 113 in the stream analysis unit 201a, the moving image stream input from the encoder 112 is first analyzed.

  The stream analysis unit 201 a transmits the analysis result of the moving image stream to the SPS and PPS total data amount setting unit 205.

  The SPS / PPS total data amount setting unit 205 obtains the total data amount of the entire SPS / PPS based on the analysis result received from the stream analysis unit 201a, and outputs the total data amount of SPS / PPS.

  In the moving image recording apparatus 101a according to the second embodiment, the total data amount of SPS and PPS having a data amount smaller than that of the SPS and PPS files themselves is filed and recorded on the recording medium 131a, thereby improving the efficiency of the recording process. The recording capacity of the recording medium can be efficiently used for other moving image streams.

  Next, the data structure written to the recording medium 131a by the recording medium writing unit 128a will be described. In FIG. 7A, SPS and PPS are added to the head of the moving image file. As another file, there are SPS and PPS data amount files (decoding parameter set data amount information). In FIG. 7B, as a moving image file, for example, a format capable of unifying a plurality of data into one file, such as a moving image and SPS / PPS data amount data, such as a RIFF format file (for example, an avi file). This is the case where the file is saved in one file.

  Further, as shown in FIG. 7, the moving image file includes one or more pictures 300 (300-1 to 300-n). Note that the pictures include, for example, I picture, P picture, and B picture types.

(About the third embodiment)
Next, a moving image reproduction device (or moving image processing device) according to the third embodiment will be described with reference to FIG. FIG. 8 is a block diagram illustrating an example of a schematic configuration of a moving image reproduction apparatus according to the third embodiment.

  The moving image playback device 102 shown in FIG. 8 is a device capable of outputting a moving image stream already recorded on the recording medium 131 to the display unit 139 and displaying the moving image stream on the screen.

As shown in FIG. 8, the moving image playback apparatus 102 includes a control unit 113b, a recording medium reading unit 123, a parameter extraction unit 135, a moving image extraction unit 136, a multiplex 137, a decoder 138, and a display. Part 139. Note that the moving image playback apparatus 102 according to the third embodiment will be described by taking an example in which the display unit 139 is provided as an integral type, but the present invention is not limited to this example. The device 102 may be connected to the display unit 139 that can be externally connected as a separate unit.

  The control unit 113b controls the processing of the recording medium reading unit 123, the parameter extraction unit 135, the moving image extraction unit 136, the multiplex 137, and the decoder 138 of the moving image reproduction apparatus 102.

  When the recording medium reading unit 123 receives a signal to start reading of the recording medium 131 from the control unit 113b, the recording medium reading unit 123 reads the SPS, PPS file, and moving image file from the recording medium 131.

  The read SPS and PPS files are transmitted to the parameter extraction unit 135, and the parameter extraction unit 135 extracts SPS and PPS parameter sets from the SPS and PPS files and outputs them to the multiplex 137.

  On the other hand, the read moving image file is transmitted to the moving image extraction unit 136, and the moving image extraction unit 136 extracts the moving image stream of the payload portion from the moving image file including the header information and the like. Is output.

  The parameter extraction unit 135 and the moving image extraction unit 136 remove the SPS, PPS file, or the header included in the moving image file, and then extract the SPS, PPS data and the moving image stream, respectively.

  The multiplex 137 inserts the extracted SPS and PPS data into the moving image stream in accordance with a command from the control unit 113 b, that is, multiplexes and outputs to the decoder 138.

  The moving picture reproducing apparatus 102 according to the third embodiment has been described by taking the case where the multiplex 137 is provided as an example. However, for example, the moving picture reproducing apparatus 102 receives SPS and PPS data such as a selector as a moving picture. The present invention is not limited to this example as long as a member having a function that can be inserted into the image stream is provided.

  When the decoder 138 inputs the data output from the multiplex 137, the decoder 138 receives the H.264 data. In the H.264 system, decoding is started in accordance with a decoding start command from the control unit 113b.

  Since the SPS, PPS file, and the moving image file are recorded as separate files on the recording medium 131, and the moving image file only needs to have at least one SPS, PPS at the beginning, the moving image reproducing apparatus 102 reads it at the time of decoding. Can be decoded and played back with good image quality.

  Next, an insertion process by the multiplex 137 of the moving image reproduction apparatus 102 according to the third embodiment will be described with reference to FIG. As shown in FIG. 9, SPS and PPS are added to the head of the moving image stream in the data input to the multiplex 137. Therefore, if the multiplex 137 is output to the decoder 138 as it is, the decoder 138 can also decode SPS and PPS if it is decoded from the head of the moving image stream. it can.

Also, different from that shown in FIG. 9, when an input is made to the multiplex 137 without adding SPS or PPS to the head of the moving picture stream, the playback operation from the first frame of the moving picture stream is shown.

In the above case, unlike the one shown in FIG. 9, since SPS and PPS are not added to the head of the moving image stream, first, the multiplex 137 transmits from the parameter extraction unit 135 to add SPS and PPS. SPS and PPS to be multiplexed into a moving image stream.

  Next, the multiplex 137 transmits the moving image stream with SPS and PPS added to the head thereof to the decoder 138 so that the decoder 138 can add the moving image stream based on the newly added SPS and PPS. Can be decrypted.

  Next, as shown in FIG. 10, the random reproduction operation will be described. Random playback is, for example, starting decoding of a video stream from a position designated by the user to start playback, and displaying the video on the screen, but is not limited to this example. In this case, since it is necessary to add SPS and PPS data to the head of the moving picture stream every time reproduction is started, the multiplex 137 multiplexes the SPS and PPS data and the moving picture stream each time.

  The multiplex 137 outputs a moving image stream with SPS and PPS data added to the head to the decoder 138.

  As described above, the multiplex 137 inputs the moving image stream and SPS and PPS separately, and multiplexes, so that the data amount at the bit rate at the time of encoding is randomly reproduced from a free position. be able to.

  More specifically, for example, as shown in FIG. 10, when the user gives an instruction to start playback from a picture 300-4, the recording medium reading unit 123 of the moving image playback device 102 moves from the recording medium 131 to a moving image. The image file is output to the moving image extraction unit 136 and the SPS and PPS files are output to the parameter extraction unit 135.

  The parameter extraction unit 135 outputs SPS and PPS to the multiplex 137, and the moving image extraction unit 136 outputs the pictures after the picture 300-4 to the multiplex 137 as a moving image stream.

  As illustrated in FIG. 10, the multiplex 137 assigns SPS and PPS before (immediately before) the picture 300-4 and outputs it to the decoder 138. Note that the picture 300-4 is an IDR picture, for example. However, the picture 300-4 is not limited to this example as long as the subsequent picture can be correctly decoded without using the information of the picture before the picture.

  The decoder 138 converts the data from the multiplex 137 to H.264. The data is decoded and output to the display unit 139 according to the H.264 system.

(About the fourth embodiment)
Next, a moving picture reproduction apparatus (or moving picture processing apparatus) according to the fourth embodiment will be described with reference to FIG. FIG. 11 is a block diagram illustrating an example of a schematic configuration of a moving image reproduction device according to the fourth embodiment.

  In the description of the moving image playback apparatus 102a according to the fourth embodiment, the moving image playback apparatus 102a according to the fourth embodiment is compared with the moving image playback apparatus 102 according to the third embodiment. In particular, the differences will be described. Others are substantially the same unless otherwise specified, and will not be described in detail.

  As shown in FIG. 11, the moving image playback apparatus 102a includes a control unit 113c, a recording medium reading unit 123a, a parameter extraction unit 135a, a moving image extraction unit 136a, a multiplex 137a, a decoder 138, and a display. A unit 139 and an SPS / PPS extraction unit 140.

  The recording medium reading unit 123a reads the SPS and PPS total data amount file and the moving image file from the recording medium 131 in response to a reading start signal from the control unit 113c.

  The parameter extraction unit 135a and the moving image extraction unit 136a remove the file header information and the like from each of the files, and extract SPS and PPS data amount data and a moving image stream.

  The SPS / PPS extraction unit 140 extracts SPS and PPS based on the extracted SPS and PPS data amount data and the moving image stream. The extraction of the SPS and PPS will be described later.

  Next, the multiplex 137a multiplexes the input SPS and PPS data and the moving image stream in accordance with a command from the control unit 113c, and outputs the multiplexed data to the decoder 138.

  The decoder 138 follows the decoding start instruction from the control unit 113c and performs H.264 decoding. Decoding is started in the H.264 format. The moving image decoded by the decoder 138 is output on the screen of the display unit 139.

  Next, the SPS / PPS extraction unit 140 will be described in more detail with reference to FIG. FIG. 12 is a block diagram showing a schematic configuration of the SPS / PPS extraction unit 140 shown in FIG.

  As shown in FIG. 12, the SPS / PPS extraction unit 140 includes a stream analysis unit 221, a counter 223, a comparator 225, and an SPS / PPS buffer 227.

  The stream analysis unit 221 analyzes the input moving image stream and detects SPS.

  When the stream analysis unit 221 detects SPS, the counter 223 starts counting and outputs the counter value to the comparator 225.

  The comparator 225 compares the data amount of SPS and PPS with the counter value, and if the counter value is smaller than the data amount of SPS and PPS, writes the moving image stream in the SPS / PPS buffer 227. That is, the data written in the SPS / PPS buffer 227 is SPS or PPS data.

  As a result, the SPS and PPS added to the head of the moving image stream can be stored in the SPS and PPS buffer, and used for subsequent random reproduction, transfer, and the like.

  Note that the SPS and PPS insertion processing by the multiplex 137a according to the fourth embodiment is substantially the same as the insertion processing by the multiplex 137 according to the third embodiment described above. Omitted.

(About the fifth embodiment)
A moving image processing system 100 according to the fifth embodiment will be described with reference to FIG. FIG. 13 is a block diagram illustrating a schematic configuration of a moving image processing system 100 according to the fifth embodiment.

  As shown in FIG. 13, the moving image processing system 100 includes a moving image playback device 102b, an externally connected device 104, and a network 103 that connects the moving image playback device 102b and the externally connected device 104.

  The externally connected device 104 includes, for example, a decoder capable of decoding a multiplexed video stream transmitted from the video playback device 102b via the network 103.

  In the moving image playback device 102b shown in FIG. 13, the multiplex 137 inserts the SPS and PPS into the moving image stream, and transmits the moving image stream to the external connection device 104 via the network 103 such as IEEE1394. It is a device that can.

  13 includes a control unit 113b, a recording medium reading unit 123, a parameter extraction unit 135, a moving image extraction unit 136, a multiplex 137, a decoder 138, and a display unit. 139 and a network communication unit 150. Note that the above-described units included in the moving image reproduction device 102b are substantially the same as the respective units included in the moving image reproduction device 102 illustrated in FIG. 8, and thus detailed description thereof is omitted.

  Further, the case where one external connection device 104 shown in FIG. 13 is connected to the moving image playback apparatus 102b has been described as an example. However, the present invention is not limited to this example. For example, a plurality of external devices 104 are connected to the moving image playback apparatus 102b. The present invention can also be implemented when the connection device 104 is connected via the network 103, or when another external connection device 104a is connected from the external connection device 104 connected to the moving image playback device 102b.

  Next, a series of processing when the externally connected device 104 is connected to the moving image reproduction apparatus 102b according to the fifth embodiment via the network 103 will be described with reference to FIG. FIG. 14 is a flowchart showing an outline of a series of processes of the moving image playback apparatus 102 when the externally connected device 104 according to the fifth embodiment is connected.

  As shown in FIG. 14, the SPS and PPS data are output to the multiplex 137 by the parameter extraction unit 135 (S1401), and the moving image stream is output to the multiplex 137 by the moving image extraction unit 136 (S1403). That is, when the external connection device 104 capable of streaming playback is connected via the network 103 while the video stream is output from the multiplex 137 of the video playback device 102b (S1405), the multiplex 137 starts decoding, for example. Even if SPS and PPS are inserted before the first IDR picture to be processed, if it is confirmed in step S1405 that the connection has been established, the multiplex 137 revisits the SPS and PPS before the picture input in the vicinity of the connection timing. (S1407).

  In step S1407, after it is detected that the network 103 is connected, a moving image stream in which SPS and PPS are inserted is immediately transmitted to the externally connected device 104, and the decoder included in the externally connected device 104 appropriately moves the moving image. The stream can be decoded.

  Further, the processes of steps S1403 to S1407 are continuously repeated until the output of the moving image stream to the externally connected device 104 is completed (S1409).

  As described above, since the externally connected device 104 can be decrypted immediately after connection, the user can view the moving image without waiting. In addition, since the recording medium 131 from which the moving image stream is read on the moving image reproducing apparatus 102b records SPS and PPS files separately from the moving image file so that the data amount in the moving image is not compressed during encoding. Degradation of image quality is minimized.

  Conventionally, an externally connected device connected after outputting a moving image stream cannot be replayed until SPS and PPS data are newly transmitted. In this case, when connected from the middle, in the worst case, the moving image could not be decoded, and there was a possibility that the user could not watch at all. In addition, SPS and PPS are not always sent at a good timing, and the user waits until SPS and PPS are sent to the external device 104 after connection.

  As shown in FIG. 15, when the multiplex 137 receives a moving image stream with SPS and PPS added to the head without SSP and PPS being transmitted from the parameter extraction unit 135, the multiplex 137 does not perform insertion processing and performs the decoder 138 as it is. Starts output. Note that since the externally connected device 104 is not yet connected to the network 103, it cannot receive the SPS and PPS assigned to the head. FIG. 15 is an explanatory diagram showing an outline of a series of processes performed by the moving image reproduction apparatus according to the fifth embodiment.

  Next, when the multiplex 137 confirms that the connection of the external device 104 to the network 103 is completed, as shown in FIG. 15, the multiplex 137 immediately receives the SPS, SPS, The PPS data is inserted and output to the external connection device 104 via the decoder 138 or the network 103. The signal indicating completion of connection to the network 103 is transmitted from the control unit 113, for example, but is not limited to such an example.

  Therefore, since the externally connected device 104 can receive the moving image stream in which the SPS and PPS are inserted as soon as it is connected to the network 103, it can reproduce the moving image without waiting for the user.

  In addition, although the case where SPS and PPS are given to the head of the moving image stream shown in FIG. 15 has been described as an example, the present invention is not limited to such an example, and for example, SPS and PPS are not given to the head. Even if it is possible. In such a case, the multiplex 137 inserts, for example, the SPS and PPS received from the parameter extraction unit 135 at the head of the moving image stream and outputs them to the decoder 138.

  Next, referring to FIG. 16, the multiplex 137 to which the SPS and PPS transmitted from the parameter extracting unit 135 are input temporarily holds the SPS and PPS.

  Next, when the multiplex 137 receives a moving image stream including the picture 300-1 and the like from the moving image extraction unit 136, as shown in FIG. 16, the previous SPS and PPS are converted into a moving image stream at a predetermined interval. insert.

  As shown in FIG. 16, SPS and PPS are inserted before the pictures appearing at predetermined intervals, such as before pictures 300a-n + 1, before pictures 300a-2n + 1, and before pictures 300a-3n + 1. . The pictures 300 appearing at predetermined intervals to be inserted can be exemplified by IDR pictures, for example.

  Further, as shown in FIG. 16, the multiplex 137, when SPS and PPS are added at the head of the moving image stream, converts the SPS and PPS to moving images at predetermined intervals based on the head SPS and PPS. It may be inserted into the image stream. FIG. 16 is an explanatory diagram illustrating an outline of a series of processes performed by the moving image reproduction apparatus according to the fifth embodiment.

  As described above, by inserting SPS and PPS data at a predetermined interval, the external connection device 104 is newly connected to the network 103 while the video stream is being output from the multiplex 137, and the connection is connected to the video playback device 102b. Even if it cannot be detected on the side, the external device 104 can receive SPS and PPS transmitted at a predetermined interval and decode the moving image stream. In addition, since the moving image stream is stored without being compressed, the amount of data can be reproduced without deterioration of image quality, and other external devices 104 can be connected to the network during the moving image stream output. Even if it is connected to 103, SPS and PPS can be immediately transmitted to other externally connected devices 104.

  In addition, although the case where SPS and PPS are given to the head of the moving image stream shown in FIG. 16 (before the picture 300-1) is described as an example, the present invention is not limited to such an example. This can be implemented even when no SPS or PPS is added to the head of the stream. In such a case, as described above, the multiplex 137 is inserted at the head of the moving image stream based on the SPS and PPS received from the parameter extraction unit 135.

(About the sixth embodiment)
Next, a moving image processing system 100a according to the sixth embodiment will be described with reference to FIG. FIG. 17 is a block diagram illustrating a schematic configuration of a moving image processing system 100a according to the sixth embodiment.

  As shown in FIG. 17, the moving image processing system 100a includes a moving image reproducing device 102c and an external recording device 105.

  The moving image reproduction apparatus 102c includes a control unit 113b, a recording medium reading unit 123, a parameter extraction unit 135, a moving image extraction unit 136, a multiplex 137, and an external device communication unit 151. Note that the above-described units included in the moving image reproduction device 102c are substantially the same as the respective units included in the moving image reproduction device 102 illustrated in FIG. 8, and thus detailed description thereof is omitted.

  With reference to FIG. 17, a process for externally transferring a moving image stream from the moving image playback device 102c to the external recording device 105 will be described.

  When a signal for starting the process of reading data from the recording medium 131 is sent from the control unit 113b to the recording medium reading unit 123, the recording medium reading unit 123 reads the SPS, PPS file, and the moving image file, and parameter extracting unit 135 and the moving image extraction unit 136 are transmitted to each of them.

  From the file read by the recording medium reading unit 123, SPS and PPS are extracted by the parameter extraction unit 135, and the moving image extraction unit 136 removes the header of the file system and the like to extract the moving image stream.

  When the extracted SPS, PPS data and moving picture stream are input to the multiplex 137, the multiplex 137 multiplexes the SPS, PPS data and the moving picture stream in accordance with a command from the control unit 113b.

  Further, the multiplex 137 transfers a stream obtained by multiplexing the SPS and PPS data and the moving image stream.

  When the moving image stream is transferred from the moving image playback device 102c to the external recording device 105 from the head of the moving image stream, that is, from the first frame, as shown in FIG. 18, the multiplex 137 receives the head of the moving image stream. If SPS and PPS are assigned in advance, the multiplex 137 performs special insertion processing and the like because the SPS and PPS are also transferred together if they are transferred to the external recording apparatus 105 as they are sequentially from the beginning of the moving picture stream. do not do. FIG. 18 shows the transfer from the first frame when the moving image file is recorded on the recording medium by adding SPS and PPS to the moving image file in the moving image reproducing apparatus according to the sixth embodiment. It is explanatory drawing which shows the outline of operation | movement.

  In FIG. 18, the case where SPS and PPS are added to the head of the moving image stream has been described as an example. However, the present invention is not limited to this example. For example, SPS and PPS are added to the head of the moving image stream. Even if not, it can be implemented. In such a case, the multiplex 137 receives the SPS and PPS from the parameter extraction unit 135, assigns the SPS and PPS to the head of the moving image stream, and then outputs them to the external recording device 105 in order from the first frame.

  Next, as shown in FIG. 19, when transferring a moving image stream from an arbitrary position or a random position on the moving image stream from the moving image reproducing device 102c to the external recording device 105, the multiplex 137 performs moving image extraction. SPS and PPS are inserted in accordance with a command from the control unit 113b at the head (first picture corresponding to an arbitrary position) of the moving image stream received from the unit 136.

  In the above case, the multiplex 137 needs to insert SPS and PPS data into the moving image stream at least once when transferring the moving image stream to the external recording apparatus 105. Each time SPS, PPS data and a moving picture stream are multiplexed.

  More specifically, for example, as shown in FIG. 19, when transferring the picture 300-2 from the moving image playback device 102 c to the external recording device 105, the multiplex 137 uses the SPS and PPS from the parameter extraction unit 135. The moving image stream inserted before the picture 300-2 is transmitted to the external recording device 105.

  Similarly, for example, as illustrated in FIG. 19, when transferring pictures 300-4 to 300-6 from the moving image playback device 102 c to the external recording device 105, the multiplex 137 includes the SPS, A moving image stream in which the PPS is inserted before the leading picture 300-4 of the pictures is transmitted to the external recording device 105.

  FIG. 19 shows an outline of the transfer operation from an arbitrary frame when the SPS and PPS are added to the moving image file and recorded on the recording medium in the moving image reproducing apparatus according to the sixth embodiment. It is explanatory drawing shown.

  From the above, the SPS and PPS data is inserted at the head of the moving image stream transferred from the multiplex 137 to the external recording device 105, and the moving image stream with the SPS and PPS inserted is transferred to the external recording device 105. By doing so, it is possible to transfer data from any position without compressing the amount of data during encoding.

  In FIG. 19, the case where SPS and PPS are added to the head of the moving image stream has been described as an example. However, the present invention is not limited to this example. For example, SPS and PPS are added to the head of the moving image stream. Even if not, it can be implemented. In such a case, the multiplex 137 receives the SPS and PPS from the parameter extraction unit 135, assigns the SPS and PPS to the head of the moving image stream, and outputs them to the external recording device 105.

  As described above, the series of processes described above can be performed by dedicated hardware or software. When a series of processing is performed by software, a program constituting the software is installed in an information processing apparatus such as a general-purpose computer or microcomputer, and the information processing apparatus is used as the moving image recording apparatus 101 and / or moving image reproduction. It functions as the device 102.

  The program can be recorded in advance in a hard disk drive (HDD) or ROM as a recording medium built in the computer.

  Alternatively, the program is not limited to a hard disk drive, but a removable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto Optical) disk, DVD (Digital Versatile Disc), magnetic disk, and semiconductor memory. In addition, it can be stored (recorded) temporarily or permanently. Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, and is transferred from the download site to the computer wirelessly via a digital satellite broadcasting artificial satellite, or a LAN (Local Area Network) or the Internet. Such a program can be transferred to a computer via a network, and the computer can receive the program transferred in this way and install it on a built-in hard disk drive.

  Here, in this specification, the processing steps for describing a program for causing a computer to perform various processes do not necessarily have to be processed in time series according to the above-described processing order, and are executed in parallel or individually. Processing (for example, parallel processing or object processing).

  Further, the program may be processed by one computer, or may be processed in a distributed manner by a plurality of computers.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, this invention is not limited to this example. It is obvious for a person skilled in the art that various changes or modifications can be envisaged within the scope of the technical idea described in the claims, and these are naturally within the technical scope of the present invention. It is understood that it belongs.

  In the above embodiment, the case where the parameter file creation unit 115 and the motion image file creation unit 116 provided in the moving image recording apparatus 101 are separate is described as an example, but the present invention is not limited to such an example. For example, the parameter file creating unit 115 and the moving image file creating unit 116 provided in the moving image recording apparatus 101 may be integrated.

  In the above embodiment, the case where the parameter extracting unit 135 and the moving image extracting unit 136 included in the moving image reproducing apparatus 102 are separate is described as an example, but the present invention is not limited to such an example. For example, the parameter extraction unit 135 and the moving image extraction unit 136 included in the moving image reproduction apparatus 102 may be integrated.

  In the above embodiment, the case where the moving image reproducing apparatus 102 includes the multiplex 137 has been described as an example. However, the present invention is not limited to this example as long as SPS and PPS can be inserted into the moving image stream. .

  In the above embodiment, the imaging unit 111, the encoder 112, the control unit 113, the extraction unit 114, the parameter file creation unit 115, the moving image file creation unit 116, and the recording medium writing unit 128 included in the moving image recording apparatus 101 are hardware. Although the case where it consisted of the wear was demonstrated and demonstrated to the example, this invention is not limited to this example. For example, at least one of the imaging unit 111, the encoder 112, the control unit 113, the extraction unit 114, the parameter file creation unit 115, the moving image file creation unit 116, or the recording medium writing unit 128 includes one or two or more. It may be a program composed of modules or components.

  In the above-described embodiment, the control unit 113b, the recording medium reading unit 123, the parameter extraction unit 135, the moving image extraction unit 136, the multiplex 137, the decoder 138, and the display unit 139 provided in the moving image playback apparatus 102 are hardware. However, the present invention is not limited to such an example. For example, at least one of the control unit 113b, the recording medium reading unit 123, the parameter extraction unit 135, the moving image extraction unit 136, the multiplex 137, the decoder 138, or the display unit 139 includes one or more modules or components. It may be the case of the program comprised from.

  In the above embodiment, the moving image recording apparatus 101 is not limited to the case where the moving image stream is recorded on the recording medium 131. For example, the moving image recording apparatus 101 reads the moving image stream from the recording medium 131, and A recording / reproducing apparatus capable of decoding and reproducing a moving image may be used.

  In the above embodiment, the moving image playback apparatus 102 is not limited to the case where the moving image stream is read from the recording medium 131, decoded, and the moving image is played back. A recording / reproducing apparatus capable of recording the stream on the recording medium 131 may be used.

  The present invention relates to H.264. H.264 video encoding apparatus, H.264 The present invention can be applied to a moving image playback apparatus that decodes using the H.264 system.

It is a block diagram which shows an example of a schematic structure of the moving image recording device concerning 1st Embodiment. It is a block diagram which shows the schematic structure of the extraction part shown in FIG. It is explanatory drawing which shows an example of the outline of a data structure. It is explanatory drawing which shows the schematic structure of the data recorded on a recording medium. It is a block diagram which shows an example of a schematic structure of the moving image recording device concerning 2nd Embodiment. It is a block diagram which shows an example of a schematic structure of the data amount extraction part concerning 2nd Embodiment. It is explanatory drawing which shows an example of the outline of the data structure concerning 2nd Embodiment. It is a block diagram which shows an example of a schematic structure of the moving image reproducing device concerning 3rd Embodiment. It is explanatory drawing which shows the outline of the insertion process by the multiplex 137 of the moving image reproducing device concerning 3rd Embodiment. It is explanatory drawing which shows the outline of the insertion process by the multiplex 137 of the moving image reproducing device concerning 3rd Embodiment. It is a block diagram which shows an example of a schematic structure of the moving image reproducing device concerning 4th Embodiment. It is a block diagram which shows the schematic structure of the SPS * PPS extraction part shown in FIG. It is a block diagram which shows the schematic structure of the moving image processing system concerning 5th Embodiment. It is a flowchart which shows the outline of a series of processes by the moving image reproducing apparatus when the external connection apparatus concerning 5th Embodiment connects. It is explanatory drawing which shows the outline of a series of processes by the moving image reproducing device concerning 5th Embodiment. It is explanatory drawing which shows the outline of a series of processes by the moving image reproducing device concerning 5th Embodiment. It is a block diagram which shows schematic structure of the moving image processing system concerning 6th Embodiment. It is explanatory drawing which shows the outline of a series of processes by the moving image reproducing device concerning 6th Embodiment. It is explanatory drawing which shows the outline of a series of processes by the moving image reproducing device concerning 6th Embodiment. H. 2 is an explanatory diagram illustrating an outline of a format of SPS and PPS in H.264.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Moving image processing system 101 Moving image recording device 102 Moving image reproducing device 103 Network 104 External connection apparatus 105 External recording device 111 Imaging part 112 Encoder 113 Control part 114 Extraction part 115 Parameter file creation part 116 Moving image file creation part 121 Data amount Extraction unit 123 Recording medium reading unit 128 Recording medium writing unit 131 Recording medium 135 Parameter extraction unit 136 Moving image extraction unit 137 Multiplex 138 Decoder 139 Display unit 140 SPS / PPS extraction unit 201 Stream analysis unit 203 Counter 205 SPS, PPS total Data amount setting unit 207 Comparator 209 Output selection unit 210 SPS and PPS total data amount setting unit 221 Stream analysis unit 223 Counter 227 SPS / PPS buffer 300 p Tea

Claims (8)

A moving image processing apparatus that encodes moving image data and generates a stream,
Decoding parameter set generation means for generating a decoding parameter set used for decoding the stream by an encoder ;
The total data amount of the decoding parameter set is calculated from the moving image stream encoded by the encoder, and the decoding parameter set and the moving image are selectively output according to the comparison result between the total data amount of the decoding parameter set and the counter value. Extraction means to
A first file creation unit that creates a file based on the decoding parameter set generated by the decoding parameter set generation unit and creates a decoding parameter set file from the decoding parameter set output from the extraction unit;
A second file creation unit that creates a file based on the encoded stream and creates a moving image file from the moving image output from the extraction unit;
A moving image comprising: a recording medium recording unit that records the decoding parameter set file and the moving image file created by the first file creating unit and the second file creating unit on a recording medium; Image processing device.
The first file creation means further generates decoding parameter set data amount information indicating the data amount of the decoding parameter set file based on the decoding parameter set file ,
The recording medium recording means, the moving image processing apparatus according to said decoding parameter set data amount information and the moving image file, to claim 1 you and recording on the recording medium. Wherein the beginning of the moving picture stream of the moving image file recorded on the recording medium, the moving image processing apparatus according to claim 1 you, characterized in that said decoding parameter set is not granted. Recording medium reading means for reading a moving image file recorded on the recording medium ;
A moving image stream extracting unit that extracts a moving image stream that is a payload portion from the moving image file read by the recording medium reading unit ;
A parameter set that will be attached to the head during the decoding of the video stream, a decoding parameter set extracting means for extracting a decoding parameter sets used in the decoding of the video stream,
Inserting means for inserting the decoding parameter set before a picture corresponding to a position where decoding of the moving image stream starts ;
Reading the video stream, and decoding means based on the decoded parameter set which is inserted into the video stream, decodes the video stream,
Network communication means capable of outputting data to the outside ,
The inserting means, after confirming that a predetermined device is connected to the network from the outside, inserts the decoding parameter set in front of a picture corresponding to a position where decoding of the moving picture stream starts, and moving picture processor you wherein also be output from the unit. In the recording medium, a decoding parameter set file in which a decoding parameter set used for decoding a moving image stream is filed is recorded,
The recording medium reading means reads the moving image file and the decoding parameter set file from the recording medium as separate files,
Said decoding parameter set extraction unit, the moving image processing apparatus according to claim 4 you characterized by based on the decoding parameter set file read from the recording medium that to extract the decoded parameter set. Wherein the beginning of the moving picture stream of the moving image file recorded on the recording medium, the moving image processing apparatus according to claim 5 you, characterized in that said decoding parameter set is not not been granted. Said decoding parameter set, a moving image processing apparatus according to any one of claims 4 to 6 you, characterized in that to be inserted by the insertion means repeatedly at predetermined intervals. It said insertion means, the moving image processing apparatus according to claim 4 you, characterized in that to insert the decoding parameter set in the video stream at predetermined intervals.

Images