JP2008252931A - Decoding apparatus and method, encoding apparatus and method, image processing system, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform encoding processing with satisfactory image quality even if an error occurs in a bit stream, while information indicating that the error occurs in the bit stream is produced. <P>SOLUTION: A decoding apparatus 10 includes a means 12 for producing image data by decoding a bit stream, a means 12 for producing encoding parameters for each hierarchy, and a means 12 for producing an error flag that indicates whether the encoding parameters can be used effectively or not. An encoding apparatus 30 includes a means 41 for judging whether the encoding parameters can be used effectively or not based on the error flag, a means 36 for calculating encoding parameters when the encoding parameters are not effective, and means 38, 39 for performing encoding of the image data using the encoding parameters from the decoding apparatus 10 when the encoding parameters are effective, and performing encoding of the image data using the calculated encoding parameters when the encoding parameters from the decoding apparatus are not effective. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a decoding apparatus and method suitable for use in a system for re-encoding image data, an encoding apparatus and method, an image processing system, and an image processing method, and in particular, decodes a bit stream to generate image data. And a decoding apparatus and method for generating encoding parameters used when re-encoding, an encoding apparatus and method for performing re-encoding using image data and encoding parameters from the decoding apparatus, and image data by decoding a bitstream The present invention relates to an image processing system and an image processing method that generate encoding parameters used when re-encoding and generate re-encoding using image data and encoding parameters from a decoding device.

  In an editing process for an image signal encoded by the MPEG (Moving Picture Experts Group) method in a conventional encoder, an ordinary video signal is decoded when the encoded image signal is once decoded and the decoded image is re-encoded. There is a case where the image quality is greatly deteriorated as compared with the case where encoding is performed. As a cause of such image degradation, there is a mismatch between various encoding parameters such as picture type and motion vector before and after re-encoding, that is, when encoding is first performed in MPEG format and when re-encoding is performed. .

  An example of a case where image quality degradation occurs due to mismatched picture types among the mismatched encoding parameters will be described. For example, the number of pictures conforming to the MPEG system in which B0, B1, I2, B3, B4, P5, B6, B7, P8 and I (Intra) picture, P (Predictive) picture, and B (Bidirectionally predictive) picture are arranged. N = 9, there is a picture type for an input decoded image corresponding to one GOP (Group of Pictures) with a number of pictures M = 3 from one I or P picture to the next I or P picture, that is, a decoded image to be re-encoded . When the GOP arranged as described above is phase-locked with B0, B1, I2, B3, B4, P5, B6, B7, and P8, the above-mentioned reference image is used as a reference image for re-encoding. In the input decoded image, what was an I picture is used as it is.

  On the other hand, in the case where B0, I1, B2, B3, P4, B5, B6, and P7 are arranged and phase lock is not performed, the degree of image quality degradation is, for example, as in the third picture (B2). A large B picture is used as a reference image when re-encoding is performed. As a result, the re-encoding accuracy is lowered and a large image quality degradation occurs.

  In addition, as described above, in the conventional encoder, if not only the picture type but also other encoding parameters such as motion vectors are matched before and after re-encoding, the encoding parameters are recalculated at the time of re-encoding. The image quality degradation when re-encoding is smaller than when using the value calculated in the above. In the conventional encoder, image quality deterioration due to re-encoding hardly occurs by matching all encoding parameters including the picture type and motion vector before and after re-encoding.

  By the way, when an error is added in the bitstream transmission path or an encoder that encodes the original bitstream uses an encoding parameter that does not conform to the MPEG system, the syntax is included in the bitstream received by the decoder. An error may occur.

  Thus, when a syntax error occurs in the bitstream, the decoder performs error recovery by searching the next start code (32-bit synchronization code) from the position where the bitstream error occurred. To do. Since the decoder cannot decode an image from the bitstream from the position where the error occurred in the bitstream to the position where error recovery was possible, it is lost due to a syntax error using information on the already decoded image. Perform error concealment on the image portion. At this time, the decoder performs a process of copying and displaying an image from a past display image to an image portion lost due to an error, for example.

International Publication No. 98/03017

  In the case of re-encoding with all the encoding parameters including picture type and motion vector being matched before and after re-encoding as described with reference to a conventional encoder as an example, in the bit stream received by the decoder When a syntax error occurs, an image reproduced by the decoder by the error concealment is input to the encoder, and the encoding parameter at that time is not input, or an encoding parameter having an incorrect value is input. Become. When the encoder performs encoding using an incorrect encoding parameter from the decoder, an encoding result with extremely large image quality deterioration is obtained.

  As described above, conventionally, encoding is often performed on the encoder side using the encoding parameter from the decoder regardless of whether a syntax error or the like has occurred in the bitstream input to the decoder. Therefore, the encoder has a problem that normal processing cannot be performed by encoding an image using an incorrect encoding parameter.

  Therefore, the present invention has been proposed in view of the above-described circumstances, and a decoding apparatus and method capable of generating information indicating that an error has occurred in the bitstream, and an error in the bitstream. It is an object of the present invention to provide an encoding apparatus and method, an image processing system, and an image processing method that can perform encoding processing with good image quality.

  A decoding apparatus according to the present invention that solves the above-described problems includes a decoding unit that decodes an input bitstream to generate image data, and each of the units used when re-encoding the image data decoded by the decoding unit. The encoding parameter generating means for generating the encoding parameter for the layer and the encoding parameter for each layer generated by the encoding parameter generating means when the image data is re-encoded can be used effectively. And an error flag generation means for generating an error flag indicating whether or not.

  In addition, the decoding method according to the present invention generates image data by decoding an input bitstream, and generates encoding parameters for each layer used when re-encoding the decoded image data. The method further comprises generating an error flag indicating whether or not the encoding parameter for each layer can be used effectively when re-encoding the image data.

  Furthermore, the encoding apparatus according to the present invention is configured such that the image data input from the decoding apparatus is based on an error flag indicating whether the encoding parameter for each layer input from the decoding apparatus can be used effectively. Based on a determination unit that determines whether or not the encoding parameter can be used effectively when performing re-encoding, and a determination result from the determination unit that the encoding parameter from the decoding device is not effective The encoding parameter calculation means for calculating the encoding parameter using the image data from the decoding apparatus, and the code input from the decoding apparatus when the determination means determines that the encoding parameter from the decoding apparatus is valid. When the image data is encoded using the encoding parameter and the determination unit determines that the encoding parameter from the decoding device is not valid Characterized in that it comprises a coding means for coding the image data by using the coding parameters generated by the serial coding parameter calculating means.

  Furthermore, in the encoding method according to the present invention, an image input from the decoding device based on an error flag indicating whether or not the encoding parameter for each layer input from the decoding device can be used effectively. Based on a determination process for determining whether or not the encoding parameter can be used effectively when re-encoding data, and a determination result indicating that the encoding parameter from the decoding apparatus is not valid, the decoding apparatus Encoding parameter calculation processing for calculating encoding parameters using image data from the image data, and image data using the encoding parameters input from the decoding device when it is determined that the encoding parameters from the decoding device are valid When it is determined that the encoding parameter from the decoding device is not valid, the code generated by the above encoding parameter calculation process And having a coding process of coding image data by using the parameters.

  Furthermore, the image processing system according to the present invention includes a decoding unit that decodes an input bitstream to generate image data, and each layer used when re-encoding the image data decoded by the decoding unit. Whether or not the encoding parameter generating means for generating the encoding parameters and the encoding parameter for each layer generated by the encoding parameter generating means when re-encoding the image data can be used effectively A decoding device comprising an error flag generating means for generating an error flag indicating whether or not the encoding parameter is enabled when re-encoding is performed based on the error flag for each layer input from the decoding device. Determining means for determining whether or not it can be used, and determining means for indicating that the encoding parameter from the decoding device is not valid. Based on the determination result, the encoding parameter calculation means for calculating the encoding parameter using the image data from the decoding device, and the determination means determines that the encoding parameter from the decoding device is valid. The image data is encoded using the encoding parameter input from the decoding device, and is generated by the encoding parameter calculation unit when the determination unit determines that the encoding parameter from the decoding device is not valid. And an encoding device including encoding means for encoding image data using an encoding parameter.

  Furthermore, another image processing system according to the present invention includes a decoding unit that decodes an input bitstream to generate image data, and an inverse quantization unit that converts the image data from the decoding unit into a DCT transform coefficient. Quantization means, quantization means for quantizing the DCT transform coefficient from the inverse quantization means to make image data, and codes for each layer used when re-encoding the image data decoded by the decoding means Encoding parameter generating means for generating encoding parameters, and whether or not the encoding parameters for each layer generated by the encoding parameter generating means when the image data is re-encoded can be used effectively An error flag generating means for generating an error flag to be indicated, and calculating an encoding parameter based on the error flag from the error flag generating means Encoding parameter calculation means, encoding means for encoding image data from the quantization means using the encoding parameters from the encoding parameter generation means or the encoding parameter calculation means, and the error flag When the encoding parameter is valid based on the error flag generated by the generation means, the image data is encoded using the encoding parameter generated by the encoding parameter generation means, and the error flag generation means Control means for encoding image data using the encoding parameter generated by the encoding parameter calculation means when the encoding parameter is not valid based on the generated error flag. It is.

  Still further, the image processing method according to the present invention decodes an input bit stream to decode image data, a coding parameter for each layer used when re-encoding the image data, and re-encoding of the image data. A decoding process for generating an error flag indicating whether or not the encoding parameter for each layer can be used effectively when performing the above, and a dequantization process for dequantizing the image data into a DCT transform coefficient; A quantization process for quantizing the DCT transform coefficient into image data, a coding parameter calculation process for calculating a coding parameter based on an error flag from the error flag generating means, and a code based on the error flag. When the encoding parameters are valid, the image data is encoded using the encoding parameters generated by the decoding process, and the error When the coding parameters based on the lag is not valid, characterized in that it has a coding process of coding image data by using the coding parameters generated by the coding parameter calculation process.

  Since the decoding apparatus and method according to the present invention can generate an error flag indicating whether or not the encoding parameter for each layer can be used effectively when re-encoding image data, After decoding, the encoding parameters used when re-encoding can be controlled. Therefore, according to this decoding apparatus and method, for example, when a decoded encoding parameter is invalid, re-encoding is performed using the encoding parameter generated by the encoder without using the encoding parameter. Re-encoding can be performed.

  Furthermore, the encoding device and method according to the present invention encodes image data using the encoding parameter input from the decoding device when it is determined that the encoding parameter from the decoding device is valid, and decoding is performed. Since it has an encoding process for encoding image data using the encoding parameter generated by the encoding parameter calculation process when it is determined that the encoding parameter from the apparatus is not valid, the encoding parameter from the decoding apparatus When is valid, re-encoding is performed using the encoding parameter for the input image data, and when the encoding parameter from the decoding device is invalid, re-encoding is performed with the encoding parameter obtained by itself. . Therefore, according to this encoding apparatus and method, since re-encoding is not performed using the encoding parameter indicated by the error flag, re-encoding can be performed accurately, and encoding can be performed with good image quality. The result can be obtained.

  Furthermore, the image processing system according to the present invention indicates whether or not the encoding parameter for each layer generated by the encoding parameter generation means can be used effectively when re-encoding image data. Whether or not the encoding parameter can be effectively used when performing re-encoding based on the decoding apparatus having an error flag generating means for generating an error flag and the error flag for each layer input from the decoding apparatus And a coding parameter calculation unit that calculates a coding parameter using image data from the decoding device based on a determination result from the determination unit that the coding parameter from the decoding device is not valid. When the determination unit determines that the encoding parameter from the decoding device is valid, the encoding parameter input from the decoding device is determined. The image data is encoded using the data, and when the determining means determines that the encoding parameter from the decoding device is not valid, the image data is encoded using the encoding parameter generated by the encoding parameter calculating means. And an encoding device including an encoding means for performing re-encoding using the encoding parameter for the input image data when the encoding parameter from the decoding device is valid, and from the decoding device When the encoding parameter is invalid, re-encoding can be performed with the encoding parameter calculated by the encoding device. Therefore, according to this image processing system, since re-encoding is not performed using the encoding parameter indicated by the error flag, re-encoding can be performed accurately, and the encoding result can be obtained with good image quality. Obtainable.

  Furthermore, the image processing system and the image processing method according to the present invention decode the input bit stream to decode the image data, the encoding parameters for each layer used when re-encoding the image data, and the image data Decoding processing for generating an error flag indicating whether or not the encoding parameter for each layer can be used effectively when performing re-encoding, and encoding for calculating the encoding parameter based on the error flag When the encoding parameter is valid based on the parameter calculation process and the error flag, the image data is encoded using the encoding parameter generated by the decoding process, and the encoding parameter is not valid based on the error flag. Sometimes the encoding of image data using the encoding parameters generated by the above encoding parameter calculation process Therefore, re-encoding is performed using the encoding parameter generated by the decoding process when the encoding parameter is valid, and re-calculation is performed when the encoding parameter generated by the decoding process is invalid. Re-encoding can be performed with the obtained encoding parameters. Therefore, according to the image processing system and the image processing method, since the re-encoding is not performed using the encoding parameter indicated by the error flag, the re-encoding can be performed accurately, and the image quality can be improved. An encoding result can be obtained.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The present invention is applied to, for example, an image processing system 1 configured as shown in FIG.

  The image processing system 1 shown in FIG. 1 includes a decoder 10 and an encoder 30 connected in cascade, a decoder 10 to which an externally encoded bit stream conforming to the MPEG2 standard is input, and the decoder 10 decodes It comprises an encoder 30 that re-encodes image data and outputs it as a bit stream to the outside.

  The decoder 10 includes a buffer 11 to which a bit stream encoded by the MPEG2 system is input from the outside. The buffer 11 temporarily stores the bit stream and outputs it to a variable length decoding unit (VLD: Variable Length Decoder) 12.

  The variable length decoding unit 12 obtains a quantization coefficient and a motion vector by performing variable length decoding processing on the bit stream of the variable length code from the buffer 11 in units of macroblocks (MB), for example, and obtains an inverse quantization unit (Inverse Quantization: IQ) 14.

  In addition, the variable length decoding unit 12 performs the variable length decoding process as described above, detects an encoding parameter added to each MB layer (hereinafter referred to as MB parameter (MB_parameters)), and the memory 13. Output to. Furthermore, the variable length decoding unit 12 detects encoding parameters (Sequence_GOP_Picure_parameters) of the picture layer and higher, and generates parameter determination information (picture_mb_parameters_valid) indicating whether the encoding parameters and MB parameters of the picture layer are valid. And output to the control unit 41 described later.

  The Sequence_GOP_Picure_parameters includes the following parameters defined in the MPEG2 standard. That is, the bit stream input to the variable length decoding unit 12 includes a sequence extension unit (Sequence_extension), a sequence display function extension unit (Sequence_display_extension), and a sequence scalable function extension unit (Sequence_scalable_extension) arranged immediately after the sequence header (Sequence_header). And a GOP header (group_of_pictures_header) is added following these extensions.

  Further, the bitstream includes an I (Inter) picture (intra-frame encoded image), a P (Predictive) picture (inter-frame forward predictive encoded image), and a B (Bidirectionally Predictive) picture (bidirectional predictive encoded image). Extension units such as a picture header (picture_header), a picture encoding function extension unit (picture_coding_extension), a quantization matrix function extension unit (quant_matrix_extension), and a picture display function extension unit (picture_display_extension) that store information related to each picture are included.

  Furthermore, the encoding parameters beyond the picture layer include parameters such as image size (horizontal_size, vertical_size), picture_coding_type, top_field_first, repeat_first_field.

  The MB parameter is an MB layer encoding parameter and includes the following parameters defined in the MPEG2 standard. That is, the MB layer includes a macroblock address, a quantization scale indicating a quantization step size, and a flag indicating that this quantization scale is effective (when MB is a skip MB or a not coded MB, “0” is set. DCT type indicating whether DCT is frame mode or field mode, motion compensation type indicating that motion compensation is performed in frame mode, field mode or dual prime mode, forward, backward or bi-directional motion prediction. This includes an MB mode indicating that the operation is to be performed, a motion vector, a field motion vector reference field, and a dual prime auxiliary vector.

  The variable length decoding unit 12 sets the DCT type as the frame mode when the macroblock to be decoded is “not coded MB”, that is, when the macroblock has a coded_block_pattern other than the intra MB.

  Further, the variable length decoding unit 12 sets the DCT type as a frame mode when skipped MB, sets the motion compensation type as frame prediction, forward prediction when the MB mode is P picture, and immediately preceding MB when the B mode is B picture. The motion vector is 0 for a P picture and the same value as the previous MB for a B picture.

  Further, as described above, the variable length decoding unit 12 outputs various encoding parameters in the sequence layer, the GOP layer, the picture layer, and the MB layer, and as shown in Table 1 below, a 2-bit flag, That is, parameter determination information expressed by a 1-bit flag indicating whether or not the picture layer encoding parameter is valid and a 1-bit flag indicating whether or not the MB parameter is valid is generated.

  That is, the variable length decoding unit 12 sets the flag to “00” when an error (invalid) is included in the encoding parameter and MB parameter of the picture layer, and does not include an error (valid). If the MB parameter contains an error, the flag is set to “10”. If the encoding parameter of the picture layer and the MB parameter do not contain an error, the flag is set to “11” and parameter determination information is generated and output to the control unit 41. To do.

  An example of a case where an error is included in the coding parameter of the picture layer is a case where an error occurs in the bit stream during the decoding process in the variable length decoding unit 12 of the decoder 10. At this time, since the image data output by the error concealment from the decoder 10 until the next GOP header is detected and error recovery is performed does not have picture_coding_type, the coding parameter of the picture layer includes an error, The encoding parameter of the picture layer is determined to be invalid by the control unit 41 described later.

  In addition, examples of the case where an error is included in the MB parameter include a case where there is an MB layer bit stream error in the decoded picture. At this time, an error is included in the MB parameter, and the MB parameter is determined to be invalid by the control unit 41 described later.

  Further, the variable length decoding unit 12 includes a field image input to the switch unit 31 among the image data of the field image sequence output to the encoder 30 via the switch unit 21, and a field when output from the encoder 30. The top_field_first and repeat_first_field are output to the control unit 41 so that the encoder 30 performs encoding as the same image as the image.

  Here, the top_field_first is a flag indicating whether to display either the top field or the bottom field of the frame-structured picture first in the interlaced image, and the repeat_first_field is the frame-structured picture. It is a flag indicating whether or not to repeatedly display the field to be displayed first when it is displayed after the field to be displayed second.

  The memory 13 receives the MB parameter from the variable length decoding unit 12 and outputs a predetermined picture time delay, which will be described later, controlled by the control unit 41 in accordance with the processing timing on the encoder 30 side.

  The inverse quantization unit 14 performs inverse quantization on the image data including the quantization coefficient from the variable length decoding unit 12 for each 8 × 8 pixel block, for example. At this time, the inverse quantization unit 14 performs an inverse quantization process by multiplying the image data by a quantization step to obtain a DCT transform coefficient, and sends the image data to an inverse DCT (Inverse Discrete Cosine Transform: IDCT) unit 15. Output.

  The inverse DCT unit 15 performs inverse transformation of discrete cosine transformation on the image data from the inverse quantization unit 14, for example, in units of 8 × 8 pixel blocks. As a result, the inverse DCT unit 15 calculates each pixel value (luminance, color difference) for each 8 × 8 pixel block. Then, the inverse DCT unit 15 outputs image data composed of pixel values generated by performing inverse transformation to the adder 16.

  The adder 16 adds the image data from the inverse DCT unit 15 and the image data from the motion compensation unit 18 and outputs the result to the control unit 17.

  The motion compensation (MC) unit 18 includes a frame memory (FM) for storing image data positioned before and after in time, and the image data predicted from the frame memory based on the motion vector is stored for each picture type. And the adder 16 performs an addition process to compensate for motion.

  The control unit 17 changes the order of the frames so that the frame image sequence in which the frames are arranged in the decoding order from the adder 16 is a frame image sequence in which the frames are arranged in the display order. 19 output.

  When the re-encoding performed by the subsequent encoder 30 is performed by changing the image size, the image processing unit 19 performs a process of changing the image size for the image data S10 input from the control unit 17. For example, the image processing unit 19 outputs an image of 720 pixels × 480 pixels to the switch unit 21 as image data S30 of 352 pixels × 480 pixels.

  Further, the image processing unit 19 interpolates vertical and horizontal pixels in the picture for each picture constituting the image data from the control unit 17, and has a resolution higher than that when input from the control unit 17. The upsampling process to make a high picture is performed.

  In addition, the image processing unit 19 determines each picture constituting the input bitstream based on the top field first (top_field_first) and repeat first field (repeat_first_field) added to the picture header of each picture. Interlaced image from frame image.

  Further, the image processing unit 19 performs a process of converting a luminance color difference format indicating a ratio of the luminance signal and the color difference signals Cb and Cr for each picture. That is, the image processing unit 19 uses the luminance color difference format in which the color information is reduced in half in the horizontal direction from 4: 2: 0 indicating the color difference format in half in the horizontal direction and the vertical direction for each picture. A process of switching to 4: 2: 2 is performed.

  Then, the image processing unit 19 performs the image data S30 on which the processing for changing the image size, the above-described upsampling processing, the processing for rewriting the picture header to convert from the frame image to the field image, and the processing for converting the luminance color difference format are performed. Is output to the encoder 30 via the switch unit 21.

  When the re-encoding performed by the subsequent encoder 30 is performed without changing the image size, the dummy data adding unit 20 performs, for example, a horizontal size xx as shown in FIG. 2 on the image data input from the control unit 17. A process of adding dummy data to the image data of the size y in the vertical direction (FIG. 2A) (FIG. 2B) is performed, and the image data S20 to which the dummy data is added is transferred via the switch unit 21 to the encoder 30. Output to.

  At this time, when the dummy data adding unit 20 determines that the image size in units of pictures of the image data S10 from the control unit 17 is smaller than 720 pixels × 480 or 576 pixels, the dummy data adding unit 20 adds dummy data, and 720 pixels × 480. Alternatively, a process of making image data of 576 pixels is performed. That is, when the dummy data adding unit 20 determines that the image size in units of pictures of the image data S10 from the control unit 17 is smaller than 720 pixels × 480 or 576 pixels, in the horizontal direction of the picture indicated by the image data (720 -X) A process of adding dummy data of 480 pixels or 576 pixels and adding dummy data of x pixels x (480-y) or (567-y) pixels in the vertical direction of the picture is performed.

  The switch unit 21 opens and closes in accordance with a control signal from the control unit 41 to be described later. When the terminal a is connected, the switch unit 21 outputs the image data S20 from the dummy data adding unit 20 to the encoder 30 and the terminal b is connected. As a result, the image data S30 from the image processing unit 19 is output to the encoder 30.

  The control unit 41 receives the encoding parameters (Sequence_GOP_Picure_parameters) of the picture layer or higher from the variable length decoding unit 12, and outputs Sequence_parameters, GOP_parameters, and Picure_parameters to the encoder 30. In addition, the parameter determination information described with reference to Table 1 is input from the variable length decoding unit 12 to the control unit 41. The control unit 41 performs processing for storing the input encoding parameter and parameter determination information in the memory 42.

  The control unit 41 outputs control signals to the switch unit 21, the switch unit 31, and the switch unit 37 in order to control the processing timing of the decoder 10 and the encoder 30, and controls the open / close timing of each switch unit 21, 31, 37. To do. At this time, the control unit 41 refers to the parameter determination information to determine whether the encoding parameter of the picture layer is valid, determines whether the MB parameter is valid, and sets the switch unit 37. A control signal for opening / closing operation is generated. At this time, the control unit 41 stores the parameters in the memory 42 functioning as a working memory and performs the above-described processing.

  The encoder 30 includes a switch unit 31 to which the image data S20 or the image data S30 is input from the switch unit 21.

  The switch unit 31 opens and closes in accordance with a control signal from the control unit 41, inputs the image data S20 from the decoder 10 when the terminal a is connected, and outputs the image data S21 to the dummy data removal unit 32. When the terminal b is connected, the image data S30 from the decoder 10 is input and the image data S31 is output to the image processing unit 33.

  As shown in FIG. 2C, the dummy data removing unit 32 performs processing for removing dummy data on the image data S <b> 21 from the switch unit 31 and outputs the image data S <b> 40 to the motion estimating unit 34. That is, the dummy data removing unit 32 performs a process of removing the dummy data added to the image data S20 (FIG. 2B) to which the dummy data from the decoder 10 is added, and the dummy data is added. The previous image data.

  The image processing unit 33 changes the image size of the image data S31 input from the switch unit 31 when the image size of the image data S31 from the switch unit 31 is changed by the image processing unit 19 described above. Processing etc. are performed and image data S40 is output to the motion estimation part 34. FIG. For example, the image processing unit 33 outputs an image of 352 pixels × 480 pixels to the motion estimation (ME) unit 34 as an image of 720 pixels × 480 pixels.

  The image processing unit 33 performs 2-3 pull-down processing on the image data S31. That is, the image processing unit 33 performs a process of converting, for example, a film picture of 24 pictures / second into image data of 30 frames / second.

  The image processing unit 33 further reduces the resolution of each picture constituting the image data S31 by reducing the pixels in the vertical and horizontal directions in the picture so that the picture has a lower resolution than when the image data S31 is input from the switch unit 31. Perform sampling processing.

  Furthermore, the image processing unit 33 configures the input bit stream based on the top field first (top_field_first) and repeat first field (repeat_first_field) added to the header of the picture layer of each picture. Each picture is changed from an interlaced image to a frame image.

  Further, the image processing unit 33 performs a process of converting a luminance color difference format indicating a ratio of the luminance signal and the color difference signals Cb and Cr for each picture. That is, the image processing unit 33 changes the luminance color difference format in which the color information is reduced in half in the horizontal direction and the vertical direction for each picture from 4: 2: 2 indicating the color difference format in which the color information is reduced in half in the horizontal direction. A process of switching to 4: 2: 0 is performed.

  Then, the image processing unit 33 converts the image size, the above-described 2-3 pull-down process, down-sampling process, the process of rewriting the picture header to convert from an interlaced image to a frame image, and the luminance / color-difference format. The processed image data S40 is output to the motion estimation unit 34.

  The motion estimation unit 34 performs a process of calculating a motion vector in MB units for the image data S40 from the switch unit 31 or the dummy data removal unit 32. Here, the motion estimation unit 34 calculates a motion vector by performing forward prediction that performs prediction from past images, backward prediction that performs prediction from future images, and bidirectional prediction that performs predictions from past and future images. Perform the process. Then, the motion estimation unit 34 outputs the calculated motion vector and MB-unit image data to the control unit 35 and the MB parameter calculation unit 36.

  The control unit 35 changes the order of the frames so that the frame image sequence in which the frames are arranged in the display order from the motion estimation unit 34 is changed to a frame image sequence in which the frames are arranged in the encoding order. The data is output to the encoding unit 38.

  The MB parameter calculation unit 36 uses the motion vector in MB unit and the image data in MB unit from the motion estimation unit 34 to generate MB parameters and output them to the switch unit 37.

  The switch unit 37 opens and closes in accordance with a control signal from the control unit 41, inputs the MB parameter calculated by the MB parameter calculation unit 36 when the terminal a is connected, and outputs the MB parameter to the encoding unit 38. By connecting b, the MB parameter from the decoder 10 is input and output to the encoding unit 38.

  The encoding unit 38 includes an adder 51 to which image data in MB units from the control unit 35 is input, image data having DCT conversion coefficients by performing discrete cosine transform processing on the image data in MB units from the adder 51. A DCT processing unit (DCT) 52 that performs quantization, a quantization processing unit (Q) 53 that performs quantization on DCT transform coefficients that form image data from the DCT processing unit 52, an inverse quantization processing unit (IQ), An inverse DCT processing unit (IDCT) 55, an adder 56, a frame memory (FM) 57a, and a motion compensation unit (MC) 57b are included.

  Here, the motion compensation unit (MC) 57b includes a frame memory (FM) that stores image data positioned before and after in time, and reads and adds image data to be predicted from the frame memory based on a motion vector. Motion compensation is performed by causing the adder 51 and the adder 56 to perform addition processing.

  Such an encoding unit 38 is configured as described above, and encodes image data in MB units so that an image in which I pictures, P pictures, and B pictures are arranged in accordance with the MPEG2 standard. Data is generated and output to the variable length encoding unit 39.

  The variable length encoding unit 39 performs variable length compression on the MB-unit image data composed of the quantized coefficients from the encoding unit 38 by using Huffman encoding, and outputs the result to the buffer 40 as a bit stream.

  The buffer 40 temporarily stores the bit stream from the variable length encoding unit 39 and outputs the re-encoded bit stream to the outside.

  Next, in the image processing system 1 configured as described above, a timing chart when the encoding parameters of the picture layer, parameter determination information, and the like are input from the encoder decoder 10 to the encoder 30 will be described with reference to FIG. .

  According to FIG. 3, first, as shown in FIG. 3A, the decoder 10 is arranged in the order of pictures I2, B0, B1, P5, B3, and B4 over picture times t1 to t6. The bit stream is input to the buffer 11.

  Here, at the same timing that the bit stream is input to the buffer 11 over the picture times t1 to t6, as shown in FIG. (Sequence_GOP_Picure_parameters) is detected in the order of pictures I2, B0, B1, P5, B3, and B4 and output to the control unit 41.

  Further, the variable length decoding unit 12 of the decoder 10 generates parameter determination information (picture_mb_parameters_valid) every time the picture is subjected to variable length decoding, and, as shown in FIG. 3 (d), stores it in the buffer 11 shown in FIG. 3 (a). Is output with a delay of one picture time with respect to the input timing of the bit stream. Here, the variable length decoding unit 12 outputs the parameter determination information for the pictures I2, B0, B1, P5, B3, and B4 to the control unit 41 over the picture times t2 to t7.

  Further, from the switch unit 21 of the decoder 10, the image data subjected to the above-described processing by the image processing unit 19 or the dummy data adding unit 20 is input to the buffer 11 at the bit stream input timing as shown in FIG. On the other hand, the picture B0, B1, I2, B3, B4, and P5 are output to the encoder 30 over the picture time t2 to t7 with a delay of one picture time.

  In contrast to the processing on the decoder 10 side shown in FIGS. 3A to 3D, the encoder 30 outputs image data from the decoder 10 shown in FIG. 3B, as shown in FIG. 3E. On the other hand, image data is input in the order of pictures B0, B1, I2, B3, B4, and P5 over a picture time t2 to t7 without delay.

  Also, in the encoder 30, as shown in FIG. 3 (f), the encoding parameters (Sequence_GOP_Picure_parameters) of the picture layers and higher in the order of pictures I2, B0, B1, P5, B3, B4 from the decoder 10 are shown in FIG. ) Is input via the control unit 41 without delay from the output timing from the variable length decoding unit 12 shown in FIG.

  Furthermore, in the encoder 30, parameter determination information is input from the variable length decoding unit 12 without delay in the order of pictures I2, B0, B1, P5, B3, and B4 from the decoder 10, as shown in FIG. . At this time, the parameter determination information for the pictures I2, B0, B1, P5, B3, and B4 from the variable length decoding unit 12 is input to the encoder 30 through the control unit 41 over the picture times t2 to t7. .

  Furthermore, in the encoder 30, as shown in FIG. 3 (h), the delay time required to encode the GOP image data with M = 3 is 3 picture time from the input timing of the image data in FIG. 3 (e). The bit stream is output from the buffer 40 over the picture times t5 to t10 at the delay timing of. At this time, the encoder 30 outputs pictures I2, B0, B1, P5, B3, and B4 in this order.

  Here, on the encoder 30 side, it is necessary to buffer the coding parameters of the picture layer in the memory 42 for the picture time required for encoding. In this example, when the control unit 41 outputs a bit stream composed of GOPs of M = 3 in accordance with the MPEG2 standard, the control unit 41 buffers the encoding parameter for 4 picture times (t1 to t4). By performing the control, control is performed so as to perform processing for outputting an encoded bitstream to which an encoding parameter is added.

  Next, a timing chart when MB parameters and the like are input from the decoder 10 to the encoder 30 in the above-described image processing system 1 will be described with reference to FIG.

  In the decoder 10, as shown in FIG. 4A, a bit stream in which pictures are arranged in the order of pictures I 2, B 0, B 1, P 5, B 3, B 4 is input to the buffer 11 over picture times t 1 to t 6. Is done.

  Further, in the variable length decoding unit 12 of the decoder 10, as shown in FIG. 4C, the MB parameter (MB_parameters) at the same timing without delay with respect to the input timing of the bit stream to the buffer 11 over the picture times t1 to t6. ) Is generated by decoding. At this time, the variable length decoding unit 12 sequentially generates MB parameters for the pictures I2, B0, B1, P5, B3, and B4.

  Then, in the variable length decoding unit 12 of the decoder 10, the MB parameter generated at the timing shown in FIG. 4C is delayed by 4 picture times as shown in FIG. MB parameters for pictures I2, B0, B1, P5, B3, and B4 are output in order over t10.

  Here, the control unit 41 controls the MB parameter for 4 picture time (4 frames) to be buffered in the memory 13 and output to the encoder 30 when M = 3 of the GOP.

  Furthermore, as shown in FIG. 4B, the decoder 10 delays the picture timings B0, B1, I2, and the encoder 30 over the picture times t2 to t7 with a delay of one picture time with respect to the input timing of the bitstream. Image data is output in the order of B3, B4, and P5.

  In contrast to the processing on the decoder 10 side shown in FIGS. 4A to 4D, in the encoder 30, as shown in FIG. 4E, image data from the decoder 10 is output at the output timing shown in FIG. On the other hand, image data is input in the order of pictures B0, B1, I2, B3, B4, and P5 over a picture time t2 to t7 without delay.

  The encoder 30 sequentially inputs MB parameters for pictures I2, B0, B1, P5, B3, and B4 over picture times t5 to t10 without delay with respect to the output timing of MB parameters shown in FIG. Is done.

  Furthermore, in the encoder 30, as shown in FIG. 4 (f), a bit stream generated by encoding or the like in the encoding unit 38 using the MB parameter input at the input timing of FIG. The picture data is input in the order of pictures I2, B0, B1, P5, B3, and B4 over picture times t5 to t10 with a delay of 3 picture times with respect to the input timing of image data input to the encoder 30 shown in 4 (e).

  Here, on the encoder 30 side, it is necessary to buffer the coding parameters of the picture layer in the memory 42 for the picture time required for encoding. In this example, when outputting a bit stream composed of M = 3 GOPs, the control unit 41 outputs the bit stream delayed by three picture times (t2 to t4) with respect to the input timing of the image data. To control.

  Next, processing according to the parameter determination information from the decoder 10 in the image processing system 1 described above will be described.

  The image processing system 1 causes the control unit 41 to determine whether the encoding parameters and MB parameters of the picture layer and higher are valid or invalid based on the parameter determination information generated by the variable length decoding unit 12, and the control unit 41 uses the encoder 30. A process for selecting an encoding parameter to be used when encoding on the side is performed.

  Specifically, the control unit 41 detects, based on the parameter determination information from the variable length decoding unit 12, a flag indicating that the encoding parameters and MB parameters of the picture layer and higher do not include an error and are valid. Then, the encoder 30 encodes the image data input from the decoder 10 using the same encoding parameter as the encoding parameter added to the bit stream input to the decoder 10.

  In addition, in the case where the bit rate is lowered by encoding with the encoder 30, the control unit 41 performs encoding by changing only the encoding parameter indicating the quantization scale with the encoder 30, and for other encoding parameters. Control is performed so that encoding is performed using the encoding parameter added to the bitstream input to the decoder 10.

  Further, when the encoder 30 performs encoding by changing the image size by the encoder 30, the control unit 41 performs encoding using only the encoding parameters of the picture layer or higher. At this time, the control unit 41 performs control so that at least picture_coding_type, top_field_first, and repeat_first_field have the same encoding parameters for encoding and the encoding parameters added to the bit stream input to the decoder 10. Also, the control unit 41 disables the MB parameter from the variable length decoding unit 12 and switches the encoding unit 38 to perform encoding using all the encoding parameters of the MB layer calculated by the MB parameter calculation unit 36. The unit 37 is controlled.

  On the other hand, in the image processing system 1 described above, based on the parameter determination information from the variable length decoding unit 12, a flag indicating that the encoding parameter and the MB parameter of the picture layer or higher include an error and is invalid is detected. In this case, encoding is performed using the MB parameter calculated by the MB parameter calculation unit 36 of the encoder 30. Hereinafter, a process when the encoding parameter is invalid will be described with reference to a timing chart shown in FIG.

  According to the timing chart shown in FIG. 5, as shown in FIG. 5A, the pictures I2, B0, B1, P5, B3, B4, P8, B6, B7, I2, B0, A bit stream in which pictures are arranged in the order of B 1 is input to the buffer 11 of the decoder 10.

  In response to this, in the variable length decoding unit 12 of the decoder 10, as shown in FIG. 5C, the pictures I2, B0, B1, P5 over the picture times t1 to t12 at the same timing as the input timing of the bitstream. MB parameters are decoded in the order of B3, B4, P8, B6, B7, I2, B0, and B1.

  Here, as shown in FIG. 5D, when a syntax error of the MB layer for the picture P5 occurs at the picture time t4, the variable length decoding unit 12 performs parameter determination as shown in FIG. A 1-bit flag for the MB layer of information (mb_parameters_valid) is set to “0”. Then, the variable length decoding unit 12 performs error recovery with the start code of the picture B3 next to the picture P5.

  Further, the variable length decoding unit 12 sets the 1-bit flag for the MB layer of the parameter determination information to “1” without causing an error in the MB layer during the picture times t1 to t3 and t5 to t12.

  Further, as shown in FIG. 5 (f), the decoder 10 buffers the MB parameter generated by decoding by the variable length decoding unit 12 in the memory 13, and controls the input timing of the bitstream by the control from the control unit 41. The MB parameter is output to 30 in the order of pictures I2, B0, B1, P5, B3, B4, P8, B6, B7, I2, B0, and B1 with a delay of 4 picture times. .

  Further, as shown in FIG. 5B, the decoder 10 delays pictures B0, B1, I2, B3, B4 over picture times t2 to t13 with a delay of one picture time with respect to the input timing of the bit stream. Image data is output to the encoder 30 in the order of P5, B6, B7, P8, B0, B1, and I2.

  In contrast to the processing on the decoder 10 side shown in FIGS. 5A to 5F, in the encoder 30, as shown in FIG. 5G, the image data is output from the decoder 10 at the output timing shown in FIG. On the other hand, image data is input in the order of pictures B0, B1, I2, B3, B4, P5, B6, B7, and P8 over a picture time t2 to t10 without delay.

  Further, as shown in FIG. 5 (h), the encoder 30 transmits pictures I2, B0, B1, P5 over the picture time t5 to t16 without delay with respect to the output timing of the MB parameter shown in FIG. 5 (f). , B3, B4, P8, B6, B7, I2, B0, B1 are input MB parameters.

  Then, as shown in FIG. 5 (i), at the same timing as the MB parameter input timing shown in FIG. It is determined whether the MB parameter from the long decoding unit 12 is valid. Here, in FIG. 5 (i), when the control unit 41 determines that the MB parameter is valid, it is expressed as “1”, and when the control unit 41 determines that the MB parameter is invalid, it is expressed as “0”. .

  As illustrated in FIG. 5D, in response to the parameter determination information indicating that the MB parameter for the picture P5 includes an error at the picture time t4, the control unit 41 generates an image in which the picture P5 is generated by the error concealment. Recognize that it is data. Then, the control unit 41 also invalidates the MB parameters for the pictures B3, B4, P8, B6, and B7 generated by predictive coding using the picture P5. Then, the control unit 41 determines that the MB parameter is valid again from the picture I2 generated without using the picture P5. That is, the control unit 41 determines that the MB parameters for the pictures I2, B0, and B1 input at the picture times t5 to t7 and the MB parameters for the pictures I2, B0, and B1 input at the picture times t14 to t16 are valid. The MB parameters for the pictures P5, B3, B4, P8, B6, and B7 input during the picture times t8 to t13 are determined to be invalid.

  As shown in FIG. 5 (i), for the pictures I2, B0, B1 for which the MB parameter is determined to be valid, the control unit 41 controls the switch unit 37 to be connected to the terminal b, and variable length decoding is performed. Using the MB parameter input from the unit 12 to the switch unit 37, the encoding unit 38 performs processing such as prediction of a motion vector and performs control.

  On the other hand, as shown in FIG. 5I, for the pictures P5, B3, B4, P8, B6, and B7 for which the MB parameter is determined to be invalid, the control unit 41 connects the switch unit 37 to the terminal a. The encoding unit 38 performs processing such as motion vector prediction using the MB parameter calculated by the MB parameter calculation unit 36 without using the MB parameter generated by the variable length decoding unit 12 and input to the encoder 30. Control to go and encode.

  Then, the encoder 30 converts the bit stream generated by encoding or the like in the encoding unit 38 into three pictures with respect to the input timing of the image data input shown in FIG. 5G, as shown in FIG. Delayed by the time, the pictures are output in the order of pictures I2, B0, B1, P5, B3, B4, P8, B6, B7, I2, B0, and B1 over picture times t5 to t16.

  Here, on the encoder 30 side, it is necessary to buffer the coding parameters of the picture layer in the memory 42 for the picture time required for encoding. In this example, when outputting a bit stream composed of M = 3 GOPs, the control unit 41 outputs the bit stream delayed by three picture times (t2 to t4) with respect to the input timing of the image data. To control.

  In the image processing system 1 configured as described above, the variable length decoding unit 12 of the decoder 10 generates parameter determination information indicating that an error has occurred in the encoding parameter and MB parameter of the picture layer, and the encoder 30 Can be controlled by the control unit 41. Therefore, according to this image processing system 1, for example, when the encoding parameter is invalid, re-encoding is performed using the encoding parameter calculated and generated by the encoder 30 without using the encoding parameter. Re-encoding can be performed accurately.

  Specifically, according to the image processing system 1, as shown in FIG. 5 (d), when an MB layer error occurs, the MB parameter generated by the variable length decoding unit 12 of the decoder 10 is not used. By performing re-encoding by the encoding unit 38 using the MB parameter calculated by the MB parameter calculating unit 36, it is possible to prevent re-encoding using an erroneous encoding parameter.

  The encoder 30 provided in the image processing system 1 that performs the operation described with reference to FIGS. 3 to 5 described above is for image data composed of GOPs in which the interval between I pictures or P pictures is 3 (M = 3) or less. When image data composed of GOPs with M being 3 or more (M> 3) is input, the control unit 41 performs processing for changing to image data with M being 3 or less (M ≦ 3). And re-encode.

  For example, for image data in which pictures I0, B1, B2, B3, B4, and P5 are displayed in order on the encoder 30, the control unit 41 performs a process of changing the picture B3 to the picture P3 to perform the pictures I0, B1, and B2. , P3, B4, and P5 are sequentially displayed as image data. Thus, the control unit 41 sets the GOP image data of M = 5 as the GOP image data of M = 3 and M = 2.

  As described above, when changing the M, the control unit 41 refers to the picture_coding_type of the picture parameter input to the encoder 30 according to the display order of the pictures, and counts the number of B pictures that are continuously input, Control is performed so that the encoding parameter is generated and re-encoding is performed so that the B picture when the count number becomes 3 is changed to the P picture.

  The image processing system 1 that performs such processing will be described with reference to the timing chart of FIG.

  According to the timing chart shown in FIG. 6, first, as shown in FIG. 6 (a), pictures I2, B0, B1, P5, B3, B4, Pa, B6, B7, B8 over picture times t1 to t14. , B9, Pd, Bb, Bc, a bit stream in which pictures are arranged in this order is input to the buffer 11 of the decoder 10.

  Here, M for the bitstream input to the decoder 10 is obtained by detecting the number of pictures from an I picture or P picture at a certain picture time to an I picture or P picture appearing at a later picture time, As shown in FIG. 6B, the picture I2, B0, B1 becomes 3, the picture P5, B3, B4 becomes “3”, the picture Pa, B6, B7, B8, B9 becomes “5”, and the picture Pd , Bb and Bc become “3”.

  At the same timing as the input timing of the bit stream shown in FIG. 6A, the variable length decoding unit 12 of the decoder 10 has pictures I2, B0, B1 over picture times t1 to t14 as shown in FIG. 6D. , P5, B3, B4, Pa, B6, B7, B8, B9, Pd, Bb, and Bc, the MB parameters for each picture are decoded.

  Then, as shown in FIG. 6E, the decoder 10 buffers the MB parameter generated by the variable length decoding unit 12 in the memory 13 and controls the bit stream input timing to 4 with the control from the control unit 41. The MB parameter for each picture is given to the encoder 30 in the order of pictures I2, B0, B1, P5, B3, B4, P8, B6, B7, I2, B0, B1 over a picture time t5 to t16. Output.

  Further, as shown in FIG. 6 (c), the decoder 10 delays by one picture time with respect to the input timing of the bit stream and delays the pictures B0, B1, I2, B3, B4, P5 over the picture times t2 to t15. , B6, B7, B8, B9, Pa, Bb, Bc, and Pd are output to the encoder 30 in this order.

  On the other hand, as shown in FIG. 6F, the encoder 30 sends picture B0 over the picture time t2 to t15 at the same timing as the output timing of the image data from the decoder 10 shown in FIG. , B1, I2, B3, B4, P5, B6, B7, B8, B9, Pa, Bb, Bc, Pd in this order.

  Here, as shown in FIG. 6G, the control unit 41 counts the number of B pictures that are continuously input to the encoder 30 based on the input timing of the image data shown in FIG. To get the count number. The control unit 41 resets the count number to “0” when an I picture or P picture is input to the encoder 30 or when the count number becomes “3”.

  That is, the control unit 41 applies 1, 2, 0, 1, 2, 0, 1, 2, 3, 1, 0, 1, 2, to the input image data as shown in FIG. 0 and the count corresponding to the picture are obtained.

  Then, the control unit 41 determines whether or not to change M of the image data based on the count number shown in FIG. That is, the control unit 41 determines that M of the image data is to be changed when the counter number reaches “3”, and the counter number is “3” at the picture time t10 as shown in FIG. The picture B8 in "" is changed to a P picture, and M is changed.

  In this way, when the B picture is changed to the P picture, the control unit 41, as shown in FIG. 6 (i), based on the counter number shown in FIG. 6 (g), the picture type (picture_coding_type) for each picture. ). In this example, the control unit 41 performs a process of changing the picture type in FIG. 6F from the B picture to the P picture by changing the picture B8 at the picture time t10 to the P picture.

  Here, the control unit 41 controls the encoding unit 38 to perform re-encoding according to the picture type determined based on the count number, regardless of the MB parameter input from the decoder 10 to the encoder 30.

  Further, as shown in FIG. 6 (j), the encoder 30 transmits pictures I2, B0, B1, P5 over the picture time t5 to t18 without delay with respect to the output timing of the MB parameter shown in FIG. 6 (e). , B3, B4, Pa, B6, B7, B8, B9, MB parameters for Pd, Bb, Bc are sequentially input.

  Then, as shown in FIG. 6 (k), the control unit 41 is variable based on the parameter determination information from the variable length decoding unit 12 at the same timing as the MB parameter input timing shown in FIG. 6 (j). It is determined whether the MB parameter from the long decoding unit 12 is valid. Here, in FIG. 6 (k), when the MB parameter is determined to be valid by the control unit 41, it is expressed as “1”, and when the MB parameter is determined as invalid by the control unit 41, it is expressed as “0”. .

  Here, when the picture at the picture time t10 is changed from the B picture to the P picture as shown in FIG. 6 (i), the control unit 41 is a B picture that is temporally continuous with the B picture at the time of the change. Therefore, it is determined that the MB parameter (for example, motion vector) for each picture up to the later P picture in time is invalid.

  That is, the control unit 41 determines that the MB parameters from the picture B8 when the counter number is 3 and the pictures B6, B7, and B9 temporally continuous to the picture B8 to the picture Pa that is temporally later are invalid. Then, the MB parameters for other pictures are determined to be valid.

  Then, the control unit 41 re-encodes the picture corresponding to the MB parameter determined to be invalid by the encoding unit 38 using the MB parameter calculated by the MB parameter calculation unit 36, and is determined to be valid. The picture corresponding to the MB parameter is re-encoded by the encoding unit 38 using the MB parameter from the decoder 10.

  That is, the control unit 41 controls the switch unit 37 to be connected to the terminal a for the pictures B6 to Pa for which the MB parameter input from the decoder 10 is invalid, and is generated by the variable length decoding unit 12 and encoded by the encoder 30. The switch unit 37 is controlled so that the encoding is performed by the encoding unit 38 using the MB parameter (picture_coding_type) calculated by the MB parameter calculation unit 36 without using the MB parameter input to To do.

  On the other hand, for the picture for which the MB parameter input from the decoder 10 is valid, the control unit 41 uses the MB parameter input from the variable length decoding unit 12 to the switch unit 37 to predict the motion vector in the encoding unit 38. Control is performed to perform encoding.

  Then, the encoder 30 converts the bit stream generated by encoding or the like in the encoding unit 38 into three pictures with respect to the input timing of the image data input shown in FIG. 6 (f), as shown in FIG. 6 (l). The pictures are output in the order of pictures I2, B0, B1, P5, B3, B4, P8, B6, B7, Pa, B9, Pd, Bb, and Bc over time t5 to t18.

  Here, on the encoder 30 side, it is necessary to buffer the coding parameters of the picture layer in the memory 42 for the picture time required for encoding. In this example, when outputting a bit stream composed of M = 3 GOPs, the control unit 41 outputs the bit stream delayed by three picture times (t2 to t4) with respect to the input timing of the image data. To control.

  According to the image processing system 1 that performs such processing, when GOP image data with M equal to or greater than 3 is input from the decoder 10 in the encoder 30 corresponding to GOP image data with M equal to or less than 3, a B picture is displayed. Even if the picture is changed to a P picture and changed to GOP image data with M of 3 or less, the encoding parameter calculated by the MB parameter calculation unit 36 is used in place of the encoding parameter from the decoder 10 to accurately re-encode. It can be performed.

  That is, according to this image processing system 1, even if the encoding parameter input from the decoder 10 is outside the range that can be encoded by the encoder 30, the encoding parameter calculated by the MB parameter calculation unit 36 is used. Thus, re-encoding can be performed by changing the encoding parameter.

  Also, the encoder 30 sets the average quantization scale for each picture re-encoded by the encoding unit 38 even when re-encoding is performed using the quantization scale among the MB parameters input from the decoder 10. Perform the calculation process. At this time, the encoder 30 detects a flag indicating that the quantization scale is valid among the MB parameters, and calculates the quantization scale for each picture using only the effective quantization scale without an error flag. Process. The average quantization scale calculated in this way is used when learning the encoding difficulty level of encoding performed by the variable length encoding unit 39 in accordance with rate control. As a result, when the encoder 30 changes from the processing mode in which encoding is performed with the quantization scale input from the decoder 10 to the processing mode in which the MB parameter calculation unit 36 calculates the quantization scale and performs encoding, as described above. The learning using the average quantization scale calculated in (5) can be used for rate control when re-encoding is performed by the variable-length encoding unit 39.

  In the description of the image processing system 1 to which the present invention is applied, an example has been described in which the parameter determination information generated by the variable length decoding unit 12 is generated for the layer higher than the picture layer and the MB layer. Instead, for example, parameter determination information may be generated for other layers (layers) such as a GOP layer and a slice layer conforming to the MPEG standard. As a result, when the image processing system 1 performs re-encoding with the encoder 30, the image processing system 1 refers to the parameter determination information corresponding to each of the above layers and performs re-encoding using the encoding parameters from the decoder 10, or each layer The control unit 41 determines whether the encoding parameter is generated by the MB parameter calculation unit 36 and re-encoding is performed, and accurate re-encoding can be performed.

  Furthermore, in the image processing system 1 to which the present invention is applied, an example in which the parameter determination information is generated by the variable length decoding unit 12 and output to the control unit 41 has been described. However, the present invention is not limited thereto, and blanking in image data is performed. It may be written in a signal portion that is not valid image data, such as a section or a LSB (Least Significant Bit) of a chroma signal, and input from the decoder 10 to the encoder 30. At this time, it is desirable that the variable length decoding unit 12 writes not only the parameter determination information but also an encoding parameter and an MB parameter of the picture layer or more in an area other than an effective part in the image data.

  Next, another image processing system 100 to which the present invention is applied will be described. This image processing system does not decode the input bit stream into image data, but decodes the encoded information in the middle, and performs re-encoding using the encoded information.

  As shown in FIG. 7, the image processing system 100 performs variable-length decoding on the buffer 102 to which the bit stream S51 is input via the input terminal 101 and the bit stream S51 from the buffer 102 to obtain the quantized DCT coefficient. A variable-length decoding unit 103 configured as image data, and a switch unit that outputs the image data S52 from the variable-length decoding unit 103 to the inverse quantization unit 105 and outputs information S53 other than the image data S52 to the delay processing unit 109 104, the inverse quantization unit 105 which performs inverse quantization processing on the image data S52 from the switch unit 104 to obtain image data composed of DCT coefficients, and performs quantization processing on the image data from the inverse quantization unit 105 A quantizing unit 106 that generates image data S54 including quantized DCT coefficients, and a variable-length decoding unit 103 A quantization control unit 107 that controls the quantization processing of the quantization unit 106 based on the quantization parameter, a parameter generation unit 108 that generates an encoding parameter based on the parameter determination information from the variable length decoding unit 103, The delay processing unit 109 receives information S53 other than the quantized DCT coefficient from the switch unit 104, the image data S54 from the quantization unit 106, the information S53 subjected to delay processing by the delay processing unit 109, and the parameter generation unit 108 The switch unit 110 that is controlled to output any of the encoding parameters generated in the above, and the variable length encoding unit 111 that performs variable length encoding processing on the image data from the switch unit 110 and generates the bitstream S55. And once storing the bit stream from the buffer 11 via the output terminal 113 And a buffer 112 which outputs the section.

  In the image processing system 100 configured as described above, the variable length decoding unit 103 performs the same processing as the variable length decoding unit 12 described above, and converts the bit stream from the buffer 102 to a variable length, for example, in units of macroblocks (MB). By performing decoding processing, image data composed of quantized DCT transform coefficients is obtained and output to the switch unit 104.

  Further, the variable length decoding unit 103 performs variable length decoding processing as described above, detects MB parameters (MB_parameters) added for each MB layer, and encoding parameters (Sequence_GOP_Picure_parameters) higher than the picture layer to switch Output to the unit 104. Furthermore, the variable length decoding unit 102 generates parameter determination information (picture_mb_parameters_valid) indicating whether or not the encoding parameter and MB parameter of the picture layer are valid, and outputs them to the parameter generation unit 108 and the switch unit 104.

  The switch unit 104 is controlled to operate so as to be connected to the terminal A or the terminal B when a control signal is input from a control unit (not shown). The switch unit 104 is connected to the terminal A to output the image data from the variable length decoding unit 103 to the inverse quantization unit 105, and is connected to the terminal B to be encoded parameters from the variable length decoding unit 103. Is output to the delay processing unit 109.

  The inverse quantization unit 105 performs inverse quantization on the image data including the quantized DCT transform coefficient from the switch unit 104, for example, for each 8 × 8 pixel block. At this time, the inverse quantization unit 14 performs an inverse quantization process by multiplying the image data by a quantization step to obtain a DCT transform coefficient, and outputs the result to the quantization unit 106.

  The quantization unit 106 quantizes the image data composed of the DCT transform coefficients from the inverse quantization unit 105 and outputs the image data S54 composed of the quantized DCT transform coefficients to the switch unit 110.

  The delay processing unit 109 delays the input timing to the switch unit 110 by the time required for processing in the inverse quantization unit 105 and the quantization unit 106, and outputs the encoding parameter S53 to the switch unit 110.

  The quantization control unit 107 calculates the bit occupation amount of the buffer 112 based on the quantization parameter from the variable length decoding unit 103, and outputs a bit stream S55 having a predetermined bit rate or less from the buffer 112. A quantization control signal that specifies a quantization scale in the quantization process performed by the quantization unit 106 is generated.

  It is desirable that the quantization control unit 107 controls the quantization scale so that a mismatch error of the bit stream S55 due to motion compensation of a reproduced image that occurs in principle when encoding and decoding is not conspicuous.

  The parameter generation unit 108 generates an encoding parameter based on the parameter determination information from the variable length decoding unit 103. That is, the parameter generation unit 108 generates an encoding parameter in a layer where an error has occurred based on the parameter determination information, and outputs it to the switch unit 110.

  The switch unit 110 is connected to the terminal A, the terminal B, or the terminal C when the parameter determination information from the variable length decoding unit 103 and the control signal from the control unit (not shown) are input, and operates with the output timing controlled. The switch unit 110 is configured to output a coding parameter from the delay processing unit 109 to the variable length coding unit 111 when the coding parameter generated by the variable length decoding unit 103 is determined to be valid based on the parameter determination information. Connected to the terminal C so that the encoding parameter from the parameter generation unit 108 is output to the variable length encoding unit 111 when the encoding parameter generated by the variable length decoding unit 103 is determined to be invalid according to the parameter determination information. Is done.

  The variable length coding unit 111 performs variable length coding on the image data from the switch unit 110 to generate a bit stream S55 and outputs the bit stream to the buffer 112, and outputs the bit stream via the output terminal 113 at a target bit rate. To do.

  An operation when a syntax error does not occur in the bit stream input from the input terminal 101 in the image processing system 100 configured as described above will be described.

  At this time, the flag of the parameter determination information generated by the variable length decoding unit 103 is “11” as is apparent with reference to Table 1. That is, no error has occurred in the picture layer and MB parameter of the bit stream that has been variable-length decoded by the variable-length decoding unit 103 and is effective. Therefore, in this case, the bit stream is re-encoded so as to convert the bit rate by performing the variable length encoding by the variable length encoding unit 111 using the encoding parameter generated by the variable length decoding unit 103. S55 is generated.

  Next, an operation of the image processing system 100 when a syntax error occurs in the bit stream input from the input terminal 101 will be described.

  When a syntax error occurs in the input bitstream, the variable length decoding unit 103 sets the parameter determination information flag to “00” or “10”. At this time, when an error occurs at the position of the I picture or P picture, the parameter generation unit 108 outputs the picture header of the P picture and outputs the MB parameter indicating that it is a skip macroblock to the switch unit 110.

  The parameter generator 108 outputs a picture header of the B picture when an error occurs at the position of the B picture, and sends an MB parameter indicating that the motion vector is “0” and has no DCT coefficient to the switch unit 110. Output. Based on the parameter determination information, the switch unit 110 outputs the coding parameter input to the terminal A together with the coding parameter input to the terminal C to the variable length coding unit 111, and converts the bit rate. The bit stream S55 is generated by re-encoding as described above.

  Next, the operation of the image processing system 100 when an MB layer syntax error occurs in the bitstream input from the input terminal 101 will be described.

  In this image processing system 100, when an MB layer error occurs in an I picture according to parameter determination information, the parameter generation unit 108 sets an encoding parameter for a macroblock adjacent to the errored macroblock to an error. Is output to the switch unit 110 as an encoding parameter of the macroblock in which the error occurs.

  When an error occurs in the P picture, the parameter generator 108 outputs an MB parameter indicating that it is a skip macroblock to the switch unit 110.

  Further, when an error occurs in the B picture, the parameter generation unit 108 outputs an MB parameter indicating that the motion vector is “0” and has no DCT coefficient to the switch unit 110. Based on the parameter determination information, the switch unit 110 outputs the coding parameter input to the terminal A together with the coding parameter input to the terminal C to the variable length coding unit 111, and converts the bit rate. The bit stream S55 is generated by re-encoding as described above.

  In the image processing system 100 configured as described above, even when a syntax error occurs in the input bit stream, the parameter generation unit 108 uses the parameter determination information generated by the variable length decoding unit 103 to generate a picture. Since the layer and the MB parameter are generated, it can be re-encoded so as to generate a bit stream in which the syntax error is corrected and output from the output terminal 113.

It is a block diagram shown about the structure of the image processing system to which this invention is applied. (A) is a figure which shows the image data of the size x in the horizontal direction x the size y in the vertical direction, (b) is a figure for demonstrating the process which adds dummy data to image data, (c). It is a figure for demonstrating the process which removes dummy data from image data. 5 is a timing chart for explaining processing timing when picture parameters, parameter determination information, and the like are input from the decoder to the encoder in the image processing system to which the present invention is applied. 5 is a timing chart for explaining processing timing when MB parameters and the like are input from the decoder to the encoder in the image processing system to which the present invention is applied. 6 is a timing chart for explaining processing timing when an encoding parameter input from a decoder to an encoder is invalid in an image processing system to which the present invention is applied. Referring to the picture_coding_type MB parameter input to the encoder according to the display order of pictures, the processing timing when encoding is performed by changing the number of B pictures to P pictures according to the count number It is a timing chart. It is a figure for demonstrating the other image processing system to which this invention is applied.

Explanation of symbols

  1,100 image processing system, 10 decoder, 30 encoder

Claims (17)

Decoding means for decoding the input bitstream to generate image data;
Encoding parameter generation means for generating an encoding parameter for each layer used when re-encoding the image data decoded by the decoding means;
Error flag generation means for generating an error flag indicating whether or not the encoding parameter for each layer generated by the encoding parameter generation means can be used effectively when re-encoding image data; A decoding device provided.   2. The decoding apparatus according to claim 1, wherein the error flag generating means generates an error flag for encoding parameters of a picture layer and a macroblock layer conforming to the MPEG2 standard. The input bitstream is decoded to generate image data, and the encoding parameter for each layer used when re-encoding the decoded image data is generated,
A decoding method for generating an error flag indicating whether or not the above encoding parameters for each layer can be used effectively when re-encoding image data.   4. The decoding method according to claim 3, wherein an error flag is generated for coding parameters of a picture layer and a macroblock layer in conformity with the MPEG2 standard. Based on an error flag indicating whether or not the encoding parameter for each layer input from the decoding device can be used effectively, the above encoding is performed when re-encoding the image data input from the decoding device. Determining means for determining whether the parameter can be used effectively;
An encoding parameter calculation unit that calculates an encoding parameter using image data from the decoding device based on a determination result from the determination unit that the encoding parameter from the decoding device is not valid;
When the determination unit determines that the encoding parameter from the decoding device is valid, the image data is encoded using the encoding parameter input from the decoding device, and the encoding parameter from the decoding device is not effective. An encoding device comprising: encoding means for encoding image data using the encoding parameter generated by the encoding parameter calculating means when determined by the determining means.   6. The encoding apparatus according to claim 5, wherein the error flag is a flag for encoding parameters of a picture layer and a macroblock layer conforming to the MPEG2 standard.   Picture type determination means for determining a picture type of image data indicating a picture to which an error flag from a decoding device is added, wherein the determination means selects a picture to which an error flag from the picture type determination means is added; 6. The coding parameter for image data indicating a picture generated using a picture to which the error flag is added is determined to be invalid based on a picture type of the indicated image data. Encoding device. Based on an error flag indicating whether or not the encoding parameter for each layer input from the decoding device can be used effectively, the above encoding is performed when re-encoding the image data input from the decoding device. A determination process for determining whether the parameter can be used effectively;
A coding parameter calculation process for calculating a coding parameter using image data from the decoding device based on a determination result that the coding parameter from the decoding device is not valid;
When it is determined that the encoding parameter from the decoding device is valid, the image data is encoded using the encoding parameter input from the decoding device, and it is determined that the encoding parameter from the decoding device is not effective. An encoding method comprising: an encoding process for encoding image data using the encoding parameter generated by the encoding parameter calculation process.   9. The encoding method according to claim 8, wherein the error flag is a flag for encoding parameters of a picture layer and a macroblock layer compliant with the MPEG2 standard.   A picture type determination process for determining a picture type of image data indicating a picture to which an error flag from the decoding device is added, and the determination process includes the error flag determined in the picture type determination process The coding parameter for the image data indicating the picture generated using the picture to which the error flag is added is determined to be invalid based on the picture type of the image data indicating the current picture. Item 9. The encoding method according to Item 8. Decoding means for decoding input bitstream to generate image data, and encoding parameter generation means for generating encoding parameters for each layer used when re-encoding image data decoded by the decoding means And an error flag generation means for generating an error flag indicating whether or not the encoding parameter for each layer generated by the encoding parameter generation means can be used effectively when re-encoding the image data A decoding device comprising:
A determination means for determining whether or not the encoding parameter can be used effectively when re-encoding based on an error flag for each layer input from the decoding device; An encoding parameter calculation unit that calculates an encoding parameter using image data from the decoding device based on a determination result from the determination unit that the encoding parameter is not valid, and an encoding parameter from the decoding device Is determined to be effective by the determination means, the image data is encoded using the encoding parameter input from the decoding device, and the determination means determines that the encoding parameter from the decoding device is not effective. In such a case, the encoding method for encoding the image data using the encoding parameter generated by the encoding parameter calculation means. Image processing system comprising a coding device comprising a and.   12. The image processing system according to claim 11, wherein the error flag generation means generates an error flag for encoding parameters of a picture layer and a macroblock layer that conform to the MPEG2 standard.   The encoding device includes a picture type determination unit that determines a picture type of image data indicating a picture to which an error flag from the decoding device is added, and the determination unit includes an error flag from the picture type determination unit And determining that the encoding parameter for the image data indicating the picture generated using the picture to which the error flag is added is not valid based on the picture type of the image data indicating the picture to which the error flag is added. 11. The image processing system according to 11. Decoding means for decoding the input bitstream to generate image data;
Inverse quantization means for inversely quantizing the image data from the decoding means to obtain DCT transform coefficients;
Quantization means for quantizing the DCT transform coefficient from the inverse quantization means to form image data;
Encoding parameter generation means for generating an encoding parameter for each layer used when re-encoding the image data decoded by the decoding means;
Error flag generating means for generating an error flag indicating whether or not the encoding parameter for each layer generated by the encoding parameter generating means can be used effectively when re-encoding image data;
Coding parameter calculation means for calculating a coding parameter based on the error flag from the error flag generation means;
The encoding means for encoding the image data from the quantizing means using the encoding parameters from the encoding parameter generating means or the encoding parameter calculating means;
When the encoding parameter is valid based on the error flag generated by the error flag generating means, the image data is encoded using the encoding parameter generated by the encoding parameter generating means, and the error flag is set. Control means for encoding image data using the encoding parameter generated by the encoding parameter calculation means when the encoding parameter is not valid based on the error flag generated by the generation means. Image processing system.   A video buffer for outputting the bit stream encoded by the encoding means at a predetermined bit rate or less, and a bit rate of the bit stream output by the video buffer based on the bit occupancy of the video buffer. The image processing system according to claim 14, further comprising: a quantization control unit that outputs a quantization control signal for controlling a quantization step when quantization is performed by the quantization unit so as to have a bit stream or less. Decode the input bitstream to obtain image data, encoding parameters for each layer used when re-encoding the image data, and encoding parameters for each layer when re-encoding the image data A decoding process for generating an error flag indicating whether or not it can be used effectively;
An inverse quantization process for inversely quantizing the image data to obtain a DCT transform coefficient; a quantization process for quantizing the DCT transform coefficient to obtain image data;
Encoding parameter calculation processing for calculating the encoding parameter based on the error flag, and encoding of image data using the encoding parameter generated by the decoding processing when the encoding parameter is valid based on the error flag. And an encoding process for encoding image data using the encoding parameter generated by the encoding parameter calculation process when the encoding parameter is not valid based on the error flag.   Based on the bit occupancy of the video buffer that outputs the encoded bit stream at a predetermined bit rate or lower, the quantization is performed so that the bit rate of the bit stream output from the video buffer is lower than the predetermined bit stream. The image processing method according to claim 16, wherein a quantization step when performing quantization in the processing is controlled.

Images