JP2008004984A - Image processor and method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for more efficiently coding image data. <P>SOLUTION: An MPEG 2 decoding section 115 decodes image data coded by the MPEG 2 system, outputs a baseband image to a video memory 117, outputs MPEG 2 decode information to a buffer 118, and outputs a quantization scale of each macroblock and generated bits to an MBComplexity calculation section 116. The MBComplexity calculation section 116 calculates Complexity, outputs the MPEG 2 decode information from the buffer 118 and outputs the Complexity and the generated bits R to an MB adaptive coding discrimination section 119. The MB adaptive coding discrimination section 119 determines whether field coding or frame coding is to be applied to each macroblock of the baseband image data on the basis of the information supplied from the buffer 118, and an AVC coding section 120 codes the image data by the H.264/AVC system. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, a program, and a recording medium, and more particularly, to an image processing apparatus and method, a program, and a recording medium that enable image data to be encoded more efficiently.

  In recent years, MPEG (compressed by orthogonal transform such as discrete cosine transform and motion compensation is used for the purpose of efficiently transmitting and storing information, and using redundancy unique to image information. A device conforming to a scheme such as Moving Picture Coding Experts Group) is becoming popular in both information distribution in broadcasting stations and information reception in general households.

  In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system, and is a standard that covers both interlaced scanning images and progressive scanning images, standard resolution images, and high-definition images. Widely used in a wide range of applications for (business) and consumer use. By using the MPEG2 encoding method, for example, a standard resolution interlaced scanning image having 720 × 480 pixels is 4 to 8 Mbps, and a high resolution interlaced scanning image having 1920 × 1088 pixels is 18 to 22 Mbps. By assigning an amount (bit rate), it is possible to achieve a high compression rate and good image quality.

  Since MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, it did not support encoding methods having a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. In the future, with the widespread use of portable terminals and the like, the need for a lower code amount encoding method is expected to increase, and the MPEG4 encoding method has been standardized accordingly. Regarding the MPEG4 image encoding system, the standard was defined as an international standard in December 1998 as ISO / IEC 14496-2.

  Furthermore, in recent years, a standard called H.26L (ITU-T Q6 / 16 VCEG), which was originally formulated for the purpose of image coding for video conferencing, has attracted attention. H. 26L is known to achieve higher encoding efficiency than the conventional encoding schemes such as MPEG2 and MPEG4, although a larger amount of computation is required for encoding or decoding. In addition, as part of MPEG4 activities, this H.264 Based on H.26L Standardization of an encoding method that incorporates a function that is not supported by 26L and realizes higher encoding efficiency is performed as Joint Model of Enhanced-Compression Video Coding. In March 2003, an international standard called H.264 / AVC (Advanced Video Coding) was established.

  In addition, with the widespread use of these encoding methods, a technique called transcoding that converts data encoded by a certain encoding method into data encoded by another encoding method becomes important.

As a technique related to transcoding, for example, a technique as disclosed in Patent Document 1 has been proposed.
JP 2001-145113 A

  By the way, in the H.264 / AVC encoding system, a field or frame adaptive encoding function at a macroblock level, which was not found in the conventional MPEG system or the like, is added. In the H.264 / AVC encoding scheme, each macroblock can be frame-encoded or field-encoded in units of macroblock pairs formed by two upper and lower macroblocks.

  For example, when an image signal input to be encoded is in an interlace (interlace scanning) format, frame encoding is performed for the entire picture, and field encoding or frame encoding is performed on a macroblock pair. It is also possible. In the H.264 / AVC system, normally, whether to perform field coding or frame coding for each macroblock is determined, for example, when the macroblock is coded, the allocated bit amount is smaller. It is determined to be.

  However, when determining whether to perform individual macroblock field encoding or frame encoding in this manner, the encoding efficiency is improved. On the other hand, in order to determine whether to perform field encoding or frame encoding, For example, it is necessary to perform frame coding and field coding for each macroblock pair, or to perform an equivalent arithmetic process in advance, and processing added to the coding device (encoder) is added. Increases, and the encoding process takes time.

  The present invention has been made in view of such a situation, and enables image data to be encoded more efficiently.

  One aspect of the present invention is an image processing apparatus that converts image data encoded by the Moving Picture Coding Experts Group (MPEG) method into image data encoded by the H.264 / AVC (Advanced Video Coding) method. A decoding means for decoding image data encoded by the MPEG method, and information obtained when the image data encoded by the MPEG method is decoded by the decoding means; MPEG decoding information acquisition means for acquiring MPEG decoding information, which is information set for compressing data in MPEG format, and a base obtained by decoding image data encoded in MPEG format by the decoding means Baseband image acquisition means for acquiring image data of a band, and based on the MPEG decode information acquired by the decode information acquisition means, the baseband image acquisition means Encoding for encoding in H.264 / AVC format for each of the macroblock pairs composed of two macroblocks that are adjacent to each other in the image in the baseband image acquired from above A coding unit for generating image data encoded by the H.264 / AVC method by encoding each of the macroblock pairs with a determination unit for determining a method and the encoding method determined by the determination unit And an image processing device.

  Information obtained when the image data encoded by the MPEG system is decoded by the decoding means, the generated bits of each macroblock of the image data encoded by the MPEG system, and the macroblock A calculation unit that calculates a value representing complexity corresponding to each of the macroblocks based on information on a quantization scale may be further provided.

  A value representing the complexity corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the calculation result of the calculation means. Is determined to be equal to or less than the threshold, the determination means may set the encoding method of the macroblock pair to frame encoding.

  A value representing the complexity corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the calculation result of the calculation means. Is determined to be equal to or greater than the threshold, the determination means may set the encoding method of the macroblock pair to field encoding.

  Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the total value of the generated bits is equal to or less than the threshold value, the determination unit may set the encoding method of the macroblock pair to frame encoding.

  Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the total value of the generated bits is greater than or equal to the threshold value, the determination unit may set the encoding method of the macroblock pair to field encoding.

  Based on the MPEG decode information acquired by the decode information acquisition means, in two macroblocks included in the macroblock pair of the baseband image, the absolute value of the difference value of the motion vector corresponding to each macroblock is When it is determined that the value is equal to or less than the threshold, the determination unit may set the encoding method of the macroblock pair to frame encoding.

  Based on the MPEG decode information acquired by the decode information acquisition means, in two macroblocks included in the macroblock pair of the baseband image, the absolute value of the difference value of the motion vector corresponding to each macroblock is If it is determined that the value is equal to or greater than the threshold value, the determination unit may set the encoding method of the macroblock pair to field encoding.

  Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the DCTType to be performed is the frame DCT, the determination unit may set the encoding method of the macroblock pair to frame encoding.

  Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the DCTType to be performed is field DCT, the determination unit may set the encoding method of the macroblock pair to field encoding.

  Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the motion compensation prediction mode to be performed is frame motion compensation, the determination unit may set the encoding method of the macroblock pair to frame encoding.

  Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the motion compensation prediction mode to be performed is field motion compensation, the determination unit may set the encoding method of the macroblock pair to field encoding.

  One aspect of the present invention is an image processing apparatus for converting image data encoded by the MPEG (Moving Picture Coding Experts Group) method into image data encoded by the H.264 / AVC (Advanced Video Coding) method. An image processing method for decoding image data encoded by the MPEG method, and information obtained when the image data encoded by the MPEG method is decoded, wherein the baseband image data is converted to the MPEG method MPEG decoding information, which is information set for compression in the image, is acquired, baseband image data obtained by decoding the image data encoded by the MPEG method is acquired, and the acquired MPEG decoding Based on the information, for each of the macroblock pairs that are included in the acquired baseband image and are composed of two macroblocks that are adjacent vertically in the image, H.264 / A Image data encoded by the H.264 / AVC method is determined by determining the encoding method in the case of encoding by the VC method, and encoding each macroblock pair by the determined encoding method. Is an image processing method including a step of generating.

  One aspect of the present invention is an image processing apparatus for converting image data encoded by the MPEG (Moving Picture Coding Experts Group) method into image data encoded by the H.264 / AVC (Advanced Video Coding) method. A program for executing image processing, which controls decoding of MPEG-encoded image data, and is information obtained when the MPEG-encoded image data is decoded. Controls the acquisition of MPEG decode information, which is information set to compress image data in the MPEG format, and controls the acquisition of baseband image data obtained by decoding the image data encoded in the MPEG format Based on the acquired MPEG decoding information, a macro composed of two macroblocks adjacent to each other in the image included in the acquired baseband image For each of the lock pairs, it controls the determination of the encoding method when encoding with the H.264 / AVC method, and by encoding each of the macroblock pairs with the determined encoding method, H A computer-readable program including a step of controlling generation of image data encoded in the .264 / AVC format.

  In one aspect of the present invention, image data encoded by the MPEG method is decoded and information obtained when the image data encoded by the MPEG method is decoded, and the baseband image data is converted into MPEG. MPEG decoding information, which is information set for compression by a method, is acquired, baseband image data obtained by decoding image data encoded by the MPEG method is acquired, and the acquired MPEG Based on the decoding information, each of the macroblock pairs included in the acquired baseband image and composed of two macroblocks adjacent to each other in the upper and lower sides in the image is encoded in the H.264 / AVC format. The encoding method is determined, and encoding is performed for each of the macroblock pairs with the determined encoding method, thereby encoding with the H.264 / AVC method. Of image data is generated.

  According to the present invention, image data can be encoded more efficiently.

  Embodiments of the present invention will be described below. Correspondences between constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  An image processing apparatus according to an aspect of the present invention converts image data encoded by an MPEG (Moving Picture Coding Experts Group) method into image data encoded by an H.264 / AVC (Advanced Video Coding) method. An image processing apparatus that decodes image data encoded by the MPEG system (for example, the MPEG decoding unit 115 in FIG. 1) and image data encoded by the MPEG system by the decoding means. MPEG decoding information acquisition means (for example, buffer 118 in FIG. 1) that acquires MPEG decoding information that is information obtained when decoding and is set to compress baseband image data in the MPEG format And baseband image acquisition means (for example, a video memo in FIG. 1) for acquiring baseband image data obtained by decoding the MPEG encoded image data by the decoding means. 117) and 2 adjacent to the top and bottom in the image included in the baseband image acquired by the baseband image acquisition means based on the MPEG decode information acquired by the decode information acquisition means. For each macroblock pair composed of two macroblocks, a determination unit (for example, a frame encoding method or a field encoding method) for determining an encoding method in the case of encoding in the H.264 / AVC format (for example, An image encoded in the H.264 / AVC format by encoding each of the macroblock pairs with the encoding method determined by the MB adaptive encoding determination unit 119) and the determination means in FIG. Coding means for generating data (for example, the AVC coding unit 120 in FIG. 1).

  The image processing apparatus is information obtained when the decoding unit decodes the image data encoded by the MPEG method, and the generated bits of each macroblock of the image data encoded by the MPEG method And a calculation means (for example, MBComplexity calculation unit 116 in FIG. 1) for calculating a value representing complexity corresponding to each of the macroblocks based on information on the quantization scale of the macroblock. be able to.

  An image processing method according to one aspect of the present invention converts image data encoded by the MPEG (Moving Picture Coding Experts Group) method into image data encoded by the H.264 / AVC (Advanced Video Coding) method. An image processing method of an image processing apparatus, wherein image data encoded by the MPEG method is decoded (for example, the process of step S101 in FIG. 6), and the image data encoded by the MPEG method is decoded. Obtained MPEG decoding information, which is information obtained and set for compressing baseband image data in the MPEG format (for example, processing in step S103 in FIG. 6), and encodes in the MPEG format Baseband image data obtained by decoding the obtained image data is acquired (for example, the process of step S104 in FIG. 6), and the acquired base data is obtained based on the acquired MPEG decoding information. An encoding method (for example, a frame) for encoding each of the macroblock pairs formed by two macroblocks adjacent to each other in the image in the band in the band image according to the H.264 / AVC method (For example, the process of step S106 in FIG. 6), and coding each of the macroblock pairs with the determined coding method, thereby determining H.264 / This includes a step of generating image data encoded by the AVC method (for example, the process of step S108 in FIG. 6).

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example according to an embodiment of an image processing apparatus 100 to which the present invention is applied. The image processing apparatus 100, for example, compresses and encodes image data that has been compressed by the MPEG (Moving Picture Coding Experts Group) 2 method using the H.264 / AVC (Advanced Video Coding) method. Configured as a transcoder that converts data.

  In the figure, an MPEG2 decoding unit 115 decodes an input bit stream of image data encoded by the MPEG2 system, outputs baseband image data (or an image signal) to a video memory 117, and MPEG2 Decode information is output to the buffer 118. The MPEG2 decoding information output here includes a motion compensated prediction mode (MC mode) used for MPEG2 motion compensation (MC (Motion Compensation)), a DCTType that is information for distinguishing an MPEG2 frame DCT or a field DCT, and the picture Is information including a PictureType which is information indicating whether a frame picture or a field picture, a quantization scale Quant of each macroblock in MPEG2 image data, and a generated bit R in each macroblock.

  Also, the MPEG2 decoding unit 115 outputs the quantization scale Quant of each macroblock in the MPEG2 image data included in the above-described MPEG2 decoding information and the generated bit R in each macroblock to the MBComplexity calculation unit 116.

  Based on the information supplied from the MPEG2 decoding unit 115, the MBComplexity calculation unit 116 calculates Complexity, which is a value representing the complexity of each macroblock, using Equation (1). The Complexity calculated here is used for determination of the encoding method (whether field encoding or frame encoding) in the MB adaptive encoding determination unit 119, as will be described later.

  Then, the MBComplexity calculation unit 116 outputs the calculated Complexity of each macroblock and the generated bit R in each macroblock to the buffer 118.

  The buffer 118 stores MPEG2 decode information of a preset unit amount, Complexity, and generated bit R, and stores the data stored at a predetermined timing to the MB adaptive encoding determination unit 119 of the AVC encoding unit 120. Output. The buffer 118 stores, for example, MPEG2 decode information corresponding to one frame of image data, Complexity and generated bit R, and one frame of baseband image data (or image signal) from the video memory 117 is AVC encoded. At the timing supplied to the unit 120, the MPEG2 decoding information corresponding to the frame, Complexity, and generated bit R are output to the MB adaptive coding determination unit 119.

  Based on the MPEG2 decoding information supplied from the buffer 118, the Complexity and the generated bit R, the MB adaptive coding determination unit 119 determines each macroblock of the baseband image data for one frame supplied from the video memory 117. Whether to perform field encoding or frame encoding is determined. In the H.264 / AVC encoding system, a field or frame adaptive encoding function at the macroblock level, which was not found in the conventional MPEG system, is added. In the H.264 / AVC encoding scheme, each macroblock can be subjected to frame encoding or field encoding in units of a macroblock pair formed of two macroblocks, and an MB adaptive encoding determination unit 119 can be performed. In this case, a coding method (frame coding or field coding) to be applied for each macroblock pair is determined.

  The AVC encoding unit 120 encodes baseband image data supplied from the video memory 117 according to the H.264 / AVC format. A detailed configuration example of the AVC encoding unit 120 is shown in FIG.

  In FIG. 2, an input image signal is first converted into digital data by an A / D converter 201. Note that when image data is supplied as digital data from the video memory 117, the processing of the A / D conversion unit 201 may be omitted.

  Next, the screen rearrangement buffer 202 rearranges the frames in accordance with the GOP (Group of Pictures) structure of the image compression information to be output.

  The image data supplied via the screen rearrangement buffer 202 includes the pixel value of the image data, the image data stored in the frame memory 211 by the intra prediction unit 212 or the motion prediction compensation unit 213, or the screen rearrangement buffer. Difference information with respect to a pixel value generated based on the image data supplied from 202 is calculated by the adder 203 and input to the orthogonal transform unit 204.

  As described above, in the H.264 / AVC encoding method, each macroblock is frame-encoded or field-encoded in units of macroblock pairs formed by two macroblocks adjacent to each other in the vertical direction. be able to. Therefore, the difference information output from the adder 203 is output in units of macroblocks. For example, information for specifying a macroblock corresponding to each difference information is added and output.

  FIG. 3 is a diagram illustrating an example of a macro block. In the H.264 / AVC encoding system, macroblocks usually composed of 16 × 16 pixels are used, and each indicated by a square frame in the figure is an individual macroblock. For example, the macroblocks are set in order from the upper left of the image, and in this example, the macroblock on the upper left is the number 0 macroblock, and the macroblock adjacent to the lower side of the number 0 macroblock is number 1. It is a macro block. Further, the macroblock adjacent to the right side of the macroblock numbered 0 is the macroblock numbered 2, and the macroblock adjacent to the right side of the macroblock numbered 0 is the macroblock numbered 3.

  As shown in the figure, in the H.264 / AVC encoding method, frame encoding or field encoding is performed for each macroblock pair formed by two macroblocks adjacent vertically in an image. It is made to be able to choose adaptively. In this example, one macroblock pair is composed of two macroblocks of number 0 and number 1, one macroblock pair is composed of two macroblocks of number 2 and number 3, and so on. A macroblock pair is constructed.

  In addition, information indicating whether each macroblock (actually, a macroblock pair) determined by the MB adaptive encoding determination unit 119 is field-encoded or frame-encoded corresponds to each macroblock. Attached to the image data input to the screen rearrangement buffer 202 and supplied.

  When the image data input to the AVC encoding unit 120 is image data that is intra-encoded (intra-image), the image data supplied via the screen rearrangement buffer 202 includes the pixel value of the image data, The difference information with the pixel value generated by the intra prediction unit 212 based on the image data stored in the frame memory 211 is calculated by the adder 203 and input to the orthogonal transform unit 204, and a discrete cosine is obtained with respect to the difference information. Orthogonal transformation processing such as transformation (DCT) and Karhunen-Loeve transformation is performed.

  The transform coefficient output from the orthogonal transform unit 204 is subjected to quantization processing in the quantization unit 205.

  The rate control unit 213 controls the bit rate of the output data by controlling the quantization scale used for the quantization process by the quantization unit 205 by changing it as necessary.

  The quantized transform coefficient output from the quantization unit 205 is input to the lossless transform unit 206, and after lossless encoding processing such as variable length coding and arithmetic coding is performed by the lossless transform unit 206, The image data is accumulated in the accumulation buffer 207 and output as image data encoded by the H.264 / AVC format.

  On the other hand, the quantized transform coefficient output from the quantization unit 205 is also supplied to the inverse quantization unit 208 and subjected to the inverse quantization process, and then the inverse orthogonal transform unit 209 further performs the inverse orthogonal transform process. Is applied to obtain decoded image data.

  The decoded image data output from the inverse orthogonal transform unit 209 is stored in the frame memory 11 after the block distortion is removed by the deblocking filter 210.

  In the intra prediction unit 212, in accordance with the macroblock to be encoded, the forward prediction mode in which only the frame image data in the forward direction (past side) on the time axis is the reference image, and the backward direction in the time axis (future side) The backward prediction mode in which only the frame image data is used as a reference image and the bidirectional prediction mode in which both of the two frame image data are used as reference images can be applied. In the intra prediction unit 212, information on the intra prediction mode applied to the macroblock is transmitted to the lossless encoding unit 206 and a part of header information in the image data encoded by the H.264 / AVC format. Is encoded as

  On the other hand, when the image data input to the AVC encoding unit 120 is image data to be inter-coded, the image data supplied via the screen rearrangement buffer 202 is first subjected to motion prediction / compensation. Input to the unit 213. At this time, the motion prediction / compensation unit 213 extracts the image data of the reference image from the frame memory 211, and generates predicted image data by performing motion prediction / compensation processing on the image data. Note that whether or not each macroblock is frame-encoded or field-encoded in units of the above-described macroblock pairs is determined based on whether the image data input to the AVC encoding unit 120 is an inter (inter-image) code. This is performed when the image data is to be converted.

  In MPEG2, when the frame structure of image data is a frame structure, a macro block is composed of a frame block of 16 pixels × 16 lines (luminance signal) in which a top field and a bottom field are interlaced, and frame motion compensation prediction and field motion compensation prediction are performed. , And dual prime prediction, three motion compensated predictions are used.

  In the frame motion compensation prediction, motion compensation prediction is performed in a frame in which two interlaced fields are combined, and a luminance signal is predicted for each interlaced 16 pixel × 16 line block. In an interlaced signal, of two fields constituting one frame, a spatially upper field is called a top field, and a spatially lower field is called a bottom field.

  FIG. 4A is a diagram illustrating an example in which forward motion compensation prediction is performed from a reference frame one frame away. In the figure, pixels in the top field are indicated by circles, and pixels in the bottom field are indicated by rectangles, and an input frame (screen rearrangement buffer) corresponding to the reference frame according to the motion vector indicated by “MV”. The pixel position of the image data supplied via 202 is specified.

  Frame motion compensated prediction is a relatively slow motion and is an effective prediction method when moving at a constant speed while maintaining a high correlation within the frame.

  On the other hand, the field motion compensation prediction performs motion compensation for each field. As shown in FIG. 4b, the motion vector “MV1” is set in the top field, and the motion vector “MV2” is set in the bottom field. According to each motion vector of “MV1” or “MV2”, the pixel position of the input frame (image data supplied via the screen rearrangement buffer 202) corresponding to the reference frame is specified.

  The field in the reference frame corresponding to the pixel in the input frame may be a top field or a bottom field, and the field in the reference frame is set by a motion vertical field select flag in the macroblock data of the MPEG2 data. In the example of FIG. 4b, the top field in the reference frame is referred to for both the top field pixel and the bottom field pixel in the input frame. In the figure, pixels in the top field are indicated by circles, and pixels in the bottom field are indicated by rectangles. In the field motion compensation prediction, prediction is performed for each field in the macroblock. The prediction is performed in units of field blocks of the line.

  For example, in the forward and backward prediction of a P picture and a B picture, two motion vectors are required for one macroblock. In the bi-directional prediction of a B picture, four motion vectors are required for one macroblock. For this reason, in field motion compensated prediction, it is possible to increase the prediction efficiency for local motion and acceleration motion by predicting each field, but the number of motion vectors is twice that of frame motion compensation. Necessary.

  In MPEG2, two types of DCT encoding modes are used. 5a and 5b are diagrams for explaining the respective DCT coding modes.

  In the frame DCT coding mode, when the luminance signal of a macroblock is decomposed into four blocks, each block is configured to include a top field and a bottom field as shown in FIG. 5a. Disassembled. The macroblock subjected to the above-described frame motion compensation prediction is normally encoded in the frame DCT encoding mode.

  On the other hand, in the case of the field DCT coding mode, when the luminance signal of the macro block is decomposed into four blocks, each block is configured only by the top field or the bottom field as shown in FIG. 5b. Is disassembled. The macroblock subjected to the above-described field motion compensation prediction is normally encoded in the field DCT encoding mode.

  The frame motion compensated prediction and the field motion compensated prediction in the motion compensation described above with reference to FIG. 4 are selected adaptively, and two DCT coding modes, ie, frame coding, as shown in FIG. By adaptively switching between field encoding, the encoding efficiency for interlaced signals is improved.

  As described above, in the H.264 / AVC encoding method, it is possible to adaptively select frame encoding or field encoding for each macroblock pair unit. In the sequence parameter set RBSP (Raw Byte Sequence Payloads) in the H.264 / AVC bitstream, there is a parameter called mb_adaptive_frame_field_flag (macroblock adaptive frame field flag), and in the slice header, field_pic_flag There is a parameter called (field picture flag). The setting of these flags determines the encoding method (frame encoding or field encoding) in units of frames and macroblocks.

  When the image data to be encoded is in an interlace (interlace scanning) format, encoding processing at the picture level or macroblock level (individual picture or macroblock pair is subjected to frame encoding or field encoding) Can be adaptively performed. For example, if Mb_adaptive_frame_field_flag in the sequence parameter set of the H.264 / AVC bitstream is set to “1” and field_pic_flag in the slice header is set to “0”, frame coding is performed for the entire picture, and the macroblock Field encoding or frame encoding can be performed on the pair.

  In the image processing apparatus 100 of the present invention, for a predetermined macroblock pair included in a baseband image supplied from the video memory 117, the MB adaptive coding determination unit 119 should frame-code the macroblock pair. When it is determined as a macroblock pair, the motion prediction / compensation unit 213 performs the above-described frame motion compensation prediction, and further, the orthogonal transform unit 204 to the lossless encoding unit 206 perform encoding in the above-described frame DCT encoding mode. Is done.

  Further, when the MB adaptive coding determination unit 119 determines a macroblock pair to be subjected to field coding for a predetermined macroblock pair included in the baseband image supplied from the video memory 117, The motion prediction / compensation unit 213 performs the above-described field motion compensation prediction, and the orthogonal transform unit 204 to the lossless encoding unit 206 perform encoding using the above-described field DCT encoding mode.

  Returning to FIG. 2, the predicted image data output from the motion prediction / compensation unit 213 is input to the adder 203, which adds the pixel values of the image data supplied via the screen rearrangement buffer 202. The difference information with the pixel value of the predicted image data is calculated. In FIG. 2, the intra prediction unit 212 and the adder 203 are illustrated as being connected, but the image data input to the AVC encoding unit 120 is an inter (inter-image) encoded image. In the case of data, the motion prediction / compensation unit 213 and the adder 203 are connected.

  After that, as in the case of intra coding, the data output from the adder 203 is input to the orthogonal transform unit 204, stored in the storage buffer 207 through processing by the quantization unit 205 and the lossless transform unit 206, and H. It is output as image data encoded by the H.264 / AVC format.

  The quantized transform coefficient output from the quantizing unit 205 is also supplied to the inverse quantizing unit 208, and the image data decoded through the processing of the inverse orthogonal transform unit 209 is processed by the deblocking filter 210. It is stored in the frame memory 11 through processing.

  Note that the motion prediction / compensation unit 213 supplies information on the motion vector generated based on the image data supplied via the screen rearrangement buffer 202 to the lossless encoding unit 206, and the lossless encoding unit 206 The information is subjected to lossless encoding processing such as variable length encoding and arithmetic encoding, and is encoded as part of header information in image data encoded by the H.264 / AVC format.

  Next, a description will be given of determination as to whether each macroblock (actually, a macroblock pair) performed by the MB adaptive encoding determination unit 119 is field-encoded or frame-encoded.

  The MB adaptive encoding determination unit 119 performs the following conditions (A) to (E) in two macroblocks MB (m) and MB (n) that are vertically adjacent in the image supplied from the video memory 117. It is determined whether or not the five conditions are met. This determination is performed based on information supplied from the buffer 118, MPEG2 decoding information corresponding to an image supplied from the video memory 117, Complexity, and generated bit R.

  (A) When the complexity of the macroblock MB (m), the complexity of the macroblock MB (n), or the total value of the complexity of the macroblocks MB (m) and MB (n) is less than or equal to a preset threshold value .

  (B) The generated bit of the macroblock MB (m), the generated bit of MB (n), or the total value of the generated bits of the macroblocks MB (m) and MB (n) is less than a preset threshold value. If there is.

  (C) The absolute value of the difference between the motion vector of the macroblock MB (m) and the motion vector of the macroblock MB (n) is less than or equal to a preset threshold value.

  (D) When DCTType of macroblock MB (m), DCTType of macroblock MB (n), or DCTType of both macroblocks MB (m) and MB (n) is a frame DCT.

  (E) The motion compensation prediction mode of the macroblock MB (m), the motion compensation prediction mode of the macroblock MB (n), or the motion compensation prediction modes of both the macroblocks MB (m) and MB (n) If in mode.

  When any of the above conditions (A) to (E) is satisfied, it can be said that the prediction efficiency in units of macroblocks in the MPEG2 encoded state is high, and the motion of the image area corresponding to the macroblocks Is moderate or the correlation within the frame of the image region is considered high. Therefore, the MB adaptive coding determination unit 119 determines that frame coding should be performed for a macroblock pair composed of the macroblocks MB (m) and MB (n).

  Alternatively, the MB adaptive coding determination unit 119 performs the following conditions (F) to (J) in two macroblocks MB (m) and MB (n) adjacent to each other in the upper and lower sides of the image supplied from the video memory 117. It may be determined whether or not the five conditions are met.

  (F) When the Complexity of the macroblock MB (m), the Complexity of the macroblock MB (n), or the total value of the Complexity of MB (m) and MB (n) is greater than or equal to a preset threshold value.

  (G) A threshold value in which the generated bits of the macro block MB (m), the generated bits of the macro block MB (n), or the total value of the generated bits of the macro blocks MB (m) and MB (n) are set in advance. If it is above.

  (H) The absolute value of the difference between the motion vector of the macroblock MB (m) and the motion vector of the macroblock MB (n) is greater than or equal to a preset threshold value.

  (I) DCTType of macroblock MB (m), DCTType of MB (n), or DCTType of both macroblocks MB (m) and MB (n) is a field DCT.

  (J) Field prediction is based on the motion compensation prediction mode of the macroblock MB (m), the motion compensation prediction mode of the macroblock MB (n), or the motion compensation prediction mode of both the macroblocks MB (m) and MB (n). If in mode.

  When any of the above conditions (F) to (J) is satisfied, it can be said that the macroblock in the MPEG2 encoded state has a low prediction efficiency, and the motion of the image region corresponding to the macroblock is small. It is considered intense or the correlation within the frame of the image area is low. Therefore, the MB adaptive coding determination unit 119 determines that field coding should be performed for a macroblock pair configured by the macroblocks MB (m) and MB (n).

  Next, an example of transcoding processing by the image processing apparatus 100 will be described with reference to the flowchart of FIG.

  In step S101, the MPEG2 decoding unit 115 decodes image data encoded by the MPEG2 method for one frame. At this time, as described above, the baseband image data is output to the video memory 117, the MPEG2 decoding information is output to the buffer 118, the quantization scale Quant of each macroblock in the MPEG2 image data, and each macroblock The generated bit R is output to the MBComplexity calculation unit 116.

  In step S102, the MBComplexity calculation unit 116 calculates Complexity, which is a value representing the complexity of the macroblock, using the above-described equation (1).

  In step S <b> 103, the MB adaptive encoding determination unit 119 acquires MPEG2 decode information from the buffer 118. At this time, the Complexity calculated in step S102 and the generated bit R are also acquired.

  In step S104, the AVC encoding unit 120 obtains, for example, baseband image data (or signal) for one frame from the video memory 117.

  In step S105, the MB adaptive coding determination unit 119 extracts one set of macroblock pairs from the image data acquired in the process of step S104.

  In step S106, the MB adaptive coding determination unit 119 performs a coding method determination process for the macroblock pair extracted in the process of step S105. Note that the process of step S106 is executed only when the image data to be encoded in the H.264 / AVC format is image data of an interlace signal. For example, the image data to be encoded in the H.264 / AVC format is progressive. If the image data is a signal, the process of step S106 is skipped.

  Here, the details of the encoding method determination processing in step S106 in FIG. 6 will be described with reference to the flowchart in FIG.

  In step S131, the MB adaptive coding determination unit 119 conforms to the above-described condition (A) for the two macroblocks MB (m) and MB (n) included in the macroblock pair extracted in the process of step S105. It is determined whether or not to do. If it is determined in step S131 that the condition (A) is met, the process proceeds to step S136. If it is determined that the condition (A) is not met, the process proceeds to step S132.

  In step S132, the MB adaptive coding determination unit 119 determines whether or not the macro blocks MB (m) and MB (n) meet the above-described condition (B). If it is determined in step S132 that the condition (B) is met, the process proceeds to step S136. If it is determined that the condition (B) is not met, the process proceeds to step S133.

  In step S133, the MB adaptive coding determination unit 119 determines whether or not the macro blocks MB (m) and MB (n) meet the above-described condition (C). If it is determined in step S133 that the condition (C) is met, the process proceeds to step S136. If it is determined that the condition (C) is not met, the process proceeds to step S134.

  In step S134, the MB adaptive coding determination unit 119 determines whether or not the macro blocks MB (m) and MB (n) meet the above-described condition (D). If it is determined in step S134 that the condition (D) is met, the process proceeds to step S136. If it is determined that the condition (C) is not met, the process proceeds to step S135.

  In step S135, the MB adaptive coding determination unit 119 determines whether or not the macroblocks MB (m) and MB (n) meet the above-described condition (E). If it is determined in step S135 that the condition (E) is met, the process proceeds to step S136. If it is determined that the condition (E) is not met, the process proceeds to step S137. That is, when it is determined that the macro blocks MB (m) and MB (n) do not meet any of the conditions (A) to (E), the process of step S137 is performed.

  In step S136, the MB adaptive encoding determination unit 119 determines that a macroblock pair constituted by the macroblocks MB (m) and MB (n) is to be frame-encoded.

  On the other hand, in step S137, the MB adaptive encoding determination unit 119 determines that a macroblock pair configured by the macroblocks MB (m) and MB (n) is to be field encoded.

  After the process of step S136 or S137, the process proceeds to step S138, and the MB adaptive coding determination unit 119 determines whether the macroblock pair configured by the macroblocks MB (m) and MB (n) and the step S136 or Information that associates the coding method determined in the process of S137 is output.

  Alternatively, the encoding method determination process in step S106 of FIG. 6 may be executed as shown in the flowchart of FIG.

  In the same figure, in steps S151 to S155, the MB adaptive coding determination unit 119 performs each of the two macroblocks MB (m) and MB (n) included in the macroblock pair extracted in the process of step S105. It is determined whether or not the conditions (E) to (J) are met. If it is determined that the conditions (E) to (J) are met, the process proceeds to step S156, and the MB adaptive coding determination unit 119 is performed. Determines that a macroblock pair composed of macroblocks MB (m) and MB (n) is to be field-encoded.

  On the other hand, if it is determined that the macroblocks MB (m) and MB (n) do not meet any of the conditions (E) to (J) by the processes of steps S151 to S155, the process proceeds to step S157. The MB adaptive coding determination unit 119 determines that a macroblock pair constituted by the macroblocks MB (m) and MB (n) is to be frame-encoded.

  After the process of step S156 or S157, the process proceeds to step S158, and the MB adaptive coding determination unit 119 determines whether the macroblock pair configured by the macroblocks MB (m) and MB (n) and the step S156 or Information that associates the coding method determined in the process of S157 is output.

  Returning to FIG. 6, after the process of step S <b> 106, the process proceeds to step S <b> 107, and the adaptive coding determination unit 119 determines whether or not all macroblock pairs included in one frame image have been extracted. If it is determined that not all macroblock pairs have been extracted yet, the process returns to step S105, and the processes of S105 to S107 are repeatedly executed.

  If it is determined in step S107 that all macroblock pairs included in one frame image have been extracted, the process proceeds to step S108.

  In step S108, the AVC encoding unit 120 encodes the image data of the frame by the AVC encoding method. At this time, individual macroblock pairs are frame-encoded or field-encoded based on the determination result obtained in step S106.

  In step S109, the MPEG decoding unit 115 determines whether or not there is a next frame. If it is determined that there is a next frame, the process returns to step S101, and the subsequent processes are repeatedly executed. .

  If it is determined in step S109 that there is a next frame, the process ends.

  Here, an example in which decoding and re-encoding is performed for each frame has been described, but the unit of data to be decoded and re-encoded is not limited to one frame.

  In this way, transcoding is performed, and image data encoded with MPEG2 is H.264. It is converted into image data encoded by H.264 / AVC.

  For example, in the conventional H.264 / AVC system, JM (Joint Model) is used by software installed in an encoding device to determine whether to perform frame encoding or field encoding for each macroblock pair. ) Was used to determine the encoding method. For example, in the determination in JM's Low Complexity Mode (high-speed mode), the cost corresponding to each encoding method is calculated by Equation (2).

  Note that SA (T) D in Equation (2) is the sum of the total value error of the difference value between the original image and the predicted image, or the Hadamard matrix. SA (T) D0 is calculated by equation (3).

  Note that QP2Quant is a conversion equation for obtaining a quantization scale from the quantization parameter QP, and the header bit (HeaderBit) is taken into account by Equation (3).

  In the determination of the encoding method of the macroblock pair of JM, the cost when the frame encoding is performed for each macroblock pair and the case where the field encoding is performed is calculated as described above, and the encoding that minimizes the cost is performed. The method will be selected.

  However, in the determination of the encoding method of the JM macroblock pair, the amount of computation (processing amount) becomes enormous. For example, when the encoding method of the JM macroblock pair is determined when transcoding is performed, Takes a long time.

  On the other hand, in transcoding by the image processing apparatus 100 of the present invention, the encoding method of each macroblock pair is determined based on information included in MPEG2 decoding information. Therefore, the load on the apparatus is reduced, and transcoding can be performed at higher speed.

  In the above, the example in which the image data input to the image processing apparatus 100 is data encoded by the MPEG2 encoding method has been described. However, the image data input to the image processing apparatus 100 is MPEG4. The data may be encoded by the encoding method. In this case, the MPEG encoding unit 115 may decode the MPEG4 data.

  The series of processes described above can be executed by hardware, or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer 700 as shown in FIG. 9 is installed from a network or a recording medium.

  In FIG. 9, a CPU (Central Processing Unit) 701 executes various processes according to a program stored in a ROM (Read Only Memory) 702 or a program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. To do. The RAM 703 also appropriately stores data necessary for the CPU 701 to execute various processes.

  The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.

  The input / output interface 705 includes an input unit 706 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal display), an output unit 707 including a speaker, and a hard disk. A communication unit 709 including a storage unit 708, a network interface card such as a modem and a LAN card, and the like is connected. The communication unit 709 performs communication processing via a network including the Internet.

  A drive 710 is also connected to the input / output interface 705 as necessary, and a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is loaded. It is installed in the storage unit 708 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network such as the Internet or a recording medium such as a removable medium 711.

  The recording medium shown in FIG. 9 is a magnetic disk (including a floppy disk (registered trademark)) on which a program is recorded, which is distributed to distribute the program to the user, separately from the apparatus main body, Removable media consisting of optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk) (registered trademark)), or semiconductor memory It includes not only those configured by 711 but also those configured by a ROM 702 storing a program, a hard disk included in the storage unit 708, and the like that are distributed to the user in a state of being incorporated in the apparatus main body in advance.

  The steps of executing the series of processes described above in this specification are performed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the order described. It also includes processing.

It is a block diagram which shows the structural example which concerns on one Embodiment of the image processing apparatus to which this invention is applied. FIG. 2 is a block diagram illustrating a configuration example of an AVC encoding unit in FIG. 1. It is a figure explaining the example of a macroblock pair. It is a figure explaining frame motion compensation prediction and field motion compensation prediction. It is a figure explaining frame DCT encoding mode and field DCT encoding mode. It is a flowchart explaining the example of a transcode process. It is a flowchart explaining the example of an encoding system determination process. It is a flowchart explaining another example of an encoding system determination process. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Image processing apparatus, 115 MPEG decoding part, 116 MB Complexity calculation part, 117 Video memory, 118 buffer, 119 MB adaptive encoding determination part, 120 AVC encoding part, 202 Screen rearrangement buffer, 203 Adder, 204 Orthogonal transformation , 206 lossless encoding unit, 207 accumulation buffer, 212 intra prediction unit, motion prediction / compensation unit, 214 rate control unit

Claims (15)

An image processing apparatus that converts image data encoded by the MPEG (Moving Picture Coding Experts Group) method into image data encoded by the H.264 / AVC (Advanced Video Coding) method,
Decoding means for decoding image data encoded by the MPEG method;
MPEG decoding information, which is information obtained when the decoding unit decodes the image data encoded by the MPEG method, and is information set for compressing the baseband image data by the MPEG method. MPEG decoding information acquisition means for acquiring,
Baseband image acquisition means for acquiring baseband image data obtained by decoding the MPEG-encoded image data by the decoding means;
Based on the MPEG decode information acquired by the decode information acquisition means, the baseband image acquired by the baseband image acquisition means includes two macroblocks adjacent in the vertical direction in the image Determination means for determining an encoding method when encoding with the H.264 / AVC method for each of the macroblock pairs to be performed;
An image processing apparatus comprising: encoding means for generating image data encoded by the H.264 / AVC method by encoding each of the macroblock pairs with the encoding method determined by the determining means . Information obtained when the image data encoded by the MPEG system is decoded by the decoding means, the generated bits of each macroblock of the image data encoded by the MPEG system, and the macroblock The image processing apparatus according to claim 1, further comprising: a calculation unit that calculates a value representing complexity corresponding to each of the macroblocks based on information on a quantization scale. A value representing the complexity corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the calculation result of the calculation means. Is determined to be less than or equal to the threshold,
The image processing apparatus according to claim 2, wherein the determination unit uses frame encoding as an encoding method of the macroblock pair. A value representing the complexity corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the calculation result of the calculation means. Is determined to be greater than or equal to the threshold,
The image processing apparatus according to claim 2, wherein the determination unit uses field encoding as an encoding method of the macroblock pair. Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the total value of generated bits is less than or equal to the threshold,
The image processing apparatus according to claim 1, wherein the determination unit uses frame encoding as an encoding method of the macroblock pair. Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means If it is determined that the total value of the generated bits is greater than or equal to the threshold,
The image processing apparatus according to claim 1, wherein the determination unit uses field encoding as an encoding method of the macroblock pair. Based on the MPEG decode information acquired by the decode information acquisition means, in two macroblocks included in the macroblock pair of the baseband image, the absolute value of the difference value of the motion vector corresponding to each macroblock is If it is determined that it is below the threshold,
The image processing apparatus according to claim 1, wherein the determination unit uses frame encoding as an encoding method of the macroblock pair. Based on the MPEG decode information acquired by the decode information acquisition means, in two macroblocks included in the macroblock pair of the baseband image, the absolute value of the difference value of the motion vector corresponding to each macroblock is If it is determined that it is greater than or equal to the threshold,
The image processing apparatus according to claim 1, wherein the determination unit uses field encoding as an encoding method of the macroblock pair. Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means If it is determined that the DCTType to be performed is a frame DCT,
The image processing apparatus according to claim 1, wherein the determination unit uses frame encoding as an encoding method of the macroblock pair. Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the DCTType to be performed is a field DCT,
The image processing apparatus according to claim 1, wherein the determination unit uses field encoding as an encoding method of the macroblock pair. Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the motion compensation prediction mode to be frame motion compensation is
The image processing apparatus according to claim 1, wherein the determination unit uses frame encoding as an encoding method of the macroblock pair. Corresponding to at least one of the two macroblocks in two macroblocks included in the macroblock pair of the baseband image based on the MPEG decode information acquired by the decode information acquisition means When it is determined that the motion compensated prediction mode is field motion compensation,
The image processing apparatus according to claim 1, wherein the determination unit uses field encoding as an encoding method of the macroblock pair. An image processing method of an image processing apparatus for converting image data encoded by MPEG (Moving Picture Coding Experts Group) method into image data encoded by H.264 / AVC (Advanced Video Coding) method,
Decodes image data encoded in MPEG format,
MPEG decoding information that is information obtained when the image data encoded by the MPEG method is decoded and set to compress the baseband image data by the MPEG method,
Obtaining baseband image data obtained by decoding image data encoded by the MPEG method,
Based on the acquired MPEG decoding information, for each of the macroblock pairs that are included in the acquired baseband image and are composed of two macroblocks that are adjacent vertically in the image, H. Determine the encoding method when encoding in H.264 / AVC format,
An image processing method comprising: generating image data encoded by the H.264 / AVC format by encoding each of the macroblock pairs with the determined encoding format. A program that causes an image processing device to perform image processing by converting image data encoded by the MPEG (Moving Picture Coding Experts Group) method into image data encoded by the H.264 / AVC (Advanced Video Coding) method. There,
Controls decoding of image data encoded in MPEG format,
Control the acquisition of MPEG decoding information, which is information obtained when the image data encoded by the MPEG method is decoded, and is set to compress the baseband image data by the MPEG method,
Control acquisition of baseband image data obtained by decoding image data encoded by the MPEG method,
Based on the acquired MPEG decoding information, for each of the macroblock pairs that are included in the acquired baseband image and are composed of two macroblocks that are adjacent vertically in the image, H. Controls the determination of the encoding method when encoding with H.264 / AVC format,
A computer-readable program including a step of controlling generation of image data encoded in the H.264 / AVC format by encoding each of the macroblock pairs in the determined encoding format.   A recording medium on which the program according to claim 14 is recorded.

Images