JP2008004983A - Image processing apparatus and method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance the satisfaction of a user viewing a reproduced image while suppressing a bit rate of coded image data to a low rate. <P>SOLUTION: A caption detection section 155 determines whether a macroblock of image data supplied from an image rearrangement buffer 142 includes a caption, and informs a rate control section 154, a motion prediction/compensation section 153, and a quantization section 145 of information associated with the macroblock including the caption. Upon the receipt of a notice from the caption detection section 155, each of the rate control section 154, the motion prediction/compensation section 153, and the quantization section 145 is operated in a way of not deteriorating images of characters when the macro block of the caption is coded, and the image data are coded. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, a program, and a recording medium, and in particular, can improve the satisfaction of a user who viewed a reproduced image while keeping the bit rate of encoded image data low. The present invention relates to an image processing apparatus and method, a program, and a recording medium.

  In recent years, MPEG (compressed by orthogonal transform such as discrete cosine transform and motion compensation is used for the purpose of efficiently transmitting and storing information, and using redundancy unique to image information. A device conforming to a scheme such as Moving Picture Coding Experts Group) is becoming popular in both information distribution in broadcasting stations and information reception in general households.

  In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system, and is a standard that covers both interlaced scanning images and progressive scanning images, standard resolution images, and high-definition images. Widely used in a wide range of applications for (business) and consumer use. By using the MPEG2 encoding method, for example, a standard resolution interlaced scanning image having 720 × 480 pixels is 4 to 8 Mbps, and a high resolution interlaced scanning image having 1920 × 1088 pixels is 18 to 22 Mbps. By assigning an amount (bit rate), it is possible to achieve a high compression rate and good image quality.

  Since MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, it did not support encoding methods having a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. In the future, with the widespread use of portable terminals and the like, the need for a lower code amount encoding method is expected to increase, and the MPEG4 encoding method has been standardized accordingly. Regarding the MPEG4 image encoding system, the standard was defined as an international standard in December 1998 as ISO / IEC 14496-2.

  Furthermore, in recent years, a standard called H.26L (ITU-T Q6 / 16 VCEG), which was originally formulated for the purpose of image coding for video conferencing, has attracted attention. H. 26L is known to achieve higher encoding efficiency than the conventional encoding schemes such as MPEG2 and MPEG4, although a larger amount of computation is required for encoding or decoding. In addition, as part of MPEG4 activities, this H.264 Based on H.26L Standardization of an encoding method that incorporates a function that is not supported by 26L and realizes higher encoding efficiency is performed as Joint Model of Enhanced-Compression Video Coding. In March 2003, an international standard called H.264 / AVC (Advanced Video Coding) was established.

  In addition, when encoding an image using an encoding method such as MPEG2, MPEG4, or H.264 / AVC, it is common to adjust the bit rate in order to obtain higher encoding efficiency. That is, when a predetermined picture or macroblock in an image is encoded, encoding is usually performed so that the number of bits allocated to the picture or macroblock is reduced.

  As a typical method for adjusting the bit rate (rate control), for example, MPEG-2 TestModel5 (TM5) can be mentioned. The TM5 rate control method has three layers: Step 1 for distributing bits to each picture, Step 2 for performing rate control using virtual buffer control, and Step 3 for performing adaptive quantization considering visual characteristics. It is configured.

  In step 1, the allocated bit amount for each picture in the GOP (Group of Pictures) is determined based on the bit amount allocated to a picture that has not yet been encoded in the GOP including the allocation target picture. To distribute.

  In step 2, in order to match the allocated bit amount for each picture obtained in step 1 with the actual generated code amount, the quantization scale is based on the capacity of three types of virtual buffers set independently for each picture type. Is obtained by feedback control in units of macroblocks.

  In step 3, the quantization scale obtained in step 2 is quantized more finely in the flat portion where deterioration is visually noticeable, and coarser in the complicated portion of the pattern where deterioration is relatively inconspicuous. And change according to the activity of each macroblock. That is, in a macroblock with a high activity that tends to have a large allocated bit amount when encoded, the quantization scale is changed so that a large quantization scale is set. Control is performed so that the number of bits in the data is as small as possible (to reduce the bit rate).

  Furthermore, it has been proposed to efficiently increase the compression rate by changing the compression rate of the image according to the object included in the input image (see, for example, Patent Document 1).

In addition, with the widespread use of these encoding methods, a technique called transcoding that converts data encoded by a certain encoding method into data encoded by another encoding method becomes important.
JP 2005-109606 A

  By the way, in a caption portion such as a telop included in an image, an image (characters or the like) including many edges is displayed, and the activity of the caption portion is high. When encoding an image using an encoding method such as MPEG2, MPEG4, or H.264 / AVC, when TM5 rate control is performed, high activity is detected in the caption portion included in the image, and a large quantization scale is set. Will be set.

  When the quantization scale is large, it is difficult to accurately reproduce the image before being encoded in the image obtained by decoding the encoded image data. However, in general, human visual characteristics have few edges. Because it is sensitive to low-frequency components, when encoding an image with many edges, setting a large quantization scale is an effective method for reducing the number of bits of encoded data. .

  However, characters and the like are displayed in the caption portion, and a user who views the decoded image usually looks at the caption portion more carefully than other portions, and the image in the caption portion is displayed. The deterioration of the image is easily noticed by the user. For this reason, even when encoding is performed so that a more natural and beautiful image can be reproduced while keeping the bit rate low in the conventional encoder, the user who viewed the reproduced image has an impression that the image quality is low. was there.

  The present invention has been made in view of such a situation, and enables the satisfaction of a user who viewed a reproduced image to be increased while keeping the bit rate of encoded image data low. It is.

  One aspect of the present invention is an image processing apparatus that encodes image data using MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding) system, and the image data to be encoded Image data acquisition means for acquiring image data, determination means for determining whether or not a caption is included in the image of the image data acquired by the image data acquisition means, and quantum data for quantizing the image data A rate control unit that controls a bit rate of the encoded image data by changing a conversion parameter according to a feature amount of the image, and a caption is added to the image of the image data by the determination unit. If it is determined that the image is included, the rate control means is a block composed of a plurality of pixels in the portion where the caption is displayed in the image The quantization parameter that is set for an image processing device for a predetermined value irrespective of the feature value of the image.

A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When the determination unit determines that a caption is included in the image of the image data, the predicted image data generation unit includes a plurality of pixels in a portion where the caption is displayed in the image. The motion vector set for the block can be a value in a predetermined range.

  A prediction image data generation unit configured to generate image data of a prediction image corresponding to the image of the image data to be encoded, according to the motion of the image of the image data to be encoded; When it is determined that a caption is included in the image of the data, the predicted image data generation means is a block for setting a motion vector, and a plurality of pixels in the portion where the caption is displayed in the image The size of the block to be configured may be larger than a preset size.

  A prediction image data generation unit configured to generate image data of a prediction image corresponding to the image of the image data to be encoded, according to the motion of the image of the image data to be encoded; When it is determined that a caption is included in the image of the data, the predicted image data generation unit selects a block corresponding to the block composed of a plurality of pixels in the portion where the caption is displayed in the image. The field of the image used to generate the predicted image data including the same may be the same field as the image of the image data to be encoded.

  A prediction image data generation unit configured to generate image data of a prediction image corresponding to the image of the image data to be encoded, according to the motion of the image of the image data to be encoded; When it is determined that a caption is included in the image of the data, the predicted image data generation unit is set for a macroblock including a plurality of pixels in a portion where the caption is displayed in the image. The macroblock mode to be used can be a skipped macroblock.

  A prediction image data generation unit configured to generate image data of a prediction image corresponding to the image of the image data to be encoded, according to the motion of the image of the image data to be encoded; When it is determined that a caption is included in the image of the data, the prediction image data generation unit is configured to perform the prediction corresponding to the block including a plurality of pixels in a portion where the caption is displayed in the image. The image used for generating the image data is one of an image temporally preceding the image of the image data to be encoded and an image temporally subsequent to the image of the image data. Can be.

  A prediction image data generation unit configured to generate image data of a prediction image corresponding to the image of the image data to be encoded, according to the motion of the image of the image data to be encoded; When it is determined that a caption is included in the image of the data, the predicted image data generation means sets the pixel accuracy of the predicted image data to integer pixel accuracy or 1/2 pixel accuracy. Can do.

  The image processing apparatus further includes orthogonal transform processing means for performing orthogonal transform processing on difference data between the image data to be encoded and predicted image data corresponding to the image data. When it is determined that a caption is included, the orthogonal transform processing means can perform an orthogonal transform process on the data in a frame coding mode.

  The image processing apparatus further includes orthogonal transform processing means for performing orthogonal transform processing on difference data between the image data to be encoded and predicted image data corresponding to the image data. When it is determined that the caption is included, the orthogonal transform processing unit sets the value of the orthogonal transform size, which is a unit for performing the orthogonal transform processing on the data, to a value smaller than a preset size. Can be.

  An image processing method according to an aspect of the present invention is an image processing method of an image processing apparatus that encodes image data using MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding). The image data to be encoded is acquired, it is determined whether or not a caption is included in the image of the acquired image data, and it is determined that a caption is included in the image of the image data. A rate control unit that controls a bit rate of the encoded image data by changing a quantization parameter for quantizing the image data according to a feature amount of the image; A quantization parameter set for a block composed of a plurality of pixels in a portion where captions are displayed in an image is set to a predetermined value regardless of the feature amount of the image. Tsu is an image processing method including the flop.

  A program according to one aspect of the present invention is a program that causes an image processing apparatus that performs image processing to encode image data using MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding). And controlling the acquisition of the image data to be encoded, controlling whether or not a caption is included in the image of the acquired image data, and including a caption in the image of the image data A rate at which the bit rate of the encoded image data is controlled by changing a quantization parameter for quantizing the image data according to the feature amount of the image. The control means sets a quantization parameter set for a block composed of a plurality of pixels in a portion where captions are displayed in the image. Computer comprising a step for controlling to a predetermined value irrespective of the amount is readable program.

  In one aspect of the present invention, the image data to be encoded is acquired, it is determined whether a caption is included in the image of the acquired image data, and a caption is included in the image of the image data. When it is determined that the image data is included, the bit rate of the encoded image data is controlled by changing a quantization parameter for quantizing the image data according to the feature amount of the image. The quantization parameter set for the block composed of a plurality of pixels in the portion where the caption is displayed in the image is set to a predetermined value by the rate control means regardless of the feature amount of the image.

  ADVANTAGE OF THE INVENTION According to this invention, the satisfaction of the user who saw the reproduced image can be raised, suppressing the bit rate of the encoded image data low.

  Embodiments of the present invention will be described below. Correspondences between constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  An image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs encoding of image data using MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding). Whether or not a caption is included in the image of the image data acquired by the image data acquisition means (for example, the screen rearrangement buffer 142 in FIG. 1) for acquiring the image data and the image data acquisition means. The determination unit (for example, the caption detection unit 155 in FIG. 1) and a quantization parameter for quantizing the image data are changed according to the feature amount of the image, thereby encoding the image. Rate control means (for example, the rate control unit 154 in FIG. 1) for controlling the bit rate of data, and the determination means adds a caption to the image of the image data. When the rate control means determines that a quantization parameter set for a block composed of a plurality of pixels in a portion where captions are displayed in the image, A predetermined value is used regardless of the feature amount.

  The image processing apparatus includes predicted image data generation means (for example, FIG. 1) that generates image data of a predicted image corresponding to an image of the image data to be encoded, according to a motion of the image of the image data to be encoded. Motion prediction / compensation unit 153), and when the determination unit determines that the image of the image data includes a caption, the prediction image data generation unit displays the caption in the image. A motion vector set for a block composed of a plurality of pixels in a portion that has been set can be a value in a predetermined range.

  This image processing apparatus includes orthogonal transform processing means (for example, an orthogonal transform unit 144 in FIG. 1) that performs orthogonal transform processing on difference data between the image data to be encoded and predicted image data corresponding to the image data. ), And when the determining means determines that the image data includes a caption, the orthogonal transform processing means performs orthogonal transform processing on the data in a frame coding mode. Can be applied.

  An image processing method according to an aspect of the present invention is an image processing method of an image processing apparatus that encodes image data using MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding). The image data to be encoded is acquired (for example, the process of step S101 in FIG. 14), and it is determined whether or not a caption is included in the image of the acquired image data (for example, FIG. 14). In step S103), when it is determined that the image data includes a caption, a quantization parameter for quantizing the image data is changed according to the feature amount of the image. Thus, rate control means (for example, the rate control unit 154 in FIG. 1) for controlling the bit rate of the encoded image data displays the caption in the image. That the quantization parameter is set to the block composed of a plurality of pixels of the portion to a predetermined value irrespective of the feature value of the image (e.g., step S201 in FIG. 15) includes the step.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example according to an embodiment of an image processing apparatus 100 to which the present invention is applied. For example, the image processing apparatus 100 converts an input image signal into image data that has been compressed and encoded by an H.264 / AVC (Advanced Video Coding) method.

  In the figure, an input image signal is first converted into digital data by an A / D converter 141.

  Next, the screen rearrangement buffer 142 rearranges the frames in accordance with the GOP (Group of Pictures) structure of the image compression information to be output.

  For the image data supplied via the screen rearrangement buffer 142, the adder 143 calculates difference information between the pixel value of the image data and the pixel value supplied from the intra prediction unit 152 or the motion prediction / compensation unit 153. And input to the orthogonal transform unit 144.

  When the image data corresponding to the input image signal is image data encoded intra (intra-image), the image data supplied via the screen rearrangement buffer 142 includes the pixel value of the image data, the intra Difference information with the pixel value generated by the prediction unit 152 based on the image data stored in the frame memory 151 is calculated by the adder 143 and input to the orthogonal transformation unit 144, and discrete cosine transform is performed on the difference information. Orthogonal transformation processing such as DCT (Discrete Cosine Transform) and Karhunen-Loeve transformation is performed.

  The transform coefficient output from the orthogonal transform unit 144 is subjected to quantization processing in the quantization unit 145. In the following, an example will be described in which the orthogonal transform unit 144 performs DCT processing as orthogonal transform processing.

  The rate control unit 154 controls the bit rate of the output data by controlling the quantization scale used for the quantization processing by the quantization unit 145 by changing the quantization scale as necessary.

  The quantized transform coefficient output from the quantization unit 145 is input to the lossless transform unit 146, and after lossless encoding processing such as variable length coding and arithmetic coding is performed by the lossless transform unit 146, The data is stored in the storage buffer 147 and output as image data encoded by the H.264 / AVC format.

  On the other hand, the quantized transform coefficient output from the quantization unit 145 is also supplied to the inverse quantization unit 148 and subjected to inverse quantization processing, and then further subjected to inverse orthogonal transformation processing in the inverse orthogonal transformation unit 149. Is applied to obtain decoded image data.

  The decoded image data output from the inverse orthogonal transform unit 149 is stored in the frame memory 151 after the block distortion is removed by the deblocking filter 150.

  In the intra prediction unit 152, the forward prediction mode in which only the frame image data in the forward direction (past side) on the time axis is used as a reference image according to the macroblock to be encoded, and the backward direction (future side) in the time axis. The backward prediction mode in which only the frame image data is used as a reference image and the bidirectional prediction mode in which both of the two frame image data are used as reference images can be applied. In the intra prediction unit 152, information on the intra prediction mode applied to the macroblock is transmitted to the lossless encoding unit 146 and part of header information in the image data encoded by the H.264 / AVC format. Is encoded as

  On the other hand, when the image data corresponding to the input image signal is image data that is inter (inter-image) encoded, the image data supplied via the screen rearrangement buffer 142 is first a motion prediction / compensation unit. 153 is input. At this time, the motion prediction / compensation unit 153 retrieves the image data of the reference image from the frame memory 151, and generates predicted image data by performing motion prediction / compensation processing on the image data.

  The predicted image data output from the motion prediction / compensation unit 153 is input to the adder 143, and the adder 143 includes the pixel value of the image data supplied via the screen rearrangement buffer 142 and the pixel of the predicted image data. The difference information with the value is calculated. Although FIG. 1 shows that the intra prediction unit 152 and the adder 143 are connected, the image data corresponding to the input image signal is inter (inter-image) encoded image data. In this case, it is assumed that the motion prediction / compensation unit 153 and the adder 143 are connected.

  After that, as in the case of intra coding, the data output from the adder 143 is input to the orthogonal transform unit 144, stored in the storage buffer 147 through the processing of the quantization unit 145 and the lossless transform unit 146, and H. It is output as image data encoded by the H.264 / AVC format.

  The quantized transform coefficient output from the quantizing unit 145 is also supplied to the inverse quantizing unit 148, and the image data decoded through the processing of the inverse orthogonal transform unit 149 is processed by the deblocking filter 150. It is stored in the frame memory 151 through processing.

  Note that the motion prediction / compensation unit 153 supplies information on motion vectors generated based on the image data supplied via the screen rearrangement buffer 142 to the lossless encoding unit 146, and the lossless encoding unit 146 The information is subjected to lossless encoding processing such as variable length encoding and arithmetic encoding, and is encoded as part of header information in image data encoded by the H.264 / AVC format.

  Further, the image processing apparatus 100 is provided with a caption detection unit 155 that detects, for example, captions such as telops and captions included in the image in the image data supplied via the screen rearrangement buffer 142.

  The caption detection unit 155 detects captions in units of macroblocks by detecting (specifying) a macroblock including a character image, for example, by detecting the number of edges of a predetermined macroblock of the image. Then, when a caption is detected, the caption detection unit 155 associates a control signal indicating that the caption is detected with information identifying the macroblock in which the caption is detected, the orthogonal transformation unit 144, the motion prediction / The compensation unit 153 and the rate control unit 154 are supplied.

  Here, the macroblock is a block composed of, for example, 16 × 16 pixels in the image data to be encoded, and is a processing unit when performing the encoding process in the H.264 / AVC format. The

  In the image processing apparatus 100 of the present invention, when detected by the caption detection unit 155, image data encoding processing is performed so as to suppress degradation of the caption portion in the image as much as possible. In other words, when the encoded image data is decoded and displayed, the image processing apparatus 100 processes the image so that the caption portion noted by the user in the image is not deteriorated as much as possible.

  When encoding an image using an encoding method such as H.264 / AVC, it is common to adjust the bit rate in order to obtain higher encoding efficiency. That is, when a predetermined picture or macroblock in an image is encoded, encoding is usually performed so that the number of bits allocated to the picture or macroblock is reduced.

  As a typical method for adjusting the bit rate (rate control), for example, MPEG-2 TestModel5 (TM5) can be mentioned. The TM5 rate control method has three layers: Step 1 for distributing bits to each picture, Step 2 for performing rate control using virtual buffer control, and Step 3 for performing adaptive quantization considering visual characteristics. It is configured.

  In step 1, the allocated bit amount for each picture in the GOP is distributed based on the bit amount allocated to the picture that has not yet been encoded in the GOP including the allocation target picture.

  In step 2, in order to match the allocated bit amount for each picture obtained in step 1 with the actual generated code amount, the quantization scale is based on the capacity of three types of virtual buffers set independently for each picture type. Is obtained by feedback control in units of macroblocks.

  In step 3, the quantization scale obtained in step 2 is quantized more finely in the flat portion where deterioration is visually noticeable, and coarser in the complicated portion of the pattern where deterioration is relatively inconspicuous. And change according to the activity of each macroblock.

For example, the activity act _j of the j-th macroblock is obtained by Expression (1).

  Here, var sblk is a value that represents a variance value of pixel values of the divided sub-blocks by dividing one macroblock into four sub-blocks composed of 8 × 8 pixels. , Which are obtained by the equations (2) and (3).

  Here, Pk is a value representing a pixel value in one macroblock.

  That is, according to the equation (1), the macroblock is divided into four subblocks composed of 8 × 8 pixels, and the case of the frame DCT coding mode for each of the divided subblocks and the field The dispersion value of the pixel value for the two cases in the DCT encoding mode is obtained, and the smallest one of the dispersion values (var sblk) of the pixel values of the eight sub-blocks thus obtained is selected. Will be.

  Then, the normalized activity Nactj whose value is in the range of 0.5 to 2 is obtained by the equation (4).

  Here, avgact is an average value of actj in the picture encoded immediately before.

  The finally obtained quantization scale code mquantj is given by equation (5) based on the quantization scale code Qj obtained in step 2.

  That is, in step 3 of TM5 rate control, the quantization scale is changed so that a large quantization scale is set in a macroblock having a high activity in which the allocated bit amount when encoded is likely to be large. .

  As described above, when the quantization scale is changed based on the activity, it is possible to control the encoded image data so that the number of bits is as small as possible (to reduce the bit rate). In the caption portion, it is easy for the user to be aware of image degradation.

  FIG. 2 shows an example of a character image included in the caption portion. In the drawing, the character “A” is shown, and one character is displayed in four macroblocks of macroblocks (MB) A to D. In the same figure, it is assumed that macroblocks A to D are composed of 16 × 16 pixels, and in the above-described activity calculation, each of macroblocks A to D is composed of 8 × 8 pixels. Assume that it is divided into four sub-blocks.

  Consider a case where the macro block B in FIG. 2 is encoded. The macroblock B is divided into subblocks B-1 to B-4, and the subblocks B-1, B-3, and B-4 include part of the character image. However, the sub-block B-2 does not include a character image.

  As described above, in step 3 of TM5 rate control, the macroblock is divided into four sub-blocks composed of 8 × 8 pixels according to the equation (1), and the sub-blocks of the divided sub-blocks are divided. The smallest of the variance values of the respective pixel values is selected. For example, in the macroblock B, the variance value of the pixel value is larger as the image having more edges and smaller as the image having fewer edges. Therefore, in the macroblock B, the subblock B− A variance value of 2 will be selected.

  Since the sub-block B-2 does not include a character image, there is no edge and the pixel dispersion value (activity) is low. Therefore, in step 3 of the rate control of TM5, the macroblock B is regarded as a macroblock with low activity, and a large quantization scale is not set.

  On the other hand, in macroblocks A, C, and D, every sub-block includes a part of the character image. Therefore, in step 3 of TM5 rate control, macroblocks A, C, and D is regarded as a macroblock with high activity, and a large quantization scale is set.

  When the quantization scale is large, it is difficult to accurately reproduce an image before encoding in an image obtained by decoding encoded image data. FIG. 3 shows an image obtained by encoding the image of FIG. 2 using normal TM5 rate control and decoding the encoded image data.

  As shown in the figure, in the image obtained by decoding, the image is clearly displayed at the position of macroblock B, but the image is unclear at the positions of macroblocks A, C, and D. (Blurred) is displayed. As described above, when a part of the character is blurred and displayed, there is a high possibility that the user viewing the displayed image has an impression that the image quality is low.

  Therefore, in the image processing apparatus 100 of the present invention, when a caption is detected, the macro block including the caption is not subjected to the process of step 3 of TM5 rate control.

  That is, when the caption detection unit 155 detects a caption in the image supplied from the screen rearrangement buffer 144, information indicating that the caption is detected and information specifying the macroblock of the detected caption are displayed. The data is output to the control unit 154.

  For example, when an image including the image of FIG. 2 is input to the caption detection unit 155, the caption detection unit 155 includes information indicating that the caption has been detected and information specifying each of the macroblocks A to D (for example, , Position information, etc.) to the rate control unit 154, the caption detection unit 155 notifies the rate control unit 154 that the macroblocks A to D are caption macroblocks. At this time, since the caption detection and notification is performed in units of macro blocks, not in units of sub-blocks, the macro block B having the sub-block B-2 that does not include a character image is naturally also rated as a macro block of captions. The control unit 154 is notified.

  When receiving the notification of the caption macroblock from the caption detection unit 155, the rate control unit 154 prevents the processing of TM5 rate control step 3 from being performed on the caption macroblock. That is, for the macroblocks A to D, the quantization scale obtained by the TM2 rate control step 2 processing is kept as it is regardless of the macroblock activity level (without the step 3 processing). Applied.

  As a result, caption macroblocks (macroblocks containing character images) have high activity, but a large quantization scale is not set, and in an image obtained by decoding encoded image data. The image before encoding can be reproduced almost accurately.

  In addition, when the caption control unit 154 has not received a caption macroblock notification from the caption detection unit 155, the rate control unit 154 performs normal TM5 rate control processing (including the processing in step 3) on these macroblocks. Process). That is, for macroblocks that do not include character images, TM5 rate control steps 1 to 3 are performed.

  As a result, for macroblocks that do not include a character image, the rate control unit 154 controls the quantization unit 145 based on the activity to change the quantization scale, so that encoding processing is performed. It is possible to perform control so that the number of bits in the image data is reduced as much as possible (to reduce the bit rate).

  In general, human visual characteristics are sensitive to low-frequency components with few edges, so when encoding an image that contains many edges, as in TM3 rate control step 3, set a large quantization scale. This is an effective method for reducing the number of bits of encoded data.

  However, characters and the like are displayed in the caption portion, and a user who views the decoded image usually looks at the caption portion more carefully than other portions, and the image in the caption portion is displayed. The deterioration of the image is easily noticed by the user. For this reason, even if encoding is performed so that the decoded image becomes a more natural and beautiful image while keeping the bit rate low in an encoder, the impression that the image quality is low for the user who actually viewed the decoded image. There was a case that gave.

  On the other hand, in the image processing apparatus 100 of the present invention, when the macro block of the caption portion is encoded, the process of step 3 of the TM5 rate control is not performed, so that the user who viewed the decoded image can be There is no sense of incongruity. In addition, although the encoding efficiency in the caption portion of the image is lowered, the processing in step 3 of the TM5 rate control is performed in the portion other than the caption portion, so that the efficiency of the entire image in consideration of human visual characteristics. Encoding will be performed. As a result, according to the present invention, it is possible to increase the satisfaction level of the user who viewed the reproduced image, while keeping the bit rate of the encoded image data low.

  Up to this point, an example has been described in which encoding is performed so as to minimize degradation of the caption portion by changing the rate control method by the rate control unit 154. However, degradation of the caption portion is minimized by other methods. It can also be encoded as follows.

  First, an example will be described in which the generation of predicted image data by the motion prediction / compensation unit 153 is encoded so as to minimize the deterioration of the caption portion by appropriately controlling.

  FIG. 4A and FIG. 4B are diagrams for explaining the process of motion compensation prediction by the motion prediction / compensation unit 153. FIG. In image data encoded by H.264 / AVC, when the frame structure of the image data is a frame structure, the macroblock is a frame block of 16 pixels × 16 lines (luminance signal) in which the top field and the bottom field are interlaced. Motion compensation prediction called frame motion compensation prediction or field motion compensation prediction is used.

  In the frame motion compensation prediction, motion compensation prediction is performed in a frame in which two interlaced fields are combined, and a luminance signal is predicted for each interlaced 16 pixel × 16 line block. In an interlaced signal, of two fields constituting one frame, a spatially upper field is called a top field, and a spatially lower field is called a bottom field.

  FIG. 4A is a diagram illustrating an example in which forward motion compensation prediction is performed from a reference frame (a frame of predicted image data output from the motion prediction / compensation unit 153) separated by one frame, for example. In the figure, pixels in the top field are indicated by circles, and pixels in the bottom field are indicated by rectangles, and an input frame (screen rearrangement buffer) corresponding to the reference frame according to the motion vector indicated by “MV”. The pixel position of the frame of the image data output from 142 is specified.

  The frame motion compensated prediction is an effective prediction method when, for example, the motion is relatively slow and the motion in the frame is moving at a constant speed with a high correlation.

  On the other hand, the field motion compensation prediction performs motion compensation for each field. As shown in FIG. 4b, the motion vector “MV1” is set in the top field, and the motion vector “MV2” is set in the bottom field. According to each motion vector of “MV1” or “MV2”, the pixel position of the input frame corresponding to the reference frame is specified.

  Further, the field in the reference frame corresponding to the pixel in the input frame can be a top field, or can be a bottom field. In the example of FIG. 4b, the top field in the reference frame is referred to for both the top field pixel and the bottom field pixel in the input frame. In the figure, pixels in the top field are indicated by circles, and pixels in the bottom field are indicated by rectangles. In the field motion compensation prediction, prediction is performed for each field in the macroblock. The prediction is performed in units of field blocks of the line.

  When characters are displayed in the image, it is usually considered that the displayed characters are unlikely to move in the screen as time passes. For example, the shape of the character “A” Is the same shape in images that are temporally changed. Therefore, in the image processing apparatus 100 of the present invention, when a caption is detected, for a macroblock including the caption, the motion vector set to the macroblock is fixed to “0”, for example.

  When normal motion compensation prediction processing is performed on a macroblock including a character or a part of a character, a different motion vector may be set for each macroblock. That is, the motion vector may be set based on the background image or the like instead of the character included in the macroblock. In H.264 / AVC encoding, one macroblock is divided into four subblocks each composed of 8 × 8 pixels, and a motion vector for each subblock is set. .

  FIG. 5 shows an example of an image obtained by encoding the image of FIG. 2 by performing a normal motion compensation prediction process and decoding the encoded image data. In the figure, each arrow shown in each frame (sub-block) represents a motion vector. In this example, for example, a separate motion vector is set for each of the sub-blocks in accordance with the motion of the background image, and as a result, the character “a” shown in the image of FIG. 5 is distorted. It is displayed in the state with.

  On the other hand, in the image processing apparatus 100 of the present invention, the motion vector set for the macroblock including the caption is fixed to “0”, for example.

  For example, when an image including the image of FIG. 2 is input to the caption detection unit 155, the caption detection unit 155 includes information indicating that the caption has been detected and information specifying each of the macroblocks A to D (for example, , Position information, etc.) to the motion prediction / compensation unit 153, the caption detection unit 155 notifies the motion prediction / compensation unit 153 that the macroblocks A to D are macroblocks of the caption. Become.

  When receiving the notification of the caption macroblock from the caption detection unit 155, the motion prediction / compensation unit 153 fixes the set motion vector to “0” for the caption macroblock. . That is, for macroblocks A to D, motion compensation in the predicted image data is not performed.

  As a result, a caption macroblock (a macroblock including a character image) is obtained by decoding the encoded image data with the motion vector fixed to “0” even if the background image moves. In the image, it is possible to reproduce the image of the character (in this case, “A”) almost accurately.

  Also, when the motion prediction / compensation unit 153 has not received a caption macroblock notification from the caption detection unit 155, the motion prediction / compensation unit 153 sets a motion vector for the macroblocks as usual. That is, for a macroblock that does not include a character image, an appropriate prediction image corresponding to the motion of the image is generated.

  As a result, the user who sees the decoded image does not feel uncomfortable, and the entire image is encoded so that a beautiful image in consideration of motion can be decoded.

  Here, it has been described that the motion vector is fixed to “0”, but the fixed value is not limited to “0”, for example, a predetermined relatively small value close to “0”. You may make it fix to the value of a range.

  Alternatively, when the motion prediction / compensation unit 153 receives the notification of the caption macroblock from the caption detection unit 155, the size of the subblock in which the motion vector is set is set for the macroblock including the caption. Alternatively, it may be set larger.

  That is, in setting a motion vector, as described above, normally, one macro block is divided into four sub-blocks composed of 8 × 8 pixels, and the motion vector for each sub-block is set. However, when a caption is detected, the size of the sub-block may be a block composed of, for example, 16 × 16 pixels for a macroblock including the caption. In this way, only one motion vector is set for each of the macroblocks A to D. For example, as the background image moves as described above with reference to FIG. Thus, different motion vectors are set for each of the individual sub-blocks, and it is possible to prevent the characters of the image from being displayed in a distorted state.

  Alternatively, when a caption is detected, only the frame motion compensated prediction shown in FIG. 4a may be performed on the macroblock including the caption, and the field motion compensated prediction may not be performed. .

  6 (a) to 6 (c) are diagrams in which the entire image of the character “A” is displayed, a diagram in which only the top field of the interlace is displayed, and a display in which only the bottom field of the interlace is displayed. FIG. As shown in the figure, an image of a character usually does not move. Therefore, when encoding, a part belonging to a top field in the character preferably refers to the top field, and a bottom of the character It is desirable to refer to the bottom field for the part belonging to the field. By doing so, it is possible to accurately display the image of the character “A” when the encoded image is decoded.

  However, in the field motion compensation prediction, as described above with reference to FIG. 4B, the top field pixel is referred to the bottom field pixel, or the top field pixel is referred to the bottom field pixel. Therefore, when field motion compensation prediction is performed on a macroblock including a caption, when the encoded image is decoded, the image of the character “a” is accurately Display may not be possible.

  Therefore, only the frame motion compensated prediction shown in FIG. 4A is performed for the macroblock including the caption. In this case, when the motion prediction / compensation unit 153 receives a caption macroblock notification from the caption detection unit 155, the prediction macroblock performs only frame motion compensation prediction to generate predicted image data. You just have to do it. When only frame motion compensated prediction is performed in H.264 / AVC encoding, the reference picture number (ref_idx) set for the macroblock is “0” or a preset value. Should be set to.

  As a result, the caption macroblock (macroblock including the character image) is the same as the character (in the image obtained by decoding the encoded image data, as in the case where the motion vector is fixed to “0”. In this case, the image “a”) can be reproduced almost accurately.

  In addition, when the caption detection unit 155 has not received a caption macroblock notification from the caption detection unit 155, the motion prediction / compensation unit 153 applies frame motion compensation prediction or field motion compensation prediction to the macroblocks as usual. To generate predicted image data. That is, for a macroblock that does not include a character image, an appropriate prediction image corresponding to the motion of the image is generated.

  As a result, the user who sees the decoded image does not feel uncomfortable, and the entire image is encoded so that a beautiful image in consideration of motion can be decoded.

  Alternatively, when the motion prediction / compensation unit 153 receives a notification of a caption macroblock from the caption detection unit 155, the macroblock mode is set to “skipped macroblock” for the macroblock including the caption. The macroblock mode may be fixed.

  For macroblocks for which the macroblock mode of “skip macroblock” is set, the difference from the reference plane image (predicted image data image) is not extracted, and the encoded image data is decoded. As a result, the obtained image becomes the same image as the image of the reference plane.

  The image of the character included in the caption is usually almost non-moving. For example, the shape of the character “A” is the same shape in the images that move forward and backward, so the image of the reference plane is In many cases, it is possible to display an image of a character accurately when displayed as it is.

  FIG. 7A shows an example of an image (reference plane image) of predicted image data generated by the motion prediction / compensation unit 153 when an image of the character “A” is encoded. FIG. 7B illustrates an example of an image obtained by decoding encoded data obtained by extracting a difference from the predicted image data illustrated in FIG. As shown in the figure, the image of FIG. 7B is displayed as a distorted image as compared with the image of FIG.

  Differences from the reference plane image (predicted image data image) are not extracted for macroblocks for which the macroblock mode of “skipped macroblock” is set, so that the macroblock includes a character image. That is, if a macroblock mode of “skip macroblock” is set for the caption macroblock, an image with less distortion as shown in FIG. 7A is displayed. Is possible.

  Note that it is not always necessary to set the macro block mode called “skip macro block” for the macro block of the caption, and “skipped macro block” is set more than the other macro block modes. You should make it easy to be done.

  Alternatively, when the motion prediction / compensation unit 153 receives a caption macroblock notification from the caption detection unit 155, the reference direction (prediction direction) is limited to one direction for the macroblock including the caption. You may make it do.

  For example, when a macroblock to be encoded by the H.264 / AVC format is a B-slice macroblock, three types of predictive encoding are possible: forward prediction, backward prediction, or bidirectional prediction. That is, the forward prediction / motion prediction / compensation unit 153 that generates and encodes prediction image data based on an image temporally prior to the macroblock image to be encoded is encoded by the motion prediction / compensation unit 153. Prediction image data based on an image temporally earlier than an image to be encoded by the backward prediction and motion prediction / compensation unit 153 that generates and encodes predicted image data based on an image temporally after the power image Among the bi-directional predictions that generate and encode predicted image data based on the subsequent images, only forward prediction or backward prediction may be performed on a macroblock including a caption.

  FIGS. 8A to 8C show examples of images (reference plane images) of predicted image data generated by the motion prediction / compensation unit 153 when an image of the character “A” is encoded. Yes. 8A is an example of a reference plane image in the case of forward prediction, FIG. 8B is an example of a reference plane image in the case of backward prediction, and FIG. It is an example of the image of the reference surface in the case of direction prediction. As shown in the figure, the image of FIG. 8C is displayed as a distorted image as compared with the image of FIG. 8A or FIG. 8B.

  In the case of bi-directional prediction, predicted image data is generated by the average value of pixels included in each macroblock of an image temporally preceding and temporally subsequent to an image of a macroblock to be encoded. Therefore, when there is even a slight spatial shift or the like in the temporally preceding image and the temporally subsequent image, the image of the predicted image data to be generated is shown in FIG. Will be distorted.

  An image of predicted image data generated by performing only forward prediction or backward prediction on a macroblock including a caption is shown in FIG. 8A or FIG. 8B. Such an image with less distortion can be obtained. As a result, even in an image obtained by decoding the encoded data, the character image can be accurately displayed.

  Alternatively, when the motion prediction / compensation unit 153 receives the notification of the caption macroblock from the caption detection unit 155, the motion compensation accuracy for the macroblock including the caption is set to integer pixel accuracy, or You may make it restrict | limit to 1/2 pixel precision.

  For example, in H.264 / AVC encoding, motion compensation accuracy can be integer pixel accuracy, 1/2 pixel accuracy, and 1/4 pixel accuracy. For example, when motion compensation with 1/2 pixel accuracy is performed, if there is no pixel at the position specified by the motion vector in the image of the reference plane, the pixel is intermediate between the two pixels based on the two neighboring pixel values. A process of virtually generating the pixel value of the pixel located is performed.

  FIG. 9 is a diagram for explaining an example of the accuracy of motion compensation. Assume that there are six pixels E, F, G, H, I, and J in the image of the reference plane. For example, in motion compensation in the H.264 / AVC format, a pixel value of a ½ precision pixel is generated by performing a filtering process called a 6 tap Fair Filter. For example, when generating the pixel value of the pixel b in the figure, the pixel value of the pixel b is generated by performing a filtering process that multiplies each of the pixel values of the pixels E to J by a preset coefficient. The

  For example, when the pixel value of the pixel a or c is generated, motion compensation with 1/4 pixel accuracy is performed. In this case, since the pixel value cannot be generated only from the actual pixels E to J in the image of the reference plane, the pixel value of the pixel a or c is generated by motion compensation with 1/2 pixel accuracy. The pixel value of the generated pixel b is also used.

  That is, when motion compensation with ¼ pixel accuracy is performed, virtual pixels are generated based on nonexistent pixels in the reference plane image.

  FIGS. 10A to 10C show examples of predicted image data images (reference plane images) generated by the motion prediction / compensation unit 153 when an image of the character “A” is encoded. Yes. FIG. 10A is an example of a reference plane image in the case of integer-precision motion compensation, and FIG. 10B is an example of a reference plane image in the case of half-pixel precision motion compensation. FIG. 10C is an example of an image of the reference surface in the case of motion compensation with ¼ pixel accuracy. As shown in the figure, the image of FIG. 10C is displayed as a distorted image as compared with the image of FIG. 10A or 10B.

  The image of the character included in the caption is usually almost non-moving. For example, the shape of the character “A” is the same shape in the images that are temporally changed, so it is virtually generated. When predictive image data is generated based on the obtained pixels, the image is often distorted.

  For macroblocks including captions, the accuracy of motion compensation is limited to integer pixel accuracy or ½ pixel accuracy, so that an image of predicted image data generated is displayed as shown in FIG. a) or an image with less distortion as shown in FIG. 10 (b). As a result, even in an image obtained by decoding the encoded data, the character image can be accurately obtained. Can be displayed.

  Next, an example will be described in which processing related to orthogonal transformation by the orthogonal transformation unit 144 is encoded so as to minimize the deterioration of the caption portion by appropriately controlling.

  In the MPEG2 encoding method, when DCT is performed as orthogonal transform processing by the orthogonal transform unit 144, two types of DCT encoding modes are used. FIG. 11A and FIG. 11B are diagrams for explaining the respective DCT coding modes.

  In the case of the frame DCT coding mode, when the luminance signal of the macroblock is decomposed into, for example, four subblocks, each subblock has a top field and a bottom field, as shown in FIG. It is decomposed | disassembled so that it may be comprised including.

  On the other hand, in the case of the field DCT coding mode, when the luminance signal of the macroblock is decomposed into, for example, four subblocks, each of the subblocks is the top as shown in FIG. It is disassembled to consist of only the field or bottom field.

  In the H.264 / AVC coding system, whether to perform frame coding or field coding in units of macroblock pairs composed of two macroblocks that are adjacent vertically (in the vertical direction) in an image is adaptive. It has been made to be able to select. In the sequence parameter set RBSP (Raw Byte Sequence Payloads) in the H.264 / AVC bitstream, there is a parameter called mb_adaptive_frame_field_flag (macroblock adaptive frame field flag), and in the slice header, field_pic_flag There is a parameter called (field picture flag). The setting of these flags determines the encoding method (frame encoding or field encoding) in units of frames and macroblocks.

  When the image data to be encoded is in an interlace (interlace scanning) format, encoding processing at the picture level or macroblock level (individual picture or macroblock pair is subjected to frame encoding or field encoding) Can be adaptively performed. For example, if Mb_adaptive_frame_field_flag in the sequence parameter set of the H.264 / AVC bitstream is set to “1” and field_pic_flag in the slice header is set to “0”, frame coding is performed for the entire picture, and the macroblock Field encoding or frame encoding can be performed on the pair.

  By the way, in an image obtained by performing orthogonal transformation processing and quantization processing on a macroblock or sub-block, compressing and encoding, and decoding the encoded data, the edge periphery due to the quantization error, Mosquito noise is generated.

  The quantization error when encoding image data occurs in the frame unit in the above-described frame DCT encoding mode in the MPEG2 encoding method, and in the field DCT encoding mode in the above-described field DCT encoding mode. Will occur. That is, when encoding in the frame DCT encoding mode is performed, mosquito noise due to quantization error is generated almost uniformly in both the top field and the bottom field, whereas encoding in the field DCT encoding mode is performed. In some cases, mosquito noise due to quantization errors may be more prominent in either the top field or the bottom field.

  FIGS. 12A and 12B show examples of encoded data obtained by encoding an image of the character “A” and an image obtained by decoding. FIG. 12A is an example of an image in the case of the frame DCT encoding mode, FIG. 12B is an example of an image in the case of the field DCT encoding mode, and FIG. In the figure, horizontal mosquito noise is generated in the horizontal direction. As described above, when more remarkable mosquito noise is generated in either the top field or the bottom field (regularly) in the character image, the mosquito noise is generated in the entire character. ), The distortion becomes easier to be seen than in the case of the image.

  Therefore, in the image processing apparatus 100 of the present invention, when the orthogonal transform unit 144 receives the notification of the caption macroblock from the caption detection unit 155, the frame code is not applied to the macroblock pair of the caption macroblock. Only encoding by encoding is performed.

  By doing in this way, it is possible to suppress the occurrence of mosquito noise that is easy for the user to see as shown in FIG. 12B in the image obtained by decoding the data obtained by encoding the caption image.

  In addition, when the orthogonal transform unit 144 performs encoding by frame encoding on the macroblock pair, the motion prediction / compensation unit 153 simultaneously performs the frame motion compensation prediction described above with reference to FIG. You may do it.

  Alternatively, when the orthogonal transform unit 144 receives a notification of a caption macro block from the caption detection unit 155, the macro block including the caption is converted into a sub-block along with the orthogonal transform process. When dividing, the size of the sub-block to be divided (so-called orthogonal transform size) may be limited to a sub-block composed of 4 × 4 pixels.

  As described above, mosquito noise is generated due to quantization error in an image obtained by compressing and encoding by performing orthogonal transform processing and quantization processing, and decoding the encoded data. Noise propagates in units of sub-blocks subjected to orthogonal transform processing.

  FIGS. 13A and 13B show examples of encoded data obtained by encoding an image of the character “A” and an image obtained by decoding. FIG. 13A is an example of an image when the so-called orthogonal transformation size is 4 × 4 (pixels), and FIG. 13B is an example of the so-called orthogonal transformation size of 8 × 8 (pixels). In this example, the distortion of the image in FIG. 13B is more easily visible than that of the image in FIG.

  In other words, if the orthogonal transform size is large, once a quantization error occurs, mosquito noise occurs in a wide range in the image, but if the orthogonal transform size is small, even if a quantization error occurs, the mosquito noise is When encoding a character image, mosquito noise is more visible in the image obtained by decoding the encoded data when the orthogonal transform size is smaller. It will be difficult.

  In this way, by limiting the orthogonal transform size to 4 × 4, the image obtained by decoding the data obtained by encoding the caption image is easily visible to the user as shown in FIG. 13B. Generation of mosquito noise can be suppressed.

  Next, the encoding process by the image processing apparatus 100 will be described with reference to the flowchart of FIG.

  In step S101, the caption detection unit 155 acquires image data of an image to be encoded from now. At this time, for example, image data supplied via the screen rearrangement buffer 142 is acquired by the caption detection unit 155.

  In step S102, the caption detection unit 155 analyzes the image of the image data acquired in the process of step S101. At this time, for example, by detecting the number of edges of a predetermined macroblock of the image, it is checked whether or not the macroblock contains characters.

  In step S103, the caption detection unit 155 determines whether or not a caption is detected in the macroblock of the image based on the analysis result obtained in step S102.

  If it is determined in step S103 that a caption has been detected, the process proceeds to step S104. At this time, the caption detection unit 155 associates, for example, a control signal indicating that the caption is detected with information identifying the macroblock in which the caption is detected, the orthogonal transform unit 144, the motion prediction / compensation unit 153, And supplied to the rate control unit 154.

  In step S104, a caption handling process is executed as described later with reference to FIGS.

  FIG. 15 is a flowchart for explaining a detailed example of the caption handling process in step S104 of FIG.

  In step S201 in the figure, the rate control unit 154 that has received the notification of the caption macroblock from the caption detection unit 155 performs only the TM5 rate control step 1 and step 2 for the caption macroblock. . That is, the processing of step 3 of TM5 rate control is not performed on the caption macroblock.

  As a result, as described above with reference to FIG. 3, in the image obtained by decoding the encoded image data, a part of characters is prevented from being blurred.

  FIG. 16 is a flowchart for explaining another example of the details of the caption handling process in step S104 of FIG.

  In step S221 in the figure, the motion prediction / compensation unit 153 that has received the notification of the caption macroblock from the caption detection unit 155 sets the motion vector set for the caption macroblock to “0” or “0”. A value in a predetermined range close to “”.

  Accordingly, as described above with reference to FIG. 5, in the image obtained by decoding the encoded image data, a separate motion vector is set for each of the sub-blocks, and the characters are distorted. Display in a certain state is suppressed.

  FIG. 17 is a flowchart illustrating yet another example of the details of the caption handling process in step S104 of FIG.

  In step S241 in the figure, the motion prediction / compensation unit 153 that has received the notification of the caption macroblock from the caption detection unit 155 determines the size of the sub-block in which the motion vector is set for the caption macroblock. The block is composed of 16 × 16 pixels.

  As a result, as described above with reference to FIG. 5, in the image obtained by decoding the encoded image data, a separate motion vector is set for each sub-block, and characters are Displaying in a distorted state is suppressed.

  FIG. 18 is a flowchart illustrating yet another example of the details of the caption handling process in step S104 of FIG.

  In step S261 in FIG. 9, the motion prediction / compensation unit 153 that has received the notification of the caption macroblock from the caption detection unit 155 sets the reference picture number (ref_idx) set for the caption macroblock to “0”. Or set to a preset value.

  As a result, only the frame motion compensation prediction shown in FIG. 4A is performed for the macroblock including the caption, and distortion in the image of the predicted image data due to the field motion compensation prediction is generated. Deterred.

  FIG. 19 is a flowchart illustrating yet another example of the details of the caption handling process in step S104 of FIG.

  In step S281 in the figure, the motion prediction / compensation unit 153 that has received the notification of the caption macroblock from the caption detection unit 155 performs “skipped macroblock” for the macroblock mode set for the caption macroblock. "Is prioritized and set.

  As a result, as described above with reference to FIGS. 7A and 7B, the encoded data obtained by extracting the difference from the predicted image data and decoding it is not an image obtained by decoding. The possibility that the image of the predicted image data is displayed as it is is increased, and an image with less distortion can be displayed.

  FIG. 20 is a flowchart for explaining yet another example of the details of the caption handling process in step S104 of FIG.

  In step S <b> 301 in FIG. 6, the motion prediction / compensation unit 153 that has received the notification of the caption macroblock from the caption detection unit 155 performs forward prediction or backward prediction on the caption macroblock.

  Accordingly, as described above with reference to FIGS. 8A to 8C, the image of the predicted image data generated by the bidirectional prediction is prevented from being distorted.

  FIG. 21 is a flowchart for explaining yet another example of details of the caption handling process in step S104 of FIG.

  In step S321 in the figure, the motion prediction / compensation unit 153 that has received the caption macroblock notification from the caption detection unit 155 changes the pixel accuracy of the motion compensation for the caption macroblock to the integer pixel system or 1/2 The pixel system.

  Accordingly, as described above with reference to FIGS. 10A to 10C, the image of the predicted image data is prevented from being a distorted image by the motion compensation with ¼ pixel accuracy.

  FIG. 22 is a flowchart illustrating yet another example of the details of the caption handling process in step S104 of FIG.

  In step S341 in the figure, the orthogonal transform unit 144 that has received the notification of the caption macroblock from the caption detection unit 155 performs frame coding (frame DCT coding mode) on the macroblock pair of the caption macroblock. Encoding is performed.

  Thus, as described above with reference to FIGS. 12A and 12B, field encoding is performed on the caption image, and an image obtained by decoding the encoded data is obtained. Therefore, it is possible to suppress the occurrence of mosquito noise that is easily visible to the user.

  FIG. 23 is a flowchart illustrating yet another example of the details of the caption handling process in step S104 of FIG.

  In step S361, the orthogonal transform unit 144 that has received the caption macroblock notification from the caption detection unit 155 sets the orthogonal transform size of the macroblock pair of the caption macroblock to 4 × 4.

  As a result, as described above with reference to FIGS. 13A and 13B, encoding is performed in a state where the orthogonal transform size is large with respect to the caption image, and the data obtained by encoding the image is decoded. It is possible to suppress the occurrence of mosquito noise that is easily visible to the user in the obtained image.

  As described above, the caption handling process is executed. In step S104 of FIG. 14, one of the processes described above with reference to FIGS. 15 to 23 may be executed, or all of the processes described above with reference to FIGS. 15 to 23 may be performed. You may be made to be. Furthermore, in step S104 of FIG. 14, a plurality of processes among the processes described above with reference to FIGS. 15 to 23 may be appropriately selected and executed.

  After the process of step S104 in FIG. 14, the image data is encoded by the H.264 / AVC format in step S105. At this time, as described above with reference to FIG. 1, the adder 143 to the rate control unit 154 operate. However, when it is determined that the caption is detected in the process of step S103, the rate control unit 154 and the motion prediction Each of the compensation unit 153 and the quantization unit 145 is assumed to operate corresponding to the processing described above with reference to FIGS.

  In step S106, it is determined whether or not all image data has been encoded. If it is determined that all image data has not been encoded yet, the process returns to step S101, and the subsequent processes are repeated. Executed.

  If it is determined in step S106 that all image data has been encoded, the encoding process ends.

  In the above, in the image processing apparatus 100, the H.264 Although an example in which encoding by H.264 / AVC is performed has been described, the present invention can be applied even when encoding by another encoding method (compression encoding method) such as MPEG4 is performed.

  The series of processes described above can be executed by hardware, or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer 700 as shown in FIG. 24 is installed from a network or a recording medium.

  24, a CPU (Central Processing Unit) 701 executes various processes according to a program stored in a ROM (Read Only Memory) 702 or a program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. To do. The RAM 703 also appropriately stores data necessary for the CPU 701 to execute various processes.

  The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.

  The input / output interface 705 includes an input unit 706 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal display), an output unit 707 including a speaker, and a hard disk. A communication unit 709 including a storage unit 708, a network interface card such as a modem and a LAN card, and the like is connected. The communication unit 709 performs communication processing via a network including the Internet.

  A drive 710 is also connected to the input / output interface 705 as necessary, and a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is loaded. It is installed in the storage unit 708 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network such as the Internet or a recording medium such as a removable medium 711.

  Note that this recording medium is a magnetic disk (including a floppy disk (registered trademark)) on which a program is recorded, which is distributed to distribute the program to the user separately from the apparatus main body shown in FIG. Removable media consisting of optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk) (registered trademark)), or semiconductor memory It includes not only those configured by 711 but also those configured by a ROM 702 storing a program, a hard disk included in the storage unit 708, and the like that are distributed to the user in a state of being incorporated in the apparatus main body in advance.

  The steps of executing the series of processes described above in this specification are performed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the order described. It also includes processing.

It is a block diagram which shows the structural example which concerns on one Embodiment of the image processing apparatus to which this invention is applied. It is a figure which shows the example of the macroblock in which the image of a character is contained. It is a figure which shows the example of the image obtained by decoding the image data in which the image of FIG. 2 was encoded. It is a figure explaining frame motion compensation prediction and field motion compensation prediction. It is a figure which shows the example of the macroblock containing the image of the character in which the separate motion vector was set. It is a figure which shows the example of the image of an interlace, the image of a top field, and the image of a bottom field. It is a figure which shows the example of the image extracted by extracting the difference of the image of prediction image data, and the image of prediction image data. It is a figure which shows the example of the image of the prediction image data produced | generated by forward prediction, backward prediction, and bidirectional | two-way prediction. It is a figure explaining the precision of motion compensation. It is a figure which shows the example of the image of the estimated image data produced | generated by the motion compensation of integer precision, 1/2 pixel precision, and 1/4 pixel precision. It is a figure explaining frame DCT encoding mode and field DCT encoding mode. It is a figure which shows the example of the image obtained by decoding the data encoded by each of frame DCT encoding mode and field DCT encoding mode. It is a figure which shows the example of the image obtained by decoding the data encoded with each of orthogonal transformation size 4x4 and 8x8. It is a flowchart explaining an encoding process. It is a flowchart explaining a caption corresponding process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. It is a flowchart explaining another example of a caption corresponding | compatible process. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

  100 image processing apparatus, 142 screen rearrangement buffer, 143 adder, 144 orthogonal transform unit, 145 quantization unit, 146 lossless encoding unit, 147 accumulation buffer, 153 motion prediction / compensation unit, 154 rate control unit, 155 caption detection Part

Claims (12)

MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding) format image processing apparatus for encoding image data,
Image data acquisition means for acquiring the image data to be encoded;
Determination means for determining whether or not a caption is included in the image of the image data acquired by the image data acquisition means;
A rate control unit that controls a bit rate of the encoded image data by changing a quantization parameter for quantizing the image data according to a feature amount of the image;
When it is determined by the determination means that the image of the image data includes a caption, the rate control means is configured to a block composed of a plurality of pixels in a portion where the caption is displayed in the image. An image processing apparatus that sets a quantization parameter set to a predetermined value regardless of the feature amount of the image. A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When the determination unit determines that a caption is included in the image of the image data, the predicted image data generation unit includes a plurality of pixels in a portion where the caption is displayed in the image. The image processing apparatus according to claim 1, wherein the motion vector set for the block is a value in a predetermined range. A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When the determination unit determines that the image data includes a caption, the prediction image data generation unit is a block for setting a motion vector, and the caption is displayed in the image. The image processing apparatus according to claim 1, wherein a size of the block including a plurality of pixels in a certain portion is larger than a preset size. A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When the determination unit determines that a caption is included in the image of the image data, the predicted image data generation unit includes a plurality of pixels in a portion where the caption is displayed in the image. The image processing apparatus according to claim 1, wherein a field of an image used for generating the predicted image data including a block corresponding to the block is the same field as an image of the image data to be encoded. A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When the determination unit determines that a caption is included in the image of the image data, the predicted image data generation unit includes a plurality of pixels in a portion where the caption is displayed in the image. The image processing apparatus according to claim 1, wherein the macroblock mode set for the macroblock is a skipped macroblock. A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When the determination unit determines that a caption is included in the image of the image data, the predicted image data generation unit includes a plurality of pixels in a portion where the caption is displayed in the image. An image used to generate the predicted image data corresponding to the block is an image temporally preceding the image data image to be encoded, or an image temporally subsequent to the image data image. The image processing apparatus according to claim 1, wherein the image processing apparatus is any one of them. A prediction image data generation unit configured to generate image data of a prediction image corresponding to an image of the image data to be encoded, according to a motion of an image of the image data to be encoded;
When it is determined by the determination unit that the image of the image data includes a caption, the predicted image data generation unit sets the pixel accuracy of the predicted image data to integer pixel accuracy or 1/2 pixel. The image processing apparatus according to claim 1, wherein the image processing apparatus has accuracy. Further comprising orthogonal transform processing means for performing orthogonal transform processing on difference data between the image data to be encoded and predicted image data corresponding to the image data;
2. The orthogonal transform processing unit performs orthogonal transform processing on the data in a frame coding mode when the determination unit determines that a caption is included in the image of the image data. The image processing apparatus described. Further comprising orthogonal transform processing means for performing orthogonal transform processing on difference data between the image data to be encoded and predicted image data corresponding to the image data;
When the determining unit determines that the image data includes a caption, the orthogonal transform processing unit sets an orthogonal transform size value, which is a unit for performing orthogonal transform processing on the data. The image processing apparatus according to claim 1, wherein the value is smaller than a preset size. An image processing method of an image processing apparatus that encodes image data by MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding) system,
Obtaining the image data to be encoded;
Determining whether a caption is included in the image of the acquired image data;
When it is determined that a caption is included in the image of the image data, the image data is encoded by changing a quantization parameter for quantizing the image data according to the feature amount of the image. A rate control means for controlling the bit rate of the image data uses a quantization parameter set for a block composed of a plurality of pixels in a portion where captions are displayed in the image as a feature amount of the image. Regardless of this, an image processing method including a step of setting a predetermined value. An MPEG (Moving Picture Coding Experts Group) 4 or H.264 / AVC (Advanced Video Coding) format image processing apparatus that performs image processing on an image processing apparatus,
Controlling the acquisition of the image data to be encoded,
Controlling whether or not a caption is included in the image of the acquired image data;
When it is determined that a caption is included in the image of the image data, the image data is encoded by changing a quantization parameter for quantizing the image data according to the feature amount of the image. A rate control means for controlling the bit rate of the image data uses a quantization parameter set for a block composed of a plurality of pixels in a portion where captions are displayed in the image as a feature amount of the image. Regardless of this, a computer-readable program including a step for controlling to a predetermined value.   A recording medium on which the program according to claim 11 is recorded.

Images