JP2007259149A - Encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem of deterioration of encoding efficiency when a macro-block of reference place through movement prediction is shifted out of a ROI region. <P>SOLUTION: A ROI region is traced by a ROI tracing unit 64, and positional information of ROI region in an encoding objective frame is held in a ROI information holding unit 67. In a movement research unit 66, an objective macro-block of the encoding objective frame is subjected to movement research, and the reference macro-block is specified at the reference frame. In a ROI inside and outside decision unit 68, the position information of ROI region of the reference frame is acquired from the ROI information holding unit 67. It is determined whether the reference macro-block detected by the movement research unit 66 is located in the ROI region of the reference frame or not. If at least part of the reference macro-block is located outside the ROI region of the reference frame, pixels outside the ROI region of the reference frame out of pixels in the reference macro-block are filled up by a padding unit 70 using pixel data of the reference macro-block in the ROI region. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an encoding method for encoding an image, and more particularly to an encoding method for encoding a moving image for each region.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. Also, in one stream, images with different image quality (for example, high and low image quality), different resolution (for example, high and low resolution), and different frame rates (for example, high and low frame rates) depending on the code amount H. can be compressed and decompressed. There is a next-generation image compression technique such as SVC (Scalable Video Coding), which is being standardized as an extension of H.264 / AVC.

  SVC, the next-generation image compression technology, encodes moving images with various scalability such as spatial scalability, temporal scalability, and SNR scalability so that moving images can be played at multiple different resolutions, frame rates, and image quality. Turn into. Coding can be performed by arbitrarily combining these scalability, and the scalability function of SVC is very flexible.

  ROI coding is a technique for coding a region of interest (ROI) of an image with a different image quality from other regions. In SVC, interactive ROI (Interactive Region of Interest; IROI) coding is listed as one of the requirements. The SVC interactive ROI encoding allows the user to successively specify the position and size of the attention area on the screen while viewing the image, and enables the attention area to be reproduced with different quality. is there. In SVC, since a moving image is encoded with various scalability, it is possible to decode a region of interest designated by the user at the time of reproduction with a quality different from that of other regions.

In compression encoding of a moving image, motion compensation is performed. In Patent Document 1, a pixel set having high correlation is searched for each divided region of the encoding target frame within the important region of the previous frame, and data of each divided region, data of the searched pixel set, and There is disclosed a moving picture coding apparatus provided with inter-frame prediction means for taking the difference between the two.
JP 2006-14086 A

  When encoding a region of interest, pixel data in the region of interest is stored in a buffer, and a motion vector is detected in units of macroblocks in the region of interest. If the reference destination of the motion vector deviates from the attention area, only the pixel data in the attention area is held in the buffer, and the reference destination pixel data cannot be obtained. Although the motion vector search range can be limited to the region of interest, the accuracy of the motion search is reduced, so that the coding efficiency is deteriorated and the image quality is lowered.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an encoding technique capable of improving the image quality when encoding a moving image by motion compensation.

  In order to solve the above-described problem, an encoding method according to an aspect of the present invention provides a region to be encoded independently on a picture constituting a moving image, and an encoding target block in the region in the encoding target picture. It is determined whether or not the reference block when the motion search is performed is in the area in the reference picture, and if at least a part of the reference block is outside the area in the reference picture, the block is in the area Motion prediction is performed after the pixels of the reference block outside the region are generated using the pixel data of the reference block.

  Here, a picture is a unit of coding, and its concept includes a frame, a field, a VOP (Video Object Plane), and the like.

  According to this aspect, when encoding is performed by providing a region to be encoded with different image quality on the moving image, even if the reference destination by the motion search is outside the region, the reference destination pixel is converted into pixel data within the region. Since the reference image can be generated by using the image, the image quality of the moving image encoded by the motion compensation prediction is improved.

  The independently encoded region may be encoded with a quality different from other regions. Encoding with different quality includes encoding with different spatial resolution, frame rate, image quality level, SNR level and the like.

  Track movement between pictures of the independently encoded region, determine whether the reference block is in the region in the tracked reference picture, and track the reference block in the tracked reference picture The position information of the area may be included in the encoded data of the moving image. Tracking movement of the region between pictures may be performed by motion search or by extracting a specific object that should be in the region. When the region itself set on the moving image moves, the region can be tracked for motion compensation. Further, since the position information of the tracked area is included in the encoded data, it is possible to easily determine whether or not the reference block is out of the tracked area on the decoding side.

  When the entire reference destination block is outside the independently encoded region in the reference picture, pixel data in the independently encoded region in the reference picture that is closest to the reference destination block is used. Thus, the pixels of the reference block may be generated. Alternatively, when the entire reference destination block is outside the area in the reference picture, the pixels in the area in the reference picture closest to the reference destination block overlap with at least a part of the reference destination block. The reference block may be moved so that the reference destination of the motion search indicates the reference block after the movement. With respect to the reference block after being moved in this way, at least part of the reference block overlaps the area in the reference picture, so that the area using the pixel data of the reference block in the area Pixels of the reference block outside can be generated.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, in the encoding and decoding of a moving image using motion compensation, the image quality of a moving image can be improved.

Embodiment 1
FIG. 1 is a configuration diagram of an encoding apparatus 100 according to Embodiment 1. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding apparatus 100 according to the present embodiment includes an MPEG (Moving Picture Experts Group) series standard (MPEG-1, MPEG, standardized by ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission)). -2 and MPEG-4), standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector), which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264 respectively).

  In the MPEG series standards, I (Intra) frames are used for intra-frame coding, P (Predictive) frames are used for inter-frame predictive coding with a past frame as a reference frame, and past and future frames. A frame that performs bidirectional inter-frame predictive coding using a frame as a reference frame is called a B frame.

  On the other hand, H. In H.264 / AVC, a frame that can be used as a reference frame may be a past two frames as a reference frame or a future two frames as a reference frame regardless of the time. Further, three or more frames can be used as reference frames regardless of the number of frames that can be used as reference frames. Therefore, in MPEG-1 / 2/4, the B frame refers to a Bi-directional prediction frame. Note that in H.264 / AVC, the B frame refers to a bi-predictive prediction frame, since the time of the reference frame does not matter.

  In the embodiment, a frame is used as an example of the encoding unit, but the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The encoding apparatus 100 receives an input of a moving image in units of frames, encodes the moving image, and outputs an encoded stream.

  The block generation unit 10 divides the input moving image frame into macro blocks. Macroblocks are formed in order from the upper left to the lower right of the frame. The block generation unit 10 supplies the generated macroblock to the differentiator 12 and the motion compensation prediction processing unit 60.

  If the frame supplied from the block generation unit 10 is an I frame, the difference unit 12 outputs the frame to the DCT unit 20 as it is. However, if the frame is a P frame or a B frame, the difference unit 12 predicts from the motion compensation prediction processing unit 60. The difference from the frame is calculated and supplied to the DCT unit 20.

  The ROI initial setting unit 62 initializes an ROI (Region of Interest) region on the moving image. The ROI area may be selected by the user specifying a specific area on the image, or a predetermined area such as the center area of the image may be selected. In addition, an important area such as an area in which a person or a character is shown may be automatically extracted as the ROI area.

  The parameters characterizing the ROI area include the coordinate value of the upper left corner of the ROI area, the width and height of the ROI area, the relative image quality level of the ROI area with respect to other areas, the shape of the ROI area, and the like.

  The encoding apparatus 100 encodes the ROI region with a quality different from that of other regions. For example, the ROI region is encoded with a spatial resolution, a frame rate, an image quality level, an SNR level, or a combination thereof different from other regions.

  For example, when the ROI region is encoded with higher image quality than other regions, the quantization step applied to the ROI region may be reduced by using a different quantization table during quantization. Thus, by encoding a larger number of effective bits, for example, by reducing the number of lower bits that are discarded, encoding is performed with higher image quality than other areas. Further, the ROI area may be encoded with a higher resolution than other areas.

  Note that the ROI area is not necessarily set only for reproduction with high image quality. For example, for the purpose of protecting privacy, it is necessary to reproduce a region of interest in which a person's face is captured with low image quality. Therefore, it is possible to designate an area requiring privacy protection as the ROI area and encode it with a lower image quality than other areas in advance.

  The ROI initial setting unit 62 may reset the ROI area for the moving image. Here, the resetting of the ROI area is also included in the initial setting. The ROI initial setting unit 62 gives the position information of the initially set ROI region to the motion compensation prediction processing unit 60.

  The motion compensation prediction processing unit 60 uses the past or future frame stored in the frame buffer 80 as a reference frame, and performs motion compensation for each macroblock of the P frame or B frame input from the block generation unit 10. Generate motion vectors and prediction frames. The motion compensated prediction processing unit 60 supplies the generated motion vector to the variable length encoding unit 90 and supplies the prediction frame to the differentiator 12 and the adder 14.

  The motion compensation prediction processing unit 60 tracks the ROI region initially set by the ROI initial setting unit 62, and performs a motion search within the ROI region. The motion compensation prediction processing unit 60 performs an exception process to be described later when a macroblock referred to by a motion vector is out of the ROI region for the macroblock in the ROI region of the encoding target frame. The motion compensation prediction processing unit 60 provides the variable length encoding unit 90 with the position information of the ROI region of the encoding target frame.

  The subtractor 12 obtains a difference between the current encoding target frame output from the block generation unit 10 and the prediction frame output from the motion compensation prediction processing unit 60 and outputs the difference to the DCT unit 20. The DCT unit 20 performs a discrete cosine transform (DCT) on the difference frame given from the differentiator 12 and gives a DCT coefficient to the quantization unit 30.

  The quantization unit 30 quantizes the DCT coefficient and provides it to the variable length coding unit 90. The variable length coding unit 90 performs variable length coding on the quantized DCT coefficient of the difference frame together with the motion vector supplied from the motion compensated prediction processing unit 60 to generate a coded stream. The variable length encoding unit 90 performs processing of rearranging the encoded frames in time order when generating the encoded stream.

  The variable length coding unit 90 also appropriately encodes the position information of the ROI region given from the motion compensation prediction processing unit 60 and includes it in the coded stream. The position information of the ROI area may be included in the header portion of the encoded stream. Various parameters other than the position information of the ROI area are also stored in the encoded stream.

  The quantization unit 30 supplies the quantized DCT coefficient of the frame to the inverse quantization unit 40. The inverse quantization unit 40 inversely quantizes the supplied quantized data and supplies the quantized data to the inverse DCT unit 50. The inverse DCT unit 50 performs inverse discrete cosine transform on the supplied inverse quantized data. Thereby, the encoded frame is restored. The restored frame is input to the adder 14.

  If the frame supplied from the inverse DCT unit 50 is an I frame, the adder 14 stores it in the frame buffer 80 as it is. If the frame supplied from the inverse DCT unit 50 is a P frame or a B frame, the adder 14 is the difference frame supplied from the inverse DCT unit 50 and supplied from the motion compensation prediction processing unit 60. By adding the predicted frames, the original frame is reconstructed and stored in the frame buffer 80.

  The reconstructed frame stored in the frame buffer 80 is used as a reference frame for motion compensation by the motion compensation prediction processing unit 60.

  In the case of P frame or B frame encoding processing, the motion compensation prediction processing unit 60 operates as described above. However, in the case of I frame encoding processing, the motion compensation prediction processing unit 60 does not operate, and here Then, although not shown, intra-frame prediction is performed.

  FIG. 2 is a configuration diagram of the motion compensation prediction processing unit 60.

  The ROI tracking unit 64 tracks the ROI region based on the ROI region position information initialized by the ROI initial setting unit 62 and the motion vector information detected by the motion search unit 66. Specifically, the movement of an object or the like in the ROI area is detected, the movement amount and movement direction from the initial position of the ROI area are obtained, and the position of the ROI area in the encoding target frame is determined.

  The ROI tracking unit 64 provides the ROI area position information (referred to as “ROI position information”) of the encoding target frame to the variable length encoding unit 90 for inclusion in the encoded data, and holds it in the ROI information holding unit 67. . The ROI position information of the encoding target frame held in the ROI information holding unit 67 is used as the ROI position information of the reference frame at the time of motion compensation.

  Until the ROI area is reset by the ROI initial setting section 62, the ROI tracking section 64 automatically tracks the subsequent ROI area based on the latest ROI position information held in the ROI information holding section 67. I do.

  The motion search unit 66 uses the reconstructed frame stored in the frame buffer 80 as a reference frame for the macroblock (referred to as “target macroblock”) of the encoding target frame given from the block generation unit 10, The inside is searched in units of macroblocks, and the macroblock having the smallest difference from the target macroblock is selected as the reference macroblock. Thereby, a motion vector indicating the motion from the target macroblock to the reference macroblock is determined. The motion search unit 66 gives the motion vector of the target macroblock to the variable length coding unit 90.

  The motion search unit 66 simply supplies the pixel data of the reference macroblock to the motion prediction unit 72 as it is for the target macroblock in the normal region other than the ROI region of the encoding target frame, but the ROI of the encoding target frame. For the target macroblock in the area, if the reference macroblock is not in the ROI area, exception processing is performed on the reference macroblock, and the reference macroblock after the exception processing is sent to the motion prediction unit 72. Supply.

  For the target macroblock in the ROI region, the motion search unit 66 gives the position information of the reference macroblock corresponding to the target macroblock to the ROI inside / outside determination unit 68. The ROI inside / outside determination unit 68 acquires the position information of the ROI region of the reference frame from the ROI information holding unit 67 and determines whether the reference macroblock detected by the motion search unit 66 is within the ROI region of the reference frame. . When at least a part of the reference macroblock is outside the ROI area of the reference frame, the ROI inside / outside determination unit 68 instructs the padding unit 70 to execute the padding process as an example of the exception process.

  The padding unit 70 performs a padding process to compensate pixels outside the ROI area of the reference frame among the pixels in the reference macroblock using the pixel data of the reference macroblock in the ROI area. Thereby, even when at least part of the reference macroblock detected by the motion search is out of the ROI region, the pixel data of the reference macroblock can be generated. The padding unit 70 gives the pixel data of the reference macroblock subjected to the padding process to the motion prediction unit 72.

  The motion prediction unit 72 appropriately uses the pixel data of the reference macroblock after the padding process given from the padding unit 70 in addition to the pixel data of the reference macroblock given from the motion search unit 66 to appropriately generate a motion-compensated prediction frame. It is generated and given to the differentiator 12 and the adder 14.

  FIGS. 3A and 3B are diagrams illustrating the relationship between the target macroblock of the encoding target frame and the reference macroblock of the reference frame.

  3A illustrates the ROI area 210 of the encoding target frame 200, and FIG. 3B illustrates the ROI area 212 of the reference frame 202. Since the focused objects 230 and 232 move, the ROI regions 210 and 212 also move between the two frames. The ROI areas 210 and 212 are tracked by the ROI tracking unit 64.

  It is assumed that the target macroblock 220 in the ROI area 210 of the encoding target frame 200 refers to the reference macroblock 222 of the reference frame 202 based on the motion vector detected by the motion search unit 66. As shown in FIG. 3B, a part of this reference macroblock 222 is outside the ROI area 212 of the reference frame 202.

  In this case, outside the ROI area 212 of the reference frame 202, since the reference frame 202 is not ROI-encoded, only pixel data with low image quality exists. If a pixel value is determined using pixel data that is not ROI encoded outside the ROI area 212 for the pixel of the reference macroblock 222 that is out of the ROI area 212, the target macro to be ROI encoded Consistency with the block 220 cannot be obtained, and generally, a prediction error becomes large and coding efficiency deteriorates.

  Therefore, the padding unit 70 performs padding processing for pixel values of the reference macroblock 222 outside the ROI area 212 using pixel data of the reference macroblock 222 inside the ROI area 212 to compensate for the pixel value.

  4A to 4C are diagrams illustrating an example of padding processing. FIG. 4A shows a case where the reference macro block 222 straddles the right end boundary of the ROI region 212. The right half (indicated by diagonal lines) of the reference macroblock 222 is outside the ROI area 212, and the left half of the reference macroblock 222 is inside the ROI area 212.

  In this case, since there is no pixel data of the ROI area 212 for the right half of the reference macroblock 222, the padding unit 70 is the pixel of the reference macroblock 222 within the ROI area 212 and the outermost pixel. Are directly copied to the pixels of the reference macroblock 222 outside the ROI area 212. As shown in the figure, the rightmost pixel columns a to g of the ROI region 212 are horizontally padded to the pixel column of the reference macroblock 222 outside the ROI region 212, and the right half pixel of the reference macroblock 222 is To be compensated.

  FIG. 4B shows a case where the reference macroblock 222 crosses the upper boundary of the ROI area 212. The upper half of the reference macroblock 222 is outside the ROI area 212, and the lower half of the reference macroblock 222 is , Inside the ROI region 212. In this case, the padding unit 70 pads the upper half of the reference macroblock 222 by padding the pixel columns a to g at the upper end of the ROI region 212 vertically upward with the pixel columns of the reference macroblock 222 outside the ROI region 212. To fill in the pixels.

  FIG. 4C shows a case where the reference macroblock 222 straddles the upper right corner of the ROI area 212. In this case, the padding unit 70 pads the pixel columns a to d at the upper end of the ROI region 212 in the vertical upward direction, and pads the pixel columns d to g at the right end of the ROI region 212 in the horizontal right direction. Further, the padding unit 70 pads the pixel d at the upper right corner of the ROI region 212 in the diagonal upper right direction. As a result, all the pixels in the portion of the reference macroblock 222 that protrudes from the ROI area 212 are compensated.

  FIGS. 5A and 5B are diagrams illustrating padding processing when the reference macroblock 222 is completely removed from the ROI area 212. FIG.

  As shown in FIG. 5A, when the entire reference macroblock 222 is on the right side of the right end of the ROI area 212, the padding unit 70 determines that the rightmost pixel of the ROI area 212 has the same horizontal position as the reference macroblock 222. By duplicating the columns a to g as they are in the pixel columns of the reference macroblock 222, all the pixels of the reference macroblock 222 are compensated.

  As another method, when the entire reference macroblock 222 is out of the ROI area 212, the padding unit 70 moves the reference macroblock 222 to the nearest side of the ROI area 212 by one pixel row as shown in FIG. Just move to overlap. Then, the padding unit 70 pads the pixel macros a to g at the right end of the ROI area 212 in the horizontal right direction with respect to the moved reference macroblock 224 as described in FIG. Then, each pixel column of the reference macro block 224 is compensated. At this time, the reference destination of the motion vector is changed to indicate the reference macroblock after the movement.

  FIG. 6 is a flowchart for explaining the procedure of motion compensation prediction by the motion compensation prediction processing unit 60.

  The motion search unit 66 detects a motion vector for each macroblock between the encoding target frame of the moving image and the reference frame (S10). The ROI tracking unit 64 automatically tracks the ROI region in each frame of the moving image using the motion vector information (S12).

  The ROI inside / outside determination unit 68 determines whether or not the reference macroblock referenced by the motion vector is in the ROI region of the reference frame for the target macroblock in the ROI region of the encoding target frame (S14).

  If the reference macroblock is not in the ROI area (N in S14), the padding unit 70 compensates the pixels of the reference macroblock outside the ROI area by the padding process (S16), and proceeds to step S18. If the reference macroblock is in the ROI area (Y in S14), the process proceeds to step S18 without performing the padding process. The motion prediction unit 72 generates a prediction frame using a reference macroblock that has been subjected to padding processing as necessary (S18).

  FIG. 7 is a configuration diagram of decoding apparatus 300 according to Embodiment 1. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding apparatus 300 receives an input of the encoded stream, decodes the encoded stream, and generates an output image.

  The variable length decoding unit 310 performs variable length decoding on the input encoded stream, supplies the decoded image data to the inverse quantization unit 320, and supplies motion vector information to the motion compensation processing unit 360.

  The inverse quantization unit 320 inversely quantizes the image data decoded by the variable length decoding unit 310 and supplies the image data to the inverse DCT unit 330. The image data inversely quantized by the inverse quantization unit 320 is a DCT coefficient. The inverse DCT unit 330 restores the original image data by performing inverse discrete cosine transform (IDCT) on the DCT coefficients inversely quantized by the inverse quantization unit 320. The image data restored by the inverse DCT unit 330 is supplied to the adder 312.

  When the image data supplied from the inverse DCT unit 330 is an I frame, the adder 312 outputs the image data of the I frame as it is and also generates a reference frame for generating a predicted frame of a P frame or a B frame. Is stored in the frame buffer 380.

  When the image data supplied from the inverse DCT unit 330 is a P frame, the adder 312 receives the difference frame supplied from the inverse DCT unit 330 and the motion compensation processing unit 360 because the image data is a difference frame. By adding the supplied prediction frames, the original image data is restored and output.

  The motion compensation processing unit 360 generates a predicted frame of P frame or B frame using the motion vector information supplied from the variable length decoding unit 310 and the reference frame stored in the frame buffer 380, and sends it to the adder 312. Supply.

  The motion compensation processing unit 360 acquires the position information of the ROI area of each frame from the variable length decoding unit 310, and the macroblock referenced by the motion vector is out of the ROI area for the macroblock in the ROI area of the decoding target frame. In such a case, exception processing similar to that performed by the motion compensation prediction processing unit 60 of the encoding device 100 is performed.

  FIG. 8 is a configuration diagram of the motion compensation processing unit 360.

  The ROI acquisition unit 364 acquires the position information of the ROI area of the decoding target frame from the variable length decoding unit 310 and stores it in the ROI information holding unit 367. The ROI position information of the decoding target frame held in the ROI information holding unit 367 is used as the ROI position information of the reference frame in motion compensation.

  The motion specifying unit 366 acquires the motion vector of the decoding target frame from the variable length decoding unit 310, and specifies the reference macroblock that the motion vector refers to for the target macroblock of the decoding target frame.

  For the target macroblock in the normal region other than the ROI region of the decoding target frame, the motion identification unit 366 only supplies the pixel data of the reference macroblock to the motion compensation unit 372 as it is, but within the ROI region of the decoding target frame. For the target macroblock, if the reference macroblock is not in the ROI area, exception processing is performed on the reference macroblock, and then the reference macroblock after the exception processing is supplied to the motion compensation unit 372. .

  For the target macroblock in the ROI area, the motion identification unit 366 gives the position information of the reference macroblock corresponding to the target macroblock to the ROI inside / outside determination unit 368. The ROI inside / outside determination unit 368 acquires the position information of the ROI region of the reference frame from the ROI information holding unit 367 and determines whether the reference macroblock specified by the motion specifying unit 366 is within the ROI region of the reference frame. . When at least a part of the reference macroblock is outside the ROI area of the reference frame, the ROI inside / outside determination unit 368 instructs the padding unit 370 to execute the padding process as an example of exception processing.

  The padding unit 370 performs a padding process for compensating for pixels outside the ROI area of the reference frame among the pixels in the reference macroblock using the pixel data in the ROI area. Thereby, even if at least a part of the reference macroblock specified by the motion vector is out of the ROI region, the pixel data of the reference macroblock can be generated. The padding unit 370 gives the pixel data of the reference macroblock subjected to the padding process to the motion compensation unit 372.

  The padding process performed by the padding unit 370 is the same as the padding process performed by the padding unit 70 of the encoding device 100.

  The motion compensation unit 372 generates a motion-compensated prediction frame using the pixel data of the reference macroblock given from the motion identifying unit 366 or the pixel data of the reference macroblock after the padding process given from the padding unit 370. To the adder 312.

  According to the present embodiment, in ROI encoding and decoding processing that specifies an ROI region and encodes and decodes the ROI region with different image quality from other regions, the macroblock in the ROI region is referred to by motion prediction. Even if the previous macroblock is out of the ROI region, the pixel of the macroblock out of the ROI region can be compensated using the pixel data in the ROI region.

  This makes it possible to generate a more accurate prediction frame even when the reference macroblock is out of the ROI region, and prevents deterioration in encoding efficiency due to motion compensation prediction, as well as moving images in ROI encoding and decoding. Image quality can be maintained. In particular, when the ROI area moves with time, the reference macroblock often deviates from the ROI area, so the effect of the padding process described in this embodiment is increased.

Embodiment 2
FIG. 9 is a configuration diagram of encoding apparatus 100 according to Embodiment 2. The components common to the first embodiment are denoted by the same reference numerals, and description thereof is omitted as appropriate.

  In the encoding apparatus 100 according to the first embodiment, the reference macroblock that protrudes from the ROI region is subjected to padding processing that is compensated with the pixel value in the ROI region. In the encoding apparatus 100 according to the second embodiment, in addition to the padding process using the pixel values in the ROI area, the pixel values in the non-ROI area at the position of the reference macroblock are used instead of padding with the pixel values in the ROI area. Substitute pixel substitution processing can be executed, and processing that is advantageous in terms of coding efficiency can be selected.

  The motion compensation prediction processing unit 60 generates both a prediction frame based on the reference macroblock generated by the padding process and a prediction frame based on the reference macroblock generated by the pixel substitution process, and adds each prediction frame to the differentiator 12. Is supplied to the container 14. The DCT unit 20, the quantization unit 30, and the variable length coding unit 90 of the encoding apparatus 100 according to the present embodiment process these two types of prediction frames, respectively, and generate differentially encoded data based on each prediction frame.

  The code amount determination unit 92 acquires the code amount of the encoded frame differentially encoded using each of the two types of prediction frames from the variable length encoding unit 90, selects the one with the smaller code amount, The variable length coding unit 90 is provided with a flag for identifying a predicted frame with a smaller number of frames. The variable length coding unit 90 employs a coded frame based on the prediction frame identified by the identification flag, and discards the other coded frame. The variable length coding unit 90 includes information on an identification flag indicating which type of prediction frame is adopted in each coded frame in the coded stream.

  In the above description, either padding processing or pixel substitution processing is selected for each encoding target frame. However, either padding processing or pixel substitution processing may be selected for each macroblock in the encoding target frame. In this case, information of an identification flag is provided for each macroblock and encoded in the encoded stream. Further, either padding processing or pixel substitution processing may be selected for each group including a plurality of encoding target frames.

  FIG. 10 is a configuration diagram of the motion compensation prediction processing unit 60. The motion compensation prediction processing unit 60 according to the present embodiment is different from the motion compensation prediction processing unit 60 according to the first embodiment in that a substitute pixel acquisition unit 74 is provided in addition to the padding unit 70.

  When the ROI inside / outside determination unit 68 determines that at least a part of the reference macroblock is outside the ROI region of the reference frame, the padding unit 70 performs the padding processing of the reference macroblock outside the ROI region as in the first embodiment. A reference macroblock is generated by supplementing the pixels, and is provided to the motion prediction unit 72. In parallel with this padding process, the substitute pixel acquisition unit 74 generates a reference macroblock by using the pixel value of the non-ROI area at the same position or in the vicinity of the pixel of the reference macroblock outside the ROI area. To the motion prediction unit 72.

  Similar to the first embodiment, the motion prediction unit 72 appropriately uses the pixel data of the reference macroblock after the padding process given from the padding unit 70 in addition to the pixel data of the reference macroblock given from the motion search unit 66. Generate a motion-compensated prediction frame. At the same time, in addition to the pixel data of the reference macroblock given from the motion search unit 66, the motion prediction unit 72 receives the pixel data of the reference macroblock after substitution processing by the pixels in the non-ROI region given from the substitution pixel acquisition unit 74. A motion-compensated prediction frame is generated as appropriate. The motion prediction unit 72 gives these two types of prediction frames to the differentiator 12 and the adder 14.

  FIG. 11 is a configuration diagram of the decoding apparatus 300 according to the second embodiment. The components common to the first embodiment are denoted by the same reference numerals, and description thereof is omitted as appropriate.

  Decoding apparatus 300 according to the present embodiment has switching flag acquisition section 390. The switching flag acquisition unit 390 acquires an identification flag indicating whether padding processing or pixel substitution processing is employed in encoding when the reference macroblock is out of the ROI region, and provides the flag to the motion compensation processing unit 360. This identification flag may be any of a frame unit, a macroblock unit, and a group unit including a plurality of frames.

  Based on the identification flag acquired by the switching flag acquisition unit 390, the motion compensation processing unit 360 generates a reference macroblock by switching to a padding process or a pixel substitution process for a reference macroblock that protrudes from the ROI region. Is used to generate and output a prediction frame.

  FIG. 12 is a configuration diagram of the motion compensation processing unit 360. The motion compensation processing unit 360 of the present embodiment is different from the motion compensation processing unit 360 of the first embodiment in that a substitute pixel acquisition unit 374 and a switching unit 376 are provided in addition to the padding unit 370.

  The switching unit 376 refers to the identification flag obtained from the switching flag acquisition unit 390, selects either the padding unit 370 or the substitute pixel acquisition unit 374, and performs switching control so that only the selected one operates.

  When switching to the padding unit 370 is performed by the switching unit 376, the padding unit 370 performs padding processing to compensate for the pixel of the reference macroblock outside the ROI region using the pixel data in the ROI region. The pixel data of the reference macroblock is given to the motion compensation unit 372. When the switching unit 376 switches to the substitute pixel acquisition unit 374, the substitute pixel acquisition unit 374 uses the pixel values of the non-ROI region at the same position or in the vicinity thereof for the reference macroblock pixels outside the ROI region. Pixel substitution processing is performed, and pixel data of the reference macroblock after pixel substitution processing is given to the motion compensation unit 372.

  Due to the action of the switching unit 376, for the reference macroblock that has been subjected to the padding process in the encoding apparatus 100 of the present embodiment, the reference macroblock that has been subjected to the padding process as in the encoding apparatus 100 is used for motion compensation. In addition, for the reference macroblock that has been subjected to the pixel substitution process in the encoding apparatus 100 of the present embodiment, the reference macroblock that has been similarly subjected to the pixel substitution process is used for motion compensation.

  According to the present embodiment, in ROI encoding, with respect to macroblocks in the ROI region, when the reference macroblock by motion prediction is out of the ROI region, padding processing is performed using pixel data in the ROI region, and Compared with the case where the pixel data in the non-ROI region is used instead of the padding process, the one with the better code efficiency can be selected.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above description, exception processing is described when a motion search reference macroblock deviates from the ROI region when a motion search is performed by providing an ROI region on a moving image. However, the present invention is called an ROI region. The present invention is not limited to area division, and can also be applied to a case where an area is divided for other factors and motion search is performed in units of areas. For example, when a picture constituting a moving image is divided into slices and encoded with different image quality in units of slices, the same exception processing may be performed when the motion search reference destination is outside the slice region. .

  Further, although the padding process has been described as an example of the exception process, exception processes other than this may be used. For example, if the macroblock of the reference destination is out of the ROI area of the reference frame, but is in the ROI area of another frame other than the reference frame, the pixel at the corresponding position in the ROI area of the other frame In this case, the pixel of the reference macroblock may be supplemented.

  The exception processing at the time of ROI encoding and motion search described in the embodiment can be similarly applied to scalable hierarchical encoding in SVC. Scalable hierarchical coding is the coding of images hierarchically with scalability. For example, encoding with different playback quality of moving images such as spatial resolution, frame rate and image quality level, and multiple playback qualities. In other words, a moving image that is scalable hierarchically encoded in this manner includes a generation of encoded data of a level, and has scalability that an arbitrary reproduction quality level can be selected and decoded. In SVC, the ROI area can be hierarchically encoded with scalability, so that the reproduction quality of the ROI area can be varied in multiple stages for decoding.

1 is a configuration diagram of an encoding apparatus according to Embodiment 1. FIG. It is a block diagram of the motion compensation prediction process part of FIG. It is a figure explaining the relationship between the object macroblock of an encoding object frame, and the reference macroblock of a reference frame. It is a figure explaining the example of a padding process. It is a figure explaining the padding process in case a reference macroblock completely remove | deviates from a ROI area | region. It is a flowchart explaining the procedure of the motion compensation prediction by the motion compensation prediction process part of FIG. 1 is a configuration diagram of a decoding device according to Embodiment 1. FIG. It is a block diagram of the motion compensation process part of FIG. FIG. 6 is a configuration diagram of an encoding apparatus according to Embodiment 2. It is a block diagram of the motion compensation prediction process part of FIG. 6 is a configuration diagram of a decoding apparatus according to Embodiment 2. FIG. It is a block diagram of the motion compensation process part of FIG.

Explanation of symbols

  10 block generation unit, 20 DCT unit, 30 quantization unit, 40 inverse quantization unit, 50 inverse DCT unit, 60 motion compensation prediction processing unit, 62 ROI initial setting unit, 64 ROI tracking unit, 66 motion search unit, 67 ROI Information holding unit, 68 ROI inside / outside determination unit, 70 padding unit, 72 motion prediction unit, 74 substitute pixel acquisition unit, 80 frame buffer, 90 variable length encoding unit, 92 code amount determination unit, 100 encoding device, 300 decoding device 310 variable length decoding unit, 320 inverse quantization unit, 330 inverse DCT unit, 360 motion compensation processing unit, 364 ROI acquisition unit, 366 motion identification unit, 367 ROI information holding unit, 368 ROI inside / outside determination unit, 370 padding unit, 372 motion compensation unit, 374 substitute pixel acquisition unit, 376 off Department, 380 frame buffer, 390 switching flag obtaining unit.

Claims (6)

  Whether an independently encoded region is provided on a picture constituting a moving image, and whether a reference block in the reference picture when the motion search is performed on the encoding target block in the region in the encoding target picture is in the region in the reference picture And if at least a part of the reference block is outside the region in the reference picture, the reference block outside the region using the pixel data of the reference block in the region A coding method characterized by performing motion prediction after generating the pixels.   The encoding method according to claim 1, wherein the independently encoded regions are encoded with a quality different from other regions.   Track movement between pictures of the independently encoded region, determine whether the reference block is in the region in the tracked reference picture, and determine whether the reference block in the tracked reference picture The encoding method according to claim 1 or 2, wherein position information of a region is included in encoded data of the moving image.   If the entire reference block is outside the independently encoded region in the reference picture, the pixel data in the independently encoded region in the reference picture that is closest to the reference block is used. 4. The encoding method according to claim 1, wherein a pixel of the reference block is generated.   When at least a part of the reference block is outside the independently encoded region in the reference picture, the pixel data of the reference block outside the region is used in the reference picture. A first reference block generated by generating a pixel of the reference block and a pixel of the reference block outside the area are pixels at a corresponding position outside the area in the reference picture. Selecting one of the second reference destination blocks generated by substituting to perform motion prediction, and including information for identifying the selected reference destination block in the encoded data of the moving image The encoding method according to any one of claims 1 to 4.   The code amount of motion prediction encoding by the first reference block is compared with the code amount of motion prediction encoding by the second reference block, and the reference block with the smaller code amount is selected. The encoding method according to claim 5, wherein:

Images