JP5998862B2 - Moving image processing device - Google Patents

Description

  The present invention relates to a moving image processing apparatus.

  The moving image processing apparatus encodes moving image data constituting a video and compresses the data, and decodes the compressed stream data to generate original moving image data. In MPEG2, MPEG4, H.264, etc., which are widely used as video compression, the encoder uses the motion vector data based on the correlation with the past image and the future image in time. A difference image between an image generated with a motion vector and an original image, or an uncorrelated image that cannot be generated with a motion vector is converted into data subjected to DCT processing, quantization processing, and compression processing. The compressed stream data is transmitted via a transmission medium or stored in a storage medium. The decoder decodes the stream data and generates original moving image data.

  High-definition video encoding and decoding technologies are already widely used in terrestrial digital broadcasting and BS digital broadcasting. High-definition images have a resolution of 2K1K (1920 x 1080), and encoders and decoders that have already been widely used also process moving image data of this resolution.

  On the other hand, as next-generation high-resolution moving image data, video formats having 4K2K and 8K4K resolutions in which the resolution of 2K1K high-definition images is four times or sixteen times have been proposed and are being developed.

  For such 4K2K and 8K4K ultra-high resolution video data, instead of expensive dedicated encoders and decoders, multiple existing high-definition video encoders and decoders are provided to divide the screen into 2K1K resolution areas. Thus, it has been proposed that each encoder and decoder share the processing. With such a configuration, a moving image processing apparatus with high cost performance can be realized. For example, it is described in the following patent documents.

Japanese Unexamined Patent Publication No. 2007-67499 JP 11-196299 A JP 2007-108447 JP 2009-294273 A

  However, if one screen is divided into a plurality of image areas and a plurality of encoders and decoders process each of the plurality of divided images, the image quality of the peripheral area of the divided 2K1K divided image deteriorates. Furthermore, since each encoder encodes each divided image according to the image content, the image quality on both sides of the boundary line of the decoded divided image is different. Therefore, when the decoder synthesizes the decoded images of the divided images, the boundary lines thereof become conspicuous and the quality of the synthesized image is degraded.

  Patent Document 1 proposes that one screen is divided into a plurality of image regions including an overlap region, encoded by a plurality of encoders, and decoded by a plurality of decoders. Then, the decoder discards the overlap area in the peripheral area where the image quality of the decoded divided image is degraded, and avoids the degradation of the image quality of the boundary line in the composite image.

  However, since the encoding process in the adjacent divided images is different depending on each image, the image quality at the boundary portion of the decoded adjacent divided image is different, and the boundary portion of the synthesized image becomes unnatural. In other words, if the image quality of one divided image is high but the image quality of the other divided image is low, the unnaturalness of the boundary portion of the composite image appears.

  Accordingly, an object of the present invention is to provide a moving image processing apparatus that suppresses deterioration in image quality at the boundary portion between adjacent divided images.

Therefore, the first aspect of the moving image processing apparatus includes a plurality of encoders that generate encoded divided image data by encoding a plurality of divided images obtained by dividing a moving image so as to have overlapping regions. An encoder unit,
A plurality of decoders that respectively decode the plurality of encoded divided image data from the data input from the encoder unit and extract motion vector information, and a plurality of decoded divided images generated by the plurality of decoders respectively decoding A decoder unit having a synthesizing unit for blending in the overlap region and outputting an image of the moving image,
The synthesizing unit determines a blend ratio of the overlap region based on the motion vector information output from each of the plurality of decoders.

  According to the first aspect, it is possible to improve the deterioration of the image quality at the boundary line of the composite image.

It is a block diagram of a moving image processing apparatus. It is a figure which shows the structural example of an encoder. It is a figure which shows the structural example of a decoder. It is a figure explaining the 1st cause from which a picture quality differs in a boundary part. It is a figure explaining the 2nd cause from which a picture quality differs in a boundary part. It is a figure explaining the encoding and decoding process in this Embodiment. It is a figure explaining the encoding and decoding process in this Embodiment. It is a figure which shows the example of the blend rate in the overlap area | region in this Embodiment. It is a figure explaining the setting of the blend rate of FIG. It is a figure explaining the image composition process in the encoder unit in this Embodiment. It is a figure explaining the image composition process in the encoder unit in this Embodiment. It is a figure which shows the example of the variable control of a blend rate. It is a block diagram of the moving image processing apparatus in this Embodiment. 3 is a diagram illustrating a configuration example of an image composition unit 24. FIG. It is a block diagram of the moving image processing apparatus in 2nd Embodiment. It is a figure which shows the relationship between the bit rate and overlap width OLW in 2nd Embodiment.

  FIG. 1 is a configuration diagram of a moving image processing apparatus. The resolution of the input image constituting the moving image is, for example, 4K2K, which is four times the resolution of the high-definition image (2K1K). Therefore, the moving image processing apparatus of FIG. 1 includes an image dividing unit 10 that divides input image data into four divided images A, B, C, and D, and four encoders that respectively encode the divided image data. The encoder unit EN includes an encoder group 12 and a stream synthesizing unit 14 that synthesizes the encoded stream data.

  Further, the moving image processing apparatus of FIG. 1 receives the stream data 16 generated by the encoder unit EN and separates it into stream data of divided images A, B, C, and D, and a stream of each divided image. Including a decoder group 22 including four decoders that generate divided image data A, B, C, and D and an image composition unit 24 that combines the divided image data and outputs a 4K2K output image Have a unit.

  By using the four encoders of the encoder group 12 and the four decoders of the decoder group 22 that are already widely used for high-definition images, the moving image processing apparatus can be configured at low cost.

  FIG. 2 is a diagram illustrating a configuration example of the encoder. This encoder has a configuration corresponding to a standard such as MPEG2. First, current image data P1 of a current frame constituting a moving image is input and stored in the frame memory 122 via the memory control unit 121. In the frame memory 122, for example, reference image data P2 of the previous frame is stored. Then, the encoder performs an encoding process on the current image data P1 of the current frame, for example, in units of macroblocks of 16 × 16 pixels.

  First, the motion prediction unit 123 searches for a corresponding image in the reference image data P2 for a macroblock image in the current image data P1, and outputs a motion vector MV when it detects it. Further, the motion compensation unit 124 generates the image data P3 of the motion compensation frame from the reference image data P2 and the motion vector MV.

  Next, the difference calculation unit 125 calculates a difference between the image data P3 of the motion compensation frame and the current image data P1 of the current frame, and generates prediction error image data P4. This prediction error image data P4 is image data of the difference between the image data P3 that can be reproduced by the reference image and the motion vector MV and the current image data P1.

  The selector 126 selects the prediction error image data P4 when the motion vector MV is detected, and the current image data P1 when it is not detected. Therefore, the output image data P5 of the selector 126 is the pixel data of the prediction error image data P4 or the current image data P1, and is 16 × 16 pixel spatial region data.

  Next, the orthogonal transform unit 127 performs DCT processing in units of blocks of 8 × 8 pixels, for example, and transforms the spatial domain data P5 into frequency domain data P6. The frequency domain data P6 is composed of a DC component and coefficients of frequency components from low frequency to high frequency, and is 8 × 8 = 16 coefficient data.

  Then, the quantization unit 128 divides the frequency domain data P6 by an 8 × 8 division matrix, and further divides by a single Q value common to 16 pixels. The quantization matrix is obtained by reducing the value for the DC component and the low-frequency component and increasing the value for the high-frequency component, and is invariant within one frame. When divided by the division matrix, the data amount of the direct current component and the low frequency component does not decrease so much, while the data amount of the high frequency component greatly decreases. On the other hand, the Q value is variably controlled for each macroblock (or for each block that is a DCT processing unit) within one frame, and the stream data after quantization and encoding is controlled to be within a certain bit rate. The

  The encoding unit 129 compresses the quantized data P7 and the motion vector MV data, for example, by run-length encoding, and outputs stream data. As a result, when the motion vector MV is detected, the stream data becomes data obtained by compressing the motion vector MV and the data P7 obtained by orthogonally transforming and quantizing the prediction error data P4, while the motion vector MV is not detected. In this case, the data P7 is obtained by compressing the data P7 obtained by orthogonal transform and quantizing the current image data P1.

  Therefore, when the motion vector MV is detected, the data amount of the spatial domain data P5 is small, and the Q value corresponding to the quantization rate can be reduced even under the constraint that the stream data is within a certain bit rate. Therefore, the image after decoding tends to have high image quality. On the other hand, when the motion vector MV is not detected, the data amount of the spatial domain data P5 is large, and the Q value has to be increased under the above-mentioned restrictions, and the image quality after decoding tends to be low.

  The encoder of FIG. 2 further includes an inverse quantization unit 130 that multiplies the Q value and the value of the division matrix, an inverse orthogonal transform unit 131 that inversely converts frequency domain data into spatial domain data, and spatial domain data P8 output from the inverse orthogonal transform unit 131. Are provided with a difference adding unit 132 for adding the image data P3 of the motion compensation frame and a selector 133, and these circuits perform local decoding processing. When the motion vector MV is detected, the selector 133 selects the output data of the difference adding unit 132. When the motion vector is not detected, the selector 133 selects the spatial region data P8 and selects the local decoded image, that is, Image data P9 used as a reference image in the subsequent encoding process is output. The output image data P9 of the selector 133 is stored in the frame memory 122 as reference image data.

  FIG. 3 is a diagram illustrating a configuration example of a decoder. The decoder also performs processing in units of macroblocks. First, the decoding unit 221 inputs stream data and performs run-length decoding to obtain a motion vector MV and the prediction error data P4 in FIG. 2 or data P17 corresponding to the original image data P1 when there is no motion vector. Output. Then, the motion compensation unit 222 generates motion compensation frame image data P13 from the reference image data P2 and the motion vector MV stored in the frame memory 228. In addition, the inverse quantization unit 223 multiplies the data P17 by the Q value received from the encoder and the fixed value division matrix, and the inverse orthogonal transform unit 224 performs inverse DCT transform to convert the frequency domain data into a spatial frequency. Convert to data P18.

  Further, the difference adding unit 225 adds the image data P13 of the motion compensation frame and the spatial frequency data P18 that is prediction error data. The selector 226 selects the image data output from the difference adding unit 225 when the motion vector MV is detected, and selects the spatial frequency data P18 when the motion vector MV is not detected, and decodes the image data. P19 is output. The decoded image data P19 is stored in the frame memory 228 by the memory control unit 227 and output as an encoded image output.

  Next, a description will be given of image quality degradation when encoding and decoding are performed for each divided image obtained by dividing an ultra-high resolution image.

  In a divided image, when filtering such as blurring the image or conversely enhancing an edge is performed, the pixels are cut off at the division boundary, and thus cannot be appropriately filtered. This degrades the image quality. In addition to this cause, first, the image of the boundary part of adjacent divided images has different image quality on both sides of the boundary line due to the difference in the Q value of quantization processing between adjacent divided images. . Second, when motion vectors are detected or not detected on both sides of the boundary line between adjacent divided images, the image quality on the side where the motion vector is detected is high, and the side on which no motion vector is detected is detected. Image quality is low.

  FIG. 4 is a diagram for explaining a first cause that the image quality is different at the boundary. FIG. 4 shows four divided images A, B, C, and D by dividing a 4K2K ultra-high resolution image into four equal parts. The stationary object 30 is on the boundary line between the divided images A and B. In this case, when the divided images A and B are encoded, no motion vector is detected in any of the divided images, and both are encoded by intra prediction (orthogonal transformation, quantization processing, encoding processing) or other than the stationary object 30. A motion vector is detected for each of the images, and the prediction error image is encoded by intra prediction.

  In this case, different frequency domain data is generated by orthogonally transforming different images in the divided images A and B. Therefore, the Q value in the quantization process for the frequency domain data on both sides of the boundary between the divided images A and B may be different in the subsequent quantization process. As a result, when the Q value is small, the image quality is high, and when the Q value is large, the image quality is low. For example, in the divided image A, the data amount of the coefficient of the high-frequency component in the lower right part close to the boundary is small and the Q value can be reduced, while in the divided image B, the low middle frequency component in the lower left part close to the boundary is high. Since the data amount of the coefficient is large and the Q value can be increased, the image quality becomes low.

  As a result, in the decoded divided image, a high-quality area and a low-quality area exist on both sides of the boundary, and the boundary is expected to be conspicuous in the composite image.

  In this way, in moving image compression (encoding) processing such as MPEG2, if the efficiency of motion prediction is excluded (that is, when the efficiency of motion prediction is the same for both divided images A and B), the encoded image quality is orthogonal. The coefficient and Q value of the division matrix in the quantization processing after conversion are dominant. The coefficient of the division matrix is usually fixed within one frame, but the Q value is variably controlled depending on the type of image in order to keep the bit rate of the stream data within a constant value. For this reason, even when there is no difference in motion compensation between the two divided images A and B, the image quality is different on both sides of the boundary.

  FIG. 5 is a diagram for explaining a second cause in which the image quality is different at the boundary. FIG. 5 shows four divided images A, B, C, and D obtained by dividing a 4K2K ultra-high resolution image into four equal parts. Then, there is an object 30 that moves in the direction from the divided image A to B on the boundary line between the divided images A and B. That is, the object 30 exists only in the divided image A at time t, and the object 30 moves to the boundary region between the divided images A and B at the next frame time t + 1.

  In this case, when the divided images A and B at time t + 1 are encoded, a motion vector is detected in the macroblock in the boundary area in the divided image A, and a motion vector is not detected in the divided image B because the object 30 first appears at time t + 1. . As a result, the boundary area of the divided image B with poor motion prediction efficiency (no motion vector is detected) has a larger amount of data before quantization than the boundary area of the divided area A with high motion prediction efficiency. .

  Therefore, in order to make the amount of data after quantization equal between the divided areas A and B, the Q value in the divided image B is controlled to be larger than the Q value in the divided image A, and the vicinity of the boundary of the divided image B is It is expected that the image quality is low near the boundary of the low-quality image and divided image A.

  In Patent Document 1, an overlap region is provided near the boundary, and the data of the overlap region is discarded from the decoded image data. However, no matter how much it is discarded, it is impossible to eliminate the difference in image quality in the vicinity of the boundary when the images are combined, and the effect of suppressing the deterioration in image quality is limited.

  6 and 7 are diagrams for explaining the encoding and decoding processing in the present embodiment. According to this process, in the encoding process, for example, an input image that is a 4K2K ultra-high resolution image is divided into four divided images A, B, C, and D so as to have an overlap region OL that crosses the boundary line. The Then, the four divided images are encoded by the four encoders.

  Next, in the decoding process, the encoded stream data is separated into four divided image streams and decoded by four decoders. Then, except for the overlap region OL, the decoded output image data (pixel data) of each divided image becomes the image data of the composite image as it is. On the other hand, the overlap region OL is divided into three regions as follows, and the decoded output image data of both divided images is blended at each blend rate.

  In other words, the overlap area OL of the decoded output image A is changed from the periphery to the inside, the area A3 where the blend rate is zero, the area A2 where the blend rate gradually increases from zero to the maximum value, and the maximum blend rate. Classify with area A1. Other decoded output images B, C, and D are classified in the same manner.

  In the overlapping areas of the divided images A and B, the areas A1 and B3 are blended at a blend rate of 100% and 0%, and the areas A2 and B2 are changed from 0% to 100% (or from 100% to 0%). Blend processing is performed at a gradually changing blend rate, and regions A3 and B1 are blended at a blend rate of 0% and 100%. As a result of the blending process, an unnatural image due to the difference in image quality is improved near the boundary of the synthesized output image.

  FIG. 8 is a diagram showing an example of the blend ratio in the overlap region in the present embodiment. The blend ratios αa1 and αb1 indicated by solid lines in FIG. 8 will be described. The blend ratio of the pixels of the divided image A is as follows: the area A3 where the blend ratio is 0%, the area A2 where the blend ratio gradually changes from 0% to 100%, It is divided into the maximum 100% area A3. The blend ratio of the pixels of the divided image B is also the same as that of the divided image A although it is reversed left and right.

When the divided images A and B are combined, in the overlap region OL, the gradation values of the pixels at the same position in the regions A1 and B3 are blended at a blend rate αa1 = 100% and αb1 = 0%. This blending operation is, for example, an operation for obtaining the gradation value PAB of the pixel after the blending process by performing the following operation on the gradation values PA and PB of the pixels at the same position in the divided images A and B.
PAB = PA * αa1 + PB * αb1
With this blend ratio αa1 = 100% and αb1 = 0%, the pixel data in the region B3 is substantially discarded, and the pixel data in the region A1 is adopted as it is. This is because the region B3 is a peripheral region of the divided image B, and the image quality is deteriorated due to discontinuity of pixels. On the other hand, the area A1 is an internal area of the divided image A, and the pixels are continuous and the image quality is not deteriorated.

  Furthermore, in the overlap region OL, the gradation values of the pixels at the same positions in the regions A2 and B2 are blended ratios αa1 (changed from 0% to 100%) and αb1 (changed from 100% to 0%). Blended. Thereby, even if the image quality differs between the divided images A and B of the decoded output image, it is possible to improve the unnaturalness due to the image quality difference near the boundary of the composite image.

  In the overlap region OL, the gradation values of the pixels at the same positions in the regions A3 and B1 are blended with the blend ratios αa1 = 0% and αb1 = 100%. Due to this blend ratio, the pixel data of the area A3 is discarded in the embodiment, and the pixel data of the area B1 is adopted as it is.

  The area A3, B3 where the pixel data is discarded is preferably set to, for example, at least 16 pixels for one micro block. If these areas A3 and B3 are widened, it is desirable that the pixel data with low image quality is discarded as much as possible, but conversely, the overlap area is expanded accordingly, and the peripheral pixels of the composite image are appropriately set due to the resolution restrictions of the encoder and decoder. It will not be encoded and decoded.

  In the present embodiment, the blend ratio is controlled to an optimum value according to the efficiency of motion prediction of the divided images A and B in the overlap region OL. That is, in the blend regions A2 and B2, the region with a higher blend rate than the adjacent side of the adjacent encoded divided images has higher motion prediction efficiency in the blend regions A2 and B2 in the overlap region (more motion vectors) The divided image side is set wider than the divided image side where the motion prediction efficiency is lower (the motion vector is smaller).

  In other words, compared to the case where the motion prediction efficiency in the blended regions A2 and B2 is the same for the left and right divided images, the motion prediction efficiency is higher (the number of motion vectors is higher) and the blend ratio on the divided image side is higher. It is set and variably controlled so that the blend rate on the divided image side is set lower, with the motion prediction efficiency being lower (the motion vector being smaller).

  In FIG. 8, the blending rates αa2 and αb2 when the efficiency of motion prediction is higher on the divided image A side than on the divided image B are indicated by broken lines. When the motion prediction efficiencies in the blend regions A2 and B2 are equal, blend rates αa1 and αb1 are set. That is, the region where the blend ratio between the divided images A and B is higher than the adjacent side is set wider on the divided image A side and narrower on the divided image B side.

  In other words, the blend rate αa2 on the divided image A side is set higher and the blend rate αb2 on the divided image B side is set lower than the blend rates αa1 and αb1 of the solid lines.

  As in the example shown in FIG. 5, when there is an object from the divided image A toward the boundary, a motion vector is detected in the divided image A, the motion prediction efficiency is increased, and a high image quality of the decoded output image can be expected. . On the other hand, in the divided image B, no motion vector is detected, the efficiency of motion prediction becomes low, and the image quality of the decoded output image is expected to deteriorate.

Therefore, in FIG. 8, the region A1 with a blend rate of 100% is expanded to the right (the divided image B side), and the region A2 near the boundary of the divided image A is narrowed. Accordingly, the area B3 with a blend rate of 0% in the divided image B is expanded to the right side, and the area B2 is also narrowed. As a result, for the divided region A, when the blend rate αa1 of the solid line near the boundary and the blend rate αa2 of the broken line are compared,
αa1 <αa2
It has become. On the other hand, for the divided region B, when the blend rate αb1 of the solid line near the boundary and the blend rate αb2 of the broken line are compared,
αb1> αb2
It has become.

  FIG. 9 is a diagram for explaining the setting of the blend ratio in FIG. In FIG. 9, (A) the motion vector information determines that there is little motion in the left and right divided images (when the motion prediction efficiency is the same), and (B) the motion vector information indicates that the object on the left Region A1, A2, A3, B1, B2, B3 in left and right divided images A, B, and C, D when moving from right to right (when the efficiency of motion prediction is higher on the left side than on the right side) C1, C2, C3, D1, D2, and D3 are shown, respectively.

  (A) When the motion prediction is the same, for example, for the divided images A and B, the area sizes are set to A1 = B1, A2 = B2, and A3 = B3. Therefore, a region having a higher blend ratio than the adjacent side in the overlapping region between the divided images A and B (part of the regions A1 and A2 in the divided image A, part of the regions B1 and B2 in the divided image B) However, it is set to the same size as the adjacent side.

  (B) If divided image A is more efficient in motion prediction than divided image B (the number of motion vectors when moving from left to right), A1> B1 and A3 <B3 are set. Yes. Therefore, a region having a higher blend ratio than the adjacent side in the overlap region between the divided images A and B (partial region A1 and region A2 in the divided image A, partial region B1 and region B2 in the divided image B ) Is set wider than the adjacent side.

  10 and 11 are diagrams for explaining image composition processing in the encoder unit according to the present embodiment. FIG. 10 shows an image at time t and an image at time t + 1. At time t, there is an object 30 that moves from left to right in the divided image A, and there is no object 30 in the divided image B. On the other hand, at time t + 1, the object 30 moving from left to right moves into the overlap area OL between the two divided areas A and B.

  In such a case, a motion vector is generated for the object 30 in the divided image A at time t + 1, but in the divided image B at time t + 1, the object 30 exists in the divided image B at time t. Since there was no motion vector, no motion vector is generated for the object 30. Therefore, in the image at time t + 1, in the divided image A, since the object 30 is encoded by the motion vector for the object 30 in the reference image at time t and the object 30 is encoded by inter prediction, the efficiency of motion prediction is high. On the other hand, in the divided image B, the object 30 does not exist in the reference image at time t, and no motion vector is generated, and the object 30 is encoded by intra prediction, so that the motion prediction efficiency is low.

  In FIG. 11, at time t + 1, the object 30 moves from left to right in the overlap region OL of the divided images A and B. The encoder unit searches for motion vectors in units of micro blocks MB. In the example of FIG. 11, on the divided image A side, motion vectors (arrows in the figure) for the object 30 are detected by MB lines 1, 2, and 3. However, in all the macroblocks MB of MB lines 0 and 4 and the macroblock MB in which the object 30 in the MB lines 1, 2 and 3 does not exist, for example, the background image remains unchanged, and therefore a motion vector of 0 No motion) is detected.

  On the other hand, on the divided image B side, a motion vector due to the motion of the object 30 is not detected. Then, in the macroblock MB in which the object 30 does not exist, since the background image remains unchanged as described above, a zero motion vector (no motion) is detected, and the macroblock MB (MB) in which only the object 30 exists. In 2 MB) of line 2, no motion vector of 0 is detected and only intra prediction (I) is performed. In macroblock MB (0 ′) in which object 30 exists in a partial area, motion vector of 0 and object There are 30 intra predictions in the region.

  In this way, in the overlap region OL, it is desirable that the motion compensation comparison (motion vector comparison) in the left and right divided images A and B is performed in units of rows of the macroblock MB, which is a unit for searching for motion vectors. . That is, for each of a plurality of macroblocks MB in the same macroblock MB row, the left and right motion vectors or the left and right motion vectors from the left and right of the macroblocks MB at the same position in the left and right divided images A and B Compare the components. Then, the blend rate is adjusted in accordance with the total of the comparison results in units of rows of the macroblock MB. Note that the overlap region OL between the upper and lower divided images A and C is performed in units of columns of the macroblock MB.

  As shown in the blend ratio of the divided images A and B shown in FIG. 11, in the MB lines 0 and 4, no motion vector in the left to right direction is detected in the left and right divided images A and B. Therefore, in the overlap region OL having a lateral width of 4 MB, the blend rate changes from 100% to 0% or from 0% to 100% in the middle 2 MB.

  On the other hand, in MB lines 1, 2, and 3, a motion vector from the left to the right is detected in the left divided image A, but not detected in the right divided image B. Therefore, in the overlap area OL, on the divided image A side, the blend rate is set to 100% with the two MBs on the left side, and the blend rate is reduced from 100% to 0% with the one MB on the right side. The blend rate is set to 0% for one MB. On the other hand, on the divided image B side, the blend rate is set to 100% for the leftmost one MB in the overlap region OL, and the blend rate is reduced from 100% to 0% with one MB on the right side. The blend rate is set to 0% in the two rightmost MBs.

  In this way, the blend rate is variably controlled according to the difference in the efficiency of motion compensation between the left and right divided images A and B for each row of the macroblock MB. A region where the blend ratio on the divided image A side in the MB lines 1, 2 and 3 is higher than that on the B side is set wider than that in the MB lines 0 and 4. In other words, the region where the blend rate on the divided image A side in the MB lines 1, 2 and 3 is higher than that on the B side is set wider than the region where the blend rate on the divided image B side is higher than that on the A side.

  FIG. 12 is a diagram illustrating an example of variable control of the blend ratio. FIG. 12 shows an example of blend ratios in the regions A1, B3, A2, B2, A3, and B1 of the divided images A and B. In FIG. 12, the blend rates 100%, 50%, and 0% shown at the left end are the blend rates for the divided image A, and the blend rates 100%, 50%, and 0% shown at the right end are the blend rates for the divided image B. Rate.

  The example of FIG. 12A is the same as the blend rate of MB lines 0 and 4 in FIG. That is, the area having a higher blend rate than the divided image B in the divided image A is the same area as the same area in the divided image B. The example of FIG. 12 (2) is the same as the blend rate of the MB lines 1, 2, and 3 in FIG. That is, the boundary (the boundary between A1 and A2) that decreases from 100% in the divided image A is moved to the right as indicated by the arrow. As a result, a region having a higher blend rate than the divided image B in the divided image A is wider than the same region in the divided image B.

  As a modification of the example of FIG. 12 (2), (3) and (4) are shown. In the example of FIG. 12 (3), the boundary between the area A2 where the blend rate in the divided image A decreases and the area A3 where the blend rate is 0% is moved to the right as shown by the arrow. However, in this example, the low-quality area A3 of the divided image A in (1) is blended. In the example of FIG. 12 (4), the boundaries of the areas A1 and A2 and the boundaries of the areas A2 and A3 in the divided image A are both moved to the right as indicated by arrows. 12 (2), (3), and (4), the area of the region having a higher blend ratio than the divided image B in the divided image A is larger than the area of the same region in the divided image B. Yes.

  FIG. 13 is a configuration diagram of the moving image processing apparatus according to the present embodiment. The encoding unit EN includes, for example, an image dividing unit 10 that divides a 4K2K input image into divided images including a 2K1K overlap area, and an encoder group that includes four encoders that respectively encode the divided images A, B, C, and D. 12, a stream synthesizing unit 14 that synthesizes the encoded divided image data and outputs stream data, and a control unit 16 that controls the encoder group 12 and the like based on input setting information. The configuration of the encoder in the encoder group 12 is as described in FIG.

  The setting information includes an overall bit rate that is the number of bits per unit time of the stream data. And the control part 16 sets the bit rate BR in each encoder calculated | required from the input whole bit rate to each encoder. As described with reference to FIG. 2, each encoder controls the Q value in the quantization process so that it is within the set bit rate BR. The Q value is normally variably controlled for each macroblock MB. Further, the Q value is controlled based on the amount of data to be quantized and the set bit rate BR. For example, if the data amount is the same, the Q value can be controlled to be small if the bit rate BR is large. In this case, the image quality can be increased, but the Q value must be controlled to be large if the bit rate BR is small. In that case, the image quality will be low.

  As described above, the stream data includes a bit stream obtained by compressing motion vectors, quantized data of prediction error image data, Q values, and the like in addition to attribute data of I picture, P picture, and Q picture.

  On the other hand, the decoding unit DE includes a stream separating unit 20 that separates stream data into data of divided images A, B, C, and D, a decoder group 22 including four decoders that respectively decode the separated stream data, and a decoding unit An image composition unit 24 that synthesizes the divided images and a control unit 25 that controls the entire decoder are included.

  In the present embodiment, the decoders and the image synthesis unit operate in synchronization with the VSYNC and VCLK signals from the control unit 25, and the image control unit 24 for synthesizing the decoded divided images generated by the decoders includes: Based on the information of the motion vector MV input from each decoder, the blending rate of the overlap area of the decoded divided image is variably set and the synthesis process is performed.

  FIG. 14 is a diagram illustrating a configuration example of the image composition unit 24. The image synthesis unit 24 includes a synthesis unit 24i that synthesizes pixel data of both divided images corresponding to the same pixel in the synthesized image based on the blend rate α. The synthesis unit 24i that synthesizes the divided images A, B, C, and D multiplies the pixel data Ai, Bi, Ci, and Di of the divided images corresponding to the same pixel by the respective blend rates αai, αbi, αci, and αdi. And an adder (+) for adding the multiplication results and outputting the combined pixel data Xi. Further, the image composition unit 24 includes a composition control unit 240. The synthesizing control unit 240 operates in synchronization with the decoder unit DE by inputting VSYNC and VCLK signals, recognizes the pixel position in the screen, and moves the motion vectors MVA, MVb of the pixel position and the divided images A, B, C, D respectively. Then, blend ratios αai, αbi, αci and αdi are determined from MVc and Mvb. For example, within the overlap region of two divided images to be synthesized at the A and B boundaries, MVA and MVb from the row unit decoder unit of the macroblock MB, and the left and right motion vectors of both divided images A and B The motion vector components corresponding to the object heading (or from the right side to the left side) are compared, and the blending ratios αai and αbi are controlled according to the comparison result.

  The motion vector comparison method is, for example, when K macroblocks MB are included in each MB row in the overlap region, and the left side for each corresponding macroblock MB in both divided images A and B. Compare the motion vector components corresponding to the object from right to left (or from right to left), and the total of the K comparison results is the divided image with the higher motion compensation efficiency of both divided images A and B To decide.

  The control method of the blend rate according to the comparison result is that, if the efficiency of motion compensation is the same, the area where the blend rate is higher than the adjacent side in both divided images is made the same size on both sides, and the motion compensation efficiency is increased. If the difference is different, for the region where the blend ratio is higher than that of the adjacent side, the method in which the motion compensation efficiency is higher is made wider.

  In the above embodiment, the region of the higher blend rate than the adjacent side is widened for the divided image having the higher motion compensation efficiency. However, as a modification, in addition to or instead of comparing the efficiency of motion compensation, a region with a higher blend rate than the adjacent side is broadened for a divided image having a smaller Q value during quantization processing. You may make it do.

  That is, the quantization process is performed on prediction error image data (difference image data between an original image and a motion compensated image) that cannot be generated by motion compensation. Based on the bit rate and the like, a Q value that is a divisor at the time of quantization processing is variably set for each macroblock. In general, when the Q value increases, the image quality decreases, and when the Q value decreases, the image quality increases. Moreover, this Q value is included in the stream data and transmitted from the encoder side to the decoder side.

  Therefore, the image composition unit 24 in the decoder unit determines the quality of the decoded image quality of both divided images based on the Q value for each row of the macroblock MB in the overlap region, Therefore, it is preferable to control so as to increase the region having a high blend ratio. In particular, when the motion compensation efficiency of both divided images is comparable, the decoded image with higher image quality can be reflected in the composite image by variably controlling the blend rate based on the Q value in the overlap region. And the image quality of the composite image can be improved.

[Second Embodiment]
FIG. 15 is a configuration diagram of a moving image processing apparatus according to the second embodiment. 15 differs from FIG. 13 in that (1) the control unit 16 sets the overlap area width OLW of the separated image according to the information on the entire bit rate input to the control unit 16 of the encoder unit EN. , Setting in the image dividing unit 10, (2) the stream synthesizing unit 14 including the set overlap area width OLW in the stream data, and (3) in the decoder unit DE, the image synthesizing unit 24 is to synthesize the decoded divided image in accordance with the overlap width OLW. The image composition unit 24 further controls the blend rate α for composition according to the overlap width OLW.

  FIG. 16 is a diagram illustrating the relationship between the bit rate and the overlap width OLW in the second embodiment. In the encoder unit EN, the overall bit rate for the stream data is set. The overall bit rate is the amount of stream data per unit time. If the overall bit rate is higher, the Q value in the quantization process can be set smaller, so it is expected that the decoded image will be less degraded. Therefore, the boundary line of the divided image is not noticeable. Conversely, if the overall bit rate is lower, the Q value must be set higher so that it fits within the bit rate, resulting in more degradation of the decoded image and making the borders of the split image more noticeable .

  Therefore, it is desirable that the width OLW of the overlap region is narrowed when the overall bit rate is higher and wider when it is lower. That is, the control unit 16 in the encoder unit EN variably controls the width OLW of the overlap area according to the set overall bit rate.

  When the overall bit rate is low and the image quality is expected to deteriorate, the image quality of the decoded divided images on both sides deteriorates, and the image quality difference tends to increase. This tendency is more pronounced for moving images than for still images. That is, in the moving image, as shown in FIG. 10, when the moving object straddles between the divided images, motion compensation is performed on the divided image A side, and the data amount of the prediction error image is small, while the divided image B side No motion compensation is performed, and the data amount of the prediction error image increases. In this way, when there is a difference in the data amount of the prediction error image to be quantized, if the overall bit rate is low, the image quality degradation on the divided image B side where the data amount of the prediction error image is large is reduced. The amount of data is significantly smaller than that of the divided image A side.

  In such a case, it is particularly preferable to set the overlap region width OLW wider to suppress the influence on the divided image B side where the image quality is severely deteriorated. In the first embodiment, the blend rate is changed and controlled based on the efficiency of motion compensation, but the blending is limited to the overlap region. Therefore, when the image quality is degraded beyond the overlap region, changing the blend rate is not sufficient to suppress the degradation of the image quality.

  Therefore, in the second embodiment, the above-described image quality deterioration can be more appropriately suppressed by variably controlling the width of the overlap area according to the set overall bit rate on the encoding unit side. . For example, when the overall bit rate is higher, the overlap area is set to the minimum blend rate 0% area A3 as 1 micro block MB, the blend area A2 as 1 micro block MB, and the overlap area width is set to 2 MB. It is preferable to do this.

  As described above, according to the present embodiment, the blend ratio when combining the decoded divided images is determined based on the motion compensation efficiency of both divided images. , Make it wider than the split image on the opposite side. As a result, even if there is a difference in image quality between both decoded divided images based on the efficiency of motion compensation, it is possible to suppress deterioration in image quality at the boundary portion of the composite image by optimally selecting the blend rate.

  The above embodiment is summarized as follows.

(Appendix 1)
An encoder unit having a plurality of encoders for encoding encoded divided image data by encoding a plurality of divided images obtained by dividing a moving image so as to have overlapping regions;
A plurality of decoders that respectively decode the plurality of encoded divided image data from the data input from the encoder unit and extract motion vector information, and a plurality of decoded divided images generated by the plurality of decoders respectively decoding A decoder unit having a synthesizing unit for blending in the overlap region and outputting an image of the moving image,
The moving image processing apparatus, wherein the synthesizing unit determines a blend ratio of the overlap region based on information on the motion vector output from each of the plurality of decoders.

(Appendix 2)
In Appendix 1,
The moving image processing apparatus according to claim 1, wherein the blend ratio is higher as the number of motion vectors in the overlap region is larger.

(Appendix 3)
The moving image processing apparatus according to Supplementary Note 2, wherein the motion vector that determines the blend ratio is a motion vector having a component in a direction going out from the overlap region to one of the divided images.

(Appendix 4)
In any one of Supplementary Notes 1, 2, and 3,
The synthesizing unit has a higher blend ratio on the divided image side where the efficiency of the motion prediction is higher than that in the case where the efficiency of motion prediction in the overlap region in the adjacent encoded divided image data is equal, A moving image processing apparatus in which a blend ratio on a lower divided image side is set lower.

(Appendix 5)
In Appendix 1 or 2,
From the periphery of the decoded divided image to the inside, the overlap region of the decoded divided region includes a first region where the blend rate is zero, a second region where the blend rate gradually increases, and the first region where the blend rate is maximum. A moving image processing apparatus having three regions.

(Appendix 6)
In Appendix 5,
The third region is a moving image processing apparatus in which the third region is wider when the number of motion vectors is larger than when the number of motion vectors is smaller than that of the adjacent divided images.

(Appendix 7)
In Appendix 6,
The moving image processing apparatus in which the second region is narrower when the number of motion vectors is larger than when the number of motion vectors is smaller than that of the adjacent divided image side.

(Appendix 8)
In Supplementary Note 1 or 2, The synthesizing unit is a moving image processing apparatus that sets the blend rate for each row or column of a macroblock that is a search unit of the motion vector in the overlap region.

(Appendix 9)
In Appendix 1 or 2,
The encoder unit has a dividing unit that divides the moving image.
The dividing unit narrows the overlap region when the bit rate set for the encoded divided image data is a second bit rate higher than the first bit rate than the first bit rate. A moving image processing apparatus.

(Appendix 10)
In Appendix 1 or 2,
The encoder includes a quantization processing unit that quantizes data to be quantized by dividing by Q value,
The decoder includes an inverse quantization unit that performs inverse quantization by multiplying the inverse quantization target data by the Q,
The synthesizing unit is configured such that a region having a higher blend ratio than the adjacent divided image side in the overlap region is wider on the divided image side where the Q value in the overlap region is smaller than on the larger divided image side. The moving image processing apparatus to be set.

(Appendix 11)
A plurality of encoded divided image data generated by encoding a plurality of divided images obtained by dividing a moving image so as to have overlapping regions are input, and the plurality of encoded divided image data are respectively decoded. A plurality of decoders for extracting motion vector information,
A synthesis unit that blends a plurality of decoded divided images generated by each of the plurality of decoders in the overlap region and outputs an image of the moving image;
The moving image processing apparatus, wherein the synthesizing unit determines a blend ratio of the overlap region based on information on the motion vector output from each of the plurality of decoders.

(Appendix 12)
In Appendix 11,
The moving image processing apparatus is such that the blend rate is higher as the number of motion vectors in the overlap region is larger.

(Appendix 13)
The moving image processing apparatus according to attachment 12, wherein the motion vector for determining the blend ratio is a motion vector having a component in a direction going out from the overlap region to one of the divided images.

EN: Encoder unit 10: Image division unit 12: Encoder group 14: Stream composition unit
DE: Decoder unit 20: Stream separation unit 22: Decoder group 24: Image composition unit
MV: motion vector
Q: Q value

Claims (8)

An encoder unit having a plurality of encoders for encoding encoded divided image data by encoding a plurality of divided images obtained by dividing a moving image so as to have overlapping regions;
A plurality of decoders that respectively decode the plurality of encoded divided image data from the data input from the encoder unit and extract motion vector information, and a plurality of decoded divided images generated by the plurality of decoders respectively decoding And a synthesizing unit that outputs the image of the moving image by blending the decoded divided image having the larger number of motion vectors in the overlap region with a higher blend ratio. A moving image processing apparatus. 2. The moving image processing apparatus according to claim 1 , wherein the motion vector for determining the blend ratio is a motion vector having a component in a direction going out from the overlap region to any one of the divided images. In claim 1 ,
The overlap region includes a first region where the blend rate is zero, a second region where the blend rate gradually increases, and a third region where the blend rate is maximum, from the periphery to the inside of the decoded divided image. A moving image processing apparatus comprising: In claim 3 ,
The moving image processing apparatus according to claim 1, wherein the third region is wider when the number of motion vectors is larger than when the number of motion vectors is smaller than that of adjacent divided images. The moving image processing apparatus according to claim 1 , wherein the synthesis unit sets the blend rate for each row or column of a macroblock that is a search unit for the motion vector in the overlap region. In claim 1 ,
The encoder unit has a dividing unit that divides the moving image.
The dividing unit narrows the overlap region when the bit rate set for the encoded divided image data is a second bit rate higher than the first bit rate than the first bit rate. A moving image processing apparatus. In claim 1 ,
The encoder includes a quantization processing unit that quantizes data to be quantized by dividing by Q value,
The decoder includes an inverse quantization unit that performs inverse quantization by multiplying the inverse quantization target data by the Q value ,
The synthesizing unit is configured such that a region having a higher blend ratio than the adjacent divided image side in the overlap region is wider on the divided image side where the Q value in the overlap region is smaller than on the larger divided image side. A moving image processing apparatus characterized by being set. A plurality of encoded divided image data generated by encoding a plurality of divided images obtained by dividing a moving image so as to have overlapping regions are input, and the plurality of encoded divided image data are respectively decoded. A plurality of decoders for extracting motion vector information,
A plurality of decoded divided images generated by decoding by each of the plurality of decoders are blended in the overlap region, and the decoded divided image having a larger number of motion vectors in the overlap region is blended at a higher blend rate. to the moving image processing apparatus and a combining unit for outputting an image of the moving image.

Images