JP6485159B2 - Moving picture coding apparatus, moving picture coding method, and moving picture coding program - Google Patents

Description

  The present invention relates to a moving image encoding apparatus, a moving image encoding method, and a moving image encoding program.

  As an example of a method for encoding moving image data, there are an intra prediction encoding method and an inter prediction encoding method. The intra prediction encoding method is a method of compressing the data amount by removing spatial redundancy in a frame. The inter prediction encoding method is a method of compressing the data amount by removing temporal redundancy between a plurality of frames.

  In the inter prediction coding method, when an error occurs in a certain frame, the error is propagated to other frames that refer to the frame in which the error has occurred. For this reason, a technique for inserting an intra frame that has been intra-encoded at a predetermined period has been proposed.

  A technique for predicting a current picture based on at least one of a current video processing block generated for a viewpoint different from a viewpoint from which the intra picture of the previous video processing block and the intra picture of the previous video processing block are generated Has been proposed.

  In addition, even when an error occurs during decoding of a video encoded stream including a plurality of channels that are multi-view coded and have a reference relationship, there is little risk of hindering recognition of the contents of the image, and this has an effect on human vision. Techniques to reduce it have been proposed.

  In addition, a code used in a multi-view video coding environment that performs a memory management operation in a decoded image buffer is provided, and the memory management operation is a technology that removes a reference image related to a specific view based on control information. It has been proposed (see, for example, Patent Documents 1 to 3).

Special table 2010-506530 gazette JP 2011-130029 A Special table 2010-507339

  In multi-view video encoding, predictive encoding is performed on a reference image and a non-reference image. When the intra prediction encoding method is applied to multi-view video encoding, intra encoding is performed on both the reference image and the non-reference image. For this reason, the amount of codes increases, and the coding efficiency decreases.

  When the inter prediction encoding method is applied to multi-view video encoding, an intra frame is inserted at a predetermined cycle in order to correct error propagation. The amount of information increases due to this intra frame. In order to correct the disturbance of the moving image due to error propagation at an early stage, if the frequency of inserting an intra frame increases, the amount of information for the intra frame increases.

  As one aspect, an object of the present invention is to perform efficient encoding with a reduced amount of information when performing multi-view video encoding.

In one aspect, the moving image encoding apparatus includes a first predictive encoding unit that performs first predictive encoding in the reference image while shifting a predetermined area for each frame with respect to the reference image, and a non-reference Among the images, a second region corresponding to the first region where the first predictive encoding has been performed on the reference image is determined for each frame based on the parallax between the reference image and the non-reference image. An area setting unit for setting a width based on the parallax in the same direction as the direction in which the first area is shifted on the reference image by the first predictive encoding performed while shifting the predetermined area. The region setting unit that sets the region delayed from the first region as the second region on the non-reference image, and the reference region with respect to the second region of the non-reference image. A second with reference to the first region of the image; And a second predictive coding unit that performs predictive coding.

  According to one aspect, when performing multi-view video encoding, efficient encoding with a reduced amount of information can be performed.

It is a figure which shows an example of the moving image encoder of embodiment. It is a figure which shows an example which applied intra prediction encoding to multiview video encoding. It is a figure which shows an example of the area | region of the vicinity of a refresh boundary among images. It is a figure which shows an example of the prediction mode which performs intra prediction encoding. It is a figure which shows an example of the reference relationship in multiview video encoding. It is a figure which shows an example of the setting of the refresh area | region of a left eye image and a right eye image. It is a figure which shows the specific example of embodiment. It is a flowchart (the 1) which shows an example of the flow of a process of embodiment. It is a flowchart (the 2) which shows an example of the flow of a process of embodiment. It is a flowchart (the 3) which shows an example of the flow of a process of embodiment. It is a figure which shows an example of the moving image encoder of a modification. It is FIG. (1) explaining the specific example in a modification. It is FIG. (2) explaining the specific example in a modification. It is FIG. (The 3) explaining the specific example in a modification. It is a flowchart which shows an example of the flow of the process in a modification. It is a figure which shows an example of the hardware constitutions of a moving image encoder.

<Example of moving picture encoding apparatus of embodiment>
Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 shows an example of a moving picture encoding apparatus 1 according to the embodiment. The video encoding device 1 according to the embodiment is a device that performs multi-view video encoding. The multi-view video encoding device is, for example, a three-dimensional video encoding device.

  The moving image encoding device 1 includes a left eye image encoding unit 10L and a right eye image encoding unit 10R. The left-eye image encoding unit 10L inputs left-eye image data among multi-view video data. The right-eye image encoding unit 10R inputs right-eye image data among multi-view video data.

  Hereinafter, the left-eye image data is described as a left-eye image, and the right-eye image data is described as a right-eye image. The left-eye image encoding unit 10L receives the left-eye image and performs predictive encoding of the left-eye image. The right-eye image encoding unit 10R receives the right-eye image and performs predictive encoding of the right-eye image. The left eye image is an example of a reference image (base view), and the right eye image is an example of a non-reference image (non-base view).

  The left-eye image encoding unit 10L includes a prediction error signal generation unit 11L, an integer conversion unit 12L, a quantization unit 13L, an entropy encoding unit 14L, an inverse quantization unit 15L, an inverse integer conversion unit 16L, a reference image generation unit 17L, and a filter. A processing unit 18L and a frame memory 19L are provided.

  The left-eye image encoding unit 10L includes a motion detection unit 20L, a motion compensation unit 21L, an intra-frame prediction unit 22L, an encoding control unit 23L, and a switching unit 24L. The intra-frame prediction unit 22L of the left-eye image encoding unit 10L is an example of a first prediction encoding unit. Prediction coding performed by the intraframe prediction unit 22L is an example of first prediction coding.

  The right-eye image encoding unit 10R includes a prediction error signal generation unit 11R, an integer conversion unit 12R, a quantization unit 13R, an entropy encoding unit 14R, an inverse quantization unit 15R, an inverse integer conversion unit 16R, a reference image generation unit 17R, and a filter. A processing unit 18R and a frame memory 19R are provided.

  The right-eye image encoding unit 10R includes a motion detection unit 20R, a motion compensation unit 21R, an intra-frame prediction unit 22R, an encoding control unit 23R, a switching unit 24R, and a region setting unit 25. The motion detection unit 20R of the right-eye image encoding unit 10R is an example of a second predictive encoding unit. Predictive coding performed by the motion detection unit 20R is an example of second predictive coding.

  Of the left-eye image encoding unit 10L and the right-eye image encoding unit 10R, each unit other than the region setting unit 25 is common. However, the motion detection unit 20L and the motion detection unit 20R perform different processes. Also, the encoding control unit 23L and the encoding control unit 23R perform different control.

  Hereinafter, each part will be described. In the following description, “L” indicates a function of the left-eye image encoding unit 10L, and “R” indicates a function of the right-eye image encoding unit 10R.

  The prediction error signal generation unit 11 receives the original image signal and the prediction image signal. The prediction error signal generation unit 11 calculates a difference between the original image signal and the prediction image signal and generates a prediction error signal. The integer conversion unit 12 performs integer conversion on the prediction error signal input from the prediction error signal generation unit 11 and outputs the signal after the integer conversion to the quantization unit 13.

  The quantization unit 13 quantizes the signal after the integer conversion, and outputs quantized data in which the code amount of the prediction error signal is reduced to the entropy encoding unit 14. The entropy encoding unit 14 performs entropy encoding on the quantized data.

  The entropy encoding unit 14 outputs encoded image data subjected to entropy encoding. The entropy encoding unit 14L outputs the encoded image data as a left eye image. The entropy encoding unit 14R outputs the encoded image data as a right eye image. For example, entropy coding is coding using a coding method in which a variable-length code is assigned according to the symbol frequency.

  The inverse quantization unit 15 receives the quantized data from the quantization unit 13 and inversely quantizes the input quantized data. The inverse integer transform unit 16 performs an inverse integer transform process on the data input from the inverse quantization unit 15. Thereby, a signal comparable to the prediction error signal before encoding is obtained.

  The reference image generation unit 17 adds the pixel data of the macroblock motion-compensated by the motion compensation unit 21 and the prediction error signal decoded by the inverse quantization unit 15 and the inverse integer transform unit 16. Accordingly, the reference image generation unit 17 generates a motion-compensated reference image (also referred to as a reference picture).

  The filter processing unit 18 performs a deblocking filtering process on the macroblock data of the reference image, and stores the frame of the reference image after suppressing the occurrence of block noise in the frame memory 19.

  As described above, the motion detection unit 20 performs different processing between the left-eye image encoding unit 10L and the right-eye image encoding unit 10R. The motion detection unit 20L of the left-eye image encoding unit 10L performs predictive encoding with reference to temporally changing images (frames). This predictive coding is also referred to as inter prediction.

  The motion detection unit 20R of the right-eye image encoding unit 10R performs inter-view prediction or inter prediction. Inter-view prediction is predictive coding with reference to frames with different viewpoints at the same time. Inter-view prediction is also referred to as inter-view prediction. In the case of the embodiment, the motion detection unit 20R performs inter-view prediction with reference to the region of the left-eye image set by the region setting unit 25.

  The motion compensation unit 21 detects the motion of frames that are temporally forward and backward, and performs motion compensation for the detected motion with respect to the temporally previous frame. For example, the motion compensation unit 21 detects a motion vector on the basis of temporally moving frames.

  The intra-frame prediction unit 22 performs predictive encoding within the same image (frame). This predictive coding is also called intra prediction. The intra-frame prediction unit 22 generates a macro block of the predicted image based on the surrounding pixels in the same image, and performs predictive coding.

  The encoding control unit 23L controls whether the prediction encoding performed by the left-eye image encoding unit 10L is intra prediction or inter prediction. The encoding control unit 23R controls whether the prediction encoding performed by the right-eye image encoding unit 10R is intra prediction, inter prediction, or inter-view prediction.

  For example, H.M. In the H.264 standard, a prediction mode for performing intra-refresh with 4 × 4 pixels as a sub-block is formulated. When the sub-block is in contact with the boundary of the intra-refreshed area, any pixel in the sub-block may refer to a pixel in the unrefreshed area.

  In this case, performing intra refresh is limited. The encoding control unit 23L controls the intra-frame prediction unit 22L to perform intra refresh when the sub-block is not subject to intra refresh restriction. On the other hand, when the sub-block is subject to intra refresh restriction, the encoding control unit 23L controls to perform inter-refresh.

  For this reason, the encoding control unit 23L controls the switching unit 24L. When the sub-block is not subject to intra refresh restriction, the coding control unit 23L connects the switching unit 24L to the intra-frame prediction unit 22L. When the sub-block is subject to intra refresh restriction, the encoding control unit 23L connects the switching unit 24L to the motion compensation unit 21.

  The region setting unit 25 sets a region for performing inter-view prediction encoding in the right eye image. The region setting unit 25 sets a region with a predetermined amount of difference from a region where the left-eye image encoding unit 10L performs intra refresh on the left-eye image as a target region for inter-view predictive encoding. This difference in the predetermined amount is based on the parallax between the left eye image and the right eye image.

<An example of predictive coding in multi-view video coding>
Next, an example of predictive coding in multi-view video coding will be described. FIG. 2 shows an example in which intra prediction encoding is applied to multi-view video encoding. Intra prediction encoding is also referred to as intra refresh.

  In the example of FIG. 2, the refresh area is set to a part of the left eye image and the right eye image having a predetermined width (refresh area). Further, the refresh area is shifted by a predetermined width over time. As described above, the intra refresh performed by sequentially shifting the refresh area may be referred to as an intra slice.

  In predictive coding by intra slice, intra refresh is performed in the refresh area. For this reason, error propagation is suppressed. In addition, since the refresh area is distributed to images at different times, intra refresh is not performed on all areas of the image at one time. Therefore, the encoding efficiency is improved.

  FIG. 3 shows an example of an area in the vicinity of the refresh boundary in the image. In FIG. 3, it is assumed that the macroblock is an area of 16 × 16 pixels, for example. The refresh boundary is a boundary between a refreshed area (shaded area) and an unrefreshed area (non-shaded area).

  An intra line (a hatched region) is a vertical line of a macroblock that is in contact with a refresh boundary. When intra refresh is performed on an intra-line macroblock, intra refresh that refers to pixels in an unrefreshed area is limited.

  FIG. 2 shows a prediction mode for performing intra refresh defined in the H.264 standard. Pixels to be subjected to intra refresh are 4 × 4 pixels (sub blocks). This sub-block is included in a macroblock.

  In the example of FIG. 4, pixels that have not been subjected to intra refresh are denoted as unrefreshed pixels. In addition, a pixel referred to by an unrefreshed pixel when intra refresh is performed is referred to as a reference pixel. In the example of FIG. 4, the reference pixels are shaded, and the unrefreshed pixels are not shaded.

  In the example of FIG. 4, in the prediction modes other than the prediction modes 3 and 7, the reference pixel referred to by each pixel of the sub-block is not included in the unrefreshed area. On the other hand, in prediction modes 3 and 7, reference pixels referred to by a plurality of pixels among the pixels of the sub-block are included in the unrefreshed area.

  Intra refresh is not performed on each pixel in the unrefreshed area. For this reason, in prediction modes 3 and 7, sub-blocks that are in contact with the refresh boundary are restricted from performing intra-refresh.

  FIG. 5 shows an example of a reference relationship in multi-view video encoding. In the example of FIG. 5, “I” indicates an I frame (Intra-Coded Frame). The I frame is a frame that is predictively encoded by intra prediction.

  “P” indicates a P frame (Predicted Frame). The P frame is a frame that is predictively encoded by inter prediction. The P frame is a frame on which predictive encoding is performed with reference to a frame at a previous time (past frame).

  “B” indicates a B-frame (Bi-direction Frame). The B frame is a frame that is predictively encoded by inter prediction. The B frame is a frame on which predictive encoding is performed with reference to a frame at a previous time, a frame at a later time, or a frame at a previous or subsequent time.

  The left-eye image has one cycle of an I frame, a B frame, a B frame, and a P frame. The right-eye image has one cycle of P frame, B frame, B frame, and P frame. Further, the refreshing of the right eye image is performed with reference to the refreshed left eye image. As a result, inter-view prediction encoding is performed.

  In the example of FIG. 5, the refreshing of the right eye image is performed by inter prediction or inter-view prediction. Therefore, since there is a possibility of error propagation, a frame (hereinafter referred to as an intra frame) in which intra prediction is performed on the right eye image is inserted in order to recover the moving image from the error early. For this reason, the amount of information increases due to the inserted intra frame.

  As in the example of FIG. 2, if the intra-refresh is performed independently on the left-eye image and the right-eye image, it is not necessary to insert an intra frame. However, since the intra-refresh is performed independently for the left-eye image and the right-eye image, the code amount increases. For this reason, encoding efficiency falls.

  Therefore, in the embodiment, as illustrated in FIG. 6, the left-eye image encoding unit 10 </ b> L performs an intra-refresh on the refresh region L with a region having a predetermined width of the left-eye image as a refresh region L. At time T, the left-eye image encoding unit 10L performs intra refresh on the refresh area L.

  The right-eye image encoding unit 10R performs inter-view predictive encoding on the refresh region R with a region having a width narrower than the refresh region L of the left-eye image at the same time T as the refresh region R. The width of the refresh region R at time T is based on the parallax between the left eye image and the right eye image.

  The left-eye image encoding unit 10L and the right-eye image encoding unit 10R perform refresh by shifting the refresh area in the same direction over time. At this time, the refresh area R is delayed from the refresh area L by the width based on the parallax.

  The right-eye image encoding unit 10R performs intra-refresh on the region corresponding to the refresh region R in the refresh region L of the left-eye image at time T after the left-eye image encoding unit 10L performs a view on the refresh region R. Inter prediction encoding is performed.

  As a result, when the right-eye image encoding unit 10R refreshes the sub-block that is in contact with the refresh boundary in the refresh region R, the left-eye image encoding unit 10L has already performed intra-refresh for the region that has exceeded the refresh boundary. There is a high possibility.

  For this reason, there is a high possibility that the sub-block in contact with the refresh boundary of the right-eye image refers to the refreshed pixel. Therefore, the right-eye image encoding unit 10R can perform inter-view prediction encoding on a sub-block for which intra refresh is restricted.

  At time T + 1, the left-eye image encoding unit 10L sets, as the refresh area L at time T + 1, an area shifted from the refresh area L refreshed at time T by the width of the refresh area L with respect to the left-eye image. Then, the left-eye image encoding unit 10L performs intra refresh on the set refresh area L.

  At time T + 1, the right-eye image encoding unit 10R sets, as the refresh region R at time T + 1, a region where the width of the refresh region L of the left-eye image is shifted from the refresh region R refreshed at time T with respect to the right-eye image. .

  At this time, the right-eye image encoding unit 10R sets the width of the refresh region R at time T + 1 to the same width as the width of the refresh region L. As a result, the refresh area R and the refresh area L at time T + 1 have the same size.

  After time T + 1, the left-eye image encoding unit 10L sets a designated area (an area surrounded by a thick line in FIG. 6). The designated region is a region in which the right-eye image encoding unit 10R has performed intra refresh in the current frame and one or more past frames.

  After time T + 1, the right-eye image encoding unit 10R performs refresh by inter-view reference for the set refresh area L with reference to the designated area. The refresh boundary of the right eye image is behind the tip of the designated area.

  Therefore, when the right-eye image encoding unit 10R performs inter-view predictive encoding, the sub-block can be refreshed even if there is a restriction on intra-refresh on the sub-block that is in contact with the refresh boundary of the refresh region R.

<Specific Example of Embodiment>
Next, a specific example of the embodiment will be described with reference to FIG. The width direction of the left eye image and the right eye image is defined as the X direction, and the direction orthogonal to the width direction is defined as the Y direction. Further, of both ends of the left-eye image and the right-eye image, the end portion where the refresh area is first set is the last end, and the opposite end portion is the most advanced end.

  When performing intra prediction, the encoding control unit 23L of the left-eye image encoding unit 10L controls the switching unit 24L so that the intra-frame prediction unit 22L is connected. The intra-frame prediction unit 22L sets an area having a width n from the end of the left-eye image as the refresh area L.

  Then, the intra-frame prediction unit 22L sequentially performs intra-refresh for each frame from the last end to the forefront. Thereby, the intra-frame prediction unit 22L performs intra refresh on the left eye image.

  The region setting unit 25 of the right-eye image encoding unit 10R sets the width a of the refresh region R. The width a is based on the parallax between the left eye image and the right eye image. In a multi-view video, a subject is photographed from a plurality of viewpoints. Accordingly, a parallax is generated between the left eye image and the right eye image.

  For example, the region setting unit 25 may obtain a shift amount based on parallax by comparing the left eye image and the right eye image at time T. Since the left-eye image and the right-eye image are taken of the same subject, the region setting unit 25 can obtain the shift amount as the parallax by comparing the two images. The region setting unit 25 may set the obtained shift amount to the width a.

  The width n is assumed to be larger than the width a. Therefore, at time T, the refresh area L is wider than the refresh area R. The right-eye image encoding unit 10R performs inter-view prediction encoding with reference to the refreshed refresh area L in the left-eye image.

  When the right-eye image encoding unit 10R performs inter-view prediction encoding, the switching unit 24R is connected to the motion compensation unit 21R. The motion detection unit 20R refers to the left-eye image in the frame memory 19L of the left-eye image encoding unit 10L and performs inter-view prediction encoding on the refresh region R in the right-eye image.

  At time T + 1, the intraframe prediction unit 22L performs intra refresh on the refresh region L having a width n of the left-eye image. The left-eye image encoding unit 10L sets a designated area including a refresh area L of the current frame and a refreshed area of one or more past frames. The designated area may be an area from the front end to the rear end of the refresh area L.

  The region setting unit 25 of the right-eye image encoding unit 10R sets the region of width n among the designated regions of the left-eye image as the refresh region R. The refresh region R is a region having a width n from a position where the width a is shifted from the end.

  The motion detection unit 20R performs inter-view prediction encoding with reference to the designated region of the left-eye image for the refresh region R set by the region setting unit 25 in the right-eye image. Thereby, the motion detection unit 20R performs the refresh region R of the right-eye image at time T + 1.

  At time T + T1, the refresh area L makes a round of the left eye image and returns to the initially set position again. In the example of FIG. 7, the designated area is divided. A part of the designated area is an area having a width n from the rear end, and the remaining part is an area having a predetermined width from the forefront.

  At time T + T1, the refresh area R of the right eye image is also divided corresponding to the designated area of the left eye image. The area setting unit 25 sets the area having the width “na” from the forefront of the left-eye image and the area having the width a from the rear end to the refresh area R based on the designated area of the left-eye image.

  The motion detection unit 20R performs inter-view prediction encoding with reference to the designated region of the left eye image at time T + T1. As a result, the refresh region R of the right eye image is refreshed at time T + T1.

  Therefore, the right-eye image encoding unit 10R performs the refresh with a delay of the width a from the left-eye image encoding unit 10L. The width a is based on the parallax between the left eye image and the right eye image. The right-eye image encoding unit 10R performs inter-view prediction encoding with reference to a designated region that has been intra-refreshed in the left-eye image.

  For this reason, the right-eye image encoding unit 10R can refresh the right-eye image without performing intra-refresh. Therefore, since intra refresh is not performed on each of the left eye image and the right eye image, encoding efficiency is improved.

  Further, the right-eye image encoding unit 10R performs inter-view prediction encoding with reference to the designated region of the left-eye image at the same time. Therefore, the right-eye image encoding unit 10R can refresh the right-eye image without inserting an intra frame. For this reason, since an intra frame is not inserted, the amount of information generated is suppressed.

<Flowchart of Embodiment>
Next, the processing flow of the embodiment will be described with reference to the flowcharts of FIGS. The encoding control unit 23L of the left-eye image encoding unit 10L sets a refresh area L in the left-eye image (Step S1). For example, the encoding control unit 23L may equally divide the width of the left-eye image in the X direction into a plurality of areas and set the equally divided area n as the refresh area L.

  The encoding control unit 23L sets an encoding block in all regions of the refresh region L (step S2). For example, the encoding control unit 23L may set the encoding block to a 4 × 4 pixel sub-block.

  The left-eye image encoding unit 10L starts a loop that performs predictive encoding in the raster scan order for all the encoded blocks in the left-eye image (step S3). The encoding control unit 23L determines whether or not the encoded block is an intra refresh restriction target (step S4).

  As described above, intra prediction encoding (intra refresh) in which an encoding block refers to a pixel in an unrefreshed region is limited. Therefore, when the encoded block is a restriction target for intra refresh (YES in step S4), the intra-frame prediction unit 22L performs intra prediction encoding with reference to the refreshed region (step S5).

  For this reason, in the case of YES in step S4, the encoding control unit 23L performs control so that the switching unit 24L is connected to the intra-frame prediction unit 22L. On the other hand, when the encoded block is not a restriction target for intra refresh (NO in step S4), the encoding control unit 23L controls to perform inter encoding or intra encoding (step S5).

  For example, when the coding block is inside the refresh region L, the coding control unit 23L performs control such that the switching unit 24L is connected to the intra-frame prediction unit 22L. As a result, the intra-frame prediction unit 22L performs intra coding on the coded block inside the refresh area L.

  When the coding block is outside the refresh area L, the coding control unit 23L performs control such that the switching unit 24L is connected to the motion compensation unit 21L. As a result, the motion detection unit 20L performs inter coding on the coding block outside the refresh region L with reference to frames that are temporally adjacent.

  The left-eye image encoding unit 10L performs loop control so that the process of step S5 or step S6 is executed for all the encoded blocks in one image of the left-eye image (step S7). When the loop from step S3 to step S7 ends, the process proceeds to the next step.

  The region setting unit 25 of the right eye image encoding unit 10R sets the refresh region R of the right eye image (step S8). The region setting unit 25 sets the encoding block to be refreshed in all regions inside the set refresh region R (step S9). Then, the process proceeds to “A”.

  The setting of the refresh region R of the right eye image will be described with reference to the flowchart of FIG. The region setting unit 25 determines whether or not the refresh region L set by the left-eye image encoding unit 10L is the refresh region L that is initially set for the left-eye image (step S11).

  When the refresh area L is the refresh area L that is set first (YES in step S11), the area setting unit 25 sets the width a of the refresh area R of the right-eye image (step S12). The width a is a width based on the parallax between the left eye image and the right eye image, and is a fixed value in the embodiment.

  The region setting unit 25 determines whether or not the refresh region L of the left-eye image is set at the last end (step S13). When the refresh area L of the left-eye image is set at the last end (YES in step S13), the designated area is divided into a predetermined area from the last end and a predetermined area up to the forefront.

  Accordingly, the region setting unit 25 sets the width of the refresh region R of the right-eye image by dividing the width into the region of the width a and the region of the width “na” from the end to the forefront (step S14). If the refresh area L of the left-eye image is not set at the end (NO in step S13), the area setting unit 25 sets the width of the refresh area R of the right-eye image to n (step S15).

  The processing after “A” will be described with reference to FIG. The right-eye image encoding unit 10R starts a loop that performs predictive encoding in the raster scan order for all the encoded blocks in the right-eye image (step S20).

  The encoding control unit 23R determines whether or not the encoded block is a restriction target for intra refresh (step S21). If the encoded block is a restriction target for intra refresh (YES in step S21), the right-eye image encoding unit 10R performs inter-view predictive encoding on the encoded block in the designated region of the left-eye image (step S22). ).

  In this case, the encoding control unit 23R of the right-eye image encoding unit 10R performs control such that the switching unit 24R is connected to the motion compensation unit 21R. Thereby, the right-eye image encoding unit 10R performs inter-view prediction encoding with reference to the designated region of the left-eye image.

  Further, in the case of YES in step S21, the right eye image encoding unit 10R may perform intra prediction encoding with reference to the refreshed area. In this case, the encoding control unit 23R performs control such that the switching unit 24R is connected to the intra-frame prediction unit 22R.

  If the encoded block is not a restriction target for intra refresh (NO in step S21), the right-eye image encoding unit 10R performs prediction using an arbitrary method (intra prediction encoding, inter prediction encoding, or inter-view prediction encoding). Encoding is performed (step S23).

  When the right-eye image encoding unit 10R performs predictive encoding of the right-eye image by intra prediction encoding, the encoding control unit 23R performs control such that the switching unit 24R is connected to the intra-frame prediction unit 22R.

  When the right-eye image encoding unit 10R performs predictive encoding of the right-eye image by inter prediction encoding or inter-view prediction encoding, the encoding control unit 23R performs control such that the switching unit 24R is connected to the motion compensation unit 21R. .

  The right-eye image encoding unit 10R performs loop control so that the process of step S22 or step S23 is executed for all the encoded blocks in one image of the right-eye image (step S37). When the loop from step S21 to step S24 ends, the process ends.

<An example of a video encoding device according to a modification>
Next, an example of a moving image encoding apparatus according to a modification will be described with reference to FIG. Description of functions common to the above-described moving image encoding device 1 in the moving image encoding device 1 of the modification is omitted.

  As shown in the example of FIG. 11, the right-eye image encoding unit 10 </ b> R of the moving image encoding apparatus 1 according to the modification includes a candidate area setting unit 26, a holding unit 27, and a statistics unit 28. The candidate area setting unit 26 sets a candidate for the refresh area R in which the right-eye image encoding unit 10R performs predictive encoding in the right-eye image. Hereinafter, the refresh region R candidates may be referred to as refresh candidate regions.

  The holding unit 27 holds the inter-view prediction vector for each encoded block. The motion detection unit 20R performs inter-view prediction encoding. The holding unit 27 holds a prediction vector for one or more frames when the motion detection unit 20R has performed inter-view prediction encoding in the past. The prediction vector when inter-view prediction encoding is performed indicates the parallax between the left eye image and the right eye image.

  The statistical unit 28 performs statistical processing on the prediction vector of each encoded block of one or more past frames held by the holding unit 27. The result of statistical processing performed by the statistical unit 28 is used as an evaluation value. Therefore, since the evaluation value is a value obtained by performing statistical processing on the prediction vector, it is a value indicating the parallax between the left eye image and the right eye image.

  The statistical unit 28 may perform statistical processing of the prediction vector of each encoded block of the immediately preceding one frame. However, the greater the number of samples, the better the accuracy of statistical processing. For this reason, it is preferable that the statistics part 28 performs the statistical process of the prediction vector of each encoding block of the past several frames.

  For example, the statistical unit 28 may use an average value of prediction vectors of each encoded block of a plurality of past frames as an evaluation value. Further, the statistical unit 28 may use the maximum value of the prediction vector of each encoded block of a plurality of past frames as the evaluation value.

<Specific example of modification>
Next, a specific example of the modification will be described with reference to FIGS. In FIG. 12, the candidate area setting unit 26 of the right-eye image encoding unit 10 </ b> R sets the width c of the refresh candidate area (the area surrounded by the thick broken line in the figure). Here, as shown in FIG. 13, the width from the front end of the refresh area L where the left-eye image encoding unit 10L performs intra-refresh to the left-eye image to the last end of the left-eye image is set to xn.

  This area of width xn is the area of the refresh area L and the refreshed area that has already undergone intra refresh in the left-eye image. Also, the width of the refreshed area in the right-eye image that has been refreshed is b.

  The candidate area setting unit 26 sets the width c of the refresh candidate area to “c = xn−b”. FIG. 12 shows the refresh area L of the left eye image and the refresh area R of the right eye image at the first time T when setting the refresh area.

  Similar to the above-described embodiment, at the first time T, the left-eye image encoding unit 10L sets a region having a width n from the end of the left-eye image as the refresh region L. At the first time T, the left-eye image has no refreshed area.

  The width xn is an area between the refresh area L and the refreshed area. Then, at the first time T, there is no refreshed area in the left eye image. Therefore, the width xn is the width n of the refresh area L set for the left-eye image.

  Also, at the first time T, there is no refreshed area in the right eye image. Therefore, the candidate area setting unit 26 sets the width c of the refresh candidate area to “c = xn−b = n”. At the first time T, the region setting unit 25 sets the width a based on the parallax between the left eye image and the right eye image to the width m of the refresh region R.

  Next, the left eye image refresh area L and the right eye image refresh area R at time T + T2 will be described with reference to FIG. At time T + T2, the refresh area L of the left-eye image has advanced in the X direction from time T.

  The width xn is the width of the refresh area L and the refreshed area. Therefore, the width of the refreshed area is “xn−n”. The designated area may be all areas having a width xn.

  The candidate area setting unit 26 of the right-eye image encoding unit 10R sets the width c of the refresh candidate area. It is assumed that there is a width b of the refreshed area of the right eye image at time T + T2.

  Accordingly, the candidate area setting unit 26 sets the width c (= xn−b) of the refresh candidate area. The region setting unit 25 sets the refresh region R for each frame based on the refresh candidate region width c set by the candidate region setting unit 26.

  The region setting unit 25 sets the width m of the refresh region R to “m = cv”. The width c is the width of the refresh candidate area. The width v is a width based on an evaluation value obtained by performing statistical processing by the statistical unit 28.

  The statistical unit 28 performs statistical processing on the prediction vector held by the holding unit 27 in units of encoded blocks, and obtains an evaluation value. This evaluation value is a value based on a prediction vector when inter-view prediction encoding is performed in a past frame. Therefore, the evaluation value is a value based on the parallax between the past left-eye image and right-eye image.

  Therefore, the region setting unit 25 sets the width m of the refresh region R reflecting the width v based on the evaluation value for each frame. As described above, the width m of the refresh region R is a width obtained by subtracting the width v based on the evaluation value from the width c of the refresh candidate region.

  The width v based on the evaluation value is a value reflecting the parallax between the left eye image and the right eye image. Therefore, the region setting unit 25 sets the width m of the refresh region R to “m = c−v”, so that the width m of the refresh region R becomes a variable value according to the parallax between the left eye image and the right eye image. .

  Thus, since the width n of the refresh region R of the right eye image is synchronized with the width n of the refresh region L of the left eye image, the right eye image encoding unit 10R refers to the refreshed region of the left eye image and Refresh can be performed.

  FIG. 14 shows an example of the refresh area L and the refresh area R at time T + T3. The left-eye image encoding unit 10L performs refresh by intra-encoding on the left-eye image. At time T + T3, the left-eye image is made a round, and the refresh area L returns to the last end of the left-eye image.

  As described in the above-described embodiment, the left-eye image encoding unit 10L divides the designated region into a partial region from the rearmost end of the left-eye image to a partial region up to the forefront. In this case, the candidate area setting unit 26 of the right-eye image encoding unit 10R sets the width c of the refresh candidate area of the right-eye image to the same width as the width n of the refresh area L of the left-eye image.

  In addition, the region setting unit 25 divides the width c of the refresh candidate region into a width v and a width m, sets the region with the width v as the region up to the forefront of the right-eye image, and sets the region with the width m as the right-eye image. Set the area from the end.

  Here, the area of width v is included in the designated area of the corresponding left eye image. Therefore, the right-eye image encoding unit 10R can perform inter-view prediction encoding with reference to the designated region of the left-eye image corresponding to the region of width v.

  The region setting unit 25 sets the region of the width v as a partial region of the refresh region R. The width v is a width based on the evaluation value. On the other hand, the region setting unit 25 also sets a region having a width m (m = cv) as a partial region of the refresh region R. The width m is narrower than the width n.

  Therefore, the right-eye image encoding unit 10R can perform inter-view prediction encoding with reference to the designated region of the left-eye image corresponding to the width m. Therefore, the region setting unit 25 sets the region of width m as a partial region of the refresh region R.

  Accordingly, at time T + T3, when the designated area of the left eye image is divided and the refresh area L is in contact with the end of the left eye image, the area setting unit 25 selects an area having the same width n as the refresh area L of the left eye image. Set to refresh region R. In this case, the width of the refresh region R is the widest.

  As described above, the region setting unit 25 changes the size of the refresh region R of the right-eye image based on the parallax between the left-eye image and the right-eye image for each time frame of the moving image data. Therefore, the right-eye image encoding unit 10R can increase the possibility of referring to the refreshed region of the left-eye image when performing inter-view reference encoding on the refresh region R of the right-eye image.

<Flow chart of modification>
Next, a flowchart of a modification will be described. The process of the modified example is different from the process of the embodiment in the process of setting the refresh region R of the right-eye image among the processes of the embodiment. That is, in the modification, the process of step S8 in FIG. 8 is different from the process of the embodiment.

  Various processes other than the process of setting the refresh region R of the right-eye image are the same as the processes described in the embodiment. Therefore, description of processing similar to that in the embodiment is omitted in the modification.

  A process for setting the refresh region R of the right-eye image in the modification will be described. The region setting unit 25 determines whether or not the refresh region L set by the left-eye image encoding unit 10L is the refresh region L that is initially set for the left-eye image (step S31).

  When the refresh area L is the refresh area L that is set first (YES in step S31), the area setting unit 25 sets the width a of the refresh area R of the right-eye image (step S32). The width a is a width based on the parallax between the left eye image and the right eye image.

  The candidate area setting unit 26 determines whether or not the refresh area L of the left-eye image is set at the last end (step S33). If the refresh area L of the left-eye image is not set at the end (NO in step S33), the candidate area setting unit 26 sets the width c of the refresh candidate area of the right-eye image (step S34).

  The candidate area setting unit 26 obtains the width c by subtracting the width b from the width xn. Then, the candidate area setting unit 26 sets the area of the width c as the refresh candidate area for the right eye image. The region setting unit 25 sets a region corresponding to the width obtained by subtracting the width v based on the evaluation value from the width c as the refresh region R (step S35).

  If the candidate area setting unit 26 determines that the refresh area L of the left-eye image is set at the end (YES in step S33), the candidate area setting unit 26 sets the width c of the refresh candidate area of the right-eye image to the width n. (Step S36).

  Then, the area setting unit 25 sets the width of a part of the refresh area R of the right-eye image to the same width as the width v based on the evaluation value (step S37). The area setting unit 25 sets an area having a width v from the forefront of the right-eye image as a partial area of the refresh area R.

  The area setting unit 25 sets the width of a part of the refresh area R of the right-eye image to a width obtained by subtracting the width v based on the evaluation value from the width c of the refresh candidate area (step S38). The area setting unit 25 sets an area having a width “cv” from the end of the right-eye image as a partial area of the refresh area R.

<Example of Hardware Configuration of Video Encoding Device>
Next, an example of the hardware configuration of the moving image encoding device 1 will be described with reference to the example of FIG. As shown in the example of FIG. 16, a processor 111, a RAM 112, a ROM 113, an auxiliary storage device 114, a medium connection unit 115, and a communication interface 116 are connected to the bus 100.

  The processor 111 is an arbitrary processing circuit. The processor 111 executes a program expanded in the RAM 112. As a program to be executed, a program for performing the processing of the embodiment may be applied. The ROM 113 is a non-volatile storage device that stores programs developed in the RAM 112.

  The auxiliary storage device 114 is a storage device that stores various types of information. For example, a hard disk drive or a semiconductor memory may be applied to the auxiliary storage device 114. The medium connection unit 115 is provided so as to be connectable to the portable recording medium 118.

  As the portable recording medium 118, a portable memory or an optical disc (for example, Compact Disc (CD), Digital Versatile Disc (DVD), semiconductor memory, etc.) may be applied. A program for performing the processing of the embodiment may be recorded on the portable recording medium 118.

  Each unit of the moving image encoding device 1 may be realized by the processor 111 executing a predetermined program. Moreover, each part of the moving image encoding device 1 may be realized by an FPGA or the like. FPGA is an abbreviation for Field Programmable Gate Array, and is a programmable semiconductor integrated circuit.

  The RAM 112, the ROM 113, the auxiliary storage device 114, and the portable recording medium 118 are all examples of a tangible storage medium that can be read by a computer. These tangible storage media are not temporary media such as signal carriers.

<Others>
The present embodiment is not limited to the above-described embodiment, and various configurations or embodiments can be taken without departing from the gist of the present embodiment.

DESCRIPTION OF SYMBOLS 1 Moving image encoder 10L Left eye image encoding part 11L Right eye image encoding part 20L, 20R Motion detection part 22L, 22R Intraframe prediction part 23L, 23R Encoding control part 24L, 24R Switching part 25 Area | region determination part 26 Candidate area | region Determination unit 27 Holding unit 28 Statistics unit 111 Processor 112 RAM
113 ROM

Claims (10)

A first prediction encoding unit that performs first prediction encoding in the reference image while shifting a predetermined region for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. An area setting unit that is set for each frame, and is based on the parallax in the same direction as the direction in which the first area is shifted on the reference image by the first predictive encoding performed while shifting the predetermined area the region which is delayed by the width of the said first region, and the region setting unit that sets on the non-reference image as the second region,
A second predictive coding unit that performs second predictive coding with reference to the first region of the reference image for the second region of the non-reference image;
A video encoding device comprising: A first prediction encoding unit that performs first prediction encoding in the reference image while shifting a predetermined region for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. An area setting unit that is set for each frame, and when the first area is set again at a position where the first area is initially set, the second area touches both ends of the non-reference image. The area setting unit for dividing the area into two areas ;
A second predictive coding unit that performs second predictive coding with reference to the first region of the reference image for the second region of the non-reference image;
A video encoding device comprising: A first prediction encoding unit that performs first prediction encoding in the reference image while shifting a predetermined region for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. a region setting unit that sets for each frame, based on the evaluation value performed statistical processing on the prediction vector based on the parallax between the non-reference image and the reference image, of the second region for each frame The area setting unit for changing the size ;
A second predictive coding unit that performs second predictive coding with reference to the first region of the reference image for the second region of the non-reference image;
A video encoding device comprising: The region setting unit uses the maximum value of the prediction vectors or the average value of the prediction vectors as the evaluation value.
The moving image encoding apparatus according to claim 3 . Performing a first predictive encoding within the reference image while shifting a predetermined area for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. It is set for each frame, and the width based on the parallax is set in the same direction as the direction in which the first area is shifted on the reference image by the first predictive encoding performed while shifting the predetermined area. Setting a region delayed from the first region only as the second region on the non-reference image;
Performing second predictive encoding with reference to the first region of the reference image for the second region of the non-reference image;
A video encoding method in which a computer executes a process including: Performing a first predictive encoding within the reference image while shifting a predetermined area for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. Set for each frame, and when the first area is set again at the position where the first area was initially set, the second area is divided into two areas in contact with both ends of the non-reference image. When,
Performing second predictive encoding with reference to the first region of the reference image for the second region of the non-reference image;
A video encoding method in which a computer executes a process including: Performing a first predictive encoding within the reference image while shifting a predetermined area for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. The size of the second region is changed for each frame based on an evaluation value that is set for each frame and statistical processing is performed on a prediction vector based on a parallax between the reference image and the non-reference image. When,
Performing second predictive encoding with reference to the first region of the reference image for the second region of the non-reference image;
A video encoding method in which a computer executes a process including: Performing a first predictive encoding within the reference image while shifting a predetermined area for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. It is set for each frame, and the width based on the parallax is set in the same direction as the direction in which the first area is shifted on the reference image by the first predictive encoding performed while shifting the predetermined area. Setting a region delayed from the first region only as the second region on the non-reference image;
And be made to the second region of the previous SL non-reference image, the second predictive coding with reference to the first region of the reference image,
A moving picture encoding program for causing a computer to execute a process including: Performing a first predictive encoding within the reference image while shifting a predetermined area for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. Set for each frame, and when the first area is set again at the position where the first area was initially set, the second area is divided into two areas in contact with both ends of the non-reference image. When,
Performing second predictive encoding with reference to the first region of the reference image for the second region of the non-reference image;
A moving picture encoding program for causing a computer to execute a process including: Performing a first predictive encoding within the reference image while shifting a predetermined area for each frame with respect to the reference image;
Based on the parallax between the reference image and the non-reference image, a second region corresponding to the first region in which the first predictive encoding is performed on the reference image among the non-reference images. The size of the second region is changed for each frame based on an evaluation value that is set for each frame and statistical processing is performed on a prediction vector based on a parallax between the reference image and the non-reference image. When,
Performing second predictive encoding with reference to the first region of the reference image for the second region of the non-reference image;
A moving picture encoding program for causing a computer to execute a process including:

Images