JP5100572B2 - Encoder - Google Patents

Description

  The present invention relates to an encoding device that encodes image data.

  As a compression encoding method of moving image data, MPEG-2 (Moving Picture Experts Group phase 2), H.264 / AVC (Advanced Video Coding) and the like are known.

  H. In the H.264 / AVC standard, a redundant slice is defined for a normal main slice in order to enhance resistance against transmission errors. This generally enhances error tolerance by adding a region of interest such as a person as a redundant slice to encoded data. As another method, the number of divisions of the main slice is increased by using a method of changing the slice order (arbitrary slice order) or a feature in which an error does not propagate to different main slices at the time of decoding. The method is known.

In general, when an error occurs in the transmission path, the decoding apparatus executes an error concealment process or an interpolation process in which the error part is replaced with another image. As an error concealment processing method, a technique for concealing an error part using a frame decoded immediately before is known (see Patent Document 1). Also, a technique is known in which the transmission side (encoding device) transmits encoded data according to the priority of data within the range of the transmission capacity allowed in the transmission path, assuming that an error occurs in the transmission path. (See Patent Document 2). Specifically, redundant slices are transmitted to the full transmission band using information at the time of encoding such as inter coding and intra coding.
JP-A-9-093589 JP 2002-112265 A

  Redundant slices can separately include the region of interest as redundant slices in the encoded data. However, it is only defined as a standard, and it is not clear what part should be a redundant slice. Therefore, it is not clear what effect can be obtained by error concealment in the decoding apparatus. Arbitrary slice order is intended to prevent error propagation and is not very effective for error concealment. The method of increasing the number of divisions of the main slice is the same, and the overhead is large in terms of transmission band and code amount.

  When the attention area is set as a redundant slice by the technique disclosed in Patent Document 1, although the effect is large in an image in which the attention area has a large movement, the effect is not so great in an image in which the movement is small, and the transmission band is wasted. End up.

  In the technique disclosed in Patent Document 2, only redundant slices are transmitted to the full transmission band, and image characteristics are not considered. Therefore, there is no effect on error concealment against transmission errors.

  It is an object of the present invention to provide an encoding apparatus that solves such a problem.

In order to achieve the above object, an encoding apparatus according to the present invention is an encoding apparatus that encodes an input image, and includes an attention area selection unit that selects an attention area from the input image, and the input image. And a motion detection unit for detecting a motion amount for each region, and a detection result of the motion detection unit for the region including the region of interest selected by the region of interest selection unit. Redundant slice setting means for setting the priority of the area to be set in the redundant slice carrying the image, encoded data for encoding the plurality of areas, and encoding means for outputting the encoded data of the redundant slice; And the redundant slice setting means preferentially sets the redundant slice from an area where the motion between frames is large based on the detection result of the motion detection means .

  Another encoding apparatus according to the present invention is an encoding apparatus that encodes an input image, and includes an attention area selection unit that selects an attention area from the input image, and a plurality of areas that constitute the input image. On the other hand, the image of the region of interest is conveyed according to the detection result of the motion detection unit for the region including the region of interest selected by the region-of-interest selecting unit and the motion detecting unit that detects the amount of movement for each region. A redundant slice setting means for setting the priority of the area to be set in the redundant slice; and an encoding means for outputting the encoded data obtained by encoding the plurality of areas and the encoded data of the redundant slice. Features.

  According to the present invention, since the redundant slice for carrying the image of the attention area is set according to whether or not the attention area has been selected and the movement of the area including the attention area, the image quality at the time of error concealment in decoding is set. Will improve.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of the present invention. The image data of each screen constituting the video signal is input to the frame rearrangement division unit 12 of the encoding device 10. The frame rearrangement division unit 12 rearranges the input image data in the frame order according to the encoded picture type, and divides each frame into a matrix called a unit composed of a predetermined number of pixels. The macroblock is composed of, for example, 16 pixels × 16 pixels, 8 pixels × 8 pixels, 4 pixels × 4 pixels, or the like, depending on the encoding method.

  The subtracter 14 subtracts the later-described predicted image data from the image data from the frame rearrangement division unit 12 and outputs the difference value to the orthogonal transformation unit 16 as image residual data. The orthogonal transform unit 16 orthogonally transforms the image residual data from the subtractor 14 using discrete cosine transform, wavelet transform, and the like, and outputs transform coefficients to the quantization unit 18. The quantization unit 18 quantizes the transform coefficient from the orthogonal transform unit 16 according to a predetermined quantization parameter and outputs the quantized transform coefficient to the entropy coding unit 20.

  The quantized transform coefficients from the quantizing unit 18 are also supplied to the inverse quantizing unit 26 for local decoding, that is, for generating predicted image data. The inverse quantization unit 26 inversely quantizes the quantized transform coefficient from the quantization unit 18 and outputs a transform coefficient representative value. The inverse orthogonal transform unit 28 performs inverse orthogonal transform on the transform coefficient representative value from the inverse quantization unit 26 and outputs a local decoded value corresponding to the image residual data. The adder 30 adds the predicted image data to the output of the inverse orthogonal transform unit 28. The output data of the adder 30 corresponds to locally decoded image data.

  The output image data of the adder 30 is written in the frame memory 32. The output image data of the adder 30 is also written in the frame memory 36 after being filtered by a deblocking filter 34 for removing block noise.

  The motion detection unit 38 detects the motion vector by comparing the image data to be encoded with the image data in the frame memory 36 in units of macroblocks. The motion detection unit 38 supplies motion vector information of the detected motion vector to the inter prediction unit 42 and the entropy encoding unit 20. Although details will be described later, the motion detection unit 38 also supplies the global vector to the stream transmission unit 24.

  The intra prediction unit 40 generates predicted image data from the image data stored in the frame memory 32 by intra-frame prediction processing. On the other hand, the inter prediction unit 42 performs inter-frame prediction processing on the image data stored in the frame memory 36 based on the motion vector detected by the motion detection unit 38 to generate predicted image data.

  The switch 44 selects predicted image data from the intra prediction unit 40 or predicted image data from the inter prediction unit 42 according to a method for encoding the current image to be encoded. A coding control device (not shown) controls the switch 44 in this way. The predicted image data selected by the switch 44 is supplied to the subtracter 14 and the adder 30.

  The entropy coding unit 20 entropy codes the quantized transform coefficient from the quantization unit 18 and multiplexes the motion vector information from the motion detection unit 38. The output data of the entropy encoding unit 20 is temporarily written in the stream buffer 22 for rate adjustment and generated code amount monitoring, and is sequentially read out by the stream transmission unit 24.

  The stream transmission unit 24 outputs the encoded data read from the stream buffer 22 at a bit rate according to the target bit rate. The stream transmission unit 24 also multiplexes the global vector information from the motion detection unit 38 with the encoded data and outputs it to the transmission path. The data output from the stream transmission unit 24 to the transmission path includes various pieces of encoding information such as a quantization parameter in the quantization unit 18 and a redundant slice.

  With reference to FIG. 2, a method for obtaining a global vector will be described. FIG. 2 shows a schematic diagram of the global vector region.

  In the example illustrated in FIG. 2, the encoding target image 50 is divided into a total of nine blocks of 3 × 3. The image 52 is an image one frame before the encoding target image 50 and is blocked in the same size as the encoding target image 50.

  For example, the divided area 54 will be described as an example. A search area 58 (a part surrounded by a broken line) centering on an area 56 at the same position as the area 54 in the image 52 one screen before the encoding target image 50 is set for the divided area 54. Then, in the search area 58, the correlation with the image of the area 54 is obtained. Specifically, an area having the same size as the area 54 is set in the area 58, the area 58 is moved horizontally and vertically in units of one pixel to a plurality of pixels, and a correlation is obtained for each moved place. When the location having the strongest correlation is obtained, the direction and distance from the region 56 to the location are calculated and set as a global vector. As a correlation value representing the strength of the correlation, an absolute difference value of luminance values for each pixel is used.

  In this embodiment, correlation values are calculated for all pixels in the region, and the sum in the region is obtained. This is called a sum of absolute differences (SAD). When the SAD value is used as the correlation value, the smaller the SAD value, the stronger the correlation.

  The operation of this embodiment will be described with reference to FIGS. FIG. 3 is a flowchart of the redundant slice control operation of this embodiment. FIG. 4A shows a schematic diagram of a decoded image example in the conventional example, and FIG. 4B shows a schematic diagram of a decoded image example in the present embodiment.

  First, as described with reference to FIG. 2, the motion detection unit 38 divides the input image into a plurality of regions, and calculates a global vector for each region (S1). An attention area selection unit (not shown) selects or sets an attention area in the input image (S2). The attention area is, for example, a portion where a person, a face, or an object such as an airplane or a train is shown. For example, a region detected when a specific object such as a face is included is selected or set as a region of interest using image pattern matching or a well-known face detection technique.

  The stream transmission unit 24 outputs encoded data obtained by encoding the plurality of areas at a bit rate according to the target bit rate (S3).

  The stream transmission unit 24 determines whether the output encoded data has a margin with respect to the band of the transmission path or the band determined by the recording format (S4). If there is a margin, the process proceeds to step S5. If there is no room (S4), this flow is terminated.

  A redundant slice setting unit (not shown) determines whether or not a region of interest is included for each region divided in step S1 (S5). A redundant slice setting unit (not shown) outputs, as encoded slices, only the margin of the band as redundant slices in the descending order of the global vector for the region including the region of interest. Specifically, the redundant slice is preferentially set from the region including the region of interest such as the face, in which the motion between frames is large. In other words, the redundant slice setting unit sets a redundant slice that carries an image of the region of interest in accordance with the selection result of the region of interest selection unit and the motion detection result of the motion detection unit 38, and prioritizes the region to be a redundant slice. Set the degree. The encoded data of one or a plurality of redundant slices set in this way is transmitted following the original encoded data.

  As described above, in the present embodiment, redundant slices are output as encoded data by the amount that has a margin with respect to the bandwidth of the transmission path or the bandwidth determined by the recording format by the global vector and the region of interest. As a result, error tolerance is increased for a portion with a large movement, and the quality of the reproduced image is improved.

  FIG. 5 is a flowchart of another redundant slice control operation according to this embodiment. FIG. 6 is an explanatory diagram of global vector calculation. In FIG. 6, the screen is divided into nine regions 60 to 76, and regions 68 and 74 mainly include images of the person who is the subject.

  Similar to step S1, the motion detection unit 38 divides the input image into a plurality of regions and calculates a global vector for each region (S11). An attention area selection unit (not shown) sets an attention area in the input image (S12). The stream transmission unit 24 outputs the encoded data at a bit rate according to the target bit rate (S13).

  The stream transmission unit 24 determines whether or not the output encoded data has a margin with respect to the band of the transmission path or the band determined by the recording format (S14), and if there is a margin, proceeds to step S15. If there is no room (S14), this flow is terminated.

  The redundant slice setting unit (not shown) calculates how much the area of interest occupies in terms of area with respect to the area for calculating each global vector for each area divided in step S11 (S15). When there is only an area where the ratio of the attention area is equal to or less than the threshold value (S15), this flow ends. In other cases (S15), a redundant slice setting unit (not shown) determines whether or not a region of interest is included for each region used in step S11. The redundant slice setting unit (not shown) uses only the surplus bandwidth as encoded data as redundant slices in the descending order of the global vector for the region obtained in step S14 (in the example shown in FIG. 6, region 68 and region 74). Output.

  As described above, a redundant slice is defined according to the ratio of the region of interest to the region for calculating each global vector, and encoding is performed to the extent that there is a margin for the bandwidth of the transmission path or the bandwidth determined by the recording format. Output as data. As a result, error tolerance is enhanced for a portion with a large movement, and the quality of the reproduced image is improved.

  Although the embodiment has been described in which the global vector is obtained by SAD, the global vector can be controlled using information obtained from information (zoom, pan / tilt, AF (autofocus), etc.) from a camera (not shown). is there.

  As described above, it is possible to improve the image quality at the time of error concealment in the decoding apparatus by effectively adding redundant slices to the encoded data according to the characteristics of the image and the bandwidth of the transmission path.

  In addition, this invention is not limited to the said Example, In the range of the technical idea prescribed | regulated by a claim, it can change suitably.

It is a schematic block diagram of one Example of this invention. It is explanatory drawing of a global vector area | region. It is a flowchart of the redundant slice control operation | movement of a present Example. It is an example of the encoding image of a prior art example (A) and a present Example (B). It is a flowchart of the redundant slice control operation | movement of 2nd Example. It is explanatory drawing of the global vector area | region in 2nd Example.

Explanation of symbols

10: Encoder 12: Frame rearrangement division unit 14: Subtractor 16: Orthogonal transformation unit 18: Quantization unit 20: Entropy encoding unit 22: Stream buffer 24 Stream transmission unit 26: Inverse quantization unit 28: Inverse orthogonal Conversion unit 30: Adder 32: Frame memory 34: Deblocking filter 36: Frame memory 38 Motion detection unit 40: Intra prediction unit 42: Inter prediction unit 44 Switch

Claims (6)

An encoding device for encoding an input image,
A region of interest selection means for selecting a region of interest from the input image;
Motion detection means for detecting a motion amount for each region for a plurality of regions constituting the input image;
Redundant slice setting means for setting a priority of an area to be set as a redundant slice for conveying an image of the attention area according to a detection result of the motion detection means for an area including the attention area selected by the attention area selection means When,
Encoding means for outputting encoded data obtained by encoding the plurality of areas and encoded data of the redundant slice;
Have
The coding apparatus according to claim 1, wherein the redundant slice setting means preferentially sets the redundant slice from an area where a motion between frames is large based on a detection result of the motion detection means. 2. The encoding apparatus according to claim 1, wherein the attention area selection unit detects an area including a specific object using pattern matching, and selects the detected area as the attention area.   The encoding according to claim 1 or 2, wherein the redundant slice setting means further sets the redundant slice according to a bandwidth of a transmission path for transmitting data output from the encoding means. apparatus. A method for controlling an encoding device for encoding an input image, comprising:
A region of interest selection step of selecting a region of interest from the input image;
For a plurality of regions constituting the input image, a motion detection step of detecting a motion amount for each region;
A redundant slice setting step for preferentially setting the redundant slice from a region having a large motion between frames based on a detection result of the motion detection step for a region including the attention region selected in the attention region selection step;
An encoding step of encoding the plurality of regions and outputting the encoded data of the redundant slice;
  The control method of the encoding apparatus which has. 5. The method of controlling an encoding device according to claim 4, wherein, in the attention area selection step, an area including a specific object is detected using pattern matching, and the detected area is selected as the attention area. . 6. The encoding according to claim 4, wherein the redundant slice setting step further sets the redundant slice in accordance with a bandwidth of a transmission path for transmitting data output from the encoding means. Control method of the device.

Images