JP2008258769A - Image encoding device and control method thereof, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding device, an image encoding method and a program thereof, which can lead out an optimum prediction value at a low operation cost while maintaining a wide search range. <P>SOLUTION: A reference image 200 and a macroblock image 201 to be encoded are respectively resolution-converted and binarized. Block matching is performed between both the resolution-converted images and a search range is narrowed by a search range restriction part 206. The narrowed range is applied to a binarized image of the reference image as it is and further narrowed by a binary image searching range restriction part 208. The range narrowed by the resolution-converted image and the binarized image is finally applied to the reference image 200 and block matching between the reference image 200 and the macroblock image 201 to be encoded is performed to detect a moving vector. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving image encoding apparatus, a control method thereof, and a computer program.

  Currently, MPEG2, MPEG4, or H.264 is the mainstream as a method for compressing and encoding a moving image in units of macroblocks using the correlation between frames. In these methods, it is possible to reduce the amount of data by encoding the difference value of the pixel between the reference image before or after in time and the macroblock to be encoded. At that time, instead of always taking a difference between pixels having the same relative position in the encoding target macroblock and the frame, block matching is performed to search for a position where the difference value is minimized.

  Information indicating the difference between the encoding target macroblock and the position of the reference image is encoded separately from the difference value in the form of a motion vector. Such an encoding method is called interframe prediction. That is, it is a method of encoding by predicting how an object in the encoding target macroblock is moving at a position on the screen.

  When encoding a fast-moving subject, the encoding efficiency changes depending on the block matching range. If the subject in the macroblock to be encoded exists at a position that exceeds the block matching search range in the reference image, optimal interframe prediction becomes difficult. In such a case, inter-frame prediction with a difference value (part where the difference value is relatively small) from a subject different from the actual one is performed, or inter-frame prediction itself is not performed. On the other hand, if the range in which block matching is performed is expanded, the possibility of detecting a position where the difference value from the reference pixel is minimum increases, but the calculation cost increases. Specifically, by increasing the search range, it is inevitable that the number of block matching calculations and the memory for holding pixel data in the search range will increase.

  Therefore, the search range at the time of inter-frame prediction must be changed according to the product to which encoding is applied. For example, in the case of a product such as a video camera, if the search range is large, power consumption increases, and it is not possible to cope with long-time recording. Further, the transfer amount of the reference image becomes enormous, and the memory bandwidth becomes insufficient.

  For this reason, a general method is to narrow the search range in consideration of power consumption and bus transfer capability, or to roughly perform block matching for a wide search range. However, for the reasons described above, it cannot be said that an optimal predicted value is necessarily obtained.

  Therefore, a method is known in which a reference image is converted into a plurality of resolutions, and a search range is gradually narrowed down from a low resolution hierarchy (see Patent Document 1). According to this technique, inter-frame prediction can be efficiently performed without narrowing the search range.

Specifically, in this method, a plurality of motion vectors at points that are minimal values are detected from the low-resolution layer, and are used as the initial motion vector search value for the next layer. For example, an image first reduced to a resolution of 1/64 is prepared and a search is performed to obtain an initial search value, and then an initial search value is similarly obtained for an image further reduced to a resolution of 1/32. , And so on, gradually change the resolution. Thus, the image memory and the number of operations can be reduced by reducing the resolution.
Japanese Unexamined Patent Publication No. 7-154801

  However, the method of narrowing down the search range using an image with a reduced resolution as described above is effective when narrowing down from a wide range to some extent, but it turns out that the prediction accuracy deteriorates as the range becomes narrower. Yes. Therefore, after narrowing down the range to some extent, it is necessary to switch to a normal prediction method that does not reduce the resolution. Furthermore, if it is narrowed down too much, there is a possibility that an optimum predicted value cannot be obtained.

  The present invention has been made in view of such circumstances, and maintains an wide search range, suppresses the calculation cost, and can derive an optimal prediction value and an image encoding method. And to provide the program.

The present invention has been made to solve the above-described problem, and uses a frame image temporally different from an encoding target image as a reference image, and uses a motion vector of each macroblock image included in the encoding target image. And a video encoding device that performs inter-frame encoding processing based on the motion vector,
Reference image binarization means for binarizing the reference image to generate a reference binarized image;
Reference resolution conversion means for converting the reference image to a low resolution to generate a reference resolution conversion image;
Macroblock image binarization means for binarizing the macroblock image to generate a macroblock binarized image;
Macroblock resolution conversion means for converting the macroblock image to a low resolution to generate a macroblock resolution conversion image;
First range determining means for determining a first range to be cut out from the reference binarized image based on block matching between the reference resolution converted image and the macroblock resolution converted image;
First cutout means for cutting out the image of the first range as a reference binarized block image from the reference binarized image;
The image processing apparatus includes a motion vector calculation unit that calculates the motion vector for the macroblock image based on block matching between the reference binarized block image and the macroblock binary image.

  According to the present invention, by performing prediction using two types of images, a resolution-converted image and a binarized image, in the inter-frame prediction calculation, efficient moving image encoding can be performed while reducing the memory capacity.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration example of a moving image encoding apparatus as an image processing apparatus according to the first embodiment of the present invention. The moving image encoding apparatus 100 performs moving image encoding by a method such as MPEG2 or H264.

  In FIG. 1, the moving image encoding apparatus 100 includes the following components. First, an imaging unit 101 including a camera unit such as a lens or a CCD, a motion vector detection unit 102, a motion compensation unit 108, a subtractor 103, a DCT (direct transform) unit 104, and a quantization unit 105. Further, a variable length encoding unit 106, a recording unit 107, a recording medium 109, a code amount control unit 110, and an inverse quantization unit 111. Furthermore, an IDCT (inverse orthogonal transformation) unit 112, an adder 113, a display unit 117, and a frame memory 118 are provided.

  Furthermore, the operation of each component unit including the code amount control unit 110 is configured to be controlled by the system controller 120. The system controller 120 controls operation of the entire apparatus 100, and can also perform operation control of the entire apparatus 100 in accordance with an instruction from a user using the operation unit 121.

  A series of image signals obtained by imaging the subject by the imaging unit 101 as the imaging means in the present invention are sequentially stored in the frame memory 118 in the order of the first frame, the second frame, the third frame,. It will be stored. From the frame memory 118, for example, image data is output in the order of encoding such as the third frame, the first frame, the second frame,. The encoding system in the present embodiment includes “intra coding” for encoding only with image data in a frame and “inter encoding” for encoding including inter-frame prediction.

  First, a picture to be subjected to intra coding is called an I picture. The reason why the encoding order is different from the frame input order is to enable prediction (backward prediction) using temporally future frames. Next, there are two pictures in the encoding target image (picture) to be subjected to inter coding (interframe coding processing). The first is a P picture that performs forward prediction with one reference frame for a unit of motion compensation (macroblock). The second is a B picture that performs bi-directional prediction with up to two reference frames for a macroblock. As described above, the present embodiment describes an invention in which moving picture coding is performed using first, second, and third types of image data, which are I pictures, P pictures, and B pictures.

  When intra coding is performed, image data divided into blocks serving as coding units is read from the frame memory 118 and subjected to direct conversion by a direct conversion unit (DCT) 104. As this orthogonal transformation, DCT transformation is performed in this embodiment. A transform coefficient that is an output from the direct transform unit 104 is quantized in a quantizer (Q) 105. The quantized transform coefficient output from the quantization unit 105 is subjected to variable length coding in the variable length coding unit 106, and then a recording signal to the recording medium is generated by the recording unit 107, so that the recording medium 109 is recorded.

  The quantization amount in the quantization unit 105 is calculated by the code amount control unit 110 from the feedback of the code amount generated by the variable length coding unit (VLC) 106. Further, the quantized transform coefficient that is the output of the quantization unit 105 is inversely quantized by an inverse quantization unit (IQ) 111. The inverse orthogonal transform unit (IDCT) 112 performs an inverse orthogonal transform process to obtain a decoded image signal, and the image signal is stored in the frame memory 118.

  On the other hand, when inter coding is performed, the image data divided into macroblocks serving as coding units is read from the frame memory 118 and input to the motion vector detection unit 102. At the same time, the motion vector detection unit 102 reads a reference image, which is a temporally different frame image, from the frame memory 118, and detects a motion vector using the encoded image and the reference image.

  The motion compensation unit 108 performs motion compensation according to the motion vector and generates a predicted image. The difference between the encoded image and the predicted image is calculated by the subtracter 103, and a difference image is generated. The difference image is output to the orthogonal transform unit 104 and orthogonally transformed. Since the processing performed after the orthogonal transform unit 104 is the same as that in the case of the above-described intra coding, the description is omitted.

  Next, the operation of the motion vector detection unit 102 will be described in detail. FIG. 2 is a diagram illustrating an example of a specific configuration of the motion vector detection unit 102.

  Hereinafter, the components of FIG. 2 will be described. Reference numeral 200 denotes a reference image read from the frame memory 118. Reference numeral 201 denotes a macroblock (hereinafter referred to as MB) image cut out from a coding target image with a predetermined size (for example, 16 × 16 pixels). A reference image binarization unit 202 generates a reference binarized image by binarizing the reference image with an arbitrary threshold, and a reference image resolution conversion unit 203 generates an image with a reduced resolution of the reference image. . Reference numeral 204 denotes an MB resolution conversion unit (macroblock image resolution conversion unit) that generates an MB image (macroblock resolution conversion image) in which the resolution of the encoding target MB image is reduced. Reference numeral 205 denotes an MB image binarization unit (macroblock image binarization unit) that binarizes the encoding target MB image with the above-described arbitrary threshold value and generates an MB binarized image (macroblock binarized image). is there.

  A search range limiting unit (first range determination unit) 206 performs block matching between the reference image obtained from the reference image resolution conversion unit 203 and the encoding target MB obtained from the MB image resolution conversion unit 204. To narrow the search range. In addition, the search range limiting unit 206 determines a range (first range) to be cut out from the reference binarized image.

  Reference numeral 207 denotes a binary that extracts a reference binarized block image from the reference binarized image generated by the reference image binarizing unit 202 based on the range determined by narrowing down the search range in the search range limiting unit 206. It is an image cutout unit (first cutout unit).

  Reference numeral 208 denotes a binary image search range limiting unit (second range determination unit). First, a reference binarized block image and an MB binarized image of the encoding target MB from the MB image binarizing unit 205 are shown. To narrow the search range. In addition, the binary image search range limiting unit 208 determines a range (second range) to be cut out from the reference image 200.

  Reference numeral 209 denotes a reference image cutout unit (second cutout unit) that cuts out a reference block image from the reference image 200 based on the range determined by narrowing down the search range in the binary image search range limiting unit 208. A prediction generation unit 210 performs block matching between the encoding target MB image 201 and the reference block image to calculate a motion vector. 211 denotes a predicted value as a motion vector generated and output by the prediction generation unit 210.

  The operation of the motion vector detection unit 102 in the first embodiment based on the above configuration will be described with reference to FIG. FIG. 3 is a diagram describing an example of the size of the macroblock and the size of the search range in the present embodiment.

  The reference image 200 is binarized by the reference image binarization unit 202 without changing the resolution. Considering the case of an image having 256 bits of 8 bits, when the threshold value is set to “125”, for example, the pixel value is “0” for pixels with pixel values 0 to 124, and the pixel values are 125 to 255. For the pixels up to, the pixel value is “255”. According to this binarization, the data amount becomes 1/8. This binarization method is merely an example, and a method of binarization or a filter using any other threshold is conceivable, but the binarization method itself is not limited.

  At the same time, the reference image 200 is converted into an arbitrary resolution by the reference image resolution conversion unit 203. Resolution conversion is performed in the direction of decreasing only the resolution (decreasing the number of pixels) without changing the gradation, thereby reducing image information. As a resolution conversion method, for example, pixels may be simply thinned out. The resolution conversion method itself is not particularly limited. For example, when the resolution is converted to 1/64 (horizontal 1/8, vertical 1/8), the information amount can be reduced to 1/64. In this embodiment, an image generated in this way by resolution conversion is called a “hierarchical image”. However, since the resolution conversion is performed only in one stage in this embodiment, the hierarchy is one.

  Similarly, the encoding target MB image 201 is also binarized by the MB image binarization unit 205 by the same method as described above, and at the MB image resolution conversion unit 204, the resolution is converted at the same resolution conversion rate as described above. Converted. In the present embodiment, the resolution conversion rate is assumed to be 1/64 as an example, and the following description will be given. By the resolution conversion of 1/64, the macro block of 16 × 16 pixels is reduced to the size of 2 × 2 pixels. The size of the macroblock at this time is as indicated by 301 in FIG.

  Next, the search range limiting unit 206 performs block matching between the reference resolution conversion image generated by the reference image resolution conversion unit 203 and the MB resolution conversion image generated by the MB image resolution conversion unit 204 to perform search. Narrow the range. In FIG. 3, for example, an area 302 of 8 × 8 pixels can be used as a range for performing block matching in the reference resolution converted image. At this time, since the resolution conversion rate is 1/64, the area 302 corresponds to an area of 64 × 64 pixels in terms of the original resolution of the reference image. The area 302 is set based on the position in the reference image corresponding to the macroblock position in the encoding target image. Note that the size of the region 302 for performing block matching is merely an example, and is not limited to the above.

  The search range narrowing in the search range limiting unit 206 can be performed as follows, for example. That is, an area having the strongest correlation with the MB resolution conversion image is searched from the resolution conversion image, and a search range having a predetermined size is output based on the position of the area. The search range of the predetermined size can be, for example, a range of 4 × 4 pixels with respect to the center of the region having the strongest correlation, but this is only an example, and even if narrowed down to a range of other sizes Good. The strength of the correlation can be determined using, for example, a value (MAE) obtained by integrating the absolute difference values for each pixel between the resolution-converted image and the MB resolution-converted image, and the MAE is minimized within the detection range. Determine the center position of the region.

  The range narrowed down by the search range limiting unit 206 is given to the binary image cutout unit 207. As information given from the search range limiting unit 206 at this time, for example, the size to be cut out from the binarized reference image and the center position of the cut-out area are given. Here, the cut-out size is 4 × 4 pixels becomes 32 × 32 pixels based on the resolution conversion ratio 1/64. If the cutout size is fixed, only the information on the center position may be given.

  The binary image cutout unit 207 cuts out a reference binarized block image in a region corresponding to the range from the binarized reference image. The reference binarized block image is, for example, as shown at 304 in FIG. 3, and has a size of 32 × 32 pixels. The resolution of the image 302 is the original resolution of the encoding target image, but the pixel value is expressed in binary.

  Next, in the binary image search range limiting unit 208, between the reference binarized block image generated by the binary image cutout unit 207 and the MB binarized image generated by the MB image binarization unit 205. Block matching is performed to narrow down the search range in the binarized image. Reference numeral 303 in FIG. 3 denotes an MB binarized image.

  The search range can be narrowed down as follows, for example. That is, an area having the strongest correlation with the MB binarized image is searched from the reference binarized block image, and a search range of a predetermined size is output based on the position of the area. The search range of the predetermined size can be, for example, a range of 18 × 18 pixels with respect to the center of the region having the strongest correlation, but this is merely an example, and even if the range is narrowed to a size other than this, Good. The strength of the correlation can be determined using, for example, MAE obtained by integrating the absolute difference values for each pixel between the reference binarized block image 304 and the MB binarized image 303, and the MAE is minimum within the detection range. The center position of the area to be determined is determined.

  The range narrowed down by the binary image search range limiting unit 208 is given to the reference image cutout unit 209. As information given from the binary image search range limiting unit 208 at this time, for example, the size to be cut out in the reference image 200 and the center position of the cut-out area are given. Here, the cutout size is 18 × 18 pixels. If the cutout size is fixed, only the information on the center position may be given.

  The reference image cutout unit 209 cuts out a region corresponding to the range from the reference image 200 as a reference block image. The reference block image is, for example, as indicated by 305 in FIG. 3 and has a size of 18 × 18 pixels. The resolution of the reference block image 305 is the original resolution of the encoding target image, and the gradation of the pixel value is expressed by 256 gradations (8 bits).

  The prediction generation unit 210 performs block matching between the reference block image 306 and the MB image 201. Specifically, the reference block image 306 is searched for an area having the strongest correlation with the MB image 201, the center position of the area is determined, and the position of the MB image 201 in the encoding target image is determined. Based on the above, a motion vector can be determined. The strength of the correlation can be determined using, for example, MAE obtained by integrating the absolute difference values for each pixel between the reference block image 306 and the MB image 201, and the center position of the region where the MAE is minimum within the detection range. decide. In this way, a motion vector is finally obtained as the predicted value 211 and output to the motion compensation unit 108.

  In this embodiment, the range determined by narrowing down the search range is converted into the original resolution of the reference image every time processing is performed by the search range limiting unit 206 and the binary image search range limiting unit 208. The feature is that it becomes narrower. For example, as shown in FIG. 3, the area 302 is an area of 64 × 64 pixels in the original resolution, whereas the area 304 narrowed down by the search range limiting unit 206 is as narrow as 32 × 32 pixels. It has become. In addition, the area 305 narrowed down by the binary image search range limiting unit 208 is further narrowed to 18 × 18 pixels.

  Note that FIG. 2 shows only one configuration example of the motion vector detection unit 102. For example, the process in the prediction generation unit 210 may be omitted, and the binary image search range limitation unit 208 may calculate a motion vector based on block matching between the reference binarized image and the MB binarized image. Note that the motion vector calculation method is the same as that described in relation to the prediction generation unit 210.

  If the amount of memory is the same, it is possible to search a wider range if the resolution is lowered, but a detailed search becomes difficult because the resolution is low. In addition, the amount of data can be reduced by using a binary image (an image with a reduced gradation), but if the information representing a pixel is 1 bit, it may be erroneously detected when used widely. Get higher.

  On the other hand, in the present embodiment, a wide area is set as a search area with a low resolution, and after narrowing the range to some extent, the range is further narrowed down while reducing the data amount using a binary image at the original resolution. Thereby, it is possible to efficiently narrow down the search range using the original resolution and gradation.

  According to the above, it is possible to reduce the memory transfer amount and calculation cost of the reference image while maintaining a wide search range, and to detect an optimal motion vector.

<Second Embodiment>
Next, a second embodiment of the invention will be described with reference to FIGS. In the first embodiment described above, there is only one resolution conversion layer, that is, one resolution conversion rate, but in this embodiment, a plurality of (n) resolution conversion rates are used. An example corresponding to a hierarchy (n hierarchy) will be described.

  In FIG. 4, each component given a reference number from 200 to 211 corresponds to each component having the same reference number in FIG. 2. Reference numeral 400a denotes a reference image resolution conversion unit in the first layer, 400b denotes a reference image resolution conversion unit in the second layer, and 400c denotes a reference image resolution conversion unit in the nth layer. Similarly, 401a is an MB image resolution conversion unit in the first layer, 401b is an MB image resolution conversion unit in the second layer, and 401c is an MB image resolution conversion unit in the nth layer. Reference numeral 402a denotes a search range limiting unit in the first hierarchy, 402b a search range limitation unit in the second hierarchy, and 402c a search range limitation unit in the third hierarchy.

  In 400a to 400c and 401a to 401c, a smaller number in the hierarchy indicates a higher resolution (a higher resolution conversion rate). However, even in the first layer, the image is not converted to a resolution higher than the original resolution of the encoding target image and the reference image. Here, “n” in the nth layer is a positive integer. The reference image 200 is converted into a predetermined resolution in the reference image resolution conversion units 401a to 401c using the resolution conversion rate specified in each layer.

  In FIG. 4, it is expressed that n must take a value of 3 or more. However, this is an example, and in actuality, n may be 2 or more. The search range is narrowed first from the n-th layer, that is, the lowest resolution layer. Block matching is performed between the n-th layer reference image resolution conversion unit 400c and the n-th layer MB image resolution conversion unit 401c, and the n-th layer search range limiting unit 402c narrows down the search range in the n-th layer.

  In the first embodiment, the result is sent to the binary image cutout unit 207. In the present embodiment, the result is transmitted to the reference image resolution conversion unit in the next layer, that is, the (n-1) th layer. The reference image resolution conversion unit in the (n-1) th layer converts the resolution of the reference image for the search range obtained by narrowing down, and outputs it to the search range limiting unit in the (n-1) th layer. The search range limiting unit in the (n-1) th layer further narrows down the search range using this image and the MB image having the same resolution, and the result is further transmitted to the (n-2) th layer. This is repeated, and finally the first layer is sequentially narrowed down.

  Note that the processing in the search range limitation unit 402 of each hierarchy is the same as the processing in the search range limitation unit 202 of the first embodiment, except for the difference in resolution.

  Block matching between the reference resolution conversion image generated by the first layer reference image resolution conversion unit 400a and the MB resolution conversion image generated by the first layer MB image resolution conversion unit 401a in the first layer search range limitation unit 402a. Is done. The result of this block matching is transmitted to the binary image cutout unit 207. The subsequent operation is the same as that of the first embodiment.

  Next, the hierarchical search procedure of this embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of the relationship between the reference resolution converted image and the MB resolution converted image in the search range limiting unit when n = 2 and a two-stage resolution hierarchy is set.

  In FIG. 5, the resolution conversion rate in the second layer is 1/64 (second resolution conversion rate), and the resolution conversion rate in the first layer is 1/16 (first resolution conversion rate). That is, this corresponds to the case where n = 2 in FIG.

  In FIG. 5, 501 indicates an MB resolution converted image (second macroblock resolution converted image) in the second layer, and 502 indicates a reference resolution converted image (second reference resolution converted image) in the second layer. Show. The MB resolution converted image (second macroblock resolution converted image) 501 is obtained by converting the resolution of a 16 × 16 pixel macroblock image to 1/64, and has a size of 2 × 2 pixels. The reference resolution converted image (second reference resolution converted image) 502 is a resolution converted from a 64 × 64 pixel area to 1/64, and has a size of 8 × 8 pixels.

  Using the MB resolution converted image 501 and the reference resolution converted image 502, the second hierarchical search range limiting unit (third range determining unit) 402b narrows down the search range (third range). As a result of this, for example, a method of setting a range of a predetermined size (for example, 6 × 6 pixels) based on the center position of the area where the MAE is minimized can be considered. In this case, since the search range in the next hierarchy is constant, realization with hardware is easy. This example will be described below.

  Note that it is also possible to set the range so as to include all regions where the MAE is below a certain threshold or the correlation between matching targets is stronger than a certain threshold. Further, the range can be set so as to include an arbitrary number from the smaller MAE, or to include an area having a certain correlation with the macroblock. In these cases, the search range in the next hierarchy is variable, but a search range with higher accuracy can be formulated. In this way, the search range for the next hierarchy is limited. In any method, the search range in the next hierarchy is smaller than the search range in the current hierarchy. Note that details of other processing related to narrowing down have already been described in the first embodiment, and are omitted here.

  In the first hierarchy reference image resolution conversion unit 400a, a block image obtained by cutting out a predetermined area from the reference image and converting the resolution based on the search range obtained from the second hierarchy search range limitation unit 402b is converted into a first hierarchy search range limitation unit. Output to 403a. At this time, the search range obtained from the second hierarchy search range limiting unit 402b is a 48 × 48 pixel area in terms of the original resolution, and the resolution is converted at a resolution conversion rate of 1/16. That is, a reference resolution conversion image in the first hierarchy of 12 × 12 pixels is obtained.

  503 in FIG. 5 shows the MB resolution conversion image (first macroblock resolution conversion image) in the first layer, and 504 shows the reference resolution conversion image (first reference resolution conversion image) in the first layer. . The MB resolution conversion image 503 is a 16 × 16 pixel macroblock image obtained by converting the resolution to 1/16, and has a size of 4 × 4 pixels. Further, the reference resolution converted image 504 has a size of 12 × 12 pixels as described above.

  Using the MB resolution converted image 503 and the reference resolution converted image 504, the first hierarchy search range limiting unit 402a narrows down the search range. As a result, for example, the search range may be narrowed down to an 8 × 8 pixel search range, but this is only an example, and the size of the search range is not limited to this. Details of the processing related to the narrowing down have already been described in the first embodiment, and are omitted here.

  The binary image cutout unit 207 cuts out a predetermined area from the binarized reference image based on the search range obtained from the first hierarchy search range limiting unit 402a, and outputs it to the binary image search range limiting unit 208. To do. At this time, the search range obtained from the first hierarchy search range limiting unit 402a is a 32 × 32 pixel region in terms of the original resolution.

  As described above, according to the present embodiment, it is possible to detect the optimal motion vector while suppressing the memory transfer amount and calculation cost of the reference image while maintaining a wide search range with variable resolution.

<Third Embodiment>
Next, a third embodiment of the invention will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of the configuration of the motion vector detection unit 102 according to the third embodiment of the invention.

  In the second embodiment described above, the resolution images of the respective layers are individually generated from the reference pixels having the original resolution in each resolution conversion unit. Therefore, while conversion to a free resolution is possible, a filter for preparing each resolution-converted image is required individually, and an increase in calculation cost is inevitable. Therefore, in the present embodiment, first, the first layer reference image resolution conversion unit 600a performs resolution conversion with the highest resolution held, and the next layer performs resolution conversion using the output. This is repeated up to n layers, and each resolution conversion unit holds a resolution conversion image in advance.

In that case, it is not necessary to prepare a filter separately by making the ratio of resolution conversion between hierarchies constant, and only one type of filter may be used. For example, if the first layer reference image resolution conversion unit 600a converts the resolution to ½, and the second layer reference image resolution conversion unit 600b converts the resolution to ¼, the used filter reduces the resolution to ½. Only a filter for conversion is required. Similarly, resolution conversion of 1/8, 1/16... And 1/2 ⁿ can be performed as the hierarchy progresses. The MB image resolution conversion units 601a to 601c also perform resolution conversion using the same single filter. Since the number of pixels in the macroblock is 256 (16 × 16) pixels, resolution conversion is not performed over 1/256.

  In FIG. 6, it is expressed that n must take a value of 3 or more. However, this is an example, and in actuality, n may be 2 or more. The search range is narrowed first from the n-th layer, that is, the lowest resolution layer. Block matching is performed between the n-th layer reference image resolution conversion unit 600c and the n-th layer MB image resolution conversion unit 601c, and the n-th layer search range limiting unit 402c narrows down the search range in the n-th layer.

  The result of this narrowing down is transmitted to the next layer, that is, the reference image resolution conversion unit in the (n-1) th layer. The reference image resolution conversion unit in the (n-1) th layer cuts out a block image from the reference resolution conversion image in the (n-1) th layer based on the search range obtained by narrowing down and serves as a search range limiting unit in the (n-1) th layer. Output. The search range limiting unit in the (n-1) th layer further narrows down the search range using this image and the MB image having the same resolution, and the result is further transmitted to the (n-2) th layer. This is repeated, and finally the first layer is sequentially narrowed down.

  In FIG. 6, each component having a reference number from 200 to 211 corresponds to each component having the same reference number in FIG. 2.

  Reference numeral 600a denotes a reference image resolution conversion unit in the first layer, 600b denotes a reference image resolution conversion unit in the second layer, and 600c denotes a reference image resolution conversion unit in the nth layer. Similarly, 601a is an MB image resolution conversion unit in the first layer, 601b is an MB image resolution conversion unit in the second layer, and 601c is an MB image resolution conversion unit in the nth layer. Reference numeral 402a denotes a search range limiting unit in the first hierarchy, 402b a search range limitation unit in the second hierarchy, and 402c a search range limitation unit in the third hierarchy. Moreover, each component which attached | subjected the reference number 402a-402c respond | corresponds to each component which has the same reference number in FIG.

  In FIG. 6, reference numeral 600a is a reference image resolution conversion unit in the first layer, 600b is a reference image resolution conversion unit in the second layer, and 600c is a reference image resolution conversion unit in the nth layer. Similarly, 601a is an MB image resolution conversion unit in the first layer, 601b is an MB image resolution conversion unit in the second layer, and 601c is an MB image resolution conversion unit in the nth layer. As described above, each resolution conversion unit performs resolution conversion using the resolution conversion result in the resolution conversion unit on one layer using a common filter.

  According to the above, resolution conversion can be performed while suppressing an increase in calculation cost.

<Other embodiments>
In each of the above-described embodiments, the horizontal and vertical are described with the same magnification and the number of pixels. However, the present invention is not limited thereto, and the horizontal and vertical may have different magnification and the number of pixels.

  In each of the above-described embodiments, the processing in each component shown in FIGS. 1, 2, 4 and 6 is not limited to hardware. The function may be realized by reading a program for realizing the function of each processing unit from the memory and executing it by a CPU (Central Processing Unit).

  Further, the present invention is not limited to the configuration described above. All or some of the functions of the processes in the components shown in FIGS. 1, 2, 4, and 6 may be realized by dedicated hardware. The memory from which the CPU reads the program may be a non-volatile memory such as an HDD, a magneto-optical disk device, or a flash memory, or a recording medium such as a CD-ROM that can only be read. Furthermore, the recording medium may be a computer-readable / writable recording medium using a volatile memory other than the RAM or a combination thereof.

  The “computer-readable and writable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, and a hard disk built in a computer system. Further, it also includes a volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  A program product such as a computer-readable recording medium in which the above program is recorded can also be applied as an embodiment of the present invention. The above program, recording medium, transmission medium, and program product are included in the scope of the present invention.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes a design and the like within the scope not departing from the gist of the present invention.

It is the figure which showed an example of the structure of the image coding apparatus corresponding to the 1st Embodiment of this invention. It is the figure which showed an example of the structure of the motion vector detection part 102 corresponding to the 1st Embodiment of this invention. It is a figure which describes an example of the size of the macroblock corresponding to the 1st Embodiment of this invention, and the size of a search range. It is the figure which showed an example of the structure of the motion vector detection part 102 corresponding to the 2nd Embodiment of this invention. It is a figure which describes an example of the size of the macroblock corresponding to the 2nd Embodiment of this invention, and the size of a search range. It is the figure which showed an example of the structure of the motion vector detection part 102 corresponding to the 4th Embodiment of this invention.

Explanation of symbols

200 Reference Image 201 Encoding Target Macroblock Image 202 Reference Image Binarization Unit 203 Reference Image Resolution Conversion Unit 204 MB Image Resolution Conversion Unit 205 MB Image Binarization Unit 206 Binary Image Clipping Unit 207 Search Range Limiting Unit 208 Binary Image search range limiting unit 209 Reference image cutout unit 210 Prediction generation unit 211 Prediction value

Claims (8)

Using a frame image temporally different from the encoding target image as a reference image, a motion vector of each macroblock image included in the encoding target image is calculated, and interframe encoding processing is performed based on the motion vector. A moving image encoding device for performing,
Reference image binarization means for binarizing the reference image to generate a reference binarized image;
Reference resolution conversion means for converting the reference image to a low resolution to generate a reference resolution conversion image;
Macroblock image binarization means for binarizing the macroblock image to generate a macroblock binarized image;
Macroblock resolution conversion means for converting the macroblock image to a low resolution to generate a macroblock resolution conversion image;
First range determining means for determining a first range to be cut out from the reference binarized image based on block matching between the reference resolution converted image and the macroblock resolution converted image;
First cutout means for cutting out the image of the first range as a reference binarized block image from the reference binarized image;
A moving image code comprising: motion vector calculating means for calculating the motion vector for the macroblock image based on block matching between the reference binarized block image and the macroblock binary image Device. The motion vector calculation means includes
Based on block matching between the reference binarized block image and the macroblock binarized image, the second range that is included in the first range and is narrower than the first range is cut out from the reference image. A second range determining means for determining a range;
A second cutout unit that cuts out the image of the second range as a reference block image from the reference image;
The moving image encoding apparatus according to claim 1, wherein the motion vector for the block image is calculated based on block matching between the reference block image and the macroblock image. The reference resolution conversion means includes
First reference resolution conversion means for converting the reference image into a first reference resolution conversion image having a first resolution lower than the resolution of the reference image based on a first resolution conversion rate;
Based on a second resolution conversion ratio smaller than the first resolution conversion ratio, a second reference resolution conversion image is converted into a second reference resolution conversion image having a second resolution lower than the first resolution. A reference resolution conversion means,
The macroblock resolution conversion means includes:
First macroblock resolution conversion means for converting the macroblock image into a first macroblock resolution conversion image of the first resolution based on the first resolution conversion rate;
Second macroblock resolution conversion means for converting the macroblock image into a second macroblock resolution conversion image of the second resolution based on the second resolution conversion rate;
The moving image encoding device is:
Based on the block matching between the second reference resolution conversion image and the second macroblock resolution conversion image, the first reference resolution conversion means performs the third conversion of the reference image for resolution conversion at the first resolution. Further comprising third range determining means for determining the range of
The third range corresponds to a range wider than the first range when converted by the resolution of the reference image,
The first range determining means includes
Based on block matching between a reference block image of a first resolution generated by the first reference resolution conversion means converting the resolution of a third range of the reference image, and the first macroblock resolution conversion image, The moving image encoding apparatus according to claim 1, wherein the first range is determined. The reference resolution conversion means includes
First reference resolution conversion means for converting the reference image into a first reference resolution conversion image having a first resolution lower than the resolution of the reference image based on a first resolution conversion rate;
Second reference resolution conversion means for converting the first reference resolution conversion image into a second reference resolution conversion image having a second resolution lower than the first resolution based on the first resolution conversion rate. And
The macroblock resolution conversion means includes:
First macroblock resolution conversion means for converting the macroblock image into a first macroblock resolution conversion image of the first resolution based on the first resolution conversion rate;
Second macroblock resolution conversion means for converting the first macroblock resolution conversion image into the second macroblock resolution conversion image of the second resolution based on the first resolution conversion rate;
The moving image encoding device is:
Third range determining means for determining a third range cut out from the first reference resolution conversion image based on block matching between the second reference resolution conversion image and the second macroblock resolution conversion image. Prepared,
The third range corresponds to a range wider than the first range when converted by the resolution of the reference image,
The first range determining means includes
For block matching between a reference block image having a first resolution cut out from a third range of the first reference resolution conversion image by the first reference resolution conversion means and the first macroblock resolution conversion image. 3. The moving image encoding apparatus according to claim 1, wherein the first range is determined based on the first range.   At least one of the first to third ranges is determined as a predetermined range including the center of an area determined to have the strongest correlation between matching targets in the block matching. The moving picture encoding device according to claim 3 or 4.   At least one of the first to third ranges is determined so as to include an area in which the correlation between matching targets is determined to be stronger than a predetermined threshold in the block matching. The moving image encoding apparatus according to claim 3 or 4. Using a frame image temporally different from the encoding target image as a reference image, a motion vector of each macroblock image included in the encoding target image is calculated, and interframe encoding processing is performed based on the motion vector. A method for controlling a moving image encoding device, comprising:
A reference image binarization step for binarizing the reference image to generate a reference binarized image;
A reference image resolution conversion step of converting the reference image to a low resolution to generate a reference resolution conversion image;
A macroblock image binarization step for binarizing the macroblock image to generate a macroblock binarized image;
A macroblock image resolution conversion step of converting the macroblock image to a low resolution to generate a macroblock resolution conversion image;
A first range determining step for determining a first range cut out from the reference binarized image based on block matching between the reference resolution converted image and the macroblock resolution converted image;
A first cutout step of cutting out the image of the first range as a reference binarized block image from the reference binarized image;
A moving image code comprising: a motion vector calculating step of calculating the motion vector for the macroblock image based on block matching between the reference binarized block image and the macroblock binarized image Control method.   A computer program for causing a computer to function as the moving picture encoding apparatus according to any one of claims 1 to 6.

Images