JP2006345186A - Moving vector detection processing time calculating device and method, moving vector detecting device, and image encoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently and precisely detect moving vectors. <P>SOLUTION: Difficulty levels in movement compensation of respective areas in a picture are calculated from an input frame and a reference frame. Then processing times are adaptively allocated to the respective areas according to the calculated difficulty levels in movement compensation and the processing performance of a moving vector detection processing section. In each area, a search range or the like of a moving vector is adaptively varied in accordance with the given processing time. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a motion vector detection technique, and more particularly to a motion vector detection technique that can be suitably applied to an image coding technique using inter-frame prediction using motion compensation.

  In order to effectively use digitalized images (image data), it is necessary to efficiently store and transmit images, and as a result, compression encoding of image data is essential. An outline of a JPEG (Joint Photographic Experts Group) encoding method widely used as a compression encoding method for image data is as follows.

  First, image data to be compressed is divided into a plurality of blocks (encoded blocks), and image data of each macroblock is converted into orthogonal transform coefficients composed of a DC component and an AC component using orthogonal transform. The orthogonal transform coefficient is quantized to reduce the code amount, and the quantized orthogonal transform coefficient is entropy-coded to obtain a final encoded result.

  When quantization is performed, low quantization components are used for sensitive low-frequency components, and large quantization values are used for insensitive high-frequency components, effectively suppressing visual degradation. The amount of codes can be reduced. As another image compression coding method, an inter-frame prediction moving image coding method such as MPEG-2 (Motion Picture Experts Group Phase 2) is known. This method detects motion between images, compensates for motion, and then performs orthogonal transform, quantization, and entropy coding. It effectively reduces the amount of code using the correlation between frames of moving images. can do.

  FIG. 8 shows a conceptual diagram of inter-frame prediction in the MPEG-2 system. The current frame is divided into, for example, blocks of 16 pixels × 16 lines (referred to as a prediction block or a motion compensation unit). The motion vector is detected with respect to a reference frame which is moving image data obtained by decoding moving image data encoded in the past, for example. Specifically, based on the region at the position corresponding to the processing target block of the current frame in the reference frame, the surrounding pixel group including the region is used as a motion search range, and the relative position of the region with the highest degree of correlation is determined. Ask. This degree of correlation is obtained by comparing the processing target block of the current frame and a region (hereinafter referred to as a reference region) having the same size as the current processing target block in the motion search range while shifting. For the measurement of the degree of correlation, the sum of absolute differences of pixels at positions corresponding to each other in the region is often used, and it is evaluated that the correlation is higher as the value of the sum of absolute differences is smaller.

FIG. 7 shows a configuration example of a moving picture encoding apparatus using such inter-frame prediction.
7 includes a subtractor 701, an orthogonal transformer 702, a quantizer 703, a scan processor 704, an entropy encoder 705, an inverse quantizer 706, an inverse orthogonal transformer 707, and an adder. 708, a storage device 709, a motion vector detector 710, and a reference region generator 711. Note that the size of the encoded block and the size of the prediction block that performs motion compensation do not necessarily match. However, in order to facilitate explanation and understanding, the description will be made assuming that the encoded block and the predicted block are the same block. .

  In the case of performing intraframe encoding in which encoding is performed using only information in the encoding target frame, the subtraction processing in the subtractor 701 is not performed, and the original image data is input to the orthogonal transformer 702 as it is. On the other hand, when performing inter-frame predictive coding using temporally different reference frames, the subtracter 701 subtracts the reference image data from the reference region generator 711 from the original image data, and the subtraction result (prediction) Error) to the orthogonal transformer 702.

  The orthogonal transformer 702 performs orthogonal transformation on the input data from the subtractor 701 in units of blocks, and outputs the obtained orthogonal transformation coefficient to the quantizer 703. The quantizer 703 quantizes the orthogonal transform coefficient using the quantization table value corresponding to the position in the block and the quantization scale value of the block, and converts all the quantized orthogonal transform coefficients into the scan processor 704 and the inverse quantum. Output to the generator 706. The scan processor 704 performs scan processing such as zigzag scanning according to the encoding mode. The entropy encoder 705 entropy encodes the output of the scan processor 704, and outputs a code as a compression encoding result.

  In the moving picture coding apparatus in FIG. 7, local decoding processing is performed using an inverse quantizer 706 and an inverse orthogonal transformer 707 in order to perform motion vector detection and motion compensation.

  The inverse quantizer 706 performs inverse quantization on the quantized orthogonal transform coefficient of the target block using the quantization scale value of the target block, and outputs the inverse quantized coefficient to the inverse orthogonal transformer 707. The inverse orthogonal transformer 707 performs inverse orthogonal transformation on the inversely quantized orthogonal transformation coefficient, and outputs the decoded prediction error to the adder 708. The adder 708 adds the prediction value output from the reference block generator 711 and the decoded prediction error from the inverse orthogonal transformer 707, and stores the addition result in the storage device 709 as a decoded reference image. The motion vector detector 710 performs motion vector detection for the next processing target block from the processing target block and the reference frame stored in the storage device 709 as shown in FIG. The reference block generator 711 generates reference block data for performing motion compensation from the motion vector detected by the motion vector detector 710 and the reference frame stored in the storage device 709, and outputs the reference block data to the subtractor 701. To do.

  FIG. 9 is a diagram showing, in time series, processing performed by each unit when the moving picture encoding apparatus using such inter-frame prediction processes one frame of image data. Here, the processing target frame is divided into N blocks for processing, and numerals 1,, N-1, and N in the figure indicate the number of blocks being processed.

  As is clear from FIG. 9, the motion vector detector and the processing units other than the motion vector detector such as the orthogonal transformer and the quantizer process different blocks at the same time and operate in parallel. A motion vector detected by the motion vector detector is used to perform processing such as orthogonal transformation and entropy coding, and during that time, the motion vector detector performs processing of the next block. In such a system, the processing speed of the motion vector detector determines the processing speed of the entire encoding device. Therefore, it is possible to improve the processing speed of the entire coding apparatus by operating the motion vector detector efficiently.

  The processing time of the motion vector detection circuit can be determined from the detection method of the motion vector and the processing performance of the motion vector detection circuit. For example, when the motion vector detection circuit is realized by a processor type circuit, the processing time required for motion vector detection is obtained from the number of instructions for performing motion vector detection, the number of cycles per instruction, and the operating frequency of the processor. be able to. When motion vector detection is realized by a dedicated processing circuit, the processing time can be obtained from the throughput and operating frequency of the processing circuit.

  Also, the wider the search range of motion vectors and the greater the number of searches, the better the coding efficiency, but the longer the processing time, there is a correlation between the processing time of motion vector detection and the coding efficiency. The current general inter-frame prediction video encoding apparatus generally applies the same motion vector detection / motion compensation method to each block, which is a unit for performing motion compensation in a frame. Accordingly, the motion vector detection processing time for each block is considered to be substantially constant, and the processing time for motion vector detection is determined from the frame rate and frame size of the moving image.

  It has also been proposed to change the motion vector detection method. Patent Document 1 obtains an encoding target time and an actual encoding time after the end of encoding of one frame, and calculates the processing time of the next frame from the deviation. A motion compensation prediction method that determines a motion vector detection method and realizes encoding in real time has been proposed. In Patent Document 2, the calculation amount is optimized by providing a threshold for motion vector candidates in the motion vector search process, and the excess calculation amount is added to the calculation amount necessary to obtain the next motion vector. There has been proposed a video encoding method that widens the search range of motion vectors.

  In addition, recently established H.264 In the H.264 video standard encoding system, in order to further improve the encoding efficiency, a system for adaptively changing the size of a unit region (predicted block or motion compensation unit) for performing motion compensation is defined. FIG. The conceptual diagram of the prediction between frames in a H.264 encoding system is shown. In the MPEG-2 system shown in FIG. 6, the size of the motion compensation unit is determined to be fixed at 16 pixels × 16 lines. In the H.264 encoding method, as shown in the figure, a 16 pixel × 16 line macroblock, which is an encoding unit, is divided by various division methods, and a motion compensation method using the divided area as a motion compensation unit is defined. The degree of freedom of compensation is increasing.

There is a correlation between the size of the motion compensation unit and the code amount. The constituent elements of the code amount in the moving image encoding method are classified into the following three.
1) Code amount of orthogonal transform coefficient 2) Code amount of motion vector 3) Code amount of header information other than motion vector

  Although the code amount of the orthogonal transform coefficient can be reduced by increasing the quantization scale value, there is a problem that the image quality deteriorates. The code amount of the motion vector is uniquely determined from the number of motion vectors, the result of motion vector detection, and the motion vectors in the surrounding macroblocks. Also, the header information has a smaller code amount than the orthogonal transform coefficient and motion vector, and it is generally difficult to reduce the header information. Therefore, whether to improve the coding efficiency without degrading the image quality depends on how efficiently the code amounts of the orthogonal transform coefficient and the motion vector are reduced.

JP 2002-112274 A JP 2004-048512 A

  In the method described in Patent Document 1, the processing time for detecting the motion vector of each block is determined without taking into account the motion information for each block in the moving image frame. For this reason, there is a problem that the processing time is wasted in a certain block, and the processing time is insufficient in another block, so that the code amount cannot be reduced sufficiently. For example, an optimum motion vector can be detected at an early stage of motion vector detection in a block having almost no motion, so that the processing time can be increased or decreased (the number of motion vector searches is small or large). The coding efficiency does not change much. On the other hand, in blocks with intense motion, the longer the processing time, that is, the wider the motion vector search range and the greater the number of searches, the more likely it is that an optimal motion vector can be detected. Efficiency is improved.

  In the method described in Patent Document 2, information on the entire screen is not taken into consideration. Therefore, when there is no movement on the entire screen, the processing time of the entire frame is shortened, and the processing time may be wasted. For example, it is difficult to realize an adaptive process such as “reducing the processing time of a region with little motion only when another block has a portion with high motion”.

  H. With respect to the H.264 encoding method, a method for determining a motion compensation unit that efficiently reduces both the orthogonal transform coefficient and the motion vector has not been established. One technique is H.264. A method may be considered in which coding is performed for all motion compensation units defined in the H.264 coding scheme, and the motion compensation unit with the smallest code amount is finally determined as the optimum motion compensation unit. However, in such a full-search method, the processing increases, so the circuit scale becomes enormous to realize real-time encoding, and real-time encoding cannot be realized in a limited circuit scale. End up.

  The present invention has been made in view of such problems of the prior art, and an object thereof is to enable detection of a motion vector that contributes to further improvement in coding efficiency.

  According to the present invention, the above-described object is to provide a motion vector that determines, for each region, a time for motion vector detection processing for detecting the motion vector of the input frame image with respect to the reference frame image for each region constituting the input frame image. A detection processing time calculation device, comprising: a motion compensation difficulty level calculating means for determining a motion compensation difficulty level for an area of an input frame image and a corresponding reference frame image area; and the entire input frame image A processing time for determining a motion vector detection processing time for each of the regions by assigning a given motion vector detection processing time to a region having a large motion compensation difficulty so as to be longer than a region having a small motion compensation difficulty This is achieved by a motion vector detection processing time calculation device characterized by having a determining means.

  In addition, the object described above is a motion having the motion vector detection processing time calculation apparatus according to the present invention and a detecting means for detecting a motion vector in the region by equally applying the processing time determined for the region in the region. It is also achieved by a vector detection device.

  The above-described object is also achieved by an encoding apparatus that performs motion compensation interframe predictive encoding of an input frame image using a motion vector detected by using the motion vector detection apparatus of the present invention.

  With the above configuration, according to the present invention, an efficient frame is obtained by obtaining a motion compensation difficulty level for each area of an input frame image and determining a motion vector detection method and detection range according to the motion compensation difficulty level. It can contribute to inter prediction video coding.

<< First Embodiment >>
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a processing time calculator 210 of a motion vector detector included in an inter-frame prediction video encoding apparatus according to an embodiment of the present invention. FIG. 15 is a flowchart for explaining motion vector detection method determination processing in the frame predictive video encoding apparatus according to this embodiment.

  The processing time calculator 210 includes a subtractor 101, a spatial differentiator 102, an intra-region maximum motion compensation difficulty level calculator 103, a storage device 104, and a processing time calculator 105. The subtractor 101 calculates a difference value between the reference frame (reconstructed frame) output from the storage device 209 and the input frame, and outputs the difference value to the spatial differentiator 102 (S101). In the present embodiment, it is assumed that the motion compensation difficulty level calculation and the motion vector detection are performed using only the luminance value component of the moving image data. The spatial differentiator 102 performs a spatial differential operation on the difference value in order to extract a change in the difference, and outputs the spatial differential value of the difference at the pixel position as a motion compensation difficulty level for each pixel (S103). Here, the spatial differentiator 102 employs a method of performing spatial differentiation operation in the horizontal direction and the vertical direction independently, but spatial differentiation can be performed by any method capable of extracting a change in difference.

  The intra-region maximum motion compensation difficulty calculator 103 stores, in the storage device 104, the maximum value in a predetermined region among the input motion compensation difficulty for each pixel as the motion compensation difficulty of the region ( S105). This "area" is a unit that gives a processing time, and there are a plurality of "areas" in the frame. For example, "4 macroblocks (encoding unit and motion compensation unit in MPEG-2 system) are set as one" area "". Can be arbitrarily set according to the encoding method or the like. If this “region” becomes small, it is possible to assign a fine processing time, but the calculation for calculating the processing time becomes complicated. Conversely, by obtaining the motion compensation difficulty level in units of “areas” that are larger than the macroblock, it is possible to calculate the processing time for each area at a higher speed and allocate the optimum processing time to each area.

When the motion compensation difficulty level for all “regions” in the frame is calculated (S107) and stored in the storage device 104, the processing time calculator 105 reads the motion compensation difficulty for each “region” read from the storage device 104. Using the degree DMC _ij , the processing time TMC _ij for detecting the motion vector in each “region” is calculated as in the following equation (1) (S109, S111). Here, the subscripts i and j indicate positions (for example, numbers in the vertical and horizontal directions) in the frame of each region.
TMC _ij = (DMC _ij * LMC _total ) / ΣDMC _ij (1)

Here, LMC _total is the number of cycles per frame allowed to guarantee the frame rate (for example, 30 frames / second) of a moving image, and is a variable determined by the processing performance of the motion vector detection processing unit described above. . The denominator ΣDMC _ij represents the sum of the motion compensation difficulty levels of each “region” in the frame (motion compensation difficulty level for the entire frame). In this way, the number of cycles per frame determined from the processing performance of the motion vector detection unit is proportionally distributed to each region according to the degree of motion compensation difficulty, thereby reducing the processing time for motion vector detection in each region. Has been decided.

  Generally, whether or not there is motion in the screen is determined from a motion vector, but it is difficult to use motion vector information in real-time encoding. In this embodiment, instead of using the motion vector information, the motion compensation difficulty level of the entire frame and each region is obtained from a simple calculation using the processing target frame and the frame of the reconstructed moving image. It can also be used in

  In the present embodiment, specifically, the motion compensation difficulty level of the entire frame and each region is obtained by simple calculation of difference calculation and spatial differentiation with respect to the difference value. In general, in a region where there is no motion or very little, the spatial differential value of this difference is close to zero, and conversely, this value increases as there is motion. Therefore, an area composed of pixels having a small spatial differential value of the difference is an area where there is little motion, and it is considered that an optimal motion vector can be easily detected by a short-time motion vector detection process (hereinafter, such an area is referred to as an area). It is said to be an area where motion compensation difficulty is low).

  On the other hand, since a large amount of motion is generated in a region including a pixel having a large spatial differential value of the difference, it is considered that the region should be improved in coding efficiency by detecting a motion vector over a processing time (hereinafter, referred to as “encoding efficiency”) Such a region is called a region having a high degree of motion compensation difficulty). Thus, in the present embodiment, the motion compensation difficulty level can be considered as information representing the magnitude of motion. Of course, if there is a correlation with the motion compensation difficulty level, that is, the motion vector detection difficulty level, the motion guarantee difficulty level may be obtained using a factor other than the motion magnitude.

  This motion compensation difficulty level is a value that should be calculated for the entire frame. However, if the motion compensation difficulty level is complicated when the frame size of the moving image is large, the processing speed of the entire coding processing apparatus will be reduced. I will. However, it is possible to calculate the motion compensation difficulty level of the entire frame at high speed by realizing the calculation of the motion compensation difficulty level with a simple calculation such as calculating the difference and applying spatial differentiation as in this embodiment. Become.

  Then, the processing time allowed for the motion vector detection processing of each region is determined from the motion compensation difficulty level obtained for each region and the entire frame and the processing performance of the motion vector detection unit. In areas where the degree of motion compensation difficulty is high, searching for motion vectors over the processing time increases the possibility of more accurate motion vector detection than before, and as a result, improved coding efficiency is expected. Is done. In addition, since it is not necessary to increase the processing time per frame, the present invention is particularly suitable for applications that require a processing speed such as real-time encoding.

  FIG. 2 is a block diagram when the processing time calculator of the motion vector detector shown in FIG. 1 is applied to an intra-frame predictive video encoding device. As shown in the figure, a subtracter 201, an orthogonal transformer 202, Quantizer 203, scan processor 204, entropy encoder 205, inverse quantizer 206, inverse orthogonal transformer 207, adder 208, storage device 209, motion vector detection processing time calculator 210, motion vector detector 211 and a reference region generator 212.

  In addition, since each structure of 201-209 and 212 may be the same as each structure of 701-709 and 711 in FIG. 7, description here is abbreviate | omitted.

  In FIG. 2, the motion vector detection processing time calculator 210 calculates the motion compensation difficulty level for each “region” of the input frame from the reconstructed frame and the input frame read from the storage device 209 and Calculate the motion compensation difficulty for the entire frame, which is the sum. Then, using these values and the processing performance value of the motion vector detector determined in advance, the processing time of motion vector detection in each region is determined based on the equation (1), and output to the motion vector detector 211. (S101 to S111).

  The motion vector detector 211 determines a motion vector detection method using the threshold values T1 and T2 (T1 <T2) from the given processing time (S113).

FIG. 3 is a diagram schematically illustrating an example of a motion vector detection method determination method in the motion vector detector 211. In the example of FIG. 3, one of three types of vector detection schemes having different motion vector search ranges is determined according to the magnitude relationship between the processing time of each given area and the threshold value. As shown in FIG. 3, motion vector detection is performed for a wider motion vector search range as the given processing time TMC _ij is longer.

  In this way, when one is determined from motion vector detection methods that differ only in the motion vector search range, the motion vector search range and the time required for the search process are obtained, and thresholds T1 and T2 can be determined based on the time. it can.

FIG. 4 is a diagram showing, in time series, processing performed by each unit when the moving picture encoding apparatus shown in FIG. 2 processes one frame of image data.
As shown in FIG. 4, in the moving picture encoding apparatus of the present embodiment, the processing time calculator 210 of the motion vector detector determines the motion compensation difficulty level of the processing target frame before processing the processing target frame. . Based on the determination result, the motion vector detection processing time in each “region” of the frame is allocated so as to improve the encoding efficiency. FIG. 4 shows a case where the “region” giving the motion compensation difficulty level is equal to the macroblock.

  In the same FIG. 9 explaining the conventional moving image coding apparatus, the processing time of the motion vector detector for each “region” is constant, whereas in FIG. It is shown that the processing times are different. In addition, the motion vector detection processing time calculator 210 can operate in parallel with other processing units such as the motion vector detector 211 and the orthogonal transformer 202, thereby reducing the processing speed of the encoding device. It can be seen that the optimum processing time can be determined for each region.

  In this embodiment, the determination of a different motion vector detection method is realized by adaptively changing the motion vector search range corresponding to a given processing time. However, the motion vector detection method to be selected according to the processing time is not limited to a method in which only the motion vector search range is different. For example, the number of motion vectors defined per unit area as used in the MPEG-4 system may be varied depending on the processing time.

  In any case, the threshold values T1 and T2 may be determined based on the processing time required when applying each candidate of the motion vector detection method. It is also possible to select one from four or more types. In this case, the number of thresholds may be increased.

  The effects of this embodiment will be further described with reference to FIG. 6 showing the processing time for each region in the frame and the amount of code generated. As shown in FIG. 6A, since the processing time of each region is constant in the prior art, a highly accurate motion vector can be detected within the processing time in a region with small motion, and the code amount is small, but it is constant. However, the code amount could not be reduced so much in a region where the motion was intense so that a highly accurate motion vector could not be detected within this processing time.

  On the other hand, according to the present embodiment, it is possible to efficiently use the processing time given to the entire frame by allocating a longer processing time to an area where the movement is intense than an area where there is little movement. Therefore, it is possible to improve the accuracy of motion vectors detected in areas with large motion while suppressing the influence on the coding amount of areas with low motion, which can detect highly accurate motion vectors with less processing time. Encoding efficiency can be improved. Furthermore, since variation in motion vector detection accuracy is suppressed, it is possible to suppress variation in code amount in each region in the frame.

  As a result, as shown in FIG. 6B, the code amount of each region approaches so that the code amount is constant, and it is easy to control when performing the code amount control in the screen, and as a result, further higher image quality is realized. It becomes possible. In addition, it is possible to determine the processing time of motion vector detection in each region at high speed, and a configuration suitable for real-time encoding can be achieved.

  Further, expandability can be given as a further feature of the present embodiment. The method for determining the processing time of each region in this embodiment is independent of the motion vector detector implementation, and can be applied when the motion vector detector is implemented by a processor or a dedicated circuit. is there. Furthermore, the processing performance of the motion vector detection processing unit can be changed according to the implementation form. Therefore, by replacing the motion vector detector with a higher-performance one in the inter-frame prediction video encoding apparatus and changing the processing performance value of the motion vector detection processing unit, it is possible to further improve the image quality with the same configuration. .

<< Second Embodiment >>
In the present embodiment, the motion vector detection processing time calculator 210 used in the first embodiment is used to determine the size of a motion compensation unit for detecting a motion vector. In an encoding scheme in which the size of a motion compensation unit is variable, such as the H.264 encoding scheme, it is possible to appropriately set a motion compensation unit.

  FIG. 11 is a block diagram of an inter-frame predictive video coding apparatus according to the second embodiment of the present invention, and the same reference numerals are used for the same components as those of the first embodiment shown in FIG. The description is omitted. As is clear from the comparison between FIG. 2 and FIG. 11, in the encoding apparatus according to the present embodiment, motion compensation for determining the size of the motion compensation unit from the processing time output from the processing time calculator 210 for motion vector detection. A unit calculator 112 is provided.

In the inter-frame prediction video encoding apparatus according to the present embodiment, the motion vector detection processing time calculator 210 determines the motion compensation difficulty (information that gives the degree of motion) for 4 macroblocks (32 pixels × 32 lines). The processing time TMC _ij for each “region” is calculated from the equation (1) in the same manner as in the first embodiment _except that the “region” is given.

  In general, in a region where there is no motion or very little, the spatial differential value of this difference is close to zero, and conversely, this value increases as there is motion. Therefore, an area composed of pixels with a small spatial differential value of the difference is an area with little motion, and even with a large motion compensation unit, the code amount of the orthogonal transform coefficient can be sufficiently reduced by motion compensation, and Since the number of motion vectors defined is small, the code amount of the motion vector is also small. Conversely, if the unit of motion compensation is reduced, the code amount of the orthogonal transform coefficient can be sufficiently reduced, but the code amount of the motion vector may be increased. Therefore, it can be said that a larger motion compensation unit is better in a region where there is little motion.

  On the other hand, a large amount of motion occurs in a region including pixels with a large spatial differential value of the difference. If the motion compensation unit is large and the motion compensation unit is large, the code amount of the motion vector does not increase so much. However, since fine motion compensation cannot be performed, the residual after motion compensation is large and the code amount of the orthogonal transform coefficient becomes enormous. . Here, by reducing the motion compensation unit, the code amount of the motion vector increases, but the code amount of the orthogonal transform coefficient can be reduced more than the increase in the motion vector code amount. Therefore, it can be said that the smaller the motion compensation unit is better in the region where the motion is large.

  Also, from the viewpoint of the relationship between the motion compensation unit and the processing time, the smaller the motion compensation unit, the more motion vectors per frame, and motion vector detection is performed by the number of motion vectors that have increased. Processing time becomes longer. In order to perform real-time encoding, there is a limit to reducing the motion compensation unit, and the motion compensation unit must be determined in consideration of the processing performance of the motion vector detector.

  In view of such a situation, in the present embodiment, as shown in FIG. 12, a large amount of processing time is allocated to a portion where the extracted motion information is large to reduce the unit of motion compensation, and the code amount of the orthogonal transform coefficient is set. Reduce.

On the other hand, when the extracted motion information is small, the processing time is allocated to be small to increase the unit of motion compensation, and the code amount of both the orthogonal transform coefficient and the motion vector is reduced.
By changing the motion compensation unit according to the motion information extracted in this way, it is possible to realize an inter-frame prediction video encoding device with high encoding efficiency in real time.

The motion compensation unit calculator 112 uses the processing time TMC _ij given from the motion vector detection processing time calculator 210 and predetermined threshold values T1, T2, and T3 (T1 <T2 <T3) as follows. The motion compensation unit in the “region” is determined (FIG. 13).
When TMC _ij <T1: T1 ≦ TMC _ij <T2 with 32 pixels × 32 lines as a motion compensation unit: When T2 ≦ TMC _ij <T3 with 16 pixels × 16 lines as a motion compensation unit: 8 pixels × When T3 ≦ TMC _ij with 8 lines as motion compensation unit: 4 pixels × 4 lines with motion compensation unit As described above, each “region” is set so that the motion compensation unit becomes smaller as the given processing time increases. ”Is determined.

  The magnitude of the motion compensation unit determined by the motion compensation unit calculator 112 is output to the motion vector detector 113 and the reference region generator 212. The motion vector detector 113 performs a motion vector search in a given motion compensation unit, and outputs a motion vector determined to have the highest degree of correlation to the reference region generator 212.

  H. In the H.264 encoding method, the maximum motion compensation unit is a macroblock (16 pixels × 16 lines), but in this embodiment, the motion compensation unit is defined as 32 pixels × 32 lines (including four macroblocks inside). In such a case, motion vector detection is performed with 32 pixels × 32 lines. Then, the motion vector determined in units of 32 pixels × 32 lines is applied to the four internal macroblocks to determine the motion vector of each macroblock. In this way, motion compensation units not defined in the standard encoding method are also defined, and the number of motion vectors is reduced to speed up the processing. As a result, the processing time is allocated to the block with more intense motion, and the encoding efficiency can be improved as in the first embodiment.

  The reference region generator 212 generates a reference moving image for performing motion compensation using the determined motion vector and motion compensation unit, and outputs it to the subtractor 201.

FIG. 15 is a flowchart of the motion compensation region size determination process in the second embodiment. This process corresponds to a process of executing step S201 for calculating the size of the motion compensation region instead of step S113 in the first embodiment.
FIG. 14 is a diagram showing, in time series, processing performed by each unit when the moving image encoding apparatus illustrated in FIG. 11 processes one frame of image data.

As shown in FIG. 14, in the moving picture encoding apparatus of the present embodiment, the processing time calculator 210 of the motion vector detector determines the motion compensation difficulty level of the processing target frame before processing the processing target frame. . Based on the determination result, the motion vector detection processing time in each “region” of the frame is allocated so as to improve the encoding efficiency. Based on this processing time, the motion compensation unit calculator 112 determines the size of the motion compensation unit in each “region”. This series of motion compensation unit determination processing can be operated in parallel with other processing units such as the motion vector detector 113 and the orthogonal transformer 202, and without reducing the processing speed of the encoding device. The optimal motion compensation unit for the region can be determined.
FIG. 14 shows a case where the “region” giving the motion compensation difficulty level is equal to the macroblock.

  As described above, according to the present embodiment, the processing time of the motion vector detection processing unit of each region is determined according to the motion information of the moving image data, and the size of the motion compensation unit is determined using the determined processing time. By adaptively changing the coding efficiency, the coding efficiency can be improved, and the same effect as in the first embodiment can be realized. In particular, in an encoding method in which the size of a motion compensation unit is variable, it is possible to realize efficient encoding by applying this embodiment.

<< Other Embodiments >>
FIG. 5 is a block diagram illustrating a configuration example of a video camera to which the inter-frame predictive video encoding device according to the first or second embodiment of the present invention is applied.
The video camera 500 includes a lens 501, an image sensor 502, a signal processing circuit 503, a storage device 504, a moving image encoding device 505, and a recording medium 506 as illustrated. An image input from an imaging target is input to the image sensor 502 through the lens 501 and converted into an image signal. The converted video signal is subjected to color space conversion, noise removal, and the like in the signal processing circuit 503.

  The video signal output from the signal processing circuit 503 is stored in the temporary storage device 504 in order to adjust the processing speed. The video signal read from the storage device 504 is input to the moving image encoding device 505 and output as encoded data. The output encoded data is recorded on a recording medium 506 such as a semiconductor memory, an optical / magnetic disk, or a magnetic tape.

  Since the encoding efficiency is improved by applying the inter-frame prediction moving image encoding apparatus according to the present embodiment as the moving image encoding apparatus 505, the same recording time is required for a recording medium 506 having a limited capacity. A moving image with high image quality can be recorded for a longer time if the image quality is the same. In addition, since the configuration enables high-speed real-time encoding, the capacity of the temporary storage device 504 can be reduced, and a video camera can be configured at a lower cost.

  In addition, a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly or to a system or apparatus having a computer that can execute the program using wired / wireless communication. The present invention includes a case where an equivalent function is achieved by a computer executing the supplied program.

  Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, a magnetic recording medium such as a flexible disk, a hard disk, a magnetic tape, MO, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-R, DVD- There are optical / magneto-optical storage media such as RW, and non-volatile semiconductor memory.

  As a program supply method using wired / wireless communication, a computer program forming the present invention on a server on a computer network, or a computer forming the present invention on a client computer such as a compressed file including an automatic installation function A method of storing a data file (program data file) that can be a program and downloading the program data file to a connected client computer can be used. In this case, the program data file can be divided into a plurality of segment files, and the segment files can be arranged on different servers.

  That is, the present invention includes a server device that allows a plurality of users to download a program data file for realizing the functional processing of the present invention on a computer.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to the user, and key information for decrypting the encryption for a user who satisfies a predetermined condition is provided via a homepage via the Internet, for example. It is also possible to realize the program by downloading it from the computer and executing the encrypted program using the key information and installing it on the computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on an instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU of the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

It is a block diagram which shows the structural example of the processing time calculation circuit of the motion vector detection which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the inter-frame prediction moving image encoder which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the determination method of the motion vector detection method of the frame participation side road which concerns on 1st Embodiment of an encoding apparatus. It is the figure which showed the process which each part performs in time series, when the moving image encoder which concerns on 1st Embodiment processes the image data of 1 frame. It is a block diagram which shows the structural example of the video camera to which the inter-frame prediction moving image encoder which concerns on embodiment of this invention is applied. It is a figure which shows the relationship between the processing time with respect to each area | region in a flame | frame, and the code amount to generate | occur | produce in the moving image encoder which concerns on embodiment, and the conventional moving image encoder. It is a block diagram which shows the structural example of the conventional inter-frame prediction moving image encoder. It is a figure which shows typically the inter-frame prediction method in MPEG-2 system. It is the figure which showed the process which each part performs when the conventional moving image encoding apparatus processes the image data of 1 frame in time series. H. It is a figure which shows typically the inter-frame prediction method in a H.264 encoding system. It is a block diagram which shows the structural example of the inter-frame prediction moving image encoder which concerns on the 2nd Embodiment of this invention. It is a figure which shows typically the setting method of the magnitude | size of a motion compensation unit in the inter-frame prediction moving image encoder which concerns on 2nd Embodiment. It is a figure which shows the specific example of the setting method of the magnitude | size of a motion compensation unit in the inter-frame prediction moving image encoder which concerns on 2nd Embodiment. It is the figure which showed the process which each part performs in time series, when the moving image encoder which concerns on 2nd Embodiment processes the image data of 1 frame. It is a flowchart explaining the principal part operation | movement of the inter-frame prediction moving image encoder which concerns on embodiment of this invention.

Claims (10)

A motion vector detection processing time calculation device for determining a time for a motion vector detection process for detecting a motion vector of an input frame image with respect to a reference frame image for each region constituting the input frame image,
Motion compensation difficulty level calculating means for determining a motion compensation difficulty level for the area from the input frame image area and the corresponding reference frame image area;
Each of the regions is assigned by assigning a motion vector detection processing time given to the entire input frame image to a region having a large motion compensation difficulty so as to have a longer time than a region having a small motion compensation difficulty. A motion vector detection processing time calculation device comprising: processing time determination means for determining a motion vector detection processing time for. The motion compensation difficulty is information indicating the magnitude of motion of the region of the input frame image,
The processing time determination means assigns the motion vector detection processing time given to the entire input frame image to each of the regions by assigning a region with a large amount of motion to be longer than a region with a small amount of motion. The motion vector detection processing time calculation apparatus according to claim 1, wherein the motion vector detection processing time is determined.   The motion compensation difficulty level calculation unit calculates the motion compensation difficulty level based on a spatial differential value of a difference value between a region of the input frame image and a corresponding region of the reference frame image. The motion vector detection processing time calculation device according to claim 1 or 2. The motion vector detection processing time calculation device according to any one of claims 1 to 3,
A motion vector detection apparatus comprising detection means for detecting a motion vector in the region by equally applying the processing time determined for the region to the region. The motion vector detection processing time calculation device according to any one of claims 1 to 3,
A motion vector detection apparatus having detection means for performing vector detection in a smaller region unit in a region where the determined processing time is longer.   An image encoding device that performs motion compensation interframe predictive encoding of the input frame image using a motion vector detected by using the motion vector detection device according to claim 4 or 5.   The image coding apparatus according to claim 6, wherein the reference frame image is an image obtained by decoding a coding result of a past frame image from the input frame image. A motion vector detection processing time calculation method for determining, for each region, a time for motion vector detection processing for detecting a motion vector of an input frame image with respect to a reference frame image for each region constituting the input frame image,
A motion compensation difficulty level calculating step for determining a motion compensation difficulty level for the area from the input frame image area and the corresponding reference frame image area;
Each of the regions is assigned by assigning a motion vector detection processing time given to the entire input frame image to a region having a large motion compensation difficulty so as to have a longer time than a region having a small motion compensation difficulty. A motion vector detection processing time calculation method comprising: a processing time determination step for determining a motion vector detection processing time for.   A control program for causing a computer to execute each step of the motion vector detection processing time calculation method according to claim 8.   A computer-readable recording medium storing the control program according to claim 9.

Images