JP4124436B2 - Motion estimation device, program, storage medium, and motion estimation method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to a motion estimation device, a program, a storage medium, and a motion estimation method.
[0002]
[Prior art]
  Due to advances in image input technology and output technology, the demand for higher definition of images has increased greatly in recent years. For example, taking a digital camera as an example of an image input device, the price of a high-performance charge-coupled device (CCD) having a number of pixels of 3 million or more has been reduced, and the price range has become widespread. It has come to be widely used in products. And the product of 5 million pixels is coming soon. And it is said that this increasing trend in the number of pixels will continue for a while.
[0003]
  On the other hand, with regard to image output / display devices, for example, products in the hard copy field such as laser printers, ink jet printers, sublimation printers, and flats such as CRTs, LCDs (liquid crystal display devices), and PDPs (plasma display devices). The high definition and low price of products in the soft copy field of panel displays are remarkable.
[0004]
  Due to the market launch of these high-performance, low-priced image input / output products, high-definition images have become popular, and it is expected that demand for high-definition images will increase in all situations. In fact, the development of technology related to networks such as personal computers and the Internet is accelerating these trends. In particular, recently, mobile devices such as mobile phones and notebook personal computers have become very popular, and opportunities for transmitting or receiving high-definition images from any point using communication means are rapidly increasing.
[0005]
  Against this background, it is inevitable that the demand for higher performance or higher functionality for image compression / decompression technology that facilitates the handling of high-definition images will become stronger in the future.
[0006]
  Therefore, in recent years, a new method called JPEG2000, which can restore high-quality images even at a high compression rate, is being standardized as one of image compression methods that satisfy these requirements. In JPEG2000, it is possible to perform compression / decompression processing in a small memory environment by dividing an image into rectangular regions (tiles). That is, each tile becomes a basic unit for executing the compression / decompression process, and the compression / decompression operation can be performed independently for each tile.
[0007]
  Further, such a JPEG 2000 image of one frame can be converted into a moving image by being continuously displayed at a predetermined frame rate (the number of frames reproduced per unit time). As a standard for moving a JPEG2000 image frame by frame and displaying a moving image, there is an international standard called Motion JPEG2000.
[0008]
  In addition, as in the Motion JPEG2000 system, there has also been proposed one in which image data is compression-coded using discrete wavelet transform (see, for example, Patent Document 1).
[0009]
  In the technique disclosed in Patent Document 1, not only the pixel values are subjected to discrete wavelet transform and compression coding, but also the correlation between the images between different frames, and the moving image data when there is no image movement between the frames. Therefore, the data compression rate can be further improved.
[0010]
[Patent Document 1]
JP 2001-309281 A
[0011]
[Problems to be solved by the invention]
  However, according to the method described in Patent Document 1, it is necessary to go through a complicated process of decoding an orthogonal transform coefficient value encoded for obtaining a correlation between frames and further performing inverse quantization. Therefore, there is a problem that processing takes time. Furthermore, an amount of memory for storing a previous frame used for obtaining a correlation between frames is also required.
[0012]
  An object of the present invention is to provide a motion amount estimation device, a program, a storage medium, and a motion amount estimation method capable of obtaining a motion amount of an image with high speed and accuracy.
[0013]
[Means for Solving the Problems]
  The motion amount estimation apparatus according to the first aspect of the present invention is hierarchically compression-coded by dividing into a plurality of blocks for each frame of an interlaced image constituting a moving image and subjecting pixel values to discrete wavelet transform for each block. Means for selecting subblocks corresponding to high-frequency subbands of the LH component (1LH) and HL component (1HL) of one layer from the code string data, and the code amount of the selected 1LH subblock, The code amount of the 1HL sub-block is calculated, and a ratio between the two (1LH code amount / 1HL code amount) is calculated, and the calculated ratio is compared with a threshold value.Sub-blockThe amount of motion is estimated to be fast,Sub-blockAnd a means for estimating that the amount of motion is low.
[0014]
  High frequency subbandThe code amount of the sub-block included in the block is calculated in block units, and based on the code amount of this sub-block, code block unitetcThe amount of motion at is estimated. As a result, it is not necessary to take the difference between frames, so it is possible to reduce memory consumption and shorten the processing time.etcIt is possible to estimate the amount of motion at.In particular, by comparing the 1LH component of the first layer of the wavelet transform coefficient where the horizontal edge of the image appears strongly with the 1HL component of the first layer of the wavelet transform coefficient where the vertical edge of the image appears strongly, the motion amount (speed) of the interlaced image ) Is reliably estimated.
[0015]
  According to a second aspect of the present invention, in the motion amount estimating apparatus according to the first aspect, the high speed is estimated for the entire frame.Means for dividing the number of sub-blocks or the number of sub-blocks estimated to be low by the total number of sub-blocks, and comparing the result of the division with a threshold value, the amount of motion of the entire frame is high or low Means to estimateIt has further.
[0016]
  Therefore, estimatedSub-blockThe motion amount of the entire frame is estimated based on the unit motion amount. As a result, it is possible to perform rough adjustment of image quality based on the amount of motion of the entire frame and fine adjustment of image quality based on the amount of motion for each code block, etc., thus enabling efficient image quality control. . Moreover, since the motion amount of the entire frame image is estimated according to the ratio of the comparison result between the code amount of the subband 1LH and the code amount of the subband 1HL for all subblocks included in the high frequency subband, the entire frame image It is possible to easily estimate the amount of movement.
[0017]
  According to a third aspect of the present invention, in the motion amount estimation apparatus according to the first or second aspect, the calculated code amount of the sub-block is a lossless compressed code amount. Therefore, it is possible to improve the estimation accuracy of the motion amount.
[0018]
  According to a fourth aspect of the present invention, in the motion amount estimation apparatus according to any one of the first to third aspects, the calculated code amount of the sub-block is a code amount before bit truncation. did Accordingly, it is possible to improve the estimation accuracy of the motion amount.
[0019]
  A fifth aspect of the present invention is a program that causes a computer to execute the function of each unit of the motion amount estimating apparatus according to any one of the first to fourth aspects. A sixth aspect of the present invention is a computer-readable storage medium storing a program that causes a computer to execute the function of each means of the motion amount estimating apparatus according to any one of the first to fourth aspects. Therefore, it is possible to obtain the same operation as in the first to fourth aspects by executing the program directly on the computer or by reading the program recorded on the storage medium into the computer and executing it.
[0020]
  The motion amount estimation method according to the invention of claim 7 is hierarchically compression-coded by dividing into a plurality of blocks for each frame of an interlaced image constituting a moving image and subjecting the pixel value to discrete wavelet transform for each block. Selecting subblocks corresponding to high-frequency subbands of the LH component (1LH) and HL component (1HL) in one layer from the code string data, and the code amount of the selected 1LH subblock, The code amount of the 1HL sub-block is calculated to calculate the ratio between the two (the code amount of 1LH / 1 the code amount of 1HL), and the calculated ratio is compared with a threshold value.Sub-blockThe amount of motion is estimated to be fast,Sub-blockThe step of estimating that the amount of motion is low. Therefore, it is possible to obtain the same effect as that of the first aspect.
[0021]
  The invention according to claim 8 is the motion amount estimation method according to claim 7, wherein the high-speed is estimated for the entire frame.The number of sub-blocks or the number of sub-blocks estimated to be low is divided by the total number of sub-blocks, and the result of the division is compared with a threshold, and the amount of motion of the entire frame is high or low And estimating stepIt has further. Therefore, it is possible to obtain the same effect as that of the second aspect.
[0022]
  According to a ninth aspect of the present invention, in the motion amount estimation according to the seventh or eighth aspect, the calculated code amount of the sub-block is a lossless compressed code amount. Therefore, it is possible to obtain the same effect as that of the third aspect.
[0023]
  According to a tenth aspect of the present invention, in the motion amount estimation method according to any one of the seventh to ninth aspects, the calculated code amount of the sub-block is a code amount before bit truncation. Therefore, it is possible to obtain the same effect as that of the fourth aspect.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
  First, an outline of the “hierarchical encoding algorithm” and the “JPEG2000 algorithm” that are the premise of the present embodiment will be described.
[0025]
  FIG. 1 is a functional block diagram of a system that realizes a hierarchical encoding algorithm that is the basis of the JPEG2000 system. This system includes color space transform / inverse transform unit 101, two-dimensional wavelet transform / inverse transform unit 102, quantization / inverse quantization unit 103, entropy encoding / decoding unit 104, and tag processing unit 105. It is configured.
[0026]
  One of the biggest differences between this system and the conventional JPEG algorithm is the conversion method. In JPEG, discrete cosine transform (DCT) is used, whereas in this hierarchical coding algorithm, the two-dimensional wavelet transform / inverse transform unit 102 uses discrete wavelet transform (DWT). Yes. DWT has the advantage that the image quality in the high compression region is better than DCT, and this is one of the main reasons why DWT is adopted in JPEG2000, which is a successor algorithm of JPEG.
[0027]
  Another major difference is that in this hierarchical encoding algorithm, a functional block of the tag processing unit 105 is added in order to perform code formation at the final stage of the system. The tag processing unit 105 generates compressed data as code string data during an image compression operation, and interprets code string data necessary for decompression during the decompression operation. With the code string data, JPEG2000 can realize various convenient functions. For example, the compression / decompression operation of a still image can be freely stopped at an arbitrary layer (decomposition level) corresponding to octave division in block-based DWT (see FIG. 3 described later).
[0028]
  In many cases, color space conversion / inverse conversion 101 is connected to the input / output portion of the original image. For example, the RGB color system composed of R (red) / G (green) / B (blue) components of the primary color system and the Y (yellow) / M (magenta) / C (cyan) components of the complementary color system This corresponds to the part that performs conversion or reverse conversion from the YMC color system consisting of the above to the YUV or YCbCr color system.
[0029]
  Next, the JPEG2000 algorithm will be described.
[0030]
  As shown in FIG. 2, in a color image, each component 111 (RGB primary color system here) of an original image is generally divided by a rectangular area. This divided rectangular area is generally called a block or a tile. In JPEG2000, it is generally called a tile. Therefore, such a divided rectangular area is hereinafter referred to as a tile. (In the example of FIG. 2, each component 111 is divided into a total of 16 rectangular tiles 112, 4 × 4 in length and breadth). When such individual tiles 112 (R00, R01,..., R15 / G00, G01,..., G15 / B00, B01,..., B15 in the example of FIG. 2) execute the image data compression / decompression process. It becomes the basic unit. Therefore, the compression / decompression operation of the image data is performed independently for each component and for each tile 112.
[0031]
  At the time of encoding image data, the data of each tile 112 of each component 111 is input to the color space conversion / inverse conversion unit 101 in FIG. A dimensional wavelet transform (forward transform) is applied to divide the space into frequency bands.
[0032]
  FIG. 3 shows subbands at each decomposition level when the number of decomposition levels is three. In other words, the tile original image (0LL) (decomposition level 0) obtained by tile division of the original image is subjected to two-dimensional wavelet transform, and the subbands (1LL, 1HL, 1LH shown in the decomposition level 1) , 1HH). Subsequently, the low-frequency component 1LL in this hierarchy is subjected to two-dimensional wavelet transformation to separate the subbands (2LL, 2HL, 2LH, 2HH) indicated by the decomposition level 2. Similarly, the low-frequency component 2LL is also subjected to two-dimensional wavelet transform to separate subbands (3LL, 3HL, 3LH, 3HH) shown in the decomposition level 3. In FIG. 3, the subbands to be encoded at each decomposition level are indicated by shading. For example, when the number of decomposition levels is 3, subbands (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, 1HH) indicated by shading are to be encoded, and the 3LL subband is encoded. It is not converted.
[0033]
  Next, the bits to be encoded are determined in the specified encoding order, and the context is generated from the bits around the target bits by the quantization / inverse quantization unit 103 shown in FIG.
[0034]
  The wavelet coefficients that have undergone the quantization process are divided into non-overlapping rectangles called “precincts” for each subband. This was introduced to use memory efficiently in implementation. As shown in FIG. 4, one precinct consists of three rectangular regions that are spatially coincident. Further, each precinct is divided into non-overlapping rectangular “code blocks”. This is the basic unit for entropy coding.
[0035]
  The coefficient values after the wavelet transform can be quantized and encoded as they are, but in JPEG2000, in order to increase the encoding efficiency, the coefficient values are decomposed into “bit plane” units, and each pixel or code block is divided. Ranking can be performed on “bitplanes”.
[0036]
  Here, FIG. 5 is an explanatory diagram showing an example of a procedure for ranking the bit planes. As shown in FIG. 5, this example is a case where the original image (32 × 32 pixels) is divided into four 16 × 16 pixel tiles, and the size of the precinct and code block at the composition level 1 is Each is 8 × 8 pixels and 4 × 4 pixels. The numbers of the precinct and the code block are assigned in raster order. In this example, the number of assigns is assigned from numbers 0 to 3, and the code block is assigned from numbers 0 to 3. A mirroring method is used for pixel expansion outside the tile boundary, wavelet transform is performed with a reversible (5, 3) filter, and a wavelet coefficient value of decomposition level 1 is obtained.
[0037]
  An explanatory diagram showing an example of the concept of a typical “layer” configuration for tile 0 / precinct 3 / code block 3 is also shown in FIG. The converted code block is divided into subbands (1LL, 1HL, 1LH, 1HH), and wavelet coefficient values are assigned to the subbands.
[0038]
  The layer structure is easy to understand when the wavelet coefficient values are viewed from the horizontal direction (bit plane direction). One layer is composed of an arbitrary number of bit planes. In this example, layers 0, 1, 2, and 3 are made up of bit planes of 1, 3, 1, and 3, respectively. A layer including a bit plane closer to the LSB (Least Significant Bit) is subject to quantization first, and conversely, a layer close to the MSB (Most Significant Bit: most significant bit) is quantized to the end. It will remain without being. A method of discarding from a layer close to the LSB is called truncation, and the quantization rate can be finely controlled.
[0039]
  The entropy encoding / decoding unit 104 illustrated in FIG. 1 performs encoding on the tile 112 of each component 111 by probability estimation from the context and the target bit. In this way, encoding processing is performed in units of tiles 112 for all components 111 of the original image. Finally, the tag processing unit 105 performs a process of combining all the encoded data from the entropy encoding / decoding unit 104 into one code string data (code stream) and adding a tag thereto.
[0040]
  FIG. 6 shows a schematic configuration for one frame of the code string data. The head of this code string data and the head of the code data (bit stream) of each tile are a header (a main header (tile header) that is tile boundary position information, tile boundary direction information, and the like). Is added, followed by encoded data for each tile. Note that the main header (Main header) describes coding parameters and quantization parameters. A tag (end of codestream) is placed again at the end of the code string data.
[0041]
  On the other hand, at the time of decoding, the image data is generated from the code string data of each tile 112 of each component 111, contrary to the case of encoding the image data. In this case, the tag processing unit 105 interprets tag information added to the code string data input from the outside, decomposes the code string data into code string data of each tile 112 of each component 111, and Decoding processing (decompression processing) is performed for each code string data of each tile 112. At this time, the position of the bit to be decoded is determined in the order based on the tag information in the code string data, and the quantization / inverse quantization unit 103 determines the peripheral bits (that have already been decoded) of the target bit position. Context is generated from the sequence of The entropy encoding / decoding unit 104 performs decoding by probability estimation from the context and code string data, generates a target bit, and writes it in the position of the target bit. Since the data decoded in this way is spatially divided for each frequency band, the two-dimensional wavelet transform / inverse transform unit 102 performs two-dimensional wavelet inverse transform on each of the components of the image data. The tile is restored. The restored data is converted to original color system image data by the color space conversion / inverse conversion unit 101.
[0042]
  The above is an outline of the “JPEG2000 algorithm”. A “motion JPEG2000 algorithm” is an extension of a still image, that is, a method for a single frame to a plurality of frames. That is, “Motion JPEG2000”, as shown in FIG. 7, is a moving image by continuously displaying a JPEG2000 image of one frame at a predetermined frame rate (the number of frames reproduced per unit time). .
[0043]
  Hereinafter, a first embodiment of the present invention will be described. Here, an example relating to a moving image compression / decompression technique typified by Motion JPEG 2000 will be described, but it goes without saying that the present invention is not limited to the contents of the following description.
[0044]
  FIG. 8 is a block diagram showing a schematic configuration of a movie camera system 1 to which the present invention is applied. As shown in FIG. 8, a movie camera system 1 to which the moving image display system of the present invention is applied includes an image recording device 1a that is a movie camera and a moving image playback device 1b that is a personal computer, over a network 1c that is the Internet. It is connected via.
[0045]
  In the following, an image recording apparatus 1a that exhibits the characteristic functions of the present invention will be described. The moving image playback apparatus 1b may be a standard system capable of decompressing code string data compressed by the Motion JPEG2000 method, and thus detailed description thereof is omitted.
[0046]
  As shown in FIG. 8, the image recording device 1a includes an image input device 2 that captures a moving image and an image compression device 3 that compresses and encodes the captured image data. The image compression apparatus 3 implements the image processing apparatus of the present invention that performs compression processing of moving image data.
[0047]
  FIG. 9 is a block diagram illustrating an example of a hardware configuration of the image recording apparatus 1a. As shown in FIG. 9, the image recording apparatus 1 a includes a CPU (Central Processing Unit) 11 that is a main part of a computer and controls each part centrally. The CPU 11 includes various ROMs (Read Only). A memory 12 that is a storage medium including a memory (RAM) and a RAM (Random Access Memory), a predetermined communication interface 13 that communicates with the network 1c, and an operation panel 18 that receives various operations from the user via the bus 14. It is connected.
[0048]
  In the image recording apparatus 1 a, in addition to the image input apparatus 2 and the image compression apparatus 3 described above, a logic circuit 19 is connected to the CPU 11 via the bus 14.
[0049]
  Control programs such as a moving image processing program for processing moving images are stored in the memory 12 (ROM) of the image recording apparatus 1a having such a configuration. This moving image processing program implements the program of the present invention. And the function of the code sequence converter 4 is implement | achieved by the process which CPU11 performs based on this moving image processing program.
[0050]
  As the memory 12, various types of media such as various optical disks such as CD and DVD, various magnetic disks such as various magneto-optical disks and flexible disks, and semiconductor memories can be used. Alternatively, the program may be downloaded from the network 1 c and installed in the memory 12. In this case, the storage device storing the program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), and in that case, the OS may take over the execution of some of the various processes described below, It may be included as a part of a group of program files constituting the application software or OS.
[0051]
  Here, the operation of each part of the image recording apparatus 1a will be briefly described. The image input device 2 of the image recording device 1a captures a moving image in units of frames using a photoelectric conversion device such as a CCD or a MOS image sensor, and outputs a digital pixel value signal of the moving image to the image compression device 3. is there.
[0052]
  The image compression device 3 of the image recording device 1a compresses and encodes the digital pixel value signal of the moving image according to the “Motion JPEG2000 algorithm”. As shown in FIG. 10, the image compression apparatus 3 includes a color space conversion unit 31, a two-dimensional wavelet conversion unit 32, a quantization unit 33, an entropy encoding unit 34, a post quantization unit 35, and an arithmetic encoding unit 36. Consists of Various functions in these units are realized by processing performed by the CPU 11 in accordance with the above-described moving image processing program. In addition, when real-time property is regarded as important, it is necessary to speed up the processing. For this purpose, it is desirable to realize various functions in each unit by the operation of the logic circuit 19.
[0053]
  Next, the operation of each part constituting the image compression apparatus 3 will be briefly described. The color space conversion unit 31 converts the digital pixel value signal of the moving image input from the image input device 2 from RGB to YUV or YCbCr, and the two-dimensional wavelet conversion unit 32 performs two-dimensional wavelet conversion for each color component. Then, the quantization unit 33 divides the Wavelet coefficient by an appropriate quantization denominator, creates a lossless code by the entropy coding unit 34, performs bit truncation (code discard) by the post quantization unit 35, and performs arithmetic coding. The unit 36 forms a code in the JPEG2000 code format. Through such a series of processing, the moving image data of the R, G, and B components of the original moving image is divided into one or a plurality of (usually a plurality of) tiles for each frame, and each tile is hierarchical. The encoded data is compressed and encoded.
[0054]
  Here, the post-quantization unit 35 that exhibits a characteristic function in the present embodiment will be described in detail. FIG. 11 is a block diagram schematically showing the configuration of the post quantization unit 35. As shown in FIG. 11, the post quantization unit 35 includes a speed estimation unit 41, a quantization table determination unit 42, a code discard unit 43, and a masking control unit 44.
[0055]
  The speed estimation unit 41 functions as a motion amount estimation device, estimates the motion amount (speed) of an image from information in the code block created by the entropy encoding unit 34, and estimates the motion of the image. The amount (speed) is sent to the masking control unit 44.
[0056]
  The masking control unit 44 adjusts the truncation amount (bit plane cutting amount) of the quantization table for each code block.
[0057]
  The quantization table determination unit 42 determines a quantization table according to the compression rate given from the CPU 11 and the amount of motion (speed) of the image estimated by the speed estimation unit 41, and the quantization table is a code discarding unit. 43.
[0058]
  The code discarding unit 43 uses the quantization table determined by the quantization table determining unit 42 and the truncation amount (bit plane cutting amount) for each code block adjusted by the masking control unit 44 for each code block. Thus, the code is discarded from the code in a state where the bit plane (or the sub bit plane) is not deleted until a predetermined compression rate is reached.
[0059]
  Here, a method for estimating the amount of motion (speed) of the image by the speed estimation unit 41 will be described. FIG. 12 describes a basic idea of an image motion amount (speed) estimation method by the speed estimation unit 41. In general, when a motion occurs in an interlaced image, the edge of the moved object becomes a comb-like tooth shape (hereinafter referred to as an interlaced comb) in the frame data. As shown in FIG. 12, in an interlaced image, an image in which an object is moving at high speed has a long interlaced comb shape in the horizontal direction. On the other hand, in an image where an object is moving at low speed, the interlaced comb is short in the horizontal direction. Further, it is known that the horizontal edge of the image appears strongly in the 1LH component of the Wavelet transform coefficient. That is, since a high-frequency moving image has a longer horizontal edge at a higher speed, the higher the code block, the larger the sum of absolute values of 1LH coefficients and the larger the sum of lossless 1LH codes. Become. The speed estimation unit 41 uses this characteristic to estimate the motion amount (speed) of the image independently for each frame.
[0060]
  FIG. 13 is a block diagram schematically showing the configuration of the speed estimation unit 41. As illustrated in FIG. 13, the speed estimation unit 41 includes a block selection unit 51, a feature amount calculation unit 52, and a speed determination unit 53. The block selection unit 51 selects a code block to be calculated. For example, code blocks in one-layer subband 1LH are selected in raster order. Alternatively, all the code blocks may be selected. The feature amount calculation unit 52 calculates the feature amount by calculating the coefficient or code amount of the code block selected by the block selection unit 51. The speed determination unit 53 estimates the motion amount (speed) of the image in units of code blocks using the feature amount obtained by the feature amount calculation unit 52.
[0061]
  Here, estimation of the motion amount (speed) of the image in units of code blocks will be described. As described above, the code block is a subband divided into finer blocks. That is, the code block is a sub-block. In the present embodiment, the interlaced image is compared by comparing the 1LH component of the first layer of the Wavelet transform coefficient in which the horizontal edge of the image appears strongly with the 1HL component of the first layer of the Wavelet transform coefficient in which the vertical edge of the image appears strongly. The amount of motion (speed) is estimated. Further, the motion amount (speed) of the image in units of code blocks is estimated by comparing code blocks that are in the same position when decoded. Here, FIG. 14 shows a one-layer subband having four code blocks. That is, as shown in FIG. 14, the code block 1HL_1 and the code block 1LH_1 that are at the same position during decoding are compared. Similarly, 1HL_2 and 1LH_2, 1HL_3 and 1LH_3, and 1HL_4 and 1LH_4 are also compared. When the code block is one subband, the comparison is performed in units of subbands instead of units of code blocks. Then, the amount of motion (speed) of the image in units of code blocks is estimated according to the comparison result.
[0062]
  The process for estimating the amount of motion (speed) of the image in units of code blocks as described above will be described with reference to the flowchart shown in FIG. As shown in FIG. 15, among the code blocks selected by the block selector 51, the first code block is acquired (step S1: sub-block acquisition means), and the 1LH and 1HL bit planes of the code block are obtained. The code amount before cutting is calculated (steps S2 and S3: code amount calculation means), divided (1LH / 1HL), and the calculated ratio is used as a feature amount (rate) for motion amount estimation (step S4). . The processing in steps S1 to S4 is executed by the feature amount calculation unit 52.
[0063]
  Then, the result (rate) of the division (1LH / 1HL) is compared with the threshold (th0) (step S5), and if the result (rate) of the division (1LH / 1HL) is larger than the threshold (th0) (step S5). Y) Assuming that the horizontal edge of the image appears strongly, the motion amount (speed) of the image in units of code blocks is estimated to be high (step S6). On the other hand, if the result (rate) of the division (1LH / 1HL) is not larger than the threshold (th0) (N in step S5), it is assumed that the vertical edge of the image appears strongly, and the motion of the image in units of code blocks The amount (speed) is estimated to be low (step S7). The processing in steps S5 to S7 is executed by the speed determination unit 53. Here, the function of the sub-block motion estimation means is executed.
[0064]
  The processes in steps S2 to S7 are repeated until all the code blocks selected by the block selection unit 51 are completed (steps S8 and S9).
[0065]
  That is, when there are four code blocks in one layer subband as shown in FIG. 14, the above process is repeated for the number of code blocks of the coefficient in one layer. • The speed for the block will be estimated. Here, the number of code blocks in one layer is, for example, when the size of a code block is 32 × 32 and the size of a 1LH coefficient is 256 × 128, the number of code blocks is (256/32). ) × (128/32) = 8 × 4 = 32.
[0066]
  Here, the code amount of the subblock included in the high frequency subband is calculated in units of blocks, and the amount of motion in units of code blocks is estimated based on the code amount of the subblocks. As a result, it is not necessary to take the difference between frames, so it is possible to reduce memory consumption and reduce the processing time, so it is possible to estimate the amount of motion in units of code blocks with high speed and accuracy. Become.
[0067]
  The motion amount (speed) in units of code blocks estimated in this way is sent from the speed estimation unit 41 to the masking control unit 44, so that an image in which a fast moving object and a slow moving object are mixed is mixed. Each can be subjected to optimum processing (masking processing, etc.).
[0068]
  Next, a second embodiment of the present invention will be described with reference to FIGS. The same parts as those described in the first embodiment are denoted by the same reference numerals, and description thereof is also omitted. In the first embodiment, the motion amount (speed) of the image in units of code blocks is estimated. However, in this embodiment, the motion amount (speed) of the image in units of code blocks and the motion amount of the entire frame image are estimated. (Speed) is estimated.
[0069]
  FIG. 16 is a block diagram schematically showing the configuration of the speed estimation unit 41 according to the second embodiment of this invention. As illustrated in FIG. 16, the speed estimation unit 41 includes a block selection unit 61, a feature amount calculation unit 62, and a speed determination unit 63. The block selection unit 61 selects a code block to be calculated. For example, code blocks in one-layer subband 1LH are selected in raster order. Alternatively, all the code blocks may be selected. The feature amount calculation unit 62 calculates the feature amount by calculating the coefficient or code amount of the code block selected by the block selection unit 61. The speed determination unit 63 estimates the motion amount (speed) of the image in units of code blocks and the motion amount (speed) of the entire frame image using the feature amount obtained by the feature amount calculation unit 62. .
[0070]
  Since the estimation of the motion amount (speed) of the image in units of code blocks has been described in the first embodiment, the description thereof is omitted.
[0071]
  Next, estimation of the motion amount (speed) of the entire frame image using the estimation result of the motion amount (speed) of the image in units of code blocks will be described. Note that, as described in the first embodiment, the description will be made on the assumption that the motion amount (speed) of an image in units of code blocks is estimated for four code blocks.
[0072]
  In the present embodiment, the motion amount (speed) of the entire frame image is estimated according to the ratio of the motion amount (speed) of the image for each code block unit. More specifically, as shown in FIG. 17, the ratio between the high speed and the low speed of the motion amount (speed) of the image for the four code blocks is
High speed: Low speed = 3: 1
Then, since the high-speed ratio is high, it is estimated that the motion amount (speed) of the entire frame image is high.
[0073]
  Note that this is just an example, and using the estimation result of the motion amount (speed) of the image in units of code blocks, it is easy to determine the motion amount (speed) of the entire frame image at a high speed or a low speed. It may be configured so that it can be easily set.
[0074]
  The process of estimating the amount of motion (speed) of the image in units of code blocks and the amount of motion (speed) of the entire frame image as described above will be described with reference to the flowchart shown in FIG. As shown in FIG. 18, after initializing the counter (step S21) and setting the total number of code blocks selected by the block selection unit 51 (step S22), 1LH and 1HL bits for each corresponding code block The code amount before cutting the plane is calculated (steps S23 and S24), and division (1LH / 1HL) is performed (step S25).
[0075]
  Then, the result (rate) of the division (1LH / 1HL) is compared with the threshold (th1) (step S26), and if the result (rate) of the division (1LH / 1HL) is larger than the threshold (th1) (step S26). Y), the number of code blocks whose count (1LH / 1HL) result (rate) is greater than the threshold (th1) is counted (step S27).
[0076]
  The processes in steps S23 to S27 are repeated until all the code blocks selected by the block selection unit 51 are completed.
[0077]
  When the processing of steps S23 to S27 is completed for all code blocks selected by the block selection unit 51 (Y in step S28), the code whose division (1LH / 1HL) result (rate) is greater than the threshold (th1) The number of blocks is divided by the total number of codes and the number of blocks, and the calculated ratio is set as a feature amount (speed) for motion amount estimation (step S29). The processes in steps S21 to S29 are executed by the feature amount calculation unit 52.
[0078]
Then, the feature amount (speed) obtained by the feature amount calculation unit 52 is compared with the threshold value (th2) (step S30), and high speed and low speed are determined based on the comparison result. That is, if the feature amount (speed) obtained by the feature amount calculation unit 52 is larger than the threshold value (th2) (Y in step S30), the high-speed ratio of the image motion amount (speed) in each code block is high. As a thing, it is estimated that the motion amount (speed) of the entire frame image is high (step S31). On the other hand, if the feature amount (speed) obtained by the feature amount calculation unit 52 is not larger than the threshold value (th2) (N in step S30), the low-speed ratio of the image motion amount (speed) in each code block is As a high value, it is estimated that the motion amount (speed) of the entire frame image is low (step S32). The processes in steps S30 to S32 are executed by the speed determination unit 53. Here, the function of the frame motion amount estimation means is executed.
[0079]
  Here, the code amount of the sub-block included in the high frequency sub-band is calculated in block units, and the motion amount in code block unit is estimated based on the code amount of the sub-block, and the estimated sub-block The motion amount of the entire frame is estimated based on the unit motion amount. The motion amount (speed) of the entire frame estimated in this way is sent from the speed estimation unit 41 to the quantization table determination unit 42, so that the quantization table determination unit 42 has a quantization table suitable for the motion amount (speed). Can be selected. That is, it is possible to perform coarse adjustment of image quality based on the amount of motion of the entire frame and fine adjustment of image quality based on the amount of motion for each code block, so that efficient image quality control can be performed.
[0080]
In the above description, the image recording apparatus 1a of the present invention is applied to a movie camera. However, the image recording apparatus 1a is applied to an information terminal apparatus such as a personal digital assistant (PDA) or a mobile phone. You can also.
[0081]
【The invention's effect】
  According to the present invention, the following effects can be obtained.
(1) It is necessary to calculate inter-frame differences by calculating the code amount of sub-blocks included in the high-frequency subband in units of blocks and estimating the amount of motion in units of code blocks based on the code amounts of the sub-blocks Therefore, the memory consumption can be suppressed and the processing time can be shortened, so that the motion amount in units of code blocks can be estimated with high speed and accuracy. In particular, by comparing the 1LH component of the first layer of the wavelet transform coefficient in which the horizontal edge of the image appears strongly with the 1HL component of the first layer of the wavelet transform coefficient in which the vertical edge of the image appears strongly, the motion amount (speed) of the interlaced image ) Can be reliably estimated.
[0082]
(2) By estimating the motion amount of the entire frame based on the estimated motion amount of each sub-block, the image quality is roughly adjusted by the motion amount of the entire frame, and the image quality is finely adjusted by the motion amount of each code block. Since adjustment can be performed, efficient image quality control can be performed. In addition, the motion amount of the entire frame image according to the ratio of the comparison result between the code amount of the subband 1LH and the code amount of the subband 1HL by the subblock motion amount estimation means for all subblocks included in the high frequency subband. It is possible to easily estimate the amount of motion of the entire frame image.
[0083]
(3) Since the code amount of the sub-block is a lossless-compressed code amount, the motion amount estimation accuracy can be improved.
[0084]
(4) Since the code amount of the sub-block is the code amount before bit truncation, it is possible to improve the estimation accuracy of the motion amount.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of a system that realizes a hierarchical encoding algorithm that is the basis of a JPEG2000 system that is a premise of the present invention.
FIG. 2 is an explanatory diagram showing a divided rectangular area of each component of the original image.
FIG. 3 is an explanatory diagram showing subbands at each decomposition level when the number of decomposition levels is 3. FIG.
FIG. 4 is an explanatory diagram showing a precinct.
FIG. 5 is an explanatory diagram showing an example of a procedure for ranking bit planes;
FIG. 6 is an explanatory diagram illustrating a schematic configuration of one frame of code string data.
FIG. 7 is an explanatory diagram showing the concept of Motion JPEG2000.
FIG. 8 is a block diagram showing a schematic configuration of the movie camera system according to the first embodiment of the present invention.
FIG. 9 is a block diagram illustrating an example of a hardware configuration of an image recording apparatus.
FIG. 10 is a block diagram schematically showing a configuration of an image compression apparatus.
FIG. 11 is a block diagram schematically showing a configuration of a post quantization unit.
FIG. 12 is an explanatory diagram of a basic idea of an image motion amount (speed) estimation method performed by a speed estimation unit;
FIG. 13 is a block diagram schematically showing a configuration of a speed estimation unit.
FIG. 14 is an explanatory diagram showing a one-layer subband having four code blocks;
FIG. 15 is a flowchart showing the flow of estimation processing of the motion amount (speed) of an image in units of code blocks.
FIG. 16 is a block diagram schematically illustrating a configuration of a speed estimation unit according to the second embodiment of this invention.
FIG. 17 is an explanatory diagram illustrating an example of an estimation result of a motion amount (speed) of an image in units of code blocks.
FIG. 18 is a flowchart showing a flow of estimation processing of the motion amount (speed) of an image in units of code blocks and the motion amount (speed) of the entire frame image.
[Explanation of symbols]
  12 storage media
  41 Motion estimation device

Claims (10)

One layer of LH components (1LH) is obtained from code string data that is hierarchically compression-coded by dividing the block into a plurality of blocks for each interlaced image frame constituting the moving image and subjecting the pixel values to discrete wavelet transform for each block. ) And means for selecting a sub-block corresponding to the high-frequency sub-band of the HL component (1HL) in units of blocks,
Means for calculating a code amount of the selected 1LH sub-block and a code amount of the 1HL sub-block, and calculating a ratio between the two (a code amount of 1LH / a code amount of 1HL);
A means for comparing the calculated ratio with a threshold and estimating that the motion amount of the sub-block is high if it is large, and estimating that the motion amount of the sub-block is low if it is not large;
A motion amount estimation apparatus comprising: Means for dividing the number of sub-blocks estimated to be fast or the number of sub-blocks estimated to be slow by the total number of sub-blocks in the entire frame ;
Means for comparing the result of the division with a threshold and estimating that the amount of motion of the entire frame is high or low;
The motion amount estimation apparatus according to claim 1 , further comprising:   The motion amount estimation apparatus according to claim 1 or 2, wherein the code amount of the sub-block is a lossless compressed code amount.   The motion amount estimation apparatus according to claim 1, wherein the code amount of the sub-block is a code amount before bit truncation.   The program which makes a computer perform the function of each means of the motion amount estimation apparatus as described in any one of Claims 1 thru | or 4.   A computer-readable storage medium storing a program for causing a computer to execute the function of each means of the motion amount estimation apparatus according to claim 1. One layer of LH components (1LH) is obtained from code string data that is hierarchically compression-coded by dividing the block into a plurality of blocks for each interlaced image frame constituting the moving image and subjecting the pixel values to discrete wavelet transform for each block. ) And a sub-block corresponding to the high-frequency sub-band of the HL component (1HL) are selected in units of blocks;
Calculating a code amount of the selected 1LH sub-block and a code amount of the 1HL sub-block, and calculating a ratio between the two (a code amount of 1LH / a code amount of 1HL);
Comparing the calculated ratio with a threshold, estimating that the amount of motion of the sub-block is fast if it is large, and estimating that the amount of motion of the sub-block is slow if not large;
A motion amount estimation method characterized by comprising: Dividing the number of sub-blocks estimated to be fast or the number of sub-blocks estimated to be slow by the total number of sub-blocks over the entire frame ;
Comparing the result of the division with a threshold and estimating that the amount of motion of the entire frame is high or low;
The motion amount estimation method according to claim 7, further comprising:   9. The motion amount estimation method according to claim 7, wherein the code amount of the sub-block is a lossless compressed code amount.   The motion amount estimation method according to claim 7, wherein the code amount of the sub-block is a code amount before bit truncation.

Images