FI110745B - Encoding successive images has reference image divided into parts which includes 4x4 pixels referred to with indexes and number is formed from the values of pixels in each part to describe pixel values in a given part - Google Patents

Abstract

The method has the following steps: an image and reference image are divided into parts which includes 4x4 pixels referred to with indexes and a number is formed from the values of the pixels in each part to describe pixel values in a given part; a search area is defined in the reference image; a block image is encoded using the motion vector candidate that gives the lowest cost function value, the motion between the image block to be encoded and a candidate block in the search area of reference image defined by the motion vector candidate. The image is processed into an indexed image and the reference image into an indexed reference image. The block to be encoded in the indexed image is searched for and a cost function for which a SAD function (Sum of Absolute Differences) is used is calculated for each motion vector candidate using the indexed reference image. Independent claims are also included for the following: (1) An apparatus for encoding successive images; and (2) A computer program on a carrier for encoding successive images.

Description

1,110,745

Method and apparatus for encoding sequential images

Area

The invention relates to a method and apparatus for encoding sequential images.

5 Background

The coding of consecutive images, such as video, is used to reduce the amount of data so that it can be more effectively stored on a storage medium or transmitted over a communication link. An example of a video encoding standard is MPEG-4 (Moving Pictures Expert Group). There are a variety of flow constants used, for example, cif is 352 x 288 pixels and qcif is 176 x 144 pixels.

Typically, a single image is divided into blocks containing information about luminance, color, and location. The block data is compressed block by block using the desired coding method. Compression is based on the removal of less significant data. Compression methods are mainly divided into three categories: Spectral Redundancy Reduction, Spatial Redundancy Reduction, and Temporal Redundancy: '. · Reduction. Typically, various combinations of these methods are used for compression.

For example, a YUV color scheme is used to reduce spectral redundancy. The YUV model takes advantage of the fact that the human eye is ··. ·; more sensitive to variations in luminance, or luminance, than chrominance, or color. · ·. changes. The YUV model has one luminance component (Y) and two chrominance components (U and V, or Cb and Cr). For example, in the H.263 video coding standard, the luminance block is 16 x 16 pixels and each chrominance block covers the same region as the luminance block, 8 x 8 pixels. The combination of one luminance block and two chrominance blocks is called a macroblock. Each pixel, in both the luminance and chrominance blocks 30, can have a value between 0 and 255, that is, two pixels are required to represent one pixel. For example, a value of 0 for a luminance pixel is black and a value for 255 is white.

• · Discrete Cosine Transform (DCT) is used, for example, to reduce space redundancy. In discrete cosine conversion, the pixel representation of a block is converted to a space frequency representation. In addition, in the block of Fig. 110745, only the signal frequencies that occur therein have high amplitude coefficients, and those signals which do not occur in the block have coefficients close to zero. The discrete cosine transform is basically a lossless transform and the signal is only disturbed by quantization.

5 The goal is to reduce temporal redundancy by taking advantage of the fact that successive images tend to resemble each other, so that instead of compressing each single image, motion data of the blocks is generated. This is called motion compensation. For the block to be coded, the best possible pre-coded reference block is searched from the previous reference image stored in the memory, the motion between the reference block and the block to be coded is modeled and the calculated motion vectors are transmitted to the receiver. The difference between the block to be coded and the reference block is expressed as difference data. Such coding is called inter-coding, which means utilizing the similarities between the images in the same image sequence.

Typically, the reference image defines a search area from which a block like the one to be encoded is searched. The best match is found by calculating the cost function between the pixels between the block in the search area and the block to be coded, for example the sum of .20 Absolute differences (SAD).

The motion estimation in the search area can be represented by the formula: · '·' · 'MV {x, y) = min (1) •. -! 6 <*, v <16 TTt1 1 •. / = 0 j-0. '· Where MV is the final motion vector, fxy is the pixel of the macroblock to be encoded, and rxy is the pixel of the reference image in the search area.

In accordance with the prior art, full search is used, i.e. all or almost all possible motion vectors are set as motion vector candidates. The problem with using full search is the high number of calculations required. For example, if the search area is 48 x 48 pixels, . the number of pixels with a resolution of one pixel is 33 x 33, and the size of the luminance block is 16 x 16 pixels, so as to calculate one sum of the absolute differences. 16 x 16 = 256 calculations are needed, and so to compute the sum of all possible motion vectors · "33 x 33 x 256 = 278,784: ··. ' calculation per macroblock: For example, a qcif image has 99 macroblocks, which means 99x278,784 = 27,599,616 calculations -35.

3 110745

Efforts have been made to reduce the number of calculations by using less comprehensive search methods, such as Three Step Search, Spiral Search, and Hierarchical Motion Estimation instead of full-search, which may cause problems with image quality.

5 Brief Description

An object of the invention is to provide an improved method and an improved device. In one aspect of the invention, there is provided a method according to claim 1 for encoding sequential images. In one aspect of the invention there is provided an apparatus for encoding sequential images according to claim 9. In one aspect of the invention, there is provided an apparatus for encoding sequential images according to claim 17. Other preferred embodiments of the invention are claimed in the dependent claims.

The invention is based on the fact that motion estimation is performed using an indexed image and an indexed reference image.

The solution according to the invention provides almost the same image quality as classic full-search, but with lighter computation.

List of figures

Preferred embodiments of the invention will be described by way of example with reference to the accompanying drawings, in which: Figure 1 shows a device for coding consecutive images; :. ·; Fig. 2 shows the distribution of a qcif image in blocks; : '·. Figure 3 shows a portion of an image to be encoded; Figure 4 shows a portion of a reference view; Figures 5, 6, and 7 illustrate an indexing operation; Fig. 8 is a flowchart illustrating a method for coding sequential images.

Description of Embodiments

Video encoding is well known to those skilled in the art, based on standards: "and textbooks, for example, by reference in this book, Vasudev Bhaskara and Konstantinos Konstantinides: '' Image and Video. ! Wer Academic Publishers 1997, Chapter 6: '' The MPEG Video Standards '' and 'Digital Video Processing', Prentice Hall Signal Processing Series, Chapter 6: '' Block

Based Methods. "

110745

The sequential images to be encoded are typically moving images, such as video. The video image consists of single sequential images in the camera. The camera generates a matrix representing the image in pixels, for example, as described at the beginning, with 5 luminance and chrominance matrices of their own. The data stream representing the image in pixels is exported to an encoder. Of course, it is also possible to construct a device in which the data stream is received by an encoder, for example, via a data transmission connection or, for example, from a computer storage medium. The purpose here is that the uncompressed video image is compressed with an encoder, for example for retransmission or recording. The compressed video image generated by the encoder is transmitted using a channel to the decoder. The decoder does basically the same thing as the encoder did when composing the image, but in reverse. The channel may be, for example, a fixed or wireless data connection. The channel may also be interpreted as a transmission path through which a video image is stored on a storage medium, such as a laser disk, and by which the video image is then read from the storage medium and processed by a decoder. The compressed video image transmitted in the channel may also be subjected to other coding, for example channel coding by the channel encoder. Channel coding is decoded by the channel decoder. The encoder and decoder may be located in various devices, such as computers, subscriber terminals of various radio systems, such as mobile stations, or other devices. devices that want to process video. The encoder and decoder can also be combined with the same device, which can then be called a video codec. Referring to Fig. 1, the structure of a sequential image encoder or encoder is described. Figure 1 illustrates the operation of an encoder at the theoretical level of theo-25. In practice, the structure of the encoder is much more complex when the skilled artisan adds the necessary things according to the prior art, for example, the necessary timing and block processing. Successive images 130 are temporarily brought to be stored in frame buffer 102. A single frame 132 is taken from frame buffer 102 to block 104 where the desired encoding format 30 is selected. The operation of the device is controlled by a control part 100 which e.g. selects the desired encoding format and announces the selected encoding format. 156, 158 for block 104 and block 120. The coding format may be intracoding or inter-coding. Motion compensation is not applied to an inter-coded image, while motion compensation is applied to an •: · * inter-coded image. Usually, the first image, .V 35, is inter-coded, and the subsequent pictures are inter-coded. Even after the first image, the image may be transmitted, for example, if good motion vectors are not found for the image to be encoded.

Next, the operation of the device when intracoding is selected in block 104 is described.

For the intranet image, block 104 only receives an image 132 from the frame buffer 102. The image 132 obtained from the frame buffer 102 is applied as such to a discrete cosine transform block 106, where the discrete cosine transform described above is performed.

The discrete cosine transformed image 136 is applied to a quantization block 10108 where quantization is performed, i.e., in principle, each element of the discrete cosine transformed image is divided by a constant and the result of the division is rounded to an integer. This constant may vary between different macroblocks. The quantization parameter from which these dividers are calculated is usually in the range of 1-31. The more zeros a block receives, the better the block is compressed, since 15 zeros are not transmitted to the channel.

The quantized and discrete cosine transformed image 138 is then applied to a variable length encoder 110, the output of which becomes an encoded image 140 produced by the device.

Quantized and discrete-20 cosine transformed image 138 from quantization block 108 is also applied to a variable-length encoder 110 but also inverse quantization block 112 which performs inverse quantization of the imported quantized and dis - ;; · Cretaceous transformed image 138, i.e. returns it '·' 142 is introduced to an inverse discrete cosine transform block 114 where an inverse discrete cosine transform is performed. Since the discrete cosine transform is a lossless transform but the quantization does not, picture 144 does not fully correspond to picture 134. The purpose of inverse quantization and inverse discrete cosine conversion is to produce in the encoder a picture similar to that of the decoder corresponding to the encoder. The 'decoded' image 144 is then exported to block 124, where the portion removed from it is added to the image, eroded if the image were intercoded.

> I

Since this is an intracoded image, nothing is added to it. With this oh-; . the division is made by a block 120 with intracoding selected, whereby the input of the block 120 has nothing, and the output connected to its block 124 also contains nothing. Intranet image 146 is then stored in frame buffer 116.

. V 35 Thus, the reconstructed image, i.e., the encoded image, is stored in the frame buffer 116 in the form in which it is in the decoder after decoding.

6 110745

Thus, there are two frame buffers, the first 102 storing an image arriving at the device, and the second 116 reconstructing a "previous" image. Thus, the image for which intracoding was selected in blocks 104 and 120 was thus processed.

When working on the following figure, motion compensation 5 can be started.

Intercoding is then selected in blocks 104 and 120. Thus, the image stored in frame buffer 116 is now a reference image, and the image to be encoded is the following image 132 from frame buffer 102. As seen in Figure 1, the following image is applied not only to block 104 but also to motion estimation block 118. 10 Motion estimation block 118 also provides reference frame 150 The operation of the motion estimation block 118 will be described in more detail later, but a search is made to find blocks in the reference image that correspond to the blocks in the image to be encoded. Inter-block offsets are expressed as motion vectors 152, 166, which are applied 15 to both variable length encoder 110 and frame buffer 116.

A reference image 148 of frame buffer 116 is provided to block 122. Block 122 subtracts reference image 148 from encoded image 132 to extract 164 which is output from block 104 through discrete cosine conversion block 106 and quantization block 108 to variable length encoder 110.

The variable length encoder 110 thus encodes the eroded data 138 and the motion vectors 166, whereby the output 140 of the variable length encoder 110 produces an in - '; Thus, variable length encoder 110 receives inputs discrete cosine transformed and quantized eroded data 138 and motion vectors 166. Thus, encoder output 140 provides compressed data representing an encoded image 25, depicting the encoded image relative to the reference image and representing the motion image. difference data. Motion estimation is performed using luminance blocks, but the encoded difference data is calculated for both luminance and chrominance blocks.

The eroded data 138 of said intercoded image is also subjected to inverse quantization 30 in inverse quantization lattice 112 and inverse discrete cosine transform in inverse discrete cosine transform block 114. . the erod data 144 is applied to block 124, where it is incremented with the previous image 154 subtracted from the position indicated by the motion vector when encoding said inter picture. Then, from block 124, the sum 146 of erode data and previous image Λ '35 is applied to frame buffer 116 to obtain a reconstructed image. The reconstructed image corresponds to the image obtained in the decoder when decoding the inter-encoded image 1101045. Thus, the frame buffer 116 again has a reference image ready for encoding the next image 132 from the frame buffer 102.

The control block 100 controls the operation of the encoder. In addition to the castle of encoding mode va-5, it controls e.g. selecting the correct quantization ratio 160 and performing variable-free coding 162. Similarly, the control block 100 may control other blocks of the encoder, although not illustrated in Figure 1. For example, the operation of the motion estimation block 118 is controlled by the control block 100.

Referring now to the flowchart of Fig. 8, a method 10 for encoding sequential images will be described. The coding is provided specifically for temporal redundancy reduction, and other methods for reducing redundancy are not described herein. Execution of the method is started in block 800 where the encoder is started. In block 802, the following image is retrieved from the frame memory. For example, the image may be a qcif-15 image depicted in Figure 2 with 176 x 144 luminance pixels. The image is divided into macroblocks 200 of 16 x 16 pixel luminance sections with eleven columns and nine rows. Luminance pixels can be represented by the matrix / o, 0 f0.1 f0.2 ··· / 0.175 ... f 1,0 f \, lf 1,2 - / 1,175 fij = f2,0 Λ, Ι f2,2 ·· / 2,175> (2) • · t. ·· · »· ··· ·.« ·· .

'· _ / L43,0 / l43, l / l43,2 · "/ 43,175 _' ·" 20 where i = 0,1,2 ..... 143 and j = 0,1,2 .... .175.

In block 804, the image is processed into an indexed image such that:, the image is subdivided into index-referenced portions, and the pixel values in each portion are converted into a number representing the pixel values in that portion.

In one embodiment, the pixels contained in the part form a square, as this is advantageous according to the tests performed by the applicant. Kuvako-

»I

(L. size and block size place some restrictions on the size of the indexed part used in the method. For the Qcif image, according to the applicant's experiments, it is preferable that the part contains 4x4 pixels. In this case, matrix 2 can be described as follows: * »» »

I I

t I I »8 F F F F i 110745 * 0.0 * Yl ^ 0.2" M), 172 ^., 0 Ft, Fl2 ... ^ 1,172

Fjj = F20 F2i F22 ... F2m, (3) • ^ 140.0 - ^ 140.1 ^ 140.2 - ^ 140.172 where 1 = 0.1.2, ..., 140 and J = 0, 1.2 ..... 172.

Thus, in addition to indexing, the values of the pixels in each section are formed into a number representing the pixel values in that section. By -5 most often this number is obtained by summing the number values of the pixels in that section. The number of each part of matrix 3 is obtained from matrix 2 by the following formula: / + 3 J + 3 3 · ,, = ΣΣΛ w / = / j = j

That is, for example, the number of the part referenced by the index Fo.o is given by the calculation 10 = ΣΣ / * · (5) / = 0 ./=0 The number referenced by the index F0, i is obtained by the calculation ^, 'ΣΣΛ- (®)

/ -o J.I

«· •.,; the number referred to by index Fi, 0 is obtained by the calculation V 'is iu-tt / * · <n ·, * / = 17 7 = 0. 'And the number of the part referred to by the index Fi, i is obtained by calculation (8); '* / = 1 7 = 1

As can be seen by comparing the matrices 2 and 3, the parts referred to by the indices which are close to each other overlap partly. By this is meant that, for example, the regions referenced by the indices F0, o and Fii0 contain twelve identical pixels of the matrix, i.e., pixels fi, o, fi, i, fi, 2, fi, 3. h, o, f2.i, f2.2,;, 'f2,3, f3, o. f3, i, f3,2, and f3,3-Then, in one embodiment, when calculating the numbers of two adjacent index-referenced parts, the number already calculated for the second part is utilized in calculating the number of the first part. This principle of the sliding las-25 cell can be represented as follows. As noted above, formula 5 can be used to calculate the number of the part to be referred to by the index F 0, o. In this case, the other numerical values are slidably obtained such that (9) 9 110745 7 = 0 7 = 0

Fi, ° = K '> - t.fu + i, f5j do »7 = 0 7 = 0

The same principle can be applied to the calculation of all numerical values, even when moving from the left to the right side of matrix 2.

After the image has been processed in block 804 into an image indexed using the principles set forth above, it is possible to proceed to block 806 where the encoding format, intracoding or intercoding to be used is selected.

If the selected coding mode is an intracoding mode, proceed to block 806 in accordance with the direction of the arrow 808 to block 810 where the picture is encoded, that is carried out for 10 diskreettikosinimuunnos and quantization, but no motion estimation. Then proceed to block 826, where the intracoded image is stored as a reference image. Subsequently, in block 828, the indexed image is stored as an indexed reference image containing the parts referenced by the indexes, and a number representing pixel values in that part-15 is formed from the pixel values in each section. Note that this utilizes the fact that the image is processed in block 804 into an indexed image. Similarly, it should be noted that the actions of the blocks 826 and 828 may be optional, but the reference image need not be an encoding image immediately * ..! previous image, but may be an earlier image.

; *. 20 from block 828 goes to block 830 to check if there are still pictures to be encoded. If the images are no longer left, move the arrow 832 • "In accordance with block 834, which terminated the performance of the method. If the images while" "· is left, then move to the arrow 836 in accordance with block 802, where the retrieved :: in the next image frame memory.

25 If in block 806 the selected coding mode is interkoodaus, then move to block 806 according to arrow 812 to block 814, which defines an indexed search area in the reference image to be scanned in an indexed reflect. ·. block to be encoded. Intercoding cannot therefore be performed on a first image, since it requires that at least one reference image exist. Thus, in our example, there must already be one intracoded image, i.e., the operations of blocks 810, 826, and 828 must be performed on the reference image in question. In the future, of course, the reference image may be some previously in- tercoded image.

10 1 10745

Figures 3 and 4 illustrate the execution of a search. Figure 3 shows a portion of an image to be encoded 300 and Figure 4 shows a portion of a reference image 400. Parts 300, 400 illustrate the same portion of the qcif image shown in Figure 2. The image 300 to be encoded thus consists of 5 x 16 luminance blocks. The chrominance blocks are generally 8x8 pixels in size, but are not illustrated in Figures 3 and 4 because the chrominance blocks are not utilized in motion estimation. It should be noted that Figures 3 and 4 illustrate the actual contents of the images, without indexing, for clarity.

In our example, suppose that the image has nothing but a suitable image for one block 302 of the exact 10, consisting of a transverse stroke and an "H". The reference image 400, our method indexed reference image, thus defines an image element in block 302 of the search area 402 to be searched in the coded image, our method indexed coded image. The search for motion vectors is usually limited to a search area 402 of 15 [-16,16]. one of nine blocks of 16 x 16 pixels. Nine blocks of the search area 402 are located around the location reference code 400 of the coding block 302 in the coding image 300 as shown in FIG. 4. The search area 402 is thus 48 x 48 pixels. In this case, the possible motion vectors: ·. ·. The number of 20 den, or motion vector candidates, is 33 x 33.

'·.! Then, we proceed to block 816, where the cost function of each motion vector candidate is calculated using an indexed image and an indexed reference image. Figures 5, 6 and 7 illustrate an indexing operation that illustrates, by pixel, 502 portions of the 25 search areas 402 depicted in Figure 4. Figure 5 illustrates which index-referenced areas 500 are required to compute a motion vector candidate [0.0]; ^ 0.0 - ^ 0.4 - ^ 0.8 ^ 0.12 F = ^ 4. ° F4> 4 FFFF ^ ... 1 8.0 1 8.4 1 8.8 1 8.12. · · 'J * 12.0 ^ 12.4 ^ 12.8 ^ 12.12 _; As shown in Figure 6, the following elements are required for calculating the motion vector candidate [1.0] from matrix 3: 11 110745

Fo, ^ 0.5 ^ 0.9 ^ 0.13 F_ ^ -5 ^ 49 ^ 413 / -J2 \ / ¾ .. ^ 8.5 ^ 8.9 ^ 8,, 3 '• ^ 12.1 ^ 12.5 - ^ 12.9 ^ 12.13 _

By the same principle, the numbers needed to calculate all possible motion vector candidates are obtained from matrix 3, for example, the following elements-5 of matrix 3 are required for the motion vector candidate [4,0] as shown in Figure 7: ^ 0.4 ^ 0.8 - ^ 0.12 ^ 0.16

^ 4,4 ^ 4,8 ^ 4,12 ^ 4,16 M

b = p F F F (12) rS, 4 rS, S ^ 8.12 "8.16 _ ^ 12.4 ^ 12.8 ^ 12.12 ^ 12.16_

The motion estimation in the search area can be represented by the formula: 3 3 MV (x, y) = | min ^ ΣΣΙ ^ i * 4, / * 4 ^ i * 4 + xtj * 4 + y | (13) 'y / = 0 7 = 0 where MV is the final motion vector, Fxy is the number calculated for the index of the image to be encoded, and Rxy is the number calculated for the index of the reference image. Comparing Formula 13 used in our method with Formula 1 of the prior art, our method requires 33 x 33 x 4 x 4 = 17424 calculations, which is 16 times less than the traditional full-search method. Calculation of indices is not taken into account here,. 15 but need only be counted once for the whole image. In our example, the cost function uses the Sum of Absolute Differences (SAD) function, but it is obvious to one skilled in the art that other cost functions can be used if they can be calculated using the image indexing method.

In our example of Figures 3 and 4, a block 404 corresponding to the coding block 302 in the (indexed) coding image 300 was found in the (indexed) reference picture 400. With respect to the block 404 found in the reference picture 400 of the coding block 302, motion vector 406. for example, the leftmost part of block 302 to be encoded; 25 man in the upper corner of the pixel motion vector. Of course, the other pixels in block 302 also move in the direction of the motion vector in question.

The origin of the image (0, 0) is usually a pixel in the upper left corner of the image. In video coding terminology, movements are expressed as right motion positive, left negative, up negative and 12 110745 down positive. The motion vector 406 is (12, -4), i.e. the motion is twelve pixels in the X axis to the right and four pixels in the Y axis up.

Block 816 is then moved to block 818, which encodes the image to be encoded using a motion vector-5 didate which gives the value of the least cost function, which motion vector determines the movement between the image coding block and the candidate block in the reference image search area.

Then proceed to block 820 to test whether there are still blocks to be encoded in the picture to be encoded. If the coded blocks is still a hereinafter-10 remaining, go to the arrow 822 in accordance with block 814, where the search starts seu raavaa-encoded block in the reference image corresponding to the block. The arrow 822 of the loop, therefore, is repeated until the encoded image blocks have been processed as desired, either all or part of them.

If the coded blocks are no longer left, then move to a si-15 820 avenge the direction of arrow 824 in accordance with the first block 826 and then to block 828, where, if desired, stored in a just coded picture as a reference picture and an indexed indexed coded picture as a reference picture. Then, block 830 checks whether there are still pictures to be encoded. If the coded images are no longer left, then move to the arrow 834 in accordance with block 834, where; ·· 20 terminated the performance of the method, otherwise proceed in accordance with the direction of the arrow 836, · · · block 802, which looks for the next picture to be encoded.

In the method illustrated in FIG. the ringing to an indexed image is utilized such that the result of the same processing '· * can also be stored as an indexed reference image. This reduces the 25 computations required, but if you want to optimize memory usage, you can in-. the dexterated reference image is calculated only from the reference image recorded before it is used.

In one embodiment, when the minimum value of the cost function is found Illustrative motion vector of one pixel accuracy, searching-30 to around the half pixel motion vector accuracy improved * * · rac .- motion vector. This is illustrated in Figure 8 by block 840. The half-pixel: · element found in the motion estimation, the 16 x 16 block is checked for another half-pixel resolution. Normally, this requires an 18x18 matrix for interpolating pixels, but the index-based procedure we have proposed allows the area to be interpolated to be a 6x6 index-based matrix. A half-pixel motion vector obtained by applying the formula 13 110745 13, wherein the procedure is also 16 times that requires less calculation than the conventional half-pixel resolution motion estimation.

In one embodiment, the best half-pixel resolution motion-vektorikandidaatti look as follows: 5 discovered interpolated motion vector of one pixel corresponding to an indexed refe renssikuvassa kandidaattilohkolle and around the half-pixel values; is calculated for each half-pixel motion vector cost function indexed image and the interpolated indexed candidate blocks used 10-crystallization; is coded image block to be coded using the minimum value of the cost Advisory half pixel motion vector.

In one embodiment, the known fact is utilized that current coding standards also allow out-of-motion motion vectors. In the indexed motion estimation illustrated, this is achieved by overfilling the index table so that there are 16 pixels on each side of the image, both top, bottom and sides, copied from the outer edges of the actual image area. This operation can be performed in block 842 in FIG. 8. The size of the indexed matrix 3 is the image size minus three, i.e., 173 x 141 in our example. The overfill image size is 173 + 32 x 141 + 32, or 176 + 29 x 144 + 29. Of course, the number 29 is obtained by reducing the space required by the overfill, that is, the number 32 from the number three.

»» *; 'The following pseudocode using the C programming language syntax describes how to calculate an indexed table and overfill the indexes.

25 typedef struct {[u8 * image; / * Cursor to image to be indexed 7

u16 pels; / * Number of columns in the image V

u16 Iines; / * Number of lines in the image V

u16 * mv; / * Pointer to index table 7 30} index_s; ; . void Melndex (index_s * vop) {·: * u16 i, j; 35 u16pelsMv; u16 * tmp = NULL; * »» »14 1 1074 5 u16 * tmp1 = NULL; u16 * put = NULL; u16 * start = NULL; u8 'block = NULL; 5 / * Temporary memory for one index column * / tmp = (u16 *) malloc (sizeof (u16) * vop-> lines); / * Number of columns in index table, 7 10 / * space required for overfill is taken into account 7 pelsMv = vop-> pels + 29; / * Pointer to first index, represents motion vector (0,0) 7 start = vop-> mv + pelsMv * 16 + 16; 15 / * Calculate first column of index table 7 block = image; put = start; for (j = 0; j <4; j ++) {20 tmp [j] = block [0] + block [1] + block [2] + block [3]; block + = vop-> pels; ; put [0] = tmp [0] + tmp [1] + tmp [2] + tmp [3]; tmp1 = tmp; 25 for (j = 4; j <vop-> lines; j ++) {· .. · tmp1 [4] = block [0] + block [1] + block [2] + block [3]; putfpelsMv] = put [0] + tmp1 [4] -tmp1 [0]; block + = vop-> pels; TMP1 ++; put + = pelsMv; }: ·,: 30 / * Calculate the remaining columns of the index table * / · for (i = 1; i <vop-> pels-3; i ++) {block = image + i; put = start + i; 35 for (j = 0; j <4; j ++) {..... tmp [j] = tmpö] + block [3] -block [-1]; 15 110745 block + = vop-> pels; } put [0] = tmp [0] + tmp [1] + tmp [2] + tmp [3]; tmp1 = tmp; 5 for (j = 4; j <vop-> lines; j ++) {tmp1 [4] = tmp1 [4] + block [3] -block [-1]; put [pelsMv] = put [0] + tmp1 [4] -tmp1 [0]; block + = vop-> pels; TMP1 ++; put + = pelsMv; } 10} free (tmp); / * Overflow index table * / 15 IndexOverfill (vop); return; } 20 void lndexOverfill (index_s * vop); ··: {. · .U16 i, j; : 'u16 pelsMv; '· U16 * put = NULL; / * Auxiliary variables, pointer to index table * / 25 u16 * get = NULL; / * Auxiliary variables, pointer to index table * / / * Number of columns in index table, 7 / * Space overflow required 7 pelsMv = vop-> pels + 29; : '·.' 30 / * Overflow center of table from top 7: · put = vop-> mv + 16; get = vop-> mv + pelsMv * 16 + 16; * "'for (j = 0; j <16; j ++) {. 35 for (i = 0; i <vop-> pels; i ++) {put [i] = get [i]; 16 110745} put + = pelsMv;} 5 / * Overflow center of table from bottom 7 put = vop-> mv + (vop-> lines + 13) * pelsMv + 16; get = vop-> mv + (vop-> lines + 12) * pelsMv + 16 ; for G = 0; j <16; j ++) {for (i = 0; i <vop-> pels; i ++) {10 put [i] = get [i];} put + = pelsMv;} 15 / * Overflow table from left 7 put = vop-> mv; get = vop-> mv + 16; for (j = 0; j <vop-> lines + 29; j ++) {for (i = 0; i <16; i ++) {20 put [i] = * get;} put + = pelsMv; get + = pelsMv;} 25 / * Overflow table from right 7 put = vop-> mv + vop-> pels + 13; get = vop-> mv + vop- > pels + 12; for (j = 0; j <vop-> lines + 29; j ++) {for (i = 0; i <16; i ++) {30 put [i] = * get;.: ··} : · Put + = pelsMv; get + = pelsMv;} .. · 35 return;} "110745

The method described may be implemented, for example, using the encoder illustrated in Figure 1. The encoder or apparatus for encoding sequential images illustrated in Figure 1 comprises means 110 for encoding an image coding block using a motion vector candidate having the value of a least cost function, which motion vector determines the motion between the image coding block and the candidate block in the reference image search area. The apparatus further comprises means 118 - process the image into an indexed image and a reference image into an indexed reference image such that the image and the reference image are divided into indexed portions and the pixel values in each portion are converted into a number representing pixel values in that portion; - defining, in the indexed reference image, the search area from which the block to be encoded in the indexed image is searched; and - compute the cost function of each motion vector candidate using the indexed 15 images and the indexed reference image.

The device may also be configured to encode an image coding block using a motion vector candidate having the value of a least cost function, which motion vector determines the movement between the image coding block and the candidate block in the reference image search area. In this case, 20 devices are further configured:

• I

- processing the image into an indexed image and the reference image into an indexed reference image, dividing the image and the reference image into an indexed image;

* I

! the parts to be referred to therein and the values of the pixels in each part are converted into a figure representing the values of the pixels in that part; 25 - define a search area in the indexed reference image from which:. retrieving the block to be encoded in the indexed image; and - calculate the cost function of each motion vector candidate using an indexed image and an indexed reference image.

The encoder blocks shown in Fig. 1 may be implemented as one or more application-specific integrated circuits (ASICs). Other implementations are also possible, for example; a circuit built from separate logic components, or a processor with software.

A mixed form of these different embodiments is also possible. One skilled in the art will consider, for example, the size and power consumption of the device when selecting an embodiment; 35 requirements, processing power, manufacturing costs and production volumes. Said means may be placed in the encoded blocks of image 110745, or they may be implemented as new blocks associated with the described blocks. For example, the means for processing the image into an indexed image and the reference image into an indexed reference image may be implemented in block 118 or in frame buffer 102, or using a new block 5 in connection with frame buffer 102. Device configuration can also be implemented using described or new blocks.

Although the invention has been described above with reference to the example of the accompanying drawings, it is to be understood that the invention is not limited thereto, but that it can be modified in many ways within the scope of the inventive idea of the appended claims. Thus, the size of the images to be processed may vary from the qcif size used in the example, and will not cause significant changes in the practice of the invention.

»F I

»1 I i t t r i <• · i»

Claims (24)

A method of coding successive images, which comprises the following steps: an encodable block of an image is encoded (818) using a motion vector candidate that provides the minimum cost function value, which motion vector candidate defines a motion between the the encodable block of an image and a candidate block of a reference image's search field; characterized in that, before encoding: (804) the image is processed into an indexed image and the reference image into an indexed reference image such that the image and the reference image are divided into reference parts and the pixel values in each part form a tai describing the pixel values in said part ; in the indexed reference image, (814) a search bar is defined from which the encodable block in the indexing image is retrieved; and the respective function vector candidate's cost function is calculated (816) using the indexing image and the indexing reference image. Method according to claim 1, characterized in that the pixels in said part form a square. : ..; 20 Method according to claim 2, characterized in that said *; · · That part includes 4x4 pixels. A method according to any of the preceding claims, characterized in that a SAD (Sum of Absolute Differences) function is used as a cost function. : *] 25 5. Method according to any of the preceding claims, characterized in that the parts referred to by index refer to one another partially cover each other. 6. A method according to claim 5, characterized in that da tai is calculated for two adjacent areas by means of index-referenced parts, utilizing the tai already calculated for the second part da of the first part. . One is calculated. 7. A method according to any of the preceding claims, characterized in that when the motion vector candidate that gives the least cost function value has been found at the precision of a pixel, the best motion vector candidate is searched for half a pixel precision around said motion vector candidate. . Method according to claim 7, characterized in that, at half a pixel precision, the best motion vector candidate is searched as follows: the found motion vector candidate comprising a pixel is interpolated in a reference image to a corresponding indexed candidate block and a half pixel value around said candidate block; a cost function is calculated for each motion vector candidate comprising half a pixel using the indexed image and the interpolated indexed candidate block; the encodable blocks of the image are encoded using the motion vector candidate comprising a half pixel which provides the minimum cost function value. An apparatus for encoding successive images, comprising: means (110) for encoding an encodable block of an image by employing a motion candidate that provides the least cost function value, which motion vector candidate defines a motion between the encodable block of an image and a candidate block on a reference image's search field; characterized in that the device further comprises::. Means (118) for processing the image into an indexed image and a reference image such that the image and reference image are divided into reference parts by means of indexes, and of the pixel values in each portion, a tai describing the pixel values is formed. in said part; Means (118) for defining in the indexed reference image a search range from which the encodable block in the indexed image is retrieved; as well. means (118) for calculating the cost function of the respective motion vector candidate using the indexed image and the indexed reference image. Device according to claim 9, characterized in that the image; . 30 points in said part form a square. 11. Device according to claim 10, characterized in that said portion comprises 4x4 pixels. Device according to any of the preceding claims 9-11, characterized in that a SAD (Sum of Absolute Differences) function is used as a cost function. 25 110745 Device according to any of the preceding claims 9-12, characterized by means of the index referring parts positioned near each other partially covering each other. Device according to claim 13, characterized in that the tai is calculated for two adjacent parts by means of index-referenced parts, the tai already calculated for the second part is then used when the number of the first part is calculated. 15. Device according to any of the preceding claims 9-14, characterized in that the motion vector candidate which gives the minimum cost function value has been found at the precision of a pixel, the best motion vector candidate is searched at half a pixel precision around said motion vector candidate. 16. Apparatus according to claim 15, characterized by a half pixel precision, the best motion vector candidate is searched in the following way: the found motion vector candidate comprising a pixel is interpolated in a reference image to a corresponding indexed candidate block and half a pixel value around said candidate block; a cost function is calculated for each motion vector candidate comprising half a pixel by using the indexed image and the interpolated indexed candidate block; '; * '. the encodable blocks of the image are encoded using motion vector channels; The 'doata comprising half a pixel gives the least cost function>>' 'value. 17. A device for encoding successive images, configured to: • ·:, encode a codable block of an image using a motion vector candidate that provides the least cost function value, which motion vector candidate defines a motion between the codeable block of the image and a candidate block of the search frame of a reference image; Characterized in that the device is further configured to:. process the image into an indexed image and the reference image into an indexed reference image such that the image and the reference image are divided by reference into indices and the pixel values in each portion form a tai which describes the pixel values in said portion; Define in the indexed part a search frame from which the codable block in the indexed image is retrieved; and calculate the cost function of each motion vector candidate using the indexed image and the indexed reference image. Device according to claim 17, characterized in that the pixels in said part form a square. Device according to claim 18, characterized in that said part comprises 4x4 pixels. Device according to any of the preceding claims 17-19, characterized in that the device is configured to use a SAD (Sum of Absolute Differences) function is used as a cost function. Apparatus according to any of the preceding claims 17-20, characterized in that the parts referred to by index refer to one another partially cover each other. Device according to claim 21, characterized in that the device is configured da tai is calculated for two adjacent sides by means of index referencing parts to utilize the tai already calculated for the second part when the number for the first part is calculated. Device according to any of the preceding claims 17-22,:. * Characterized in that when the motion vector candidate that gives the least cost function value has been found at the precision of a pixel, the device is configured to look for the best motion vector candidate at half a pixel precision around said motion vector candidate. 24. Apparatus according to claim 23, characterized in that the device is configured to look for the best of half a pixel precision:,, the motion vector candidate in the following way: the found motion vector candidate comprising a pixel is interposed in a reference image to a corresponding indexed candidate block and half a pixel value around said candidate block; A cost function is calculated for each motion vector candidate in * comprising half a pixel using the indexed image and ~; . the interpolated indexed candidate block; the encodable blocks of the image are encoded using the motion vector candidate comprising a half pixel which gives the minimum cost function value, v '35. • t} t