WO2007113767A1

WO2007113767A1 - Video processor comprising a pixel filter

Info

Publication number: WO2007113767A1
Application number: PCT/IB2007/051186
Authority: WO
Inventors: Aleksandar Beric; Ramanathan Sethuraman
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2006-04-04
Filing date: 2007-04-03
Publication date: 2007-10-11

Abstract

A video processor comprises a pixel filter (INT) for calculating a weighted sum of pixels that are present in a particular area of a picture. The pixel filter comprises a multiplexer arrangement (MUXA) that forms an array of input pixels (PI1, PI2,.. PI64) on the basis of a selection of pixels that are present in the particular area of the picture. An array of multipliers (Ml, M2,.., M64) multiplies the array of input pixels with an array of weighting coefficients (K1, K2,.., K64) so as to obtain an array of weighted pixels (PW1, PW2,.., PW64). An accumulator arrangement (ACCA) forms respective combinations of weighted pixels in the array of weighted pixels.

Description

Video processor comprising a pixel filter.

FIELD OF THE INVENTION

An aspect of the invention relates to a video processor that comprises a pixel filter. The video processor may be, for example, a video encoder, a video decoder, or a video display system capable of carrying out picture-rate conversion or de-interlacing, or a combination of these operations. Other aspects of the invention relate to a method of video processing, a computer program product for a programmable processor, and a video apparatus.

BACKGROUND OF THE INVENTION There are many types of video processing that involve pixel filtering. Pixel filtering equates with calculating a weighted sum of pixels, which are present in a particular area of a picture. Examples of types of video processing that involve pixel filtering include video coding with motion estimation and motion compensation, video decoding, video format conversion, picture-rate up-conversion, de-interlacing, de-blocking, up-scaling, and motion- compensated filtering for noise reduction. In particular, pixel filtering can be used for the purpose of interpolation. A weighted sum of pixels can be considered to constitute an interpolated pixel, which has a spatial position defined by respective weighting coefficients. The spatial position of the interpolated pixel need not necessarily be precisely on a grid point of a pixel grid. Accordingly, a sub-pixel resolution can be achieved in the aforementioned types of video processing.

There are various parameters that characterize a pixel filtering operation. One parameter is the size of the area in terms of number of pixels for which a weighted sum needs to be calculated. For example, the area may comprise 2, 4, or 8 pixels. Another parameter is the shape of the area. For example, the area may be rectangular or circular- like. Many different shapes are possible. In case the area is rectangular, the area has a given aspect ratio, which constitutes yet another parameter. For example, a rectangular area that comprises 4 pixels may be 2 pixels wide (horizontal dimension) and 2 pixels high (vertical dimension). The rectangular area may also be, for example, 4 pixels wide and 1 pixel high. The aspect ratio is 1:1 in the first case and 4:1 in the second case. Yet another parameter relates to sub- sampling, which can be applied in establishing a weighted sum of pixels for a particular area of interest. In case sub-sampling is applied, not all pixels in the area of interest are taken into account for establishing the weighted sum. For example, only 1 out of 2 pixels are taken into account in accordance with a regular pattern. Sub-sampling allows a reduction of computational effort.

US patent number 6,526,430 describes a coprocessor that assists in the computation of sum of absolute differences, symmetrical row/column finite impulse response filtering with a downsampling (or upsampling) option, row/column discrete cosine transform and inverse discrete cosine transform, and generic algebraic functions. The coprocessor consists of 8 multiply-accumulate hardware units connected in parallel and routed and multiplexed together.

SUMMARY OF THE INVENTION

It is an object of the invention to allow an improvement in video processing, in particular by providing a pixel filter that can efficiently be used in many different applications. The independent claims define various aspects of the invention. The dependent claims define additional features for implementing the invention to advantage.

The invention takes the following points into consideration. In principle, a pixel filter can entirely be implemented by means of software. That is, a processor executes a set of instructions, which define a pixel filtering operation that the processor carries out. Such a software-based implementation will generally be relatively complex and relatively slow. This is particularly true if the software has to handle many different types of pixel filtering operations, each of which has a particular set of parameters that have been described hereinbefore. A pixel filter can also be implemented in the form of a data path. The coprocessor that the aforementioned US patent describes comprises a data path that consists of 8 multiply-accumulate hardware units connected in parallel and routed and multiplexed together. However, this data path requires a relatively complicated formatting of pixels for which a weighted sum needs to be calculated. That is, complicated manipulations are required in order to apply appropriate input data to the respective multiply-accumulate hardware units. Consequently, the coprocessor that the aforementioned US patent describes will require relatively complicated software when used for the purpose of pixel filtering. This is particularly true if the co-processor needs to cope with the various different parameters that have been described hereinbefore, in particular sub-sampling. The aforementioned US patent does not explain how sub-sampling could be achieved within the co-processor.

In accordance with the invention, a pixel filter comprises a multiplexer arrangement that forms an array of input pixels on the basis of a selection of pixels that are present in a particular area of the picture. An array of multipliers multiplies the array of input pixels with an array of weighting coefficients so as to obtain an array of weighted pixels. An accumulator arrangement forms respective combinations of weighted pixels in the array of weighted pixels.

The pixel filter in accordance with the invention can cope with the various parameters that have been mentioned hereinbefore. The pixel filter can provide weighted sums for many different shapes and sizes of picture areas. Many different sub-sampling patterns can be implemented by appropriately controlling the multiplexer arrangement. What is more, the pixel filter can be configured in accordance with a particular set of parameters with relatively simple software. An implementation of the invention advantageously comprises one or more of following additional features, each of which provides advantages in terms of flexibility, scalability, and ease of programming. Different sets of additional features, which correspond with different dependent claims, are presented as different paragraphs.

The accumulator arrangement comprises a ring of adders coupled to receive at least a portion of the array of weighted pixels. An adder within the ring provides a combination of weighted pixels.

The multiplexer arrangement comprises a first set of multiplexer groups, which receives pixels from odd lines, and a second set of multiplexer groups, which receives pixels from even lines. A multiplexer group comprises a plurality of multiplexers. Each multiplexer of the multiplexer group receives a different group of pixels and selects one of these pixels as an input pixel.

The accumulator arrangement comprises a first adder ring and a second adder ring. The first adder ring forms respective combinations of weighted pixels on the basis of input pixels, which the first set of multiplexer groups provides. The second adder ring forms respective combinations of weighted pixels on the basis of input pixels that the second set of multiplexer groups provides. The accumulator arrangement comprises a plurality of composition sections. A composition section receives a plurality of combinations of weighted pixels and comprises different composition units for different pixel windows.

A detailed description with reference to drawings illustrates the invention summarized hereinbefore, as well as the additional features.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates a video coding system.

FIG. 2 is a pictorial diagram that illustrates a set of pixels from a reference picture, which is used for the purpose of motion estimation.

FIG. 3 is a pictorial diagram that illustrates a register memory in which the set of pixels is stored.

FIG. 4 is a pictorial diagram that illustrates an interpolated pixel that is formed on the basis of 8 pixels. FIG. 5 is a pictorial diagram that illustrates an interpolated pixel that is formed on the basis of 4 pixels.

FIG. 6 is a pictorial diagram that illustrates an interpolated pixel that is formed on the basis of 2 pixels.

FIG. 7 is a block diagram that illustrates an interpolator that forms part of the video coding system illustrated in FIG. 1.

FIG. 8 is a block diagram that illustrates a multiplexer arrangement that forms part of the interpolator illustrated in FIG. 7.

FIG. 9 is a circuit diagram that illustrates a multiplexer group that forms part of the multiplexer arrangement illustrated in FIG. 8. FIG. 10 is a circuit diagram that illustrates another multiplexer group that forms part of the multiplexer arrangement illustrated in FIG. 8.

FIG. 11 is a block diagram that illustrates an accumulator arrangement that forms part of the interpolator illustrated in FIG. 7.

FIG. 12 is a circuit diagram that illustrates an eight pixel window composition unit that forms part of the accumulator arrangement illustrated in FIG. 11.

FIG. 13 is a circuit diagram that illustrates a four pixel window composition unit that forms part of the accumulator arrangement illustrated in FIG. 11.

FIG. 14 is a circuit diagram that illustrates a two pixel window composition unit that forms part of the accumulator arrangement illustrated in FIG. 11. FIG. 15 is a pictorial diagram that illustrates a set of interpolated pixels that the interpolator illustrated in FIG. 7 provides.

FIG. 16 is a pictorial diagram that illustrates another set of interpolated pixels that the interpolator illustrated in FIG. 7 provides. FIG. 17 is a pictorial diagram that illustrates yet another set of interpolated pixels that the interpolator illustrated in FIG. 7 provides.

DETAILED DESCRIPTION

FIG. 1 illustrates a video coding system VCS, which may be in the form of, for example, a digital camcorder. The video coding system VCS comprises a camera CAM, an input memory MEMI, a reference memory MEMR and various functional entities: a subtractor SUB, a prediction error coder PEC, a motion compensator MC, a motion estimator ME, and a system controller SCT. The motion estimator ME comprises a register memory RGM, an interpolator INT, and a motion vector finder MVF. Any of these functional entities may be implemented by means of software or hardware, or a combination of software and hardware. For example, each of these functional entities may be implemented by suitably programming a processor. In such a software-based implementation, a software module may cause the processor to carry out specific operations that belong to a particular functional entity. As another example, each of the aforementioned functional entities may be implemented in the form of a dedicated circuit. This is a hardware-based implementation. Hybrid implementations may involve software modules as well as one or more dedicated circuits.

The video coding system VCS basically operates as follows. The camera CAM provides an input video VI that represents a scene, which the camera CAM captures. The input memory MEMI temporarily stores the input video VI. The input video VI comprises a sequence of pictures. Each picture is divided into blocks of pixels, which are individually encoded in the following manner. The subtractor SUB receives a block of pixels BI, which forms part of the input video VI. The subtractor SUB subtracts a predicted block of pixels BP from the block of pixels BI that is to be encoded. Accordingly, the subtractor SUB provides a block of prediction error pixels BE.

The prediction error coder PEC encodes respective blocks of prediction error pixels BE so as to obtain a coded video VC. In addition, the prediction error coder PEC decodes the coded video VC so as to obtain a decoded video VD. The decoded video VD comprises a sequence of decoded pictures. A decoder that receives the coded video VC will produce the same sequence of decoded pictures. The reference memory MEMR stores decoded pictures that serve as a reference for encoding pictures in the input video VI.

The motion compensator MC provides the predicted block of pixels BP on the basis of a reference picture RP, which is stored in the reference memory MEMR, and a motion vector MV. The motion vector MV points to an area in the reference picture RP that is similar to block of pixels BI to be encoded. More specifically, the motion vector MV defines a spatial displacement between the block of pixels BI to be encoded and the area in the reference picture RP that is similar. The motion vector MV may define this displacement in sub-pixel units. That is, the motion vector MV has a horizontal component and a vertical component, each of which may comprise an integer part and a fractional part. In case the motion vector MV comprises a fractional part, the predicted block of pixels BP is established by means of interpolation between pixels in the area concerned of the reference picture RP.

The motion estimator ME establishes the motion vector MV by means of a search. A search typically involves testing various different candidate motion vectors. Each candidate motion vector defines a particular reference block of pixels, which can be synthesized from the area in the reference picture RP to which the candidate motion vector points. The search aims to establish the reference block of pixels that best matches the block of pixels BI to be encoded. The candidate motion vector that provides this best matching reference block of pixels constitutes the motion vector MV that is applied to the motion compensator MC. The candidate motion vectors may have sub-pixel accuracy. In case a candidate motion vector comprises a fractional part, the reference block of pixels is synthesized by means of interpolation.

The search for the motion vector MV, which the motion estimator ME carries out, involves the following operations. The motion estimator ME reads a set of pixels PS from the reference picture RP that is stored in the reference memory MEMR. The set of pixels PS is stored in the register memory RGM. The interpolator INT receives search data SRD from the motion vector finder MVF, which specifies a candidate motion vector. The interpolator INT provides a set of interpolated pixels XPS on the basis of the search data SRD and the set of pixels PS that is stored in the register memory RGM. The motion vector finder MVF may specify a sequence of various different candidate motion vectors by means of the search data SRD. Accordingly, the interpolator INT may provide various different sets of interpolated pixels for different candidate motion vectors on the basis of a single set of pixels PS that is present in the register memory RGM. A set of interpolated pixels XPS, which the interpolated provides for a particular candidate motion vector, represents a portion of the reference block of pixels that the candidate motion vector defines. The motion vector finder MVF uses this portion to establish a degree of matching between the reference block of pixels, which the candidate motion vector defines, and the block of pixels BI to be encoded. FIG. 2 illustrates the set of pixels PS that is stored in the register memory

RGM. The set of pixels PS comprises 64 pixels, which are organized in the form of a matrix with 16 columns and 4 rows. Each pixel P(i,j) has a column number "i" and a row number "j". Indices "i", "j" thus denote a horizontal and a vertical position, respectively, of the pixel P_1J within the set of pixels PS. FIG. 2 illustrates this by designating pixels P_1;1, Pis,i, and P_1;4. FIG. 2 further illustrates a pixel grid by means of dotted lines. The pixel grid defines integer vertical positions and integer horizontal positions for a pixel. Each pixel is centered on a grid point, which is formed by a crossing of a vertical dotted line and a horizontal dotted line. Consequently, each pixel of the set of pixels PS has an integer vertical position and an integer horizontal position that corresponds with the column number and the row number of the pixel, respectively.

FIG. 3 illustrates the register memory RGM in which the set of pixels PS are stored. The register memory RGM comprises 16 registers RGl, RG2, .., RG16. Each register is capable of storing four pixels, which have the same row number "j". Each register may be, for example, 32 bits wide and each pixel may comprise comprises 8 bits so that a register is capable of storing four pixels as FIG. 3 illustrates.

FIG. 4 illustrates an interpolated pixel XPi ₈ that is formed on the basis of eight pixels, which are within a window of 4 x 2 pixels. A rectangle in broken lines illustrates this 4 x 2 pixel window. The interpolated pixel XPi ₈ is a weighted combination of the eight pixels within the 4 x 2 pixel window. Respective weighting coefficients for these respective eight pixels define a position of the interpolated pixel XPi ₈. The interpolated pixel XPi ₈ is not necessarily centered on a grid point of the pixel grid, which is illustrated by means of dotted lines as in FIG. 2. The respective weighting factors define, as it were, a center of gravity that need not necessarily correspond with a grid point.

FIG. 5 illustrates an interpolated pixel XPi 4 that is formed on the basis of four pixels, which are within a window of 2 x 2 pixels. A rectangle in broken lines illustrates this 2 x 2 pixel window. The interpolated pixel XPi 4 is a weighted combination of the four pixels within the 2 x 2 pixel window.

FIG. 6 illustrates an interpolated pixel XPi 2 that is formed on the basis of two pixels, which are within a window of 2 x 1 pixels. A rectangle in broken lines illustrates this 2 x 1 pixel window. The interpolated pixel XPi 2 is a weighted combination of the two pixels within the 2 x 1 pixel window.

In effect, FIGS. 4, 5, and 6 illustrate three different interpolation functions. Each interpolation function has a particular pixel window and a particular set of weighting coefficients. The set of weighting coefficients defines the center of gravity: the position of an interpolated pixel within the pixel window. The respective interpolated pixels illustrated in FIG. 4, 5, and 6 have respective positions that do not coincide with a grid point of the pixel grid. A set of weighting coefficients that provides such an interpolated pixel, corresponds with a particular fractional part in a candidate motion vector. For example, a set of weighting coefficients may correspond with a motion vector of which the horizontal component and the vertical component have fractional parts equal to 0.25 and 0.75, respectively. Another set of weighting coefficients may correspond with a motion vector of which the horizontal component and the vertical component have fractional parts equal to 0.5 and 0.875, respectively. The interpolator INT illustrated in FIG. 1 can provide numerous various different interpolation functions including the three interpolation functions that FIG. 4, 5, and 6 illustrate. The system controller SCT may specify the pixel window of a particular interpolation function by means of window configuration data WD, which the system controller SCT applies to the interpolator INT. For example, the system controller SCT may specify that the interpolator INT should apply the 4 x 2 pixel window that FIG. 4 illustrates. The search data SRD, which the motion vector finder MVF applies to the interpolator INT, may specify the set of weighting coefficients of a particular interpolation function. In a motion vector search, the motion vector finder MVF may group several candidate motion vectors that have identical fractional parts. An identical set of weighting coefficients can be applied for these candidate motion vectors.

FIG. 7 illustrates the interpolator INT in greater detail. The interpolator INT comprises a multiplexer arrangement MUXA, 64 multipliers Ml, M2, .., M64, an accumulator arrangement ACCA, and a control module CTM. The interpolator INT thus comprises a number of multipliers that corresponds with the number of pixels in the set of pixels PS, namely 64.

The interpolator INT basically operates as follows. The control module CTM receives the window configuration data WD, which defines a particular pixel window. The control module CTM further receives the search data SRD, which defines a candidate motion vector or a group of candidate motion vectors. The control module CTM derives multiplexer control data MC for the multiplexer arrangement MUXA from the window configuration data WD and the search data SRD. The control module CTM further derives accumulator control data AC for the accumulation arrangement from the window configuration data WD. The control module CTM further derives respective multiplier coefficients K₁, K₂, .., K₆₄ for the respective multipliers Ml, M2, .., M64 from the window configuration data WD and the search data SRD.

The multiplexer arrangement MUXA receives the set of pixels PS that FIG. 2 illustrates. The multiplexer arrangement MUXA provides respective input pixels PI₁, PI₂, .., PI₆₄ for the respective multipliers Ml, M2, .., M64. An input pixel for a multiplier corresponds with a particular pixel in the set of pixels PS. The multiplexer control data MC defines for each multiplier which pixel from the set of input pixels constitutes the input pixel for that multiplier. This will be described in greater detail hereinafter.

The respective multipliers Ml, M2, .., M64 multiply the respective input pixels P₁, PI₂, .., PI₆₄ with the respective multiplier coefficients K₁, K₂, .., K₆₄ that the control module CTM provides. Accordingly, the respective multipliers Ml, M2, .., M64 provide respective weighted pixels PWi, PW₂, .., PW₆₄. There is a total of 64 weighted pixels; each multiplier produces one weighted pixel, which is the input pixel for that multiplier multiplied by the multiplier coefficient for that multiplier.

The accumulator arrangement ACCA effectively forms different sets of weighted pixels. Each set corresponds with a pixel window that has a size and a shape as defined by window configuration data WD. The accumulator arrangement ACCA sums the weighted pixels in each set. Accordingly, respective sums of weighted pixels are obtained. These respective sums constitute respective interpolated pixels XP₁ k_i, .., XPi k N that form the set of interpolated pixels XPS, which the motion vector finder MVF illustrated in FIG. 1 receives.

FIG. 8 illustrates the multiplexer arrangement MUXA. The multiplexer arrangement MUXA comprises two sets of multiplexer groups. A first set comprises multiplexer groups MUXGl, .., MUXG8. Each of these multiplexer groups receives pixels that have odd row numbers in the set of pixels PS illustrated in FIG. 2. Accordingly, multiplexer group MUXGl receives pixels Pi₁I, .., P_16;1 and pixels P_1;3, .., P_16;3. The same applies to the other multiplexer groups in the first set. The first set of multiplexer groups MUXGl, .., MUXG8 provides input pixels PIi, .. PI₃₂ for multipliers Ml, .. M32, respectively. That is, the first set of multiplexer groups MUXGl, .., MUXG8 provides a first half of the respective input signals for the respective multipliers. The second set comprises multiplexer groups MUXG9, .., MUXG 16. Each of these multiplexer groups receives pixels that have even row numbers. Accordingly, multiplexer group receives pixels Pi₁₂, .., P₁₆,₂ and pixels P_1;4, .., P_16;4. The same applies to the other multiplexer groups in the second set. The second set of multiplexer groups MUXG9, .., MUXG16 provides input pixels PI₃₃, .. PI₆₄ for multipliers M33, .., M64, respectively. That is, the second set of multiplexer groups MUXG9, .., MUXG16 provides a second half of the respective input signals for the respective multipliers.

FIG. 9 illustrates multiplexer group MUXGl, which belongs to the first set of multiplexer groups. Multiplexer group MUXGl comprises four multiplexers MUXl 1, MUX12, MUX13, MUX14. Each multiplexer provides an input pixel for a particular multiplier: multiplexers MUXl 1, MUX12, MUX13, MUX14 provide input pixels PIi, PI₂, PI₃, PI₄, respectively. Each multiplexer receives eight different pixels from the set of pixels PS, which is present in the register memory RGM. Multiplexer MUXl receives four pixels from the first row and four pixels from the third row. There is a fixed step size of four pixel units between the pixels that are on the same row. A similar step size exists for the other multiplexers MUX 12, MUX13, MUX 14, which equally receives four pixels from the first row and four pixels from the third row. The other multiplexer groups of the first set are identical to multiplexer group MUXGl illustrated in FIG. 9.

FIG. 10 illustrates multiplexer group MUXG9, which belongs to the first set of multiplexer groups. Multiplexer group MUXG9 has a structure that is identical to that of multiplexer group illustrated in FIG. 9. Multiplexer group comprises four multiplexers MUX91, MUX92, MUX93, MUX94 that provides input pixels PI₃₃, PI₃₄, PI₃₅, PI₃₆, respectively. Each of these four multiplexers receives four different pixels from the second row and four different pixels from the fourth row in the set of pixels PS, which is present in the register memory RGM. The respective pixels that the respective multiplexers MUX91, MUX92, MUX93, MUX94 receive follow a pattern that is identical to that in multiplexer group MUXGl illustrated in FIG. 9. The only difference is a shift of one row with regard to the respective pixels that the respective multiplexers MUXl 1, MUX 12, MUXl 3, MUX 14 receive. The other multiplexer groups of the second set are identical to multiplexer group MUXG9 illustrated in FIG. 10.

FIG. 11 illustrates the accumulator arrangement ACCA. The accumulator arrangement ACCA comprises a first adder ring ADDRl, a second adder ring ADDR2, and an interpolated pixel composer IPC. The first adder ring ADDRl comprises 32 adders, which can be divided in eight adder sections, each of which comprises four adders. FIG. 11 illustrates one of these eight adder sections: adder section ADDSl 1. The second adder ring ADDR2 has an identical structure. The second adder ring ADDR2 also comprises 32 adders, which can be divided in eight adder sections, one of which is shown in FIG. 11: adder section ADDS21. The interpolated pixel composer IPC comprises eight composition sections, which are identical. FIG. 11 illustrates one of these eight composition sections: composition section COSl. Composition section COSl comprises three different composition units: an eight pixel window composition unit 8PWl, a four pixel window composition unit 4PWl, and a two pixel window composition unit 2PWl. The other composition sections are identical.

The accumulator arrangement ACCA operates as follows. The first adder ring ADDRl receives weighted pixels PWi, .. PW32. These weighted pixels are derived from input pixels PI₁, .., PI32, which the first set of multiplexer groups MUXGl, .. MUXG8 illustrated in FIG. 8 provides. The first adder ring ADDRl can therefore be associated with the first set of multiplexer groups MUXGl, .. MUXG8 and, as a result, can be associated with the pixels on the first row and the third row of the set of pixels PS, which is present in the register memory RGM as illustrated in FIG. 3.

The second adder ring ADDR2 receives weighted pixels PW33, .. PWό4. These weighted pixels are derived from input pixels PI33, .., PIό4, which the second set of multiplexer groups MUXG9, .. MUXG16 illustrated in FIG. 8 provides. The second adder ring ADDR2 can therefore be associated with the second set of multiplexer groups MUXG9, .. MUXG 16 and, as a result, can be associated with the pixels on the second row and the fourth row of the set of pixels PS, which is present in the register memory RGM as illustrated in FIG. 3.

Each adder in the first adder ring ADDRl and the second adder ring ADDR2 adds two weighted pixels so as to provide a weighted pixel combination. FIG. 11 illustrates that adder section ADDSl 1, which comprises four adders, provides four weighted pixel combinations A₁₊₂, A2₊3, A3₊4, A4₊5. Weighted pixel combination A1₊2 results from an addition of weighted pixels PWi and PW₂. Weighted pixel combination A2₊3, A3₊4, and A4₊5 result from an addition of weighted pixels PW₂ and PW3, PW3 and PW4, and PW4 and PW5, respectively. Another adder, which belongs to another adder section, will provide a weighted pixel combination that results from an addition of weighted pixels PW5 and PW₆. Yet another adder, which belongs to yet another adder section, will provide a weighted pixel combination that results from an addition of weighted pixels PW3₂ and PWi, which can be considered as closing the ring. Accordingly, the first adder ring ADDRl provides 32 different weighted pixel combinations. In a similar fashion, the second adder ring ADDR2 equally provides 32 different weighted pixel combinations. FIG. 11 illustrates that adder section ADDS21, which comprises four adders, provides four weighted pixel combinations A33₊34, A34₊35, A35₊36, A36₊37. Weighted pixel combination A33₊34 results from an addition of weighted pixels PW33 and PW34. Weighted pixel combination A34₊35, A35₊36, and A36₊37 result from an addition of weighted pixels PW34 and PW35, PW35 and PW36, and PW36 and PW37, respectively.

Composition section COSl receives the four weighted pixel combinations A₁₊₂, A₂₊₃, A₃₊₄, A₄₊₅, which adder section ADDSl 1 in the first adder ring ADDRl provides, and receives the four weighted pixel combinations A33₊34, A34₊35, A35₊36, A36₊37, which adder section ADDS21 in the second adder ring ADDR2 provides. Composition section COSl can thus be associated with adder section ADDSl 1 and adder section ADDS21 in the first adder ring ADDRl and the second adder ring ADDR2, respectively. In a similar fashion, each of the other composition sections in the interpolated pixel composer IPC can be associated with a particular adder section in the first adder ring ADDRl and a particular adder section in the second adder ring ADDR2.

Composition section COSl provides one or more interpolated pixels on the basis of the weighted pixel combinations Ai₊₂, A₂₊3, A₃₊₄, A₄₊₅, A₃₃₊₃₄, A₃₄₊₃₅, A₃₅₊₃₆, A₃₆₊₃7 that composition section COSl receives. Only one of the three composition units is involved in this process: either the eight pixel window composition unit 8PWl, or the four pixel window composition unit 4PWl , or the two pixel window composition unit 2PWl . The composition unit that is involved in providing one or more interpolated pixels depends on the pixel window that the window configuration data WD specifies.

Let it be assumed, for example, that the window configuration data WD specifies a 4 x 2 pixel window, which comprises eight pixels from the set of pixels PS, which is present in the register memory RGM. In that case, the eight pixel window composition unit 8PWl will provide one interpolated pixel on the basis of the weighted pixel combinations A₁₊₂, A₂₊₃, A₃₊₄, A₄₊₅, A₃₃₊₃₄, A₃₄₊₃₅, A₃₅₊₃₆, A₃₆₊₃7 that composition section COSl receives. As another example, the four pixel window composition unit 4PWl will provide two interpolated pixels if the window configuration data WD specifies a 2 x 2 pixel window. FIG. 12 illustrates the eight pixel window composition unit 8PWl. The eight pixel window composition unit 8PWl comprises three adders. A first adder provides a sum of weighted pixel combinations. A second adder provides a different sum of weighted pixel combinations. A third adder provides a sum of the respective aforementioned sums. This sum constitutes interpolated pixel XPi ₈ i, which is a weighted combination of input pixels PI₁, PI₂, PI₃, PI₄, PI33, PI₃₄, PI35, PI₃₆.

FIG. 13 illustrates the four pixel window composition unit 4PWl. The four pixel window composition unit 4PWl comprises two identical circuits. A first circuit, which comprises two multiplexers MUXl 11, MUXl 12 and an adder, provides interpolated pixel XPi 4 1. A second circuit, which comprises two multiplexers MUXl 13, MUXl 14 and an adder, provides interpolated pixel XPi ₄_₂. In the first circuit, multiplexer MUXl 11 receives weighted pixel combinations A₁₊₂, A4₊5, , A34₊35, A35₊36 and applies one of these weighted pixel combinations to the adder. Multiplexer MUXl 12 receives weighted pixel combinations A2₊3, A₃₊₄, A₃₃₊₃₄, A₃₆₊37 and applies one of these weighted pixel combinations to the adder. The adder provides a sum of the respective weighted pixel combinations that the adder receives from multiplexers MUXl 11, MUXl 12. The same applies to the second circuit in which multiplexers MUXl 13, MUXl 14 play the same roles as multiplexers MUXl 11, MUXl 12, respectively, in the first circuit. FIG. 14 illustrates the two pixel window composition unit 2PWl . The two pixel window composition unit 2PWl comprises four multiplexers MUXl 15, MUXl 16, MUXl 17, MUXl 18. Each multiplexer receives a different pair of weighted pixel combinations and provides one of the weighted pixel combinations in the pair as an interpolated pixel. More specifically, multiplexer MUXl 15 receives weighted pixel combinations A₁₊₂, A2₊3. One of these weighted pixel combinations is selected to constitute interpolated pixel XPi 2 1. Multiplexer MUXl 16 receives weighted pixel combinations A3₊4, A4₊5, one of which is selected to constitute interpolated pixel XPi 2 2- Multiplexer MUXl 17 receives weighted pixel combinations A₃₃₊₃₄, A₃₄₊₃₅, one of which is selected to constitute interpolated pixel XPi ₂_₃. Multiplexer MUXl 18 receives weighted pixel combinations A₃₅₊₃₆, A₃6₊₃7, one of which is selected to constitute interpolated pixel XPi ₂_₄.

FIG. 15 illustrates eight interpolated pixels XPi s i, XPi ₈_₂, XPi s 3, XPi 8 4, XPi 8 5, XPi 8 6, XPi 8 7, XPi 8 8 that have different positions. Each interpolated pixel is formed on the basis of a 4 x 2 pixel window. The respective 4 x 2 pixel windows for the respective interpolated pixels are presented by means of rectangles in broken lines. The interpolator INT can provide the eight interpolated pixels in parallel. This involves the following control actions.

The control module CTM illustrated in FIG. 7 appropriately configures the accumulator arrangement ACCA illustrated in FIG. 11 by means of accumulator control data AC. This causes composition section COSl to apply the eight pixel window composition unit 8PWl illustrated in FIG. 12. Seven other composition sections in the interpolated pixel composer IPC will each apply a similar eight pixel window composition unit.

The control module CTM further applies appropriate multiplier coefficients K₁, K₂, .., K54 to the multipliers Ml, M2, .., M64, respectively, in the interpolator INT. The multiplier coefficients K₁, K₂, .., K₆₄ comprise eight identical sets of eight multiplier coefficients. Each set of multiplier coefficients belongs to a particular 4 x 2 pixel window and a particular interpolated pixel illustrated in FIG. 15. The multipliers Ml, M2, .., M64 may correspondingly be considered to comprise eight sets of eight multipliers. Each set of multipliers receives a set of multiplier coefficients and is associated with a particular interpolated pixel illustrated in FIG. 16.

The control module CTM further causes the multiplexer arrangement MUXA to apply appropriate input pixels PI₁, PI₂, .., PI₆₄ to the multipliers Ml, M2, .., M64, respectively. The multiplexer control data MC, which the multiplexer arrangement MUXA receives from the control module CTM, effectively causes the multiplexer arrangement MUXA to form the respective pixel windows illustrated in FIG. 15.

For example, interpolated pixel XPi s i is formed on the basis of weighted pixel combinations Ai₊₂, A3₊4, A33₊34, A35₊36, which is illustrated in FIG. 12. These weighted pixel combinations are formed on the basis of input pixels PI₁, PI₂, PI3, PI4, PI33, PI₃₄, PI35, PI36, which multiplexer groups MUXGl and MUXG9 illustrated in FIGS. 9 and 10, respectively, provide. The multiplexer control data MC may cause multiplexer MUXl 1 to select pixel Ps_;1 as input pixel PI₁. Multiplexers MUX12, MUX13, MUX14 may select pixels P_2;1, P_3;1, P_4ji as input pixels PI₂, PI₃, PI₄, respectively. In addition, multiplexers MUXl 1, MUX12, MUX13, MUX14 may select pixels Ps,₂, P₂,₂, P₃ ,₂, P4,₂ as input pixels PI₃₃, PI₃₄, PI35, PI₃₆, respectively. Accordingly, the multiplexer arrangement MUXA effectively defines the pixel window associated with interpolated pixel XPi s i in response to the multiplexer control data MC. The respective pixel windows for the other interpolated pixels XPi ₈_₂, .., XPi ₈_₈ are defined in a similar fashion through an appropriate configuration of the multiplexer arrangement MUXA.

Let it be assumed that the motion estimator ME illustrated in FIG. 1 is carrying out a motion vector search. This may require the interpolator INT to provide a further eight interpolated pixels similar to those illustrated in FIG. 15 but having different positions. That is, the respective pixel windows illustrated in FIG. 15 may need to be shifted one or more pixel units in a horizontal or in a vertical direction. Such a shift in position only requires a reconfiguration of the multiplexer arrangement MUXA through an appropriate modification of the multiplexer control data MC. The control module CTM need not modify multiplier coefficients K₁, K₂, .., K54 or accumulator control data AC. That is, the multiplier coefficients K₁, K₂, .., K54 and the accumulator control data AC can be static during at least a portion of the motion vector search. This allows the control module CTM to be of relatively simple design. Moreover, the interpolator INT can provide various different interpolated pixels belonging to various different candidate motion vectors in a relatively fast manner.

FIG. 16 illustrates 15 interpolated pixels that have different positions. Only two interpolated pixels XPi 4 1, XPi ₄_₂, are designated with a reference sign for reasons of simplicity. Each interpolated pixel is formed on the basis of a 2 x 2 pixel window. The respective 2 x 2 pixel windows for the respective interpolated pixels are presented by means of rectangles in broken lines. The interpolator INT can provide the 15 interpolated pixels in parallel, which involves the following control actions.

The control module CTM illustrated in FIG. 7 appropriately configures the accumulator arrangement ACCA illustrated in FIG. 11 by means of accumulator control data AC. This causes composition section COSl to apply the four pixel window composition unit 4PWl illustrated in FIG. 13. Seven other composition sections in the interpolated pixel composer IPC will each apply a similar four pixel window composition unit. Each four pixel window composition unit 4PWl comprises four multiplexers, each of which selects one out of four weighted pixel combinations as illustrated in FIG. 13. The accumulator control data AC controls the respective multiplexers in the respective four pixel window composition units. Accordingly, the accumulator control data AC contributes in defining the respective pixel windows which are associated with the interpolated pixels illustrated in FIG. 16.

The control module CTM further applies appropriate multiplier coefficients K₁, K₂, .., K54 to the multipliers Ml, M2, .., M64, respectively, in the interpolator INT. The multiplier coefficients K₁, K₂, .., K54 comprise 15 identical sets of four multiplier coefficients. Each set of multiplier coefficients belongs to a particular 2 x 2 pixel window and a particular interpolated pixel illustrated in FIG. 16. The multipliers Ml, M2, .., M64 may correspondingly be considered to comprise 15 sets of four multipliers. Each set of multipliers receives a set of multiplier coefficients and is associated with a particular interpolated pixel illustrated in FIG. 16. The control module CTM further causes the multiplexer arrangement MUXA to apply appropriate input pixels PI₁, PI₂, .., PIό4 to the multipliers Ml, M2, .., M64, respectively. The multiplexer control data MC, which the multiplexer arrangement MUXA receives from the control module CTM, and the accumulator control data AC effectively cause the multiplexer arrangement MUXA to form the respective pixel windows illustrated in FIG. 16.

For example, interpolated pixel XPi 4 1 is formed as follows. Referring to FIG. 13, multiplexer MUXl 11 selects weighted pixel combination A₃₄₊₃₅ and multiplexer MUXl 12 selects weighted pixel combination A2₊3. Weighted pixel combination A2₊3 is formed on the basis of input pixels PI₂ and PI3. Weighted pixel combination A34₊35 is formed on the basis of input pixels PI34 and PI35. Referring to FIG. 9, multiplexers MUXl 1 and MUX13 select pixels P_2;1 and P_3;1 as input pixels PI₂ and PI₃, respectively. Referring to FIG. 10, MUX91 and MUX93 select pixels P2,2 and P_3;2 as input pixels PI34 and PI35, respectively. Interpolated pixel XPi 4 2 is formed on the basis of input pixels in a similar manner through appropriate multiplexer control.

FIG. 17 illustrates 30 interpolated pixels that have different positions. Only four interpolated pixels XPi 2 1, XPi 2 2, XPi 2 3, XPi 2 4 are designated with a reference sign for reasons of simplicity . Each interpolated pixel is formed on the basis of a 2 x 1 pixel window. The respective 2 x 1 pixel windows for the respective interpolated pixels are presented by means of rectangles in broken lines. The interpolator INT can provide the 30 interpolated pixels in parallel, which involves the following control actions.

The control module CTM illustrated in FIG. 7 appropriately configures the accumulator arrangement ACCA illustrated in FIG. 11 by means of accumulator control data AC. This causes composition section to apply the two pixel window composition unit 2PWl illustrated in FIG. 14. Seven other composition sections in the interpolated pixel composer IPC will each apply a similar two pixel window composition unit. Each two pixel window composition unit 2PWl comprises four multiplexers, each of which selects one out of two weighted pixel combinations as illustrated in FIG. 14. The selected weighted pixel combination constitutes an interpolated pixel. The accumulator control data AC controls the respective multiplexers in the respective two pixel window composition units. Accordingly, the accumulator control data AC contributes in defining the respective pixel windows which are associated with the interpolated pixels illustrated in FIG. 17.

The control module CTM further applies appropriate multiplier coefficients K₁, K₂, .., Kό4 to the multipliers Ml, M2, .., M64, respectively, in the interpolator INT. The multiplier coefficients K₁, K₂, .., K₆₄ comprise 30 identical sets of two multiplier coefficients. Each set of multiplier coefficients belongs to a particular 2 x 1 pixel window and a particular interpolated pixel illustrated in FIG. 17. The multipliers Ml, M2, .., M64 may correspondingly be considered to comprise 30 sets of two multipliers. Each set of multipliers receives a set of multiplier coefficients and is associated with a particular interpolated pixel illustrated in FIG. 17.

The control module CTM further causes the multiplexer arrangement MUXA to apply appropriate input pixels PI₁, PI₂, .., PIό4 to the multipliers Ml, M2, .., M64, respectively. The multiplexer control data MC, which the multiplexer arrangement MUXA receives from the control module CTM, and the accumulator control data AC effectively cause the multiplexer arrangement MUXA to form the respective pixel windows illustrated in FIG. 17.

For example, interpolated pixel XPi 2 1 is formed as follows. Referring to FIG. 14, multiplexer MUXl 15 selects weighted pixel combination A2₊3. Weighted pixel combination A2₊3 is formed on the basis of input pixels PI₂ and PI3. Referring to FIG. 9, multiplexers MUX12 and MUX 13 select pixels P_2;1 and P_3;1 as input pixels PI₂ and PI₃, respectively. Interpolated pixels XPi 2 2, XPi 2 3, XPi 2 4 are formed on the basis of input pixels in a similar manner through appropriate multiplexer control. It should be noted that the interpolator INT illustrated in FIG. 7 is particularly suited to establish respective weighted sums for pixels on a sub-sampled grid. A sub-sampled grid is obtained, for example, by leaving out even rows and even columns in the pixel grid that FIG. 2 illustrates by means of dotted lines. Such a sub-sampling can be realized without any particular difficulty through an appropriate control of the multiplexer arrangement MUXA.

It should further be noted that the motion compensator MC in the video coding system VCS, which FIG. 1 illustrates, may comprise an interpolator similar to the interpolator INT illustrated in FIG. 7. The interpolator can provide the predicted block of pixels BP when the motion vector MV has sub-pixel accuracy.

CONCLUDING REMARKS

The detailed description hereinbefore with reference to the drawings is merely an illustration of the invention and the additional features, which are defined in the claims. The invention can be implemented in numerous different manners. In order to illustrate this, some alternatives are briefly indicated.

The invention may be applied to advantage in any type of product or method that relates to video processing and which involves pixel filtering, in general, or pixel interpolation in particular. A video encoding system is merely an example. The invention may equally be applied in, for example, a video decoder. There are numerous different pixel windows that a pixel filter in accordance with the invention can provide. Figures 4, 5, and 6 merely illustrate a few examples. As another example, a pixel window may be 2 pixels wide and 4 pixels high. Any shape or size is possible. The size need not necessarily be a power of 2, although this is advantageous. There are numerous different manners to implement a multiplexer arrangement within a pixel filter in accordance with the invention. Figures 8, 9, and 10 merely illustrate an example, which is based on a set of pixels stored in a register memory as illustrated in figure 3. In this example, respective multiplexers receive respective groups of 8 pixels. In other embodiments, a multiplexer may receive a greater number of pixels or a smaller number of pixels. A different number of pixels may be appropriate if, for example, a different register memory is used in terms of size or configuration, or both.

The term "picture" should be understood in a broad sense. This term includes a frame, a field, and any other entity that may wholly or partially constitute an image or a picture. There are numerous ways of implementing functions by means of items of hardware or software, or both. In this respect, the drawings are very diagrammatic, each representing only one possible embodiment of the invention. Thus, although a drawing shows different functions as different blocks, this by no means excludes that a single item of hardware or software carries out several functions. Nor does it exclude that an assembly of items of hardware or software or both carry out a function.

The remarks made herein before demonstrate that the detailed description with reference to the drawings, illustrate rather than limit the invention. There are numerous alternatives, which fall within the scope of the appended claims. Any reference sign in a claim should not be construed as limiting the claim. The word "comprising" does not exclude the presence of other elements or steps than those listed in a claim. The word "a" or "an" preceding an element or step does not exclude the presence of a plurality of such elements or steps.

Claims

CLAIMS:

1. A video processor (ME) comprising a pixel filter (INT) for calculating a weighted sum of pixels that are present in a particular area of a picture (RP), the pixel filter comprising: a multiplexer arrangement (MUXA) for forming an array of input pixels (PI₁, PI₂, .. PI₆₄) on the basis of a selection of pixels that are present in the particular area of the picture; an array of multipliers (Ml, M2, .., M64) for multiplying the array of input pixels with an array of weighting coefficients (K₁, K₂, .., K₆₄) so as to obtain an array of weighted pixels (PWi, PW₂, .., PW₆₄); - an accumulator arrangement (ACCA) for forming respective combinations of weighted pixels in the array of weighted pixels.

2. A video processor according to claim 1, the accumulator arrangement (ACCA) comprising a ring of adders (ADDRl; ADDR2) couples to receive at least a portion of the array of weighted pixels (PWi, PW₂, .., PW₆₄), an adder within the ring providing a combination of weighted pixels.

3. A video processor according to claim 1, the multiplexer arrangement (MUXA) comprising a first set of multiplexer groups (MUXGl, .. MUXG8) coupled to receive pixels from odd lines and a second set of multiplexer groups (MUXG9, .. MUXG 16) coupled to receive pixels from even lines.

4. A video processor according to claim 3, a multiplexer group (MUXGl) comprising a plurality of multiplexers (MUXl 1, MUX12, MUX13, MUX14), each multiplexer of the multiplexer group being coupled to receive a different group of pixels and being arranged to select one of these pixels as an input pixel.

5. A video processor according to claim 2 and 3, the accumulator arrangement (ACCA) comprising a first adder ring (ADDRl) for forming respective combinations of weighted pixels that are formed on the basis of input pixels provided by the first set of multiplexer groups (MUXGl, .. MUXG8), and a second adder ring (ADDR2) for forming respective combinations of weighted pixels that are formed on the basis of input pixels provided by the second set of multiplexer groups (MUXG9, .. MUXG16).

6. A video processor according to claim 2, the accumulator arrangement (ACCA) comprising a plurality of composition sections, a composition section (COSl) being coupled to receive a plurality of combinations of weighted pixels (A1₊2, A2₊3, A3₊4, A4₊5, A33₊34, A34₊35, A35₊36, A36₊37) and comprising different composition units (8PWl, 4PWl, 2PW1) for different pixel windows.

7. A video processor according to claim 6, a composition section (COSl) comprising: a first composition unit (8PWl ) that comprises an adder tree for summing the plurality of combinations of weighted pixels that the composition section receives; a second composition unit (4PWl ) that comprises multiplexers (MUXl 11, .., MUX 114) for selecting particular combinations of weighted pixels and that comprises adders for summing respective groups of selected combinations of weighted pixels; and a third composition unit (2PW 1) that exclusively comprises multiplexers (MUXl 15, .., MUX 118) for selecting particular combinations of weighted pixels.

8. A method of video processing that involves calculating a weighted sum of pixels that are present in a particular area of a picture (RP), the method comprising: a multiplexer control step for causing a multiplexer arrangement (MUXA) to form an array of input pixels (PI₁, PI₂, .. PI64) on the basis of a selection of pixels that are present in the particular area of the picture; a multiplier control step for causing an array of multipliers to multiply the array of input pixels with an array of weighting coefficients (K₁, K₂, .., K54) so as to obtain an array of weighted pixels (PWi, PW₂, .., PW₆4); - an accumulator control step for causing an accumulator arrangement (ACCA) to form respective combinations of weighted pixels in the array of weighted pixels.

9. A computer program product for a programmable processor, the computer program product comprising a set of instructions that, when loaded into the programmable processor, causes the programmable processor to carry out the method according to claim 8.

10. A video apparatus (VCS) comprising a memory (MEMR) for storing at least a portion of a picture (RP) and a video processor (ME) according to claim 1 for providing a set of interpolated pixels (XPS) on the basis of the at least portion of the picture that is stored in the memory.