CA2360295A1 - System and method for image warping - Google Patents
System and method for image warping Download PDFInfo
- Publication number
- CA2360295A1 CA2360295A1 CA002360295A CA2360295A CA2360295A1 CA 2360295 A1 CA2360295 A1 CA 2360295A1 CA 002360295 A CA002360295 A CA 002360295A CA 2360295 A CA2360295 A CA 2360295A CA 2360295 A1 CA2360295 A1 CA 2360295A1
- Authority
- CA
- Canada
- Prior art keywords
- filter
- target pixel
- coefficients
- pixel
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims description 31
- 238000001914 filtration Methods 0.000 abstract description 3
- 238000013507 mapping Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000013461 design Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Classifications
-
- G06T3/18—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
Abstract
A system creates a target pixel in a target grid from a plurality of source pixels in a source grid. The system comprises a filter controller, a left portion lookup circuit, a right portion lookup circuit, a filter reconstruction unit, and a filter. The filter controller determines a first distance between the target pixel and a source pixel to the left of the target pixel, and a second distance between the target pixel and a source pixel to the right of the target pixel. The left portion lookup circuit determines filter coefficients for a left portion of the target pixel in accordance with the first difference. The right portion lookup circuit determines filter coefficients for a right portion of the target pixel in accordance with the second difference. The filter reconstruction unit creates a filter in accordance with the left and right portion filter coefficients for interpolating the target pixel from the plurality of source pixels. The filter performs the filtering operation according to the output of the filter reconstruction unit.
Description
System and Method For Image Warping The present invention relates generally to digital signal processing, and more particularly to a system and method for image warping.
Background of the Invention Image warping, also know as 3-Dimensional (3-D) rendering or texture mapping, can be used to create a 3-D projection from an original 2-Dimensional (2-D) first image. Image warping is used to create visual effects in different visual media including film, television 3D graphics and computer displays. It can be expressed as the process of mapping pixels from a source sample grid to a target sample grid. Typically, the target grid is sampled at a different frequency than the source grid. In general, the output grid does not vary in a constant manner with respect to the input grid. The variations from pixel to pixel between the source grid and the target grid can make the process of image warping extremely complex.
Several systems and methods for image warping are known. Detailed discussions of prior art image warping and systems are discussed in Wolberg, George, "Separable Image Warping: Implications and Techniques", Phd. Thesis, Department of Computer Science, Columbia University, New York, 1990 and "Digital Image Warping", IEE Computer Society Press, Los Alomos, California, 1990. The prior art systems described in these documents are typically complicated and expensive to implement due to the large number of computations required.
U.S. Patent No. 5,594,676 (herein referred to as the '676 patent) titled "Digital Image Warping System", issued to Greggain et al discloses a particular prior art implementation for image warping. The '676 patent overcomes some of the problems of the prior art by using an intermediate grid, upsampled from the source grid, but having corresponding pixels to the target grid. In this manner, the source pixels are first mapped to the upsampled grid, and then readily mapped to the target grid.
Despite offering certain advantages over the prior art, the '676 patent still presents a relatively complicated solution because it requires an intermediate, upsampled grid:
Upsampling requires extra circuitry and processing steps in order to achieve the desired digital image warping result. Furthermore, upsampling introduces distortion and can soften the image, which are often undesirable effects.
Therefore, it is an object of the present invention to obviate or mitigate at least some of the above mentioned disadvantages.
Summary of the Invention In accordance with an aspect of the present invention, there is provided a system for generating a target pixel in a target grid from a plurality of source pixels in a source grid.
The system comprises a filter controller, a left portion lookup circuit, a right portion lookup circuit, a filter reconstruction unit, and a filter. °'The filter controller determines a first distance between the target pixel and a source pixel to the left of the target pixel, and a second distance between the target pixel and a source pixel to the right of the target pixel. The left portion lookup circuit determines filter coefficients for a left portion of the target pixel in accordance with the first difference. The right portion lookup circuit determines filter coefficients for a right portion of the target pixel in accordance with the second difference. The filter reconstruction unit creates a filter in accordance with the left and right portion filter coefficients for interpolating the target pixel from the plurality of source pixels. The filter performs the filtering operation according to the output of the filter reconstruction unit.
In accordance with a fiu ther aspect of the invention the left portion and the right portion of the filter are calculated separately without reconstructing the filter.
In accordance with another aspect of the invention, there is provided a method for generating a target pixel in a target grid from a plurality of source pixels in a source grid.
The method comprises the following steps. A first distance between the target pixel and a source pixel to the left of said target pixel, and a second distance between the target pixel and a source pixel to the right of the target pixel are determined. A
first set of filter coefficients for a left portion of the target pixel is determined in accordance with the first distance. A second set filter coefficients for a right portion of the target pixel is determined in accordance with the second distance. The first and second sets of coefficients are combined for creating a final filter for the target pixel.
The final filter is used for generating the target pixel.
Brief Description of the Drawings An embodiment of the present invention will now be described by way of example only with reference to the following drawings, in which:
Figure 1 is a schematic representation of a plurality of source pixels and a target pixel derived therefrom;
Figure 2 is a block diagram of a system for image warping; and Figure 3 is a flowchart of a method for image warping.
Detailed Description of the Preferred Embodiment The present invention relates to a method and system for image warping, also called texture mapping, which processes the pixels first horizontally and then vertically. The description of the preferred embodiments will be of the horizontal process only, however the vertical process is similar with the input data being scanned vertically instead of horizontally. Likewise, the process could be performed vertically first and horizontally second as desired. The invention allows for the design of filters that are asymmetric on either side of each output pixel by using filters that have been designed for asymmetry.
The embodiments describe the method of design for the filters and a method for their use in image warping or texture mapping.
In one embodiment, the method selects filter coefficients for the left portion of the target output pixel by analyzing the distance in the input space between the target output pixel and the output pixel to the left of the target output pixel. The method then selects filter coefficients for the right portion of the target output pixel by the distance in the input space between the target output pixel and the output pixel to the right of the target output pixel. The filter coefficients are combined by simple addition and applied to the input data for producing the target pixel.
The filters used may be created from any poly-phase filter set using the simple technique for separating the filters into left and right portions. The filters may have an even number or an odd number of taps provided that both the left portion and right portion for any target pixel are selected from filters with an even number of taps or both selected from filters with an odd number of taps. The filter for the left portion of the target pixel may have a different frequency response and even a different number of taps than the filter used for the right portion of the target pixel.
For convenience, like numerals in the description refer to like elements in the drawings.
Referring to Figure 1, an array of target pixels and source pixels is illustrated generally by numeral 10. A plurality of source pixels 12 are used for generating a plurality of target pixels 14. The position of the target pixels may be determined by any method that is known in the art, and thus will not be described in detail. Examples of such methods include accessing a file containing the locations of the output or target pixels; using a polynomial description of the location of the output pixels with respect to the input grid;
and implementing a set of wire frames that approximates the surface of the object onto which the 2D image is to be projected.
The target pixels 14 are generated based on their spatial relationship with neighboring pixels as well as the source pixels 12. For example, a first target pixel To is separated by a horizontal distance D1 from a second target pixel T1, which is separated by a second horizontal distance D2 from a third pixel T2. A phase value ~L is determined by the distance between the target pixel Tl and a nearest input pixel S1 to the left of target pixel Tl. Alternately, phase ~R can be calculated as the distance between the target pixel Tl and a source pixel S2 that is the nearest input pixel to the right of the target T1. Either calculation can be performed, since ~R=1-~,,.
Background of the Invention Image warping, also know as 3-Dimensional (3-D) rendering or texture mapping, can be used to create a 3-D projection from an original 2-Dimensional (2-D) first image. Image warping is used to create visual effects in different visual media including film, television 3D graphics and computer displays. It can be expressed as the process of mapping pixels from a source sample grid to a target sample grid. Typically, the target grid is sampled at a different frequency than the source grid. In general, the output grid does not vary in a constant manner with respect to the input grid. The variations from pixel to pixel between the source grid and the target grid can make the process of image warping extremely complex.
Several systems and methods for image warping are known. Detailed discussions of prior art image warping and systems are discussed in Wolberg, George, "Separable Image Warping: Implications and Techniques", Phd. Thesis, Department of Computer Science, Columbia University, New York, 1990 and "Digital Image Warping", IEE Computer Society Press, Los Alomos, California, 1990. The prior art systems described in these documents are typically complicated and expensive to implement due to the large number of computations required.
U.S. Patent No. 5,594,676 (herein referred to as the '676 patent) titled "Digital Image Warping System", issued to Greggain et al discloses a particular prior art implementation for image warping. The '676 patent overcomes some of the problems of the prior art by using an intermediate grid, upsampled from the source grid, but having corresponding pixels to the target grid. In this manner, the source pixels are first mapped to the upsampled grid, and then readily mapped to the target grid.
Despite offering certain advantages over the prior art, the '676 patent still presents a relatively complicated solution because it requires an intermediate, upsampled grid:
Upsampling requires extra circuitry and processing steps in order to achieve the desired digital image warping result. Furthermore, upsampling introduces distortion and can soften the image, which are often undesirable effects.
Therefore, it is an object of the present invention to obviate or mitigate at least some of the above mentioned disadvantages.
Summary of the Invention In accordance with an aspect of the present invention, there is provided a system for generating a target pixel in a target grid from a plurality of source pixels in a source grid.
The system comprises a filter controller, a left portion lookup circuit, a right portion lookup circuit, a filter reconstruction unit, and a filter. °'The filter controller determines a first distance between the target pixel and a source pixel to the left of the target pixel, and a second distance between the target pixel and a source pixel to the right of the target pixel. The left portion lookup circuit determines filter coefficients for a left portion of the target pixel in accordance with the first difference. The right portion lookup circuit determines filter coefficients for a right portion of the target pixel in accordance with the second difference. The filter reconstruction unit creates a filter in accordance with the left and right portion filter coefficients for interpolating the target pixel from the plurality of source pixels. The filter performs the filtering operation according to the output of the filter reconstruction unit.
In accordance with a fiu ther aspect of the invention the left portion and the right portion of the filter are calculated separately without reconstructing the filter.
In accordance with another aspect of the invention, there is provided a method for generating a target pixel in a target grid from a plurality of source pixels in a source grid.
The method comprises the following steps. A first distance between the target pixel and a source pixel to the left of said target pixel, and a second distance between the target pixel and a source pixel to the right of the target pixel are determined. A
first set of filter coefficients for a left portion of the target pixel is determined in accordance with the first distance. A second set filter coefficients for a right portion of the target pixel is determined in accordance with the second distance. The first and second sets of coefficients are combined for creating a final filter for the target pixel.
The final filter is used for generating the target pixel.
Brief Description of the Drawings An embodiment of the present invention will now be described by way of example only with reference to the following drawings, in which:
Figure 1 is a schematic representation of a plurality of source pixels and a target pixel derived therefrom;
Figure 2 is a block diagram of a system for image warping; and Figure 3 is a flowchart of a method for image warping.
Detailed Description of the Preferred Embodiment The present invention relates to a method and system for image warping, also called texture mapping, which processes the pixels first horizontally and then vertically. The description of the preferred embodiments will be of the horizontal process only, however the vertical process is similar with the input data being scanned vertically instead of horizontally. Likewise, the process could be performed vertically first and horizontally second as desired. The invention allows for the design of filters that are asymmetric on either side of each output pixel by using filters that have been designed for asymmetry.
The embodiments describe the method of design for the filters and a method for their use in image warping or texture mapping.
In one embodiment, the method selects filter coefficients for the left portion of the target output pixel by analyzing the distance in the input space between the target output pixel and the output pixel to the left of the target output pixel. The method then selects filter coefficients for the right portion of the target output pixel by the distance in the input space between the target output pixel and the output pixel to the right of the target output pixel. The filter coefficients are combined by simple addition and applied to the input data for producing the target pixel.
The filters used may be created from any poly-phase filter set using the simple technique for separating the filters into left and right portions. The filters may have an even number or an odd number of taps provided that both the left portion and right portion for any target pixel are selected from filters with an even number of taps or both selected from filters with an odd number of taps. The filter for the left portion of the target pixel may have a different frequency response and even a different number of taps than the filter used for the right portion of the target pixel.
For convenience, like numerals in the description refer to like elements in the drawings.
Referring to Figure 1, an array of target pixels and source pixels is illustrated generally by numeral 10. A plurality of source pixels 12 are used for generating a plurality of target pixels 14. The position of the target pixels may be determined by any method that is known in the art, and thus will not be described in detail. Examples of such methods include accessing a file containing the locations of the output or target pixels; using a polynomial description of the location of the output pixels with respect to the input grid;
and implementing a set of wire frames that approximates the surface of the object onto which the 2D image is to be projected.
The target pixels 14 are generated based on their spatial relationship with neighboring pixels as well as the source pixels 12. For example, a first target pixel To is separated by a horizontal distance D1 from a second target pixel T1, which is separated by a second horizontal distance D2 from a third pixel T2. A phase value ~L is determined by the distance between the target pixel Tl and a nearest input pixel S1 to the left of target pixel Tl. Alternately, phase ~R can be calculated as the distance between the target pixel Tl and a source pixel S2 that is the nearest input pixel to the right of the target T1. Either calculation can be performed, since ~R=1-~,,.
Referring to Figure 2, a system for image warping is indicated generally by numeral 20.
The system 20 includes a data controller 24, a data buffer 26, a time step controller 28, a delay element 36, a target pixel lookup 40, a filter reconstruction block 48, and a mapping filter 52. Input is entered to the data buffer 26, which is coupled with both the mapping filter 52 and the data controller 24. The data controller 24 is coupled to the time step controller 28, as is the target pixel lookup 40. Furthermore, target pixel lookup 40 is also coupled with the time step controller 28 via the delay element 36. The target pixel lookup 40 is coupled to the filter reconstruction block 48, which is coupled to the mapping filter 52.
The time step controller 28 measures the difference between pixels and calculates the phase values ~L and ~R. The time step controller communicates this information to the filter lookup 40. The phase value ~L for the left portion lookup is sent directly to the filter lookup 40, while the phase value ~ for the right portion lookup is sent to the filter lookup 40 via the delay element 36. This reduces the circuitry required by allowing the left portion filter lookup and right portion filter lookup to share the same circuitry.
Alternately, a delay is not required if separate left and right portion look up circuitry is used, as will be appreciated by a person skilled in the art.
Once the left and right filter portions have been determined, the filter reconstruction block 48 combines them for creating a complete set of filter characteristics.
The filter characteristics are then provided to the mapping filter 52 for filtering the data. The data is received at the mapping filter 52 from the data buffer 26. The data flow into and out of the data buffer 26 is controlled by the data controller 24, which also receives the timing from the time step controller.
An example of the operation of the various components of the system will now be described with reference to Figure 1, Figure 2, and Figure 3. It is assumed that for the present embodiment, a four-tap filter size is used. However, it will be apparent to a person skilled in the art that other filter sizes can be used, as desired.
Furthermore, a plurality of filter sizes can be accommodated on either side of the target pixel.
The system 20 includes a data controller 24, a data buffer 26, a time step controller 28, a delay element 36, a target pixel lookup 40, a filter reconstruction block 48, and a mapping filter 52. Input is entered to the data buffer 26, which is coupled with both the mapping filter 52 and the data controller 24. The data controller 24 is coupled to the time step controller 28, as is the target pixel lookup 40. Furthermore, target pixel lookup 40 is also coupled with the time step controller 28 via the delay element 36. The target pixel lookup 40 is coupled to the filter reconstruction block 48, which is coupled to the mapping filter 52.
The time step controller 28 measures the difference between pixels and calculates the phase values ~L and ~R. The time step controller communicates this information to the filter lookup 40. The phase value ~L for the left portion lookup is sent directly to the filter lookup 40, while the phase value ~ for the right portion lookup is sent to the filter lookup 40 via the delay element 36. This reduces the circuitry required by allowing the left portion filter lookup and right portion filter lookup to share the same circuitry.
Alternately, a delay is not required if separate left and right portion look up circuitry is used, as will be appreciated by a person skilled in the art.
Once the left and right filter portions have been determined, the filter reconstruction block 48 combines them for creating a complete set of filter characteristics.
The filter characteristics are then provided to the mapping filter 52 for filtering the data. The data is received at the mapping filter 52 from the data buffer 26. The data flow into and out of the data buffer 26 is controlled by the data controller 24, which also receives the timing from the time step controller.
An example of the operation of the various components of the system will now be described with reference to Figure 1, Figure 2, and Figure 3. It is assumed that for the present embodiment, a four-tap filter size is used. However, it will be apparent to a person skilled in the art that other filter sizes can be used, as desired.
Furthermore, a plurality of filter sizes can be accommodated on either side of the target pixel.
Referring to figure 3, the system receives a plurality of source pixels at step 100. At step 110, the system receives target pixel information, including the desired location of the target pixels. At step 115, the received source and target pixel information is used to calculate the distances D1, D2, ~1, and ~z, referred to in figure 1.
At step 120, the left-potion pixel filter is determined. The current iteration of the target pixel calculation is for target pixel T1. The distance D1 between target pixel Tl and target pixel To is used for selecting from a plurality of filters to be used for the left portion. In the present embodiment, there are two filters, Filter 1 and Filter 2, from which to select.
However, it will be apparent to a person skilled in the art that a lesser or greater number of filters may be used.
The appropriate filter is selected using a threshold comparison. If the distance DI lies between a pair of predefined thresholds, then the first filter, Filter 1, is selected. If, however, D~ lies between a different pair of predefined thresholds, then the second filter, Filter 2, is selected. For the present example, it is assumed that the distance D1 results in the second filter, Filter 2, being selected.
The phase of the second filter is determined by the distance ~ between the target pixel Tl;
and the closest source pixel to its left Sl. The unit of measurement is the distance between source pixels so, therefore, ~, will always be less than 1. For the present example, it is assumed that the distance ~, is equal to 0.7.
The filter coefficients for the left portion are provided in a lookup table.
An example of a filter coefficient lookup table having two possible filters and five different phases is provided below in Table 1.
At step 120, the left-potion pixel filter is determined. The current iteration of the target pixel calculation is for target pixel T1. The distance D1 between target pixel Tl and target pixel To is used for selecting from a plurality of filters to be used for the left portion. In the present embodiment, there are two filters, Filter 1 and Filter 2, from which to select.
However, it will be apparent to a person skilled in the art that a lesser or greater number of filters may be used.
The appropriate filter is selected using a threshold comparison. If the distance DI lies between a pair of predefined thresholds, then the first filter, Filter 1, is selected. If, however, D~ lies between a different pair of predefined thresholds, then the second filter, Filter 2, is selected. For the present example, it is assumed that the distance D1 results in the second filter, Filter 2, being selected.
The phase of the second filter is determined by the distance ~ between the target pixel Tl;
and the closest source pixel to its left Sl. The unit of measurement is the distance between source pixels so, therefore, ~, will always be less than 1. For the present example, it is assumed that the distance ~, is equal to 0.7.
The filter coefficients for the left portion are provided in a lookup table.
An example of a filter coefficient lookup table having two possible filters and five different phases is provided below in Table 1.
Table 1 Filter Coefficients Filter Phase h_2 111 hl hi 0.9 -1 48 88 -7 0.7 -2 58 78 -6 1 0.5 -4 68 68 -4 0.3 -6 78 58 -2 0.1 -7 88 48 -1 0.9 2 15 98 13 0.7 8 31 69 20 2 0.5 16 48 48 16 0.3 20 69 31 8 0.1 13 98 15 2 The filter coefficients h_2, h_1, hl, and h2 are applied to filter taps starting from the second closest pixel to the left of the target pixel and ending with the second closest source pixel to the right of the target pixel. Therefore, for the present example where the target pixel is T1, h_2 is applied to S0, h_1 is applied to S1, hl is applied to S2, and h2 is applied to S3.
Furthermore, it can be seen from Table 1 that the nominal sum of the filter coefficients is 128 for each phase. In the present embodiment the nominal sum is selected to be 128 and the output of the filter is divided by 128 in order to be normalized. In point of fact, filters may be designed so that their coefficients have any sum. In general a power of 2 is selected to be the nominal sum and the filter is normalized so that that the power of 2 represents a gain of 1. The output of the filter is divided by the power of 2 to be normalized.
The filter coefficients for each phase can be separated into a left portion and a right portion. The filter coefficients for the left portion are listed below in Table 2.
Furthermore, it can be seen from Table 1 that the nominal sum of the filter coefficients is 128 for each phase. In the present embodiment the nominal sum is selected to be 128 and the output of the filter is divided by 128 in order to be normalized. In point of fact, filters may be designed so that their coefficients have any sum. In general a power of 2 is selected to be the nominal sum and the filter is normalized so that that the power of 2 represents a gain of 1. The output of the filter is divided by the power of 2 to be normalized.
The filter coefficients for each phase can be separated into a left portion and a right portion. The filter coefficients for the left portion are listed below in Table 2.
Table 2 Filter Coefficients Filter Phase h_2' h_1' hl' h2' 0.9 -1 48 17 0 0.7 -2 58 8 0 1 0.5 -4 68 0 0 0.3 -6 70 0 0 0.1 -7 71 0 0 0.9 2 15 47 0 0.7 8 31 25 0 2 0.5 16 48 0 0 0.3 20 43 0 0 0.1 13 51 0 0 Note that the nominal sum of the left portion filter coefficients are 64, half of that for the overall filter. The left portion filter coefficients are calculated as follows:
h_2' = h_2 h2' = 0 temp='h-(112+h_1) If temp > 0 h_i' = h_1 hl' = temp else h_1' _ ~2 - h_2 111' = 0 where'h = half the nominal sum = %2 (128) = 64, for the present example.
Since the distance ~L is 0.7, the left portion coefficients are {8, 31, 25, 0}. The contribution to the left portion from the source pixels So and S1 on the left side of the target pixel Tl is {8, 31 }. Since the sum of their contribution is equal to 39, which is less than 64, an adjustment from the closest source pixel S2 to the right of the target pixel T1 is required. The value of the contribution from source pixel S2 is {25}, which compensates the difference. Generally, the compensation CR required from source pixel S2 is:
CR=64-(1z2+h_1) Once the left portion filter coefficients are determined, step 120 is complete. At step 130, similar steps are taken to calculate the right portion filter coefficients.
For the right portion, the distance D2 between target pixel Tl and target pixel T2 is used for selecting from the two filters, Filter 1 and Filter 2.
Similar to the left portion, the appropriate filter is selected using a threshold comparison.
If the distance DZ lies between a pair of predefined thresholds, then the first filter, Filter 1, is selected. If, however, D2 lies between a different pair of predefined thresholds, then the second filter, Filter 2, is selected. For the present example, it is assumed that the distance D2 results in the first filter, Filter 1, being selected.
The phase of the first filter is also determined by the distance ~, between the target pixel Tl and the closest source pixel to its left Si. Therefore, the distance ~L is equal to 0.7.
The filter coefficients for the right portion of the filter are listed below in Table 3.
Table 3 Filter Coefficients Filter Phase m hlm h2m 0.9 0 0 71 -7 0.7 0 0 70 -6 1 0.5 0 0 68 -4 0.3 0 8 58 -2 0.1 0 17 48 -1 0.9 0 0 51 13 0.7 0 0 43 20 2 0.5 0 0 48 16 0.3 0 25 31 8 0.1 0 47 15 2 It can be seen from Table 3 that the nominal sum of the right portion filter coefficients is 64 for each phase. The right portion filter coefficients are calculated as follows:
h2 » - h2 h2> > - 0 temp = %z - (h2 + hl) If temp > 0 h1 » = hl h_1" = temp else hl»=,~2-h2 h1» = 0 Since the distance ~L is 0.7, the right portion coefficients are {0, 0, 70, -6}. The contribution to the right portion from the source pixels S2 and S3 on the right side of the target pixel Tl is {70, -6}. Since the sum of their contribution is equal to 64, an adjustment from the closest source pixel Si to the left of the target pixel T1 is not required. Generally, the however, compensation CL required from source pixel S1 is:
Cr.=64-(h2+hi) Note that the table has been constructed such that the phase is determined by the distance ~L. However, since ~L = 1- ~R, it is also possible to determine the phase of the filter for the right portion by examining the distance ~R.
Once the right portion filter coefficients are determined, step 130 is complete and an overall target pixel filter is reconstructed at step 140. The filter reconstruction combines the selected left portion coefficients and right portion coefficients by addition. Therefore, the resulting constructed filter is {8, 31, 95, -6}. The sum of the reconstructed filter coefficients is 128.
At step 150, the time step control determines whether or not more target pixels need to be created. In the present example, there is yet another target pixel T2 to be created.
Therefore, the process returns to step 120 and the left portion filter coefficients for T2 are determined.
The distance D2 between T2 and T1 is used for determining which filter to select for the left portion filter coefficients. Therefore, the same filter, Filter 2, is selected for the left portion filter coefficients for target pixel T2 as was selected for the right portion filter coefficients for target pixel Tl. However, the phase is different, since the distance between target pixel T2 and source pixel S3 is different from that between target pixel T1 and source pixel S 1. For the target pixel T2, the phase is calculated to be 0.1. Therefore, from Table 3, the left portion coefficients are {13, 51, 0, 0}.
At step 130, the right portion filter coefficients for T2 are determined. A
distance D3 between target pixel T2 and target pixel T3 is used for determining which of the two filters to use for the right portion filter coefficients. Using the predefined thresholds, it is determined that the first filter, Filter 1 is to be used. From Table 2, the right portion coefficients are {0, 17, 48 -1 }.
At step 140, filter coefficients are reconstructed by addition of the left portion filter coefficients and the right portion filter coefficients. Therefore, the reconstructed filter coefficients are { 13, 68, 48, -1 }.
In the present embodiment, filters having an even number (four) of taps were illustrated.
However, it is possible to implement the invention using filters have an odd number of taps. In an alternate embodiment, five-tap filters are used instead of the four-tap filters described in the previous embodiment. The general process is similar to the previous embodiment, the main difference being the separation of the overall filter coefficients into left portion and right portion coefficients.
Table 4 is an example of a five-tap filter, and its corresponding adjusted coefficients.
h Originals-tap20 -10 98 -20 40 filter Adjusted filter~2 hl h~ ~ hl h2 ortions Left portion 20 -10 54 0 0 0 Right portion0 0 0 44 -20 40 The center of the five taps, ho is separated into a left center tap hL,o and a right center tap , hRo. The left portion of the filter is determined by preserving the filter coefficients for the filter taps left of the centermost tap. The sum of the left portion of the filter coefficients is given by LP = h_2 + 1L1 +'/Zho. The left center tap is calculated as hL,o ='/z +'hho - LP. A similar procedure is followed for the right center tap, where RP
= h2 + hl +'/2ho and hRo ='h + ~2hQ - RP.
When the filter tap coefficients are reconstructed, hl,o of the selected left portion coefficients is added to hRa of the selected right portion coefficients for reconstructing the center coefficient ho.
Furthermore, it is common in resampling systems to use filters of different lengths for different bandwidths. For example, a half band filter with four taps could be used for reducing the number of samples by a factor of two. Similarly, a quarter band filter with eight taps could be used for reducing the number of samples by a factor of four.
It will be apparent to a person skilled in the art how to apply the invention to filters of differing lengths for the left and right portions. For example, a six-tap filter is used for the left portion and a four-tap filter is used for the right portion. The resulting filter is a five-tap filter (h_3, h_2, 1y, hl, h2).
Although the invention has been described with reference to certain specific embodiments, various modifications thereof will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the claims appended hereto.
h_2' = h_2 h2' = 0 temp='h-(112+h_1) If temp > 0 h_i' = h_1 hl' = temp else h_1' _ ~2 - h_2 111' = 0 where'h = half the nominal sum = %2 (128) = 64, for the present example.
Since the distance ~L is 0.7, the left portion coefficients are {8, 31, 25, 0}. The contribution to the left portion from the source pixels So and S1 on the left side of the target pixel Tl is {8, 31 }. Since the sum of their contribution is equal to 39, which is less than 64, an adjustment from the closest source pixel S2 to the right of the target pixel T1 is required. The value of the contribution from source pixel S2 is {25}, which compensates the difference. Generally, the compensation CR required from source pixel S2 is:
CR=64-(1z2+h_1) Once the left portion filter coefficients are determined, step 120 is complete. At step 130, similar steps are taken to calculate the right portion filter coefficients.
For the right portion, the distance D2 between target pixel Tl and target pixel T2 is used for selecting from the two filters, Filter 1 and Filter 2.
Similar to the left portion, the appropriate filter is selected using a threshold comparison.
If the distance DZ lies between a pair of predefined thresholds, then the first filter, Filter 1, is selected. If, however, D2 lies between a different pair of predefined thresholds, then the second filter, Filter 2, is selected. For the present example, it is assumed that the distance D2 results in the first filter, Filter 1, being selected.
The phase of the first filter is also determined by the distance ~, between the target pixel Tl and the closest source pixel to its left Si. Therefore, the distance ~L is equal to 0.7.
The filter coefficients for the right portion of the filter are listed below in Table 3.
Table 3 Filter Coefficients Filter Phase m hlm h2m 0.9 0 0 71 -7 0.7 0 0 70 -6 1 0.5 0 0 68 -4 0.3 0 8 58 -2 0.1 0 17 48 -1 0.9 0 0 51 13 0.7 0 0 43 20 2 0.5 0 0 48 16 0.3 0 25 31 8 0.1 0 47 15 2 It can be seen from Table 3 that the nominal sum of the right portion filter coefficients is 64 for each phase. The right portion filter coefficients are calculated as follows:
h2 » - h2 h2> > - 0 temp = %z - (h2 + hl) If temp > 0 h1 » = hl h_1" = temp else hl»=,~2-h2 h1» = 0 Since the distance ~L is 0.7, the right portion coefficients are {0, 0, 70, -6}. The contribution to the right portion from the source pixels S2 and S3 on the right side of the target pixel Tl is {70, -6}. Since the sum of their contribution is equal to 64, an adjustment from the closest source pixel Si to the left of the target pixel T1 is not required. Generally, the however, compensation CL required from source pixel S1 is:
Cr.=64-(h2+hi) Note that the table has been constructed such that the phase is determined by the distance ~L. However, since ~L = 1- ~R, it is also possible to determine the phase of the filter for the right portion by examining the distance ~R.
Once the right portion filter coefficients are determined, step 130 is complete and an overall target pixel filter is reconstructed at step 140. The filter reconstruction combines the selected left portion coefficients and right portion coefficients by addition. Therefore, the resulting constructed filter is {8, 31, 95, -6}. The sum of the reconstructed filter coefficients is 128.
At step 150, the time step control determines whether or not more target pixels need to be created. In the present example, there is yet another target pixel T2 to be created.
Therefore, the process returns to step 120 and the left portion filter coefficients for T2 are determined.
The distance D2 between T2 and T1 is used for determining which filter to select for the left portion filter coefficients. Therefore, the same filter, Filter 2, is selected for the left portion filter coefficients for target pixel T2 as was selected for the right portion filter coefficients for target pixel Tl. However, the phase is different, since the distance between target pixel T2 and source pixel S3 is different from that between target pixel T1 and source pixel S 1. For the target pixel T2, the phase is calculated to be 0.1. Therefore, from Table 3, the left portion coefficients are {13, 51, 0, 0}.
At step 130, the right portion filter coefficients for T2 are determined. A
distance D3 between target pixel T2 and target pixel T3 is used for determining which of the two filters to use for the right portion filter coefficients. Using the predefined thresholds, it is determined that the first filter, Filter 1 is to be used. From Table 2, the right portion coefficients are {0, 17, 48 -1 }.
At step 140, filter coefficients are reconstructed by addition of the left portion filter coefficients and the right portion filter coefficients. Therefore, the reconstructed filter coefficients are { 13, 68, 48, -1 }.
In the present embodiment, filters having an even number (four) of taps were illustrated.
However, it is possible to implement the invention using filters have an odd number of taps. In an alternate embodiment, five-tap filters are used instead of the four-tap filters described in the previous embodiment. The general process is similar to the previous embodiment, the main difference being the separation of the overall filter coefficients into left portion and right portion coefficients.
Table 4 is an example of a five-tap filter, and its corresponding adjusted coefficients.
h Originals-tap20 -10 98 -20 40 filter Adjusted filter~2 hl h~ ~ hl h2 ortions Left portion 20 -10 54 0 0 0 Right portion0 0 0 44 -20 40 The center of the five taps, ho is separated into a left center tap hL,o and a right center tap , hRo. The left portion of the filter is determined by preserving the filter coefficients for the filter taps left of the centermost tap. The sum of the left portion of the filter coefficients is given by LP = h_2 + 1L1 +'/Zho. The left center tap is calculated as hL,o ='/z +'hho - LP. A similar procedure is followed for the right center tap, where RP
= h2 + hl +'/2ho and hRo ='h + ~2hQ - RP.
When the filter tap coefficients are reconstructed, hl,o of the selected left portion coefficients is added to hRa of the selected right portion coefficients for reconstructing the center coefficient ho.
Furthermore, it is common in resampling systems to use filters of different lengths for different bandwidths. For example, a half band filter with four taps could be used for reducing the number of samples by a factor of two. Similarly, a quarter band filter with eight taps could be used for reducing the number of samples by a factor of four.
It will be apparent to a person skilled in the art how to apply the invention to filters of differing lengths for the left and right portions. For example, a six-tap filter is used for the left portion and a four-tap filter is used for the right portion. The resulting filter is a five-tap filter (h_3, h_2, 1y, hl, h2).
Although the invention has been described with reference to certain specific embodiments, various modifications thereof will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the claims appended hereto.
Claims (8)
PROPERTY OR PRIVELEGE IS CLAIMED ARE DEFINED AS FOLLOWS:
1. A system for generating a target pixel in a target grid from a plurality of source pixels in a source grid, said system comprising:
(a) a filter controller for determining a first distance between said target pixel and a source pixel to the left of said target pixel, and a second distance between said target pixel and a source pixel to the right of said target pixel;
(b) a left portion lookup circuit for determining filter coefficients for a left portion of said target pixel in accordance with said first difference;
(c) a right portion lookup circuit for determining filter coefficients for a right portion of said target pixel in accordance with said second difference; and (d) a filter reconstruction unit for creating a filter in accordance with said left and right portion filter coefficients, said created filter for generating said target pixel from said plurality of source pixels.
(a) a filter controller for determining a first distance between said target pixel and a source pixel to the left of said target pixel, and a second distance between said target pixel and a source pixel to the right of said target pixel;
(b) a left portion lookup circuit for determining filter coefficients for a left portion of said target pixel in accordance with said first difference;
(c) a right portion lookup circuit for determining filter coefficients for a right portion of said target pixel in accordance with said second difference; and (d) a filter reconstruction unit for creating a filter in accordance with said left and right portion filter coefficients, said created filter for generating said target pixel from said plurality of source pixels.
2. A system as defined in claim 1, wherein said left portion lookup circuit and said right portion lookup circuit share a common lookup circuit, wherein input to said common circuit is offset by a predetermined delay.
3. A method for generating a target pixel in a target grid from a plurality of source pixels in a source grid, said method comprising the steps of:
(a) determining a first distance between said target pixel and a source pixel to the left of said target pixel, and a second distance between said target pixel and a source pixel to the right of said target pixel;
(b) determining a first set of filter coefficients for a left portion of said target pixel in accordance with said first distance;
(c) determining a second set filter coefficients for a right portion of said target pixel in accordance with said second distance;
(d) combining said first and second sets of coefficients for creating a final filter for said target pixel; and (e) using said final filter for generating said target pixel.
(a) determining a first distance between said target pixel and a source pixel to the left of said target pixel, and a second distance between said target pixel and a source pixel to the right of said target pixel;
(b) determining a first set of filter coefficients for a left portion of said target pixel in accordance with said first distance;
(c) determining a second set filter coefficients for a right portion of said target pixel in accordance with said second distance;
(d) combining said first and second sets of coefficients for creating a final filter for said target pixel; and (e) using said final filter for generating said target pixel.
4. A method as defined in claim 3, wherein said first and second distance are horizontal differences when said source pixels are processed horizontally and vertical differences when said pixels are processed vertically.
5. A method as defined in claim 3, wherein each of said first and second sets of coefficients are selected from a plurality of sets of coefficients in accordance with said first and second distances.
6. A method as defined in claim 5, wherein each of said first and second sets of coefficients are selected by determining which of a plurality of ranges said first and second distances are included, and selecting a corresponding filter from a plurality of filters, said filter.
7. A method as defined in claim 6, wherein said plurality of filters are decomposed into left portion coefficients and right portion coefficients, said left portion coefficients for associating with said corresponding first set of coefficients and said right portion coefficients for associating with said corresponding second set of coefficients.
8. A method as defined in claim 7, wherein said left and right portion coefficients each have a sum of half a nominal value of said filter.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US09/984,001 US7154512B2 (en) | 2001-10-26 | 2001-10-26 | System and method for image warping |
| CA002360295A CA2360295A1 (en) | 2001-10-26 | 2001-10-26 | System and method for image warping |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US09/984,001 US7154512B2 (en) | 2001-10-26 | 2001-10-26 | System and method for image warping |
| CA002360295A CA2360295A1 (en) | 2001-10-26 | 2001-10-26 | System and method for image warping |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2360295A1 true CA2360295A1 (en) | 2003-04-26 |
Family
ID=27623537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002360295A Abandoned CA2360295A1 (en) | 2001-10-26 | 2001-10-26 | System and method for image warping |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US7154512B2 (en) |
| CA (1) | CA2360295A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4102973B2 (en) * | 2002-04-24 | 2008-06-18 | 日本電気株式会社 | Encoding method and decoding method of moving image, apparatus and program using the same |
| KR100510685B1 (en) * | 2003-04-22 | 2005-08-31 | 엘지전자 주식회사 | Apparatus for anti-aliasing in backward mapping algorithm |
| US8325282B2 (en) * | 2008-04-08 | 2012-12-04 | Mitsubishi Electric Visual Solutions America, Inc. | Television automatic geometry adjustment system |
| US8698747B1 (en) | 2009-10-12 | 2014-04-15 | Mattel, Inc. | Hand-activated controller |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4602285A (en) * | 1983-04-08 | 1986-07-22 | Ampex Corporation | System and method for transforming and filtering a video image |
| GB2197766B (en) * | 1986-11-17 | 1990-07-25 | Sony Corp | Two-dimensional finite impulse response filter arrangements |
| US5204944A (en) * | 1989-07-28 | 1993-04-20 | The Trustees Of Columbia University In The City Of New York | Separable image warping methods and systems using spatial lookup tables |
| US5175808A (en) * | 1989-09-12 | 1992-12-29 | Pixar | Method and apparatus for non-affine image warping |
| US5440653A (en) * | 1993-09-24 | 1995-08-08 | Genesis Microchip Inc. | Image mirroring and image extension for digital filtering |
| US5809322A (en) * | 1993-12-12 | 1998-09-15 | Associative Computing Ltd. | Apparatus and method for signal processing |
| US5594676A (en) * | 1994-12-22 | 1997-01-14 | Genesis Microchip Inc. | Digital image warping system |
| US6061477A (en) * | 1996-04-18 | 2000-05-09 | Sarnoff Corporation | Quality image warper |
| US5977947A (en) * | 1996-08-19 | 1999-11-02 | International Business Machines Corp. | Method and apparatus for resizing block ordered video image frames with reduced on-chip cache |
| US6188803B1 (en) * | 1996-10-18 | 2001-02-13 | Sony Corporation | Image processing device |
| US6573889B1 (en) * | 1999-02-08 | 2003-06-03 | Adobe Systems Incorporated | Analytic warping |
| US6384828B1 (en) * | 1999-10-22 | 2002-05-07 | Northshore Laboratories, Inc. | Digital processing apparatus and method for use in enlarging the size of an image displayed on a 2D fixed-pixel screen |
-
2001
- 2001-10-26 CA CA002360295A patent/CA2360295A1/en not_active Abandoned
- 2001-10-26 US US09/984,001 patent/US7154512B2/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| US20030095136A1 (en) | 2003-05-22 |
| US7154512B2 (en) | 2006-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5921794B1 (en) | Method and apparatus for interpolating video using asymmetric interpolation filter | |
| KR101658808B1 (en) | Method and apparatus for interpolating image using smoothed interpolation filter | |
| US6219464B1 (en) | Source data interpolation method and apparatus | |
| US8907973B2 (en) | Content adaptive image restoration, scaling and enhancement for high definition display | |
| JP4216830B2 (en) | Video processing apparatus and method, and computer-readable storage medium | |
| EP2850835A1 (en) | Estimation, encoding and decoding of motion information in multidimensional signals through motion zones, and auxiliary information through auxiliary zones | |
| JP2003324750A (en) | Electronic still camera and image processing method | |
| EP1133757B1 (en) | Image processing method | |
| JP2004348117A (en) | Video interpolation system and video interpolation method | |
| KR20070110252A (en) | Single-pass image resampling system and method with anisotropic filtering | |
| JP2003069998A (en) | Image processor and image processing method | |
| CA2360295A1 (en) | System and method for image warping | |
| Wang et al. | A fast computation of 2-D cubic-spline interpolation | |
| EP1433133B1 (en) | A method and apparatus for executing an affine or non-affine transformation | |
| JPH04369989A (en) | Coding method and coder | |
| JPH05225323A (en) | Picture interpolation method | |
| EP3364369B1 (en) | Decoding a bayer-mask or like coded image | |
| KR100909024B1 (en) | Low Complexity Image Interpolation Apparatus and Method Considering Precision Gradient Characteristics | |
| JP2004064194A (en) | Device for special effect image | |
| WO2020031362A1 (en) | Image processing device and image processing method | |
| JPH06113120A (en) | Resolution conversion device for picture processor | |
| JPH08223407A (en) | Image processing unit | |
| Kim et al. | An adaptive image interpolation using the quadratic spline interpolator | |
| JP2005198219A (en) | Estimation of frequency and amplitude of halftone screen for digital screening cancel of document | |
| JPH04345393A (en) | Picture data encoder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued | ||
| FZDE | Discontinued |
Effective date: 20051026 |