Classifications

• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
  • G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
  • G09G5/005—Adapting incoming signals to the display format of the display terminal
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2340/00—Aspects of display data processing
  • G09G2340/04—Changes in size, position or resolution of an image
  • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas

Definitions

• the present invention relates to scaling operations in a video processing system, to resize image content with minimal artifacts.
• Scaling generally refers to processes that alter the size of an image. Scaling may be performed as a format conversion operation in a video system. For example, when video images of low resolution (e.g., 48Op) are displayed on a high resolution panel (e.g., a full high definition (108Op) panel), the video images need to be scaled to 108Op. This kind of conversion has been referred to as an up-conversion. Also, video images of high resolution may need to be converted to be displayed on a low resolution panel (e.g., 108Op to 48Op) or a smaller display area (e.g., Picture in Picture (PIP)). This kind of conversion has been referred to as a down-conversion or decimation.
• PIP Picture in Picture
• scaling involves adjusting size of video images to be displayed. For example, an 4:3 image frame with resolution of 48Op has 720x480, or 345,600 pixels and an 16:9 image frame with resolution of 108Op has 1920x1080, or 2,073,600 pixels. Thus, additional pixels need to be added to perform an up-conversion from 480p to 108Op and existing pixels need to be eliminated and/or combined to perform a down-conversion from 108Op to 48Op.
• NYOl 19427% vl - 1 - 13641/396210 artifacts can cause distortions to images and affect the image quality after either an up-conversion or a down-conversion, but the existing techniques using complex hardware structures do not reduce these undesirable artifacts effectively.
• FIG. 1 illustrates a schematic diagram of an imaging system according to an embodiment of the present invention.
• FIG. 2 illustrates a parametric Farrow structure according to an embodiment of the present invention.
• FIG. 3 illustrates a set of equations for determining coefficients of a parametric
• Farrow structure according to an embodiment of the present invention.
• FIG. 4 illustrates a new pixel y(A) being added by a parametric Farrow structure according to an embodiment of the present invention.
• FIG. 5 illustrates a set of conditions to determine the values for the adjustable parameter alpha according to an embodiment of the present invention.
• FIG. 6 illustrates magnitude responses for different phases of a conventional polyphase filter.
• FIG. 7 illustrates magnitude responses for a parametric Farrow structure according to an embodiment of the present invention.
• FIG. 8 illustrates a process to use a parametric Farrow structure for down-conversion according to an embodiment of the present invention.
• FIG. 9 illustrates a down -conversion process using a parametric Farrow structure according to an embodiment of the present invention.
• FIG. 10 illustrates an overshoot control according to an embodiment of the present invention.
• Embodiments of the present invention are directed to an image processing system.
• the image processing system may comprise a content detection module having an input to receive a sequence of input pixeis and configured to generate an adjustable parameter based on detected differences between adjacent pairs of input pixeis, and a digital filter having an input for the sequence of input pixels and a control input coupled to an output of the content detection module.
• the digital filter may adjust filtering coefficients according to the parameter.
• FIG. 1 illustrates a schematic diagram of an imaging system 100 according to an embodiment of the present invention.
• the imaging system 100 may comprise a content detection module 104, a parametric Farrow structure 106 and an overshoot control module 108.
• the parametric Farrow structure 106 may be a digital filter that processes input pixels and generate output pixels by applying coefficients to respective input pixels (e.g., by interpolation).
• the parametric Farrow structure 106 may be coupled to pixel input 102 to receive the input pixels.
• the content detection module 104 may determine a parameter aipha 112 based on input pixels to adjust the coefficients for the parametric Farrow structure 106.
• the overshoot module 108 control may limit output pixels from the Farrow structure 106 to be within a certain range of input pixels.
• both the content detection module 104 and the overshoot control module 108 may be coupled to the parametric Farrow structure 106.
• the parametric Farrow structure 106 may generate output pixels based on the input pixels and the parameter alpha 112.
• the generated output pixels may be transmitted to the overshoot control module 108.
• the overshoot control module 108 may process the generated output pixels for overshoot control and output the generated pixels to the pixel output 110.
• Embodiments of the present invention may apply interpolation/decimation to a variety of color spaces for images (e.g., red-green-blue (RGB), YIQ, YUV).
• the content or frequency detection may be based on a YUV color space.
• RGB input may need to be converted to YUV before being scaled and to be converted back from YUV to RGB after being scaled.
• alpha decision may apply on Y channel
• UV may share the same alpha with Y.
• the content or frequency detection may be based on RGB color space.
• the 3 channels of RGB color space may have separate alpha decision module respectively.
• FIG. 2 shows a parametric Farrow structure 200 according to the an embodiment of the present invention.
• the parametric Farrow structure 200 may comprise an input signal line 202, a position signal line 220, an output signal line 230, a plurality of deiay elements 216.1-216.5 (collectively delay elements 216), four amplifiers 218.1-218.4 (collectively amplifiers 218), eight adders 224.1-224.8 (collectively adders 224) and two multipliers 222.1-222.2 (collectively multipliers 222).
• the input signal line 202 may receive a sequence of input pixels x(m) (e.g., x(0), x(l)) one dimension of an image (e.g., horizontally or vertically). Each received input pixel may be applied to the amplifiers 218 and transmitted to delay elements 216.
• the amplifiers 218 may each amplify an input signal by a factor of alpha( ⁇ ). For example, for an input of x(m), the output may be .zx(m).
• the delay elements 216 may each add one delay to a pixel. For example, if input signal at input signal line 202 is x(n), then the signal at point 204 and 210 (one delay element 216.1 and 216.4 away from x(n) respectively) is a previous pixel x(n-l), the signal at point 206 and 212 (two delay elements 216.1 and 216.2, and 216.4 and 216.5 away from x(n) respectively) is a previous pixel x(n-2), and the signal at point 208 (three delay elements 216.1, 216.2, and 216,3 away from x(n)) is a previous pixel x(n-3).
• each delay element 216 may be a storage device, such as, but not limited to, a register.
• the position signal line 220 may receive a position indicator ⁇ k identifying where a new pixel is to be generated.
• the position indicator ⁇ may be multiplied (e.g., at the multipliers 221) to signals in the parametric Farrow structure 200.
• An output signal y(A) (e.g., a new pixel) may be generated on the output signal line 230 for the position identified by the position indicator ⁇ k based on input pixels.
• the parametric Farrow structure 200 may be a four-point piecewise parabolic Farrow structure and referred to as a quadratic interpolator.
• FIGs. 3 and 4 illustrate operation of the parametric Farrow structure 200 according to an embodiment of the present invention.
• FIG. 4 shows a new pixel y(A) to be added at a position ⁇ k based on four input existing pixels (e.g., x(-l), x(0), x(l) and x(2)).
• FIG. 3 illustrates a set of equations for determining coefficients Ci, C 0 , Ci and C 2 that may be applied to the four existing pixels according to an embodiment of the present invention.
• the new pixel y(A) may be located at the position ⁇ k between two existing
• the parametric Farrow structure 200 may use two existing pixels before the position ⁇ k where the new pixel is to be added (e.g., x(-l) and x(0)) and two existing pixels after the position ⁇ / de.g,, x(l) and x(2)).
• ⁇ k may be a value between zero (inclusive) and one (exclusive).
• the new pixel y(/r) may be generated by the four input pixels x(-l), x(0), x(l) and x(2) as follows:
• y(A) C 2 x(2) + C 1 X(I) + Qx(O) + C 1 XC-I).
• the signal at points 204 and 210 may be x(l), the signal at points 206 and 212 may be x(0), and the signal at point 208 may be x(-l), respectively.
• x(2) may be amplified by the amplifier 218.1 and added to the adder 224.1.
• Another input at the adder 224.1 may be a negative input from the amplifier 218.2, which amplifies signal x(l) at point 204. Accordingly, the adder 224.1 may generate an output of cx(2) - ⁇ x(l) to be input to the adder 224.2.
• the adder 224.2 may also have a negative input (e.g., - ⁇ x(0)) from the amplifier 218.3, which amplifies signal x(0) at point 206.
• the adder 224.3 may generate an output of ox(2) - c ⁇ (l) - cx(0) to be input to the adder 224.5.
• the adder 224.5 may add the output from the adder 224.3 and an output from the amplifier 218.4 (e.g., a ⁇ (- I)) and have an output of c ⁇ 2) - a ⁇ .) - ox(0) + o ⁇ (-l).
• the output of the adder 224.5 may be multiplied to the position signal ⁇ k at the multiplier 222.1.
• the multiplier 222.1 may thus have an output of (ox(2) - cx(l) - cx(0) + ⁇ x(-l)) ⁇ *>
• input to the adder 224.2 are signals from the amplifier 218.1 (negative input), amplifier 218.2 and point 210.
• the adder 224.2 thus may generate an output of - e ⁇ (2) + ⁇ x(l)+ x(l).
• the adder 224.4 may add the output from the adder 224.2 to the signal from the amplifier 218.3 (e.g., + ox(0)) and subtract the signal from the point 212 (e.g., - x(0)). Accordingly, the adder 224.4 may generate an output signal of - ax(2) + (cH-l)x(l) + (tf-l)x(O).
• the output from the adder 224.2 may be added at the adder 224.6 with a negative signal from the amplifier 218,4 (e.g., - ox(-l)).
• the adder 224.6 may have an output of
• the sum of output of the multiplier 222.1 and the adder 224.6 may be multiplied by ⁇ k at the multiplier 222.2.
• the generated signal y(A) may be generated at the adder 224.8, which may sum up the output signal from the multiplier 222.2 with the
• the generated signal y(A) may be as follows and the coefficients Ci, C 0 , C 1 and C 2 for each existing input pixels x(-l), x(0), x(l) and x(2) may be determined as shown in FIG. 3;
• y(A) (flx(2) - cx(l) - a ⁇ (0) + a ⁇ (-X)) ⁇ / + (- cx(2) + ( ⁇ +l)x(l) + ( ⁇ -l)x(0) - Gx(-l)) ⁇ k +x(0).
• FIG. 5 illustrates a set of conditions to determine the values for the adjustable parameter alpha according to an embodiment of the present invention.
• Value of a new pixel to be inserted may depend on values of input pixels before and after the position of insertion.
• the value of the new pixel may be depend on differences between adjacent input pixels. For example, the difference between the input pixels immediate before and after the position of insertion may play a bigger role in contributing to the new pixel then differences between pixels further away from the position of insertion.
• the content detection module 104 may determine the value of the parameter alpha dynamically based on content (e.g., differences between adjacent pixels). As shown in FIG.
• values of a sequence of existing pixels may be used to determine the value of the parameter alpha for new pixel(s) to be inserted between the second and third pixel of the sequence (e.g., x(n-l) and x(n-2)).
• the value of alpha may be determined by checking the set of conditions A, B, C, D, E, F, G and H.
• the set of conditions A, B, C, D, E, G and H may then derive an appropriate alpha for a new pixel to be inserted between x(n-2) and x(n-l) (e.g., x(0) and x(l) of FIG. 4).
• the content detection module e.g., content detection module 104 of FIG. 1
• 5 levels e.g., five values for the adjustable parameter alpha
• 5 levels may be used: high, intermediate high, middle, low, off.
• the five levels may correspond to: 1, 0.75, 0.5, 0.25, 0.
• Conditions A, B, C, D and E may be directed to differences between the input pixels immediately before and immediately after a newly inserted pixel (e.g., x(n-2) and x(n-l)).
• Conditions G and H may be directed to differences between neighboring pairs of input pixels close to the position of insertion.
• the value of the parameter alpha may be determined solely on the difference between the pixels immediately before and immediately after the insertion position if the
• NYOl 1942796 vl - 6 - 13641/396210 difference is in a certain range (e.g., difference between values of x(n-2) and x(n-l) in a certain range).
• alpha may be high (e.g., 1) if the difference between x(n-2) and x(n-l) is greater than or equal to a top threshold (e.g., the absolute value of the difference may be greater than or equal to 96 shown in FIG. 5 as condition A).
• alpha may be intermediate high (e.g., 0.75) if the difference between x(n-2) and x(n-l) is between the top threshold and an intermediate threshold (e.g., the absolute value of the difference may be between 96 (exclusive) and 64 (inclusive) shown in FIG. 5 as condition B).
• alpha may be middle (e.g., 0.5) if the difference between x(n-2) and x(n-l) is between the intermediate threshold and an intermediate low threshold (e.g., the absolute value of the difference may be between 64 (exclusive) and 32 (inclusive) shown in FIG. 5 as condition C).
• alpha may be middle (e.g., 0.5) if the differences of neighboring pairs of input pixels are greater than an upper threshold (e.g., both absolute values of difference between x(n-3) and x(n-2), and difference between x(n-l) and x(n) are greater than 64, shown as condition G in FIG. 5).
• alpha may still be middle (e.g., 0.5) if the differences of neighboring pairs of input pixels are greater than a lower threshold (e.g., both absolute values of difference between x(n-3) and x(n-2), and difference between x(n-l) and x(n) are greater than 32, shown in FIG. 5 as condition H),
• alpha may be low (e.g., 0.25) if either or both differences of neighboring pairs of input pixels are less than or equal to the upper threshold (e.g., either or both absolute values of difference between x(n-3) and x(n-2), and difference between x(n-l) and x(n) are less than or equal to 64, shown as condition G equaling to zero in FIG. 5).
• alpha may be set to 0 if either of both of the differences of neighboring pairs of input pixels are less than or equal to the lower
• NYOl 1942796 vl - 7 - 13641/396210 threshold e.g., not both absolute values of difference between x(n-3) and x(n-2), and difference between x(n-l) and x(n) are greater than 32, shown in FIG. 5 as condition H-O).
• the vaiues for various thresholds may be adjusted and/or determined by a system designer.
• the top threshold value may be 100
• the intermediate threshold value may be 50
• the intermediate low threshold value may be 25
• the bottom threshold value may be 10
• the upper threshold may be 48
• the lower threshold may be 24.
• the value of alpha may be determined by sequence of input pixels.
• the coefficients of an Farrow structure according an embodiment of the present invention may be adaptive based on the sequence of input pixels (e.g., existing pixels before and after the inserted the pixel), and interpolation may be based on the image content (e.g., differences between the pixels).
• the criteria for the determination according an embodiment of the present invention may be adjusted as needed.
• a signal processing structure may act as a signal filter.
• the signal the filter typically has different effects for different frequencies.
• the horizontal axis may be frequency and the vertical axis may be magnitude response of the traditional polyphase structure for a respective phase (e.g., ⁇ k ). If the filter is an ideal low pass filter(LPF), the frequency response should be all equal to 1.
• the first phase's response is 1 (e.g., point 602)
• the second and fourth phases both have a response of about 0.75 (e.g., point 604 and 608 respectively); and as shown in Figure 6(c), the third phase's response is about 0,4 (e.g., point 606).
• a traditional polyphase structure is far from an ideal LPF and may cause a dramatic distortion to an image being processed. This dramatic distortion in fact is a direct cause of saw tooth edge, which is a well-known undesirable artifact in signal processing.
• Embodiments of the present invention may adjust the parameter alpha (thus the coefficients of the parametric Farrow structure) according to the frequency level of an image to make it have consistent response at different phases.
• the adjustment may be have the effect of changing parameters of a digital filter to achieve desired frequency responses for a variety of frequencies dynamically.
• a parametric Farrow structure according to an embodiment of the present invention with a parameter alpha having a value of 0.25 may have a magnitude response of less than 0.5 (e.g., point 702 in FIG. 7(a)).
• FIG. 7(b) e.g., point 704
• the parameter alpha may be adjusted to make the spectrum flat, that is, make the magnitude response to be higher (e.g., to approach one) for a frequency and thus make the magnitude response more like the ideal LPF spectrum.
• the content detection module may detect frequency levels of existing pixels and adjust and/or choose a value for the parameter alpha that may cause the parametric Farrow structure to generate a consistent magnitude response.
• the parameter alpha may be configured to one of: high, intermediate high, middle, low and off (e.g., 1, 0.75, 0.6, 0.25 and 0). For example, as
• Embodiments of the present invention may help reduce undesirable artifacts.
• One of the artifacts when applying up-conversion (e.g., scale up) on image may be staircase.
• Staircase is caused mainly by unequal frequency responses on different phases (e.g., FIG. 6).
• a parametric Farrow structure may achieve a better or equal gain at the same frequency point (e.g., FIG. 7).
• the staircase artifact may be greatly reduced.
• Another of the artifacts when applying up-conversion on images may be blurring.
• the whole frequency response may be bigger than any traditional filters.
• the embodiment may increase the sharpness of an edge.
• FIG. 8 illustrates a process 800 to use a parametric Farrow structure for down- conversion according to an embodiment of the present invention.
• the process 800 may comprise two steps.
• the parametric Farrow structure may be used as a low pass filter. Existing pixels may be processed by the low pass filter to merge several adjacent pixels into one generated intermediate pixel.
• the parametric Farrow structure may be used two perform bilinear interpolation from the generated intermediate pixels to derive a final pixel from original existing pixels.
• the parametric Farrow structure may perform a linear interpolation first in one direction , and then again in the other direction (e.g., horizontal then vertical, or vertical then horizontal).
• FIG. 9 illustrates the process 800 by an example of merging four existing pixels into one pixel.
• Four existing pixels x(-l), x(0), x(l) and x(2) may be input to a parametric Farrow structure according to an embodiment of the present invention (e.g., the parametric Farrow structure of FIG. 2).
• the parametric Farrow structure may be used as a 3 points low pass filter (LPF).
• the 3 points LPF may process the first three consecutive pixels x(-l), x(0) and x(l) to generate a first intermediate pixel x'(0).
• the parametric Farrow structure may then process the next three consecutive pixels x(0), x(l) and x(2) to generate a second intermediate pixel x'(l).
• the parametric Farrow structure may apply linear interpolation to the intermediate pixels x'(0) and x'(l) to generate a final pixel ⁇ (k) at the position ⁇ k between the existing pixel x(0) and x(l).
• This process may be applied in two dimensions thus may be referred to as a bilinear interpolation.
• the 3 points LPF may generate the intermediate pixel by summing one quarter of a preceding pixel, one half of a middle pixel and one quarter of a succeeding pixel.
• the first intermediate pixel x'(0) may be generated as 1 A x(- 1) + V% x(0) + TA x(l)
• the second intermediate pixel x'(l) may be generated as 1 A x(0) + Vz x(l) + 1 A x(2).
• the parametric Farrow structure may be a parametric Farrow structure shown in FIG. 2 adjusted by additional switches. The additional switches may select the signals in FIG.
• y(A) may be the sum of (1 - /4)x'(0) and ⁇ *x'(l) (e.g., a linear interpolation for two points).
• Down scaling or decimation is another important technology in image scaling. It has application such as PIP (picture in picture).
• PIP picture in picture
• moire pattern is usually induced due to frequency aliasing (e.g., known as aliasing artifact).
• an anti-aliasing filter e.g., LPF
• LPF anti-aliasing filter
• the pass-band of the LPF should be proportional to decimation scale.
• the parameter of Farrow structure filter may be selected to achieve a proper frequency response (e.g., suppressing the high frequency content to a large level) and greatly reduce the moire artifact without additional hardware.
• Embodiments of the present invention may apply overshoot/undershoot control to any scaling (e.g., interpolation or decimation) process.
• FIG. 10 illustrates an overshoot control according to an embodiment of the present invention. Overshoot (or undershoot) may occur during signal processing when an output signal of a filter fall outside a certain scope of the maximum (or minimum) of the input signal, for example, if the output signal y(k) of Figure 4 is much larger than the maximum of four input signals (e.g., x(0)), or much
• the output signal may be determined according to a formula shown in Figure 10. Max and m ⁇ n may represent maximum and minimum value of the four input pixels. Offset may be a global parameter applied to all images being scaled. Therefore, the output pixel's value may be limited to a certain range based on the four input pixels and thereby the overshoot control according to the present invention may prevent the new interpolation to be too large or too small in value and appearing overshoot.
• embodiments of the parabolic Farrow structure according to the present disclosure may become adaptive LPFs and thus, decimation may share the same computing circuit with interpolation as described above with respect to FIGs 8 and 9.
• Embodiments according to the present disclosure may generate new pixels adaptively and produce smoother and sharper images.
• the performance may be obtained by adjusting frequency response for different frequency spectrum.
• a parametric quadratic kernel is employed which has lower cost and higher high frequency response compared with traditional Lagrange cubic.
• the frequency response may be adjusted by modifying a parameter, which may be determined by content of the image.
• the parametric Farrow structure according to an embodiment of the present invention may be more cost efficient.
• the coefficients of the parametric Farrow structure 200 may be calculated dynamically, thus eliminating look up tables (LUT).
• the dynamically generated coefficients may be adjusted based on the parameter alpha.
• the parameter alpha may be adjusted according to content of an image being processed. That is, the coefficients may be generated dynamically based on existing surrounding pixels.
• the interpolation may be performed as vertical first, then horizontal.
• Embodiments of the present invention may have a parametric quadratic kernel.
• the parametric quadratic kernel may have following characteristics. First, it may have a low cost. For example, compared to a traditional Farrow structure (e.g., a traditional cubic Farrow structure), for a four-tap Farrow structure (e.g., using four existing pixels), only 2 multiplications and 8 adders in each direction and each channel may be needed. Second, the coefficients may be adaptive. For example, a parameter could be adjusted to make the coefficients have best frequency spectrum while the original Lagrange Farrow's kernel
• NYOl 1942796 vl - 12 - 13641/396210 function is fixed. Third, it may achieve a ringing reduction. Because the ringing region in image is considered to be iow frequency, adaptive frequency response may mitigate the effects and will not magnify the ringing.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Hardware Design (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Processing (AREA)
• Television Systems (AREA)

Priority Applications (2)

Applications Claiming Priority (4)

Publications (1)

Family

ID=43050399

Family Applications (1)

Country Status (4)

Families Citing this family (2)

Citations (4)

Family Cites Families (6)

• 2009
  • 2009-07-20 US US12/505,626 patent/US8346021B2/en active Active
• 2010
  • 2010-05-04 CN CN201080026512.5A patent/CN102460561B/zh not_active Expired - Fee Related
  • 2010-05-04 JP JP2012509900A patent/JP5603414B2/ja not_active Expired - Fee Related
  • 2010-05-04 WO PCT/US2010/033544 patent/WO2010129548A1/en not_active Ceased

Patent Citations (4)

Also Published As

Similar Documents

Legal Events