Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
  • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
  • H04N7/012—Conversion between an interlaced and a progressive signal
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/14—Picture signal circuitry for video frequency region
  • H04N5/142—Edging; Contouring
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/14—Picture signal circuitry for video frequency region
  • H04N5/144—Movement detection
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/14—Picture signal circuitry for video frequency region
  • H04N5/20—Circuitry for controlling amplitude response
  • H04N5/205—Circuitry for controlling amplitude response for correcting amplitude versus frequency characteristic

Definitions

• This invention relates to image processing.
• Video image capture represents a spatial and temporal sampling process. An image is captured as a set of pixels arranged in rows or lines. Successive images are captured at spaced instants in time.
• the smallest detail that can be reproduced in the image is a single pixel.
• each of the smallest details of the original scene should be represented by one pixel in the captured image.
• some of the details in the scene fall between pixel lines, so two pixel lines are required for such small details.
• This factor (70%) is known as the KeIl- factor.
• Other authorities assert that the Kell- factor relates only to the degradation caused by interlacing.
• This invention provides image processing apparatus in which output pixels of an output image are generated with respect to input pixels of an input image, a first subset of the output pixels being identical to respective input pixels and a second subset of the output pixels being derived from respective groups of one or more input pixels, the apparatus comprising: a spatial filter arrangement for filtering the second subset of pixels, the filter not being applied to filter the first subset of output pixels, the filter arrangement having a response such that high spatial frequencies are attenuated in the output image.
• the invention recognises that a simple approach to Kell-factor filtering of an image generated as defined above (i.e. just to filter the entire output image with a 70% bandwidth vertical low-pass filter) would not be expected to produce a useful result. This is because some of the pixels in the output image correspond to pixels in an input image which has already been captured or generated by a technique which limits the vertical bandwidth by the Kell-factor. In contrast, in the present invention, such pre-existing pixels are not altered by the filtering operation.
• the input image is an interlaced image; and the output image is a progressive scan image.
• the invention is applicable to any case where there is a mixture of source pixels and modified pixels. Further respective aspects and features of the invention are defined in the appended claims.
• Figure 1 schematically illustrates a flat-screen display arrangement
• Figure 2 schematically illustrates video mixing operation in a studio environment
• Figure 3 schematically illustrates an interlace to progressive converter
• Figures 4a to 4c schematically illustrate gradient detection
• Figures 5 and 6a to 6e schematically illustrate a spatial block matching operation
• Figures 7a and 7b schematically illustrate an alias situation
• Figures 8a to 8d schematically illustrate alias detection techniques
• Figure 9 schematically illustrates a Kell-factor filtering operation
• Figures 10 and 11 schematically illustrate stages in a Kell-factor filtering operation
• Figure 12a schematically illustrates a motion adaptive interpolator
• Figure 12b schematically illustrates motion detection between successive video fields
• Figure 13 schematically illustrates a high frequency check operation
• Figures 14a and 14b schematically illustrates a variation which reduces the need for image scaling
• Figures 15, 16a, 16b, 17a and 17b schematically illustrate a primary and an alternative pixel testing strategy
• Figure 18 schematically illustrates the generation of a difference value
• Figure 19 schematically illustrates an alternative embodiment of the interlace to progressive converter
• Figures 20a, 20b, 22a and 22b schematically illustrate variations of the alias detection techniques of Figures 8a to 8d;
• Figures 21a and 21b schematically illustrate an example of alias detection techniques applied to a thin line
• Figure 23 schematically illustrates a Kell-factor filtering operation applied in the horizontal direction.
• Figure 1 schematically illustrates a flat screen display arrangement 10 comprising a source of interlaced video material 20, an interlace to progressive scan converter 30 and a display panel 40 such as a liquid crystal (LCD) or plasma display.
• a broadcast signal received by the source of interlaced material 20 is used to generate an interlaced signal for display.
• This is passed to the interlace to progressive scan converter 30 to generate a progressive scan signal from the interlaced signal. It is the progressive scan signal which is passed to the display 40.
• the source of interlaced material 20 need not be a broadcast receiver, but could be a video replay apparatus such as a DVD player, a network connection such as an internet connection and so on.
• Figure 2 schematically illustrates a video mixing operation in a studio environment, in order to give another example of the use of interlace to progressive scan conversion.
• a source of interlace material 50 and source of progressive scan material 60 are provided. These sources could be cameras, video replay apparatus such as video tape recorders or hard disk recorders, broadcast receivers or the like.
• the interlaced output from the source of interlaced material 50 is supplied to an interlace Jo progress scan converter 70 to generate a progressive scan, signal.
• the progressive scan output of the vision mixer 80 can be converted back to an interlaced format if required, e.g. for subsequent broadcast or recording.
• the vision mixer 80 is just one example of video processing apparatus; instead, a digital video effects unit, for example, could be used at this position in Figure 2.
• Figure 3 schematically shows an interlace to progressive scan converter.
• the converter comprises a spatial interpolator 100, a motion adaptive interpolator 110 and a set of three field stores 120.
• the converter of Figure 3 operates to generate output progressive scan frames at the same repetition frequency as the input interlaced fields. Therefore, a main requirement of the converter is to generate the "missing" pixels in each interlaced field to turn that interlaced field into a progressive scan frame.
• the spatial interpolator 100 generates the "missing" pixels by spatial interpolation within the field concerned. In other words, this is an intra-field operation.
• the motion adaptive interpolator generates the missing pixels by inserting pixels from an adjacent field of the opposite polarity.
• the output of the spatial interpolator 100 is used at image positions where image motion is detected, while the output of the motion adaptive interpolator 110 is used at pixel positions where image motion is not detected.
• the spatial interpolator operates at each pixel position, and the motion adaptive interpolator either selects the output of the spatial interpolator, or selects a pixel from another field of the opposite polarity for output, or mixes the two.
• the motion adaptive interpolator will be described in more detail below. First, the spatial interpolator will be described.
• the spatial interpolator comprises a 1-2 horizontal pixel sealer 130, a spatial block matcher 140, a minimum error selector 150, a diagonal interpolator 160, a dot noise reducer 170 and a Kell-factor corrector 180. The operation of each of these will be described in more detail below.
• the sealer 130 uses horizontal linear interpolation to generate one additional pixel value between each two pixels of the input interlaced field. So, the horizontal resolution (at least in terms of number of available pixel values) is doubled, but no difference is made at this stage to the vertical resolution.
• the overall operation of the spatial block matcher 140 and the diagonal interpolator 160 is to detect the orientation of an image feature relevant to a pixel position where a new pixel is to be interpolated, and then to apply an interpolation along that image feature direction. So, if a current pixel position to be interpolated lies within a diagonal image feature (a line, an edge etc.) at, say, 45° to the horizontal, interpolation of that new pixel would take place along that 45° direction. This can tend to give a better output result than restricting the interpolation to horizontal or vertical interpolation. A key part of this process, clearly, is therefore to detect the direction of an image feature at each pixel position.
• Figure 4A schematically illustrates a successful block match between two blocks 200, 210 of pixels around the position of an unknown pixel 220 (a circle with a question mark inside).
• the notation used in the present drawings is that a square indicates a known pixel where as a circle indicates a pixel to be interpolated by the diagonal interpolator 160.
• the shading in Figures 4A to 4C is a schematic representation of an image feature. So, referring to Figure 4A, a successful block match is obtained between the blocks
• an image feature is vertical and there is again a successful block match between overlapping blocks 230, 240.
• the image feature has a gradient of 1/1. It is not possible to obtain a successful block match with the blocks at integral pixel positions. A successful match between blocks 250, 260 occurs at a half integral pixel position. Accordingly, in order to detect gradients of this nature (indeed any gradients sharper than 1/2), it is necessary to operate at a sub-pixel accuracy. In the present case, a half-pixel accuracy was adopted. If a greater accuracy still was used, (e.g. quarter-pixel accuracy) then gradients yet closer to vertical could be detected. In this embodiment, half-pixel accuracy was selected as a balance between processing requirements and accuracy at very acute gradients.
• FIGS 5 and 6A to 6E schematically illustrate the spatial block matching operation.
• spatial block matching is carried out at sub-pixel accuracy; in this case half-pixel accuracy.
• a range of block sizes is used, with corresponding search ranges (maximum displacements relative to the pixel position under test).
• search ranges maximum displacements relative to the pixel position under test.
• Figure 5 schematically illustrates a block match operation between two blocks of 3v x 13h pixels 300, 310, around an unknown pixel position 320.
• the block width is 13h in terms of scaled (interpolated) pixels; only the real (source) pixels are shown here.
• the variable d in fact a displacement of 4 interpolated pixels signifies a horizontal displacement of the block's horizontal centre from the pixel position under test.
• a condition applied to the block matches is that the blocks must always overlap the pixel position under test.
• the blocks are shown displaced in integral numbers of real pixel displacements (so a displacement of m corresponds to a displacement of 2m interpolated pixels).
• Figure 5 allows five possible tests at a displacement of -4 pixels (Figure 6A) -3 pixels, -2 pixels ( Figure 6B), -1 pixel, 0 pixels ( Figure 6C), +1 pixel, +2 pixels ( Figure 6D), +3 pixels and +4 pixels (Figure 6E).
• the displacement is indicated as a displacement from the centre, in scaled pixels.
• the two blocks are displaced by equal amounts, though in opposite directions. Symmetrical displacements are used because otherwise the block matching could detect lines or edges which are not relevant to the pixel under test.
• SAD sum of absolute differences
• the SAD values for three colour components are combined, and a minimum normalised SAD value determines a gradient for interpolation.
• Various checks are made to avoid poor interpolation, as described below.
• Figures 7A and 7B illustrate a possible alias situation.
• a block match between blocks 340 and 350 suggests that an unknown pixel 330 should be a dark grey colour.
• the block match is 100% successful and so the SAD value would be zero (note that this is a schematic example).
• a digitised line segment is considered to have two properties. Firstly, it is monotonic along the central scan line row of the block in question, and secondly there is a vertical transition between scan lines in the block in question. The way in which these tests are applied will be described with reference to Figures 8A to 8D.
• a source field contains multiple diagonal lines.
• Figure 8B schematically illustrates one row of pixels within the image of Figure 8A.
• Figures 8C and 8D illustrate the two edges of the diagonal line shown in Figure 8B. It will be seen that each of these edges has a region of pixels which show a monotonic variation in luminance. Also, referring back to Figure 8A, it can be seen that such segments exhibit a vertical transition between adjacent rows of pixels.
• the tests are performed separately in respect of each of the colour components (e.g. R, G and B). All three tests must be passed separately. Alternatively, for example to save hardware, fewer than three tests could be performed. For example, only the luminance, or only one colour component, might be tested. Of course, a YCbCr or YPbPr representation could be tested instead.
• a second rule, hinted at above, which is applied to block matching is that the minimum normalised SAD value determines the gradient for subsequent interpolation.
• the direction is selected from the block match with the smallest error (smallest normalised SAD).
• the Kell-factor correction of the corrector 180 will not later be used if the normalised SAD is less than a threshold KPCthr e s h -
• the normalised SAD value is similar (within a threshold amount) for multiple block offsets or if the smallest normalised SAD value obtained across all of the block matches is greater than a predetermined maximum value, then vertical interpolation will be used, Kell-factor correction will always be used in these cases and the normalised SAD value is set to the predetermined maximum value mentioned above.
• a third rule applicable to block matching is as follows.
• the diagonal interpolator 160 is a simple pixel averager: given a direction it picks the pixel in that direction on the line below and the pixel in that direction on the line above and averages them.
• the dot noise reducer 170 involves a process which is applied to the output of the diagonal interpolator 160. A test is applied to detect whether an interpolated pixel lies within the maximum and minimum values of the four vertically and horizontally pixels, i.e. the pixels immediately above, below, left and right of the interpolated pixel. Note that the pixels above and below the interpolated pixel will be real pixels, whereas those to the left and right will be interpolated themselves.
• v be the original value of the pixel under consideration, and let v' be v, clipped to lie within the range of the four locally neighbouring pixels.
• references to the Kell-factor are simply to help explain the operation of this part of the system. What the filter is actually exploiting is simply the knowledge that the source image did not use the full bandwidth available to it, .whether that is because of scanning artefacts or because of a low pass filtering process.
• the Kell-factor is a quantity which represents a property of progressive scan and interlaced images. In order to represent the information being scanned, it is generally considered that only 70% (the Kell-factor) of the possible vertical bandwidth is (or should be) represented. Hence when performing an interlace to progressive scan conversion, it is potentially hazardous to attempt to produce a full vertical bandwidth image. Instead, a compensation to account for a Kell-factor of less than unity may be used.
• One method to compensate for the Kell-factor would be to use a 70% bandwidth filter on the frame output of any interlace to progressive scan algorithm. However, one of the fields in the frame is 'real' data - i.e. it was sampled correctly, so the content arising from that field must by definition be perfect.
• a generic Kell-factor filtering operation is schematically illustrated in Figure 9, whereby source data is filtered (in the present example, in the vertical direction) to a bandwidth of about 70% of fs/2, i.e. the Nyquist frequency.
• Figures 10 and 11 schematically demonstrate the concept. The general idea is shown in Figure 10, whereby a vertical filter is (selectively) applied to real pixels I and interpolated pixels S in a progressive scan frame.
• a hypothetical 100% vertical bandwidth progressive image 500 is considered. This does not of course exist; at best it would be 70% filtered, due to the inherent filtering in the sampling process. This forms a realisable progressive frame 510. However, even this frame is not available to us; instead, it is provided in interlaced form as a field 520.
• the field 520 is the source data to the present interlace to progressive scan conversion process.
• the interlaced field 520 is supplied to an interlace-to-progressive converter, which then generates the missing field lines to produce a reconstructed frame 530.
• the frame 530 may now contain frequencies that are above 70% of the available bandwidth. Generating such frequencies is not always desired, since the information was not present in the source data to the interlace to progressive scan converter.
• a version 540 of this frame that does not exceed this bandwidth limit should be created, and this should be achieved by adjusting just the reconstructed field lines and not the source field lines - those source field lines are from an image that has already been passed through such a filter, and are therefore 'perfect'.
• Producing the required output can therefore be thought of as the low-pass filter output of a theoretical image 550 constructed from the lines from the original 100% vertical bandwidth image interlaced with the reconstructed field.
• the original 100% vertical bandwidth image is not available, and so a technique to circumvent this problem is now described.
• F are pixels from a real field of data, and / are interpolated pixels. Note that these values are known.
• the initial symmetrical 3-tap filter has relatively poor filtering characteristics, and thus a method to improve upon this by using an initial symmetrical 5-tap filter was developed.
• An example 70% 5 tap filter is:
• the example 3-tap filter defined above is:
• the symmetrical 7 tap filter consist of four coefficients: a, b, c, d, and the full filter be described by [ a, b, c, d, c, b, a] ⁇ .
• the 5 tap filter consist of three coefficients: e,f, g, and the full filter be described by [e,f, g,f, e] ⁇ .
• the 3 tap filter consist of two coefficients j, k, and the full filter by described by [/, k,j] ⁇ .
• An example 70% 7-tap filter is:
• the example 5-tap filter defined above is:
• the example 3-tap filter defined above is:
• This equation is used on each spatially interpolated pixel.
• the response is a bandstop filter, with the stop band between 0.66 and 0.8 f s /2; however, interpreting the response is complicated as it is only used for alternate output pixels.
• FIG 12a schematically illustrates the operation of the motion adaptive interpolator 0 110.
• the interpolator 110 comprises and inter-field block matcher 600, a high frequency checker 610 and a mixer 620.
• the inter-field block matcher 600 uses data from the current input field and the three field stores 120 to carry out inter-field motion comparisons. This involves comparing blocks of pixels in the current field (F N in Figure 12b) with correspondingly positioned blocks in the 5 previous field of the same type (F N-2 ) and likewise comparing the preceding field (F N- O an d the previous field of the same type (F N - 3 ). The results of these comparisons are used to detect a degree of motion in the image. - - . - . In particular, weighted, sums . O_f_ absolute, differences _ (SWADs) are generated as follows. 0 Four block matches are performed to produce two weighted SADs. These are:
• Weighted block matches sum weighted absolute differences between coincident pixels, SWAD.
• SWADAR EA Summing the latter two SWADs gives a localised block match
• the high frequency checker 610 is arranged to detect high frequencies in the input fields. The algorithm is based on the following principle.
• the high frequency checker uses the lines above and below the currently interpolated pixel from the current field F N and the line from the preceding field F N -I that corresponds to the missing line.
• the HFC may be considered as a 5x3 pixel neighbourhood check.
• HFCt h reshi and HFC t h r es h2 be two programmable constants, with the former greater than the latter.
• SWADARE A will tend to act against the use of the motion adaptive pixel at that output position.
• the mixer 620 operates according to the criteria SWAD A R E A and SWADLO CA L and also various thresholds threshi ....
• SWAD LOCA L > thresh use only spatially interpolated field, F N - Else if SWAD AR E A > thresh 2 , use only spatially interpolated field, FN-, only Else if SWAD AR E A ⁇ thresri3, use only field F N -I
• Figures 14a and 14b schematically illustrates a variation which reduces the need for image scaling.
• Figure 14a schematically illustrates the operation of embodiment described above, where each row of pixels (indicated in this diagram as ⁇ o ⁇ o
• ... contains real pixels and 1:2 interpolated pixels from the sealer 130.
• An example block match between two blocks 700, 710 and another between two blocks 720, 730 is illustrated.
• the present embodiment retains the 1:2 interpolation operation for smaller block offsets, in order to allow such steep angles to be better detected. For example, a range of block offsets of +1 and 0 could use 1:2 interpolated pixels, whereas block offsets of greater than this need not use the 1 :2 interpolated pixels.
• each block size is used in a match operation for only its most positive and most negative offsets. This can dramatically reduce the number of block matching operations needed, while not actually losing any information, since the previous scheme repeated (for example) the block matching tests at zero displacement (vertical image feature) for each block size.
• these measures can allow the 1 :2 interpolation to be carried out "as required", for example within the block matcher, rather than as a whole field of interpolated pixels by the sealer 130. This reduces the need for a field store to hold the field of interpolated pixels. Instead, just a few interpolated pixels over a few lines need to be buffered by the block matcher.
• Figures 15, 16a, 16b, 17a and 17b schematically illustrate a primary and an alternative pixel testing strategy.
• a further alternative to the arrangement described above is another alteration to alter the rules applicable to block matching in order to reduce the hardware requirements of this operation.
• the revised test is symmetrical about the interpolated row.
• Figure 15 schematically illustrates a block match.
• Figures 16a and 16b schematically illustrate the original test for horizontal and vertical energy which would have been performed in respect of that block match.
• Figures 17a and 17b schematically illustrate the revised tests which show that the same two rows are used for both tests.
• Figure 18 schematically illustrates the generation of a difference value.
• SAD values are calculated as a rolling average, with a new value being inserted and an old one discarded. This again can cut down the processing and hardware required to implement this feature.
• the interlace to progressive converter shown in Figure 19 chiefly comprises a spatial interpolator 1000, the motion adaptive interpolator 110, and the set of three field stores 120.
• the motion adaptive interpolator 110 and the three field stores 120 are substantially the same as those described with reference to Figure 3.
• the spatial interpolator 1000 is an alternative embodiment to the spatial interpolator 100 that is described with reference to Figure 3.
• the spatial interpolator 1000 will now be described.
• the spatial interpolator 1000 comprises a delay module 1110, a control module 1120, an alias avoidance module 1130, a spatial block matcher 1140, a best block matcher 1150, a local neighbourhood checker 1160, a Kell-factor corrector 1170 and a de-multiplexer 1180.
• the delay module 1110 is operable to receive video input data and preferably comprises three line delays and pixel delays for providing data to the alias avoidance module 1130, the spatial block matcher 1140, the Kell-factor corrector 1170 and the de-multiplexer 1180. Alternatively, the pixel delays may be implemented separately in other modules.
• the delay module 1110 is also able to output data to the motion adaptive interpolator 110. In this case, extra pixel delays may be included in the data output to thejnotion adaptive interpolator so that the data is aligned with the interpolated outputs of the spatial interpolator.
• the delay module 1110 may comprise any number of line delays and/or pixel delays.
• the control module 1120 generates control signals for the alias avoidance module 1130, spatial block matcher 1140, best block matcher 1150, local neighbourhood checker 1160, Kell-factor corrector 1170 and the de-multiplexer 1180.
• the control module 1120 is able to output control signals to the local neighbourhood checker 1160 and the de-multiplexer 1180 so as to perform edge processing.
• Edge processing can be used to address the issue that, at the edges of the input field, there are not enough lines or pixels to enable the full spatial interpolation method to be used. Instead, the pixels at the edges of the input field may be interpolated according to, for example, a line repeat function at one vertical edge and a simple vertical average between vertically adjacent pixels at the other edge. It will be appreciated that any other method for interpolating pixels at the edges of the input field may be used.
• the 'line repeat 1 function and vertical interpolation are used at the edges of the image. This is because it is not possible to perform block matching unless there are two source rows above and below the source line. At the top of an image, where there is only one line above the current line, vertical interpolation is carried out using the current line and the one line above it. If there is no line above the current line then the current line is repeated.
• the same method is used at the bottom of the image, where there are fewer than two lines below the current line.
• the algorithm also includes an optional feature, whereby the vertical interpolation
• This optional feature does not change the case where there is no line above (or below) the current line.
• the control module 1120 is also able to generate processing break flags that indicate a boundary, which the spatial interlace-to-progressive processing must not cross. For example, the processing breaks could occur at the start and end of each input line of pixels and the control module 1120 could use this data to indicate to the alias avoidance module 1130 where the processing breaks occur so that anti alias processing (as described above with reference to Figures 7a, 7b and 8a to 8d and/or as will be described in more detail below) may be carried out without generating processing errors or artificial image artefacts
• the alias avoidance module 1130 is operable to carry out anti alias measures as described above with reference to Figures 7a, 7b and 8a to 8d so as to avoid problems caused by alias situations.
• the alias avoidance module 1130 generates and outputs allow flags which indicate if a particular diagonal can be used. The allow flags are passed to the spatial block matcher 1140.
• the alias avoidance module 1130 is also operable to carry out further processing to reduce the effects of alias situations as will described in more detail below.
• the spatial block matcher 1140 is operable to carry out the block matching as described above and is similar in operation to the spatial block matcher 140 described with reference to Figure 3.
• the spatial block matcher comprises block match units for carrying out the block matching.
• the input data for the units is provided by the delay module 1110 and the tests are enabled by the allow flags received from the alias avoidance module.
• the block match units generate normalised sum-of-absolute difference (NSAD) values as described above as well as generating a spatially interpolated pixel (SIP) for each matched block. These values are passed to the best block matcher 1150.
• NSAD normalised sum-of-absolute difference
• SIP spatially interpolated pixel
• the spatial block matcher 1140 is also operable to generate vertically interpolated pixel values for use when no valid diagonals are identified or for when none of the diagonals that were identified are suitable for the basis on which to interpolate the pixel under test.
• the vertically interpolated pixel values are passed to the local neighbourhood checker 1160.
• the best block matcher 1150 is operable to process the NSAD and SIP values output by the spatial block matcher 1140 to determine which of the blocks identified by the spatial block matcher 1140 should be used as the basis for interpolation. For example, the best block could be identified as the one that has the lowest NSAD value. However, a person skilled in the art will realise that other methods of selecting the best block are possible.
• the best block matcher 1150 passes the identified best block NSAD value and the SIP to the local neighbourhood checker 1160.
• the best block matcher 1150 also passes the interpolation direction associated with that identified best block to the local neighbourhood checker 1160.
• the local neighbourhood checker 1160 performs a consistency check on the output of . the best block matcher 1150 in order to see if the block selected by the best block matcher 1150 is consistent with neighbouring blocks. If the output of the best block matcher 1150 is inconsistent then this output is modified by the local neighbourhood checker.
• the Local Neighbourhood Check is given the best direction SIP and the vertically interpolated pixel (VIP). Depending on the test result it outputs either the SIP, the VIP or a mixture of the two. It will of course be appreciated that this functionality could be provided by other units within the embodiment.
• One way of describing the gradient of the interpolation direction with respect to the pixel under test is to describe the gradient in terms of a horizontal offset with respect to a pixel a line of adjacent pixels immediately above the pixel under test.
• an adjacent line of pixels immediately below the pixel under test may be used or that, in both cases, a horizontal line of pixels could be vertically displaced by some other value.
• the pixel under test could be defined as sitting at the origin of an (x, y) plot.
• the gradient could then be defined by the gradient of a line passing through the origin and through the point (x, 1).
• the axes may be reversed and/or that a positive offset could be defined to the left of the pixel under test or vice versa and that any suitable coordinate system could be used to describe the gradient of the interpolation direction.
• Any inconsistent directions are removed and replaced.
• their NSAD values are modified and the vertically interpolated pixel is output.
• the NSAD values are used in the BCFC module (see below).
• the modified NSAD value is set so that the KFC will be used for this pixel.
• the local neighbourhood checker passes the modified NSAD values and interpolated pixel (IP) values to the Kell-factor corrector 1170.
• the Kell-factor corrector 1170 receives the NSAD values and the IP values output by the local neighbourhood checker 1160 and is operable to perform Kell-factor correction as described above with reference to Figures 9, 10 and 11.
• the Kell-factor corrector 1170 is also operable to perform additional filtering as will be described in more detail below.
• the interpolated data that is generated by the Kell-factor corrector is passed to the de-multiplexer 1180.
• the de-multiplexer 1180 is operable to route the interpolated data from the Kell- factor corrector to the correct input of the motion adaptive interpolator 110. So as to ensure correct data processing, the de-multiplexer is also operable to generate any data headers or control signals that may have been lost during interlace to progressive processing by the spatial interpolator 1000.
• Figure 20a shows the application of the monotonicity test that was described above with reference to Figures 7a, 7b and 8a to 8d.
• the monotonicity test is applied to two blocks 1200 and 1210, indicated by the heavier solid lines in Figure 20a, that span the block matched regions under test. If both blocks pass the monotonicity test, then the direction indicated by the block match is accepted and the pixel under test 1230 is interpolated accordingly.
• a single block 1240 of a similar size to each of the two blocks (1200, 1210) that are used in the two-block monotonicity test is used to perform the monotonicity test.
• the single block 1240 is centred on the pixel position 1250 under test as shown in Figure 20b but other configurations of the single test block 1240 with relation to the pixel 1250 under test are possible. This method is advantageous in that it increases the likelihood that low-gradient thin lines are detected as only the area close to the interpolation position is tested and areas that lie further away may be disregarded.
• Figure 21a shows the two-block method being applied to an image region comprising a thin line.
• the direction under test is indicated by the solid black line 1250 and the pixel position under test 1230 by the circle with the question mark.
• the monotonicity test previously described would reject this direction as the two separate blocks are not monotonic.
• the test block region 1240 is monotonic and the direction under test
• FIG. 22a and 22b An example of this is shown in Figures 22a and 22b.
• a valid diagonal is detected in respect of the blocks shown in Figure 22b.
• a further test is applied in that some or all smaller sets of blocks (such as the set shown in Figure 22a) must also pass a monotonicity test for that diagonal to be accepted.
• a valid diagonal 1320 should appear in all of the smaller blocks under test that are centred on that pixel under test 1330. If any of the smaller blocks fail the monotonicity test, then the direction 1310 indicated by the original large test block of Figure 22b is not accepted as the diagonal on which to base interpolation of the pixel 1330 under test.
• the test could be applied to blocks of decreasing size from the size of the original test block.
• the monotonicity test could be applied to the smallest test block possible (3h x 3v) centred on the pixel 1330 under test and then increased in size up to the size of the original test block. As described above, the monotonicity may be given by
• x is the source pixel value
• x91,0 is the top left pixel of the monotonicity block
• y is the row coordinate
• blockWidth is the block width in pixels
• C is a constant which allows for noise in the data.
• a threshold may be applied to the individual pixel differences, as described below.
• a noise threshold may be applied to each pixel by subtracting a threshold value from each pixel-by-pixel difference and then clipping negative pixel-by-pixel difference results to zero as described below.
• the noise tolerance of the monotonicity may be based on the feature under test rather than on a global constant. For example, a scaled quantity of diffextreme may be subtracted as shown below.
• adjustment defines the pixel-by-pixel threshold and diffFactor is a scaling factor for diffext r e m e-
• the pixel-by-pixel threshold may depend on the average pixel value as shown below.
• adjustment CONSTANT * (pix(x,y) + pix(x-l,y) ) / 1024
• the table below shows example values for where the pixel-by-pixel threshold is fixed for every block and pixel. The best combination (subjectively evaluated) is shown in the column headed "Best Value”.
• the fixed pixel-by-pixel threshold can improve the appearance of low angle parallel lines as well as improving the appearance of image regions where there are many possible diagonals.
• the Kell-factor corrector 1170 is operable to selectively apply Kell-factor correction (KFC) filtering to the interpolated pixels depending on the NSAD value. This reduces the degradation of diagonals where the chosen direction appears very clear.
• KFC Kell-factor correction
• Kell-factor filtering can always be applied in the vertical direction to the interpolated pixel. However, as mentioned above, this may degrade low gradient lines. To address this, it is preferable to only apply KFC vertical filtering if the NSAD value is high.
• the Kell-factor corrector 1170 is operable to always apply KFC vertical filtering when the direction for a particular pixel is within a particular threshold.
• the KFC vertical filter may always be applied if the direction is -1, 0 or 1.
• a flag that states whether the direction is -1, 0 or 1 may be stored instead of storing the actual interpolation direction.
• the Kell-factor corrector 1170 is also operable to apply horizontal filtering (horizontal KFC filtering) so as to smooth interpolated pixels in the horizontal direction as shown in Figure 23.
• horizontal filtering horizontal KFC filtering
• the circles represent interpolated pixels and squares denote real pixels.
• the pixel 1500 to be interpolated and to which filtering is to be applied is denoted with the question mark "?". Filtering is applied in a horizontal direction.
• a 5 tap horizontal filter is- used,- over a filter range shown- schematically by a dark" border 1510, symmetrically disposed about the interpolated pixel 1500.
• a suitable filter for use as the horizontal KFC filter may be a 5- tap symmetrical filter as was described above for the KFC vertical filter.
• Suitable filter values are the same as those used for the vertical filter as described above. However, a person skilled in the art will realise that any filter suitable for performing horizontal filtering may be used.
• the KFC horizontal filter may be applied to every interpolated pixel, it is preferable to selectively apply the KFC filter so as to not degrade high-angle (near vertical lines). This may be accomplished by not applying the filter when the current pixel interpolation direction lies in a direction in which applying KFC horizontal filtering would be inappropriate.
• the arrangement may be that the KFC horizontal filter is not applied when the current, left or right direction is zero or ⁇ 1.
• a flag that states whether the direction is -1, 0 or 1 may be stored instead of storing the actual interpolation direction.
• the KFC horizontal filter may not be applied if the NSAD value of that pixel is larger than a certain threshold.
• the Kell-factor corrector 1170 is operable to not apply the KFC horizontal filter to any of the interpolated pixels. It will be appreciated that any combination of or all of the Kell-factor filtering methods described above may be applied to the interpolated pixels. Additionally, it will be appreciated that the above described Kell-factor filtering methods may be applied in any arbitrary direction with respect to the interpolated pixels. Although the embodiments have been described in respect of interlace to progressive scan conversion, the same techniques could be applicable to altering the size (e.g. height) of an image such as a progressive scan image. For example, the techniques could perform a 1 :3 or 1 :4 scaling.
• the invention can be implemented in programmable or semi-programmable hardware operating under the control of appropriate software.
• This could be a general purpose computer or arrangements such as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array).
• the software could be supplied on a storage medium such as a disk or solid state memory, or via a transmission medium such as a network or internet connection, or via combinations of these. Appendix
• 5hx4v block 12/1024 23/1024 28/1024 23/1024 12/1024 32/1024 62/1024 77/1024 62/1024 32/1024 32/1024 62/1024 77/1024 62/1024 32/1024 12/1024 23/1024 28/1024 23/1024 12/1024
• 5hx3v block 20/1024 39/1024 48/1024 39/1024 20/1024 48/1024 94/1024 117/1024 94/1024 48/1024 20/1024 39/1024 48/1024 39/1024 20/1024 Ihx2v block: 128/256 128/256 lhxlv block: 255/256

Landscapes

• Engineering & Computer Science (AREA)
• Computer Graphics (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Television Systems (AREA)
• Image Processing (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Publications (1)

Family

ID=35516029

Family Applications (1)

Country Status (5)

Families Citing this family (6)

Citations (6)

Family Cites Families (12)

• 2005
  • 2005-10-31 GB GB0522159A patent/GB2431794A/en not_active Withdrawn
• 2006
  • 2006-10-27 JP JP2008537199A patent/JP4961431B2/ja not_active Expired - Fee Related
  • 2006-10-27 WO PCT/GB2006/004037 patent/WO2007051997A1/en not_active Ceased
  • 2006-10-27 US US11/908,082 patent/US8743280B2/en not_active Expired - Fee Related
  • 2006-10-27 CN CN 200680037436 patent/CN101283587A/zh active Pending

Patent Citations (6)

Also Published As

Similar Documents

Legal Events