GB2277001A - Motion compensated video standards conversion; simultaneous spatial and temporal interpolation. - Google Patents
Motion compensated video standards conversion; simultaneous spatial and temporal interpolation. Download PDFInfo
- Publication number
- GB2277001A GB2277001A GB9307411A GB9307411A GB2277001A GB 2277001 A GB2277001 A GB 2277001A GB 9307411 A GB9307411 A GB 9307411A GB 9307411 A GB9307411 A GB 9307411A GB 2277001 A GB2277001 A GB 2277001A
- Authority
- GB
- United Kingdom
- Prior art keywords
- pixels
- input
- output
- arrays
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
- H04N7/014—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes involving the use of motion vectors
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Television Systems (AREA)
Abstract
A motion compensated video standards converter is described in which simultaneous temporal interpolation and line rate conversion is performed. Motion vectors 30, associated with input pixels 26, 28 are determined. The motion vector to be applied to a particular output pixel 32, that is non co-incident with the input pixels is taken from the spatially closest pixel 38 in the array of input pixels. The input fields between which interpolation may be made can be in an interlaced format with the motion vectors having been determined from interpolated versions of the interlaced input fields in which the input pixels have been made coincident between the fields. By using a single interpolator for both spatial and temporal interpolation, circuit cost and noise associated with an extra processing stage can be reduced. <IMAGE>
Description
MOTION COMPENSATED VIDEO STANDARDS CONVERSION
This invention relates to motion compensated video standards conversion. More particularly, this invention relates to motion compensated video standards conversion in which the pixel positions in the output image are non-coincident with the pixel positions in the input image.
A motion compensated video standards converter is described in
British Published Patent Application GB-A-2 231 749 (Sony Corporation).
The system of GB-A-2 231 749 performs motion compensated interpolation to derive a sequence of output images from a sequence of input images having a different format.
Known video standards converters incorporating motion compensation perform the conversion process in various stages.
Usually, the last stage to be performed, after temporal interpolation, is the conversion of the video from the input line rate to the required output line rate. This necessitates adding to the system a dedicated further processor to perform vertical filtering on the video data to effect this line rate conversion.
Such a line rate converting further processor has the disadvantage of increasing the complexity and cost of the system since the hardware involved in implementing the line rate conversion is considerable. In addition, the line rate conversion is a further distinct signal processing stage in the signal path and produces a degradation of the image quality due to noise injection and the like.
Viewed from one aspect this invention provides apparatus for performing motion compensated video standards conversion, said apparatus comprising:
means for detecting motion vectors associated with input image pixels between temporally adjacent input arrays of pixels each having an input array of pixel positions;
means for determining a temporal position between said input image arrays of pixels at which an output image array of pixels is to be interpolated, said output image array of pixels having an output array of pixel positions non-coincident with said input arrays of pixel positions;
means for selecting a motion vector for each output pixel in said output array of pixels from motion vectors associated with spatially adjacent input pixel positions to said output pixel position; and
means for interpolating an output pixel value at said output pixel position in dependence upon said selected motion vector.
The invention provides a motion compensated video standards conversion in which the temporal interpolation to a different frame/field rate together with the interpolation to non-coincident output pixel positions is carried out in a single stage. The ability to perform both these conversions in a single stage stems from the realisation that the motion vectors identified at the input pixel positions can still be used at the nearest output pixel positions without introducing any particular problems. By exploiting this realisation, the temporal interpolation can be made directly to the output pixel positions thereby obviating the need for a separate line rate conversion stage. This has a significant advantage both in terms of hardware cost and complexity and in removing an additional processing entity from the signal path.
The invention can be applied to any system in which the output pixel positions are non-coincident with the input pixel positions and a simultaneous temporal and spatial interpolation is desired. However, the invention is particularly suited to systems in which said input arrays of pixels are raster scanned at an input line rate and said output arrays of pixels are raster scanned at an output line rate different from said input line rate.
Examples of such conversions might be between the NTSC and PAL formats or between the high resolution television formats at 1125 lines, 60Hz and 1250 lines, 50Hz.
In such systems in which the line rate is different, the pixel positions parallel to the raster direction can be made the same by placing the sampling points for each line at the same place, but the pixel positions perpendicular to the raster direction are more problematic and require handling such as that provided by this invention. In such systems, said output array of pixel positions are non-coincident with said input array of pixel positions in a direction perpendicular to said raster scanning direction.
The means for selecting can operate to select the motion vector to be used for a given output pixel from amongst those associated with spatially adjacent input pixels. In a simple embodiment, the motion vector used might be taken from the input pixel in a fixed relation to the output pixel, i.e. the input pixel above and to the left of the output pixel. However, in preferred embodiments of the invention said means for selecting selects a motion vector associated with the spatially closest input pixel position to said output pixel position.
Selecting the motion vector from the spatially closest input pixel position provides more accurate interpolation.
In the case that the input pixels are in a raster format, this requirement is equivalent to said means for selecting selects from motion vectors associated with the spatially closest raster line of the input arrays of pixels.
Selecting the motion vectors from the closest raster line of the input arrays of pixels is particularly suited to embodiments in which the motion vectors are processed and manipulated as raster lines of motion vectors. In this case, a raster line of motion vectors to be utilised for interpolation can be either repeated or omitted such that the motion vectors closest to the output pixel positions are presented for use.
It will be appreciated that the output arrays of pixels could take a number of different forms. However, the need to perform line rate conversion in an efficient manner is particularly applicable to systems in which said output arrays of pixels comprise interlaced output fields of pixels.
The input arrays of pixels may directly represent the input video standard, such as a progressive scan standard. However, in the case where the input video standard comprises interlaced input fields, then block matching to identify motion vectors between the interlaced fields becomes complicated by the non-coincident pixel positions between the interlaced fields. In order to deal with such situations, said input arrays of pixels comprise interpolated input arrays of pixels interpolated from interlaced input fields of pixels, said interpolated input arrays of pixels having coincident interpolated input pixel positions between each interpolated input array of pixels.
In this way, the interlaced input fields are first interpolated to an interpolated input array of pixels having coincident pixel positions derived from each input field. Block matching between the interpolated input arrays of pixels is thus not complicated by the vertical displacement between the interlaced input fields.
In order to preserve image quality whilst using the above feature, said means for interpolating uses said selected motion vector associated with an interpolated input pixel position to interpolate said output pixel from pixels in adjacent interlaced input fields of pixels.
Thus, whilst the interpolated input arrays of pixels are used for block matching to identify the motion vectors, the means for interpolating reverts to the unprocessed interlaced input fields of pixels for the interpolation in order to reduce the amount of injected noise associated with the interpolated input arrays of pixels.
The means for interpolating could apply many different sorts of averaging to the identified pixels in the input arrays of pixels on either side of the output array of pixels1 for example, the average of the spatially closest previous and next input pixels pointed to by the selected motion vector could be used. However, in preferred embodiments of the invention, said means for interpolating interpolates between pixel values obtained by two-dimensionally filtering a target area of pixels within said interlaced input fields of pixels.
This two dimensional filtering of a target area allows for the situation that the motion vectors may not point to exactly one pixel within the input arrays of pixels, but at a position between pixels.
Viewed from another aspect the invention also provides a method of motion compensated video standards conversion, said method comprising the steps of:
detecting motion vectors associated with input image pixels between temporally adjacent input arrays of pixels each having an input array of pixel positions;
determining a temporal position between said input image arrays of pixels at which an output image array of pixels is to be interpolated, said output image array of pixels having an output array of pixel positions non-coincident with said input arrays of pixel positions;
selecting a motion vector for each output pixel in said output array of pixels from motion vectors associated with spatially adjacent input pixel positions to said output pixel position; and
interpolating an output pixel value at said output pixel position in dependence upon said selected motion vector.
Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
Figure 1 illustrates a known technique for temporal interpolation;
Figure 2 illustrates a known technique for line rate conversion;
Figures 3 and 4 illustrate alternative configurations of video standards converters having different input line rates and output line rates;
Figures 5 and 6 illustrate one embodiment of a video standards converter in which temporal interpolation and line rate conversion are simultaneously performed; and
Figure 7 illustrates an alternative embodiment of a video standards converter in which temporal interpolation and line rate conversion are simultaneously performed.
Figure 1 illustrates a technique of temporal interpolation. Each of the input field points 2 represents a pixel on one line of a progressive scan converted input array of pixels ("previous frame").
The input field points 4 represent pixels in another progressive scan converted input array of pixels ("next frame") between which an array of output pixels 6 is to be temporally interpolated. It will be seen from the line 8 that the pixel positions in the input and output arrays of pixels are coincident.
Considering the output pixel point 10 whose value is to be interpolated. a motion vector associated with the input pixel position 12 is selected as representing the motion of the output pixel 10. This motion vector is then projected both forward and backward from the output pixel 10 to identify source points 14, 16 in the input arrays of pixels which correspond to the output pixel 10. A two-dimensional filtering operation centred about the source points 14, 16 is then performed to provide a pixel value that is also weighted upon the relative temporal proximity of the output pixel 10 to each of the source points 14, 16.
Figure 2 illustrates line rate conversion. The pixels 6 at the input line rate must be converted to the pixels 18 at the output line rate. Considering a particular output point 20, the value at this position is derived from a convolution of a plurality of points 22 within a target area of the pixels at the input line rate. The convolution applied is schematically represented by the one-dimensional filter function 24. The filter function contains a plurality of sets of coefficients to be applied to the input pixels depending upon the relative position of the output pixel 20 to the pixels within the target area. These filter coefficients are chosen such that the convolution curve illustrated is substantially achieved for the input pixels 22.
Figure 3 illustrates one embodiment of a system for performing line rate conversion and temporal interpolation. In this system, the input data is first line rate converted in accordance with the technique illustrated in Figure 2 to produce and output at the output line rate. This output is then supplied to a vector steered temporal interpolator where the temporal interpolation to achieve a different frame/field rate is performed. It is significant to note that the input to and output from the vector steered temporal interpolator have the same line rate.
Figure 4 illustrates an alternative to the system of Figure 3.
In this case, the vector steered temporal interpolation to achieve the new field/frame rate is carried out at the input line rate and then the input line rate is converted to the output line rate.
Figure 5 illustrates the technique whereby temporal interpolation and line rate conversion are simultaneously achieved. Input arrays of pixels 26, 28 are disposed at input pixel positions that are coincident between successive arrays. One or more motion vectors 30 are identified by a block matching process to be associated with each input pixel position.
Considering the output point 32 under consideration, this forms part of an output array of pixels 34 having an array of pixel positions that are non-coincident with the array of input pixel positions, i.e.
the output is at a different line rate to the input. A consequence of this is that there is no longer a one-to-one correspondence between identified motion vectors 30 and output pixel positions. Accordingly, a selection of the spatially closest motion vector 36 associated with the input pixel position 38 is made for use with the output pixel 32.
This selected motion vector 36 is then projected backwards and forwards to the arrays of input pixels 26, 28 and a two-dimensional filtering operation upon a target area of pixels in each of the forward and backward directions is made to determine a pixel value for the output pixel 32. The contributions in the forward and backwards directions are weighted according to the temporal proximity of the output pixel 32 to each of the input arrays of pixels 26, 28. Whilst it will be seen that the motion vector 36 does not directly correspond to the output pixel position 32, this has been found to make little significant difference to the output image quality.
The motion vectors 30 are calculated and stored at an earlier stage to that of interpolation in the usual manner for motion compensated standards converters such as that described in British
Published Patent Application GB-A-2 231 749 (Sony Corporation). The motion vectors are read out of their store a raster line at a time corresponding to the raster lines of the arrays of input pixels 26, 28.
Thus, if the relative ratios of the number of lines in the input arrays of pixels and the output arrays of pixels is known, then the line of motion vectors that should be applied to a particular line of output pixels can be calculated. For example, if a conversion is being made between 1250 lines and 1125 lines, then assuming that the first and last lines in each array are co-located, the line of motion vectors that should be used for line "X" in the output array is the rounded integer value of ((1250/1125) * X). In this case, since the output has fewer lines than the input, then line of motion vectors will be periodically dropped. In the opposite case when the output has more lines than the input, then lines of motion vectors will be periodically repeated. Each motion vector is used by two horizontally adjacent pixels within each raster line (motion vector sub-sampling), e.g.
pixels 0 and 1 uses the first vector, pixels 2 and 3 use the second vector etc..
Figure 6 schematically illustrates the difference of the system according to Figure 5 over that discussed in relation to Figures 3 and 4. In the embodiment of Figures 5 and 6, the temporal interpolation and the line rate conversion take place as a single process within a single piece of hardware.
Figure 7 illustrates an alternative embodiment to that of Figure 5. In this embodiment, the input video to the processing system is in an interlaced field format. In order to properly identify motion vectors between fields. each interlaced input field is interpolated to produce an interpolated array of input pixels that are coincident between the fields. This produces arrays of pixels such as those illustrated in Figure 5.
These interpolated input arrays of pixels are used to identify the motion vectors 40. The motion vector to be used with a particular output point 42 in the output array of pixels is derived from the spatially closest interpolated input pixel in a similar manner to that illustrated in Figure 5. In the embodiment of Figure 7, the major difference is that when the interpolation of the output pixel is made between the previous and next fields, it is actual interlaced arrays of pixels that are used as the targets for the two-dimensional filters rather than the interpolated input arrays of pixels that were used to identify the motion vectors. It is better to use the source interlaced arrays of pixels rather than the interpolated input pixels since the additional processing stage of interpolation that may introduce noise into the interpolated input pixel values that should be avoided. The non-coincidence of the input pixel positions between adjacent fields is illustrated by the line 44. In other respects, the operation of the embodiment of Figure 7 follows that of Figure 5.
Claims (12)
1. Apparatus for performing motion compensated video standards conversion, said apparatus comprising:
means for detecting motion vectors associated with input image pixels between temporally adjacent input arrays of pixels each having an input array of pixel positions;
means for determining a temporal position between said input image arrays of pixels at which an output image array of pixels is to be interpolated, said output image array of pixels having an output array of pixel positions non-coincident with said input arrays of pixel positions;
means for selecting a motion vector for each output pixel in said output array of pixels from motion vectors associated with spatially adjacent input pixel positions to said output pixel position; and
means for interpolating an output pixel value at said output pixel position in dependence upon said selected motion vector.
2. Apparatus as claimed in claim 1, wherein said input arrays of pixels are raster scanned at an input line rate and said output arrays of pixels are raster scanned at an output line rate different from said input line rate.
3. Apparatus as claimed in claim 2, wherein said output array of pixel positions are non-coincident with said input array of pixel positions in a direction perpendicular to said raster scanning direction.
4. Apparatus as claimed in any one of claims 1, 2 and 3, wherein said means for selecting selects a motion vector associated with the spatially closest input pixel position to said output pixel position.
5. Apparatus as claimed in claims 3 and 4, wherein said means for selecting selects from motion vectors associated with the spatially closest raster line of the input arrays of pixels.
6. Apparatus as claimed in any one of the preceding claims, wherein said output arrays of pixels comprise interlaced output fields of pixels.
7. Apparatus as claimed in any one of the preceding claims, wherein said input arrays of pixels comprise interpolated input arrays of pixels interpolated from interlaced input fields of pixels, said interpolated input arrays of pixels having coincident interpolated input pixel positions between each interpolated input array of pixels.
8. Apparatus as claimed in claim 7, wherein said means for interpolating uses said selected motion vector associated with an interpolated input pixel position to interpolate said output pixel from pixels in adjacent interlaced input fields of pixels.
9. Apparatus as claimed in claim 8, wherein said means for interpolating interpolates between pixel values obtained by twodimensionally filtering a target area of pixels within said interlaced input fields of pixels.
10. A method of motion compensated video standards conversion, said method comprising the steps of:
detecting motion vectors associated with input image pixels between temporally adjacent input arrays of pixels each having an input array of pixel positions;
determining a temporal position between said input image arrays of pixels at which an output image array of pixels is to be interpolated, said output image array of pixels having an output array of pixel positions non-coincident with said input arrays of pixel positions;
selecting a motion vector for each output pixel in said output array of pixels from motion vectors associated with spatially adjacent input pixel positions to said output pixel position; and
interpolating an output pixel value at said output pixel position in dependence upon said selected motion vector.
11. Apparatus for performing motion compensated video standards conversion substantially as hereinbefore described with reference to
Figures 5, 6 and 7 of the accompanying drawings.
12. A method of motion compensated video standards conversion substantially as hereinbefore described with reference to Figures 5, 6 and 7 of the accompanying drawings.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9307411A GB2277001B (en) | 1993-04-08 | 1993-04-08 | Motion compensated video standards conversion |
| GB9312129A GB2277006B (en) | 1993-04-08 | 1993-06-11 | Motion compensated video signal processing |
| JP6871494A JPH06326980A (en) | 1993-04-08 | 1994-04-06 | Movement compensating type processing system of picture signal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9307411A GB2277001B (en) | 1993-04-08 | 1993-04-08 | Motion compensated video standards conversion |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| GB9307411D0 GB9307411D0 (en) | 1993-06-02 |
| GB2277001A true GB2277001A (en) | 1994-10-12 |
| GB2277001B GB2277001B (en) | 1997-09-24 |
Family
ID=10733598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB9307411A Expired - Fee Related GB2277001B (en) | 1993-04-08 | 1993-04-08 | Motion compensated video standards conversion |
Country Status (1)
| Country | Link |
|---|---|
| GB (1) | GB2277001B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0514012A2 (en) * | 1991-04-15 | 1992-11-19 | Vistek Electronics Limited | Method and apparatus for the standard conversion of an image signal |
-
1993
- 1993-04-08 GB GB9307411A patent/GB2277001B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0514012A2 (en) * | 1991-04-15 | 1992-11-19 | Vistek Electronics Limited | Method and apparatus for the standard conversion of an image signal |
Also Published As
| Publication number | Publication date |
|---|---|
| GB9307411D0 (en) | 1993-06-02 |
| GB2277001B (en) | 1997-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5600377A (en) | Apparatus and method for motion compensating video signals to produce interpolated video signals | |
| US4985767A (en) | Spatio-temporal sub-sampling of digital video signals representing a succession of interlaced or sequential images, transmission of high-definition television images, and emission and reception stages for such a system | |
| EP0395274B1 (en) | Motion dependent video signal processing | |
| EP0395265B1 (en) | Motion dependent video signal processing | |
| EP0395273B1 (en) | Motion dependent video signal processing | |
| EP0266079B2 (en) | Interpolating lines of video signals | |
| US5526053A (en) | Motion compensated video signal processing | |
| JP2832927B2 (en) | Scanning line interpolation apparatus and motion vector detection apparatus for scanning line interpolation | |
| GB2231749A (en) | Motion dependent video signal processing | |
| EP0395263B1 (en) | Motion dependent video signal processing | |
| GB2231743A (en) | Motion dependent video signal processing | |
| EP0395268B1 (en) | Motion dependent video signal processing | |
| EP0395266B1 (en) | Motion dependent video signal processing | |
| EP0395272B1 (en) | Motion dependent video signal processing | |
| EP0535066B1 (en) | Video signal processing | |
| US5021881A (en) | Motion dependent video signal processing | |
| US5485224A (en) | Motion compensated video signal processing by interpolation of correlation surfaces and apparatus for doing the same | |
| EP0395269B1 (en) | Motion dependent video signal processing | |
| EP0303316B1 (en) | Television transmission system | |
| GB2202706A (en) | Video signal processing | |
| NL8902828A (en) | MOTION DETECTION SYSTEM FOR A HIGH DEFINITION TELEVISION RECEIVER. | |
| WO1993015586A1 (en) | A partial interpolation method and apparatus for frame rate conversion | |
| JP3469626B2 (en) | Motion compensated video signal processing apparatus and video signal processing method | |
| GB2277001A (en) | Motion compensated video standards conversion; simultaneous spatial and temporal interpolation. | |
| JPH06326980A (en) | Movement compensating type processing system of picture signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PCNP | Patent ceased through non-payment of renewal fee |
Effective date: 20110408 |