GB2215552A

GB2215552A - Video signal transformation

Info

Publication number: GB2215552A
Application number: GB8805417A
Authority: GB
Inventors: Stephen Mark Keating
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 1988-03-08
Filing date: 1988-03-08
Publication date: 1989-09-20
Anticipated expiration: 2008-03-08
Also published as: JPH01272269A; GB2215552B; GB8805417D0

Abstract

A method of and apparatus for mapping digital video sample values from a first sample array to a second sample array under control of a mapping function which may involve rotation in two dimensions, for example, for digital video effects unit, comprises an address generator 2 for mapping the sample positions from the first sample array using the mapping function, a processor 4 for determining, where four mapped sample positions enclose a sample position in the second sample array, the positions of the four mapped sample positions relative to the sample positions in the second sample array, and an interpolator 6 controlled in dependence on the determination to derive an interpolated sample value for the sample position in the second sample array. <IMAGE>

Description

VIDEO SIGNAL PROCESSING This invention relates to video signal processing, and more particularly to methods of and apparatus for digital video sample mapping.

This technique is used, for example, in digital video effects units, in which digital television signals are processed to give special effects.

Such special effects are now well known to television viewers, and enable images on the cathode ray tube of a television receiver to be, for example, expanded, compressed, moved in any direction, or rotated in two or three dimensions. Such special effects can be generated by mapping video sample values from one sample array to another under control of a dynamically changing mapping function.

Consider the example shown in Figure 1 of the accompanying drawings. In an input sample array A, six sample values respectively correspond to six successive sample positions (shown as dots) along a horizontal scan line of an input digital television signal. Suppose that the requirement is to compress at least that part of the image in the ratio 4:3.

Mere compression would simply move the sample values, without change in their magnitudes to more closely spaced points in an array B'. However, the required output sample array B in fact consists of sample positions identical with those of the input sample array A, so it is further necessary to map the sample values of the array B' onto the output sample array B, so that the sample values have the correct magnitudes and are in the correct positions.

A relatively simple solution to this problem is to use for each sample position in the output sample array B that sample value from the array B' which is spatially nearest. Obviously this results in inaccuracies, and other more sophisticated solutions have been proposed, although these tend to involve complex calculations and to require a large amount of storage, particularly where more than one dimension is involved. Also, such previously proposed solutions tend not to operate satisfactorily where the mapping function is changing rapidly.

We have previously disclosed in our UK patent specification :4B-A-2 177167 a solution which can be used when the mappino 'uno..oll is changing rapidly, and which involves so-called back interpolation. In this previous proposal, the position of a point in the input sample array which, using the mapping function, would map onto a sample position in the output sample array is computed, and then the required sample value for that sample position in the input sample array, and hence the required sample value for the output sample array, is calculated by interpolation using available sample values from the input sample array.

According to the present invention there is provided a method of mapping digital video sample values from a first sample array to a second sample array under control of a mapping function, the method comprising: mapping the sample positions from the first sample array using said mapping function; where four mapped sample positions enclose a sample position in said second sample array determining the positions of said four mapped sample positions relative to said sample position in said second sample array; and in dependence on said determination controlling an interpolator to derive an interpolated sample value for said sample position in said second sample array.

According to the present invention there is also provided apparatus for mapping digital video sample values from a first sample array to a second sample array under control of a mapping function, the apparatus comprising: means for mapping the sample positions from the first sample array using said mapping function; means for determining, where four mapped sample positions enclose a sample position in said second sample array, the positions of said four mapped sample positions relative to said sample position in said second sample array; and an interpolator controlled in dependence on said determination to derive an interpolated sample value for said sample position in said second sample array.

The invention will now be described by way of example with reference to the accompanying drawings, in which: Figure 1 shows diagrammatically parts of horizontal scan lines of a digital television signal for explaining the problem of the present invention; Figure 2 shows diagrammatically the mapped positions of sample positions from an input sample array relative to an output sample array, where the mapping involves compression; Figures 3 and 4 show diagrammatically the mapped positions of sample positions from an input sample array relative to an output sample array where the mapping involves rotation; Figure 5 shows in block diagrammatic form an embodiment of apparatus according to the present invention, and Figures 6 and 7 show respective alternative forms of part of the embodiment of Figure 5 in more detail.

The embodiment to be described is for use in a digital video effects unit in which a digital television signal is processed to give special effects in a subsequently displayed image. The method involved will first be described, and then an apparatus for performing the method will be described.

Many special effects which are required involve the image which is to be displayed on a cathode ray tube being expanded or compressed in size relative to an input image. In some cases movement, which may be linear or involve rotation, and which is additional to the movement inherent in any expansion or compression which is also taking place, may be involved. The present invention is particularly, but not exclusively, concerned with special effects which involve two-dimensional rotation and compression in size of at least part of an image under control of a dynamically changing mapping function, this rotation and compression in size being effected by processing of the input image information, followed by storage of the processed information.The invention can also be applied to special effects which involve other changes, such as expansion in size, although in the case of expansion the input information is stored unchanged, and the necessary processing of the input information is done on reading out the stored information, because to do the processing before storage would unnecessarily increase the storage capacity required.

The method will first be described with reference to Figures 2 and 3 As described above with reference to Figure 1, when using a mapping function to map sample values from an input sample array A onto an output sample array B, it is generally the case that the sample positions in the input sample array A will not map exactly onto the sample positions in the output sample array B. The problem then is to specify the site of an actual sample position in the output sample array relative to the mapped sites of sample positions from the input sample array.Thus in Figure 2, where the mapping function causes compression of the portion of the input image shown in-the ratio 5:4, the points P11. P12, P21 and P22 are the mapped sites of sample positions from the input sample array separated by one pixel horizontally (P11 to P12) and one horizontal scan line vertically (P11 to P21). The origin 0 is the sample position in the output sample array nearest to these four points, that is, the sample position which is enclosed by the rectangle defined by the points P1l, P12, P22 and P21.

Consider now the more general example of Figure 3 where the- mapping function involves a rotation. Provided that the four points P11, P12, P21 and P22 at least substantially define a parallelogram, the ratios la/ha and lb/hb can be calculated using only the positions of the points P12, 21 and P22 as follows.

If the x,y coordinates of the points P1l, etc. are taken to be x11, y11, etc, then the line P12P22 is given by the equation: y = x(y12-y22)/(x12-x22)+y22-x22(y12-y22)/(x12-x22) and the line P21P22 is given by the equation: y = x(y21-y22)/(x21-x22)+y22-x22(y21-y22)/(x21-x22) And the line through the origin 0, that is the required sample position in the output array, parallel to the line P12P22 is given by the equation: y=x(y12-y22)/(x12-x22) and the line through the origin 0, parallel to the line P21 P22 is given by the equation: y = x(y21-y22)/(x21-x22) To obtain the point Pa, combine equations 2 and 3: xa(y12-y22)/(x12-x22)=xa(y21-y22)/(x21-x22)+y22-x22(y21-y22)/(x21-x22) Cross multiplying: [(y12-y22)(x21-x22)-(y21-y22)(x12-x22)]xa= y22(x12-x22)(x21-x22)-x22(y21-y22)(x12-x22) Therefore: xa = (x12-x22)[y22(x21-x22)-x22(y21-y22)]/ [(y12-y22)(x21-x22)-(y21-y22)(x12-x22)] Putting this into equation 3: =[y22(x21-x22)-x22(y21-y22)]/ [(y12-y22)(x21-x22)-(y21-y22)(x12-x22)] ...(6) The length la is given by: la=(xa-x21)+(ya-y21) Similarly: ha = (x22-x21)+(y22-y21) ...(8) Therefore the site of the sample position # along the line P21P22 is given as the ratio:

where xa and ya are given in equations 5 and 6 respectively.

Similarly, it may be shown that: xb = (x21-x22)[y22(x12-x22)-x22(y12-y22)]/ [(y21-y22)(x12-x22)-(y12-y22)(x21-x22)] ...(10) yb = [y22(x12-x22)-x22(y12-y22)]/ [(y21-y22)(x12-x22)-(y12-y22)(x21-x22)] ...(11) and that:

The ratios la/ha and lb/hb given by equations (9) and (12) are the controls required by a two-dimensional interpolator of known form to derive the magnitude of the sample value in the input sample array which would map exactly onto the sample position 0 in the output sample array, so the interpolator can be controlled regardless of the mapping function involved.

As noted above, the ratios la/ha and lb/hb are calculated using only the positions, that is the addresses, of the three points P12, P21 and P22.

Figure 4 shows an alternative case where the points P1l, P12, P21 and P22 at least substantially define a rectangle, in which case the ratios la/ha and lb/hb can be calculated using only the positions that is the addresses, of the two points P21 and P22. This is a rather less general case, and will therefore usually (depending on the particular mapping function) give somewhat less accurate results for the sample value derived by the interpolator.

In the case of the rectangle shown in Figure 4, equations (1) and (2) are unchanged. However,-in place of equation (3), the equation of the line at right angles to the line P21P22 can be taken: y = x(x22-x21)/(y21-y22) and in place of equation (4), the equation of the line at right angles to the line P12P22 can be taken: y = x(x22-x12)/(y12-y22) By working similar to that above we can derive: a = (y21-y22)[x22(y21-y22)-y22(x21-x22)]/ F(y21-y22)2 + (x21-x22) =(x22-x21)2[x22(y21-y22)y-y22(x21-x22)]/ L(y21-y22) + (X21-x22) 2 and hence also expressions for the ratios la/ha and lb/hb, with the difference that in this case the ratios can be calculated using only the positions of the two points P21 and P22.

The steps involved are to map the sample positions of the input sample array, that is the current field, onto the output sample array using the dynamically changing mapping function, the addresses of the mapped sample positions being temporarily stored in two line stores. At any given time, therefore, the mapped sample positions corresponding to two horizontal scan lines of the input sample array are held in the line stores.

The mapped sample positions are checked in groups of four; two corresponding to two adjacent sample positions in the same scan line of the input sample array and two corresponding to the two adjacent sample positions which in the input sample array were in the next scan line of the field and respectively vertically below the first two, to see if the group of four mapped sample positions encloses a sample position in the output sample array. If it does not (as may commonly occur for example in the case of substantial compression) that group is discarded. This process steps sample position by sample position and scan line by scan line over the whole field. Each time the group of four mapped addresses does enclose a sample position in the output sample array the addresses of two or three of the four mapped sample positions are used to determine the above ratios la/ha and Ib/hb.These ratios la/ha and lb/hb then control an interpolator which in known manner determines the magnitude of the sample value for the output sample position from a moving matrix of 3 x 3 sample values from the input sample array.

The embodiment of apparatus according to the present invention will now be described with reference to Figure 5. The apparatus comprises an output memory 1, an address generator 2, a mapping function generator (or source) 3, a processor 4, a coefficient look-up table device 5, and a twodimensional interpolator 6 of known form.

The output memory 1 can store data relating to one field of a video signal. It is assumed that the data, that is the sample values corresponding to each sample position in the field, are in the form of respective 8-bit words. Each sample position in the field is designated by two address words used to identify the sample position in the horizontal and vertical, that is the X and Y, directions respectively. These address words may, for example, be an 11-bit word for the horizontal direction and a 10-bit word for the vertical direction.

The address generator 2 generates successive addresses each in the form of two address words designating X and Y, and moreover, under the control of the mapping function, which may or may not be dynamically changing, supplied by the mapping function generator 3, generates the addresses of the sample positions from the input sample array as mapped onto the output sample array. The mapping function, which if dynamically changing may change from sample position to sample position, specifies the compression and/or rotation and/or other effect which is to be applied to the image in that field, and the address generator 2 operates to calculate the addresses of the points, to which the mapping function will cause each sample position in the input sample array to be mapped.

Each mapped address X, Y is supplied to the processor 4, which comprises two line stores and at least one programmable read only memory (PROM), and which operates, as described above, for each group of four mapped addresses which encloses a sample position in the output sample array, to determine using two or three of the four mapped sample positions (P12 P21 and P22 in Figure 3 or P21 and P22 in Figure 4), the above ratios la/ha and lb/hb for supply to the coefficient look-up table device 5.

Figure 6 shows the processor 4 in more detail for the parallelogram case (Figure 3), where it comprises a PROM 11, a sample store 12 and a line store 13 connected as shown. The X,Y addresses are supplied to the two inputs, and the PROM 11 is programmed to derive the ratios la/ha and lb/hb as described above.

Figure 7 shows the processor 4 in more detail for the rectangle case (Figure 4), where it comprises two PROMs 21 and 22, a sample store 23 and a line store 24 connected as shown. The X,Y addresses are supplied to the four inputs, and the PROMs 21 and 22 are programmed to derive the ratios la/ha and lb/hb respectively, as described above.

Referring again to Figure 5, the coefficient look-up table device 5, which may be a PROM, supplies weighting coefficients in dependence in the ratios la/ha and lb/hb to the interpolator 6, to which the 8-bit input data words are also supplied. The coefficient look-up table device 5 and the interpolator 6 together operate, generally in the manner of such elements in a television standards converter, to calculate using a moving matrix of 3 x 3 sample values from the current field, that is the input sample array, a sample value for the output sample position, which calculated sample value is supplied to the memory 1 to be stored at the location corresponding to the appropriate address X, Y for subsequent read-out when the field stored therein is to be displayed, recorded or transmitted.

The required calculated sample value is derived by multiplying the sample value at each of the nine sample positions by the respective weighting coefficient, and summing the resulting nine products to derive the calculated sample value.

Claims

1. A method of mapping digital video sample vanities from a first sample array to a second sample array under control of a mapping function, the method comprising: mapping the sample positions from the first sample array using said mapping function; where four mapped sample positions enclose a sample position in said second sample array determining the positions of said four mapped sample positions relative to said sample position in said second sample array; and in dependence on said determination controlling an interpolator to derive an interpolated sample value for said sample position in said second sample array.

2. A method according to claim 1 wherein said mapping function involves rotation.

3. A method according to claim 2 wherein said rotation is in two dimensions.

4. A method according to claim 1, claim 2 or claim 3 wherein the positions of said four mapped sample positions relative to said sample position in said second sample array are expressed as two ratios which represent the distances of said four mapped sample positions from said sample position in said second sample array in two directions.

5. A method according to any one of the preceding claims wherein said mapping function is dynamically changing.

6. A method according to any one of the preceding claims wherein said interpolation uses a moving matrix of sample values from said first sample array, and there is derived for said sample position in said second sample array a plurality of weighting coefficients, each sample value in said moving matrix is multiplied by a respective said weighting coefficient, and the resulting products are summed to form said interpolated sample value.

7. A method according to any one of the preceding claims wherein said interpolated sample value is supplied to a memory for storage prior to display, recording or transmission.

8. A method according to claim 7 wherein said memory is a field store.

9. Apparatus for mapping digital video sample values from a first sample array to a second sample array under control of a mapping function, the apparatus comprising: means for mapping the sample positions from the first sample array using said mapping function; means for determining, where four mapped sample positions enclose a sample position in said second sample array, the positions of said four mapped sample positions relative to said sample position in said second sample array; and an interpolator controlled in dependence on said determination to derive an interpolated sample value for said sample position in said second sample array.

10. Apparatus according to claim 9 wherein said mapping function involves rotation in two dimensions.

11. Apparatus according to claim 10 wherein said means for determining comprise an address generator for generating the addresses of said mapped sample positions, and a processor for deriving from said addresses the positions of said four mapped sample positions relative to said sample position in said second sample array expressed as two ratios which represent the distances of said four mapped sample positions from said sample position in said second sample array in two directions.

12. A method of mapping digital video sample values from a first sample array to a second sample array under control of a mapping function, the method being substantially as hereinbefore described with reference to Figure 3 or Figure 4 of the accompanying drawings.

13. Apparatus for mapping digital video sample values from a first sample array to a second sample array under control of a mapping function, the apparatus being substantially as hereinbefore described with reference to Figure 5, or Figures 5 and 6, or Figures 6 and 7 of the accompanying drawings.