GB2224410A - Video image magnification system - Google Patents

Abstract

An input video signal is digitised, 10, and stored, 11. The portion of the image to be magnified is defined by its centre point and address generator 14 calculates the number of pixels to be read out according to the required magnification. The store, 11, is read out column by column directly and via a delay, 20, to an interpolator, 17, which generates a fixed number of interpolated pixels (Fig. 4). A data selector 21 selects the appropriate number of interpolated pixels for the degree of magnification selected. The pixels, now at increased line density, are again stored, 25, and read out line by line with repetition of pixels according to the selected magnification. A D-A converter, 29, generates an analogue signal which is filtered, 30, to have the effect of interpolation in the line direction. <IMAGE>

Description

Video Image Magnification System This invention relates to a video image magnification system and method providing electronic step-wise magnification of any portion of a video image.

According to the invention, the video image magnification system comprises digital means for interpolating the values of pixels in a number of lines intermediate the lines in a portion of the original image according to the magnification required, and means for interpolating pixel values between the pixels in each line of the portion of the original image.

One preferred aspect of the invention provides a system comprising a first storage means for storing digitally line by line the values of pixels in a selected portion of a frame of a video signal and for outputting said pixel values column by column, first interpolating means for interpolating the values of a number of additional pixels between each pair output from the first storage means, the number depending on the magnification to be achieved such that the selected portion, when magnified, occupies the full frame, a second storage means for storing digitally column by column the resulting values of the pixels, and for outputting said pixel values line by line, and second interpolating means for interpolating the values of the said number of additional pixels between each pair output from the second storage means.

According to another preferred aspect of the invention, a video image magnification system comprises: (a) an analogue-to-digital converter for providing a digital video signal; (b) means for selecting the magnification factor; (c) means for designating a portion of the image to be magnified by the magnification factor to the size of the whole image; (d) first storage means arranged to store selectively data in lines representing the lines of pixels in the designated portion of one frame of the image.

(e) means for reading out the pixel data serially column-by-column from the storage means; (f) first interpolating means for interpolating between the data for each vertical pair of pixels data representing a series of additional pixels intermediate said pairs, the number interpolated being determined according to the magnification factor selected; (g) second storage means arranged to store column-by-column the pixel data received from the first interpolating means representing the designated portion of the frame; (h) means for reading out from the second storage means data representing lines of pixels to form the magnified image at a rate within each line inversely proportional to the magnification factor and at a line rate such that the output frame rate is equal to the original frame rate;; (i) second interpolating means for interpolating between the data for each pair of pixels data representing a series of additional pixels intermediate said pair, the number interpolated being determined according to the magnification factor selected; and (j) a digital-to-analogue converter for converting the serial data to an analogue video signal.

The system of the invention may suitably be provided with a signal from a television camera and may provide a picture representing a portion of the image provided by the camera, magnified by a factor of, say, 2, 4 or 8, with a delay of no more than that required for 2 frames of the picture to be completed, thus effectively providing real-time electronic enlargement.

The invention also provides a method of magnifying a selected portion of a video image, comprising storing line by line in a first storage means digital values of the pixels in the selected portion, outputting the pixel values from the first storaage means column by column, interpolating a number of additional pixel values between each pair of pixel values output from the first storage means, the number depending on the magnification to be achieved such that the selected portion, when magnified, occupies the full frame, storing column by column in a second storage means the resulting digital values of the pixels, outputting the pixel values from the second storage means line by line, and interpolating the said number of additional pixel values between each pair of pixel values output from the second storage means.

Reference is made to the drawings, in which: Figure 1 is a schematic diagram of the system according to one embodiment of the present invention.

Figure 2 is a diagrammatic representation of a video image showing a portion designated for magnification by the system; Figure 3 is a flow diagram showing the operation of the counters forming part of the address generator of the system illustrated in Figure 1; Figure 4 shows in schematic form the digital interpolation device for use in the system shown in Figure 1; Figure 5 is a schematic diagram of the system according to a second embodiment of the invention; and Figure 6 shows in schematic form a digital interpolator device for use in the system shown in Figure 5.

Referring to Figure 1, a CCIR video signal is input to an analogue-to-digital converter 10, which provides a digital video output to a store 11 and synchronisation signal outputs on four separate output lines shown as a single line 12. These synchronisation outputs consist of pixel clock, line synchronisation pulse, frame synchronisation pulse, and field synchronisation pulse signals. These are supplied to a control bus 13 to which are also supplied magnification select signals.

The store 11 comprises two 128K RAMS, which can be toggled between an input function and an output function. Two address generators 14 and 15 control the input and output to either RAM. The first address generator 14 controls input to either RAM according to its toggled state, while the second address generator 15 controls the output from the RAMS. The first address generator 14 receives signals via the control bus 13 indicating the size and position of the portion of the image to be magnified, the size being a function of the magnification. In the embodiment described, the magnification provided for is in three steps, namely 2 times, 4 times, and 8 times.The size is designated by the magnification selected, the zoom selector providing a signal on two lines as follows: Magnification Magnification Size of Valid Select Factor Area (pixels) Z1 Z0 0 0 x 2 256 x 256 0 1 x 4 128 x 128 1 0 x 8 64 x 64 1 1 x 1 Not Applicable The first address generator 14 includes an X co-ordinate counter which counts the pixels along a line, incrementing with the pixel clock pulses and resetting with the line sync. pulses. Referring to Figure 2, the total screen consists of 512 lines, each of 512 pixels. The shaded area represents an area of 128 x 128 pixels designated for magnification at a factor of x 4. The position of the designated area is expressed in terms of x0 and y0, being the coordinates for the centre of the designated area. The patch size Py and Px in each coordinate direction give the valid area. The Y counter increments on line sync. pulses and resets on field sync. pulses. Thus, the X counter counts pixels along a line and generates a signal which starts the storing of digital pixel information in one of the RAMS of the store 11 when the X count is in the valid area about the x off-set position. The Y counter counts lines down a field-(not a frame) and indicates when the Y count is in the valid area.

Referring now to Figure 3, the flow chart shows the operation of the X counter, operation of the Y counter being essentially the same except that, because CCIR video signals consist of two fields of, in this example, 256 lines interleaved to form a frame, the two tests in the flow chart will test the count for equality to -(P /4) and +(P /2), respectively. The address y y generator counters are enabled by the relevant line active and field active timing signals so that they only count in the active area of the video signals and not through blanking.

When the digital pixels for the complete area have been stored in the RAM, the RAMS are toggled so that the second RAM can now receive at the appropriate time information for the next frame, while the first RAM comes under the control of the second address generator 15. The data is stored in the first RAM in a form representing an array of lines of pixel data representing the lines in the designated area. The second address generator 15 reads data out in a manner representing columns of pixels from the designated area of the image. The data are fed along a line 16 to an interpolator 17 which has twb entry ports 18 and 19, the first 18 fed direct, while the second 19 is fed via a latch 20 which serves as a delay of 1 byte so that a pair of successive bytes representing successive pixels is presented at the input to the interpolator 17.

The operation of the interpolator 17 will be more clearly understood by reference to Figure 4. The interpolator is provided with 8 output ports 40a-h. The first of these, 40a, is connected direct to the input port 19 receiving the delayed byte representing the first of a pair of pixels. Both input ports 18 and 19 are also connected directly to a first adder 41 which adds together the two bytes and then deletes the least significant bit of the sum, effectively producing the average of the two bytes. This average appears at output port 40e, and is also fed to two further adders 42 and 43. The first of these 42 has as its second input a direct line to the input port 19, and therefore produces a signal on port 40c which is the average of that at 40a and that at 40e.Similarly adder 43 has a direct connection to input port 18 as its second input and produces an output which is the average of the value at 40e and that on the input line 18. Further adders 44 to 47 are similarly arranged to produce values which are the average between 40a and 40c, 40c and 40e, 40e and 40g, and 40g and 18, respectively. Thus, the output from the interpolator 17 is a series of seven pixels whose values are equally spaced between each pair of pixels, together with the first of the pixels, giving eight pixels in total.

Reverting now to Figure 1, the output from the interpolator 17 is fed to a data selector 21 controlled by a sequencer 22 which is in turn controlled by the system controller 23, which controls the operation of the whole system. The data selector 21 selects which of the interpolated pixels on lines 40a to 40h are to be used, in accordance with the selected magnification.

Thus, when x 8 magnification is required, all eight interpolated pixels -will be used. Where x 4 magnification is required, every second pixel is discarded. Where x 2 magnification is required, only the outputs on lines 40a and 40e are selected. The rate at which data are output from the data selector to line 24 as a serial output is a function of the zoom factor.

The data on line 24 are read into a first 128K RAM of a second store 25, under the control of a third address generator 26. The data are stored as an effective array, the data being input as columns in the array.

Each column has a length which is equal to the magnification factor multiplied by the column length of the original sample. The line widths are the same.

Thus, in the example illustrated by Figure 2, the stored data will be 128 columns of 512 bytes (representing pixels).

When the complete frame comprising two interleaved fields has been stored in the first RAM, the two RAMS forming the store 25 are toggled so that second RAM is ready to receive data representing the next frame, while the first RAM i.s ready for the data to be read out. The second RAM is controlled by a fourth address generator 27 whose function is similar to that of the third address generator 26, the two address generators 26 and 27 being toggled with the RAMS and running different programmes according to the toggled state. The address generator 26 controls the output of a video signal to an 8 bit output bus 28 line by line at a rate equal to the input clock rate divided by the magnification factor.

Each pixel is therefore output for a time proportional to.the magnification factor.

The resulting digital video signal is converted back to an analogue signal in a conventional digital-to-analogue converter 29, producing an analogue signal which is stepped between the values of successive pixels in the line. Each step has a length equal to the expected length of the pixel signal multiplied by the magnification factor. This signal is then passed through a low pass filter 30 having a switched time constant in accordance with the magnification factor.

The time constant can be switched by selecting the appropriate one of a plurality of capacitors according to the magnification factor selected. The result is to produce a continuous change between the peaks and troughs of the signal, rather than steps. This has the effective of interpolating the average pixel values in the line between the original pixel values. Thus, the output from the filter 30 is an analogue signal containing 512 x 512 pixels generated from the original area of 128 x 128 pixels selected from the complete frame. It will be seen from the above description that the system can operate effectively in real time.

Referring now to Figure 5, an alternative system is illustrated, in which a mathematical interpolator is used in place of each of the interpolation stages described with reference to Figure 1. In Figure 5, those components which are the same as in the embodiment of Figure 1 are given the same reference numerals, and will not be described again in detail. The interpolator 17 and data selector 21 in the embodiment of Figure 1 are replaced by a first mathematical interpolator 50, described hereinafter in more detail with reference to Figure 6, while the digital-to-analogue converter 29 and low pass filter 30 of the Figure 1 embodiment are replaced with a second mathematical interpolator 51, with associated latch 52 and sequencer 53, and a final stage digital-to-analogue converter 54. The interpolators 50 and 51 are essentially indentical, and will now be described with reference to Figure 6, which illustrates the arrangement of the first interpolator 50. The interpolator comprises two multiplication devices 60 and 61, comprising PROM look-up tables, and an adder 62 for adding the outputs from the devices 60 and 61. Control lines 63 from the sequencer 22 (Figure 5) are connected to each multiplication device 60 and 61. Input ports 18 and 19 carry a pair of successive pixels in digital form, while the output line 24 carries a sequence of pixels resulting from the interpolation.

In use, a pair of pixels X and Y arrive at ports 18 and 19 and are input to the respective devices 60 and 61. The sequencer 22 (Figure 5) supplies to each device 60 and 61 a sequence of numbers, according to the selected magnification. Thus, for a X2 magnification, the sequence will be 0, 4, 0, 4 ....... For X4 magnification, the sequence will be 0, 2, 4, 6, 0, 2, 4, 6 , while for X8 magnification, the sequence will be 0, 1, 2, 3, 4, 5, 6, 7, 0, 1 .......

Taking X2 magnification for convenience, by way of illustration, devices 60 and 61 derive from the look-up table, for each number in the sequence, a value by which the respective pixel value X or Y is multiplied, the result, for each number in the sequence, being added to that for the other pixel of the pair to produce the output pixel Z. Thus, the look-up table will provide a value line where i is the sequence number and n is the maximum magnification factor = 8, and for X2 magnification, the output pixels Z will be Y and Y/2 + X/2 For a X4 magnification, the output pixels Z will 3y x y x 3X be Y, /4 + /41 '2 + X/2, /4 + /4.

It will be seen that, in this embodiment, the interpolator uses fewer components than that illustrated in Figure 4, and that the need for final low pass filtering is avoided.

The foregoing description applies to monochrome video imagery which has been derived from a true CCIR sensors, that performs two (offset) scans of FIELDS which are interleaved to produce the displayed FRAME.

Frame by frame processing is assumed in the description.

However, problems can arise, caused by the sensor using two field scans per frame of video. Thus the input fields are displaced in time by 20ms. When displayed on a monitor the fields are again displayed sequentially.

The result is that the different data in the two fields produces an improvement in both spatial and temporal resolution over that which could be obtained with just one field. The persistance of the eye and the monitor allow the observer to perceive smooth subject motion and good vertical resolution.

The difference between FIELDS and FRAMES leads to the two fundamental approaches to 'zooming', or magnifying a portion of, such an image, viz: (i) Process each input FIELD as an independent image to produce the corresponding field in the output CCIR video imagery; or (ii) Process each input FRAMAE (consisting of two correctly interleaved fields) as though it represents a single image. This is 'zoomed' and alternate lines are then taken from the resulting image to display as the FIELDS of the output CCIR image. (This is assumed in the description of the drawings).

Processing each FIELD as in (i) above produces good results with moving subjects because the time delay between the capture of the 2 fields is preserved in the output and thus helps to convey smooth motion.

However, the image flickers at FIELD rate (50Hz).

This effect is most prounounced at high zoom factors and at grey level boundaries in the subject. With a stationary subject a situation can resultwhere one FIELD displays a group of white lines due to zooming, in an area which is filled by black lines in the alternate FIELD. It is because each pair of lines is interpolated into a number of interleaved lines that the perceived effect is an area of screen flashing between alternate grey levels. It is possible to overcome the problem of differences between consecutive fields by either displaying only one field or by averaging between fields. However both of these 'solutions' further reduce the vertical resolution in the displayed image.

Processing each FRAME as in (ii) above ignores the time delay between field capture. It thus produces ideal results with a stationary image (where fields improve only spatial and not temporal resolution).

However a moving subject suffers 'tearing'. This 'tearing' is a horizontal band of lines of a contrasting grey level, visible on grey level boundaries in the image, and is apparent because the displayed FRAME consists of alternate bands of lines interpolated from individual lines of alternate FIELDS. However, the alternate FIELDS themselves were captured at different times, hence the subject occupied different positions in the image. Thus the eye perceives a FRAME which consistently displays the subject torn (in bands) between the positions it occupied in the consecutive input FIELDS.

The width of the bands (in lines per FRAME) is the zoom factor, hence tearing is most pronouced at large zoom factors. Also the extent of the banding effect depends upon the distance moved by the subject between consecutive FIELDS, hence a large amount of motion exacerbates the problem.

Where the superior resolution of the FRAME processing technique is essential, the subject could be tracked and minor oscillations between fields could be reduced by re-registering FIELDS of the image. This would reduce the tearing seen on the subject while preserving the resolution. It would however, casue the background to tear with a moving subject. Temporal filtering on the output of the system can also help to reduce the perceived tearing by blurring the edges of a moving subject and hence reducing the contrast of the bands seen at its edges.

Ideally, a sensor generating a single field per frame would be used to avoid these problems.

Although the invention has been described with reference to a monochrome image, it would be possible to use the same system with colour, for example by using 3 units, each processing one channel of an RGB video signal.

Claims (14)

1. A video image magnification system, comprising means for interpolating the values of pixels in a number of lines intermediate the lines in a portion of the original image according t the magnification required, and means for interpolating pixel values between the pixels in each line of the portion of the original image.

2. A video image magnification system according to Claim 1, comprising a first storage means for storing digitally line by line the values of pixels in a selected portion of a frame of a video signal and for outputting said pixel values column by column, first interpolating means for interpolating the values of a number of additional pixels between each pair output from the first store, the number depending on the magnification to be achieved such that the selected portion, when magnified, occupies the full frame, a second storage means for storing digitally column by column the resulting values of the pixels, and for outputting said pixel values line by line, and second interpolating means for interpolating the values of the said number of additional pixels between each pair output from the second store.

3. A video image magnification system according to Claim, comprising: (a) an analogue-to-digital converter for providing a digital video signal; (b) means for selecting the magnification factor; (c) means for designating a portion of the image to be magnified by the magnification factor to the size of the whole image; (d) first storage means arranged to store selectively data in lines representing the lines of pixels in the designated portion of one frame of the image.

(e) means for reading out the pixel data serially column-by-column from the storage means; (f) first interpolating means for interpolating between the data for each vertical pair of pixels data representing a series of additional pixels intermediate said pairs, the number interpolated being determined according to the magnification factor selected; (g) second storage means arranged to store column-by-column the pixel data received from the first interpolating means representing the designated portion of the frame; (h) means for reading out from the second storage means data representing lines of pixels to form the magnified image at a rate within each line inversely proportional to the magnification factor and at a line rate such that the output frame rate is equal to the original frame rate;; (i) second interpolating means for interpolating between the data for each pair of pixels data representing a series of additional pixels intermediate said pair, the number interpolated being determined according to the magnification factor selected; and (j) a digital-to-analogue converter for converting the serial data to an analogue video signal.

4. A system according to Claim 2 or 3, wherein the first and second storage means each comprise a pair of memories and means for toggling between the memories whereby data for one frame can be read into one memory and data for the preceding frame can be simultaneously read out of the other memory.

5. A system according to Claim 2, 3 or 4, wherein the first and second interpolating means each comprise means for simultaneously inputting data representing a pair of pixels and means for calculating from said data a series of values of pixels intermediate said pair of pixels, the number in said seried depending upon the magnification factor selected.

6. A system according to Claim 5, comprising a sequencer for supplying to the calculating means a sequence of numbers according to the magnification factor selected, the -calculating means comprising a first multiplication means operable on the first of the pixels in the pair and a second multiplication means operable on the second of the pixels in the pair, the multiplication means each being arranged to derive from each number in the sequence a fractional number by which the pixel value is multiplied, and summing means connected to the two multiplication means and arranged to sum the numbers output thereby to provide an intermediate pixel value.

7. A system according to any of Claims 2 to 6, wherein each frame of the video signal comprises two interleaved fields of lines, data relating to the first of the two fields being stored in said storage means in such a manner that data relating to the second field can be interleaved therewith, and alternate line data being read out of the second storage means by the reading out means, so that the resultant analogue video signal comprises successive interleaving fields.

8. A system according to any of Claims 2 to 7, wherein the first storage means comprises an address generator arranged to input to a random access memory only data representing successive pixels falling within the designated portion of the image.

9. A system according to Claim 8, wherein the address generator comprises counters for counting the lines within a field and the pixels in a line to detect the start and finish of each designated portion.

10. A system according to any preceding claim, arranged to operate at a rate such that the delay in the system is not more than the time required for two complete frames.

11. A video image magnification system, substantially as described with reference to the drawings.

12. A method of magnifying a selected portion of a video image, comprising storing line by line in a first storage means digital values of the pixels in the selected portion, outputting the pixel values from the first storaage means column by column, interpolating a number of additional pixel values between each pair of pixel values output from the first storage means, the number depending on the magnification to be achieved such that the selected portion, when magnified, occupies the full frame, storing column by column in a second storage means the resulting digital values of the pixels, outputting the pixel values from the second storage means line by line, and interpolating the said number of additional pixel values between each pair of pixel values output from the second storage means.

13. A method according to Claim 12, which comprises initially converting an analogue video signal to a digital video signal, and converting the digital video signal resulting from interpolation of the values output from the second storage means to an analogue video signal.

14. A method of magnifying a selected portion of a video image, substantially as described with reference to the drawings.