GB2253111A - 3-D fractal video scanning systems - Google Patents

Abstract

in a system for scanning a sequence of images to produce an electrical signal for transmission, e.g. a television system, the image is scanned along a locus defined by one order of a hierarchical set of three-dimensional fractal curves, the three dimensions comprising two dimensions (H and V) in the image plane, and time (T). A set of receivers of decreasing complexity equipped with appropriate signal storage can display images of decreasing quality by selecting scanning patterns of decreasing order of the fractal curves. The quality decreases not just spatially, but also temporally, so as to equalise the subjective loss of quality spatially and temporally. An algorithm can be used to develop an order of increased complexity from a scan of lower order. <IMAGE>

Description

VIDEO SCANNING SYSTEMS This invention relates to video scanning systems including apparatus for encoding a sequence of images for transmission of signals and apparatus for decoding received signals for display. One example of such system is a broadcast television system. However the term transmission is used for convenience here as that is the usual situation in practice but is also intended to cover storage or processing where appropriate.

Our British Patent Specification GB 2 193 411A discloses methods for scanning and for displaying images with a curve that is one of a family of self-similar two-dimensional fractals.

This method preserves a hierarchical property such that a recognisable image will be produced by a display using any member of the family when fed with a signal derived by scanning an image using any other member of the family. The preferred family chosen is the set of Hilbert curves, the fifth order being shown in Figure 1 of the present application. As the order rises1 the detail and hence the bandwidth of the video signal rises. Thus, if the video is subjected to a bandwidth restriction, it is as if it had been derived from a lower order source. Moreover, the resolution loss is shared between the horizontal and vertical directions which is better than the one-dimensional loss incurred with conventional raster scanning.

Reference should also be made to Proc. of the IEEE, Vol.67, No.10, October 1979, pages 1465-1466 and to Japanese Laid-open Application No. 62-264764 published 17th November 1987.

Our British Patent Specification GB 2,215,935A describes such a system adapted for interlace. Alternate samples of the fractal scan are transmitted in one picture period and the intervening samples in the next. More generally, there may be q sets of samples with l/q of the samples being transmitted on each field period in a cycle of q field periods. However, the fractal curve employed is still essentially a two-dimensional curve.

The present invention is defined in the appended claims to which reference should now be made.

In a preferred embodiment of this invention, a method of scanning is disclosed in which the scan curve is a member of a family of self-similar three-dimensional fractals, derived by Peano, in which the third dimension is chosen to be time. Thus as the order rises, the number of pictures per second as well as the spatial resolution increases. Conversely, restriction of the video signal bandwidth causes less loss of spatial resolution than in the two-dimensional case because some of the resolution loss is temporal.

The preferred embodiment of the invention will now be described in more detail, by way of example, with reference to the drawings, in which: Figure 1 (referred to above) illustrates a two-dimensional fractal scan as described in our Specification 2,193,411A; Figure 2 is a perspective view illustrating a first order three-dimensional fractal curve in H, V and T dimensions; Figure 3 is a similar view illustrating a second order three-dimensional fractal curve; Figures 4A and 4B are block diagrams of a transmitter (encoder) and receiver (decoder) in a video scanning system embodying the invention; and Figure 5 is a diagram illustrating how an n x n x n scan can be enlarged into a 2n x 2n x 2n scan.

Figure 2 shows the first order case of a three-dimensional scan joining a cube of 2 x 2 x 2 points and Figure 3 shows the second order case joining a cube of 4 x 4 x 4 points. The second order scan is obtained from the first by systematic translation and reflection of the first order pattern according to the algorithm described below with reference to Figure 5. As is seen on the figures, the scan is three dimensional in the horizontal, vertical, and time dimensions.

There are certain practical limitations in this approach.

Firstly, because time is unbounded, unlike the two spatial dimensions, the scanning of an arbitrary sequence of images must consist of a sequential series of fractal cubes representing the scanning of a series of sequences. If the starting and finishing points of the intra-cube scans are chosen to correspond to different times but with the same place, then the discontinuities caused by abutting the cubes may be minimised.

Figure 2 represents the optimum scan type for a 2 x 2 x 2 scan covering eight points in all.

Secondly, because the scan moves backwards and forwards in time, the technique implies that individual image sequences are stored in some way before being scanned. If the imaging device (camera) is discrete it is quite feasible to build storage into it so that successive images are stored as planes of samples, ready for scanning. Similarly, corresponding display devices with storage could b developed. The display of the images would, however. be subject to a delay. At first sight, this delay would seem te be equal to twice the sequence length.

This is because the information at the beginning of the sequence cannot be sent until the information at the end has occurred, and the information at the beginning cannot be displayed until the information at the end has been received. This delay can, however, be halved as the first half of the scan is confined to the first half of the sequence, and the second half of the scan is confined to the second half of the sequence.

Thirdly, there is a limit to the number of frames that should in practice be explored by a single scan, so that the storage requirements and consequent delay are realistic. The number of frames used results from a combination of setting a desired system delay and a frame rate for a particular spatial resolution. For example, suppose the frame rate is set at 48Hz for a 512 x 512 spatial resolution, corresponding approximately to present conventional scanning standards, and the system delay is set at 1/3 second. Then the delay must be 16 frames, giving a 512 x 512 x 16 cuboid and a spatio-temporal "aspect ratio" of 32. It is this quantity that remains constant as the scan order changes. Thus, the next lowest order would have 256 x 256 x 8 scans with a frame rate of 24Hz whilst the next highest would have 1024 x 1024 x 32 scans with a frame rate of 96Hz.While an equal number of horizontal and vertical points has been assumed here, they can of course be different.

Figures 4A and 4B show a schematic of the preferred embodiment. Successive images captured by the sensor 100 in the encoder of Figure 4A are transferred under the control of a write address generator 110 to one of a pair of stores 120 and 130 which can be thought of as cuboids of 2" x 2" x 2P locations containing successive image planes. The stores could be fabricated as part of the sensor, if discrete, and the precise mechanism of transfer between sensor and stores is immaterial.

When one store is full it is then read to provide a sampled video signal under the control of a read address generator 140, clocked by a pulse generator 150, which produces address sequences according to one half of an algorithm based on the above disclosure of order appropriate to the spatio-temporal resolution of the sensor. Meanwhile successive images from the sensor 100 are transferred to the other of the pair of stores. When all the contents of the first store have been read and all the contents of the second written, the roles of the two stores are reversed with the second store being read according to the other half of the scanning algorithm. This process may then be repeated indefinitely. Finally the samples are turned into an analogue signal for transmission by digital-to-analogue converter 160, clocked by generator 150, and output over a line 170.

At the receiver of Figure 4B the video signal on a line 200 is appropriately filtered by filter 210 and sampled in sampler 220, which is clocked by a pulse generator 230, at a rate corresponding to the spatio-temporal resolution of the display scan. The samples are than written under the contol of a write address generator 240, clocked by pulse generator 230, into one of a pair of stores 250 and 260 which can be regarded as cuboids of 2m x 2n x 2P' locations containing successive image planes where m'/m = n'/n = p'/p i.e. the horizontal-vertical and vertical-temporal aspect ratios remain unchanged. Generator 240 produces address sequences according to one half of an algorithm based on the above disclosure of appropriate order and forms up successive image planes in the stores 250 and 260. When one store is full its contents are transferred under the control of a read address generator 270 to the display 280. Meanwhile the sampled video signal is written into the other of the pair of stores according to the other half of the scanning algorithm. The stores could be fabricated as part of the display, if discrete, and the precise mechanism of transfer between the stores and the display is immaterial provided successive images are displayed at the appropriate times. When all the contents of the first store have been transferred and all the contents of the second written, the roles of the two stores are reversed. This process may then be repeated indefinitely.

Alternatively, the stores at the transmitter and/or receiver could hold the values in the sequence of the transmitted sequence, i.e. in the order along the scan path. It would then need the writing of the store at the transmitter and the reading of the store at the receiver to be adapted in accordance with the fractal scan, rather than vice versa as just described. In principle, of course, the order of storage in the store locations is immaterial if both writing and reading are specially adapted.

A set of receivers of decreasing complexity can display images of decreasing quality by selecting scanning patterns of decreasing order of the fractal curves. The quality decreases not just spatially, but also temporally, sb as to equalise the subjective loss of quality spatially and temporally.

DERIVATION OF THE SCAN Once a scan of desired "aspect ratio", in the sense used above, that is a scan embraced by a cuboid of the desired shape, can be found, then the scan can be "blown up" to any desired order of complexity by use of an appropriate algorithm as will now be described. It is convenient to consider a cube of equal sides having n x n x n points.

Figure 5 shows how the n x n x n scan can be enlarged to a 2n x 2n x 2n scan by reproducing then n x n x n scan in sequence in each of the eight octants of the larger scan.

Each reproduction is either a reflection about a diagonal plane of the scan cube or a rotation about a triad axis of the scan cube. These operations can be obtained by transposing coordinates or complementing them (subtracting from the highest expressible number), governed by which octant the scan is in.

If xyz axes, scan direction and octant sequences A-H are defined as shown, then the relationship between the new and old xyz coordinates (x',y',z') and (x,y,z) of a particular point in the n x n x n scan and the new most significant bits of the xyz coordinates X,Y,Z as a function of the octant is as set out in the following Table.

TABLE Octant Z Y X Equations A 0 0 0 x' = x, yl = z, z' = y B 0 1 0 x' = z, y' = x, z' = y C 0 1 1 x' = z, y' = x, z' = y D 0 0 1 x' = C - y, y' = C - x, z' = z E 1 0 1 x' = C - y, y' = C - x, z' = z F 1 1 1 x' = C - z, y' = x, z' = C - y G 1 1 0 x' = C - z, y' = x, z' = C - y H 1 0 0 x' = x, y' = C - z, z' = C - y where C is the highest expressible number for the n x n x n scan, i.e.

C=n- 1.

This can be extended to a cuboid of unequal sides by taking a part only of a fractal scan having the desired aspect ratio or cuboid shape.

Claims (8)

1. A method of scanning a sequence of images to produce a signal representative thereof, comprising scanning the images with a scanning pattern defined by one order of a heirarchical set of three-dimensional fractal curves, the three dimensions comprising two dimensions in the image plane, and time.

2. Apparatus for encoding a sequence of images to produce a signal representative thereof, comprising means for forming a signal representative of the sequence of images, the signal forming means comprising means effective to scan the images with a scanning pattern defined by one order of a heirarchical set of three-dimensional fractal curves, the three dimensions comprising two dimensions in the image plane, and time.

3. Apparatus according to claim 2, in which the said one order of the set of curves is related to a set of lower order in accordance with the algorithm defined in the foregoing Table.

4. Apparatus according to claim 2 or 3, said apparatus comprising a video camera.

5. Apparatus for decoding a received signal for display as a sequence of images, comprising means including store means for receiving the signal and applying successive elements to appropriate store locations, and means for retrieving successive images from the store means for display, in which the incoming signal elements are distributed in the store means in one pattern and retrieved from the store means in another pattern, such that incoming signals defined by one order of a heirarchical set of three-dimensional fractal curves, the three dimensions comprising two dimensions in the image plane, and time, are converted to an appropriate scan for display in time-sequential order.

6. Apparatus according to claim 5, said apparatus comprising a video display device.

7. Apparatus according to claim 5, said apparatus comprising a television receiver.

8. A method of designing a three-dimensional fractal scan curve, comprising selecting a fractal scan exploring points defined by a cuboid of desired shape (aspect ratio), and increasing the order of the curve to a desired order by translation and rotation in accordance with the algorithm defined in the foregoing Table.