GB2205704A - Reduced bandwidth video transmission - Google Patents

Abstract

The bandwidth reduction of a video signal is attained by processing with a two-component model, the first being a very economical but recognizable cartoon or line drawing which is derived from the grey-level image by valley detection 210. The second component is generated by using the cartoon as a sampling template to sample the grey-level image, so that samples, for example two, of the original image are taken at the cartoon lines. A (or several) further sample is taken mid-way between the valleys and the original image is reconstructed using two dimensional second order (quadratic) interpolation between the sampled points. To generate the most suitable line drawing the grey level image is thresholded, the discontinuous lines generated by valley detection are filled in, i.e. made continuous and this image is subjected to boundary extraction, 230 which produces a continuous line drawing with lines of one pel width. Streaks produced in the final, interpolated image may be removed by allowing distinction between valleys and edges and then applying conditional adaptive filtering to the interpolated image. <IMAGE>

Description

REDUCED BANDWIDTH VIDEO TRANSMISSION BASED ON TWO-COMPONENT MODEL This invention relates to a method and apparatus for producing a two component video signal, and also for reconstructing a visual image from such a signal. The invention finds particular application with low resolution moving grey-level images as well as colour images for transmission over narrow-band channels, though such image signals may be stored in and retrieved from a suitable storage medium, utilising a minimum of data storage space.

Digital images, being two or even three dimensional in nature (moving picture sources), tend to generate data at unrealistically high rates for transmission over a narrow band channels. It is essential therefore to adopt some coding strategy; a large number of techniques exist, such as DPCM, Transform or Hybrid Coding, but these can only achieve a moderate reduction in bandwidth.

Another technique, known as synthetic highs, simply detects the edges in one dimension along a scanning line, run-length coding the edge location and magnitude, and then generating a "synthetic highs" signal from the edge location. This signal is then added to a low-pass signal to give a good-looking picture with a reduction in bandwidth by a factor of four. This system is described in a publication by W.F.Schreiber et al., "Synthetic highs An Experimental TV Bandwidth Reduction System" J. SMPTE, volume 68, pp. 225-237, August 1959.

Another technique, called Contour-Texture Oriented Methods, reduce the bandwidth by m factor of fifty or so. One such system is described in a publication by Murat Kunt et al., "Second Generation Imaqe-Coding Techniques" Proceedings of the IEEE, volume 73, No. 4, April 1985 pp. 549-574. The system attains a bandwidth reduction by segment ng the Image into texture regions surrounded by c o n t o u r s 5 U c h t h a t the contours correspond, as much as possible, with those of the object in the image. Contours are extracted by either edge-detection or region growing.Contour and texture information are then coded separately. The image is constructed by adding these two components and applying polynomial approximation. Although the compression ratio is high this method has some drawbacks. First, the image needs to be preprocessed to reduce the local noise without affecting its contours. second, not all the regions correspond, in general, to real objects of the original image and thus become false contours. Leonardi and Kunt have improved the system by introducing the idea of split and merge to reduce the number of regions in the segmented image.The system is described in a publication by R. Leonard and M. Kunt, "Adaptive Split for Image Coding", Proceedings of the IASTED International Symposium on Applied Signal Processing and Digital Filtering, Paris, June 19-21, 1985. A similar system is described in a publication by C.G. Ward, "Image Compression by Regional Analysis", IERE Conference on Digital Processing of Signals in Communications, pp. 179-184, 22-25 April, 1985. The system attains the bandwidth reduction by segmenting the image Into closed loop regions, the edges are detected by two ede Jetection masks, the Sobel and the Marr- Hildretn operator.The edge InformatIon s coded by chain coding while the grey-level values in each region IS coded by modelling Its variation by l second order quadratic equation using least square fitting.

Yet another technique for reducing the video bandwidth is called plateau coding; this technique is first applied for coding the chrominance component of colour picture signals. The system is de described bed in a publication by John O. Limb and Charles B. Rubinstein., "Plateau Coding of the Chrominance Component of Color Picture Signal" I EE Transactions on Communications, volume Com-22, No. 6, pp. 812-820, June 1974. In this system the picture is divided into regions of approximately constant chromaticity, the amplitudes are transmitted by sending one set of chrominance values for the whole region.The luminance signal is transmitted accurately and used at the transmitter and receiver to indicate changes in the two chrominance signals, hence addressing information about the regions need not be transmitted. New chrominance samples are only transmitted when there are significant luminance changes. A similar system was described in a publication by J. Rodriguez and D. Barba "Picture Coding Scheme Based on Two-Component Model", Picture Coding Symposium, PCS 34, Cesson-Sevigne, France, July 3-5, 1984. In this system the picture is modelled by twoadditive components.The first component, called the plateau component, contains information about the localmean values, while the second component ls the texture component an contains information about the local variations of the image signal apart from its mean value. The plateau component is generated by an edgedetection, while the texture component is obtained by subtracting the decoded plateau component from the original picture signal. The plateau component is approximated by piecewise linear segments, while the texture component is coded by a Block Truncation Coding Scheme (3.T.C.).

Still another technique for reducing the video bandwidth, called composite source coding, transmits for each image two independent additive components which have well defined statistics. The first component is a low-pass signal obtained by low-pass filtering or by straight line fitting, so that it can be coded efficiently, while the second component is a high-pass signal. The high-pass signal can either be coded by transform coding or by adaptive bit assignment with variable quantization. Two different systems are described in a publication by Johnson K. Yan and David J.Sakrison, "Encoding of Images Based on a Two Component Source Model", IEEE Transactions on Communications, volume Com-25, No. 11, November 1977, and by D.K.iMitrakos and A.G.Constantinides, "Nonlinear Image Processing for Optimum Composite Source Coding", IEE Proceedings, volume 130, Pt. F, No. 5, August 1983.

The above techniques are not mutually exclusive and some have seen combined in the prior art. For instance, one such system is described by D. Barba, "sew Self-Adaptive Edge Detection Scheme with Application to Picture Coding Based on a Two-Component Model", Second International Conference on Image and its Application, lEE Conference Publication Number 265, 24-26 June 1986.

Recent studies in human and computer vision have suggested new ways of'representing and modelling images, which in turn have led to a novel approach to twocomponent coding.

According to the broadest aspect of the present invention there is provided a method for producing a two component video signal corresponding to an image derived from a video camera, in which method the first component is obtained by producing a cartoon image of the original image and then performing a boundary extraction process on that cartoon image to yield a two-level primitive sketch of the original image, and the second component is produced by sampling the original image using the primitive sketch as a sampling template, whereby an approximation of the original image may be reconstructed from said two component video signal.

It will be appreciated that the video signal of this invention may be transmitted as low-resolution (64x64 or 100x100) grey-level images, and possibly colour, over a narrow band channels, such as the integrated Services Digital Network (ISDN). The method employed Tan 5e summarized. into a two-step process. The first step -was to generate a 2r m-- fe sketch from the grey-level ::age, generated by means of valley detection (a cartoon extraction is described n a publication by D.E.Pearson and J.A.Robinson "Visuai Communication at a Very ow Data Rates" Proceedings OL the IUDE, volume 73, No. 4, pp. 795-812, 1985).The second step is to use the sketch as a sampling template to sample the image so that only luminance values along the cartoon lines are coded for transmission to the receiver. Reconstruction of the original image may then be performed by means of interpolation. Experiments have shown that if the sketch is used as a sampling template together with a few of selected mid-points, a good likeness of the original image can be constructed; mid-points are chosen since they need no address information to be transmitted. In the case of colour images, the same technique as described above may be used for coding the luminance signal. The chrominance signal can be coded in a two-step process. The first step is to subsample the chrominance signal with a reduction of 1:25 or 1:64, and use 2 bits DPCM to transmit their values.The second step is to use the statistics which are extracted from the distribution In the cartoon images to expand the chromatlcity signals to their original dimension.

The method of this invention may be performed to smooth and filter the coded images. Tests showed the interpolated images suffered from some streaks due to.

the imperfect interpolator and the fact tat the cartoon extractor has a response to both luminance valleys and luminance edges, but that these streaks could be removed by a two-step process. The first step is to classl~y each feature as a valley or an edge and pass this information to the interpolator, and the second step is to apply conditional adaptive filtering. The streaks only occur at non-sampled points, especially near midpoints; thus by detecting streaks by conventional operators and using their sharpness to control the filter coefficients, the streaks can be removed.

The sampling template used in performing this invention may be improved further to reduce the number of streaks which occur in the coded image. By checking for picture elements (pels) below some threshold level, assigning those pels as feature points and then running a cleaning routine or a boundary extraction routine to extract the occluding contours of the object, one can force continuity in the sketch and hence obtain a better sampling template.

When reconstructing the image the interpolation and filtering process, may be speeded up by scanning the cartoon image and computing the addresses of the first and last cartoon point on each line. This information is then used in a logical way to separate the object from the backgrounc. In this way, the data to be processed is reduced, as is the computation time.

In order that this invention na better be znaerstood, it will now be described in grater detail by way of example only, reference being mauve to the accompanying drawings, in which:: Figure 1 is a simplified block diagram of an embodiment of the present invention; Figure 2 is a detailed block diagram of the embodiment of the invention as shown in Figure 1i Figure 3 shows the impulse response of the filter section of the cartoon extractor; Figures 4(a) and 4(b) show the actual implementations of cartoon detectors, operating on 3x3 and 5x5 windows, respectively; Figure 5 shows test images, Figure 5(a) being at a resolution of 100x100, Figure 5(b) being 4 frames at a resolution of 64x64, and Figure 5(c) and (d) being at a resolution of 256x250; Figures (m), 6(c) and 6(d) show sketches constructed from valley points of the images of Figure 5; Figure 7(a) shows a section through a concave 3dimensional surface;; Figure 7(b) shows a section through a convex 3dimensional surface; Figure 7(c) shows the luminance profile for the surfaces of both Figures 7(a) and 7(b) for frontal illumination; Figure 7(d) shows samples taken at luminance valleys (cartoon lines) and at a single mid-point with various simple reconstructions; Figure 8 shows the boundary extractor operator; Figures (a) to 9(,) show the cartoons of the images shown in Figures 5(a) to (d), using valley and threshold extraction techniques; Figures 10(a3 to 10(d) show the sketches generated by applying boundary extractions to the images of Figure 9(a) to 9(i), with the operator of Figure 8; Figures 11(a) to 11(d) show the grey-level values along the cartoon lines of the sketches shown in Figures 10(a) to 10(d);; Figures 12(a) to 12(d) show the corresponding reconstructed images; Figures 13(a) to 13(d) show the reconstructed images with the streaks removed; Figure 1 4 shows the block diagram design of an embodiment of this invention, as applied to a colour system; Figure 15 shows a section of 10x10 pels of a colour primitive sketch; and Figures 16(a) and (b) show the original and coded colour images at a resolution of 256x256.

Figure 1 depicts in block diagram form one embodiment of this invention. Video camera 10, which may be any conventional video camera provides 256 lines and 512 pels per line. The frame store (not shown) provides the necessary vertical and horizontal synchronization signals. Sketch generator 20 operates on the grey-level image 40 and produces a two level binary nage, black and white. The steps in extracting the sketch (for our coding system) are shown in Figure 2 by blocks 210, 230 and 240. The grey-level image 40 is sampled by the coarse sampler 50 at feature points and few selected mid-points determined by the primitive sketch 250 block (Figure 2). Mid-points between two successive feature points are sampled if and only if the number of pels between these feature points exceeds four. Mid-points are chosen for two reasons, simplicity and further reduction in the video bandwidth; since no addressing information is required to be transmitted to the receiver. Interpolator 30 operates on the output of coarse sampler 50 in two dimensions and produces grey-level output. The interpolation function used in this invention is a second order quadratic.

Display 60 is a conventional black and white monitor.

Figure 2 depicts a detailed block diagram of a complete system of this invention. Channel 100 is a 64 kbits Integrated Service Digital Network (ISDN). The diagram does not show the memory needed for the three Dimensional Relative Address Coder and Decoder (3D-RAC) blocks 80 and 800 respectively. Error protection codes, known in the art, may be used in channel 100.

Primitive sketch 250 which is used as a sampling template to sample the grey-level image 40 is derived by a three step process. First a cartoon image 220 is obtained by running R threshold and valley detector 210 on the grey-levei image 40. The technique for this extractor was reported by Pearson and Robinson (supra).

Their theory says that cartoon lines must be drawn at points where lines from the camera graze the object surface tangentially; by analyzing the light falling on the object and assuming a diffuse illumination, this criterion amounts to luminance valleys in the image. A complete illustration of valley detection is reported by J.A. Robinson, "tow Data-Rate Visual Communication", Ph.D. thesis, Department of ESE, University of Essex, U.K. 1985. It is worth noting that the image feature points produce luminance valleys when all local surfaces have the same reflectance (e.g. the edge of the nose or at the edge of the finger held in front of the face); it is, however a step edge when seen against a background of different reflectance (e.g. a finger held in front a wall of greater or lesser lightness than skin).

Figure 3 shows the impulse response of the cartoon detector; this has a primary response to luminance valleys, but possesses a secondary response to luminance edges. Figure 4(a) and Figure 4(b) show examples of detectors used in generating the cartoon image.

Figure 4(a) shows a 3x3 "Edge Counting Logical Operator", operating on 3x3 peis. It is simple and fast, requiring eight neighbour pels and one counter.

Figure 4(b) shows a 5x5 Logical Valley Detector". This detector performs a serves of computations Ln four orientations. If a valley is detected in any of these, afeature point is narke. For vertical valley detection shown in Figure 4(b) the computation begins by checking the differences (l-m) and (n-m). Provided one of these is over threshold T1 , the search for a valley proceeds.

This check is Included for eiimination of unlikely points.

Examples of cartoons generated using these operators are shown in figures S(a), 6(c) and 6(d). The criterion on which size of operator to use for the purpose of extraction is subject to scaling. It was found experimentally that for low resolution images (64x64 or 100x100) the 3x3 operator performs better than the 5x5, while for high resolution images (256x256 or 512x512) the 5x5 operator generates a better sketch.

Each feature point in the cartoon is either a valley or the foot of a rising edge. Furthermore, the surface between each two feature points derived using one of the detectors described in Figure 4 is rising upwards. Figure 7 is a visual explanation of the above postulate. Figures 7(a) and (b) show a section through a smooth concave and convex three dimensional surface respectively. By assuming that the illumination is from the general direction of the camera, the luminance profile Is then as shown in Figure 7(c), which consists of two valleys separated by an upward convexity.There are a number of objects for which the above assumptions are not true, such as fabrics or highly textured surfaces; but this is nearly true for human faces ana hands. ConsiderIng now appropriate spatial positions at which to sample the luminance profile (the grey-level image) In order to approximate it, it is apparent that two samples taken at the luminance minima (correspondIng to the cartoon points) and another taken roughly midway between would be an appropriate choice. A smooth curve drawn between these samples would be an improvement on the flat white of the cartoon, which is a rectangular approximation (Figure 7(d)). The mid-point between the valleys IS chosen here since it requires no additional addressing information to be transmitted to the receiver.A better choice is to sample at the point of inflexion (e.g. maxima) but then the receiver needs to know the position of that sampled point.

Due to the discontinuity in the contour lines, if the cartoon images of Figure 6 are used as sampling templates to sample the grey-level image for the purpose of reconstruction, problems in image quality and bandwidth reduction will arise. A cartoon sketch which has a very few feature points but which occludes the objects of interest is required - i.e. continuous lines with no or little breaks. This is because the code used for transmitting the cartoon depends on the distance of each cartoon point from the previous one; the longer the distance between the feature points, the longer the word-length needed for transmitting the relatIve distances per image, and hence the greater the resultant ol rate.The same explanation for coding the grey-level valves applies here, if use is made of DPCX5 along the contour lines; the shorter the relative distances, the higher the correlation and hence the shorter the codes.

Furthermore, if the object is not completely occluded by the cartoon lines, the reconstruction circuitry will make no attempt to occlude it and hence that part of the image will be missed out from the reconstruction.

A two step method may be used to overcome the discontinuity problem. The first step is to include a black-level in-fill in the cartoon sketches, by means of thresholding the image before extraction. The second step is to include a cleaning operator or boundary extractor 230 (Figure 2), which takes the cartoon sketch with the black in-fill as its input and produces a very thin line drawing of one pel width with continuous contours. The boundary extractor operator is shown in Figure 8, and is a 3x3 window which operates on a 3x3 region of cartoon images as shown in Figure 9. This shows the cartoon images generated by applying the operators of Figure 4 together with the black in-fill.

Figure 10 shows the cartoon sketches genera-ted by applying the boundary extractor in Figure 8.

Figure 12 shows the images as reconstructed from the grey-level images of Figure 11, together with a few selected samples taken midway between each feature of Figure 11. The interpolation used was a two dimensional second-order (smooth curve) interpolation. Interpolator 30 (Figure 2), operates on the image in two dimensions in a three step-process. The first step was to interpolate the image in the horizontal direction, using the template of Figure 11 together with the selected mid-points. The second step is to interpolate the image in the vertical direction, using only the template of Figure 11 the mid-points for the vertical interpolation are derived from the output of the horizontal interpolator. The third step is to superimpose the output of the horizontal and vertical interpolation to obtain the final two dimensional reconstruction, shown in Figure 12.

It can be seen that the reconstruction of Figure 12 suffers from some streaks. The streaks only occur at non-sampled points, especially near mid-points, which is due to the imperfect interpolator and the fact that the cartoon detector has a response to both luminance valleys and luminance edges. it was found that these streaks could be removed by allowing the operator to distinguish between the valley or edge feature points and then applying conditional adaptive filtering to the interpolated images.

Referring back to Figure 2, the valley edge classifier 240 is a 3x3 window which operates mutually on the Images shown in Figure 10 and Figure respectively. For every feature point in the cartoon sketch Figure 10, it computes the =our directional arlvatives (north, south, ast and west) from the grey-level image. If one of the derivatives IS over some threshold level then the corresponding feature point is an edge. If an edge is found the sampling template figure 10 is updated accordingly. For example, assume an edge in the north direction has been detected, then the grey-level value of the pel above that feature point must be transmitted as well.

The second step in removing the streaks is in the application of the conditional adaptive filtering. The conditional adaptive filter 310 operates mutually on the output of the interpolator and the primitive sketch 2500. The filtering routine consists of two steps, the first step being the use of the primitive sketch 2500 to mask the output of the interpolator and then detecting streaks at the non-sampled points by conditional operators (streaks only occur at non-sampled points).

The second step is the use of the streak sharpness to control the filter coefficients. Figure 13 shows the final coded images after. applying conditional adaptive filtering.

A method for background removal was included to speed Lip the reconstruction process and to attain further reduction in the video bandwidth. The problem of background removal is one of pattern recognition, subject to some restriction. For example, the object to be detected may have a widely variable shape and size, so template comparison techniques are unsuitable.

The method of background removal which has been developed utilises an image processing method which detects black to white and white to black transitions in primitive sketch 2500. Object-background extractor 310 operates on primitive sketch 2500 by scanning the image line by line in the horizontal and vertical directions.

It computes the number of feature points on each line, the distance between each two adjacent features and their addresses. The object is included within the region defined by the address of the first and last feature points in each horizontal line and by the address of the first feature point in each vertical line. Therefore, in the reconstruction process a simple "oring" test is applied on each pel in the image, if it passes, the corresponding pel is in the background region, else it is in the object region. Interpolator 30 operates on the object only and the background is shaded with one grey-level value.

The data rates generated by the method described is picture dependent, but tests have shown that only 9% to 22% of the original data need be coded and transmitted to the receiver. The table below shows simplified calculations of the data rate for the coding method described, for two different sizes of picture.

In the calculations the following assumptions were made: 1) The original images are low-resolution 6-bits pictures 2) The pictures are transmitted at 6 frames/s.

3) RAC is applied for coding the cartoon images.

4) 3-bit DPCM is applied for coding the grey-level along the contour lines.

picture size lOOx100 64x64 original 360 kbits/s 147 kbits/s cartoon alone 18 kbits/s 7 kbits/s cartoon with black-level fill-in 18 kbits/s 7 kbits/s Grey-level reconstrction 99 34 kbits/s 14 kbits/s erev-level reconstruction 22% j 58 kbits/s 23 kbits/s The data compression obtained is fairly modest, but it must be remembered that the originals are lowresolution 6-bit pictures for which it is much harder to obtain high compression ratios than the 256x256 or 512x512 8-bit originals, sometimes used in the literature.The cartooning process alone produces only a 20:1 compression ratio.

The present invention may also be used for coding moving colour pictures. Figure 14 shows in block diagram form one embodiment of this invention. Camera 11 is a colour video camera that provides the red, green and blue signals (RGB). The frame store (not shown) provides the necessary vertical and horizontal synchronization signals. The red, green and blue signals are then stored on the digital frame store ready for computer processing. The first step in the processing is to derive the signals Y (luminance signal), C1 and C2 (chrominance signals) from the red, green and blue signals, via the transformation matrix 21. The equations for the luminance and chrominance signals are shown below.

Y = 0.299R + 0.587G + 0.144ss (1) C1 =0.69(R-Y)+32 (2) C = 07(B-Y) 32 (3) C1 and C2 are the colour difference signals. The numbers 0.69, 0.57 and 32 have been introduced in equations 2 and 3 to ensure that the quantized signals are represented within 6-bits. The luminance signal Y is coded as explained above. The chrominance signals C1 and C2 are coded by a two step process. The first step was to divide the chrominance signals C1 and C2 into square blocks of size nxn and the average value of each block was computed by the averaging circuit 95. The second step was to DPCM coding the averaged values by the 2-bits DPCM coder 96.

At the receiver, the colour image signals are reconstructed by a three step process. The first step is to reconstruct the luminance signal Y as explained above. The second step is to reconstruct the C1 and C2 signals via the DPCM decoder 97 and the expander circuitry 98. The third step is to regenerate the RGB signals from the Y, C1 and C2 via the retransformation matrix 22. The chrominance signals C1 and C2 are reconstructed to their original size by the expander circuitry 98 and using the statistical distribution of white and black pels in the primitive sketch 2500. The distribution is computed by counting the number of occurance of white or black pels in the corresponding block. Only one counter is needed for this.

Consider one block and suppose for example that the block size is nxn and the number of white pels is m.

Assuming that each white pel has a weight of I, each black pel has a weight of J and the average luminance value in the block is AVG, then: Number of black pels (4) n -m Sum of the units Ixm + J(n- m) (5) Sum of the luminance in the whole block is nxAVG (6) Therefore white pels will be assigned a value of the order nxAVG x I (I x AVG) + J(n- m) (7) And a black pel nxAVG x J (I x m) + J(n - m) (8) Equations 7 and 8 are the bases for the reconstruction of the chrominance signals. Figure 15 shows a 10x10 pels of the primitive sketch 2500. A0 to J9 represents the addresses of the corresponding pels.

Assume n is equal to five; i.e. a chrominance signal divided to blocks of size 5x5. At the transmitter, only four averages are computed and transmitted to the receiver. In the reconstruction process C2, C7, H2, and H7 are reconstructed by simply applying equations 7 and 8 above. Problems arise in reconstructing pels C3-C6, D2-D7, E2-E7, F2-F7, G2-G7 and H3-H6, since the average of the blocks centered on these. pels are not known. The first step in this method is to estimate a value for the corresponding block average and then apply equation 7 or 8. The estimated block average is computed by combining the knowledge of the four neighbouring blocks together with the pels' distribution of the corresponding block, which can be easily derived from the primitive sketch as explained previously.Having reconstructed the Y, C1 and C2 signals the RGB signals can be easily derived via the retransformation matrix 22. The equations for the RGB signals are shown below: R = Y + 1A49(Cl - 32) (9) B = Y t 1.754(C2 - 32) (10) G = 1.704Y - 0.509R - 0.245B (11) Figure 16 (a) shows an original colour image of resolution 256x256 while Figure 16 (b) shows the resulting image by applying the coding method presented here. The block size used for dividing the chrominance signal was of dimensions 5x5.

A comparison may be made between a two-level cartoon as a sampling template (present invention) and the use of contours for segmentation in two-component coding [eg Kunt in IEEE Proc, Rodriguez at PCS and so on]. The principle of two-component coding is to separate the image into regions, then transmit the boundaries (one component) and the interiors (the other component) using appropriate codes. For example, the boundaries may be sent using a chain code, and the interiors as coefficients of a polynomial model of the enclosed luminance surface. The segmentation may be effected by region growing, edge detection or any of a number of other schemes.As described here, in the present invention a two-level cartoon is transmitted for example using Relative Address Coding (RAC), followed by the transission of the actual values of the original at points as determined by the cartoon using DPCM along the cartoon lines.

The most fundamental difference between the two approaches is that two-component coding as conventionally used is firmly based on segmentation ìn image space. The regions produced do sometimes correspond to identifiable regions of the original objects. But in the case of smoothly shaped objects, for example, the human head, the segmentation bears little relationship to the actual geometry of the object. The approach in generating the cartoon was to examine the projection of object features into the image. Thus this treatment of the image is based on the structure of the original scene in three dimensional object space.

Although the second component is merely a sampling of grey levels, the structure of the sampling lattice is determined directly by the structure of the scene i.e.

the segmentation here is based on the physical meaning of the regions in the three dimensional space.

Claims (20)

1. A method for producing a two component video signal corresponding to an image derived from a video camera, in which method the first component is obtained by producing a cartoon Image of the original image and then performing a boundary extraction process on that cartoon image to yield A two-level primitive sketcti of the original image, and the second component is produced by sampling the original image using the primitive sketch as a saw.pllng template, whereby an approximatIon of the Drlglnai image may be reconstructed from said two component video signal.

2. A method accordIng to claim 1, in which the first component comprises a series of addresses defining the primitive sketch and the second component comprises the sampling of the picture elements of the original image at those addresses.

j. A method according to claim 1 or claim 2, in which the address of each sampling point is expressed as a relative address, with respect to the previous address.

4. A method according to any of claims 1 to 3, in which the second component includes information produced by additional sampling of the original image at a number of further points substantially mid-way between the sampling points defined by the primitive sketch, such additional sampling being performed when the distance between said sampling points is greater than some predetermined value.

5. A method according to any of the preceding claims, in which the two-level primitive sketch is produced by obtaining from the original image a series of sketch lines each of one picture element width.

6. A method according to any of the preceding claims, in which the two-level primitive sketch is obtained from the cartoon image by a process including two steps, the first being to determine whether a cartoon feature Is a valley or an edge, and the second being said boundary extraction process to obtain a primitive sketcn having substantially continuous sketch lines.

7. A method according to claim 6, in which the valley or edge determination is performed by sampling the Image in the eight picture elements surrounding a cartoon feature picture element, the decision depending upon the detected signal level in those elements relative to that in the cartoon feature picture element.

8. A method according to claim 7, in which the second component includes a sampling from an adjacent picture element to a sketch feature in the case of the detection of an edge.

9. A method according to any of the preceding claims, in which the original image is scanned line by line, and the number of sketch features on each line, the distance between each two adjacent sketch features and the addresses thereof are computed during the production of the two-level primitive sketch.

10. A method according to any of the preceding claims and adapted for use with a colour video system, in which the second component includes coded luminance signals derived from sampling of the original image.

11. A method according to claim 10, in which the second component further includes coded chrominance signals obtained by sub-sampling the original image.

12. A method according to claim 11, in which the coded chrominance signals are produced by averaging the actual chrominance signals over a block of pre-defined size, the resultant average signals over a block of predefined size, the resultant averaged signals then being subjected to DPCM coding.

1 3. A method of transmitting low resolution images comprising producing a two component video signals in accordance with a method as claimed in any of claims 1 to 12, transmitting the two component video signal over a transmission channel, and the reconstructing an approximation of the original image from the two component video signal.

14. A method according to claim 13, in which the reconstruction is performed by interpolation on the basis of the transmitted two component video signal.

15. A method according to claim 13 or claim 14 and wherein the production of the primitive sketch includes the step of classifying each cartoon feature as an edge or a valley in which method the reconstructed image is enhanced by applying conditional adaptive filtering to the interpolated reconstructed image, on the basis of the edge or valley classification of each primitive sketch feature.

16. A method according to any of claims 13 to 15, in which a decision is taken during reconstruction of the image as to whether a picture element is in the background or in an object portion of the reconstructed image by determining whether the picture element lies within the boundaries defined by the first and last sketch feature points on each horizontal scanning line and by the address of the first sketch feature in each vertical line.

17. A method according to claim 16, in which all picture elements in the background are during reconstruction given a substantially uniform value.

18. A method of transmitting low resolution changes over a relatively narrow band - width channel, substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.

19. Apparatus for transmitting a low resolution video picture over a narrow band-width channel whenever adapted to perform on the basis of a method according to any of claims 1 to 17.

20. Apparatus for transmitting a low resolution video picture over a narrow band-width channel substantially as hereinbefore described with reference to and as illustrated in any of Figures 1, 2 or 1 4 of the accompanying drawings.