GB2163026A - Still picture transmission - Google Patents

Abstract

Before transmitting the full luminance data from store 1, coarse luminance data representing an image consisting only of lines and/or areas of constant luminance is generated 2 and transmitted to provide a fast display at the receiver whilst the full data is being received for subsequent display. Chrominance information may be similarly handled. The invention is suitable for use in photovideotex systems. The line data is obtained by a method of edge detection which involves for each of the two orthogonal directions calculating for each element of the image values representing the luminance gradient in that direction, identifying those elements at which a local maximum of the luminance gradient occurs, and comparing the value of luminance gradient at the local maxima with a threshold value. Those elements at which the threshold is exceeded are designated as edges. <IMAGE>

Description

SPECIFICATION Still picture transmission Whilst televison transmission of moving pictures requires a signal channel of very considerable bandwidth, stili pictures can be transmitted with good definition over much narrower bandwidth paths since a much greater period of time can be made available for transmission of data representing a single frame. Typically this may be a period of seconds. In an interactive environment such as photovideotex, however, the time taken can be irritating for the user. Videotex (or viewdata) is a system whereby a user can all up pages of information from a database over a telephone line or similar link, and photovideotex refers to such a system having the capability of displaying (often as an insert in a page of text) half-tone pictures.Having selected a page (which may be transmitted in two or three seconds) he has to wait rather longer -- say, two minutes-for the half-tone insert, due to its much greater information content.

One proposal for photovideotex transmission is to transmit data corresponding to a transformed version of the image (e.g. Hadamard or Discrete Cosine transforms). This permits certain redundancy reduction techniques to be used: also, by transmitting the low order coefficients first, an extremely coarse-definition half-tone picture can be generated at the receiver relatively quickly and then gradually updated as more data is received. The philosophy behind this is that of transmission of the data forming the final image in a sequence such that a complete, coarse, picture can be reconstructed at an early stage.

Although representing an improvement, the initial picture is extremely coarse and the perceived information content is low.

According to one aspect of the present invention we provide a method of transmitting an image, comprising generating from a set of luminance data representing the brightness of elements of a picture to be transmitted, a set of course luminance data comprising line image data representing an image consisting only of lines and/or data representing areas of constant luminance, transmitting the coarse luminance data followed by the luminance data, receiving the coarse luminance data and displaying the corresponding first image, and receiving the luminance data and displaying in place of the first image a second image represented by the set of luminance data.

The invention involves the recognition that by removing the constraint that the first set of data to be transmitted must be a subset of the data forming the final image, one is free to generate a set of data, using a relatively small number of bits, which is selected to provide a subjectively acceptable picture. The resultant picture is essentially a line drawing representation of the full image which is considered, for many applications to provide the maximum usable information to the viewer for a given information content in terms of the number of bits transmitted; although preferably (or instead) plain areas of white or black can also be transmitted.

The overall increase in the time taken to transmit the final image is believed to be more than offset by the fact that the viewer has an acceptable image to look at in the meantime. Indeed, it may be that in some instances the line drawing image will provide the viewer with all the information he requires about that particular picture and he will not wish to view the full image. For example - in a videotex system - he might interrupt the transmission process by selecting the next page. Although a limited number of greyscale levels could be permitted for the plain areas, preferably they consists of only black and peak white.

Another advantage of the proposal is that reception of the image would be possible with equipment lacking the capability of displaying the full image. Videotex adaptors for home-computers (which generally have a medium resolution pixel graphic capability) are already available commercially, and it would be a relatively simple matter to adapt such an arrangement in most cases merely by modifying the control software (assuming a suitable transmission format).

In a preferred arrangement of the invention, for a colour system, the following method is proposed (i) transmission of the coarse luminance data (ii) transmission of coarse chrominance data (iii) transmission of the luminance data (iv) transmission of the chrominance data At the receiver, (i) the coarse luminance data is received and displayed (ii) the coarse chrominance data is received and displayed simultaneously with (i).

(ii) the luminance data is received and displayed simultaneously with the coarse chrominance colour data (iv) the (full) chrominance data is received and displayed with the "full" image In other aspects, the invention also provides transmission and reception apparatus, and a method of edge detection, as defined in the appended claims.

One embodiment of the invention will now be described, by way of example, with reference to the accompanying drawings, in which: Figure lisa block diagram of a coder according to the invention; Figure 2 depicts a typical display; Figures 3,4 and 4A are vector diagrams illustrating the representation of colour; Figure 5 is a block diagram of a decoder; and Figure 6 is a mask showing the numbering of pixels in an algorithm for calculating the luminance gradient.

Suppose that a half-tone colour picture is to be transmitted. It is assumed that, at the transmitter, the image data is available in digital form in a frame store 1 as luminance (Y) and colour difference (U, V) components. Techniques for conversion of video signals into a digitized single frame are, or course, well known.

For the purposes of illustration, it is assumed that the resolution of the store is 270 x 240 pixels, with 8-bit resolution for the Y-signai and 4 bits each for U and V, although the figures chosen will be selected to suit the particular application. The information content is 128 kbytes, which would take about 14 minutes to transmit over a telephone line at 1.2 Kbit"s (or 331 minutes at 4.8 Bits) although this could be reduced to 2 minutes using transform coding such as described in our European patent application 8330607 [BT ref: 22797] (this and figures quoted below for typical transmission times assume 1.2 Bits).

The first step at the transmitter is to generate a shaded line drawing image, which is done by coder 2, and then to enter thins coded image in a store 3.

The appearance of such a coded image is shown in Figure 2.

Essentially the coder consists of an edge-detector to identify the outlines and a thresholding device to identify the areas having a luminance below a predetermined threshold level. Once identified these outlines and dark areas are represented as black in the coded image.

Techniquesforsuch coding are known, however, conventional methods tend to produce intermittent outlines or outlines of variable width, so a coding technique with improved edge-detection is proposed in which: the pixels are scanned in order to find points at a local maximum in luminance gradient; these points are designated as outline points provided the gradient exceeds a luminancegradient threshold.

In a preferred implementation of this coding technique, the algorithm for calculating the luminance gradient can be best understood by considering those pixels (60) which, in the displayed.

image (61), would fall under a 3 x 3 mask (62) numbered P1-P9 as shown in Figure 6. A first calculation of a luminance gradient Qt is made using the'luminances (p,pg) of pixels (P1-P9) which would fall under the mask if it was in the top left of image where Q1 = P3 + 2P6 + Ps - Pi - 2P - (where indicates absolute value of) Subsequently the mask is shifted one pixel to the right and the same calculation performed to give a value of Q2. The same happens again to calculate Q3.These values of luminance gradient are now compared and if: (03 > TE) and (03 > 0i) and (03 > - that is, if Q2 is greater than a predetermined threshold TE and is a local maximum, then the central pixel (P5) in the mask corresponding to O, is made black in the coded image. Secondly, if: (p5 in the mask for Q2) < T5 that is, if the luminance of the central pixel T5 of the Q2 mask position is less than the threshold then it is made black. If neither of these conditions is satisfied then the central pixel P5 is made white.

The mask is then shifted again, CII becomes equal to the old Q2, 02 becomes equal to the old Q3, a new value for Q3 is calculated and the next pixel is coded.

The mask is moved from left to right across the image, then down a line and across again. This is repeated over the whole image so that all the vertical lines are detected and drawn. Then the horizontal lines are detected by moving the mask from top to bottom and computing: P7 + 2P8 + Ps - P1 - 2P2 - Finally the pixels around the edge of the image frame are all coded white.

It should be understood that.the implementation of this coding technique does not require the mask or any display of the image, but merely an algorithm which operates entirely on the luminance components of the image data stored in the frame store.

The coded image requires only one bit per pixel; following run-length coding this can be reduced to 0.2 bit per pixel, i.e. 26 Kbits, implying a transmission time of 40 seconds. As the image contains, in general, large plain areas of white or black, this an be reduced by standard redundancy reduction techniques (such as variable length coding), to about 20 seconds.

The following description assumes a first stage transmission of the black/white edge information, with information as to large black areas (assuming a nominal white background). However, more detailed (though still coarse by normal standards) area information may be transmitted; e.g. a four level grey scale, requiring (before coding) 2 bits per pixel. With this level of resolution, the edge information could, if desired, be omitted.

The second stage is the generation of colour shading data -viz an overlay image consisting of uniformly coloured areas - in a colour coder 4.

Essentially this involves a coarse quantisation of the chrominance information, thus defining only a small number (say five or six) possible colours.

Consider a U-V diagram as shown in figure 3.

The colour of any pixel can be represented on the diagram by plotting its U and V coordinates. The hue corresponds to the angular position of the point plotted, whilst its saturation is represented by the radius (cf Green (G), Red (R) and Pink (P)). If one disregards the saturation, then the colour of the pixel to be coded can be determined by generating a codeword indicating into which of (say) four sectors (referenced S1 to S5 respectively) corresponding to Red, Yellow, Green, Cyan, Blue or Magenta the point falls. These are the standard colours as used in videotex systems, but the colours chosen are arbitrary: for example, orange could be coded separately, as indicated at S6.

The resultant coloured display will be very stylised, not unlike a comic strip cartoon, and the eye is relatively intolerant to colour errors in such a representation; e.g. the precise colour of the front door of the house shown in figure 2 is probably immaterial to the viewer. In the case of flesh tones, however, this is not so, and the coder described is designed to accommodate this by identifying such tones and to generate a unique codeword which can be used at the receiver to generate a standard flesh tone at the receiver.

The diagram of figure 4 shows the coding areas suggested for such a scheme-viz 000 no colour 001 blue 010 green 011 cyan 100 red 101 magenta 110 yellow 111 flesh tones The U and V components are assumed to be quantised to 4-bit accuracy: i.e. 16 x 16 = 256 possible colours and the diagram of figure 5 is shown divided on a 16 x 16 matrix. Thus the colour coder might consist simply of a 256 x 3-bit read only memory containing appropriate code words read out when the U and V words are applied to its address inputs.

The seven colours shown (plus "no colour") could be coded as indicated above using a 3-bit word, whence the information content would be 270 x 240 = 194,400 bits: approximately 24K-bytes.

As with the luminance signal, known redundancy reduction techniques could be applied.

Although in principle areas having no colour could be identified as areas with low chrominance, in practice it is found to be more satisfactory to regard areas having a luminance value above or below predetermined levels as being respectively white or black, and other areas as being coloured.

Thus, in figure 1 a thresholding device 6 forces '000' (= "no colour") onto the output of the read-only memory 8.

If a limited greyscale representation is permitted by the coarse luminance signal, the colour resolution can be reduced, and, for example, a three-colour (2-bit) scheme is shown in Figure 4A.

Here, the same display colour is made to double for flesh tones and reds, which is found to be subjectively satisfactory.

The colour data is stored in a store 9. Figure 1 also shows encoders 9, 10 for the line drawing and colour data from stores 3 and 18, employing known redundancy reduction techniques such as variable length or run length coding, or by transmitting the pixel coordinates of the points at which the content of the relevantimage changes. An encoder 11 encodes the full detail luminance and chrominance using known techniques such as those described in our above-mentioned patent application. For illustrative purposes the luminance and chrominance outputs are shown separately.

The data content of the four encoded outputs would typically be Line drawing 14 Kbits with (average) redundancy reduction Coarse colour 53 (average) Full luminance 512 K-bits (unencoded) Full chrominance 512 These are fed in sequence to an output interface 12.

Transmission of the picture occurring as follows: (i) transmission of line drawing data from the store.

(ii) transmission of colour data from the store.

(iii) transmission of the Y data from the frame store.

(iv) transmission of the U, V data from the frame store.

It should be noted that, in general the quantity of data generated for (i) and (ii) above will vary according to picture content. Although a fixed transmission time could be used, varying the coding thresholds and hence the degree of picture detail transmitted, this is not necessary and it would be quite possible to allow the transmission time to vary (with, perhaps, an upper limit). The block diagram of a suitable receiver is shown in figureS. Incoming data is received via a telephone line interface 20.

Transmissions (i) ad (ii) are routed via decoding circuitry 21 to a frame store 22. The frame store 22 is assumed identical to that at the transmitter and is connected to conventional readout circuitry 23 driving an RGB video monitor 24.

The decoding circuitry 21 decodes the line drawing signal and writes either peak white or black into the corresponding locations in the Y plane of the frame store 22. The image (of figure 2) appears on the screen of the monitor 24.

A second decoder 25 now receives the coarse colour information, viz the 3-bit colour code for each pixel (or group of consecutive pixels having the same colour), and in response to the code generates U and V values defining the derived "standard" display colour corresponding to that code. Typical display colours are indicated by stars in figure 4.

When these have been written into the frame store 22, the displayed picture appears with coloured areas much as seen in popular comic strips.

The full luminance data is then received, decoded at 26, and written into the Y-plane of the frame store 22, replacing the line drawing data, so that a halftone picture is then seen, but with the stylised colour remaining. To preserve the appearance of the displayed picture during reception and decoding, a buffer store 26 may be interposed, the luminance data being rapidly transferred to the display frame store only when complete. Finally the chrominance information is received, decoded and entered into the U, V planes of the frame store 22 (optionally via buffer 27) to form the final display image.

Another embodiment of the invention envisages that the "full" image may itself be a line drawing or other stylised representation. The drawing might be created in that form, or by the process described above. The first image ("coarse luminance data") would then be a reduced resolution of the same style. This could be crudely obtained by omitting (say) three out of every four picture points in the horizontal and vertical directions, but in practice it may be desirable to adopt a more sophisticated method to avoid loss of important detail. Where runlength coding is used, data reduction in the horizontal direction could be effected by reducing the run-lengths by the appropriate factor, coupled with a suitable correction to accommodate fractional results.

As an aiternative, where the luminance data itself has been generated by an edge detection or other technique from a photographic-quaiity video image, the coarse luminance data might be obtained directly from the original image by using the same technique modified to produce a lower definition result-or the original image might be subjected to subsampling (with or without interpolation) to reduce the number of picture elements in it prior to conversion.

It is envisaged that the first image might be displayed at a reduced scale relative to the second image, so that the pixel resolution per unit area of the display screen is the same for both.

Claims (34)

1. A method of image transmission comprising generating from a set of luminance data representing the brightness of elements of a picture to be transmitted, a set of coarse luminance data comprising line image data representing an image consisting only of lines and/or data representing areas of constant luminance, transmitting the coarse luminance data followed by the luminance data, receiving the coarse luminance data and displaying the corresponding first image, and receiving the luminance data and displaying in place of the first image a second image represented by the set of luminance data.

2. A method according to claim 1 in which the coarse luminance data includes the line image data and represents an image consisting of only two luminance levels.

3. A method according to claim 1 or2further comprising generating from a set of chrominance data a set of coarse chrominance data comprising codes indicating only into which of a reduced number of colour ranges the colour of the corresponding element(s) of the picture falls, transmitting the coarse chrominance data before transmission of the luminance data and the said chrominance data, and, upon reception of the coarse chrominance data, displaying the image represented by the coarse luminance data with areas having displayed colours selected to represent each ofthe said colour ranges.

4. A method according to claim 3 in which one of the colour ranges is that corresponding to the colour of human flesh.

5. A method according to claim 3 or4 comprising transmission of, in sequence: (a) the coarse luminance data (b) the coarse chrominance data (c) the luminance data (d) the chrominance data and, upon reception, display of, in sequence: (i) the image represented by the coarse iuminance data (ii) the image represented by t,he coarse luminance data with areas having displayed colours selected to represent each of the said colour ranges (iii) the image represented bythe luminance data with areas having displayed colours selected to represent each of the said colour ranges (iv) the image represented by the luminance data with displayed colours defined by the chrominance data.

6. A method according to any one of the preceding claims employing redundancy reduction means for encoding one or more of the sets of transmitted data.

7. A method according to any one of the preceding claims in which the line image data is generated from the luminance data by the steps of (a) relative to each of two orthogonal directions of the picture, calculating for each element of the picture values representing the luminance gradient in that direction, and identifying those elements at which a local maximum of luminance gradient occurs; and (b) comparing the values of luminance gradients of the local maxima with a threshold value and coding those elements at which the threshold is exceeded as lines.

8. A method according to claim 7 in which the values representing luminance gradient for an element are calculated as a weighted sum of the luminance values of elements adjacent to that element.

9. A method according to claim 8 in which the two orthogonal directions are designated i and j, the luminance of any element is defined by pi,j and the luminance gradient is given in the first direction by Qh,,i and in the second direction by Qvi,J where Qhi,j = pi-1,j+1 +2pi,j+1 + pi+1,j+1 pi-1,j-1 - 2pi,j-1 - pi+1,j-1 and Qvi,j = pi+1,j-1 + 2pi+1,j + pi+1,j+1 i-1,j-1 - 2pi-1,j - pi-1,j+1.

10. A method according to claim 9 in which an element i,j is identified at a local maximum of luminance gradient in the first direction if Qhi,j > Qhi,j-1 and Ohi,j > Qhi,j+1 and in the second direction if Qvi,j > Qvi-1,j and Qvi,j > Qvi+i,j

11. An apparatus for image transmission comprising input means for receiving a set of luminance data representing the brightness of elements of a picture to be transmitted, means (2) arranged to generate from such data a set of coarse luminance data comprising line image data representing an image consisting only of lines and/ or data representing areas of constant luminance, and transmission means (12) arranged to transmit the coarse luminance data followed by the luminance data.

12. An apparatus according to claim 11 in which the coarse luminance data includes the line image data and represents an image consisting of only two luminance levels.

13. An apparatus according to claim 11 or 12 in which the input means (1) is arranged also to receive a set of chrominance data, and further including means (4) arranged to generate from the set of chrominance data a set of coarse chrominance data comprising codes indicating only into which of a reduced number of colour ranges the colour of the corresponding element(s) of the picture falls, the transmission means (12) being arranged to transmit the coarse chrominance data before transmission of the luminance data and the said chrominance data.

14. An apparatus according to claim 13 in which one of the colour ranges is that corresponding to the colour of human flesh.

15. An apparatus according to claim 13 or 14 in which the transmission means is arranged to transmit, in sequence: (a) the coarse luminance data (b) the coarse chrominance data (c) the luminance data (d) the chrominance data.

16. An apparatus according to any one of claims 11 to 15 including redundancy reduction means for encoding one or more of the sets of transmitted data.

17. An apparatus according to claim 11 or 12 in which the means arranged to generate a set of coarse luminance data comprises means for calculating, relative to each of two orthogonal directions of the picture, for each element of the picture values representing the luminance gradient in that direction; menas for identifying, relative to each of the respective directions those elements at which a local maximum of luminance gradient occurs; means for comparing the values of luminance gradients of the local maxima with a threshold value; and means for coding those elements at which the threshold is exceeded as lines.

18. An apparatus according to claim 17 in which the values representing the luminance gradient for an element are calculated as a weighted sum of luminance values of elements adjacent to that element.

19. An apparatus according to claim 18 in which the wo orthogonal directions are designated i and j, the luminance of any element is defined by pi,j the luminance gradient is given in the first direction by Ohi,j and in the second direction by Qvi,j where Qhi,j = pi-1,j+1 +2pi,j+1 + pi+1,j+1 - pi-1,j-1 2pi,j-1 - pi+1,j-1 and Qvi,j = pi+1,j-1 + 2pi+1,j + pi+1,j+1 pi-1,j-1 - 2pi-1,j - pi-1,j+1.

20. An apparatus according to claim 19 in which an element pij is identified at a local maximum of luminance gradient in the first direction if Qhi,j > Qhi,j-1 and Qhi,j > Qhi,j+1 andin the second direction if Qvi,j > Qvi-1,j and Ovi,j > Qvi+i,j

21. A receiving apparatus for use with a transmitter according to any one of claims 11 to 20 with means for receiving the the coarse luminance data and displaying the corresponding first image, and means for receiving the luminance data and displaying in place of the first image a second image represented by the set of luminance data.

22. An apparatus according to claim 21 when dependent on claim 13 with means for reception of the coarse chrominance data and displaying the image represented by the coarse luminance data with areas having displayed colours selected to represent each of the said colour ranges.

23. An apparatus according to claim 22 in which one of the displayed colours is that corresponding to the colour of human flesh.

24. An apparatus according to claim 22 or 23 arranged to display in sequence: (i) the image represented by the coarse luminance data (ii) the image represented by the coarse luminance data with areas having displayed colours selected to represent each of the said colour ranges (iii) the image represented by the luminance data with areas having displayed colours selected to represent each of the said colour ranges (iv) the image represented by the luminance data with displayed colours defined by the chrominance data.

25. A method of edge detection for use in processing data representing an image comprising (a) relative to each of two orthogonal directions; calculating for each element of the image values representing the luminance gradient in that direction; identifying those elements at which a local maximum of luminance gradient occurs; and (b) comparing the value of luminance gradient at the local maxima with a threshold value and designating those elements at which the threshold is exceeded as edges.

26. A method according to claim 25 in which the values representing luminance gradient for an element are calculated as a weighted sum of the luminance values of elements adjacent to that element.

27. A method according to claim 26 in which the two orthogonal directions are designated i and j, the luminance of any element is defined by pi,j and the luminance gradient is given in the first direction by Ohi,j and in the second direction by Ovi,j where Qhi,j = pi-l,j+l +2pi,j+1 + pi+1,j+1 pi-1,j-1 - 2pi,j-1 - pi+1,j-1 and Ovi,j = pi+1,j-1 + 2pi+1,j + pi+1,j+1 - pi-1,j-1 - 2pi-1,j - pi-1,j+1.

28. A method according to claim 27 in which an element i,j is identified at a local maximum of luminance gradient in the first direction if Qhi,j > Qhi,j-1 and Qhi,j > Qhi,j+1 and in the second direction if Qvi,j > Qvi-1,j and Ovi,j > Qvi+i,j

29. A method of image transmission substantially as herein described with reference to the accompanying drawings.

30. An apparatus for image transmission substantially as herein described with reference to the accompanying drawings.

31. A method of edge detection substantially as herein described with reference to the accompanying drawings.

32. A method of image transmission comprising generating a set of luminance data representing elements of a picture to be transmitted and a set of coarse luminance data having a lower information content and representing a less detailed representation of the picture, transmitting the coarse luminance data followed by the luminance data, receiving the coarse luminance data and displaying the corresponding first image, and receiving the luminance data and displaying in place of the first image a second image represented by the set of luminance data.

33. A method according to claim 32 in which the set of luminance data and the set of coarse luminance data each comprise line image data representing an image consisting only of lines andl or data representing areas of constant luminance.

34. A method according to claim 33 in which the first image is displayed at a reduced scale relative to the second image.