CA2176350A1 - Voice and image telecommunications apparatus - Google Patents

Abstract

Voice and image telecommunications apparatus (2) which is connectable to an analogue telephone network and which enables at least first and second persons with first and second visual display units at first and second remote locations to have a speech conversation while viewing and being able to manipulate in an interactive manner an image which is simultaneously seen by the first person at the first location and by the second person at the second location. The apparatus (2) has first connection means for the first person, second connection means for the second person, image data compression and decompression means (14), speech digitizing means (6), speech data compression and decompression means (12), combining means (16) for combining the compressed image data and the compressed speech data into a single integrated data stream, convertor means (22) for converting the single integrated data stream into analogue form suitable for transmission over the analogue telephone network, transmit and receive means (28, 30) for the first person; transmit and receive means for the second person: and control means for distributing authority over the apparatus (2) in order to ensure that only one person at a time has the ability to manipulate the dynamically changing shared image on all visual display units in operation.

Description

wo ss/136sa . . ~ L~ 15 ~7fi35~

VOICE AND IMAGE TELECOMMUNICATIONS APPARATUS
This invention relates to voice and image telecommunications apparatus.
Visual display units are well known and they are widely used for displaying i"rul " ~c~liul) such for example as words and/or images on a screen. An ima~e can be displayed on the screen of diflerent Yisual display units at remote locations but problems anse when persons operating these diflerent visual display units wish to discuss and alter the displayed image. More specifically, known telecommunications apparatus does not allow for the simultaneous alteration in real-time of the shared ima3e between persons in the same free and easy manner as the image may be discussed by the persons.
It is an aim of the present invention to obviate or reduce the above mentioned problem.
Accordin~ly, in one non-limiting ~" ILlu~il "~:, IL of the invention there is provided voice 3nd image telecommunications apparatus which is c~" " I~,LiL,l~ to an analogue telephone network and which enables at least flrst and second persons with first and second visual display units at first and second remote locations to have a speech conversation whilst viewin~ and being able to manipulate in an interactive manner an image which is simultaneously seen by the first person at the finst location and by the second p~rson at the second location, which apparatus comprises, i) first connection means which is for the first person and which is for providing a connection to the ana~o~ue telephone network and to a first visual display unit to be used by the first person;
SUBSTITUTE SHEET (RULE 26) _ . . . . . . . . .. . ..

WO 9~i/13680 }~,1,. 1!0244 21763So 2 ii) second connection means which is for the second person and which is for providing a conneGtion to the analogue telephone network and to a second visual display unit to be used by the second person;
iii) image data GO~ DSjUII and de~UlllUlt:bbiUIl means forco,,,~,~w;,,G and de-Gu,, ,,u, wwil ,~ image data;
iv) speech digitising means for converting speech into digitised speech data;
v) speech data cu" I,UI ~ iUI I and de~G~" ,,u, cwiOI ~ means for w" ,,u, ~w" ,!J and de~u" ,,u, ~Zbbil ISi the speech data;
vi) combininSi means for combining the cullllul~b:~ed image data and the ~.ulll,ul~bs~d speech data into a single integrated data stream;
vii) convertor means for convertinbi the sin~le integrated stream into analogue form suitable for ~lal 1~l l libl:~iUIl over the analogue telephone net~vork;
viii) transmit and receive means for the first person;
ix) transmit and receive means for the r,econd person; and x) control means for distributing authority over the voice and image teiecommunications apparatus in order to ensure that oniy one person at a time has the ability to manipulate the dynamically changing shared image on SU~STITUTE SHEET ~RULE 26) WO 95/13G80 ~ 1 7 ~ 3 ~ O . ., ~ ~ LM'~ 15 all visual display units in operation during operation of the voice and image telecommumcstions apparatus.
It will be a~ Jlb~ ,;i that the apparatus of the present invention enables the image to be aitered in an interactive manner by all pr~rsons paRy to an operative group studying the image. The image can be altered at will by different persons substantially in the same free and easy manner as they can discuss the image at will, with the proviso that whilst all the persons in the paRy can alter the image, oniy one person at a time can do so in order to prevent two or more persons alteRn~ the image at any one time and thereby causing confusion.
Preferably, the apparatus is one in whioh the manipulation of the ima3e is possible in the following two ways:
a) the oRginal data can be altered which then changes the displayed image, or;
b~ lines and markinas can be drawn as an overiay to highlight paRs of the image which has no effect on the originai data.
Preferabiy, the apparatus is one in which the control means is such as to allow cu, ",u, ~ed ~peech data signais from the ~Irst person and from the second person to pass simuitaneousiy in both directions in order to permit simultaneous speechbetween the first and the second persons, in which the controi means is such as to aiiowcu,,,u,~dimagedatasignalsfromthefirstpersonandfromthesecond person to pass one at a time in both directions ir - ;~r to permit only one person at a iime to have the ability to manipuiate the image; and in which'the control means is such as to permrt the transfer of the abiiity to manipulate the image from person to person. Preferabiy, the apparatus is one in which there are four data streams consisting of digitised and Cu~,ul~bSts i voice data, processed and ~.ulll,ul~aeLI
Si'~T~iTE SHEET (Rl~iLE 2iS~
,, _ _ . . . . . . . . . ..... . _ . . .. .

WO 95/13680 , ~ .. ,5 1 '^~445 ~7635~

image or bitmap data, vector data which communicates the position of the cursor on the screen, and systems or wntrot data which conveys instnuctions to the receivin3 screen to draw a line or rectangle For example, which are inte3rated into one combined data stream.
The apparatus may be one in which the Flrst and the second transmit and receive means are each such as to send only an ima3e as seen at the moment of disp~ay on the visual display unit, and in which the apparatus is such that the first and the second transmit and receive means receive what has been llal IDI "iilt:d but with the control means giving the person 1,~, IDI "il~i"~ control over a screen of the visual display unit of the person receiving and with the control means permitting the transfer of the ability to transmit from person to person In this case, the first and the second transmit and receive means may each include a read only memory for sendin3 instructions, a random access memory for extractin3 visual display unit screen dah, and at least one ,,,;~,,up,uc~ ,. Where two ,,,i~,,u,u,u~ re employed, then one " ~ u~u~ DDor may be for speech data cu" I,ule~Siûl I and theother Illi~lU,UIUCebDUI may be for speech and ima~e data inte3ration In an altemative t:" IJUdil 11~:1 It ûf the invention, the first and the second transmit and receive means are each such as to transmit the interactive manipulation of llrll IDI, litl~d data.
The first and the second transmit and receive means may be such that they each comprise a separate transmitter and a separate receiver, ûr they may alternatively each comprise a transceiver S~JBSTIME SHEET (RULE 26 W095/1368~ 21753~ r "~
~ " I ~ .
s Advantageously, the image data cu" "~ means i8 a I lielal ~ al image data cu",,u,eDDi"u means which _u",,u,~ v image data in a Ilielal~ al order of ,uul Lal l_e for allowing 1, al IDI, I;DDi_ll to a visual display unit in a similar 1.~ dl order of ;" ,,uu, lal l_e thereby to enable the build up of an image in a layered manner.
This enables the provision of an image on the second visual display unit which is on the first visual display unit and which has not lost quality. The I lielal ~
L, al IDI, IIDDiUI, enables first the outline of the imar~e to be formed and then the colour i"~ " "~ " to be inserted as a result oF the coarse detail of the ima~e being llal IDI "iL~e! first and then the hner detail il ,ru, 1 l laliul I thus providing the successive build up of the image. The apparatus may thus be one in which the image data culllluleDbill~meansisallielalvllivalimagedatacvlllluleaDill9meanswhich C~ JleDDeD the image data in a shape orientated build process which obtains the ~pp~uA;"~aLeoutlineblockcolour~e~eDe~,latiu~oftheimagebyassessingobject ske and shape. The technique employed may include shape and teAt recognition, shape repeat, filters and line a,u,ul UA;I I la~iui Is. The outline plus fill colour block colour Laliul I may o~ten fully descnbe the image but, where it does not, further build up techniques may be applied which include major gradients, teAture simulation and repeat, secondary outlines and fill.
Thellielalvllivv:imagedatacvlll~uleDD~ meansmayprovideawindowoffinitetime duration for the total bandwidth to be devoted to the L, al IDI, ,._~;v,, of image data to allow at least sufficient image data to be Llal IDllli~led for a le_uyl liD_L le image to be displayed by the receiving display unit, before speech interrupts. The window of finite time duration may be of any suitable and ~i~u,ulu~uli~ time duration such for example as 3.5 seconds.

The ima~e data may t~e transferred in the following hielal~ al stages:
SU4STITUTE SHEET (RULE 26) ,,, _ , ~ _ . _ .. _, _ ... _ .. . _ .. _ .. . _ .. ... ..... ..... _ .. _ _ WO95/13680 ~ ~17l~35~ ^7~
~i) image outline;
(ii) fill colour;
(iii) refined colour;
(iv) refined image; and (v) repeat stages (iii) and (iv) until the required imaae i8 obtained.
The image data is preferably transferred in the following l ,i~ al stages:
(i) oolour plates providing coarse shape c~u,ulu~ ,s (ii) smaller shapes derived from a codebook (iii) refinement of image detail from colour plates of narrower colour value vanations (iv) refinement of the image using major gradients, texture simulation and repeat secondary outlines and fill techniques (v) repeat stages (iii) and (iv) until the required image is obtained.
The apparatus of the invention may include dynamic partitionin3 means for producing data packets in order to allow the intecJration of the data in a dynamic manner to provide for the elficient 1, c,, ,~" ,is~iùl, of a data streâm over the available bandwidth.
The apparatus with the dynamic partitioning means may also include data packet shrinking means for shrinking the data packets as the line quality changes or drops in quality or data packet size changing means for changing the size of the data packets as the line quality chan~es in order to minimise the impact of all data lost since lost visual data can be re~ "ll~ d but lost speech data is ~t:""~"e,llly lost.
SUBSTIIUTE SHEET (RULE 26) wo ss/13680 ~ 1 7 ~ 3 ~ 0 The apparatus may further include dynamic data packet sizins means for dynamically sizin3 the data packets as line quality changes in order to minimise on theinputofdatathatiseitherlostorthatrequirere-~,a,,b,,,ibbiu,,, andinorderto maximise on the l,a,,b,,,isbiu~l of data. Such apparatus enables the optimum Ll~l Ibl I libbiUI I of data.
The apparatus of the invention may have the followin~ four data streams which have to be integrated;
a) voice data - diaitised and culll,ult:ss~d b~ imase or bitmap data (CUlll,Ult~b~) c) vector data - the position of the cursor on the screen d) systems or control data - in6tnuctions to the receiving screen to draw a line or rectan~le for example.
Usually, streams b, c and d ane first combined into a sinsle stream and than a difference processor combines the `combined' data stream and the voice data into a sinsle data stream. It is necessary to infonm the receiving computer of the nature andamountofthedata beins l,~,,b,,,ilL~:~iandthati,,fu,,,...`iu,, iscontained in bytes in data packet `headers' and `enders'.
It is desirable to decrease the ~ , llay~ of overhead as a proportion of the data bein~q llal Ibl I Idl~i to as low a level as possible. The overhead is always a fixed amount and so in conditions of 300d line quality, the data packet would be as lons as possible to minimise that ,~,~,~, ,Ic,u~. As qualr~y drops however, there will be a requirement to minimise the damase of lost data. Voice ~s lost forever but imase, vector and control dâta is re-~, 51 ~Sl I ~i~led. It is better to re-transmit small amounts SLIBSTITUTE SHEET (RULE 261 WO 95/~3680 ~ . , 1 '0244--21~ ~ia O

more often than large amounts because of the ttme delay and syl i~,l 1, u, li~dii problems. It is thus possible to adjust the data packet size.
The apparatus oF the invention may include monitor means for monitoring for a gap in the lla~ 1~l l ,a~ speech for the insertion of extra ima~e data.
An el, llx~il, lel ,l of the invention will now be described solely by way of ex~dmple and with reference to accompanying drawings in which:
Figure 1 is a block diagram of voice and image telecommunications apparatus;
Figure 2 is a block diagram of a printed circuit board for instal~ation into a visual display unit for converting the visual display unit into voice and imaj3e telecommunications apparatus; and Figure 3 is a block circuit diagram similar to that shown in Figure 2.
Referring to Fi3ure 1, there is shown voice and imagê telecommunications apparatus 2 which is col ,l It~ilible to an analo3ue telephone network and whichenables at ieast first and second persons with flrst and sêcond visual display units at frst and second remo~e locations to have a speech conversation whilst viewing and being able to manipulate in an interactive manner an image which is seen by the first person at the first location and by the second person at the second location.
The apparatus 2 comprises data capture means 4 for capturing i"rul 1, ,~iu" from a telephone line and/or the visual dispiay units.
The data capture means 4 is connected to speech digitising means 6 via input lines 8. The speech digitisin~ means 6 is for converting speech into digitised speech data. The speech diaitisinj3 means 6 is connected by input lines 10 to speech data S'JBSTITUTE SHEET (~ULE 2iE) WO 9~113680 ' 2 ~ 7 ~ 3 ~ d r~
cur",u.~;;J" means 12 and a~so to image data cu,.,~ J,~ means 14. The dual speech w",u,.:s;,iv" means 12 and the image data cU"l,u,~s~,u,. means 14 are connected to combining means 16 by lines 18. The combining means 16 combines the cul ",u~ d image data and the cul l l,u, ,l,~d speech data into a single integrated data stream which passes along line 20 to convertor means 22. The convertor means 22 is for converting the single integrated data stream passing alona line 20 into analogue fomm suitable for 1, al~Sl, ,i~,~iu,, over the analogue relephone network. The analogue form of the data may then be pumped by data pumping means 24 along line 26 to receiver means 28. It will thus be a,u,~l~;cK~tllat apparatus 4, 6, 12, 14, 16, 22 Forms llal lal I litL~I means 30 when data is being sent to the receiver means 28.
The apparatus 2 is such that the receiver means 28 can also operate as transmitter means, in which case the transmitter means 30 will operate as receiver means 30.This is achieved by control means (not shown) for distnbuting authority over theapparatus 2 in order to ensure that only one person at a time has the ability tomanipu~ate the image an all visual display units in operation dunng operation of the apparatus 2. More specifically, the control means may be such as to allow c(." ,~ ,ed speech data signals from the first person and from the second personto pass simuKaneously in both directions in order to pemmit simuKaneous speech bet~veen the first and the second persons. The control is i~ pt l ,.1.:"l of thellal l~ ll~iol1 of speech and image data. That is the first person may be llal ,~" ,::'i"g speech and image data to the second person but the second person may be exercising authority over the screen of the first person at the same time. The control means may also be such as to allow combined data stream (containing speech, image and systems data) to pass simultaneously in both directions. It is only the vector data, which is control~ed by systems Cul "" la~ , which can be sent in only SU~S~ITUTE SHEET (RULE 26) WO 95/13680 r~ .'0~44~
~171~350 one direction at a time. The control means will also be such as to permit the transfer of the abiiity to manipulate the image from person to person so that all persons pariy to an audiolvisual discussion haYe the facility of making an audio and a visual ccntribution to the discussion. Usually, an image sent will only be the image as seen by other persons on their visual display units.
The apparatus of the present invention, for example as illustrated generally with reference to Figure 1 can be provided as a first version which combines voice and screen data to enable each user to share the same image and manipulate that image in an interactive manner with a voice contribution being possible.
In an alternative ~:" ,L,~i" l~l ,l of the invention, the apparatus of the invention may be such that it does not include the voice L,~, IDI ~ liDDiUI 1. The apparatus of the invention will then allow the interactive manipulation of the image on the visual display units but the required vcice discussion will be over normal analogue telephone lines.
In the first above mentioned ~"~L,odi",~"l of the invention where the apparatus of the invention provides both voice and ima3e discussion facilities, then the apparatus may be comprise apparatus as shown in Fi~ure 2. Figure 2 shows a printed circuitboard 32 comprising a ~ u~u.,esDu~ 34, a speech ~ ,u.,e~sur inter~ace 36, a voice Digdal Signal Processor ~,u" Iplt:DDiUI I device 38, a random access memory 4û, a diaital to analogue converter 42, an analogue to digital converter 44, and an automatic gain control 46. Items 42, 44 and 46 have been combined into a single codec which performs these three functions. The single codec is connected to a telephone handset via a digital path 50 to a telecommunication socket 54. A modem 56 is connected to communications software 58 which is in tum connected to a Digital Access Al, el, Iyt l "t:"L (DAA) line protector 60, which is in tum connected to SUBSTiTUTE SHEET (i~ULE 2i) 1~ WO95/13C80 A~ /P?1~r 21i763~

the telecommunications socket 54. A read only memory 62 and a random access memory64 are connected as shown to the I l li-il Upl Ul~_~JI 34.
Refening now to Figure 3, similar parts as in Figure 2 have been given the same reference numerals for ease of Gu~ I l,ua~ and ul ~d~ al l iil l~. Figure 3 illustrates hrJw data ~,v" I~ul eD.?iUI, of screen data can be in a host computer or in the printed circuit board 33 at position D2.
The apparatus of the present invention i~ able to operate such thr~t di~itised audio si~nals are packaged into packets and integrated with picture, drawing, numerical or textual data, layouts and other computer held or generated i"ful " lallul~, as well as ima~es that have been scanned in or captured by image grabbing devices into a single data stream, muitiplexed and distributed at the reGeivin~ end. All changes made to the screen data on either visual display unit are echoed on the screen of the other visual dispiay unit in real time. Whilst talking, authority over both screens can be negotiated between the parties, thus enabling a document to be discussed and edited between the users as a collaborative eflort, each user heanng and seeing the actions of the other.
The apparatus of the invention may use software pemmr~ting the capture of a screen of data from a non-windows application which is then converted into a bitmapped image format which can then be imported into windows software.
The apparatus of the present invention may provide users with graphic tools to assist the manipulation and annotation of screen data. All added data overlays the original and can be removed in layers. The data added by each party is preferably in their own unique colour providing an audit trail of which par~y has added what SUBSTITUTE SHEET (RULE 26) WO 95/13680 r~

2~ 7 63~;!0 annotation. Screens can at any time be saved by erther party for later printing or display by either party.
Where it is desired to offer the apparstus of the invention without the voice facility, such as when for example, connection is to a local area network then the telephone handset, and the voice conversion and processing circuitry and the high bit ratemodem will not be required. The apparatus will then be such as to hook up and tocommunicate over a local area network for the llal ,b",i~,biu" and interactive manipulation of screen data and the distribution of the voice will be over the existing telephone network.
During operation of the apparatus 2 the data capture means 4 are used to generate a stream of data. Two devices may be used, namely a telephone handset and a computer. It will be dp~ that the `telephone handset' could be any u~l 1011~ that genenates an analogue signal and any speaker that produces sound from an analogue signal such as a telephone handset, and the `computer' could be extended to `any device that digitally stores, processes and displays data such as a personal computer.
The computer will be the computer from which ima3e data is captured. The image data is obtained from the video memory and input data is obtained from the videomemory andtor keyboard buffer which can contain keyboard inputs andtor mouse inputs or inputs from other pointing devices. The screen image can be modified and the changes captured from the video memory or the keyboard buffer as they occur.The screen image can be a display of an image file or can be the screen display of an active application. In both modes, the sending and the receiving screens are slaved and authority over both screens can be traded back and forth using a system SUBSTITUTE SHEET (RULE 26) ~ wo 9Y1368~ r 1 15 23~7~5~

or and agreed protocol, thus providin~q the users with interactive simultaneous voice and visual communications. Ima~es can be modified or overlain with drawn anl lUtelliOI Ib, and ,), " ' . ,s can be operated and up-dated in a co-operative and interactive manner in real time. Screen data can be bitmap or alpha-numeric in character.
In a possible ", "~ ' , of the apparatus, more than two devices may be inputtin~information. The apparatus may include a moving or still camera with digital or analogue input, a scanner or other electronic devices 3enerating a data stream.
The data captured by the data capture means 4 passes alon~q the lines 8 as shownin Figure 1. Each data capture means 4 generates a data stream. Two or more datacapture means 4 may be employed.
The speech di~itising means 6 may be a " ,iu, uul1ul ,e wnich transmits the speech signal to the printed circuit board board as an analogue sianal. The si~nal may be first augmented by the automatic gain control 46 to stabilise its energy level and a signal may then be converted to a digital stream usin~ the analoaue to digrtal converter chip 44 . In a possible future l l l ' ~ ' I of the invention, the input si~qnal may not require conversion.
The conversion process at the sendino end is reversed at the receiving end by passing the di~ital speeoh stream throu3h the speech cu, I I~ b~iUI I and de-Cul l I,UI elbbiUI I module 43 and then through the digital to analosue converter chip 42.
Conversion of the voice signal is not required in the non-speech version. The data captured from the video memory is a di3ital data stream and does not require conversion.
SUBSTITUTE SHE~T (RIJLE 26) WO95/13680 ~ J4--35~

The data cu" I,uleDDiul, is ef~ected such that the input signal or signals are reduced in ske so that when they are combined into a singie data stream they do not exceed the bandwidth capacity of the transport medium. For example, a ~Irst data streammay be a digitised 6peech data stream which is sent to the speech ~UllI,Ult:DDiUIl module which consists o~ a di~ital signal processor and supporting memory chips.The data stream is C~JIIl,ult:DDdd from 64,000 bits per second to good telephonequality produced at a rate o~ 6,80û bits per second or to 4,80û bits per second depending on the bandwidth that the llal IDllliDDiUI~ medium is able to support. The quality is that known as good telephone quality, that is good enough to be easily understood and to be able to be reco6nise who is speaking.
A second data stream may be data captured from the video memory which is processed and ~ull l~ aaed in a l lit:lal~ al manner so that the critical data is extracted and sulhcient data is sent to a receiving screen within a window of hnite duration, ~or example 3.5 seconds. At the receiving screen, the whole of the bandwidth is reserved for the llal IDI I liDD;~I I of the screen data. During the hnite window, an image is constructed on the receiving screen that accurately emulatesthe ima~e on the sending screen, this image being sufficient to enable a meaningful voice/visual ~u,, . ~, aaLiul I to take place. After the expiration of the time duration of the window, the voice channel re-opens for the llal~Dl l liDDiUI I of a combined voice and image data stream allowing voice communication and the continuing improvement and refinement of the Llal IDII liLL~d image.
The same approach is used if the apparatus of the invention is used without the speech facility but, because the available bandwidth would be greater because there would not be the need to share it with another (voice) data stream. The llal IDI I IlDDiOI I
SUBSlITUrE SHEET (RULE 26~

WO 95/13680 P~ ,, 3 1 ~A? ~ ~
~1~6350 of a meaningful image would take less time and the interactive manipulation of shared data could GUI 1111 lel ,~,e in less that the finite time of the window, for example the above mentioned 3.5 ~econd window. Where there is not a voice element to theapparatus of the invention, the end G~f the finite window would allow for vector and systems ~,u, "" lal IJD to be integrated with ima~e data and sent to the receivin~
screen With regard to the data stream known as data stream n, then depending upon the nature of the data bein6 captured and the bandwidth both available and required,the data may be ~,,u~,lu,u,idl~:ly processed and Gu~lluleDDéd The speech data cu" ,,ui eDSi~ ~8 means 12 and the ima3e data cu" ,,u, eDDiUI~ means 14 may be such that data from data stream one and data from data stn~am n are packaged into packets of n bits. The data packet is flag~ed both front and back with infonmation indicatina that:
a) what Follows will be a speech or image, vector andlor control data packet of n length and, b) what went before was a speech or image, vector and/or control data packet of n len~th.
Packet sizes are dynamically adjusted dependin~ on the line quality as chd, a~leriaed by the data rejection rates As the flag overhead is a flxed amount, it is therefore a hi~her proportion of a small packet than it is of a big,packet The bi~er the packet, the more il~Fu~llaLiull data is lost in the event of an ~allDI~isDiul~ fault or enor but the more data can be sent in a given time period. A balance point is desired between the speed of 1, al 1~ iun and the amount of data lost in the event of an enor.
SUBSTITUTE SHEET (RULE 26) .. .... ... . . .. . . . _ .

WO 95/13680 A ~ . 44--~17~350 , .
Where the apparatus of the invention offers both the speech and the image facility, the combined data may be Lla~ l~l to a similarly equipped temminal over the existing public service telephone network in analogue format. The nominal bandwidth is 9,600 bits per sewond but the bandwidth may range from 2,400 bits per second to 28,800 bits per second. V.32bis modem communications has become sufficiently stable that if a connection can support 2,400bps in the older V.22 standard, it can sustain 9,600bps using the V.32bis standard. We will also be compatible with the new V. FAST standard enabling data to be transferred at a rate of 28,800bps. Our effective bandwidth ranges then from 9~600 to 28,800bps. The whole of the available bandwidth may be dedicated to the 1, al IDI 11;5~ of the scneen image for the first finite period afier which the bandwidth is shared by bc~th the voice and the image data. After w,"~ ", the voice data occupies 6,800 (or 5,200) bits per second and the screen image, vector and systems data and the data IJd_h~ overhead occupies the balance. The combined data stream is dynamically partitioned in that, in periods of voice silence, the lack of a voice signal is detected and the bandwidth no longer oocupied with speech is given over to the ~l al 1~ tiUI I of screen data if required. When speech data is detected, the bandwidth is again se~ u_t~l between image and speech data reserving 6,800 (or 5,200) bits per second for speech.
The data pumping means 24 is effective to pump the combined single stream of data to the other visual display unit over the analogue telephone network. The data must therefore be wonverted to an analo~ue signal for ~lal 1~ ;ul I by an analogue data 1, c, ,~" ~iul, pump which may take the form of a modem. Modifled versions o~
the apparatus ûf the invention may enable the 1, al 1l~l l ,i~,siul, of digital data over the integrated service digital network and may therefore not requine wonversion.
SLI~STltUTE SHEET (~_LE 26) WO 95/13680 P~, I, .,, 1.'î ? 1 1~
~7~3~

-When the apparatus of the invention is connected to a local area network, as described earli~r, the data does not require conversion and the speech data conversion means 43 and the data pumping means 24 is removed.
Conversion to an external medium would not be required by the apparatus shown inthe drawin~s.
As indic~ above, the ~,~v~u, u~v, ialt data stream is pumped by the data pumpin~means 24 over the analogue or di3ital network telephone network. The ~la~ iu Cul 1.'~ ;VI~b confomm to i"Lt" "aliul ,al CCITT standards thus enabling the printed circuit boards to also function as a standard file transFer modem l~,vu"-li"g toHayes compatible AT ~u, "" lal Iv~ and as a Group 3, Class I and 11 facsimile device.
If data is pumped over the digital (ISDN) network, the apparatus of the present invention will support Group 4 facsimile standards.
In the modffled version of the invention where speech is not provided for, then a data pump as herein before descnbed would not be required.
It is to be aju~ul t:.,idL~.I that the ~" ,L,odi" 1~ of the invention described with reFerence to tlle accompanying drawings has been given by way of example only and that ". '`- ' ~n:, may be effected. Thus, For example, the apparatus of the inventionmay be used by any suitable and aupl u,u, iale~ number of persons.
SUBSrlME SHEET (RULE 26)

Claims (15)

1. Voice and image communications apparatus which is connectible to a telephone network and which enables at least first and second persons with first and second visual display units at first and second remote locations to have a speech conversation whilst viewing and being able to manipulate in an interactive manner the same or a shared image which is seen by the first person at the first location and by the second person at the second location which apparatus comprises:

(i) first connection means which is for the first person and which is for providing a connection to the analogue telephone network and to a first visual display unit to be used by the first person;

(ii) second connection means which is for the second person and which is for providing a connection to the analogue telephone network and to a second visual display unit to be used by the second person, (iii) image data compression and de-compression means for compressing and de-compressing image data;

(iv) speech digitising means for converting speech into digitised speech data;

(v) speech data compression and de-compression means for compressing and de-compressing the speech data;

(vi) combining means for combining the compressed image data and the compressed speech data into a single integrated data stream;

(vii) convertor means for converting the single integrated stream into analogue form suitable for transmission over the analogue telephone network;
(viii) transmit and receive means for the first person;
(ix) transmit and receive means for the second person; and (x) control means for distributing authority over the voice and image telecommunications apparatus in order to ensure that only one person at a time has the ability to manipulate the dynamically changing shared image on all visual display units in operation during operation of the voice and image telecommunications apparatus.

2. Apparatus according to claim 1 in which the control means is such as to allow compressed speech data signals from the first person and from the second person to pass simultaneously in both directions in order to permit simultaneousspeech between the first and the second persons, in which the control means is such as to allow compressed image data signals from the first person and from the second person to pass one at a time in both directions in order to permit only one person at a time to have the ability to manipulate the image; and in which the control means is such as to permit the transfer of the ability to manipulate the image from person to person.

3. Apparatus according to claim 2 in which there are four data streams consisting of digitised and compressed voice data, processed and compressed image or bitmap data, vector data which communicates the position of the cursor on the screen, and systems or control data which conveys instructions to the receiving screen to draw a line or rectangle for example, which are integrated into one combined data stream.

4. Apparatus according to any one of the preceding claims in which the first and the second transmit and receive means are each such as to send only an image as seen at the moment of display on the visual display unit, and in which the apparatus is such that the first and the second transmit and receive means receive what has been transmitted but with the control means giving the person transmitting control over a screen of the visual display unit of the person receiving and with the control means permitting the transfer of the ability to transmit from person to person.

5. Apparatus according to claim 4 in which the first and the second transmit and receive means each include a read only memory for sending instructions, a random access memory for extracting visual display unit screen data, a microprocessor for speech data compression and a microprocessor for speech and image data integration.

6. Apparatus according to any one of claims 1 to 3 in which the first and the second transmit and receive means are each such as to transmit the interactive manipulation of transmitted data.

7. Apparatus according to any one of the preceding claims in which the image data compressing means is a hierarchical image data compressing means which compresses the image data in a hierarchical manner in order of importance for allowing transmission to a visual display unit in a similar hierarchical order of importance thereby to enable the build up of an image in a layered manner.

8. Apparatus according to claim 7 in which the hierarchical image data compressing means provides a window of finite time duration to allow at least sufficient image data to be transmitted for a recognizable image to be displayed by the receiving visual display unit, before speech interrupts.

9. Apparatus according to claim 7 or claim 8 in which the image data is processed, transmitting and re-constituted on the receiving visual display unit in a shape orientated build up process which obtains the approximate block colour representation of the image by assessing object size and shape, the technique employed including shape and text recognition, shape codebook, shape repeat, filters and line approximations in the following hierarchical stages:
a) a coarse detail approximation of the image made up of plates of equal block colour corresponding to large ascertained shapes;
b) finer shapes and characters determined from a variable codebook;
c) finer detail refinement consisting of areas of with decreased variations in colour values, and;
d) a successive build up of the image by repeating c) with reduced colour variations with each pass, and where this technique does not result in a fully described image, further build up techniques may include major gradients, texture simulation and repeat, secondary outlines and fill.

10. Apparatus according to any one of the preceding claims and including dynamic partitioning means for producing data packets in order to allow the integration of the data in a dynamic manner to provide for the efficient transmission of a data stream over an available bandwidth.

11. Apparatus according to claim 10 and including data packet shrinking means for shrinking the data packet as the line quality changes or drops in quality, in order to minimise the impact of all data lost since lost visual data can be re-transmitted but lost speech data is permanently lost.

12. Apparatus according to claim 11 and including dynamic data packet sizing means for dynamically sizing the data packets as line quality changes in order to minimise on the input of data that is either lost or that requires re-transmission, and in order to maximise on the transmission of data.

13. Apparatus according to any one of claims 10 to 12 and including monitor means for monitoring for a gap in the transmitted speech for the insertion of extra image data.

14. Apparatus according to any one of the preceding claims and which is for more than two persons.

15. Voice and image communications apparatus substantially as herein described with reference to the accompanying drawings.