CA2379330C - Video processing method and apparatus - Google Patents

Abstract

A method and an apparatus are set forth for encoding and decoding video to achieve bandwidth compression. In one form of the disclosure, two video signals (810, 850), representative of different images, can be transmitted using only the bandwidth (880) generally allocated to a single video signal, with little or no perceived degradation of image quality. In another form of the disclosure, motion indicative signals (970) are used in a technique that dynamically modifies the frequency band information to be stored and/or transmitted.

Description

WO 91/15929 - . PCT/US91/02228 . . . . . -_- DESCRIPTTON ~_ -- _. _ :.
VIDEO-PROCESSING:METHOD AND APPARATUS "
FTELD OF THE INVENTION
This invention relates to video signals and, more particularly,: to,.apparatus_and method for encoding and decoding video signals..for-use in.televisionvand in high definition television~systems as well asrin other ' applications including storage and/or transmissionvover any suitable,medium, of:moving images,~or combinations of moving ..
and still images,_in a form: that requires reduced storage capacity and/or reduced bandwidth channels.v Some of the techniques hereof can be employed; for example, for transmitting through the air or through conducting or optical cable, a plurality of video signals~using only the bandwidth generally allocated to a single video signal, and with little or no perceived-degradation of image quality. Some of the techniques hereof can be employed, for example, in so-called "compatible" high definition television approaches, as in so-called "simulcast" approaches wherein independent high definition television signal is sent simultaneously with a conventional transmission of the same program information.
Some of the techniques hereof can also be employed in so-called "enhanced definition" approaches that send picture enhancement information (but less than the information needed for full high definition. performance) on the same channel with a conventional television program.
BACKGROUND OF. THE-INVENTION' Available spectrum is becoming increasingly burdened by ever greater demand-for_.video information channels.
Traditional airwave.spectral. space has beenvcrowded for many years, and burgeoning .video program~irig for 'such applications as home cable, teleconferencing,-picture''phonesand~computerw video transmission:has now crowded conductive and optical' '~
c~les, phone ,lines,:: and sattelite ~ conmunication ~channels.~

wo 9ins~z9 Pc-rms9l~~nxfz~
a i The desirability of techniguss for increasing the amount of video i»formation that aan bs ser~t~ °,. over these transmiss.ian media is evident. ~. A~s~c~~~.. as mere. .,via~o.'..~.nfi~'s~RiatiQn~ is~
stored, It is des~.rable to develop techniques -chat #increase the amount of video that cast be,- stcxed.in a given storage size.
As hf.gh definition tal~vis,ion (HhTV) becomes mere w '' prevalent, improved systems- ar~~. needed for. trartsmissiaw ahd v r~cepti.an of. the.; addi~i.o~al, infcrmatior~_ reqN.~red far r presenting HDTV images,. Any- new- services which provides higher de~initian te~ev,.is3.an_:than is convent3.onally broadcast (i.e., mate elements per ~.ine and lines3 per frame, and thus a widor bandwidth naaos$axy far.txar~em3ssioy ahoWld serve existing home telev~.sipn receivers with e~3sent~al~.y all the picture attribut~s and quality of which the receivers are c$pable. Also, receivers dasf.gned far new (high definition) service, should be capable af,_operating using the pre°existing transmissions and dex~.ve from them a ,result not inferior to that provided bV pre-existing . receivers:
A variety of HDTV schema hive heen proposed. in U.S.
Patent No.s 4,$17,597, 4,b2B,34y 4y52,909, 4,7~1,78~, and 4~800,42~, assigned, to the same asa~.gnee as the prBaent application, as well. as 3n the,. publ.ioation nHDT~t Compatible Txar~smisaion syabmm~~, W.r. Glenn, Hatl.onax Association of Braadcastexs, Apr~.).. 198b, there is disclosed an HDTV system that utilizes an auc~sntation approach which permits compatible tra~tsmisafon of.. HDTV. A s~parate auxiliary or ~~augmentation~~ chsnne.l 3e used to send picture c3etaix informat3.on that augnsenta conventionally'received t~rl,evis3on ~.n~ormation to nhtain high d~firtit3on performance. The disclosed techniqe~es slap have .,dpp1#.ca~Cion to viddo bandwidth compression and to redac3,ng,vid~o_starage capacity.
As r3eseribed in the rexerer~ead patents axed publication;
an ele~ctronia video signal , (e. g.~; a ~ telev3,siori signal ) can bev encoded at reduced bandwidth by Iowering:the f=ate refresh rata ~x~ tho..high apafital, gr~guenay Gongsone»ta, whf.ler . . r ,_ maintaining tl~e . iramet refresh ; rake , of : at 1~ast a ~po~t.ion ' o#'~ ~
' the ,low spatial ~.~f~equen~yr , cop~ponsnts.~ , atv th~ standard rate o~~

wV y1i I~YLy : v ... PCT/U591/02228 If done i~ a. specified. mannerwthis: will= not'- cause=- substantial-degradation in. the: ultimately displayed. image,=-~-sincew .humane ~
°~
vision cannot' perceive changes, ~in:~ high spatial'- resolution'w ~ ~ ' information at as. fas a. rate as: it caw perceive changes in -low spatial resolution-information. Accordingly; as~has~been previously , set - forth, - an. , electronic video r encoding and - ~ - ~ -decoding system can"be devised_which-takes ~advan~tage'of=this, and other,. characteristics_of,human.w vision by encodingvhigher spatial. resolution..video, components; to be.at~'a temporal information rate.which< approximately corresponds~to the w highest rate actually,: perceived by human.vision~ifor such w -components; thereby.. eliminating the.need to encode these . -components at a higher rate,-which inherently-wastes bandwidth.. Also, as shown ~in referenced patent and v publication,. the low spatial resolution information can be generated in a form which is compatible with standard television video, for examgle.NTSC video used in the U.S. It has also been recognized that a number.of-frequency components can.. be transmitted at specified rates [see e.g.
W.F. Schreiber et al., Reliable EDTV/HDTV Transmission In Low Quality Analog Channels, SMPTE: Journal, July 1989, and the abovereferenced patents of-the present assignee], with components selected according to degree of motion in order to have higher spatial resolution in-scenes with little~motion and higher temporal resolution in scenes with a great deal of motion.
Fig. I illustrates.a compatible. high definition television transmission.,and receiving system of the general type described in the above-referenced patents and publication. A transmitter 200 includes NTSC processing circuitry 21,0 which processes television signals from a source such as a television camera system (not shown) or a video recording system.(not,shown)....The'circuitry 210 is coupled to transmitting circuitry,215,.which typically --includes modulation circuitry::and other=suitable=circuitry .
for groducfng a signal:to-be,:transmitted over:a=standazd NTSC ~ -channel . The television. signals. fromr the televisidnm r camera WO 91I15~tZ9 v. .' .. PCFlE1S91I0?.Z2~ .

system, or,wideQ.;, r~entder. (wh~.ch is~~assumed~~to have a high definition__vidaa;~caga~i;lit~r)' are also prab~~sed by high .
def~.nitian.:telev~si,on., (Hp~V) processing circuitry Z6o which produces detail.; sigma#s. that can be~ ut3~li:zed to enhance corive»tional televiaibn sis~»als to'obtain HDTV signaxs, as .
described in_ the, abovsre#era»ced paten'ts~~ and publication.
[ As furthar: deac~ihdd ~.n the. referenced t1. S : pate»t I~o. ~ .
~, ss2, 909, . the,:,detail ~ aign~si can. be obtained ~ from a'. separaxe camera. The datail=~t~.c~nals are coup#ed'to ~uxther airGUitry' 275, which trans~tits ~ the detail signa~.~~o~rer a' second .(auxiliasy).:chann~3.~that ~.s typ#caZly not adjacent to the (maf.n) NTSC ahan~a~.v.uscsci fox .tranami.a~s3c~n of the staridax'd ' .
pprtion of the ~keieVision it~~oxmatfon. The NTSC signal is received by receivers auch~as.rece~,ver 31Q which has only a capabil.#~ky.. of pracl~ec#ng a te7.ev,ision picture at substantially coaventioral r~rsol;ution e.g. canvant~ivnal disp7Lay 315.
Receivers such as. receivBr 360, which have a capabixity for receiving, prACessirig, and. displaying high defir~itio~r talevis~.on sig~tals, xe~oive both the mai» chann~1 carrying the NTSC sig»a~. and the aWx#lia~ry channel carrying the detail signa3s to be used for augmentation of the NTSC vid~o signal so as to produce a hi.c~h de~i~n3tion television. signal #'or display on an HDTV dtspiay 3b5:
In the refexenceci patents and pglalicat,io», the spatia3 detail. is trans~ntitted at a relatively clew frame rate, such a,s 15 or 7.5 frames per secpnd. "Juttexr' (jerky edge motion) was obsarved when thh.detai# frat~e rate was reduced too far.
Th~.s artifact constrains the augmentation channox bar~dwidth _ to be ~.argex than wou~.d athexwise be fnd#ce~ted by psychophysicai,stud#es. Camera lag, c~iua~ed by the i»tegration o~ image energy o» the face v~ the camera tube, which attenuates, detail i.n moving araas~af tHo picture; can ' be exploited to reduce ~uttsr, but some'reductian in image dataii can be obsesved~ in mov3.riq~ ab jectw When ' their are' ~ - ..
v#~aaa~.ly tracked.., . .ttv. ia~.amv»q they obyecta~ of the present ~~ '."
.invention ;to' pro;vir~~,; improva~ritay in' perfb~ince and° is ~- . .
, bandwidth - ca~press#.ori with respect. to: ths~ techn,id~ues ~ ' ~ ' wv yii i~yiy PCT/US91/OZ228 ..
described. above, - and.. with respect to:- other-priorvartv = - -techniques . It ~ is- also; - among=. the- objects hereof' to" provide. " ' ' ' such improvements.- in a- :- system; that. can. be' made compatiblev with existing:,.or:,.future~_.; television:. standards ( for example, NTSC, or other standards.such as PAL-or SECAM). ~ ~. - ~ -As further background;to-the.invention; reference can be -made to the following U.S.:Patents.which relate;to~~~--=- ~~-~ -compression,;transmission.and/or other processing of videow-signals and/or still picture:-information:~-=U. S. Patent 4,196,,-448,_ 4,210,931;- 4,:224,678°;:4;302;775;:4,394,774;- y 4,541,012, 4,605,952, ,4,630,.099-;= 4,661,862;w4,672,425~.-' 4,675,733, 4,675,750,:,4,729,012,_ 4,774,562, 4,780,761,-4,791,598, 4,807,029,.4,821,119,-_.4,845,562, 4,85T,906;~ -4,870,489 and 4,873,573..
The FCC recently announced that it prefers planned terrestrial HDTV transmission in the U.S. to be broadcast using a simulcast format: i.e., with the same program content sent simultaneously both a conventional,.television channel and a separate HDTV channel. -It has been anticipated that, in time, television viewers,_will.replace standard NTSC receivers with high definition sets,: thereby allowing the present NTSC
channels to eventually be reassigned for other application.
In order for this concept to.work, however,wiewers~must be motivated,to purchase receivers:designed.to accept this new format. Even when wide-screen HDTV becomes available,'a significant demand-will always exist for smaller=screen receivers. The image quality of small screen-size television receivers is generally not limited. by transmission considerations, but by,human visual~acuity. The optimum viewing distance for popular 19-20'.'.:conventional receiver, for example, is between six and seven feet. A similar screen-size HDTV receiver has an optimum viewing distance of about three feet;~clearly impractical in most viewing situations:v. The goal of abandoning.-the,conventional NTSC channehs iri the foreseeable future may be::impractical because therevwill always be a..consumer,demand;,for-inexpensive smaller screen television sets . . . =. - .. _ ..

WQ 9111~9Z9 . . . , pCf/U891/QZ22$ ' s It fs~ alsQ. amongv the;. ol?jea~s: af~ the pxesen~t- invention- to ~'.. _.. .
provide. ,improvement~.~ .1» enrodingw aid decoding ~ of ~r3deo i~formatian: which addresses: the~° d~scr~.bed problems and ' ~.imitatio»s of.~the; prior_.$rt, achieves substantsal~ bandwidth ' ' savings, increases the efficiency of v,~deo transm~.ssion and storage,,aRd..proYides a capability.for~.higher definition tea,ev3s,ion xansmission=in.~he bandwidth pf~a single . , conventional te.leviaiow channel. it is a~.so among the objects of the, present.invsntion to provide a technigue whereby two video signals, represantative~~'of differe»t images can be transp~itted uainc~ only the bandwidth r~enerally olloee~'ked tc~ a es~.ng~.e v3:dcP signal, ~i~.tlt .little of »o pezceived degradation of image quality. It is also among the objects of the present invention ~o provide a method fox broadcasting video signals w~.th improved interference immunity.
~ . : : SUMMARY f~F THE INY~NTI~1N
A method and an appax~tus are sit ~oxth~fox~ encoding and decoding video to-achieve bandwidth corppt~ession. I» one form of the ~.r~vention, two video sig»als, representative of different images, can be transmitted using only the bandwidth generally allocated to a single video s~.gnal, with little ox rno pQrceivad degradation of f~nagd qe~ality. In another farm pi the i=tvention, motion indicative signals-aye uaed in a technique that dynamically ~oclifies.the'frequency band intarmation to be stared andfor transmitted.
Further foatWro$ aac~ advantaqQa 'of' the invention will become mcre,readily apparent frog the following. detaiied description when taken it con~unGtion with the accompa»yirtq drawings.
BRIEF p~SCRIP~'IU~THE hRAWINGS
Fig. 1 ~.s a siatp~.ifasd brlock diagram of' a prior campatibi.a high, dett~i~ioa teleVti3.io=i' 9ysteilt. ' .. ' ..
Fig. ~ i~_,a pc~Iar: plot .~llustrati~ig data that sie~aareg the obligtse eftect. ; _ .

WO 91/15929 . :w - PCT/US91/02228 F.ig:.. s.; 3A,.; 3B and._;3C:-. respectively- 'il~liistrate= cardinal sampling, quincunx.sampling,and=quincunx sampling-with-w-' ~ -reduced sampling rate. . .- . ... . . . . . ..
Fig s 4A, 4B and:4C respectively illustrate spectra for the Fig. 3A, 3B and 3C situations.- -. ~ -.~
Figs 5 and 6 illustrate spectra referred=to in the description.. -. . : ,... :'.= r. ~ _ : -. -:;. v v ;-: :. : . ;::
Fig. ? illustrates the four quadrant pass band of~a ~~ -two-dimensional diagonal.:filter.having its vertical and horizontal cutoff frequencies:.at half the video sampling rate . .: ::.: ~. : :.. : . ::. ~. .. ': .
Fig. 8 is a block..diagram of an embodiment of an encoder and encoding.method in accordance with a form of the invention.
Fig. 9 illustrates an example of the coefficients of a 9x7 filter kernel array....
Fig. 10 is a block diagram of circuit which can be utilized to implement two-dimensional convolution with a filter kernel.
Fig. 11 illustrates an:.example of two-dimensional spectrum folding around a diagonal which occurs as a result' of a two-dimensional image modulating a two-dimensional subcarrier.
Fig s 12A and 12B respectively show a pixel array before and after two-dimensional:modulation. . w Fig. l3 illustrates a circuit for implementing two-dimensional modulation:
Fig. 14 shows a block diagram of an embodiment of a decoder and decoding method which can be utilized to recover signals encoded in accordance with a form of the invention.
Fig s 15A, 15B,: 15C 15D and 15E shown illustrative spectra. , ,.-Fig. 16 is a block diagram.of an embodiment of an encoder and encoding_method in accordance with another form . . -. . . _ . _; _ : , ._ , o _- , . :. . ..
of the invention. . .. _ _ _. . . : ... .. . _ .
Fig. 17-is~a diagram=of-:.a.decimator Which can be " ,.
utilized in an embodiment of the invention.

WO.91/15929 ~ ~~. , . PCFlUS9llOZZZ~3 F~,g. :1~.. is: ,a, b~oc~:.,d~.agram.~.of_ anv embodiment of a decoder and decoding, msthcad. which can.. be utii.iaed to decode the encoded signals o~ a l'orm of the inrrenkion. .. _. ..._.. _._ ....... . _ .
Fig.. 19. is a diagram of a zero padder which can be utilized in embod.lments of the iltvention. -FI:q. 2p is a block,diagram of an embodiment of an Encoder and~encoding method in accordance with another .fo=m of the invant~.on. ... . . . . . . , . , .. . .
Fig. 2l is a block diagram of an.embod.iment of a decoder and decoding method whfch can be,u~iliaed ~a decode signals encoded in accordance with a further form of the invention.
Fig. .2a is a .black diagram of a» smb~ad~.ment of a high-definition t~levl.sion syt~tem and. method- in accordance with a further form of, the invention.
Fig. x3 illustrates a furthax form of the invsation that ,is used to minimize int$rferenca botween traasmi.tted v~.deo signals.
F~.g.s 24 anc3 25 illustrate exemplary. band divisions of tha Fig. 6 apactrum.
Fig. 2C is a block dl.agrata of an encoder in accordance with an embodiment. of the irwention, ~ alld. whf.ch can be used to praGtfce an embodiment of method o~:th~ fnventl.on.
F3g. 2? ~;a a black diagram of a system for encoder scan conversion.
Fig. 28 3s~a block diagram of a portion of the encoder of the Fig. 26 embodiment. _ . .
Fig. 29 is a flcW diagram a routine for coxltrol.ling the tile control processor of Fig. 2,8. .
Fig. 30: ~i~3 a blpck d#:agram of an embodiment of the mpt3on detection circuit of Fig.. a8. ~ -Fig. 31 is ~a bloick dl.agram of- a decoder fn $ccordance with as embodiment of the invention and which can be util~.z~d to practice an embodiment,, of. the, dl,~c~;ps~ decoding method.
Fig. 32,.;which includes Fig:s.32A and 328..~laced one below at~nthQx, ire a #low .diagram of ther ~cotstinc for ~ w cantro1.1.1.ng vthe attc~rtentation in~rwt ~ processor.: of~,the Fig: 3 i embodfonent. . .. , . --_. ~ .

WO 91/15929 -_ _~ PCT/US91/02228 g ,.
Fig.s.. 33 .and 34~, are- flora diagramsvof the- routine'~for" -implementing -the- spectral-to-detail converter' control ~°"
processor of , the. Fig .. 31° embodiment'.' ' " -Fig. 35 is, a block diagram of a fifo circuit utilized in the Fig. 31 embodiment..
.., DETAILED.. DESCRIPTION
Subjective:.vision_studies have indicated~that perceived resolution is anisotropic°(not equallyrprecise in all'-directions).;. The eye is more sensitive~to detail along the' horizontal and. vertical r. axes- than' to~ that along diagonals:
[See, for example, W.E. Glenn et.al:, "Imaging System Design Based On Psychophysical Data," Proc.~of the SID, Vol 26/1, pp.
71-78, Jan. (1985); NYIT STRC "Visual Psychophysical Factors as Applicable to the Design and Development of Video Systems for Use in Space, Final Report," NASA Report, May (1989); 6.C.
Higgins et al., "Variatiow of. Visual Acuity with Various Test-Object Orientations and Viewing Conditions,''' J. Opt.
Soc. Am. 40, pp. 135-137 (1950); F.W. Campbell et al., "Orientational Selectivity of the Human Visual System," J.
Physiol., 1.87, pp. 437-445, (I966); and S: Appelle, "Perception and Discrimination as a Function of Stimulus Orientation: The "Oblique Effect"'in Man and Animais,"
Psychological Bulletin, VoI. 78, No. 4, pp. 26f-278, (1972).]
Fig. 2 illustrates this oblique effect, plotted in polar form, as characterized by various researchers. While results differ somewhat due to the different types of testing employed; the curves of subjective resolution have a similar shape and diverge from the isotropic resolution shown by the outer circle. It is known that bandwidth and. display element density can be, reduced by taking advantage of the anisotropic spatial response.characteristics'of thewisual system. Fig s 3A, 38 and 3C respectively illustrate cardinal sampliwg, quincunx (or diagonal) sampling, and quincunx sampling with reduced sampling. rate., Figs 4A'~'48 and 4C show'the~
respective discrete spectra . for-- the ~ sa~epling- of Fig. s 3A~, 3B
and 3c, where fs _-. 1/D. _ The,, quincunx samplingshown in Fig:'s WA 9!/59'l9 P4~f/~fS9tlp~
1~
3B and 3C re"suits. ;irk the. rotation. of. the spectral coaxdinatr axes by 45 degrees...(see,.; for.. example, . R.C.-. Gonzales et al. , .
Digital image ProceSS~.nc~, Readi»q~ Mass. , . Add~.son-Wesley ~ . .
(1957); 5. Dubois et al.,'!Three-173mensional spectrWm and pracpssion ~of Digital NTSC color S3.gnels, ~~ SMPTE ,3ouraal, pp.
3?2-3?8, April ( 1982 ) ; and 8. Wes~dland et al _ frOn Picture duality of Same Telev3.sion Signal Frocessihg~~Teahnic~ues, ~~
5M'FTE Jaurnalr:PP~,935-9x2, Oct., I19~4j], thereby more closea.y matching_.'the characteristics of vision. This method cart be used to-. xeduce the. 3.rifo=matiort content by a factor of two withoctt degx$dation_ia perceived 3;mage quality. Half. tone pr3.nt~s anr~, more recently, CCD cameras and LcD displays axe sttecessfully uti3izf,ng ;this tech»igue: soma of the systems described in the. patents seferencsd fn the Background portion hereof utilized gti.inct~r~x sampling tp reduce the sampling rate, arid therefore thg augmentaxien ba~widtn, by a factor of two.
In an embod~.ment described below; information cont~ant is reduced bar eLi.minating high diagonal freguency components appxoxiatate.iy to the upper right of the dfagnnal line 5 in the discrete spectral domai.ri ~llustratad tn Fig. 5. The NTSC
luminance spectrum is il3.ustxated approximately in xh~ lo~wor lefthartd box'of Fig. 5. rn a subseguently described embodiment, far an aWgmentatioh System wherein the NT5C
apaatral portia» is avai~.ah~.es from a GonV8nt10»al. channel:, the approximate rs~ainiag spectrum used for transmission an the augmentation ~ahattnel .~s.shown 3.n the shaded region of Fig:
A Sow-pass filter can be used to restrict the treguency cnatpopents of a cardinally sampled fmape to the region within the diamoad~shaped perceptivity curve of Fiq..2. A telev~.sion viewer, postt3:oned at the mutate favorable vfewing distance roughly six screQt~ hoights fot conwent~.ons~, NTgC~ 525 line video); is not obis to resolve the (vertical) video raster, yet 3.e sill. able tQ,app=eciate image detail.. Fi~u. re ? shows the tour quadrant pass--bar~a ~f a.: two-dimensional ~ diagonal v .
filter hav~nq both its . verti,csl attd- horizontal cutaf f frequencies, set to~~o~e--half the video sample rate ( fox r WO 91/I5929':' PCT/US91102228 1 I -.....
example, one-half the vertical sampling rate):- One-half ttiewy total spectrah- area~~is :passedw by this-. filter: The effective two-dimensional bandwidth-for-.-imageswmatched to the characteristics of.the human..visual system is only one-half of that created by cardinal sampling. In- accordance with a feature of~the present~invention, useful video information is two-dimensionally modulated so as to be positioned in the ~w shaded portion of Fig..:7; i.e.,~ into:a spectral region that has beew effectively:unused and,genex°ally-wasted in prior art systems.. In an embodiment hereof, two different television images, for example, can. be encoded on a single transmission channel by effectively placing. one in each of the two distinct spectral regions. Applicant has discovered that each television picture. can maintain substantially the full subjective resolutiow found in the original, and is completely separable from the other television picture.
Referring to Fig. 8; there is shown a block diagram of an apparatus in accordance~with an embodiment of first form of the invention, and which can be used to practice a form of the method of the invention. Two electronic video signals are produced, as represented by the blocks 810 and 850, respectively. The electronic video signals may be generated by any suitable means, for example by video- cameras; video storage, graphics or animation generators, or medical or other imagers, etc. It wilf be understood that the blocks 810 and 850 may represent respectively different types of sources of electronic video signals. As an illustrative example, it-can be assumed that the blocks 810 and 850 represent electronic video camera systems directed at different scenes. As described hereinbelow, the signals can also be representative of different components'of~the same image.. Also, it will become understood that~the techniques are applicable to: various formats of electronic video signals and to conventional as well~as low o= high' definition video:
An example is initially:set forth-iri' terms of avmonochrome ~~
video.signal.having.conventional' television resolufion,-v although...the techniques~hereof=are- also generally acceptable ' wo 9~i~s~z9 - . ". ~cr~uss~iiQZxzu 12..
c , to color video_signals:~:~. . _ .. . " " .. ,... ._ ._ .__. .... . _ .. . .._.
. _ _.....,.. .._.
The "outputs:, of video signal sources . 81D.. and 85Q are - . - ---~- ---reapeative7.lr coupled ta.~I,ow;pa~s-:fijaers BI5 a»d- a55~ (see -.:~,.
also two-dim~ansional spectra si5A and 855A), a»d then to analog-to-digital cpnvettexs 824 and 860. The analog--to-di.gital converters can be operated atany suitable clock rate, in knowin. fashi.~», to obta~.n frames of digital pixels which are .stored-,~,n digital buffers 821. and 861, xeapea~~.vely:; The k~uf,fers may be frame buffers. or portions thereof . each pixel- of. each Exams- c$» have a luau»ance value convent~.4»ally rispresent~ed by an n-hit digital word.' Thu outputs of fray buffe~rt~. 8~1 artc~ 861 are , respectively caupled,to two-d~:mensional diagonal low-pass f3.~.tsrs 8~5 and 865. Faah of these Tilt~xs is operative to remove high fregusncy two-dimdusiona7, diagonal fregueacy componentr~ from the fr~a~tcea of digitised video r~ic~naZ. Fos example, for the approx3.mately squaxe spectrum of the first guadxa»t Fig. 7,. the filtering of the px~serit embodiment wil.I
p=efesably result i» a.spectrum having an approximately triangular shape as in. the unshaded .regfat~ in Fig. 7 ( s~:e a3so the sketches 825A and 8fi5A). It ~ri~l be understood, however, that the, line joining the highest paaseci vertical arid horizortta~. freg~~nc~.as [ ( f~« ~ constant ] ca». genera l 1y be aosslgcnsg as a-boundary. As noted in co»~;u~ct.ion with the description of.,Fig.,2,:_there are inveatigatars who have determined that event:som~ freguenciea within the i»dicated triangular regsoa will De Substantially attenua~ad by the human vfauai $y~tem. The precise shape of the fi.lto~c can b~
determined frog present and/or future studies on ths~ humavrt visua?.:system, arid/_or aan be adjusted empirically.
The twa~dime»~sic~al diago»al low-pass filter (825 and a85) aan,be.impleme~ted,by any su~aable technique. For examgle, a commercial programmable fflter kernel can be utilised,to obtain"the desired;fi3teriag function. . F~:g. g ..il.laatrat~aa ,ans.. exs~m~ale of._~GhB coatf3ci.ente~ of 3 9x7 'fi.lter. ' .
kernel array; that, cart, be., ~utllized to implement two--diate»sional diagQnar. lob-paaa_ l~il~ex3.ng.., ; The. filter. kernel. cart be applied WO 91/15929 . PCT/US91/02228 ' 13 __ by convolving the . array.-with the frame of ~ pixels tow be ' filtered. , Techniques -far--:implementation- of the filtering " _ process, are.known in.the art. Fig.~lOvshows~-a block diagram ...
of a circuit which can-be,utilized to.implement w two-dimensional convolution,.-.and which. can be employed, with appropriate weighting coefficients, in the present embodiment to implement a two-dimensional.diagonal low-pass filter: In -the circuit of Fig..: 10,: an: array of : coefficients kij, 'are ' w applied to an (mjx(nj moving group of pixels by using m line delays 1020 and n pixel~.delays which are indicated in Fig.. 10' by representative register rows 1025, each of which has individual stages with respective one. pixel delays. Shift registers or,FIFOs may be used for~this purpose. Each pixel and delayed pixel is multiplied by a coefficient, kij, with the coefficient values being implemented by applying corresponding signal levels to the multipliers 1050. The coefficients can be in accordance with the selected array for a particular filter kernel, for example the array illustrated in Fig. 9.
The outputs of multipliers 1050 are summed by a summing circuit 1080 which produces each convolved output signal as the array "moves" over the frame. It will be understood that other filter implementations, including commercially available chips, can be utilized,, if desired, and that end conditions can be handled by techniques known in the art.
Referring again to Fig. 8, the output of two-dimensional diagonal low-pass filter 825 is coupled to one input of summing circuit 880.' The output of two-dimensional diagonal low-pass filter 870 is coupled-to a two-dimensional modulator 870, which serves to fold the.spectrum of the filtered signal , into the spectral space normally. occupied by the high frequency diagonal; components. Fig. 1l illustrates an example of the two-dimensional spectrum folding which occurs when a two-dimensional image modulates a two-dimensional subcarrier having (in this casej,.,a"horizontal frequency of half the sampling rate and a vertical frequency of'half the sampling rate. In.,general, the.image spectreim will be folded:~around~~
the diagonal ,demarcation<linewand.reversedw'so-that'~hi.gh wo 9ms~ra . rc-~iu~~imzz~a i~
l r freguency hvrizonta3w co~pp~ahents:-. fl f the original: image becamQ
high freguency. vertical components and vicewersa~ as sshown ~n Fig. l1.. A carist3n~ gx'ay~level (dc~ on'thawencoded~~~ma~ge~wfll .
then appear as the highest possible.frequency which~cari be w represented with the sampling:Paremeters,v f5/2, f8/2. ~in ether wards, , the spectra. location ( t1, A ) ~ after suoh ts~o-dimensional .
modulation. wi,l~.,_be a~ (f~12, f~12) and~coristant ihtensity ~.cve3 will generally e~ppeh~c as a high freQiiericy checkerboard pattern. [For this modula~kion in the 3:mage bs3.ghtness domain.
being modulated in. this casewas brightly above the average gray 3evel as below itr: the entire image.content would be~last and appear as n blonk gxay ac~een to :x viewer more than six screen heights from the display, since the spectral content is outside the range at hcuuan percaptivity. i.e. the he-man visual system would act as a diagonal low-pass filter. The spectral location (0, tx), after much two-d.~menaiortetl modulation, will be at (~~/~, f~12-f,~), and so on. The two-dimensional t~advlation to achieve folding around the diagonal. can be implemented by reversing the po3,aritlr of every other pixel on evesy line, with the polarity order reversed every other ~.ine, as illutstrated in Figs l~A and Ira. Fi9.
12A shows a pixel arrap before two-dimensional modulation on the two-dimensional. subcarrier, anc~ Fig. lab shows the pixel arzay after such modt~lati.on. F'ig. 13 illustrates a aiFCUit far implementing this ipadulatfon. A mult~.plexer 1350 receives, at one inpux, the pixel output of two-di8ensional filter ~s5 and, at another inpux, tMe pixel output inverted by invexter 1310. The modulator receives pixel and lino ind#.cations and al~etnates~~its input line select~.on sequence in accordance with the~polarity a~quence shown in Ffg. 128.
The encoded output of summing circuit 8~0 contains video s.~gna~.s representativg of both 3.mag~as (see ewo-d3.mQnsicnal spectral sketch 8~~A.,which denotes the respective vidso~sfgnal spectra as x and Z), can be stored andlar transmitted, as rwpresented by;tha b~.aak 89p.. . . .
Referring ,to. i~~g.: :,.I4, there: is ahpwn a blocl~ diagram of a .
decoder which can be utiZized.to recovex and record and/o~c WO 91115929 .... PCf/US91/02228 display the encoded video-signals: The block 1410-represents ~~
the receiving of the encoded signal-or the reading thereof .-from storage. The two-dimensional frequency spectrum is shown in the sketch,at,1410A.,,._,An,optional pixel.storage-buffer 1415 --can be used to store frames of-information or. portions thereof. The output of buffer:1415 is coupled to a two-dimens'ionah modulator.which can operate in the same manner as modulator 865 (F.ig...8), as illustrated in conjunction with Fig s 11-13. The_previously.de5cribed spectrum folding reverses the spectral positions of the video signals. The output/of modulator 1450 is coupled to,a two-dimensional diagonal low-pass filter 1470, which may again be of-the type illustrated in ~conjunction_with Figs 9-10. The output of buffer 1415 is also coupled to such a filter.(1420): The respective outputs of two-dimensional low-pass filters 1420 and 1470 are coupled to digital-to-analog converters 1425 and 1475, and then to analog (horizontal) low-pass filters 1430 and 1480. The output analog video ignals can be recorded and/or displayed, as represented by the blocks 1435 and 1490, respectively. In operation, it is seen that the two-dimensional, modulator operates to "reverse" the spectral locations of the signals identified as "1" and "2" (see sketch 1450A). The filters 1420 and 1470 can then be utilized to filter undesired spectral components and obtain the respective separated video signals. (see also sketches 1420A
and 1470A). After conversion to analog form and suitable ....
low-pass filtering, the recovered signals can be recorded and/or displayed, as desired.
In accordance with a further form of the invention, the size of an image may be reduced,.. or squeezed, by :;ub-sampling after first filtering to remove frequencies which would otherwise cause aliasing. ,As will be described, an image can be reduced to half its original.size by throwing away, or decimating, alternate pixels, horizontally and._yertically.-Spatial decimation can.also.be;used to reduce~.the.data content and associatedytransmi sion_bandwidth of_non-squeezed~iniages -without substantially.: degrading-, the subjective resolution'.- - = w-- .

W4 91!159Z9 PGT/US9I/D22'tii 3.6 $e~ore-constder.ing' two-~dtmensi,onai~ ddctmat~,on; i~°'is ~helpfu~.
'.. _ tea examine theweffeGts of- one-dimensional decimation: ~tf it' is assumed that avcontinuc~us' ' analog s~:grial is~ satppled using ~a d#.gitizer which has an ~.nfin3.te?:y small, sampl~,ng aperture, ' the digitized i.xsput card bo considered as a conttilllat~ts dndla~
sislnal mult3.plied by a series of impulses spaced Ti secar:ds apart. The samp3ed spectrum'cnn$ists of th'e oxigin3l ana~.og spectrum convolved with spectraa. impulses located at n~~ where f~~i/Ti. Repetitions of the analog spectrum w1~.1 thus be centered about integral, rnultiple~, of the samplfitg frequency as shown in Fig. 15A. No a3;~.asing will occur if the analog ~~
base~aand~spectrum fs resxricted to frequencies less than fg/2 prior to sampling. Assume, now, that :even and odd saatples are separated using the fc~llowtn~ even and odd dec,imat~.ng functions:
Dk.''Cove~ ' ti/Z:)r1 t aos2pi*(f~/Zj(X/t~)~
(1/~)~l. + coa(pi;x)j 1~EC,~d -- (l/2) ~l - cos(pi*x?
where x s integral sample number' Apply~.ng DEC~,.A t° the data set wLll force the odd date elements to zero whii.$ pEC~4 wLll i~orca~ the even elements to 0. The spectrum of an evenly decimated image is obtained by convclving the sa~rip3ed spectrum with the spectrum of DEC~n.
rig. Z~a shows that dscimat.ion caused another repetition of the analog spectrum tc~ he c=sated about f~/2. Alids3ng wf.Il pacux for fregusncies greater than f~/4: DEC ~4 viii create a similar spectrum, but the a3.fased components wsll bd f,nverted with the =aspect to the alta~shd spectra dram DECD. When the add and even compapanta are. added, the wnslf.aeed ori,gina.l components add, but aliased components canc$l. leaw~.ng the origf.hal spectrum. While this result ~.s f:ntereeting, it is of little pract3.ca~, value in one d,~mens3on= the same result is obtained by s~amplinA the analog .i.nput at fw/~.
Spatial deGimat~on, however, does not lead te'the trivial. rasu~.t encountered in'~ one dimension;' provided a ~ ' diagonal decimating-pat~ernvis used: ~A d~:aqonal decf.matox ~ ~ .
ata~aks an image.with a checkerboard pdttern of ~a.g': black) W4 91./15929 ' PCTlUS91l02228 17 .- .
dots. The even decimator can be represented as:
', DECdia9 =. (1/2) ['1: +: cos(pi*x}cos(pi*y) l .-. _....-..
The spectrum_.of.the~decimator is-composed of two frequencies,- --one _ at DC : ( 0, 0 ) , the.: other- at half -the w horizontal and vertical.sampling rates (fh/2,f~12).~ It is analogous to the one-dimensional'decimation spectrum except that frequencies are two-dimensional. An image to be decimated can: be first. passed:: through a two-dimensional diagonal pre-filter of the type previously described. The decimated spectrum is obtained by convolving the filtered. ~ _.
image spectrum with the spectrum of the decimator. The original-filtered.spectrum is repeated about~multiples of the decimating frequency, i~.e... ..
_-(2m+1)f$/2,(2n+1)fs/2 for.all m and n a portion of which_is shown in Fig. ISC. Passing the decimated image through another diagonal filter regenerates the undecimated image without degrading the image quality.
To summarize the. foregoing discussion, when a diagonally pre-filtered image is.spatially decimated diagonally the spectral components introduced by the decimation process do not extend into the original spectral region, and hence may be removed by filtering.
Diagonally decimated images contain half the number of pixels per line of cardinally sampled images. The time necessary to send a line of.video can be reduced by half if the bandwidth is maintained constant, so that the image will be squeezed horizontally- The second half of each video'line may then be used for other purposes, such as carrying a second image or carrying high definition components of the hasic image. Squeezedwimages'can be reconstructed at the receiver by reinserting~.the zeros created by the decimator and passing it through a two-dimensional low-pass filter.
Fig. 15D is-a representation of a two-dimensional frequency spectrum.of a:decimated image which is composed of '~
repetitions of:the-cardinally-sampled spectrum about -multiples of~ the decimating frequency, 3.a. at w ' '' w' . . ( 2mi'l ) f ~~~~n2 ~ ( 2n'f'1 ) f ~u~u~ ' . ., _ ..

wo ~ii~~9z~ . pcrms~~razzz~
1e fax all intagra~. values of m and~ »:: -__. . . . . - _ .. . .
Freguency- alias,~ng, will occur .if' the image is' not prapexly pre-filtered and likewise, ~.f the'zsrc--padded' xeconstructl:on is;nvt adeguately post-filtered; F~.g. 15~
'shows that a tuo-dime»sianal fiit~r w~.th 4 diamond-shaped pass-band that can be used fos both. the pre- a»d post=
filter. It removes anly,thase specxral~compo»e»ts outside ..
the se»si~kiv#~ty range of.,the .human ,visual system. pictures with the same subjective s~ual.ity of the or3.ginal image can, therefore, be, reconstructed from a diagonally-decimated image.
Referring to Fig., 1b,. there is,, spawn a blank diagram of an embodiment af'a form of thg invention which uti~.izea pr~.nc~.p~,es dust set forth. Twc~ sources of video s.~gnal, .1610 a»d iGSh are provided, as prerr~ously described in aan~unction w~.th the desc=iptinn of f'~:g. ~ . ~e si.gnals are harizonta~.ly lAw-pass f3Ztered and a»alorl-to-cii93x$1 oouverter as ~epres~ented by the blocks 1615, 1620 and 1655, 165D, respectively (see axso sketches ~.615A and ib55A). The outputs of ths.analog-to~-digital canvexters'are respecti.vel.y two-dimensiorial~.y'low-pass filtered, such as in the manner previously described. as represented by the blocks 1625 and 16b5. The resultant ~ssgecttve two-dfmeresional spectra are rep~raaented .in. sketcttss i62SA anQ 1665A~ Tile fl-lured 3.magQs are then d~aci.mated, i» the Manner previously dgs~cribed, apt represented by the blocks,l634 and 160, respectively. The pattern of pixels selected by the decimators are illustrated in the diagrams 1631 and 16'i1, respectively. in xhe present embodiment, only the selected alternate pixels (as .illustrated) are passed by the deeira,~tors, sod this ears be implemented, far example, by using.vo~aage--controlled gates as illustrated irt Fish. 17. zn pe~xticular, li»e and pixel information provid~d to the controlled gate deteria~.nes whether the ;pixel. 3;s passed by: _ the decia~ator.y ~~,tex»at.~vely, a flip-flop, which.. is: set to. .. a different inf.tial~ status aaGh Line, could be utilized. The outputs of decf.matars ~:f30~ and 157p are respectively coup~.ed to t#:me bhse'eompressors~xb35 pcrrvs9iro2zzs and 1675 which. operate:.. to. compress each line of pii~els into respective. halves. of the- original .tine time: Time base correction is very; well .known.in the art, and any suitable time base compressor can -be.utilized for this purpose. - The compressed two-, dimensional spectra of the resultant signals are represented.- in_sketches 1635A:and I675A, respectively.
It is seen,that the horizontal component~.is doubled. A ' w multiplexes 1690;is then utilized to combine the signals during successive time. slots, as illustrated in the diagram 1690A. The resultant ;signal can be stored-andrbr transmitted, as represented by the. block 1695.
Referring to Fig. 18, there is shown a block diagram of an embodiment of a decoder: which can be=utilized to decode the encoded-video signals of-the Fig. 16 circuit. The block 1810 represents the receiving or the reading;from storage of the encoded signal. A demultiplexer 1815 is utilized to separate the signals in the two time slots of each line, and ' the demultiplexer outputs are respectively coupled to time base correction circuits 1820 and 1850, respectively, which perform the converse of the operations that were performed by corresponding time base correctors 1635 and 1675 in the encoder. The outputs of the°time base correctors are respectively coupled to pixel storage buffers 1825 and 1855, and these may comprise, for example; half-frame stores. The outputs of the buffers-are respectivelg coupled to zero paddsrs.,1830 and 1860 which operate to insert zeros at diagonally alternating pixel positions, as represented in the diagrams 1830A and 1860A, res3pectivel.y. A circuit which can be utilized to implement-the padders is illustrated in~ Fig.
19. In particular, a~multiplexer can be utilized which receives,. as inputs, the pixels output.froia the buffers (1825 or 1855) and a-signal representative pf~a zero level.
Selection of the value to be-pa~sed..by-the'padder will depend on the line and pixel information input'to-the- multiplexes, ' in accordance with the illustrated altetiiating=- pattern. The outputs of padders 1830-and:.1860~are :resgeetively coupled to ~ ~ .~ ' two-dimensional diagonal low-pass -filters-w1835 and 1865,' ~vo stiis~z~ - ' ~cr~us9~iun2~ . _... . _., xe~pectively.- which, ~s previausly.,....ae~cribed; ap~srato~ tov _ .
remove the alias,inq~ componerita.v They. outputs of~~ the' filters"
are coup~:ec~ to cifgital.-ta-analog corivsrtsrs 1840 and 1~7E1, respectsveiy,, and then carp be analog ~;ow-pass filtered lharizontaily), if desired. and diSpleyed andr'Qr recorded, as represented by the biacks'1843 and 1875,. respectively. ' 'v Referring. to Fig. gIl.:, l:hexe #a shown an embodiment of~ a .
form of thB. invention wherein two. video. si.grtals can be combin~d.; on a s~n~~.e Gha»ne~~ w~.th reduced bandwidth ' reriuirsment, aaalag pxacess~.ng hein~ utilized in this '' .
embodia~e»t. As berfore; two video in8ut signals' are received from vidcp.signa3 aos~rces (2pi0 r~tsd 2050) . The present inver~t~.or~ ( in both dig~.tai .arid anaToq prflcessine~
ia~plemert at~.ons ) many sometimes be practiced without initial two-dimensional diagonal low-pass filtering, since most rsatura3, images have ~.itt~.s spectral energy to began with ~,n the high diagonal freguency region of the spectrum. The embodiment of Fig. 20 is an example of praaessittg ~r~.thout such prefi3.tering: In the embodiment of Fig. xg 3t is assumed, also as an examp~.e, that the video comprises ~.nteriaced video. Also in this embodiment, he two-dimensional modulation, is implemeht~d by separate vertfcal asu~ riorizonta~. e~odulati.o». In part#,aular, the b~.cck 2D~a represents the vertical madulat~.on. For an intorlsced display, al.texnate lines occur in successive fields, so v~rtical modulat3:on can be obtained by invertl.nc~ during every ether field. A multiplexar ZOa~r which is contralied oa a lane 20218 by a signal des~.vwd fro~a fie~.d ~aynch~aniaation, selects either the video receiaed via an amplifier ~p2~ or the video reoaivec( vi,a. #.nverting ampli:fiar 2023. The re~uxtant "a3te~nafie line inverted" s~,gnal .fs used to drive a double balance$ modulator (txhich implements the horizontal.
modulatiort,comporent), the mixing freguency (f~) of which, received on line., a~I30B, : i.s selected as th~ maxi~au~n pass-band fragWency, for ax~upls' 4.~;M#~z.. tn oxder,:'to maintain the unipolar. nature o~ the, video, a bias equal: . ~o half the' . ~ w maxf.muta unipola,r video level r i~ subtracted prior to encoding .. ~ .., .~»~ PCTlUS91l02228 by difference_'circuit 2005;: and is ~ then ire~iiis~erted after the ' ~ ~~
modulation., process, ~ by.. summing circuit '- 2050, ~ v The modulated ' and unmodulated.video-signals are added by summing circuit 2060, low-pass,_filtered (block 2070),and'-recorded and/or -' transmitted (block 2080). ' ~ -Referring to-Fig: 21;. after storage and/or transmission, the combined signal-::iswreceived andlor read~from storage (block,21I0),-andv can be decoded.to~recover-thew original signals. The combined signal is~coupledwto=a~-two-dimensional diagonal- filter _2120 to obtain the~~lo~a two=dimensional '.
diagonal fzequencies on a linev2020C, and the high two-dimensional~frequencies on a line 20208. The diagonal ..
filtering may be implemented in analog~fashion or by converting to digital farm, separating as above, and reconverting. The. signal 21200 can be recorded .and/or displayed, as represented by block 2180.v The signal 21208 is two-dimensionally modulated-,. in one-dimensional stages analogous to the encoding process of Fig:- 20. In particular, a multiplexes 2140, which is controlled on a~line 2140B by a signal derived from field synchronization-; selects either the output of an amplifier 2130 or of inverting amplifier 2135.
The result is,then passed-through a double-balanced mixer 215 0 which receives, as its other input, a signal at the same frequency as that used at the transmitter;w i.e, a 4.2 MHz signal in.this~example. Bias can be suitably added by summing circuit 2160 and the resultant signal stored and/or .
displayed, as represented by block 2170.
The high frequency diagonal portion of~the two-dimensional spectrum can also be utilized to carry components of the primary image (for example, high definition ' components, color components, additionalv image area to modify aspect ratio or for other purposes, ete.)~, or to carry other information such. as data, still pictures, audio, etc: Fig. 22 is a block diagram of a high definition televisioru~system which two-dimensionally modulates the high''definition telev.ision;system which two-dimensionallyv~modulates the high -definition components:~into. the high frequency two=dimensional Wt.~ 91/15429 , ... pC'ftr1S91tQ2?.28 ZZ
dictganal regf.on.~ ' A_soWroe c~f. hir~h dsfinit3an (which- eon' be ' ..
considered as: en~ompass~.ng- a signalw w.ith~..substantiaily higher ~ . ..
dafin.it.idn. than current cpnventional television video) 2205 is provided. .The.outp~~,thexeo~;.is,ccupled.to the positive input ' of a difference a rcuit 220, and to a block 2210 which represents twA~,dimens~.o~tal low--passwdiagonal filtering of the , high-definition v.ideo,.siqnal. ~ This fr~itering may bar i~tp~.emented, fo,r ~examp7.e,. ,as descFibed above by converting to digital form and iptpiementing the filtering a$ described above. The two-d~:mensional ,spsctrua~ o~ thewese~ltant signal is represented at 2210A, and .~t~ is seen thax~ the original.
horixonta~.,;and vertical. freguencies, at half the high dafinitiatt sampling raga, axe reduced to within f.he diagonal hand at the .lower frec,~uency standard definition two-dimensional frequency cutoffs. The output of the fi~:ter ~21p is received by decimator 2215, which operates, irt this embodiment to decimate every other vertical lire and retain every fourth horizontal pixel. The pattern of decimation is illustrated~in anothsx park of the diagram at 250, and it wi~.l be understopd that thla decimation c$n be achieved by the decimator preuiot~sly 3l:l.u~strated, with the line and field inputs causing refection of the pixels to be retained irt accordance with the desired pattern: The resultant relative imago density is illudtraterd irt the diaqram'at zZOp.''Atter time base correction .in the manner previously described, the output a~ the dee,imatox~ i$ coupled to one input of multiplexes 212p. The output o~ the t~u~.tiplexer i~ coupled to a zero padder 2210 and to a storage buffer for storing detai3 coe~f~;cient~3. The output o~ zero phdder 2260 reconstructs the pattern shown at_226OA, using a padder of the type previ.ous~.y i~rlustrated. This signs?: is coupled to a standard definition two-dimensional .d3ac~s~na~. , low-gee Titter 2260, ~rh~3ch operates in the same . ~aruser , as ~~.lter 22111, and produces a P$atrum as iLiustrated at 2265A, without;aiiasing. The output of f.i~,ter 22b5 is one input td a summing circu~;t a2yD.. Tha other .
muitip7.exsr ~outpu~ , ih coupl,ed xo detail coefficient storage v buffer a2ao whoBe output ts;cq~pi~d.to. an inv~rse co$ine r. l.. ~ '~ , awI ..:.

WU 91/15929 '-. ' PCT/US91lOZ228 transform circuit . 2265,-: which: may be. a; chip of the' type--described in the above referenced Canadian Patent Application Serial No. 2,079,318. The output of circuit 2285 is the other ---w input to summing. circuit 2270..~The output of summing circuit 2270 is, in turn, the negative input to difference circuit 2207 whose output~is coupled.:. to cosine transform-circuit 2135;
and the output. of .'this circuit: is coupled'to'a detail' component. selection circuit 2237.. The circuits 2135 and 2137 may, again, be.of the.type described in the abovE referenced Canadian Patent Application Serial No: 2,079,318. The output of circuit 2237 is time base,corrected (block 2240), and is then the other input to multiplexes 2120. In operation, this circuit uses (as an:example), selected transform components in the time slot made available by decimation. The decoder, after demultiplexing,. can utilize circuitry in the dashed enclosure 2250 to recover the components of the signal and add them to obtain an HDTV output .. It will.be understood that the same principles could be applied for non- transformed detail components.
It will also be understood that the two-dimensional modulation of video into the high frequency~diagonal portion of the spectrum (for subsequent recovery) can be utilized to reduce interference between signals, for example in a co-channel situation. Fig. 23 illustrates transmitters A and B in separate broadcasting regions, and a receiver which can receive interfering signals from the transmitters: If one of the transmitters is utilizing a two-dimensionally modulated video signal (for subsequent: conversion, as described above), perceived interference will be substantially reduced.
It is.known that pictures can alternatively k~e represented in terms of their spectrah content. The discrete Fourier transform.of~ an entire image having N spatialv pixels will contain N unique. frequency components and therefore generally requires the..same transmissiorrwbandwidth needed to send spatial. image information.-:: In°: the' embodiment to de described.next; spectrally transforming.thevaugmentation detail can provide advantages,;for example'because the lower W4 ~lr~s~z~ . . . pcTrus~~io3zZ~

2~
spectral components, usua~,ly, sent. in- the - conventfonal channel need not be- dupi~.ca~ed,,,and~ the txansformeid augmentation comgane~ts. can be dynamical~.y. selected in a ma»ner which can better satisfy the needs of this human visual system, g~.g.s, ~9~
and 25 illt~sx.tate exemplary band divfsians of the Ffq. s spectrum into_twa arid fouW binds, respectively. In Fig. 25, tha bands have;app~oxima~ely equal:spectra~, order, and ire numbered~in descondirg: ordex cf v.iaua3~prioritx: A tour band , augmentation tech».ic~ue ~is set forth ~ i.» the embadilnent to be d8saribed, but otheac band selections (both as to the number of bands ahd the spectral shapes and apportionments thereof)~cett lie employed. , .
Refexxinq ,to F~,g. 2b,. there is shown a b~.r~ak diagram of a» encadex ~.3~ ae~ordance with a further embodi~sat of the invention, and which cart be used to practicevan embod~.ment, of thQ mQthod,of the invQntian, The enaodesr may be utiiiaed, for example, inthe type of system ille~s~rated in ~'ig. 1. Tha use of codes or synch"rorii:zing s#,~nals, or of othex means for iaentifyzng car synehxonixing channels, mey be ix~ accordance with the teachinc~~ of 11.S. Patent No. 4,8flD,~2fi~ or other abovereferenced patents ar publications:
The high definition videp signal ~a coupled to a complementary a-aitnen~s.iana~. sgatfal filter 910. The f3.lter, described fu5tther in conjanatipn.~"~ith b'ig. ~~, ~apetatea, inter alia, to separate the received ~tzTV signal into a conve»tional resvlutlo» video sigt~t~xl a»d $~dekail signal. The encoder spatial filter 91o can be 1» xre form of two fine d~imerisional low pass finite impulse respnn~se (FZR) f~.J.tars ~ one operating in the vertical spatial direction cascaded with orte operating in the hor3xo»tal spatial direction. The cascading operation acts to Convo~.ve the vertical: anci horizontal spatial images.
The two dimensiana7. spectral xespo»se is ebta3ned by , multiplyi»g the responseh p~ the tWO filtars'the~eby providictg a rectangular spectral wi»dow in ttie two di~enstonal freguency da~aair. Raferanee can, oleo be made-: to the: ah~pverefexenced U. S . Patent No . , ~ , 6~ ~, 34 ~ ., ( Of : caurae, if sepaiate hi gti ~
and~~'' low rssolntiori cameras ox.othex.sdurces. uf,video campon~nt wo 9ms9z9 Pcrius~~iozng 2 5 : -.
signals are. employed;:..the_ filtering=.may not be necessary ar may be of a different form:.j-::The conventional--resolution video signal is coupled- to~ awscanwconverter 920 which is utilized to produce.video having conventional scanlirie and element format, for_example 483 visible lines as in NTSC. The scan-converted video;is,..coupled to°encoder 930, f.or example an NTSC encoder. The encoded.. conventional resolution video signal can be stored, transmitted,'or otherwise processed~in -known fashion. A motion-indicative:signal-can also be included in the conventional. video.channei in order to obtain enhanced operation at.receivers~having certain processing capabilities. ~ , ::
In the present_.embodiment, the high definition detail - component is processed using predetermined regions (or "tiles"} of the picture.- In this. embodiment, the tiles are square and there are 8x8 pixels per tile, although other tile sizes and configurations could be utilized.
The detail augmentation channel includes transformation of the detail signal~.using an orthogonal transform into the frequency domain, such as by cosine transform, as represented by the block 950. The particular transform bands to be utilized (transmitted; in this case} are dynamically selected as a function of motion for each tile,~as represented by the blocks 960 and 970, and described further in conjunction with Fig. 28. in accordance with a feature of this embodiment., the -particular bands selected are determined, for those tiles at which there is currently no. substantial motion, as a function of the motion history of such tile. In particular, the longer the picture information at the tile remains substantially the _ same, the more"detail information'is provided (transmitted, in this case} for the. tile, until all available detail has been provided. In the present embodiment, the motion indicative signals are generated by detecting motion of the transformed tiles (e. g. Figs 28 and 30); although-it will be understood that motion could,be detected~using the detail component or~
ether suitable video representation. The selected band components (if any)~,for each_tile, and motion status signals W~ ~~~~~~~ ~~~~;~
. i are coupled, ..,in ;the, pre~~,n~: embod;ment, via mu~ltiplexer 9Qt~, . _._..
._ ..
td, the transmitter ,andlo~: .storage me~lit~m. .
F~.q. 27. illustrates a preferred- arrangement for the w -encoder scan convex~s,ion.: The low ~ or conventicana~l ) resolution output of the.2p spdtial filter.~9lQ (e.g. at 1125 tinac) aan be coupled tQ a first dawh, scan converter 2'l20 that convexts to conventional 525 line (NTSC) format, for coupl~.itg to the conventional. transmtt~er (e.g. alp in Fig: l). An up scan convertor 270, that is identica)..to one .hat will be employed .in the recei~rex's decoder te.g. block 31I~ of Fig. 31 b~low), can than bemused to up-ccpvert.back to.~.125 lines. The result can be subbrocted tram the high resolution comp~onerlt, using di~ference.circuit 2?aA; to obtain the desired high-pass detail components. A domper~ating delay 2710 can be employed, as shown, and throughout the description hereof it w~11 be understood that ,ashy neceas~sry competisaxing 4eldys can t~
emg3.ayed, as Itnown in the art .
Referring to Fig. a8, thexe is shown, in further c~e~ail, a block diagram of the portio~t of the encoder that procasa~s:
the detail s~.qnal for ccuplinq to the auc~mentati,on channs~, and which was repre~cented in Fig..~6 by the blocks 950, g60, 9~0 and 980. A demult p3.exar IIpS, line gifos x.110 and multlplexsr ~lli5. serve, in known fash3.on, to put the cletaii signal in appropxiata format to be recafved by the ~,ransform chip II~O which, is the present embodime~lt may be he INMOS I~dB ..
A7.21 Discrete Coa3.ne Transform chip. the tt~nsform chip 1120 receives a pixel cloak, which ~,8 also received by a coefficient cnuntar ilZa.. Th~ txan$~arm chip alas receives a ..next tile" irtd.~eatinn, which can be derived as the pixel count modulo b4r and this indication ~s a7a3o coup~.ed to the coefficient counter.l~Z5 and_to a tile po$ition counter 1130.
Ths til~ position counter ~.~.3o a7,sa reoefves a "next frame~~
.indic~tt3.on. .The aonf~icient counter 1125y keeps track of which of theft=anaform coefficients ~6~ of them; fcsr this exarn~ale) is .being output from the tr$nt~fo~n 'ohip I120, ahd the . ' tile position aountex,113a keeps track of:tho position (x,y)~
of the tile being pxocesssd, the t~.les typically being WU 91!15929 - -' PCTlUS91l02228 processed sequentially, a~ row at. a~ time'.-=~-R=look-up table 1135, which; may.__be a. read-only. memory- (.'-'ROM!'-j.,=-translates the' coefficient identification:information'from counter 1125 to part of an address at which-coefficients. from transform chip II20 are .to be stored,in two band storage RAMs 1150~and 1155, respectively._.;The_band storage RAM.1150 is. dual ported; that is, it;.can..be.accessed at either an input port, using and -input address;.or, independently; at an output port using an output address (which,".in this case, is--.obtained-from a tile control processor 1175). The. band storage:~RAM 1150 is used to store the band. components; that is, the groups-~o~ transform coefficients that. comprise individual bands'of the frequency spectrum, such, as, the four bands labeled 1, 2, 3 and 4 in Fig.
25. In.the present.example it.is assumed that-each of the four bands, 1-4, has five coefficients, which that means that twenty of the possible sixty-four coefficients are actually utilized to represent the shaded spectrum-shown in Fig. 6, the other forty-four coefficients approximately representing the remaining regions shown:in~Fig. 5, and.not being-necessary, as first explained above. As will be explained, the appropriate band component will be read out of the band storage RAM 1150 to a multipl,exer 1165 which also receives indications of motion flag storage status from the flag storage RAM ii70.
These signals are applied to a fifo 1185 and eventually to a transmitter (or storage; as the case may be); such as transmitter 275 of Fig. i.
The other band storage RAM 1155 may be-single ported, and is used in this embodiment for the purpose of motion detection. The RAM 1155 receives the same coefficient information as the RAM 1150, but each..time it receives a new coefficient, it reads out the corresponding coeff~.cient from the previous.frame (thatis, the~coefficient stored at the address to be occupied by,_the corresponding coefficient of the current frame),.so:,that motion;detection can be implemented by the circuit'1160,.which:is.,shown.:in:further detail in Fig. 30:v Referringnmoiaentarily~to"Fig:-..-30;.:-the: current=band component -( from transform.chip l I20.. -_ dig: ,: 28 ) and ~ the y~corresponding wo 9tns9~. . . Pcx~us9~~ozzzs previous trams.; bandL cPt~ppnent.~ ( from. harid~ storac~a~ ~~ 1155 -Fig. 28) are..,receiv~d, hlr;,a:di~terence, circwit 131Q whose output ins coupled to . an abso~.utev vaZu~e.. c.~rcuit 1320 . An accut~u~.ator 1330 accumulates the' total of the absolute values of the differences for each t~.le ,the xesett3~nq and road-oWt of the accumulator being enabled by a next tile indicattan), and its output. is compared again$ a predetera~fned thiceshald level. If the. t1'~esho~.d is exceeded, a fiction tntl~.Cation signal is output..f,rom camparatar 134Q. As seen, i.rt Fig. 2g, this signal, '.is received by flag storage RAb! 11?t~, which ~is~
alao a dual ,ported RA,~.-.~; Yn the pre:e»t embodiment there are five ~neti~rn; ~~ag, atatuses,. yea. fvl~ows: motion flag statusvl indicates atotiun, and that the first of the~four bands should be sent; motion fxag.,status 2 ind3.cdtes thht there has been nn motion for. orte frame, and that band 2 should be sent; motion ~lag status 3 t»di;cataa theft thrx~r hens beep 110 mot~.on for two frames, and that~~band 3 should beW ent: motion flag 4 .indicates that there has been no motion at the tile for three fra~ses, and t~tat band . $ . should be sent; and motion f~.ag status U lndicates,that there has been no-motion at the tile for four or more frames, at~d that nn spectrhl infotmatio» should be sent (all four bands havipg alreadx been sent). The tile avntrol proaesacrr 13.95, which may ba $ny suitablsr microprocessa=_or part thdrgof, dedioated logic, or integ=atod circuit. acntrols the flag storage RAM 1170, as will be described. yig. 29 is a flow diagram of a routine for control~,xng trie tile coritrAl proceasar 11?5.. Thd block ~~p5 represents the init$aiizi.nc~ of indices ~, and j ~rhich represent the position of tha tile in the two df~tenaf.onal tale array. A
sync coda is then cent (block X210). as indicated in Fig. 2~
by art output of, ti~:e. control processor xav a .sync code generator xiBO which outputs a s~raa aiqnal to ona input of an analog~muxtiplexe~.l=9~. The motia~e flag status for tile 1 is then., aeAt..to the fifo. 11a5. from the fl.eq~ stcrage~ RAM
t i~0 via the multip~,exex . l ib3, as rsprese~tted ~ by the block '~ ' ' . ' 1215 . This . is . ~mpletpe,~t~d, by ~nd3.ng the'~add=ess ~ ~.; ~ j~ ~to the outpWt port of the.fl~g s~urage x.1170; and'enabl3:ng the ~' s ' WV 9/15929- ~ PCT/US91/02228 multiplexer. 1165 to. pass._ the: flag status-information~-to the =
fifo 1185. .. Inquiry,. is...then made (diamond 1220) as to whether'- '- w the motion flag status is zero. If so, there has been no motion at this tile, for-at least the last four frames, and a1,1 of the band information has already been sent, so na spectrally information will-be-sent for this tile. In such case, the diamond 1250 is entered directly. If the motion~flag status is not zero, band information will be sent from the band v storage RAM 1150 to.;the,:fifo 118:5, and this is implemented by the loop 1243. An index k is initialized at zero, as _ represented by the block 1225. The index k is used to identify the five components of each band. Each time k is incremented (block 1240) the band component at the address [i,j,flag(i,j),k] is loaded from the band storage RAM 1150 into the fifo 11.85 via the multiplexes 1165, the multiplexes being controlled to load information during this phase from the band storage RAM,1150 by the line in Fig. 28 labeled "select flag or band component". The address is seen to include the tile position [(i,j)l, the motion flag status [flag {i,j) - which determines the band to be used, in accordance with the above-listed rules), and the component of the band [k]. When the loop 1243 is complete, the inquiry of diamond 1235 will be in the affirmative, and block 1245 will be entered, this block representing the updating of the motion flag status in accordance with the above-listed rules. Stated another way; the motion flag status at the address (i,j) is accessed in the flag storage RAM i170 and is incremented modulo 5. Inquiry is then made (diamond 1250) as to whether the last tile of the row has been reached. If not, l is incremented {block 1260), block 1215 is re-entered, and the Ioop 1263 is continued~until the row is complete" Index l is then initialized for the next row, and inquiry is then made (diamond 1255): as to whether the last row of tiles has been considered. If not, j is.;~incremented (I2?0), block 1215 is re-entered, and the loop l2?3 is.continued until all tiles v have been. cons dared, whereupon: the: block 1205 is re-entered for processing of.the tiles of:.the next frame: "' CA 02379330 2002-04-16 _ " _,__, WO 9y115929 . FGT/LtS9ll The., ~,nfor'm3tion.: i.n.:fi.f.Q .1195 is -clocked aot preferab~.y at' a fixed: dock . rh~.e for a given. Pox~ion of the transmitter frames, ta. di9ithl-to-a»alog convertsr~ 1190, aRd-'then- to v _.
t~cansmi.tter via analog multiplexer 1195. The t~u3tiplexer 1195 ~.s cantrolle~,d, to pass the output of di:gixal-to-~aaaLog cp»vertex~ 1190, except when a sync code is being a~aplied, as previously descr3;bed.
R$f$xring to Fig. 31, them ~.s, shown ~! Dlc~Ck diagram of an embodiment. of the decQde~. In general. the decoder receives a convehtio»a7. channel (e.g: 525 l.itte input) and converts .it f.a a =s~,atively low resolatian image at the same number of titres as tha HpTV to b~ llt.tme3tely displayed or recprded. Lnfermatfott .fr~am th$ augmentation ahannol is received aver a separate path. aRd is p~oces ed and stored n a spectral memory (block 14~a in Fig. 31). The output of the epeatraJ. memory ,~~ processad by an #nvers~e translaXm chip (block 3160 in Fig. ill, for example a» inverse cosies transform for the present embodf,~ptetet, to crbtair~ image detail.
which i.s added td the re,~atfvely Sow resolution scan--co»verted imago obtained from the aonventiona~, resolution ehd»ne~..
This is impleme»ted by the sinner 3x70 ~.n th~ Fiq. 33 embodiment. ~'h~s signal is converted to analog forip and can c3~en be coupled to a sWitable HD~!~V display and/or record3~ng means.
It can be i.»idally ;toterd that. the timing rslat3o»ship between the conventional channel, and the augmentation channel is non-cr~,tical, and aan bra txeated independently. ~Th.is-is berceuse small delays k~attresn the low xesolul~i.an components and ttie detail components will generailly not be noticeable to the viewer.]
I» the F,~g. 31 decoder emlQOdi~nent, an augmentation input processor 3140 aad a spectral-to--detai~i canvartar contxel proces'sox are employed. Theae ~unctiuns can be i~tplemented, for example, by sharing a slngae rnic~oproces~r~~ os with se~aaratv microproce~eo~sr o= py ded,icated logic or int~grated .
c.ircu.it means. . The roatfne ~gor controll~.ng the a~g~enta~ian input processor is deacribed.~n:con~unction with the flaw ' WO 91/1S929 , - PCT/US91/02228 diagram. of .Fig::.. 32,.: and. the-. routine...-.for~~implementing the ' spectral-to-detail.converter..control~processar is described in conjunction with the- floww diagrams of Fig. s 33' and 34. ~~ The' - - ~~
pixel stream.output;from the inverse transform chip 3160 is ' coupled to a fifo circuit 3165 which is described further in conjunction with Fig. 35. -Turni-ng..now to the.detailed-operation of the Fig. 31 ' embodiment,-the. video-from the conventional~receiver portion is coupled to an--analog-to-digital converter 3105 and then to a scan converter.3115, the scan converter also receiving the necessary_synchronizing information which is extracted by sync detector 3110, and is to be used by the scan'~converter 3115 and the spectral-to-detail converter control processor 3155. . w - The augmentation signal from the augmentation channel receiver portion is digitized using analog-to--digital converter 3120, and the augmentation~sync~is extracted (block 3125) and coupled to the augmentation input processor 3140 which, if in. the. middle of an operation, will asynchronously terminate the process in progress and-return t:o the start of t:he frame sequence in order to re-synchronize. The digital data output from analog-to-digital 3120 is coupled, via a demultiplexer. 3130, to the processor 3140 or to one input of a ~nultiplexer 3135, the,other input of which is a logical "0".
As will be recalled, the data has a flag status indication followed, where ,appropriate, by spectral components, and the augmentation.ingut processor 3140 operates to control the demultiplexer 3130 to couple motion flag status indications to the processor 3140 and spectral component, data; via multiplexer 3135, to the dual ported spectral data RAM 3145.
- Reference can be made at this pofnt to the flow diagram of Fig. 32 which illustrates a routine for controlling the augmentation-input.processorv3140. The diamond~1502, and the associated loop, represent the. waiting for detection of sync, whereupon;~the;-,tile position indices, i,j, are initialized (block 1504).. Inquiry is.theca madev(diamond~1506) as to whether the motion flag status is-. 0. ~'=- If . so, no- spectral data will follow.the,motion status flag, the next portion.of the wo 9m5~x9 ~ rcrms9noxzxs a rAUtine is bypassed..: ~~dJ:~~~.. diamond 15?5 ~ is en~ared~ diroctly°:
. . . .
xf tt~e. flag status, is. r~o~..'~._.irtguiry_ is made. (diamond x.508) as to whexher the ~lag status is 1. ~If ~ not, the block 1560 is entered directly. If;sp, however, motion at'the cur=ent tile is i~diCated;, dnd the h:I9hw resolution bands f.n~spectral data RAID! 3145 ~thex~#ore contain obsolete values. In such case, the next portion o~ .the ~ roW~fne is utfl~.zed 'to. remflve ~khese obsolete va~.ues , dram. hAb~ v : 315 e~nc1 xo ~.°i:nserx a"0"
via multiplexer 3135. In;, particular,. the bandvindex is initialized at 2 (b~.ock.lSID),~, it being understood that it is not necessary to zentave ex.istinq data from band l of stcraga, rinse the subseguer~~ .. r~pere~k~an, will cause inse=tiolt of new spectral data into bard ~. (t3ie motion flag status being 1 for this branch o,t the routinel. The bandcomponent index k ie then ~.n~.tialJ.zed block 15~;~), and the b7.ack 151, is entered', khis black repxecontir~g the aett'sng of the cr~mpvnent in spectral data RAM 314, at hddre~s-[i,~,b,k], try 0, xnguiry is then made (diamvo~d 1517) as, to whether the last k has beers reached. If nest, k is incremented (block l5xQ~. block 1514 is rs-entered, and the loop 1519 continues until ali band components have been considered. Then, .ingu~.xy i.s made (diamond 1525) as to whether the last band hay been reached.
1 f not, the band index ~.~ increp~entec~ . (block 1529 ) , block 1512 is re-entered and the ~.oop 1530 ~,s continued until all.
bands have been considered. The block 3560,i.s then entered, and k 3s initial.~.xed. Tht received spectra. component will then be stored in the spectral.rlatd RAM at addtest~
I 3 , ~ . tla9 ( i. ~ ) . k ] . dt~ r~~tx'esen.ted by the bloctk 15s5.
Ir~qt~3.xy is rhea made (diamond 1.560 ifs to whether the last k has been reached. It not, k is..inare~tented (lock 1569), and the loop 1570 is cp~tinued until ail component of the received spectral band have been read into their appropr~.ate addre~aes in spectral data RAl~~3i15. The diamond 1595~is than entered (and is also entered directly from the nyes~~ output branch of .
a3amond l5osj: arid ingpiry.is made as to whether xa:t i (that' .
is, the last tiles of, the rc~w~ has been ' re~ahed. If not, .~ is fnare~ae~tted (block 1578), diamohd 106 is re-entered, and the WO 91,/15929., PCT/US91 /02228 Loop 1580_.is...continued until ~he last~~i is~-reached. When this occurs, i. is initialized..to ; begin -a new row - (block 1582 )-; 'and - ~ "
inquiry is made as to whether thewlast~row has been reached (diamond 1585). If not, j is incremented (block 1587), diamond 1506. is. re-entered, and the loop 1590 is continued until all tiles have been processed, whereupon the diamond 1502 is re-entered.
Referring again to Fig. 31, the spectral-to-detail converter control processor is synchronized to the output~of scan converter 3115. When an indication of a start of frame is received by processor 3155, it begins the routine of controlling inputting of spectral data information from RAM
3145 to inverse transform chip 3160 via the multiplexer 3150.
Referring, in this regard, to the flow diagram of Fig. 33, the sync is awaited (diamond 3302 and associated loop), and the tile indices are then initialized (block 3305). A
coefficient index, c, is then initialized (block 3308), to consider all coefficients [for example,w 64 coefficients for an 8x8 pixel.tileJ to be coupled, for each tile, to the inverse transform chip 3160. Inquiry is made (diamond 3310) as to whether c is used (it being recalled- that only some of the coefficients are utilized). If not, a "0" is sent to the inverse transform chip 3160 by sending a command to the control line of multiplexer 3150. [Alternatively, if it is viable to permanently disable the not-used coefficients of inverse transform chip 3160, this operation would not be necessary.] If the coefficient is used, the block 3320 is entered, this block representing the sending to the inverse transform chip of the component in the spectral data RAM at address '[i;j,cj. Inquiry is then made (diamond 3330) as to whether the last coefficient has been reached. If not, c is incremented (block 3332), diamond 3310 is re-entered, and the loop 3335 is continued until all coefficients have been read into the inverse transform chip 3160. When this has been done for the-;current tfle, the inverse transform aperation isw initialized (block 3340, and the "start" line in Fig: 3I).
Inquiry is then made (diamond 3360) as to whether the last WU 91/lar9Z9 . FG'~'/US91/tlZZ2~

tile of the ,. , row , hays, begirt, reaahed . ~- I f ncrt, s is irtcreme3lt~ed (b~.oak, 33~~), black 330$ as: re-entered, and the ioop: 335p is continued until .the row is completed: Tt~e index i is then ' ' initializec!'for the next row block 33fi5), and inguiry i~
trade (diamond 33~9j as tp whether the last ~:ow hay been processed. If not, ~ ~.s :incremented (block 3380), bibck 3308 is ra-entered, and the loop 3385 is continued until all rows of tiles have been process~d. The diamond 3302 is then re--entered : to again away t the sync . " . .
The rout.,fne illustxa~ed by the flow dfagram of F~.c3. 3~
fs used to control the loading of tiles of output pixel data into, and then out af, the f~fa ci~auit ' 3165 of F3.g. 31, t?~e fife circu3.~ boiriq shown ire F#g. 35. ~n the present embodiment there are eight.fifos, x521-35~~, and they each receive the inputs from the 3:»verse ~r$nsfAxmch~.p 3160.
ttowsvsr, only one t~.fo is enabled to load at a time, under cc~atrol of demnitiplexer 35tp. The demult~,plexer 351Q
receives the inverse ~rattsform clock and a fifo select control from processor 3155: 1» particular, refe7crin~ tv this routine of Fig. 34, the d3,am4nd 3~1Q, and the assoc~.ated loop, represei~t$ the wa3tinc3 foxy sine of ~n~ outprat video ~:u be genex~ated~ The complsrtion of the inverse tsansfarm aompu~ation for the Current tile is th~n awaited (diamond x.715 and the c~asociated loflp) , and a pixel ~,ncie;~ is initialf zec~
(block Z7~0): The demuhtiplexer 3510 is then controlled to select the fi:fo for the current pixel Gaunt (block x,725).
Ingu3.ry is then made (diamond 1730) as to ~rhether the last pixel has bs~n rer~dhed. If not, the pixel. index is incxernented (b~.ork 1735), the block i'725 is re-entered, and the loop ~7~0 continues until a:L1 pixels for the current tile have been read into the ~f3.foe. Inquiry is then made (diamond x750) a$ to whether all t3.les have been processed. It not, the tile index is irtcreatented (block 17551, diamond 115 is re-enterBd, and the loop ly5i continues until all. tiles have peen proeessed,.wheraupon~~:he diamond i?i0 is re-entered.
The pixel in~armat~,on iri the f3.fos 1e clocked out under control of demultiplexer 350 whfch rece~.veh the video nut a clock and the l;ne out enable, as seen in Fig. 35. The demultiplexer is-.controlled by the output~of line counter 3550 which receives'thewline-out' enable, and, in the present embodiment, isya modulo_8 counter. The counter 3550 output also controls the multiplexer 3530 to select~which fifo output is coupled to:summer 3170 (Fig. 31):, so that the information is read out a line at a time, after the 8x8 pixel data is read into the fifos 3521-3528. w It will-be understood that the techniques hereof are applicable regardless of the original resolution, and could be used to advantage for bandwidth compressing moving picture video information at any original bandwidth. It will also be inderstood that when a substantial portion of the scene is still (not in motion) for'a substantial number of frame periods (e.g. more than five frame periods~or l/6 of a second), very little picture information will be transmitted [since, as noted above, most tiles wi3i be status "0"]. In such case, the additional bandwidth could be used to periodically send update. information. Also, statistical multiplexing among a number of channels of the type described could take particular advantage of the dynamic bandwidth characteristics of. each channel.

Claims (11)

1. A method for encoding a video signal, comprising the steps of:
dividing frames of the video signal into a multiplicity of tiles;
separating the picture content of each tile into a plurality of frequency bands having a predetermined priority order;
determining motion at each tile from changes in picture content of the tile from frame to frame; and generating signals for each frame that include an indication of the motion status for each tile and a representation of a frequency band for each tile; the frequency band representation for each particular tile being selected as a function of the motion status for said particular tile.

2. The method as defined by claim 1, wherein said plurality of frequency bands comprises at least three frequency bands.

3. The method as defined by claim 1 or 2, wherein said step of generating a signal that includes an indication of the motion status for each tile includes generating status signals that indicate, for each particular tile not exhibiting motion, the period for which said particular tile has not exhibited motion.

4. The method as defined by claim 3, wherein different frequency band representations are selected for each motion status.

5. The method as defined by claim 3, wherein one of said motion statuses results in selection of no frequency band representation for the tile having said one motion status.

6. The method as defined by claim 1, 3 or 5 wherein said step of separating the picture content of each tile on a frequency basis includes applying an orthogonal frequency transform to the picture content of each tile, and dividing the resultant coefficients into band groups.

7. The method as defined by claim 6, further comprising the step of discarding coefficients representative of relatively high frequency diagonal frequency components of the picture information of each tile.

8. Apparatus for encoding a video signal, comprising:
means for dividing frames of the video signal into a multiplicity of tiles;
means for separating the picture content of each tile into a plurality of frequency bands having a predetermined priority order;
means for determining motion at each tile from changes in picture content of the tile from frame to frame; and means for generating signals for each frame that include an indication of the motion status for each tile and a representation of a frequency band for each tile, the frequency band representation for each particular tile being selected as a function of the motion status for said particular tile.

9. Apparatus as defined by claim 8, wherein said means for generating a signal that includes an indication of the motion status for each tile includes means for generating status signals that indicate, for each particular tile not exhibiting motion, the period for which said particular tile has not exhibited motion.

10. A method for encoding a video signal, comprising the steps of:
separating said video signal into a relatively low resolution component and a relatively high resolution detail component;
dividing frames of the detail component into a multiplicity of tiles;
separating the picture content of each tile into a plurality of frequency bands having a predetermined priority order;
determining motion at each tile from changes in picture content of the tile from frame to frame; and generating signals for each frame that include an indication of the motion status for each tile and a representation of a frequency band for each tile, the frequency band representation for each particular tile being selected as a function of the motion status for said particular tile.

11. For use in conjunction with a method for encoding a video signal, comprising the steps of: dividing frames of the video signal into a multiplicity of tiles; separating the picture content of each tile into a plurality of at least three frequency bands having a predetermined priority order; determining motion at each tile from changes in picture content of the tile from frame to frame; and generating signals for each frame that include an indication of the motion status for each tile and a representation of a frequency band for each tile, the frequency band representation for each particular tile being selected as a function of the motion status for said particular tile; a decoding method comprising the steps of;
storing frequency band representations for each tile;
selecting stored frequency bands as a function of said motion status signals; and generating a decoded video signal from said selected stored frequency bands.