CA2140893A1 - Image processing system - Google Patents

Abstract

An image processing system (9) having a transmitter (10) capable of selecting a proportion of pixels (Umn) and deleting a proportion of pixels (Gmn, umn, gmn) of a video field for achieving data compression. A receiver (12) is provided for regenerating the deleted pixels (Gmn, umn, gmn) for regenerating the video field. The receiver (12) having a processor control (26a) for horizontally interlacing the regenerated pixels over a period of time substantially imperceptable to a viewer for relying upon the BETA
APPARENT MOVEMENT effect for providing an apparent resolution which is greater than the actual resolution of the regenerated video field.
Enhancement of the resolution of the video field is also provided by adding random noise pixels into the regenerated video field.

Description

~ WO94/03013 214 ~ 8 :9 3 ~ PCT/AU93/00368 TITLE
IMAGE PROCESSING SYSTEM
FIELD O~ THE INVENTION
The present invention relates to an image pxocessing system particularly, although not exclusively, en~isaged ~or use in reduclng the bandwidth requirements for transmission of video siynals (i.e. for high definition television(HDTV), TV, video and video-phones), increasing the resolution of a video picture for a given bandwidth 0 (HDTV) and in digitlsing signals for storage onto video cassette tapes.
~ The pres~ent invention is able to do this by taking into consideration the psychophysical attributes of the human visual perception system, namely BETA APPARENT
15 MOVEMENT and SUPER EDGIMG ( the Julesz stereopsis experiment ) .
~: ` ; ACP~GROUN~D TO THE INVENTION
The design of the modern televislon has been nfluenced by prevlous discoveries in the cinema industry and are based on three basic desiyn ~arameters:
1. Frame Format - in cinema it was discovered that a frame rate of 16 frames per second was needed to sustain smooth apparent motion. This was increased to 24~ frames per second in order to r~produce sound. Then the rate was increased to half of the mains frequency (25 Hz in Europe and ;~ ~ 30Hz in US~) so as to avoid mains interference.

2. Synchronisatlon - achieved by sequentially ~i scanning video pictures starting at the top left of the frame and progressing to the bottom right on a line by line basis, and repeating the process for subsequent frames.

3. Fiic;~er - which has important effects on selêc~iGLl of the frame frequency. If the frame frequency is ; 35 too low (say 16 Hz or less) flicker results; at medium frequencies (say 25 Hz) stro~oscopic effects are created (i.e. wheels appearing to ~ i W094/03013 ~40~9~ PCT/AU93/D0368 rotate counter to their actual movement; whereas at too high a frequency (50 Hz) the bandwidth required is too large and costs become prohibitive.
In relation to the latter, video compression techniques have been developed to reduce the bandwidth requirements. However, most of these so ~ ions are based on algorlthms which are mainly designed t~-detect and predict changes such as those described ln`~the proposed CC~TT
standard H261. .
Conventional display techniques including teIevision and electronic displays work on the principle hat for an image to be seen by a viewer, the complete image has to be represented by the illumination of the appropriate l5~ picture elements (plxels) in the form of the image. When the~e pixels are lit in their appropriate positions, the human~eye can reconstruct the complete image, preserving the positional relationships of the pixeIs.
; The biggest drawback of this method of display is 20 that for images that require a highe~ level of resolution to be~ properly displayed (i.e.~ graphlcal images, including graphical company logos like Co a-Cola), a display consisting~ of a higher number of pixels is required.
Slmlla~ly,~ ~or~ dlsplays that scroll across the display 25~screen~ to~appear to be movin~ smoothly, large numbers of columns of~these pi~xels are required, so that the resolution of the image t as it moves across the screen, can be preserved.
However, it was discovered that for moving ~objects, the human vlsual system is able to function ; perfectly~on signlficantly~less information than previously was assume~ required. An e~ample of this is an effect known as the BETA APP~RENT MOVEMENT or picket Ience ~flect, where an observer ~i ws a moving object through a pic~et fence that re~eals no more than 10~ of the object at any instant, yet the viewer is able to s~e the object perfectly. It is the mo~ement of the object behind the picket fence which is t ~94/03013 21~ 0 8 9 3 plcT/Au93/ao36s fundamental to the viewer being able to see the entire object.
Most drivers may notice that when moving (even at low speeds), the rails of bridges, low density vegetation, and the common picket fence become effectively transparent.
Similarly, objects become quite visible through the spokes of rotating wheels, or the blades of rotating propellers.
In these cases, up to 90% of an objec~ can be obscured by a moving obstacle, and still the full details of the object can be resolved The converse also applies; a moving object can be shielded by a fixed a barrier which is ~ up to 90% solid and~the full details of he object can still be seen.
This is not due to the "persistence of vision", lS which in the past was ascribed to retinal retention. It is in fact due to the "BETA APPARENT MOVEMENT" effect. This e~fect is the correct reason for humans being able to interpret and connect different images into a sensation of motion. Converse1y, the integration of partially presented images, scrolled over time, is al~p mxde possible by the ;same principle.
The BETA~effect was used in Australian Patent No.
493435 where, instead of relying on large numbers OL pixels in order to display graphical images of high resolution, the BET~ e~fect was simulated and the human visual system was tricked into perceiving a resolution much higher than was actually bei~g displayed.~
In Australian Patent No. 493435 instead of using a display fu11y covered with plxels, a relatively small number 3D of columns of pixels, spaced relatively far apart, were u~ed ;~(like the gaps in the fence that are the sole source of information). If the image is then moved across the display, either left to right or right to left, at a suitable rate, it was discovered that the human visual 35 system would "fill in the gaps", giving the viewer the ~
;perception that the viewed image was being displayed at full -~resolution. The viewer thus would not notice that the image W094/03013 2 ~ 40 ~ 3 PCT/AU93/00368 at any yiven instant contained only a small fraction of the complete image. The process of deleting a significant : portion of the image and regenerating it over time for recombination by the human visuai system is herein referred : 5 to as "IMAGE DEPLETION" - the image was~EPLETED.
There are two main adva~ ages to using image depletion. First, a display that~`only requires a column where every tenth column should b`e results in significant :~cost savings ln the production and operation of such a display. Secondly, if this display is fed with information ~through an existing information network, significantly less ;~ information has to be transmitted for each "field" of the ~isplay to represent an image. This in turn has a two-fold advantage of either transmitting at a higher field rate than would be possible for the full non-depleted image, or transmitting on a network of narrower bandwidth than would otherwise be required for the non-depleted image.
The main problem with pictures, when stored and transmltted, is that ~hey have a lot of data. Typically, an 20 :a~erage television picture needs, ~hen digitised, ~uffers running into millions Qf bytes. A PAL standard colour television picture needs three monochrome components, each o some 520,~33 bytes.: This image is assumed to have 625 x 625 = 390,625 pixels each with an aspect ratio of 4 x 3, add up to :520,833 bytes. The total, for a colour picture, e~uals~three times this number, or approximately 1,562,500 bytes. : :
::: To send such a picture on a TV channel, the ; ~:`` channel would need to be able to handle a rate of 1,562,500 ,~ .
30~ x 8j or approximately 12~:.5 megabits, 25 times each se~ondl .
Thls is equivalent to 12.5;x 25 i.e. 31~.5 mega bit/sec. . :
The challenge: is to reduce this data rate to the .
capabi~ities of ISDN network communication speeds Ol the : order of 64 kb/sec.
The first step in handling such a huge task is to ~
implement G system which can compress the pict~re, free it :
~;
of any redundant information, and further reduce ;

~ W~94/03013 21 4 0 8 9 3 ; P~T/AU93/00368 :, . .

transmission times by sending only the field and block differences rather than the whole field information. This is ~he intention of the proposed CCITT H261 JPEG/MPEG
document covering the Discrete Cosine Transform (DCT), 5 Motion Estimation Prediction (MEP) and ancillary algorithms. :
Some of the reduction in transmission bandwidth requirement can be handled by these algorithms, but they have finite limits. We have discovered, however, that enhancement to picture quality, as well as reductions in transmission bandwidth requirement, can be achieved by improving the machine-to-human interface by the use of Psychophysics.
Prior art video systems (with the exception of Australian Patents 493435 and 573024) have been designed on the basis of a machine-to-machine interface and have not taken the special needs of the human perception system into consideration.
THE ROLE OF PSYCHOPHYS ICS
In order to compress a video i~age for low : 20 bandwidth, we use the potential: of~ the visual perception : system of: the observer to fill-in missins details. This requires a series of techniques which rely upon certain unique featurés in the manner in which the human brain functlons. Hence, we can produce a considerable difference betwe~n actual and perceived resolution and so enhance the : machine:-to-human interface.
` :` :
These techniques are chosen from:-1. The BETA APPARENT MOVEMENT effect (the BETA
` effect);
, ~30 2~ Two dimensional interlacingi ::
3. ~ Statistical fill-in; and, 4. Super Edging.
:~: The latter two of which constitute a VISUAh EN~AN~Ei~lri~T
:~ I
: TECHNIQUE (VET).
~;:: 35~ The present invention relies on these techniques j ;
to enable considerable depletion of an image without causing signi~icant loss of visual intelligibility OL the image.

~ .

W094/030l3 ~ ~089 ~ ~ ~ PC~/AU93/00368 The BETA effect is used in a scanning and raster technique which works equally well whethex the video imaye is stationary or moving and is referred to as a DEPLETIOM
OPTIMISATION TECHNIQUE raster (DOT raster)~. The DOT raster adopts the BETA effect described in Australian patent 4~3435, except that, instead of requiring the video image to move with respect to a plurality of spaced apart stationary columns of video elements, a raster of columns are moved backwards and forwards :with~xespect to a fixed or moving video image. :Hence, the pixels are moved backwards and forwards to create apparent movement - even in still video :images. This is equivalent to taking the picket fence analogy of Australian patent 4934~5 and moving the picket fence with respect:to the video image instead of moving the I5 video image with respect to the stationary picket fence.
The:~ef:fect can be simulated by closing one eye, spreading ; your~fingèrs slightly apart and waving them in your field of view.
It~ is the mo~ement o~ the column raster which 2:0~ enables the~present~invention to appl~ the~3ET~ effect to a stationar~ video: image. Also, a row raster can simultaneously be used to provide a horizontal depletion -to create :a two:~dimensional interlace raster, as shown in Figure ~la~
25 ~In fact the raster movement of the pixels does not have to~ be~:limlted~to llnear~movement in rows and columns, but c~n~ be~:random ln~ Z:dimensions, as shown in Figure lb.
The DOT~ raster produces highly viewable ~ideo images~even~ where; the fields: are reduced from 625 to `6d 30pic~ture columns, e:ach with only 12~8 out of 625 vertically arranged~ picture elements (p1xels). This represents a horizontal~;depletion of lO:~l and a vertical depletion of 4:~1,:: givin:g:a~ overall depletion of 40:1. That is, each ~ -field is depleted by 40:1, but a time separated interlace of 35~ a::plurality of the fields which build u~ a full video image having a resolution of 625 x 625 pixels. :~
" : Hence, the horizontal resolution which can be WO94/030l3 2 ~ 4 0 8 9 3 PCT/AUg3/00368 achieved is not dependent upon the number of pixels in a single field, but is dependent on the number o~ unique positio~s that the DOT raster can acquire over time.
The columns can be broken up and rearranged into a two dimensional interlaced array (a checker board pattern as shown in Figures 2a, 2b and 2c). This has the effect ol allowing the entire image to build up over a number of fields and to redùce the tendency for the viewer to lock onto the moving checker board.~
Alsot the DOT raster does not preclude the use of other da~a compression techniques such as Discrete Cosine - Transform ~DCT) and motion prediction algorithms. Hence, even greater savings in bandwidth are possible.
Further, statistical fill-in methods can be used to reduce any artefacts introduced into the video image by the DOT raster. This is achieved with a VISUA~ E~HA~CEMENT
TECHNIQUE ;(VET) which relies on the injection of band limited noise into the video signal, once decoded, and manifests itself as pixels appearing between the other ;~ 20 pixels of the decoded checker board Battern. The luminance va~lue o~ the noise approximates that of the intensity of neishbouring pixels. This has the very surprising effect that the addition of noise increases the sharpness of the video image. AIso, the injected noise creates pseudo pixels 25~ and hence can double the apparent resolution. As the noise do~s~not have a fixed position within the DOT raster but is o~erlapped by true pixels, by the action of the Beta effect, the viewer perceives the average of the ~noise and active image. ` : ; `
; 30 ~ The increase in sharpness can be explained by a consideration of the SUPER EDGING effect ~see Figures 3a and 3b) r where any random structure improves the sharpness of an image, this is the property of the viewer to "seel' non-existent details between certain points. The edges of the shapes shown in Figures 3a and 3b appear extremely sharp even though the edyes are formed fr~m a random arrangement of pixels. The sharpness of the edges in these exam~les t ': ~,. I
WOg4/03013 PCT/AU93/00368 2i 40893 would be unobtainable by simple connecting lines. It is the psychophysics of the human perception system which ~; interpolates the random pixels and creates the perception of ; an extremely sharp edge - hence SUPER EDGING.
- 5The ability of the ~iewer'~;visual per~eption system is such that, lac~ing full i ~ mation, the missing lnformation is "created" in dramati~ detail It is this ability which enables a v~I~ewer to appreciate mpressionistici' pa1ntings. Further examples of the viewer "creating"~ the~ missing information to achieve a more complete image is~shown in Figures 4a and 4b. In each case ~ t~e~viewer can perceive a triangle, yet if the spots are ~; taken individually ~(by covering the other two) the viewer "sees" only what is present.
~ SUMMARY OF THE INVENTION
There~ore, it is an object of the present invent1on~ to prov1de an 1mage processing system relying upon the~BETA~APPARENT MO~EMENT effect to enhance the perceived resolu~ion of~a~ ~ideo image.
;Z O~In~ ac~cordance;~ with one ~spect~ of the invention there is~prov1ded~an~image~processing~system having: ~
; inter1Ock1ng~ means for~interlacing the~ pixels of subsequent ~ields~of pixels horizontallyi whereby~,~the inter1a~c1ng causes~ thé pixels to move 25~ ba~kwards~and~forwards at a rate substantially imperceptible to ~a~ human:~vièwer~ for creatlng a perceived reso1ut1on of a ; video~ im~àge~ formed~ by~a~plurality of the fields so interleaved, wherein the perceived resolution is greater than the actual~resolution.~ ~`
30~In~aGcordance~wit~h~another aspect of the invention there~is~;pro~ided an image~processing system for compressing ~ -a video image re~erred to as an oniginal ~ideo image, the mage proc~essing system comprising: ~
;a~digitiser ~means for digitising the original video ~ - 35 ~;image;, ~the orig1nal video image being formed of a plurality of original video fields each having M rows of pixels and N j `columns of pixels; and, ~3 , ~ W094/03~l3 . 2140893 ;~ PCT/A~93/00368 g a process control means for processing the digitised original fields, the process control means selecting one pixel out of euery d pixels and for deleting the remainder of the d-l pix~ls for generating depleted ~ields in a depleted video image, the depleted field ha~ing m rows of pixels and n columns of pixels where m is less than M and n is less than N;
whereby, a receive~ means can recei~e the depleted video image and can generate d-1 pixels from each selected pixel and can display:each selected pixel and its associated d-l generated pixels in a manner to simulate movement of the ~ pixels on the display to substantially reconstruct the original video image by relying on the BETA APPARENT
MOVEMENT effect.
15: Xn a~c~rdance with another aspect of the invention there is pro~ided a method for compressing a video image ref erred to as an oniginal video image, the method comprising the steps of:
~; digitising a field of the original ~ideo image into a pluraIity of data bytes referred to ~s pixels, each original video field having M rows of pixels and N columns of pixelsi selecting one pixel out of every d pixels;
deleting the remainder of the d-1 pixels; and, generating a depleted: field of pixels in a depleted ~ideo image, the depleted field having m rows of pixels and n columns:of pixeIs where m is less than M and n is less than N;
whereby,~ a~; receiver means can receive the depleted ;; video image, generate d-1 pixels from each selected pixel and: dlsplay each selected pixel and its associated d-l : generated pixels on a display means in a manner to simulate :
~ movement of the: pixels to :substantiaIly reconstruct the :: ~ original video image by relying on the BETA APPARENT
;: MOYEM~NT effect.
In accordance with another aspect of the invention there is provided an image processing system for decompressing a video image referred to as a deplet~d video 2 ~ 4 0 8 9 3 PCT/AU93/00368 image, the image processing system comprising:
a digitiser means for digitising the depleted video image, the depleted video image bei~g formed of a plurality of depleted video fields each havin~ m rows of pixels and n columns of pixels; and, a process control means ~f'or processing the digitised depleted fields, the process control means generating d-l :~ pixels from each selected pixel and displaying each selected pixel and its associated d-1~ generated pixels over a period lo of time imperceptible to a viewer for simulating movement of ::
the pixels on ~a ~display~to regenerate an original video - image having~M rows of pixels and N columns of pixels;
whereln,~ the process controller relies upon the BETA
APPARENT MOVEMENT effect in regenerating the original video 15 image:. ` ~
: In accordance :with another aspect of the invention :
there lS provided:a method for decompressing a video image referred to as a depleted video image, the method comprising the~steps of~
~ di~itising the~depleted video i~ge, the depleted video image being formed~ of a plurality of depleted video fields each~havlng m rows~ of pixels and n columns of pixels;
sélecting~a pixel from the:~depleted video:lmage;
generating~:d-l pixels from each selected pixel;
;;25 : ; ~displaying~ each ~selected pixel and its associated d-1 generated:~pixels: o~er a period of~ time imperceptible to a viewer~for simulating mo~ement of the pixels on a display mqan!s~for reconstrjucting an original video image wi~h the selected pixels and the generated pixels, the reconstructed 3:0:; :video lmage~havlng M rows of pixels and N columns of pixels, where M is greater than m and N in greater than n;
wherein, the simulated movement relies upon the BETA
: APPAR~NT MOVEMENT:effect in reconstructing the original video~image~.
35 ~ Preferably, the ` luminance companent of the rljected noise is congruent :with the luminance of adjacent pixel~s. Also, the injected noise is preferably band 21~ 0893 W~9~03~13 PIcrfAug3~00368 limited.
Prererably, luminance determining means is provided to determine the luminance of the adjacent Dixels and to set the luminance component of the pseudo pixe; to a value proximate the actual value but different enough so as : to induce an observer's~perception system to select visually the most probable value~.
BRIEF INTRO UCTION OF THE DRAWINGS
One embodiment, being an example only, of the present invention ;wlll now be described with reference to the accompanyi~g drawi~gs,: in which:-- Figure la shows a two dimensional interlace of pixels, being in an QDD field "O" and an EVEN field "E";
Figure lb shows a two dimensional random raster ~15 movement of pixels for a modulo-3 raster scheme;
Figures 2a :to 2c show the break-up of a plurality of columns of video (Flgure 2a~ into a single depleted field (Figure ~b) and into two interlaced depleted fields (Figure 2c); ~ :
20`~ ~Figures 3a and 3~ are diagrams i~powing the effect known : as SUPER EDGING in relation:to~a square and a triangle;
Figures 4a and 4b are diagrams showing the effect known as FILLING-IN~in relation to a triangle;
Fi~ure~5 is~ a block diaqram~ of an image processing 25~: system ln accordance with the present invention; -.
Figure 6 is a block diagram of a receiver of the image processing system shown in:Figure 5;
Figure 7~ lS a~ block diag~am showing~a transmitter of :the image processing system shown in Figure 5;
Figure 8 iæ ~a graphical representation of a two : interlaced regenerated~fields ~J and G of the receiver shown .
n Figure 6; and, ~ ~ .
~ :Figure 9 shows a modulo-4 raster scheme.
;
: DESÇRIPTION OF THE P~EFERRED_EMBODIMENT
The following relates to an embodiment of the image processing system 5 capable of providlng a video depletion of about 40:1. The image processing system 9 ~ ~ -WO94/03013 21~ PCT/AU93/00368 comprises a transmitter 10 and a receiver 12, shown in Figures 6 and 7 respecti~ely. A CODEC 14 is connected to an output of the transmitter 10 and another CODEC 16 is connected to an input of the receiver~. A video source 18 5~ is connected to an input of the transmitter 10 and a video ; monitor 20 is conne~ted to an ~put of the receiver 12.
~; Optionally, a monitor 21 is connected to the transmitter 10.
.
RECEIVER
The receiver 12 has a video signal conditioner 32a and a sync separator 32b connected to ~ video input 33 which .
:: : lS connected to the CODEC 16. The~ sync separator 32b is ; connected to a clock circuit 32c and a timlng controller 32d whlch controls a~clock timer 32e. The clock circuit 32c typically operates at a frequency of 12 megahert~ and is 15 ~hereinafter referred to as the "pixel clock'l 32c. The video conditioner 32a is connected to an analogue dis?ital converter;34a and thereby to a field store 34b. The out~ut of;the field store 34b is typically an 8-bit databus which lS shown ~as~ a thi~k line in Figure 6. The field store 34b 20~ is ~onnected to a delay circuit 34c~and thereby to an 8-bit latch 34d. The~delay of the delay circuit 34c is typically about l~microsecond so as to enable correct syncing. The 8-bit latch~ 34d typically has a refresh rate of~about 5 million times per~se~ond.
25~ The~ ~sync and colour subcarrier components are extracted from the~video~signal at the vi~eo input 33 by a colour extractor 32~ connected to~the video input 33. The colour extractor 32 is connected to a colour processor 34 which typically includes a control circuit substantially t;~e ~ ;
30 ~;same~ as the remainder of the receiver 12. The colour extractor 34~ ope~rates on standard B~Y and R-Y signals sent . 7 in~the video~signal from the transmitter 10, Typically, the B-Y and R-Y signals are at a reduced resolution, such as . 3 half the resolution of the video signal processed by the 35; plxel processor 26 (the compressed Y signal).
The analo~ue to digital converter 34a and the field store 34b digitise the video luminance signal from the $

~ 21~089~ 1 wo94/03013 :- ~ . PCr!AU93/~0368 !

video conditioner 32a. The analogue to digital converter ~, 34a generates digital values corresponding to the luminance of each pixel (hereinafter referred to as the ~Ipixels~) "on the fly" as the luminancP signal is processed by the video conditioner 32a. : The pixels are then stored in the appropriate memory address of the ~ield store 34b.
: Typically, the luminance for each pixel is determined to be, for example, one of 256 luminance levels per pixel depending ~: on the instantaneous voltage of the video signal (i.e. one i~ 2~).
~; For ease of understanding the conception of the ` ~ - receiver consider that the fie_d store 34b collects one full field of the luminance signal and stores it for further processing. That is, consider that the receiver 12 operates one field behind the incoming video signals at the video input 33.
: The clo:ck circuit 32c controls the timing of the analogue to digitaI con~erter 34a so as to digitise the s:lgnal :received from the video co~ditioner 34a at the `:
appropriate time:to correspond to eac~ pixel location of the ~ideo source 18. The sync separator 32b re-aligns the cloc~
circuit 32c at the beginning of each line of the video signal by use~ of the horlzontal sync signal contained therein.
~ The: outputs of the fieId store 34b and the ~-bit Iatch~34d are connected to a pixel processor 36. The pixel processor 36 has a process controller 36a connected to : latches and buffers 36c, 36d and 36e. The latch and buffer ~'~ 36c is connected to an ALU unit 38c and thereby to a latch and buffer 38d. The clock timer 32e is connected to the 8-bit latch 3~d, the latches and buffers 36c, 36d, 36e and 38d. The clock timer :32e combines the clock and s-~nc signals trom the sync separator 32b and the clock circuit 32c for controlling the tl~ing of the pixel processor 36. ~j `
The process controller 36a reads the pixels from ~; : the field st~re 34b and allows them to proceed to the latch and buffer 36d. Also, the process controller 36a can allow ; ~ .
~ ' I' .
W094/03013 2~4893 ~ !.. PCT/AU93/~0368 passage of the pixels to the latch and bu~fers 36c and 36e.
The pixels which pass to the latch and buffer 36d correspond to those un-depleted pixels which were transmitted to the receiver 12.
A pseudo noise generator 3~8a is connected to the ALU unit 38c via a latch 38b. The pseudo noise generator 38a allQws injection of noise pixels into the video image ~; displayed on the monitor 20. The pseudo noise generator 38a typ~cally generates pixels with luminance values which have a random value between the values of the adjacent pixels.
The outputs of the latches and buffers 36d, 36e ~ and 38d are connected to a digital to analogue converter 40 which is connected to a video output 42 and hence to the monitor 20.
~ ~ ~ The pixel from the latches and buffers 36d, 36e and 313d are recombined with the colour information at the digltal to analogue converter 40 via the colour processor 34 and with the sync information from the sync separator 32b hence forming a vldeo signa1 corresponding to a regenerated 20 ~ideo~ field for`display on the monito~ 20.
The process~controller 36a reads pixels from the fiel;d store 34b one at a time in sequential order, corresponding to ~rows of the video signal from the video source~18~i.e. U11, U13,...Ul,n (where n~is the number of 25~ col~umns~ as shown in Figure 8. However, once the pixels U11,~ U13...U1,n~ have been~read~the process controller 3~a genera~:es pixels ~12, G14,...Gln-1 and sends them to the latch~and buffer 3~d before their each of associated pixels U11, U13,...~Uln are;sent to the latch and buffer 36d via the 8-blt latch 34d. That is, the generated pixel G12 is sent ` to~he latch and buffer 36d before the depleted pixel U13 and so on. Hen~e, the process controller 36a generates the pixels between the un-depleted pixels from the video slgr.al as it reads the un-depleted pixels and sends each generated ~; 35 pixel~ for display before it sends the last pixel read from the field store 34b to the latch and buffer 36d. The pixels are referred to as '~un-depleted pixels'~ because they are the ~: : . . . . . .

f. 21~0893 WO 94/03013 . P.~/AU93/00368 i, j pixels which remain after the depletion process of the ¦ transmitter 10, described hereinafter.
The control processor 36a generates the ge~eratedpixels G by considering the luminance values (between and 0 and 255) of the last two pixels read from the field store 34b and makes the generated pixel lu~inance value statistically dependent thereon. For example, the value of the generated plxel could be a polynomial interpolation of : the values of the two un-depleted pixels.
: Alternatively, th value could be a random value with the values of the two depleted pixels as its upper and lower limitsO In this case the process controller passes control to the latch and buffer 36c and the pixel is generated by the pseudo noise generator 38a and sent to the ~: 15 DAC 40 via the latch and buffer 38d.
The process controller 36a then reads the pixels U~1, U33,...U3,n of row three of the video image. These are the next pixels in the field store 34b. The control processor 36a also generates the pixels G32, G34,...G3n+1.
Then the cvntrol processor 36a gen~rates the pixels G21, : G22,...G2n for the line of pixels between the first and third lines of pixels read from the field store 34b.
The effect of the above is that each deple~ed :~ fleld commenc~es with only: 312 out of 625 pixels across and 2~5 312 out of 625 pix~ls down in each column but results in a fu11 62:5 line video signal. Hence, each depleted field had only one quarter of the pixels of the original field and the recelver 12 regenerates the other three ~uarters of the pixels, as shown graphically in Figure 8.
:30 The above~ description relates to a mo~ulo-2 : depletion of the pixels of the original video signals. It amounts to a horizontal and vertical interlace of two fields, namely Un,m/Gn,m and un,m/gn,m.
The horizon~al interlace has the effect of moving ~ 35 the image back and forth behind "a picket fence" and hence ; induces the BETA effec~ to give the illusion of hi~her than ~:~ actual resolution to the viewer. The vertical interlace W 0 94/03013 2 ~ 4 0 a 9 3 PC~r/AU93/00368 gives higher vertical resolution.
In Figure 9 there is shown a graphical representation of a modulo-4 raster scheme. In modulo-4 four fields are effectively interlaced h~rizontally and step through positions marked "1" throug~ '7". In this scheme the receiver 12 generates pixels G~ nd g for three out of every four columns. Vertical in~èrlacing could be included in this raster scheme. Further, as shown in Figure lb the raster could be random (a random 2 dimensional modulo-3) ln which over subsequent: fields each pixel described the path shown by lines enumerated 1 to 9.
, T~ANSMITTER
The transmitter 10 has a transmission signal conditioner 22a and a since separator 22b connected to a vldeo input 23 which~is driven by the video source 18 (such as a video camera~or HDTV program or VCR or the like). The sync separator 22b is connected to a clock circuit 22c and a timing controller 22d which controls a clock timer 22e. The : clock ~circuitry 22c typically operates at a frequency of 12 megahertz and is hereinafter referre~ to as the pixel clock ~;;`: : 22c. The:video conditioner 22a is connected to an analogue digital converter 24a and thereby to a field store 24b. Th~
output of the field stoxe 24b is typically an 8-bit databus which is shown as a thick line in Figure 7. The field store 24b is connected to a delay circuit 24c and thereby to an 8-bit latch 24d. ~The delay of the delay circuit 24c is typical~y about 1 microsecond so as to enable correct syn¢ing. The 8-bit latch 24d typically has a refresh rate of about 5 million times per second.
~ The signal conditioner 22a extracts the luminance component from the video signal. A colour extractor 22 extracts the colour component from the video signal received from the video source 18. The colour extractor 22 is : : connected to a colour processor 27 which typically includes : 35 a control circuit substantially the same as the remainder of :~ the transmitter 10. The colour extractor 27 operates on ~ standard B-Y and R-Y signals sent in the video signal from WO94/030l3 21 ~ o ~ g 3 PCr/AU93/00368 the transmitter 10. Typically, the B-Y and R-Y signals are at a reduced resolution, such as half the resolution of the ~ video signal processed by the pixel processor 26 (the ¦ compressed Y signal).
!: 5 The analogue to~ digital converter 24 and the I field store 24b digitise each field received from the video I source 18 and store it for digital processing. The analogue I to digital converter 2~a generates digital values of the luminance of each pixel (hereinafter referred to as I 10 "pixels") "on the fly" as the luminance signal is processed j by the video conditioner 22a. Typlcally, the luminance is determined to be, for example, one of 256 luminance levels ~er pixel (i.e. one in 28).
The clock circuit 22c controls the timing of the analogue to digital sonverter 24a so as to digitise the signal received from the ~ideo conditioner 24a at the appropriate time to correspond to each pixel location of the video sou~ce 18. The sync separator 22b re-aligns the clock circuit ~2c at the beginning of each line of the video ~: 20 signal by use of the horizontal ~ync signal contained : ::
therein. :
The output of the field store 24b and the 8-bit latch 24d is zonnected to a pixel processor 26~ The pixel processor 26 has a process controller 26a connected to latches and buffers 26c and 26d. The latch and buffer 26c is connécted to an:~ALU unlt 28a and thereby to a latch and buffer 2~b. The cloc~ timer 22e is connected to the 8-bit latch 24d and the latches and buffers 26c, 26d and 28b. The : clock timer 22e combines the clock and sync signals from the 30 sync separator 22~b and the clock circuit ~2c ~or controlling the timing of the pixel processor 26.
; The process controller 26a reads luminance values from the field store 24b one at a time in sequential order, corresponding to rows of the video signal from the video :35 source 18. s For ease of understanding the conception of the transmitter consider that the field store 24b collects one W094~0311l3 93 PCr/AU93/1)0368 full field of the luminance signal and stores it for further processing. That is/ consider that the transmitter 10 operates one field behind the incoming video signals at the video input 23 5Ref erring to Fig~ ~ 8 the process controller 26a selects, for example, th~.pixels corresponding to the ODD
rows and the ODD columns', i.e. Un,m for all values of n and : m. These pixels are referred to as the "un-depleted pixels~' and are allowed~to~proceed to the latch and bu~fer 26d for : 10 transmission to the` receiver 12. Also, the process controller 26a allows passage of~other ones- of the pixels ^ th~ latch and buffer: 26c. These pixels are referred to as "depleted pixels". The pixels which pass to the latch and buffer 2~c are used to be displayed on a monitor 15 by the 15: ALU unlt 28a, the latch and buffer 28b ana a digital to :analogue: converter:31. The digital to anaIogue converter 31 ; is al~so:~connected to an output oL the latch and buffer 26d and therefore, the monitor 15: can show a video signal similar~to that wh1ch will be shown on the monitor 20 at the :20 rece:iver i2. :~
: A pseud~ noise~ generator 28c îs connected to the ALU unit: 28a via a latch 28d. ~ The~ pseudo noise generator 28c allows~ injection of noise pixels into the video image displayed on the monitor~15~: :
: 25 ~ The process:controller 26a has the effect of, for example, selecting ODD pixels- from OD~ video fields and ~VEN
pixels from EVEN vldeo ~ields. The combined effect is a ;~ field having ;~two fields which are interlaced both horizontally and:vertically as shown in Figure 8. That is, every second column and every second row from the original ideo signal has been deleted. Hence, the resultant video signal for transmission is re~erred to as "depleted".
,, :; The output of the latch and buIIer 26d is : connected to a digital to analogue converter 3C which is connected to the CODEC 14 for transmission viar for example, ~: an antenna. The sync separator ~2b is connected to the DAC ., 30 to control its timing.

~, ~ ;

WO94/03013 21 ~ 0 8 9 ~ PCT/AU93/U036X

The effect of the a~ove is that each field is depleted from 625 down to 312 pixels across and from 625 dow~ to 321 pixels down in each column. Hencej each depleted field has one quarter of the information of the original field. This correspo~ds to a modulo-2 depletion of the pixels of the original video signals.
The Iuminance of the depleted pixels is recombined ~: with the sync;:and colour information at the digital to analogue converter 30 via the colour processor 27 and the sync separator 22b respectively.
~~ As with the receiver 12 other forms of depletion `: ~ of the video signal could be used, such as, for examplP, ~ modulo 3, 4, 5 etc. or ~even a relatively random form of :~ depletion as shown in Flgure lb in which over subsequent :~: 15 fields each pixel describes:the path shown by lines 1 to 9.
This is a two ~imenslonal version of modulo-3.
: In modulo-4, as sho:wn in ~igure 9, each field image lS digitised and saved in the field store 2~b in the following manner:
Z0 -~ during field l~r startinq from the first pixel, ,`
on~ly 1 pixe~l in 8 is saved on each line. The ; : dis:played picture loo~s simllar to Figure 2a where part of a 64 column pattern is depicted; ~, during:field 2, starting from pixel 3, only the :25 ~ : three-modulo-8 elements are saved;
during fields 3:and 4 the same process is used for pixel 5 and 7 respectively~
:~ The combined effect is a frame of 4 fields that looks ::~; similar to ~igure 2bf in which every second line has been ,-deleted - both horizontally and vertically.
The imag~ processing system of the present invention allows ~or considerable compression of a video signal by relying on ~the psychophysical attributes of the ; : human perception system to undertake appropriate interpolation of the video image to provide a perceived resolution which is substantiall-~ the same as that of the uncompressed signal - even though the actual resolution has ~ .: ` f~:
W~94/03013 PCT/AU93/00368 ~40~9 - 20 -been severely depleted. The DOT system relies on the BETA
APP~RENT MOTION effect to achieve the high compression by deletion of all information from the video signal which is not necessary for the human perception system. The BETA
effect is ~chieved in still video images by moving the pixels, such as, backwards and forwards. Also, the ` viewability of the interlaced fields is enhanced due to the very high statistical correlation - between the pixels of subsequent fields (by stepping the pixels bac~wards and forwards). The VET then adds random noise to the signal, once received, in order to make the resultant video image sharper - by relying on the phenomenon of SUPER EDGING.
Also, the system of the present invention pref;erably operates in a digital format and so avoids analogue artefacts (i.e. overshoot and smear) which are very ; difficult to remove. Statistical manipulation of the video signal can allow for even greater compression. For example, the ~ideo signal, as compressed by the DOT raster, could be processed by a DCT to give a further compression of 40:1, thus giving a total compression of ~600:1. Hence, a video signal can be compressed and transmitted via a telephone :
line having a bandwidth of 64kHz - thus being applicable to video telephone without requiring special ox multiple telephone lines.
~lso, the DOT can be used to enhançe the resolution of a standard video signal to achieve a high definition video result. And, such high definition can be achieved simply~ by processes at the receiver. Still further, the DOT could be used to digitise video signals for video recorders and ~ideo cameras.
Modificatlon and variations such as would be apparent to a skilled addressee are considered within the scope of the present invention. For example, other systems of movement of the pixels could be used to take advantage of the BETA effect e.g. circular or random movement of pixels.

' .

Claims (12)

1. An image processing system having:
interlocking means for interlacing the pixels of subsequent fields of pixels horizontally;
whereby, the interlacing causes the pixels to move backwards and forwards at a rate substantially imperceptible to a human viewer for creating a perceived resolution of a video image formed by a plurality of the fields so interleaved, wherein the perceived resolution is greater than the actual resolution.

2. An image processing system according to claim 1, in which the interlacing means interlaces pixels horizontally and vertically.

3. An image processing system according to claim 1, in which the interlacing means interlaces the pixels in a random manner.

4. An image processing system according to any one of the preceding claims, also comprising a noise generation means for generating pixels of random luminance for interspersing between the pixels for relying upon the super edging effect for enhancing the perceived resolution of the video image.

5. An image processing system for compressing a video image referred to as an original video image, the image processing system comprising:
a digitiser means for digitising the original video image, the original video image being formed of a plurality of original video fields each having M rows of pixels and N
columns of pixels; and, a process control means for processing the digitised original fields, the process control means selecting one pixel out of every d pixels and for deleting the remainder of the d-1 pixels for generating depleted fields in depleted video image, the depleted field having m rows of pixels and n columns of pixels where m is less than M and n is less than N;
whereby, a receiver means can receive the depleted video image and can generate d-1 pixels from each selected pixel and can display each selected pixel and its associated d-1 generated pixels in a manner to simulate movement of the pixels on the display to substantially reconstruct the original video image by relying on the BETA APPARENT
MOVEMENT effect.

6. An image processing system according to claim 5, in which the process controller interlaces the pixels in a random manner.

7. A image; processing system according to claim 5, also comprising a noise generation means for generating pixels of random luminance for interspersing between the pixels for relying upon the super edging effect for enhancing the perceived resolution of the video image.

8. A method for compressing a video image referred to as an original video image, the method comprising the steps of:
digitising a field of the original video image into a plurality of data bytes referred to as pixels, each original video field having M rows of pixels and N columns of pixels;
selecting one pixel out of every d pixels;
deleting the remainder of the d-1 pixels; and, generating a depleted field of pixels in a depleted video image, the depleted field having m rows of pixels and n columns of pixels where m is less than M and n is less than N;
whereby, a receiver means can receive the depleted video image, generate d-1 pixels from each selected pixel and display each selected pixel and its associated d-1 generated pixels on a display means in a manner to simulate movement of the pixels to substantially reconstruct the original video image by relying on the BETA APPARENT
MOVEMENT effect.

9. An image processing system for decompressing a video image referred to as a depleted video image, the image processing system comprising:
a digitiser means for digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixels and n columns of pixels; and, a process control means for processing the digitised depleted fields, the process control means generating d-1 pixels from each selected pixel and displaying each selected pixel and its associated d-1 generated pixels over a period of time imperceptible to a viewer for simulating movement of the pixels on a display to regenerate an original video image having M rows of pixels and N columns of pixels;
wherein, the process controller relies upon the BETA
APPARENT MOVEMENT effect in regenerating the original video image.

10. An image processing system according to claim 9, also comprising a noise generation means for generating pixels of random luminance for interspersing between the pixels for relying upon the super edging effect for enhancing the perceived resolution of the video image.

11. A method for decompressing a video image referred to as a depleted video image, the method comprising the steps of:
digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixels and n columns of pixels;
selecting a pixel from the depleted video image;
generating d-1 pixels from each selected pixel;
displaying each selected pixel and its associated d-1 generated pixels over a period of time imperceptible to a viewer for simulating movement of the pixels on a display means for reconstructing an original video image with the selected pixels and the generated pixels, the reconstructed video image having M rows of pixels and N columns of pixels, where M is greater than m and N in greater than n;
wherein, the simulated movement relies upon the BETA
APPARENT MOVEMENT effect in reconstructing the original video image.

12. An image processing system according to claim 11, also comprising a noise generation means for generating pixels of random luminance for interspersing between the pixels for relying upon the super edging effect for enhancing the perceived resolution of the video image,