CA2237886A1 - Computer stereo vision system and method - Google Patents

Abstract

The system and the method are intended to enable a robot (20) to see the environment in which it operates, by means of a pair of identical cameras (3). The pictures from the cameras (3) are processed by a coordinator/translator (59). The system includes a memory register (51) for movement identification at time intervals.A basic shapes registry (52) identifies shapes.

Description

CA 02237886 1998-0~-13 ~L 96/00 1 I~WS O 6 F~ t9 , COMPUTER STE~EO VISION SYSTEM ANI~ METHO~

Fleld of Invenffon T~is i~ention relates to computer vision ~ ification seen in real time by mear~ of c~ x, a computer, etc. in ~r~r~l and 3-D co~r~uter v~sion ~y means of 2 or more "stereo" c ~ '7~ in pa~cular.
R~q~around of ~e InYenffon The need for c~7"~uL~l ~is~on is an eYery day necessity for Iesol~ing problems ~for use by man p~ tically in all fields: From research, astronoaIy, robo~cs, in~stIy, agciculture7 ..~...-r,.~ture, services, dnv~, seclmty, assistance to the blir~, detectionofrl~.n...~.~ etc.
Today, as the volnme of both ir~ l ar~ extemal con~?uter memory is la~sr~ ths phy~ical (li.~.xinn of c~ yuLt;r co~nents is bemg c~irmlusly reduced, aml vali~us typss of h~gh speed processors exi~t. Conse~uently, if one could create a system and a method for co~uter ~risi~ that would iderfi~y yLhil~, it sees in real time, then e~re~ uld becoqne easier and more si~ple.
Present Situa~on Present co~ uter v~sic~n, w~ch makes use of one o~ two ~ , focuses on the vision of individual, def~ned, knDwn and ~ r static object(s) ~d their i~lP.ltifi~ationis le~y, co...l~.-,.livt; ~pictu~e -~;.~t pictuIe), paItial and focused ~y~ and does not ~lyze and ider~ify ev~ryt~ng that is seen at the speed of slx~oti~ It re~ ; and uses ma~y devices such as: sensors, li~ltir~ me~iL~
~n~?~ etc., and is cumbersome, limit~1, insllffici~r~ly efficiP~ and does not provide satisfacto~y solllti~
O~iecffves of the Invenffon The puIpose of this inver~on is to E~ovide a system and a method ~ to aDalyze and iderfify all the foIms viswsd cf~n~P.~rq)oIaneously and at tl~ paCB of filmir~ by mans of ~;~II~.I.1X conr~cte~l to a computer, means for CV~ ; of <l; . l ~ lT~ti~ l other perceptible R~tln~s.
Tru re, the pu~pose of ~his invention is to enable ~my ~llt~ted device (robots, tools, c~ll~uLel~" etc.) to see by means of vaIied and ~3~r~Liate means of vision, any t~r~ble tl~, ~1~ ~n, ~imilznly to the way man can see and ider$i~ ~m, to teach ~em basic tB~ of ~tifici~l i~.lli~.n~e they nsed in order to c.~.. ~;c~e ~th their e~i~ the way man c~.. ~i.. ;~ates wqth 1;
AMENDE0 SI~E~T

A CA 02237886 1998-0~-13 ~ ~ ~ 96J0~ 1 45 ~WS O 6 F~B ~gg8 to perfoIm alm~st aTy acti~n, tas3~: and w03:k that man dk~es but more accuIately, effi~ tly, f~eter, better, etc., around the clock, a~ e~ l ~n places ~ch are physically ~ifficult to access~ or ~ ulls~ ir~cces~ihle~ bonr~, etc.
, the ~pose of t~ ~erfion is to allow stoIage, b~a~cast~ng a~/or i~ via r~ lZ-t~ inn lines, I~a~ TLR 3-D pi-iLLu~s, so that a person can'~see" that picture f~rn a remote loca~o~, by me~Js of a decodir~g so~cware progcam an~Jor ~lti-media, ~lasses aT~for special devices ~Rsi~LRd for ~i-D v~sioql, ~ection, y~ sn an~l eYen enl~
Ths o~ective of this Dention is to allow for t~R cor~truction of dev~ces accordir~, to i~, ~le ~ e~ , systerns, ~ a~ elflC~t; c~lT~r~ basic so~ware etc., e~stir~ on the "~.k~t in one foml o~ ~thP.r, SO that by their c~nnRct-7~n, cq---l7i-~ n, ~t~tion, ext~ n~ etc., de~Aces canbe CIe~R~ ar~or -ds:ie~ led according to this i~en~on, i~ln~ir~ th~s ability of ada~tion to ~llllr~ 8S. ~ aY~rage elecb~nics Rry~n~er, cor~puter e~n~er~ ~ystem ar~lyzer, etc, will be able to design, ~cll~le and cons~uct de~ices acc~ g, to t}~s i~enti~
0~ of ~e ad-v~ ps ofthis in~en~ionis its ~ 71i~ity, aUowir~ it to be used ~th ,..i.,i..~l 1imits~ti~, this iTrventiononlY canfillfiU U t~r o~ectives.

The Inven1ion The Dention consists of an ..I~lL7v-~ e system and mthod for steIeo computer v~on A}y m~teri~1 wiitten as e~71~tinn to the software and u~ich does not form paIt of the iIn~ention~cess as a whole, is a sample proposal or~y. l~s ~ose is to iUustrate the ~rocess of ope~ti~7n of coIr~puter ~ion, and does not ~RJ~tR f~m the essence of the i~ention.

2. The e~71~n~tinn g~ren below is ~.nP.~l, s~nce there vv~U be vaIious types, models and sizes of computer ~ systems (e.~ ge, smalL medi~TI, Phi~lips, etc.), in accor~nce with the d;~ta that the coq~uter ~ision w~U haYe to ...il to ~e customer it seIves, but the basic Opk~ti~ ~ R ill all of them is identical.

3. Computer vision is a means of offer~r~g a service of CCYie~' . Its task is to see a section in space, to collect d~, defin~ti~ an~l terms in ~o~ectio~ with it, AME~ 3ED SHEET

CA 02237886 1998-0~-13 ~L 9 6 / O ~ 1 4 ~
II~WS O 6 F~ ~gg8 w~le p$~eservi~ the pictLlre of the view an~ w~le c1~ ths ~iewed Enct~ into codes ar~ d~a, to decip~er it as necess~y, to store in a regi~3ter whatever needed and to l~ ~il filrt~r the ol~ect it has seen or relevar~ ~a 3-D data and ~lltimRdia may be stored, by stonng the ~ , data at aIy sh~ pace.
The~ is llifF~n~e between the tw~ systsms aT~ msthods in the trs~tmP.nf~ but in l..; . ,~;;l~le, they are identical.
~n the system and me~d of pictu~e stC ~, the co~r~son picturs must be a spa~l p3ctu~ th part of the collected tl~t~ refèmng to aach and e~rery dot such as color, Wr ( difference befween the two c~ ~J~ to the same dot), etc, the piGtures of the e~ are updated ~th the photog~a~
coordi~es and the Scillll~ly~ of the ~ pictu~ as ~ t the space picture relates to Co~l~t~ and relies o~ the location of the space picture co~rdinates as c~ d to tha callJl~ ~ict~e.
With t~s met}~ the pixels of one of the c~ are adjusted by means of the dev~ces aT~ y a~ of the coor~ t~s, in an ordsrly ..~...~., at the pace of shoating aT~or other, as against the p~xels of a p~evious pictuIe storad in the sp~ti~l m.qmnTy register, as long as there is id~y ~nd c~respo~ance ev~lyl~ , cn.- ;IlllRc~ an~ new movemer~ s}~e in ths video registered p~cture c~atss t~ck of ca~respo~snce an~ llpft~hr~ and thsn ~ ~ls of the two li~..~r~ are co~ d, pi~cel against pi~CBL the data of t~R identical pixels is recordedinthe spa~ y fo$ eachcolarpixeL Wr llilT~;.t;~e, etc., the dots of non-coT~espc,~lPnce in the picture stored in the sp~ial mRlT~y aIe reg~stersd as depthlines inthe s~ picture and the dots of ~e second picture are }egistered sepaIately in ~onnectioql, aml ag~in, m~t~ ~;-~ ths spa~ial "~ s-y begins ag~n, etc.
FuIther l~ L"~~ is si~lar and car~ Rs accordirlg to the same pl;l~ le of encoded data detectlon and corr~t~on as detailed in ~e abo~e preferred ex~rnple, but wi~h a di~ lce ~ttihl~lR, ad~usted to t}~R ~.~...~. of S~ , and a collection LR tl51tZ- collection p~ocess takes place in two ways: A Cu~ of t~R dots of one pi~ture as ~ t ~e do~s of the o~her picture of ~he same pace and/o~
,l~t~ as ~ t a dah~n among the d~a of two conss~ re pictures7 fio.r puIposR of detection of the Wr, "c~t.~ of coord~r~tes, mntinn, co~respondence, i~rtification7 etc. B. CollectiM~detection of data among thc A~AENDED SHE~

CA 02237886 1998-0~-13 ~/IL ~ U 1 4 ~
I~A/IJS ~ ~ FEB ~gg8 existing da~a of the same pic,ture, such as ~ames, geomRtrical lines, geome~cal.sha~es ~fiamesl etc., and based on all ~e d~a an~Jor a part t~LRreof, c~ are dIawn, as well as definition~, etc., for fuIther l~ , for ~rtific~ p~lli~.T~e~ for a ~-~lchir~ and i~Rnlifica~oIL key, an~l ~or cl,.. ~ .. ~i ca~io~L
6. Eneeded, pa:[t of the v~s~o~ phases may be used for data f-~ ~-;c~i~n, for ex~ple in 3-D ~roadcasti~, by d viding each type of da~a in a~e mavemer~
and d~a connpos~ion a~d ~he 1-~ -;5~ band ~d~h an~Uor al~ks n~Iy be reduced, and in ~n~*~P.r pla~e the codes ~1 da~ r~ to them m~ be t~n~1~te-1/de~ )1~J~d and displayed and projected in va~ous ways so as to obtain a 3-D ~icture, w~ich ma~ even be enl~d.
';~ 7. SiIlCB c~T~lter ~on is su~possd to senre a cll~tfmRr and th~ customer is supposed to ~lr~ limited opP,~ti~n~, his wo~k space is usually confin~A
therefors, ~ con~er vision system ~ l for tl~ cnst~nnRr ~11 lElve to ~ovide ~n ~th the slt~ he n~eds for fillfillir~ his task 8. Connputer vision and/or the robot ~refer bly) nnay use ~ liti~l PxtPrr~l aids(also for puIpose of de~ , such as tlhe ll~n~l~r system, the cn~tf~ l's nrle~ s, an~3io, touch ~eat, ~iLlL~ S~ , wind ~low, etc.~, taste, s~ n, etc, ~ich nnay be extemal to the s~em or the cll~tC~ r ~etween tl~ cll~tnm~r~s CW~ el vision systsm and the aids the~ ~11 be ~Bci~ n7~e~ corr~i~i~y, ~r~ncc, C~lPT~ti~, rwi~t~, d~a inn, etc., as maybe req~re~
9. The system canbe c~"-.l,;.~.-l with a robot and own ar~Jor s~ Le C~
,de~ce which~v.ides the r~bot ~h d~, etc.
1~. ~um~g da~a collection, basic te~ns in ~Ttifici~l intP.lli~n~e a~e dete-,ted These basic tenns are intRnfl~d to assist the deG;l~l~.;.y, system and the robot coIr~uter to decipher pictures, phP.nmT~n~ etc., as well as to infnrm and cnm-.~ tP ~th the env~--~----~-~ in w~ich the robot operates, such as other robots ~ike ar~or lnlm~n beings.

List of F~ures Fi~-l is a schRlT~ti~: sketch of a Elreferred system iTrlll~ pos~ih1P c~nRcti~
in the ~n of a colu~r~ ~li~rn;

AMEN~ED SHE~T

CA 02237886 1998-0~-13 ~L 96/0~ 1 4 l~lS ~ g ~ g~

Fi~-2 is a ~ow chaIt of a prefe~ed system for co~ re l~ of data ecei~ed ~om ths ~ and their l . ~ sion to the ~ s, encoded with the detected (1 Fi~-~ is a ll~nz~ OSS section that ill ~ .~ qs schRms.ti~lly the cer~al vie~
axes, the fields of si~ nd the paIallel c~ , lines of the fields of si~ht and the optic im~s of the system descnbed in Fi~-l.
Fi~-Af3 (x4 er~ is the parallel optic images of Fig-3.
F~g ~ is a ltnti7nnt~1 cross section ~a$ shows sch~mz *c~1ly the fields of si~ and the shapes viewed ~of ths system desc~bed in F~g-1~ m va~ous sizes an~l at v~ious ~ t~es Fig-A,~4 (x4 er~ re~resents pictures of ~e sha~es v~ewed in 3~
v~ewed sin~llt~ously by bo~h the ~ht an~l the left c~ of ~e system described inFi~-1.

Phot~r~l 'n~ ~Imin~) ~nd the Camera~
1. A ~ of identical ~ 4 ( Fi~ , co~rriin~ted in s~ an~e in~ln~liT~ va~able aml enl~.~..lf.-lreduction, c~ , fo~ the ~ x op~ical p~ll~l fields of si~ Flg.-3) CU~ 1 a co~... o.- plane ~ n~l refem~d to as the ~nTi7nnt~1 plans, in o~r wo~ds, ~e h ri7nnt~1 distanc~ with Ie~rd to sach field of sig~ lins is alw~s i~lRTfi~.s~ ~, bo~ ~ x ~m {0 0~
and up to ~ ), and is a fixed ~lixt~Tlrs of devia~ion M (F~g.-3, At3) at aIy pl~o~ xt:~rre. The veItical ll~v~ el~licularto ~e l~..i~..~;.
plane) of ~e c,.. . ~.l nx is Cu~ and identical.
The rl~t~rlls: takenbythe ~ x i~ recei~edinanir~-..~..~.y P-O and P-l ~Fi~-l) in a~co~e with the spsed of ph~o~ aT~or other) m fo~ of d~al da~a tn~ te~l either by the ~ or by a~y other e4 -;~
5, ~i~-l) ths ~ will be p~ysical upon ll~llllr~ tu~ inct~ n andfor a~ a~ other ~ven mnm~Tt ~ef~T~h1y by the rohot).
. Said tw~ or more ~ l, x, in a sir~le enclosure or in separate p~l,kZ~T~" are si~lar to video c;~ tx, in~1nAir~ CCD ~ x and inf~h1 liry~ c~ x ~
ir~e~d means fo~ coqrve~ data received in the pictures to dy~ital d~a and ;T~111~1;T~ one or mo~ of ~e follo~r~:
a) A~aptation to colo~ photo~ s at var~ous speeds and at aly li~ such as IR, Yisible and at any li~;hir~ c~rulitinn~ such as poor, by means of li~ht :~lifil~ati~S
A~AENDED S~T

~ CA 02237886 1998-05-13 ~ ~ f ~
ll~qlllS ~ g FE~ 1998 _ ~ _ b) Fnk~or ~dllction devices, i~l~lAi~ telescop~c or m~croscopqc c~ Optic i~ in.. lllAi~the des~redres~.lntinn whether s~
co~ex~ or concav~.

Calcula~on of disl -~s~ rlim~n~ons:
1. Based oql ths dE~n Wr - ~e difference between the loca~ion of a coD~esp~
d~t in the ~icture of the two ~ on the X a~s (Wr--Xb-X~) ~Fig -2), and ~ased on the fixed d~um M - t~he paIallel deviation ~fixed distance ~i~-3) between parallel lines at aTy shootiIlg distance~ the s~ze of the ~nxel Ie~reser t~ti~n can be calculated Q~lWr.
:~ 2. The calc~ ti~n of the dist~ce L between ~he C~ and aTy dot is based oql the ide~ity of the two e;1..~,J-.lx ~i~e~i~l shoo~g ar~e in the c~...s..-~ (in Cit~ l a~ ve~cal ~~)l,onthep~ysicalsi2e(theu~dthof the optic iITE~ge 1), the i~ced resnlllti~n of the optic ilrE~ XY, on ~;
Rnl~ fre~ian, M, and Wr and ~e c...~l;..-~. K~, Kl, K~, K3 m ev~ry v~e~g system B~sic formul~s (ocp~ aT~le for a sir~e En~L ~U) ~ ler of the circle in E~ucels (op~c image), (CPn) size of pi~fel ~ $~ n in the c~ u~-~e ~lanel, (C~d) size of ~el reE~reser~ti~n for st~i~rd ~ U~ d in foqm A and pLxels' Xd in d~
1. ap~~f~X*
2. U~ ~~O~a~
~. C~

4. Q~.

Dlst~nce c~lcul~tion formula~
rO - ~istam;e~ From the circle Ia~us u~ to the op~c ~rr~ge.

5. rO----~4*360f2~*~~'~l*Ko~*K
K0=3~0l2*7c*a~
Kl~*Ko r - Ths Ia~us u~ to an~ p~t.

6. r~U*~f. *7~{{Z~*360fa~3~*{MfWr}]~*~
~ Wr3( M*X*300J2*lc*a~)~Wr~K~
K3~M*X*3~f2*~*a~~*K2 AMENDED SHEET

CA 02237886 1998-0~-13 ~T~L 96100 145 ~IS O 5 F~ tg~8 L -- The dist~e i~om the QptiC ~ U~) to aIy point.

7. ~=r-rO~YWr~K3-Z*KI~3JWr~KI]

La -- The dista~e u~ to the i~1Rr~ifiR.l s~e A (i~ el calc~ t 8. IF(Qd*U~f2*7c-rO={(dl~X*36012*~*a~~]-Z~KI=
~dlX~X*KD-2;*KI--~;[K2(dlXd)-K,]
K2=X*3~f'~*7c*a~=X*KO

The co~e plane ~s the ~X~e ~n the tw~ c~... ,n.~.~ in ~ch the paIallel ~t~hiT~ d~ is ~ihl~te/1 inthe same locatioq~ the X axis.
If Wro~ then~ Q~2M ~the computer ~vill know), ~tance calculation will ~e according to: I~Z(14*M-Kl) 3. The calclll~ti~n of the si ze of d4x repr~sRnt~tif~n allaws to calcula~e si~e, leng~h, u~i~h, hei~, ~ uce area, etc.
4. T~ ord~r of d~s wi~ a~ gi~en i~me ar~or ~etwGe~L ad~acer~ ~mes etc., an~l the Ia~o of char~e of their loca~ioqL b~tw~el~ ~e X and Y ax~s allows to ~etect ths an~ linss and shz~s for e.~l~le: a strzn~ line -is a c~ penodic~y in the cha~e of X and Y and ~he~r si~ do not c~e.
The aT~Le of the line - the ~io of change ~le~c. ~ the (tangsfcob~nz~3 a~le of the line, for e~ple: at e~rery step in the periodici~y the dots chan~e by ~(~13 fi~ced s~s3, the a~e is - 45~, etc 5. Ths calcll~ n of the mavement velocity V is based on ~ (the s~otin~
r ~ 3 an~l T [T is the ~ti~n of a sin~le frequency (Fi~-2~ ~r w~ich a te~n of time exists inthe co~uter].
. ~ will also help in ~ uducing 3-D ~ctures and intime.
7. ~ ~ow chaIt 101 (Fi~-2) ~ere ~s a f~ctor U (~h rnlmh~r S that I have cl~s~n registered next to it~, this is the lo~ atio of ch~se for a change between two af~jacent points (inthe cha~t it is 1.25).

SamPle of a System and Pre~erred Method OPer~tin~ Accordin~ to the InYention All th~ s~ c, video clips, ~law chaIt, etc., ~e ~y a~l ex~r~ple ~lich, althou~h an excelle~ one, may irK 1~ el~ as in ty~ t~r~ ti~n, etc.~, inacc AhlENDED SHE~

CA 02237886 1998-05-13 ~r~L 9 6 / 0 0 7 4 5 IPE~JS ~ g FEB t~g8 g co~lectios, a~ ls to ci~ tances, etc. 1~ is e...~ i7~d ~ the ~rocess that every processor pe. r~....~ a ceItain oper~ n(s), ~his ~s an exan~ple for the ~rwess p~sses and it is depen~s upon t}~ E~rocessing speed (~?rocessors), and ~n~he order of the process phases there may also occur ceItain IT~cations.

The Process of Plcture InPut 1. AIY picture ~ir~luding color picture) received ~om arry camera t~1~ted ir~o com~uter l~ ~, will enter to one of the l~ I;R~ PO;Pl ~i~-l) to its de~i~ted place, once to line O and once to line 1 (sa~ing of space), and so O~L
Tl~ line will be ~ rtefl to the ler~ of X inthe c~ nesn111ti~>n 2. Each codes table will be given gFr~r~1 data like a hf~ er, part of ~ich received fi~n the ro~ot coIr~uter and a part fiom l~he ne~ng cull~uLer, for e~ple: in the C~ S Z - enl;~ tre~l~ti~n and C - ths a~le of in~tinn of the axes towards the hnn7~ will be given by the robots ~ich c~ols the ïno~rement ~n~ ~e cS... ~..~ ). Tt is pos~ibl~ that aU fc~ur a~les of ~e o~ical imlge fiame dots (resnlllti~ n) in every pict~e will be m~tchRd to the robot m~ l coord~nates and to ths ~ , namely, thsre u~l be da~a defir~ng their loca~io~ in space as a basis for m*f~.hiT~ and ~ tiry~ of the cou~ ;;s f~ther aqL I~keuise, there will be a d~tum of the system the rlllml~Ar of beat t [ilow ch~t ~ 2)].
. ln the flow chart (~i~-2), P.C.U-l (~i~-l) processor h~ntllAs ths co~r~ison of da~ (such as: ~l a as ~ t pixel b (Fig-2) received ~n the c~ x, detects from am~g them new ~ta that can be deduced from the cr~ data and registers them in a n~tl~.hin~ table for codes: Ql;Q2;Q3, with each such code beir~g a ~ as reg~stered ne~¢ to it) in a 4-b~ts la~,e byte. The codes are 1 acco~ng to their appe~ e in the Y colu~ and the X row (Ci~
B), and the da~a for each code dle ~ .d ~n the ~ ,hi~ tab~e ~n the same order an~llor in a~y other f~mili~r order The P.C.U-l p~ocessor h~lles the P~;P-l every time in the free line and at a pace adjusted to the pace of sh~ and it inseIts the data irrl~ the codes to PP-O;PP-l ~Fig -l), eveIy shDoting cycle to an~ther location a ln the ~low chart ~ 2~ rectan~le 1 - ~'start", rects~ e 2 - "read Ya, Yb", llP~ - "go ba~
b. Circle [lOl ~ -2)~ detects the fiIst ide~ical dot in the data of the tu~
c~m~
AME~DED St~EET

CA 02237886 1998-0~-13 P~L 96/oO 1 4 I~IS O 6 FE~ tgg8 c. Ci~le ~102 (Fi~-2)] detects on the X axis the codes Q2 - color ar~ Q3 -alea and the relevant data:
Q2:p;Wr,~Y:Xcn);~ Xx) Q3:Wr,(Y:~);q;u;(Y:X~c~
~h p bs~ color, Wr the di~erence with reg~rds to the same dDt in the two c.~ x, (n, cn) in~calion of the ~ x) - ir~ication of the dot w~th regard~ to ~ color/line dep~sur~e, f - the rnlmher of dots of the same color~ q - the ~ ~r of dots in an area, u - the ~er of colo$s in t}~ area and (Y~cn)~ the .. ~; .. i .. and ~ dots Circle [103 ~ig-2)] detsGts on the X axis ths code Ql - t}~ depth lins and data ~n ~ tic~n to it: Ql:~p);~Y:~);f~;+);~Y:Xx) ~eIr (Y:Xn~ is the ,--i--i-.. - poir~, ~(-;+) t}~ rn~l~r of do~s on the X a~as ~nsga~ve: Xa ed~ d A, pos~ive: ~ c~ B, 3-D d~ta~, the ~ sign in~3icates 'l~eeping away", the plus sign in~icates ~yl~C~' ~basic terrns in ~T~ific~
ir~lli~n~e], Y:~c - the ll~ullLIlu po~nts (due to lack of space in th~ chart, the depthline ha~e n~ beRniclR~ifiRd) e. Circle [104 ( Fi~-2)3 ill~LL~Les ~e c~ sL~,. ~P.:r~e of the ~cessor ~Mth the f. Circle [105 (Fig-2)] ~ t~Htes the distIibulion of the data to the vanous mRm~nes for fi~rther l~n~lli~
4. The da~ of each pace, in w~e or in paIt' such as: Q2:(WrX~,f, Q1:(E~ 3 the s~, Fulses) a~ ren ~y mo~ng ~ a~le of each da~um -Hn~l c~on uith the code d~si~te~l to be ~ lt;df~oadcast, for exa~ple in TV e~nnP.l~ - 3-D b~oadcastir~ and sa~i~ of I ...ix~,ion b~s, etc. In ~thRr pl~ce the ~ . H. ~ s~ wïll be received, ~e~ Hl Rd, ~ lHted an~l it ~11 be possible to enl~ it seveIal times by .. l~ i~ all the f data on the X
axis ~ y rnm~er, and to retum on each column (row) Y the same tn~nl~er of times, thlL~ the ~cture ~11 be prc~ected an~ r re~resented in aIy other way as a 3-D p~cture and in t~me.
~kewise, it wil3 be possible to store these .l~tH an~l proJectt'preser~ them in ~t~.r time or place, in any war whatsoever.

Sample o~ }?r~me Polnfs Detecffon Me~hod:
1. The p~cessol(s) P.C.U-2 (Fi~-1) }~les the detection of d~ fou~ in one of the l.~.,.~. ;es PPO;PPl the da~a for which have been c~rnrlete-l, and its t~sk is to detect in a fixed order the ~ame dots for every eQ1Or, dep~h line and area~
AMEN~ED SHEET

CA 02237886 1998-OS-13 ~L 961~ 1 4 ~RJS O 6 FEB

based on ~ enco~ and th~ existir~ en~o~ ~1~t~, an~l to ~cord a header ~or the frame d~a an~l the frame dots ln~ ne afler the other. Fr~nn ious SCa~ /T~tChi~ we k~w ths ~ point on t~ X axis, ths ~er of pc~ and the ll~illlllll~ poir~, ~ile ~ reg~l to axis Y these data a~e n~X av~ hle, the~fore, du~g detectio¢~ of the conto~ poirfs, wl~;l~sr possible, these 11~t~ st be in~icated.

The pos~ihlR coq~tour poir~ are the ~o~lo~ng:
l).Yx,f,Yu$ 2).Yx,f,X~ 3).Yx,f.~cn)~i~q); 4).Y,~(cn),f~q3, 53 Y,Xic, o3.Yn~cn3,X~ 73.Yr~cnl X ~, 8).Yn(cn3, ~ (Cn3,f~q3 2. A~'7UUIU1~ that SC~Iu~ is ~.r...... ~ fi~nn le~ to right arYl fin~nn the tcq?
d~JWlLWCl~, al~l the direc~c~l of co¢~ou~ nhfi~h ~ is aTii-clockn~ise, the E~ocessor ~xasses throny~h the codes t ~13 to colcqF, de~ nes al~l ar~as or accord~q~ to the order of their l~c~n or aCc~r~Ty~ to tylpe an~Uor ~n al~y o~herway w~atsoe~er. E~ry code enc~ d for the r-sL tim is 1TR~A and the co~leour of i~RTti~l colo~ is ~PtpcteA~ as we31 as id~iical codes am~lfolr a~jacent codes in ~e ~icturR, and a ylV~l~iiV~ head~r and r~ns are 1~ ,.J
for it (AAn;AB~ACn), for e~ lR: far the colo¢ ~ r,~,Wr,S, fc~ ths dept~h l~ne ABr~S, for the area AC~r,Wr,S. Fa-~h heacl~r will ~3 a l~d of c~nn in ~ich aU the a~e,~ contour dots will be ~:~tP~, ths on3 a~er ths ot~er, ir~lllllin~ the da~mn f~) for axis X and ax~s Y u~ to the iIitial poir~ (cl~r~ of the co~our~, wi~ le~u.l~- to ths ~h lines o~ an area fioun~l wi~n the dept~h lines, the po~rt of t~. ..-;.~l;~rl (closing of the ccrto~) can also be the e~
axes [(O:O~Y:~] of the ~ct~e coq~our (C~ B)- ~ ~ 1 P" the n for each dot ~11 be c~ 1 out a~er j~1~tific~hnn of the adjacer~
d~t.
3. Whene~er the X ~s ~-~;--;.-....-- ~;cn), the ~-...-~ of dots f~q) is copied~re~ te~1 ns~¢ to it un~l the lll~Llll~IIL onthat axis, wi~h lt;~'~]S to that c~aar. W1~ ;V~L
Y iS lll~illlllll- (X), ILR~t to it will be registered (~ ~ously o~ later ~en the Y of that X is ~ - -; ~ -; . - .... - -~, the ~lmhPr of dots u~ to the ...; ..; .... ~ that sams a~s, cnn~erJ~iT~ tha~ same L~IG.
4. ~ order to ~-aqd that ~n furthsr S~;~1UJ1~ ths 3ins a~or ~7sn sc~ull~il~ othGr lines the same code is r~t h~tell twice, Wll~ll~V~l there is need to tIea~ codes makes us~ o~ lP~ifica~ion code), ~n order to av~d MAEN~EQ SHEEr CA 02237886 1998-05-13 ~ 9 6 /0 0 1 4 5 I~JS ~ 6 FE~ 8 "

di~Lu~ ce dunr~s treahnent, the code will be changed ir~o the coqEecu~ive code ~as a loop~ c~ y (~ = change to):

Q~ ~ Q~
Q~ Q6 ~~ ~Q14 Q~ ~Q7 ~ Ql~

T}~s meth~d allows se~e~al pr~cesso~s w~ on the same ~latsl base in c~n, each of them pelrv~ , its pa~, aTld ~en i~ fir~s ~s ne~ one C~~ Rs, etc., and t}ms, by tlG~ accol~li,~ to the si7e of Wr two ~cessors can ~rocess ~i~nllt~nRously, the one from the laIgest ar~l the o~er ~om the ~llf~st.
5 ~hen s~ , for ~pose of detsction of a coQnto~ for a color, ths a~acent dot of ~e same color mllst bs (letect~ ta bass of ~hat cods, or ~
conse~ve code. W~en s-ja."~.,~, for ~ose of ~1~tRcti~n of a cor~o~ of a depth line or for an area, the adjacent dot of ths sams code or of the consecutive code ~t be dete ted.
CI. ~e dot is ~cn) the f~ el of do~s ill the X axis L',! added to S
[S is the total Tnlml~er of dDts in that contour S=S+~o~], and ths code ~l ~. ~sin acco~ ce ~th t~x list of codes and thslr consecu~ivs codss.
~les for Contour Detec~on:
1. Accor~g to the ~revious sections, the codss ars scannsd and ~Mth lt~ ~ ~, to each code ths data are copied into a tç~r~y location DDD. the ~ dahun of the cod~ w~ e the be~ , dot of the contour, ~is da~n is also the Yx of the coq~o~, the dot will be recorded and will bs ~no. 1). The datum f of Y will be mi~ and will bs c~T~lP.te~1 later. A~e51WdlJ~i, each pQir~ in the f~) LS
lly3 passed ~ough and the dot is reg~ste~ad acc~ ~ly ~in. the first line it will be (no. 2)~, one a~er the other up to the .. ;.. ;~.. ~ point Xn(cn), the d~a are co~ned fo~n the DDD and it ~11 be ~no. 33, the Y will be m~ed and ~ere u~ space for its f~ ~en the f~q3 of the X a~s is added to the S
tsee fiIrther (c)].
a st~ t line movement ~n~ l, veItical), one of the axes ~Y:X3 lt~
c~r~t~t (static), ~ile the other a~ ces (1) step m CQ~ (~,-3. The detection of an adjacent dot wi~l be camed ou~ on the right of ~e movemer~
a~s (a~ gO~ an~e), as follows~ r~vement ax~s ~ static and ~le axis was sta~ic adv~ces by (1) step (see fi~er a, c, e, g), until the dot fb~er AhlF~l~ED St~E~T

, CA 02237886 1998-05-13 ~ ~ ~ 96100 1 4 5 ~Vs ~ ~ FrB ~998 down the st~ai~ line does n~t bel~, arymol:e to the sams contour. Fr~m th~s poirt (ou~side the cor~our3 acc~ ly (see co~ l ~e al: b~ h).
In the m~Yt~ll~l~ on a ~ e ~at adjacer~ poir~s or~y ~he follou~
pos~ibiliti~s e~st: l~. 13~~, 2~.225~, 3).315~, 4).45~3 ~le two axes (Y:X) advar~e ea~h time by ~1) step. Detection of an adjacer~ dot and on the n~ht of the movement direction there are two, the one (at 45~ ar~le) and Ihe other 90~ ar~e) [~m heIe o~ see ~i k m, o)3, nT~il the po~ ther down the ~i~l line no lo~er bel~r~ to the same co~o~. Flu~ll t~is poirt (ou~side the con~our) [see filIlher on (j, L n, p~3.
Y~X-1 ) 1 ¦IY~ X~(?~ I
¦(Y+ 1 )(X~;~?~ I
2 ~Y+ 1 )(X) I ~Y+ 1 )(X) ~Y 1 )~X) J
(Y-1~X~ I
3 ~Y~X+ 1 ) ¦ 1~1 1~1 ~Y~X 1~;~?}
(Y)(X+1~;~?~
~Y~(X+ 1 ~
(Y)~X-1 ) a) (Yx)~X-1);(7-lPb);~y-lx~;~7-y~;(y+l)~;~
b3 ~ l);(?-y~i~,(Y+2~X(?-y~kx~ x+l~;(?-y~c~;(y-l)~pe Y+lxxn(cn));(?-Ipd~;~y~-l);(?-y~a~;~y~+l);>c d) ~Y+lXX~ ?-y~k);~+2k(?-Y~m~;~Y-lX~;~?-y~e e) (Yn)ÇX+l);(?-~ Y+l)~,~?-Y~);~ >e f~ (Y+~ ?-y~m);~y-2)~;(?-y~o);~-l);(?-y~;~+l)(xpa 7-rph);(y~+lk(?-y~e);~y)~-l);~g h) (~+l~;~?-y~o~,~Y)~-2);(?-y~i~,~Y+l)~k(?-y~a);~+l~

j) ~Y)~+l~;~?-y~;~Y+lX;X-l);(?-y >aX(Y+l)( X);(?-y~k);(Y-l~ÇX+13~m k) cyx+lx~cn~l);(?~ '-lx~;(?-y~ -l);(?-y~i);(y+l~x+lpk 1) ~Y-l)~;(?-y~a);~+l)~x+l);(?-y~c);cy)t~+l);(?-y~m);~-l)(x-lpo m3~yn+l)~cn~+l);(?-rpn~;(~x-l)s~?-y~a);(y+~ ?-y~k)~ x~+l~m n) ~')(X-1);(?-y~c);~-1)~+1);(?-y~e);(Y-1)~;(?-Y~ y+lxx-lpi o) ~ ~+l~;~?~ y+~ ?-y~c~ x+l);(?-y~m~;(y-l3~-l~o p) ~Y+1)~;(?-y~e);~ -1);(?-y~s(Y)~-l)~(7-y~ y+lx~pk At each cluesffon mark ~?) if it is ~ti~e and if needed the req~red dahmL
r~st ~e ~e~ste~d (in I~ ti~l to that dot an~ ths ~ ~aus ~ns, ~go to, ~D,y~yes).
AMENDED SHEET

CA 02237886 1998-0~-13 ~/IL ~ ~/ V U 1 4 ~
1~A/US ~ g~8 2. ~ a return mo~emer~ on a line (uith ~ ~l lines, the previous recordirlg of the c~tour dots ~st be taken ir~o cnn~:idP~ and to ad~ust the q ~es!inns:~, the cor~our dots will be registered tuice, once fo~ ar~ once back and the necessary o$~tinr~ are camed ou~ ~e~..l;-~, the code m~fication, ~ 1~h~n of a dot to the sum (S), 1ll;ll;lll--lll, ll~x~ll~ull, etc., all in a~cor~1~rr,e ~th the prev~ous rules.
3. Du~g sc,~ , ~ile detecti~g the cor~our for a depth line or area, or in ordiernot to get out from the depth line, the code sho~d be asked (in ln;.l~ l moYement on the Y~cn) dot OqL the s.;-,.. .;.~ code and in movemer~ on ~e dots on the consecuti~e code) an~l SO both, one a$er the other.
4. The corfour of an a~a is found wit~n and ac~acent to the corfour of a depth ~'~ line, o~ betweenthe corltours of two depth l~s and aL~acent to them, and af~er detection of the depth line it if possible to detect ths cor~our dots t~le (;n~R1T~1 and e~errEII).
g c~our ~let~ct~ ~en on ths consecutiYe codle, one sh~llld ask ~r the next dot is the inLtial point, in depth lines and areas found wi~n the depth lines, e~en if the dot is not an end dot in the p~cture da~a (it colild be an end d~t of a cor~our~.
6. The rules in~icated here are an example for cortour (letec*~n and they do notin~luAe fiill ~lP.t~, some of ~n are ~ncc~ ete and ~}erefore ~st be co~eted, and if ~e color fi~ne cantains in it i~rlA~tinr~ ames of o~her coaa~, ~AAiti~l r~es mllst be adde~
7. DuIi~g this an~or other ~cesses, Aat7~ and basic temP, of ~Ttifi~
intP.lli~n~e are detected fo the de~ lfor ro~ot systems tha~ ~11 help in decoding and c~ection. For e~ample, when one of ~ axes X or Y in aly fian~ is n~ mzn~cRA ~lh t~LR l.. ;.,;..... and ~ , it IILans t}~t u~ ha~e a st~ht line ~in~ iT~ the ~ and l~ end pc~ s). M~eover, ~e basic terms will be defined b~sed ontl~ da~a as follows: l) X is n~t defirLd forthose dots for w~ich Yx is, so there is an 'b~er hn. ;"..~1 strai~ line"; 2~ X
L~ nLX defined for th~se dots for ~ich Yn is, ~erefore there is a '~lower h~ri~ ti~l strai~ lin_". 3) Y is r~ defined for those dots for vs~ich X~ is~
~erefore there is a'~3t veItical s~ ~ine". 4) Y is nDt def~ned for those dots for w~ch ~(cn) is, theref~re therR is a "lefl: vertical st~ line" .
~3. At the end of each tre~t~RT~ 3tor in the course thereof, the codes table, the d~ the co~ours~ t~ definiti~r~ and th basic terms for ?~Ttifi~ qlli~n ~e AMENDED SHE~T

. CA 02237886 1998-0~-13 ~T~L ~/00 1 4 5 11~4qlS O ~ FEB lg98 wi~l be ~ dto the M~;M~ 2 "~n"~ 1) -~yl~yliate for each ~se, ~orfi~thertrea~ner~byP.C.U-(3;4)~cessols.

P.C.U.~3:4~ Detect l~$DvemetnJM~tlon, Geome~ l Lines, Geometric~l Shapes~ c Telms in Artiffc~l Tntf~ ence, etc.
The P C U-~ processor h~rullRs the movement~'m~il)n ~n a c~ e way betweentwo (consecutive paces) ou~ oft~ three followir~ ..~ .s inreg;~rds of ~ich the M~;M-l;M-~ d~ta h~ been cn~ tR~l and wi~ t~.hi~ of coor&tes.
1. On~ of the pos~ ilitiRs is: conrllin~tRs entered ir~o the first EnctuIe ~11 c~ uLe ~e basis for coordinates f~r ~L
. The sec~ possihi1i~y is: t~t the robot u~ll calry out c~lc~ t~ for ~e coc~ s accord~g to its ~ and ths deviatioql of the e-~
~31 te~po~ly u~ e the c~ ' shootir~ data 3. The co~e~ vision de~'lc8 Call also ~ , fo~ e~l~le? th~t in ths f;rstp3ct~e, the h~i7~ ne~s fiIst point on a strai~ht east pl~ne an~for a~y poir~ is the O point, u~ ar~ ri~ beiT~g ~ive and down and le~ being nsg;~live? ~1 each and every pQxel at "~x;".,..., Rnl~ is a COf~ line, and so it will updats t~Le coor~in~tes fi~n ths data of a E~svious picture to ~e d;~ta of a cl ." ~ e picturs.
~, 4. h~ of the coM~tes and their ~ lio~ in ths hsader, may be c~u~edout f~n ~Le co~l.lil~Ls dal:a of a previous picb~rR to t~R data of a pictllre u~ch ~s iT~R~.C to p~ onn movementJmnti~n detection? by m~tchir~ of a sir~ls ~sIr~ll~ mobile icl~Ttified contour w~ich, acc~ng to the data of the robot and the C~l l ~l i'l?; mov~IrLert is ~md in t~LR data of both paces.
5. Based on ths d~ta ~ered u~ ur~l t~is phase, a cll".~ e t~ P.rt is p~. r~ .d bet~sen two mRlT~nes re~; each contQur, dep~ lins and color sepa~ely, between ths changs in the length of Y and X aml ~ls cor~esporllling values in the sec~d previous . . ~. . ~. y, for moveme~lmotion detectioqL
6. The c~s in the data may be ~l~tecte-l r~lly, accor~ing to their order of appea~nce from left to ri~f an~ from high to low an~or acco}ding to the order of their vicil~ty to t~Le c~~ , n~nely, ~m the la~est Wr up ur~l the end of the cor~our in the ~ the ~ st Wr), and they may be detected by two or more processo~s, sho~d it be requ~led.
AMENDED SHEET

CA 02237886 1998-05-13 ~ ~ ~ 9 6 /0 ~ 1 4 M~JS ~ 6 ~B

The P.C.U~ ~cessor ~~ contou~s accordir~ to va~ous cnten~, such as rr~ca~on of the~r distance ~om the er~ or a lmif~m ~T~vement direction, belnrq~r~ to a defined detected genTnP.trical sh~e, etc, ~nd accordillg to the order of their vici~y to the Cdl~ and Y~ch 1~ not yet been i~1ertifi~,d rthey ha~e no name ~ the spa~ial .~..~.~ MM~ ~-1), nor p~or j~lR~tfi ;~ti~>n ~loca~ion of the name in the MM~ ~1 be Ir~e accordir~, to the coo~din~es of the two contours), ar~ gives them defim~ ;, if pos~ihl~, such as:
a) A~acent contours ~aTea~" coloIs and depth lin~s) irL ~ich thers has been ~ no movement ~soe~r, the ove~l contour wi~l be iflP.tt~ifiad as anadditional cor~our ~ h~ l~RT$), and it will be given a tsIr~y name BAIL The si~ific~:e ~th reg~ls to ~tifici~l it~f'.ni~t~e i~

b3 A~acer~ contouls ~ }7ad lmtf~m mo~ in SlZe ~ direction - their contour ~11 be ~1~tecte~l and registered as an s~ inrRl contour (il~
the hea~ler) and ~I:t be given a t~TT~ny name BBn. The ~i~r~ificance ~uth ls to ~tifici~ lli~t'r8 iS "mobile".
c) The coItour of BBn inside ~ich there was ~ ~ll (or inside ~ich a~y c~r~s occurred) differer~ ~oqn the ~ l in size an~ in ~ection ~11 b~ gi~en a tel~l~y namB BC~L The ~i~ific~e wil~
to ~r~ifiGi~ s~e is "alive".
d) A~acent caq~ours with (~i~ificant) Wr di~ LC8 between them and their ~ enbl8 en~"--""~,- Wi~ be g~venthe ten~porary nErne BD~ The si~cance ~hre~s to ~rtifi~i~l i~e11i~R~e is "floa~'.
~f ths~ ~s movement wi~n the contour (and it~ ;nn ~ n the given ~) it will be g~ven the tRTrl~ name BEn and ~he ~i~ific~e is ~ed ~
e~ ~n ~he eve~ that this processor has some time left an~lor aTy other liti~l p~cessor~s) will han~31e with~u~ ~e the same ~..r~y M~;M~ 3, and it ~11 detect definiti~ in the contours' dots and between the ~.aqkours that foIm a c~rq~l~te f~ame ~sha~e~, stIai~ lines, concave lines, arched (con~x) lines, ~ ~ral lines (irnluf~ le), and aIy other i~lan~ifi~hle g,s. mRtTical line, it ~1 also detect geometIical shapes such as: rectan~e, square, tri~n~la, circle, ellipse, etc. and it ~11 est~bli~ for them hasic te~ns of ~tifici~ p.nce.
AMENDED SHEET

CA 02237886 1998-0~-13 ~L 9~100 1 45 ~ FEB 199~
-- ~0 f~ l~s, co~ours~ etc. for w~ch no ge-~ntP.tncal definiti~ t; kr~wn, or the detectioll of u~ich is ler~y, ~1 bs mz~tl~.~A ~ n~t a Ieg~ster of ~asic a~ lmown s~es~corfo~s MM4 ~ig.-l) ada~ted to any coIr~puter v~sion in accordance with its filrr.ti~n and aim (in rin~irl~, not excesding 256~, by n~tcl~r~ the si~:e of ~he contours to the basic co~ours saved in order to ~ix a definitif~n for the above ~ fconto~s.
. ~ga~l/oru~?ontt,~ ;cn ofthe rr~c~ md detec~on ofthe IT~vemen~, te vaIious ca~ou~s etc., the dsc~rhPTi~ system an~o~ ths robot ~11 lmow .l.liti~T~l basic terms in ~tificis~l intP.lli~n~e, such as the follo~nng C~nBS:
a Ths leng~h Y has ~ l cu~ aIld has moved ~ch~r~ location) u~w.~ e ci~ifi~e - "ascen~
b. I}1B ler~h Y has ~ dd c~ l~l and has moved ~ch~ged location) duw lwd~ , the ~i~7ificance - "descsmled".
c. The len~Y becarne ~rr~ll~r, the ~i~r~ific~ e -"go~ shoIteI".
~ Thsleng~hXbecsnnR ~ller~ the ~nificance -"got~l;..,.-~.l".
e. The len~hY bec~ns l~er, the ~i~ificance - "got ~eI?'.
~ X an~Y becams laIgsr, t~a ~i~ifif:ance "a~ru ~ X and Y becams ~ll~r, the si~Tificance - ~ 2.1-- ul~h".
g. D~ir~ g~ ar~or at any o~}er time, Ir~y basic terms in ~Ttificiz~l ;n~R11j~n~e may be ~1atRCtR~1/deriVed aS may be needed, ~ATich may help in deco~ t~he s~es aT~or the pllRm~mRn~ an~ r the terms a~Jo¢
c..... ;c~ etc., ~nm~yways, suchas hllTr~n lsn~
10. A~ ~ and of each l .t;~ an~Jor ~Tnr~ it~ t~a t~le of co~as~ a, conto definiti~ and the basic temF of ~rtifici~ alli~en~e ~11 be sent to y MM~ (sp~ial malT~ny3 according to ~heir bel~ and their locatio~
accor~g to coordir~es, for ~er l Pr~ificatic~nby p¢ocessors P.C.U~5;6...) ~Fi~-l).
11. ThB spa~ial IrLamory - ~ J will co~rer a ceIt~in space, ~o~ e~r~ple: a~
h~n7r~ nLa 180~ and at ve~ical line 120~ an~lor a full spa~al ~e, all as desiled AMENl~ED SHE~T

CA 02237886 1998-0~-13 ~T~L 9 ~/o O 14 ~
R~US a 6 F~B 7g~

Shape Identi~cation Key, Idenliflcalion The data, contou~s, c~cteristics, ~ ~, de~nih~, ct~rrln~ion~, etc.
d du~ a~ detectiMl are ~ ~d ~n the va~ous tables acc~ding to their b~ ar~ loca~an ~n the spa~ mRm~Ty MM~ in a speci~ order, ~ted to the same order in w~ich t~ d~a of the sto$~d shapes R~e ;~ ~d) and they r~ 1~RY factors foq mR~lnnE nd id~ Rtion (da~a R~7in~t d ta) an~ r acc~ng to the '~tr~' table~ 8tc.
The nT~lR~ifi~d areas ~contours) are Yery few at each s}~oti~ ~e and ar8 usuallyat the mR~;in of the picture. The d~:a of lmi(1Rr~ified s}~es".. . ~. ~ ;1 in a special order, c~ R key f ctors, ~ich aUow, by all or part of the means, as we~l _s by fi~quency, the"tru~' table, etc., like aw~rdinadictionaIy, ~enthe cl~ification is l~ R and t~R order of the key co~poqler~s are the order of letteIs and ths o¢d as a whole is the key, to detect, match ~1 identify a EnctuIeJrecord ~ilc c~ , it to o~her pichlres/~cords an~Jor da~a cll..q~.~l to data of known and stored shapes ~ i in the same special order, adgusted according to ths key co~nents, acc~&lg to 'certain'5 '~etween and between', 'reasor~le' ar~
'possible', 1. The ;~1R~t;fi~t;~n is p~. fil....~1 betweenthe ~ta of the viewed sha3?es register MM~ and the d~a of the ~l~t~ register of the constan~, lcnown, stor~d sha~es M~I-l ~E;~-l~, inthe ~ IL~ taMM-O, the s~ze, aIl~lB andandshooting e lT~t~hirg will be p~. r~,. ..~1 according to ths distance an~or en~ hll~,ti~ t~ and accor~ ng to the l~ of s~e sh~otir~reco~di~gar~e ofthe (l;.l~l~;nn~ dataofthe sta~eds}E~e.
2. The d~ ~or shapes stored in the MM-l may be obt~ned fr~n decir~r~ ~e d~ta ~ each ~nd eveIy sha~e by preser~g the sha~es for com~uter visior~
accordi~, to s~ ze and distance adjusted to ~e m~tclliT~E, key, an~l/or they may be deiined by meaTls of logical analysis, under the same cor~itinn~.
The sh~pes will be defined and ;~ pd accord~ng to a ceItain ord~r r7ion~ such as color, ~,~.~tn~ l shape, etc., that c~...r~....~ to ~he ds~irl~Tiry~ key ar~l they will be ~ C~ifi~d as 'certain', 'between and between', 'reasonable', and'possible'.
3. There will be a movemer~ MM-2 ~Fi~-l~ ~: will store, for e~T~le, th~ a of all ~he c~s of"alive" sha~es QT~y and~or u~ to a certain distance AMENDED S~IE~T

~ CA 02237886 1998-0~-13 P~L 96/~0 1 45 IPE~US ~, 6 FEB Igg8 or~y as desired, every pulse, eveIy tenth pulse, every secor~, etc., for a ceItain tilrs def~I}ed in secon~s, hours, etc., for yllIpOSB of movementJmo~on speed calc~ i~n 4. It is possible that ~n ~is mP.mh~Ty or ~n ~n~1~r, all the cl~r~ of moYemer~motion at each pulse and pulse uill be stored~ ar~ thlls it ~11 ~e possible to obtain sto~ge of 3-D data an~or lTnllf~m~dia which, it wi31 be po~sible to display aT~or proiect, etc., in ~ r time and place, after ti~n anddecodin~ar~e~renen1~
5. It is also possible to have ~ teA Illr~ 3 ~ig-l) which will coIr~se the contouls of the space in w~ch the ro~ot 7 ÇFi~-l) wi~l fimction, irEide the space cor~ours the important s}~es and the sp~ce contours will be in~1ic~t.s~1 in the location w~sre they are ~n focm of contour ~h few ider~ri~
details7 such as: name ~w~th the ~AAitinn of one or two data an~Jor o~e or two words), ~n~l eveIy t~me t}ELt the robot is in some place it u~ll know the locatioql, and in any case of cha~ge of loca~ 7 it will knc~w that there has ~een a ch~ge of location (~ich change and whes to?3 in the coa~tours, in the location of the ~l~es7 etc., an~ ~11 u~ e the ~ te~ space in acc~lJ~de wi~h the new reality and ~th the si~le methDd of re~ an~ e will be a PM
"~~ r(Fi~_l) ~or~cei~exter~ d~a vl ' ~ r~
1. The C~ IIIHI v~sio~ system sh~l be ~uu~liately protected ~E~iT~t h 1i~7 fl~ hts of any 1~1 wha~7oever i~.1n~ laser and an~ere it may ~e situated - ~in~t aly other possible p~ysical i~ury, as ~ar as po~sih1~
. The c~uter vision system will be coqr~tible aT~ ll opeIate in accordance ~h the user7s in~ needs.
3 The computer ~ion system will corr~?~ise sofl:~re p~ns 6 (Fig-l), electr~ic and ~n~r~1 c~rcles by w~ich the system ~11 function7 aml a so~wa~
pro~am ~ich w~ll ad~ust the size :~n~l phot~ ffilmir~ ar~e of the viewed sha~e in accordance with the distance ar~for ~nl~ Jreduction data a~ in accordance with the st~ rds of si ze and fil l l l; l ~1~;cord~ng of ~he stoIed sha~e's (l;~ data, as well as any other so~ware ~n for the requ~red da~a u~Lw~ls such as for b~adcast~ 3-D
present~tion and ~nlltin~dia, and any o~her so~cware pro~L that ~ht be necessaIy AMENDED SHEEr CA 02237886 1998-05-13 ~ML ~6 / o ~ i L~ '~
IPE4/lJs n ~ rr~ l9g8 ,9 4. The receivedpictures, codes, data, calc~ ior~, ~ictuIes inthe spatial-..~..~.~r ar~ any other i~rU~ i~ concermn~, the received p3ctures or m~s or d~:a stored in aly of the sy~tem mp~ es m~y ~ ser~ out as ~re, irr.lllf1ir~ the da~ c~r~y the codes ar~ d~a, an~ in~ or sp~ti~l stereo ~ictures, to aly user such as a ro~ot, accordi~ to re~ , design and ~th a~ thod, and ~ user ~11 bc able to dIaw d~ as he wishes ~nd in ccordance wi~h the system desigrL
5. All ~le abo~ r~nT~d ope3~ti~ ~11 be camed out by ~rocessors and ~l~tinr~l means of ~rocessir~g for m~tf~hir~ (1Rtectif~n pr~cess~g and calcnl~tinn They will use C~ ~uLt~l de~ces, coIT~poner~s, ons or ~
elecb~oqic circles and aliks, and any c~ inn thereof req!I~red fior and t~ to the puIpose O~Cv~ly~ulBl v~on The work dis~bution aTr~g them will be such, ~hat each cr~er~, p~t, elec~nic circle, system, etc., will ~lr~ its task ~hou~ R- rn- ;~ ~ one 1 there will be full coolrl;.~1ifm and cf~T~hhili~ r~g a;ll of them, ~or e~rl~, prwessor no. i;.. r.. ~ ~rOCBSSa~ no. ~ by means of a af~nponer~' electr~nic circle, etc., that it has firrch~ s paIt in oq?Rl~h-nll A and that p¢ocessf~r no. 2 may Cf -~ R C~1ing 01~ its p~t in Cf~lRX Opf~tif~ A, etc.

Commercial ~mr~-~en~ation l . Co$r~lter vision may be used $~ a As aviewer(watches, analyzes, dscodes, reports, I.;t.~l.. ;lx, etc ~.
b. As a viewer that collects data and preserves them in aly da~ base or re~ t~r in any way (fi~ly, pa~ly, by aTy form of cl~x~ififa~i~ or so~ etc ~.
. Du7e to the ~ )l;city of posxi~ilities of use of co~puter Vi~lf~l, seveIal 1 computer vision systems may be used~ ~ere each ~ l system is ~-l~tRd so as to pro~ide ceItain services and perform ce~ain tasks, and thus it will be a~able '~on the shel~er", w~le for special ~ r-~t~7 a coqT~er vision system adapted to specific needs U~ll be ~RSi~R!l 3. Each c~11~uLe1 v~sion systemwi~l ~e ~x~i~;nad standard and specific ~l~lities.
4. It is possible to adapt the colTqnlter v~sion system to the specific needs aT~l r~4L il~11~1~i of aly user w~o is a designer or a constructor duI~ng ths sta~e of design, ~ to integrate it as palt of the user, or as a se~i~Le u~t tha~ serves the user AMENDED SHEET

Claims (9)

1. A system for stereo computer vision including:
a. A pair of identical, aligned, coordinated cameras at fixed distance M, with coordinated shooting angle and enlargement/reduction, creating for the photographing cameras optical parallel fields of sight, coordinated, aligned, with identical field of sight line in both cameras from (0:0) and up to (Y:X), at a fixed distance M at any photographing distance, and the pictures of said cameras are received by means of input-memory devices in the computer, translated to computer language;
b. Based on step a, the pixels of the pictures that were received may be treated with various methods, such as: 1. the pixels of the pictures are matched by means of the devices the one against the other, data are collected/detected for a header for encoding, including pulse/pace, enlargement/reduction, color, the difference of dots Wr between the two cameras and the same points is equal, location, the number of adjacent horizontal dots of the same color and/or distance or depth lines for 3-D, etc., 2. The pixels of one of the cameras are matched by the devices through matching of coordinates by order according to the speed of shooting and/or other, as against the pixels of a previous picture stored in the spatial memory register, when any new movement and/or shape create lack of matching and then updating, the pixels of both pictures are matched the one against the other, the data of identical pixels are recorded in the spatial memory for each color pixel, Wr difference, etc., the non-matching pixels are recorded as depth line dots in the space picture and the pixels of the other picture are recorded separately, in connection, etc.;
c. Based on step b, the devices and the datum Wr allow to calculate sizes, dimensions, distances, etc., and they also help in distributing the work between processors and in reproducing 3-D pictures and in time, etc.;
d. Based on the previous steps, contours for colors, depth lines and areas are detected, as well as contours for groups of areas, depth lines that are adjacent, and/or having a uniform movement, which are close, etc, and they are added as data tables to existing data, and the size of vertical lines in each frame is detected, geometrical lines and/or geometrical shapes are detected, and through the matching of the size and the angle of the contour with the contours of stored basic shapes, etc., throughout the detection/comparison, definitions are derived, such as: straight line, circle, triangle, concave, inanimate, floating alive, etc., for artificial intelligence,while external auxiliary terms may be attained, such as: measurements, audio, taste, smell, palpation, etc., that will help identification and/or communication with the environment;
e. Based on previous steps, the data of consecutive pictures are compared by means of the devices, by matching coordinates and/or coordinates are matched to consecutive data by matching a contour to an identified static shape found in both paces, then the shooting deviation angle which allows the robot balancing is detected, movement/motion of the horizontal and vertical lines in the various contours is detected, the extent of change, direction, angle, etc., is calculated, the data of the changes, stored in paces at determined intervals allow, through matching of coordinates, to calculate the speed of movement/motion, part of the detected data may be stored in various ways, pace changes may be stored one after the other, may be stored, broadcast, transmitted, etc., in various ways, as 3-D data for projection/presentation in any way, at any time and any place after translation/deciphering and even enlargement;
f. Based on previous steps, part or all the data collected by means of the devices may be stored in the spatial memory, each datum according to its belonging and by matching of coordinates, the data of unidentified shapes, arranged in a particular order, form key components, allow, by means of all or part of the devices and by incidence, "truth" table, etc., and according to a recorded picture/record/datum size, angle and distance, to perform matching/detection/identification of a picture/record/datum as against pictures/records/data of the known and stored shapes arranged in the same special order, adjusted to the key components, according to 'certain', 'between and between', 'reasonable', and 'possible';
g. During detection, the unidentified contours and their data will be recorded by a temporary name and the identified shapes will be recorded by their own name, artificial memory may be added to the environment in which the robot operates in the form of contour lines with indication of the names of the contours, with the essential shapes and a few identifying data which will be occasionally updated by the robot and/or the vision system.

2. The computer vision system claimed in claim 1, wherein said pair and/or more cameras can be packaged in a single casing or separately, are similar to video cameras, including CCD cameras and including cameras in which there are combined means for converting data received from pictures taken by the cameras into digital data, and includes one or more of the following:
a) Adaptation for color photographing, at various speeds and at any light such as IR, visible and at any lighting conditions such as meager, by means of light amplification;
b) Enlargement or reduction devices, including telescopic or microscopic means;
c) Optic image including the desired resolution, whether straight, convex, or concave.

3. A system of computer vision as in claim 1 and 2, in which the main devices for collecting and storing detected, compared and/or computed, defined, derived, viewed, etc., information on data that can be defined as a datum (such as:
codes table, codes of size, color, contour(s), geometrical lines, geometrical shapes, definitions and/or normal term(s) and/or in artificial intelligence etc.), which allow access to the data in order to identify stored shapes, as the mattermay be, with regards to dimensions according to matching of size and angle to the size and angle of the stored data and by matching of coordinates and classification according to the key components in one or more hierarchic order that has been established in advance, as 'certain'. 'between and between', 'reasonable' and 'possible', whether internal or external, they include one of the following:
a. Input memory for receiving the pictures of the cameras/the viewed pictures;
b. Spatial memory for one or two cameras and/or data for 3-D that will be integrated by marching of coordinates, with the method of picture preservation, all the pictures obtained from fixed enlargement/reduction recording and other, in another memory, and in addition to other data, separately and in connection, for a second picture, with the method of encoding, which includes the data of both cameras, the data and the tables will be joined together in relation to a certain section of a defined space, updated and adjusted at each and every pulse, the pictures and data will be stored expressed in horizontal and vertical circumferential coordinates, and will be updated all the time;
c. A register in the form of table(s) for stored shapes that are known, which will include pictures, maps, sign, (including data and tables related to them), data, codes, etc., adjacent, above, in connection, etc., to any shape that can be defined and/or given a name and/or identified separately (such as a screwdriver, a finger, a wall, a background, a sign, a phenomenon, and anything else), registered in a particular, known, order adjusted to key elements. data which are dimensions will be adjusted to the photographing/recording standard depending upon the size of the shape, its d. A register for the detection of the movement at various paces, the changes inpace are preserved and adjusted at defined and known time interval(s), and based upon the extent of change and the number of paces and shooting frequency, the size and speed of movement can be calculated;
e. A register for storing all the changes in the codes, data and horizontal and vertical dimensions at a pace one after the other, will allow at another time and in another place to present/project the pictures again, after translation/deciphering and even enlargement;
f. A register for contours of basic shapes which cannot be defined and detected within a reasonable period of time, which will allow to detect geometrical shapes as compared to the contour that has been detected, as against the basic contours, and thus definitions will be derived as a supplement to the key data;
g. Memories, registers for collection and handling of table(s), encoding of their data, tables for contours and their data, table(s) for terms, trigonometric and other definitions, basic terms of artificial intelligence, allas may be required, etc.;
h. Registers for auxiliary data received from devices such as the robot, audio devices, sensing devices, etc.;
i. Register for software programs with computing detection, comparing handling and identifying capacities, for performing any operation required for the functioning of the computer vision device, in order for the device to fulfill its function.

4. A method for stereo computer vision including:
a. A pair of identical, aligned, coordinated cameras at fixed distance M, with coordinated shooting angle and enlargement/reduction, creating for the photographing cameras optical parallel fields of sight, coordinated, aligned, with identical field of sight line in both cameras from (0:0) and up to (Y:X), at a fixed distance M at any photographing distance, and the pictures of said cameras are received by means of input-memory devices in the computer, translated to computer language;
b. Based on step a, the pixels of the pictures that were received may be treated with various methods, such as: 1. the pixels of the pictures are matched by means of the devices the one against the other, data are collected/detected for a header, for encoding, including pulse/pace, enlargement/reduction, color, the difference of dots Wr between the two cameras and the same points as equal, location, the number of adjacent horizontal dots of the same color and/or distance or depth lines for 3-D, etc., 2. The pixels of one of the cameras are matched by the devices though matching of coordinates by order according to the speed of shooting and/or other, as against the pixels of a previous picture stored in the spatial memory register, when any new movement and/or shape create lack of matching and then updating the pixels of both pictures are matched the one against the other, the data of identical pixels are recorder in the spatial memory for each color pixel, Wr difference, etc., the non-matching pixels are recorder as depth line dots in the space picture and the pixels of the other picture are recorded separately, in connection, etc,;
c. Based on the step b, the devices and the datum Wr allow to calculate sizes, dimensions, distances, etc., and they also help in distributing the work between processors and in reproducing 3-D pictures and in time, etc.;
d. Based on the previous steps, contours for colors, depth lines and areas are detected, as well as contours for groups of areas, depth lines that are adjacent, and/or having a uniform movement, which are close, etc, and they are added as data tables to existing data, and the size of vertical lines in each frame is detected. geometrical lines and/or geometrical shapes are detected, and through the matching of the size and the angle of the contour with the contours of stored basic shapes, etc., throughout the detection/comparison, definitions are derived, such as: straight line, circle, triangle, concave, inanimate, floating, alive, etc., for artificial intelligence, while external auxiliary terms may be attained, such as: measurements, audio, taste, smell, palpation, etc., that will help in identification and/or communication with the environment;
e. Based on previous steps, the data of consecutive pictures are compared by means of the devices, by matching coordinates and/or coordinates are matched to consecutive data by matching a contour to an identified static shape found in both paces, then the shooting deviation angle which allows the robot balancing is detected, movement/motion of the horizontal and vertical lines in the various contours is detected, the extent of change, direction, angle, etc., is calculated, the data of the changes, stored in paces at determined intervals allow, through matching of coordinates, to calculate the speed of movement/motion, part of the detected data may by stored in various ways, pace changes may be stored one after the other, may be projection/presentation in any way, at any time and any place after translation/deciphering and even enlargement;
f. Based on previous steps, part or all the data collected by means of the devices may be stored in the spatial memory, each datum according to its belonging and by matching of coordinates, the data of unidentified shapes, arranged in a particular order, form key components, allow, by means of all or part of the devices and by incidence, "truth" table, etc., and according to a recorded picture/record/datum size, angle and distance, to perform matching/detection/identification of a picture/record/datum as against pictures/records/data of the known and stored shapes arranged in the same special order, adjusted to the key components, according to 'certain', 'between and between', 'reasonable', and 'possible';
g. During detection, the unidentified contours and their data will by recorded by a temporary name and the identified shapes will by recorded by their own name, artificial memory may by added to the environment in which the robot operates in the form of contour lines with indication of the names of the contours, with the essential shapes and a few identifying data which will be occasionally updated by the robot and/or the vision system.

5. A method for computer vision as in claim 4, wherein said pair and/or more camera can be packaged in a single casing or separately, are similar to video cameras, including CCD cameras and including cameras in which there are combined means for converting data received from pictures taken by the cameras into digital data, and includes one or more of the following:
a) Adaptation for color photographing, at various speeds and at any light such as IR, visible and at any lighting conditions such as meager, by means of light amplification;
b) Enlargement or reduction devices, including telescopic or microscopic means;
c) Optic image including the desired resolution, whether straight, convex, or concave.

6. A method for computer vision as in claim 4 and 5, in which the main devices for collecting and storing detected, compared and/or computed, defined derived, viewed, etc., information on data that can be defined as a datum (such as: codes table, codes of size, color, contour(s), geometrical lines, geometrical shapes, definitions and/or normal term(s) and/or in artificial intelligence etc.), which allow access to the data in order to identify stored shapes, as the mattermay be, with regards to dimensions according to matching of size and angle to the size and angle of the stored data and by matching of coordinates and classification according to the key components in one or more hierarchic order that has been established in advance, as 'certain', 'between and between', 'reasonable' and 'possible', whether internal or external, they include one of the following:
a. Input memory for receiving the pictures of the cameras/the viewed pictures;
b. Spatial memory for one or two cameras and/or data for 3-D that will be integrated by matching of coordinates, with the method of picture preservation, all the pictures obtained from fixed enlargement/reduction recording and other, in another memory, and in addition to other data, separately and in connection, for a second picture, with the method of encoding, which includes the data of both cameras, the data and the tables will be joined together in relation to a certain section of a defined space, updated and adjusted at each and every pulse, the pictures and data will be stored expressed in horizontal and vertical circumferential coordinates, and will be updates all the time;
c. A register in the form of table(s) for stored shapes that are known, which will include pictures, maps, signs, (including data and tables related to them), data, codes, etc., adjacent, above, in connection, etc., to any shape that can be defined and/or given a name and/or identified separately (such as a screwdriver, a finger, a wall, a background, a sign, a phenomenon, and anything else), registered in a particular, known, order adjusted to key elements, data which are dimensions will be adjusted to the photographing/recording standard depending upon the size of the shape, its dimensions and the photographing/shooting angle;
d. A register for the detection of movement at various paces, the changes in pace are preserved and adjusted at defined and known time interval(s), and based upon the extent of change and the number of paces and shooting frequency, the size and speed of movement can be calculated;
e. A register for storing all the changes in the codes, data and horizontal and vertical dimensions at a pace one after the other, will allow at another time and in another place to present/project the pictures again, after translation/deciphering and even enlargement;
f. A register for contours of basic shapes which cannot be defined and detected within a reasonable period of time, which will allow to detect geometrical shapes as compared to the contour that been detected, as against the basic contours, and thus definitions will be derived as a supplement to the key data;
g. Memories, register for collection and handling of table(s), encoding of their data, tables for contours and their data, table(s) for terms, trigonometric and other definitions, basic terms of artificial intelligence, allas may be required, etc.;
h. Register for auxiliary data received from devices such as the robot, audio devices, sensing devices, etc.;
i. Register for software programs with computing, detection, comparing handling and identifying capacities, for performing any operation required for the functioning of the computer vision device, in order for the device to fulfill its function.

7. A computer vision system and method as in any of claims 1-6 wherein: a computer, computer components, components, electronic circuits, a processor, a computerized data processing system and alike, one or more of them, form one or more of the said means or any combination thereof, the said means, known software programs, new and compatible, with an ability to perform whatever if necessary, said means for converting data obtained from photographed pictures into digital data; the said means for matching, comparison, detection and definition of color, code, characterization identity, of regular type and/or by artificial intelligence, etc., with regards to a viewed point and/or any adjacent point and/or contours generated in the cameras pictures and/or stored in any register whatsoever, said means for preparation and detection of tables for codes, data, for contours, terms, features, definitions,calculations, basic term in artificial intelligence, etc.; said means for calculation, comparison and adjustment of distance, dimensions, coordinates, angles, speed, etc.; said means for movement detection at time intervals, type of movement/motion; said means for handling of frequent and/or "true" table(s);
said means for data collection for key composition, said means for comparison and matching of registers according to a constant, defined, known order, and/or according to a key; said means for storing various types of information; said means for receiving, transmitting and drawing of data; the said means include input, data protection, said means, the system, the method and alike.

8. A computer vision system and method as in any of claims 1-7 in which: the input and stored information that has been collected before, during and after identification, including calculations, data, features, regular definitions and conclusions and/or basic terms for artificial intelligence which are well known and familiar, that have been collected such as with regard to areas, shapes, is immediately or after a short while transmitted forward in form of data information, speech, picture, etc., in a regular manner and/or by stereo in formof 3-D and/or multimedia, and which the system preserves and/or provides and/or which are accessible upon request and/or automatically to the user, such as a robot, a device, a blind person, by means of adequate interfacing means.

9. A computer vision system and method essentially including any of the innovations herein or any combinations thereof, as described, referred to, explained, illustrated, shown or implied, in detail and in the above claims, or in explained, illustrated, shown or implied, in detail and in the above claims, or in the figures attached hereto.