CN1202239A

CN1202239A - Computer stereo vision system and method

Info

Publication number: CN1202239A
Application number: CN96198304A
Authority: CN
Inventors: 摩西·拉宗
Original assignee: Individual
Current assignee: Individual
Priority date: 1995-11-14
Filing date: 1996-11-12
Publication date: 1998-12-16
Also published as: JP2000500236A; WO1997018523A2; AU738534B2; IL115971A0; KR19990067273A; WO1997018523A3; AU7331696A; BR9611710A; EP0861415A4; IL115971A; EP0861415A2

Abstract

The system and the method are intended to enable robots to see the environment in which they operate, by means of a pair of identical cameras, aligned, compatible and coordinated with their fields of sight being parallel and adjusted in parallel divergence between each and every line in the fields of sight, and to identify anything viewed by the cameras immediately and at the rate of photographing/filming. By means of the system and the method, the computer vision receives the pictures from the cameras into a designated place for this purpose and a backup is created for the field of sight from the pictures stored in the spatial-memory, and it calculates the distance for each and every point in the picture, as well as the dimensions of the shapes and registers their various features. The system includes a memory register for movement identification at time intervals and calculation of movement, motion, speed and direction of each and every shape in the pictures of the leading camera. The data, the features, definitions and conclusions enable to compose keys for the unidentified shapes, compatible with the recognized stored shapes register, and thus identification of the unidentified shapes is carried out fully and almost immediately.

Description

Computer stereo vision system and method

The present invention relates to watch computer picture and identification in real time, particularly, relate to by 2 or more a plurality of " three-dimensional " video camera and watch the 3D computer picture by video camera, a device such as computing machine.

It is necessary that the demand of computer picture is that deal with problems every day, and by people apply to research, uranology, robotics, industry, agricultural, manufacturing industry, service sector, driving, safely, help all spectras such as blind, phenomenon detection.

Nowadays, along with both capacity of inside and outside computer memory increase gradually, the physical size of machine element constantly reduces, and existing various high speed processors.Therefore, if someone can create a kind of can Real time identification its anything computer image system and method for being seen, it is easier and simpler that everything will become.

It is independent, that limit, known and mainly be static object that the existing computer picture that utilizes one or two video camera concentrates on, its identification is tediously long, contrast, the part and only is a focus, and do not analyze and discern the everything of seeing simultaneously by take a picture (shooting).This computer picture needs also to use many devices, for example sensor, light fixture, measuring gage etc., this computer picture trouble, restricted, efficient is low and satisfied resolution can not be provided.

The objective of the invention is by the video camera that is connected to a computing machine, the device that is used for driven dimension, motion and other discernable characteristic provides the system and method for all forms that can see simultaneously with the velocity analysis of making a flim and identification.In addition, the objective of the invention is to make any automatic equipment (instrument, computing machine, robot etc.) see any tangible thing and phenomenon with suitable image device in the mode that the people can see and discern them by different.Make they help and the execution people can make almost any action, task and work, but carry out more accurate and effectively, faster, better etc., and be long-time, muscle power be difficult to approaching or dangerous, can not enter, irksome any place and place etc.

In addition, the objective of the invention is to allow the people by the present invention, by using glasses and/or isolated plant as the 3-D graphical design, or be used for the circuit and the common supervision screen of data transmission, simultaneously even use the multimedia that has multimedia software to watch space around specific region of ＂ with 3-D form of expression ＂ from remote location.

In addition, the objective of the invention is to allow design according to the present invention to construct these devices, use equipment existing a kind of form or other form, system, circuit and electronic component, basic software etc. on the market simultaneously, so that pass through their connections, combination, assembling, expansion etc., according to the invention and/or assemble these the device, these devices comprise the ability that conforms.Any ordinary electronic slip-stick artist, Computer Engineer, systems analyst etc. can design, assemble and construct these devices according to the present invention.

The present invention is made of a kind of improvement system and method for preferred stereo computer image, comprises and operates as follows: preamble

1. that write as software manual and do not constitute as any material of a whole invention process part and only simply mention.Its objective is the process of explanation computer picture operation, rather than belittle essence of the present invention.

2. explanation given below is general, must send the user's data that it is served to according to computer picture, exist the computer image system (for example large and small, medium, Phillips screwdriver etc.) of dissimilar, pattern and size, but their basic principle of operation is identical.

3. computer graphic similarly is the device that a kind of ＂ of providing watches the ＂ business.Its task is to watch certain part in the space, as required to its decoding, when needs it is stored in the register, and further transmits object or the related data of being seen.

If desired, also can design the data transmission of 3-D form, or ＂ multimedia ＂ (comprising the shape relevant with watching object, place, position, size, color, distance etc.), all these depend on the purpose of computer image system.

4. the imagination computer graphic similarly is to be user's service, and imagines this user and carry out limited operation, and his work space is always limited.Therefore, being suitable for this user's computer picture system will be necessary for this user and provide him to finish the data of its required by task.By the suitable data and the help of software, this computer image system can ＂ be familiar with the environment of ＂ user job, distinguishes the shape of the object that can find in this zone, and its sensation is very important concerning the user.In addition, equipment and the instrument in all working place that the user runs into will be distinguished by this system, and these equipment and instrument are that this user must relate to and use in its limitation of liability, reproduce by grade equally.

5. computer picture can help and use the such additional aids of measurement mechanism that resembles storage system, user, or this computer picture can be this system or user's an outside attaching device.Between user's computer picture system and servicing unit, existence is connected each other, compatible, quote, negotiation, reciprocity, data transmission etc.

Fig. 1 has provided according to the synoptic diagram that has the optimum decision system of the possible connection of the present invention includes of two processors with the block scheme form;

Fig. 2 is the synoptic diagram according to the part of system that the present invention includes formation and operation;

Fig. 3 is the parallel qualification line in the center optical axis, the visual field and the visual field of the system schematically described in the key diagram 1 and the horizontal sectional view of optical imagery;

Fig. 3 A is the collimating optics image (amplifying 4 times) of Fig. 3;

Fig. 4 schematically shows with different size with at (system of describing among Fig. 1 is seen) visual field that different distance is seen and the horizontal sectional view of shape;

Fig. 4 A sees in Fig. 4 and by two shape pictures (amplifying 4 times) that video camera is seen simultaneously of the right of the system of describing among Fig. 1 and a left side.Photography (shooting) and video camera

1. identical video camera 3 a pair of alignment, that coordinate in shooting angle (Fig. 1 and 2) comprises variable and multiple that amplify/dwindle, and for video camera produces the parallel light visual field (Fig. 3) simultaneously on common plane, this common plane is called as surface level hereinafter.In other words, in two video cameras, be identical with respect to the distance of every line of sight, be increased to (Y:X) from (0:0), and be a fixed range M (Fig. 3 and 3A) at any shooting distance.The vertical movement of video camera (vertical with surface level) is constant and identical.

The image of being taken by video camera receives this image according to speed of making a flim and/or other factors in input store A/51 and B/51 (Fig. 1) after converting computerese (numeral) to by video camera or any miscellaneous equipment.Can carry out the physics adjusting in installation and/or any other given time by computer image system.

2. the described two or more video cameras in a unique casing or separate packages are identical video cameras, comprise ccd video camera, comprise the device that is used for the data-switching that picture receives is become numerical data with the video camera one, and comprise following one or more devices:

A) be suitable at friction speed and any light, for example IR, visible light and at any lighting condition, for example light difference is carried out the colored device of taking by light amplification,

B) amplification or reduction means comprise and mould far away or microscope equipment;

Whether c) optical imagery 25 (Fig. 3 and 3A) comprises desirable resolution, is straight line, convexity or concave.

3. provide required utility appliance to these video cameras according to the instruction works that provides by computer image system.

A) regulate camera lens (at factors such as distance, light).

B) mobile camera (side direction, around, upwards, to inferior).

C) set (telescope, microscope, starlight amplifier etc.).

D) be computer image system and video camera service, and with they bring into any given time, can with suitably, wish and mode is moved computer picture accurately state.Transducer

Transducer A/59 and B/59 (Fig. 1) can constitute the integral part of video camera, and/or go/become to go from the computer image system two-way changing/from operating, measure, connect the device to any external unit, and/or any servicing unit, it can be served all parts of system or only serve a part of system, or from top any one to computer image system.The processing of picture input

1. any picture (comprising colour picture) that receives from any video camera will enter input store A/51 and B/51 (Fig. 1) to being its appointed positions according to its former state, and will occupy the space that is directly proportional with resolution of video camera.The picture of all video cameras will have all colours and enter into input store at disconnected position with the speed of shooting/shooting, image simulation memory device (for example ＂ screen card ＂) for example, in addition, the picture of guiding video camera A after through color filter 53 (Fig. 1) for example with entry time Motion Recognition register 54 (Fig. 1) at interval.

2. picture is sent to its position in space memories A/52 (Fig. 1), be sent to a video camera or two video cameras of being used for the 3-D shooting, space memories A/52 deposits constant, all pictures and/or other picture that amplify/dwindle of reception, if this computer image system constitutes user's a part, motion 6 (Fig. 2) according to the user, and/or according to the motion of video camera on level and Z-axis, these pictures will be preserved and will be brought in constant renewal in as the profile picture in level and vertical coordinate.According to the calculating from action, motion, level and the vertical coordinate of every width of cloth picture of each and all video cameras is come frame update.Also can upgrade them according to the motion of space counter 57 (Fig. 1).

3. the picture that receives from video camera according to the velocity sweeping of making a video recording/making a flim by an A/55 processor (Fig. 1), during scan process, for unrecognized or each shape in the picture that move/advances still still, calculate the distance of itself and video camera, and any other data that can calculate and/or can obtain from this picture, for example color, in addition, these pictures are sent to appropriate location and any other place of space storage register, for example the user place.Scanning and matching process

1. according to the speed of making a video recording/making a flim and/or other situation scanning picture data, suppose to begin from left to right and from top to bottom to right video camera, hereinafter referred to as the guiding video camera, picture in pixel (0:0) scan, then to the pixel in the left camera views (0:0) scanning, the point (Y:X) in arriving each camera views.3-D manifests if desired, be not only the guiding video camera but the scan-data of each video camera in two video cameras all will be respectively and picture coupling by the space storage, there not being when coupling,, their are upgraded and will find between the picture that obtains mate according to coordinate and space counter.

2. for range observation and/or profile identification, the point in the picture that has changed and these pictures must mate, and except matches criteria, they also will mate mutually.

3. scan for coupling, and calculate horizontal line X in the picture of two video cameras and the distance L (Fig. 4 and 4A) of the some D among the row Y, X1 represents to come the quantity of the pixel of the point (Y:0) in the bootstrap video camera, the quantity of the pixel of same point X2 represents from point (Y:0) to another video camera, then, obtain difference result: Wr=X2-X1 from following formula, so that can computed range.

4. be illustrated in the pixel of a point of seeing in right and the left video camera, top point be positioned at overlap apart from outside (pixel is represented 2M and 0=Wr (1/2), can not detect the variation of distance from this distance and farther distance.Make the same point from two pictures that video camera received be positioned at same position (Y:X), and represent that their pixel is identical.

5. can draw from this coincidence distance, distance is short more, and the difference between the pixel Wr increases big more.In other words, shape will increase difference between the pixel Wr to the path of shape nearby from afar, and will reduce the difference the pixel Wr from shape nearby to the path of distant place shape.Therefore, exist near during the guiding video camera picture neutralization away from the time another video camera picture in absent variable point, but will pay attention to when mating the computer-chronograph picture system.Work as consecutive hours, the point of these depth directions also forms the cut-off rule between outline line and/or the zones of different.

6. in first aiming, except the motion detection register, image computer will stop additional screens intermittently to enter, and will transmit the coupling that scans with the first aiming picture to pixel.

Computer image system will make the same pixel matching in the picture of a pixel (0:0) and another video camera in the picture of guiding video camera; If coupling, difference then is 0=Wr, and system will forward next pixel in two width of cloth pictures that must be transmitted and mate or the like to, up to finding unmatched pixel.Exist under the unmatched situation, difference 0 ≠ Wr, computer image system is incited somebody to action the same pixel matching in pixel that make the picture that guides video camera at first one by one and the picture that has unmatched second video camera, up to finding coupling, and correspondingly continue to transmit and coupling, or be 0=Wr up to difference, do not match if find, computing machine then turns back to the same pixel in the picture that has unmatched guiding video camera, and one by one with the pixel and the same pixel matching of the picture of second video camera, till coupling or the like, use boundary according to the area limiting of interval motion detection register detection.

In two pictures of same distance in delegation continuous performance pixel will be in order coupling mutually, if two pixels do not match, should suppose that the distance of a point has changed, and will therefore mate computing.

8. in the matches criteria screening, when not matching between the picture that store in input picture and space, computing machine will begin to mate between the picture that unmatched same position received from every width of cloth picture, and detect according to the information of front and whether to have any form to move, changed or new, and calculate the distance of each point in each coupling, up to discovery coupling between the picture of picture that is received and space storage, and coupling is proceeded down as usual.Distance calculation is handled

1. handle for compute and picture, can distribute counting (Fig. 1) (index lines) to coordinate 57, for example from (0,0) to (Y, X) or from (Y ,-X) to (Y, X), it is a kind of coordinate grid of picture of the storage of arrangement space in the above.For example, for horizontal positioned and direction according to compass heading ＂ north ＂ video camera northwards, it is that an Op (Fig. 3 and 3A) (mid point of picture) is read at the center of each video camera on the surface level that ＂ 0 ＂ names a person for a particular job.

When the transverse movement of video camera eastwards or upwards for just, and westwards or downwards when negative, according to the automatic update calculation machine of motion in the picture that causes by external factor or by a compass or backup system the size of pixel angle (for example by), and receive the corresponding data of video camera.

The peripheral level point of coordinate representation, the peripheral vertical point of another coordinate representation.

2. in the zone, to locate, for example three dimension counters 57 (Fig. 1) will be arranged, the dimension counter will with the central vision relevant (mid point of video camera) of guiding video camera, for example: counter (1) advances from the motion of the video camera of the western ＂ of east orientation to the main ＂ 0 ＂ point ＂ that will set and counts; Counter (2) is counted advancing from the motion of the video camera of north orientation south ＂ from the main ＂ 0 ＂ point ＂ that will set; Counter (3) is to rising and (highly) ＂ counting that descends for for example sea level or any other plane of being set up from ＂ 0 ＂ point ＂.

3. for each picture that receives in the input store, calculate for making the counter coordinate coupling relevant, and resulting picture is sent to the picture of space storage according to this calculating and relevant coordinate with first that in picture, discerns.

4. under the situation of using video camera (CCD) or any similar video camera, himself rotation on level and vertical plane of camera intrinsic is so that can draw peripheral picture (taking by standard) by coordinate according to the pixel quantity in the camera views.Therefore, with representing that its center is that the coordinate of the angle in the spheroid enveloping surface on angle top of video camera camera angle is drawn this space.When finishing a whole circle rotation or when level or Z-axis arrive the end in the space of photographing/watch, computer picture will be discerned, the quantity of pixel/coordinate for example, and ＂ understood its coordinate must return the beginning in photography in this process or the motion/watch space time of ＂.

5. during amplifying, for example, when video camera shifts near when reducing the angle of shooting picture, the corresponding coordinate of adding will be a decimal fraction, for example behind the radix point three.

6. the pixel quantity that is taken by every kind of shape in the space will depend on the physical size of quantity, the shape of pixel in the optical imagery, the distance and the visual field, visual angle of itself and video camera.Pixel quantity is high more, and the visual field, visual angle of optical imagery is more little, with the form (constant with the distance of video camera) of the more same size of pixel performance occurring in nature, can improve the discriminating to shape.In addition, enlargement factor is big more, and the quantity of pixel that shows same size shape is big more.Computing method

The distance L of calculating from video camera to this match point, this calculating are according to difference Wr, carry out according to the resolution of homogeneity, physical size and optical imagery, coupling and constant distance M (parallel deviation) between the video camera.

1. calculate from the angle part of visual angle α radius r up to optical imagery ₀Formula as follows, its width (on X-axis) is X ₀And amplification/minification A:

r ₀=360 ^*A ^*X ₀/ 2 ^*π ^*α (360 is angles of a whole circle); K ₀Be all constants in the formula, thereby K is arranged ₀=360 ^*X ₀/ 2 ^*π.

2. the computing formula of the radius r from visual angle α to any point of being seen is: r=360 ^*A ^*X ^*M/2 ^*π ^*Wr ^*α, wherein X is the quantity of horizontal pixel in the optical imagery (on the X-axis), if K is all constants in the formula, then has: K=360 ^*X ^*M ^*/ 2 ^*π.

3.K ₁Be any proportionality constant of being directly proportional with amplification/minification and be inversely proportional to that its formula is K with visual angle α ₁=A/ α.

4. the formula of computed range L is: L=K ₁(K/Wr-K ₀) (coefficient of variable Wr).

5. calculate any indirect distance L ₁, Q wherein ₁Be by resembling the size of the such standard size of known human body height in the pixel performance of the position of any form.

Computing formula is:

L ₁＝360 ^*A ^*Q ₁ ^*X/2 ^*π ^*α-r ₀＝K ₁(K ^*Q ₁-K ₀)

K2 is whole fixed datas: K2=360 in the formula ^*X/2 ^*π

So obtain: the size that can come calculating pixel to show at the point that pixel is represented to calculating to the distance of these points, help/permission detect the framework of these affiliated areas and such as width, highly, the degree of depth, size, angle, feature and resemble any calculating required data of state (swing, suspension etc.) definite.Area limit and data collection process

Cognizance code of each appointment (interim name) to zone 11 and 12 (Fig. 4 A).Zone can be in other zone of the different dimensions of performance in himself scope.Each zone must always have a profile (except that a point and line), such as this regional fixed range, with environmental differences and color distortion or outline line and/or should the zone and boundary, regional movement and the zone state (swing, suspension etc.) of peripheral region.Discern and define these and other data by computer image system by B/55 processor (Fig. 1).

1. color separation 58 (Fig. 1) software 56 (Fig. 1) handle the color that receives picture and separate these colors, use other zone definitions auxiliary area definition formerly, and detect in this zone the additional definitions of this color-ratio in all colours at the mass-tone in a zone.For example can be divided into 4 kinds of forms:

A. colored amount is: the 100%-90% of color in the zone;

B. colored amount is: the 90%-70% of color in the zone;

C. colored amount is: the 70%-50% of color in the zone;

D. colored amount is: in the zone 50% of color and below.

In addition, I also want the mode that Accent colour looses (spaced point, determine the point of size etc.).

2. except that normal reception, when receiving, picture to the guiding video camera carries out colour filter 53 (Fig. 1) (resource preservation), and be kept in the time interval motion detection register 54 (Fig. 1), duplicate and check in the interval in required time in the mode (can be several) that fixing amplification/minification and desirable and known amplification/minification are taken.According to formerly receiving the data that picture gathers (color for example, tone, calculate etc.), can be zone/form 11 and 12 (Fig. 4) detects and/or calculates such as motion, move, speed, angle (the angle relevant with observed object, and the angle of the observed object relevant) with other object in video camera and the space, and mobile (for example eyes move) in the zone, motion in regional envelope (position for example, the action of hand etc.) and so on data, so that whether the check basis is stable, whether top moves (for example tree), regional movement whether flow (for example river), direction (north, south, east, the west, upwards, downwards, forward, backward, right and left), speed, and the situation of motion and/or mobile type and form, and, and help the framework in the zone that definition detects not yet according to the space counter position of detection computations machine image and zone and shape also.

3. basic and essential basic configuration (for example geometric configuration) register C/52 (Fig. 1), basic configuration by have in basic configuration everywhere line and/or the black/white outline line of several points, with can with computer image system, the simple form of the shape of its task and purpose compatibility constitutes.By allow immediate access really definite sequence preserve these basic configurations, so that the shape with basic configuration and input after the shape of input being carried out suitable framework processing compares, dimensionally with they couplings, so that obtain (approx) comparing data (a kind of shape can be mated with several basic configurations) about the processing shape.For the definition of setting regions profile, for example rectangle, inclination, circular arch, wedge angle etc., basic configuration (being no more than 256 in principle) storer will depend on the quantity of preserving shape B/52 (Fig. 1) (can have hundreds and thousands of).

4. the table 58 (Fig. 1) such as the true ＂ table of ＂ can cooperate with described computer picture, table after another table/inside, be used for zone/morphological data included and/or that obtain from a table among them, these tables use described data by particular order, partly with current table, next table etc. are relevant, are used to examine characteristic survey definition and deduction.Collected all or any characteristic, definition and infer to be added to and/or to be connected to and be used to the key element data that detect, mate and discern.Can set bells and whistles detects, defines and reaches a conclusion, for example: abiotic, movable, float, live, for example: if should the zone between an intended size and another size, if object moves and motion is arranged in this scope and/or object inside and/or move etc., definite object of each list processing (LISP), for example handle possible motion and/or move, handles possible color, processing is from the variable in distance of surrounding environment etc.

5. be used to be reproduced in that shooting distance finds, be adjusted to the storage shape and to occur in picture be the very rational tables of data 58 of form.In these cases, just in time when receiving picture and driven dimension and the additional independent data resemble the color, computer image system detects their reproduction, and these data are configured for and the register pair of storing shape shines into the key element of capable detection and Identification.

6. if require, need and possibility, also the recognition data such as heat, radiation, sound, taste and smell 57 (Fig. 1) can be added to existing recognition data.

7. can be according to the computer picture requirement, desirable ratio and computerese (numeral) be factor (for example space counter, compass etc.), external factor (for example video camera, from amplification/minification of user, translational speed etc.), printer, velograph and/or any backup system internally, and for example compass, telescope, stadimeter (claiming that hereinafter these are servicing unit) and other related data obtain auxiliary data 57 (Fig. 1).

Near any form that can be defined and/or provide title and/or separately identification (for example abiotic, plant, biology, background, symbol, phenomenon and any other thing)/on picture, map, symbol and/or text data storage and recognize that form B/52 (Fig. 1) by with specific and known sequential storage, mates with key element in one of following four kinds of forms:

A. as by comprising what colored video camera received.

B. black/white picture.

C. reduced form-(synoptic diagram)

D. the form that has the table of title and data, or the written form of card index for example.Picture and map are the shooting standard that depends on geomery and shooting distance.Can with the shape store of visual format storage in several places, and in each place never the visual angle of ipsilateral preserve picture so that can be from each direction identification.For example, for a 3-D picture, can store 6 different pictures, one in each Descartes's direction.The identification of zone in the picture and with the process of the form fit of storage

1. because the shape of being stored can not be represented each possible size of this shape with different distance in the space, produce size match between the suitable shooting distance of the shooting distance of the true shape that the processing 56 of a ＂ logarithm ＂ system and/or any other form and B/52 (Fig. 1) will be in the space receive in the picture of given time and the shape of storage.

2. a zone can comprise an additional areas in its own inside.Coupling and identification will begin from the zone of maximum and carry out to less zone.If zone identification is a part of data that are identified shape this zone in, more generally say, if they be finish sign (for example position) that they will be identified requisite, regional identification can be preserved the necessary data of identification of interior zone.

3. form by data, characteristic, definition and deduction, the key element of arranging by the particular order of same sequence in the database that is adjusted to the shape of known as memory 58 (Fig. 1) allows the key element of having collected is carried out Fast Classification and arrangement, and allows unidentified zone is detected, mated and discerns.At normal visual field, because in the speed of any photography/make a flim and unrecognized regional considerably less, therefore, identification is almost carried out immediately, and as look into word inspection in dictionary, classification is as language, and the order of key element is as alphabet, whole word is as a key word, and an ingredient may be that determine, marginal or possible.Contrast with the register of distinguishing the shape of storing and to discern, this register for example comprises the data of relevant shape with the form that is kept at picture, map, symbol and/or text data in the register (with table, nearby, superincumbent form), when picture is relevant with map, also can discern by size and photography/shooting angle according to the storage shape photography standard of the purpose of the physical size that depends on shape, photo distance and computer picture.

Object subordinate zone, partly, the shape that needs so further to analyze such as subordinate details, age, type, position handles, and more identification and detailed data are provided, and comprises any data that requested image provides for the user of its service.Symbol detection and identification

A kind of software that is used for the significant character of detection and Identification picture.For example:

1. software program detects in picture symbol (for example mark, literal, street symbol, traffic lights etc.), mates and discerns.Software is understood the place (for example position of vehicle registration number) that has literal or certain symbol in the shape for example according to recognition data, and for mate, discern and with size with literal of being stored or symbol coupling and understand this place of visit.

2. in order to help testing process, provide catalogue or dictionary [symbol, literal (literal of local language or English etc., title), form, street symbol, traffic lights etc.].Parts of images

1. parts of images appears when a part of shape is covered, and must be according to the whole shape of appreciable part detection and Identification.

2. under these situations, computer picture detects those parts that occur in the picture, and all details and its visible part relevant with this concrete shape with respect to those part indications.Unidentified shape

During coupling and cognitive phase, if there is not the shape of the form fit in discovery and the picture.

1. this shape is kept in the register in (zone, partly, details etc.), and preserves its all related datas in given place.

2. computer image system is counted the number of times that it runs into shape.If no longer run into this shape in the cycle at a reasonable time, system then will should be used as other analogue in the place in the register.

Computer image system can comprise to or from external factor, for example facsimile recorder interface, monitor and printer 31 and 32 (Fig. 2), audio system, compunication etc. send and receive message, and with its identification not yet according to the shape of system storage identification.Cartographic analysis

The computer image system that can be in buildings, city, field, rural area the device service of moving can be analyzed picture or the process of being seen, and the same place in the map of exactly these data and system register being stored compares.

1. computer image system will be understood its initial place map according to the space counter or from an external factor, or find its position on map by recognition data from real picture.For example, this system will read a street title and with the title in its another street that intersects and this street on two numbers of a house in the same side, and by suitable plan, comprise that its city of finding discerns the place.

2. map will provide relevant road and along the data of road both sides:

The data of the pavement markers of a. relevant obstacle, crossing, railway, intersection, intersection, ramp, traffic sign, both sides, road outside etc.These data appear at above drafting figure, the map etc., and wherein some data is relevant with road, and other data then are different modes.

B. can be to represent the shape or the literal of tables of data (for example literal, code, symbol, traffic sign etc.), the form registration data of code.Rotate

Suppose that computer image system has been seen because the result that the shape of being seen moves, and/or because video camera makes progress, downwards and the result of lateral tilt, and/or the user is around the result of any rotation, and/or the user that video camera is attached thereto moves the space that changes and change.

The picture of space storage will cover the whole angular range of circumference and the visual field of desirable described computer image system.When video camera receives a width of cloth picture at any angle, system detects the variation in the coupling between the picture of being stored in last picture and the space memories by suitable coordinate, and different with it part in the part replacement final picture in the picture of preserving with space memories.Preserve and manifest the space with 3-D and multimedia

The computer picture system will keep from each the complete space stored picture of computing machine.

1. these pictures will play the effect of whole circumference motion and preserve every width of cloth picture respectively, produce a composite space picture around each video camera.

2. the space picture will allow computer image system to send to any other factors by 3-D display routine (needn't constitute the part of computer image system), show or present the data in the space that it moves or finds, comprise showing identification details and data.

3. can see the space that centers on computer image system with the 3-D form, as in actual life by connecting manifesting of external display or any other type.

4. can pass through an interface 33 (Fig. 2), send data by the physics transmission or by transmission mode to the display of screen or any other type, this does not constitute a computer image system part, but user or additional part to system or user.Zone/shape is followed the tracks of

1. in order to follow the tracks of, with reference to first that scans in zone/shape in the picture of guiding video camera and envelope and/or they with the coordinate of the annex point in the shape that is instructed to unrecognized area stores under one's name temporarily, and the form of having discerned is stored in himself under one's name so that at any time the position of tracing area/shape, orientation and/or move and/or motion and/or scope in a shape part and/or the variation in its inside (for example animal, plant etc.).

2. under the situation of needs, preserve from picture, calculating, by the Motion Recognition in the time interval with from the morphological data storehouse of preserving and all data of collecting from any recognition device of relevant shape, leave the picture of the space storage of video camera up to this shape.Be kept in the register

If necessary, the register of use after computer image system is sent to the picture of storage (whole, sampling, part etc.) or shape (picture, figure, symbol, code etc.).Computer image system must store or stored picture, so that they are recovered when needing later on, this process is carried out by following mode:

1. continuous picture and/or the data of complete sum from a particular moment to another picture constantly (for example are sent to suitable register to computer image system definition, store the particular event that someone want before incident/sees after the event procedure neutralization, so that understanding particular procedure or data etc.).

2. frame update or map and replace picture and map in the register with new and the picture that upgrades or map.

3. transmit and anything and/or relevant process and/or the data of Any shape seen with any method.The outside connection

1. computer image system has a coupling arrangement 50 (Fig. 1), and this coupling arrangement can connect a computing machine, robot, user, device 20 (Fig. 1), printer, facsimile recorder, controlled machine, microphone, loudspeaker, an order wire etc.

Certainly, each computer image system can only have required coupling arrangement (can have an I/O and be used to disconnect or tuning first code) as required for transmitted image data, exchanges data, direction reception, notice etc.

2. between coordinator A/59 and B/59 (Fig. 1) Coordination calculation machine picture system and video camera, the user etc. and any one being connected from them to computer image system.Transmit

Will serve after according to target and customer requirements identification details and its data, this system is sent to user that require with data needs by suitable transmission or interface arrangement.

1. computer image system calculates canned data in its storer according to the picture that receives, handle, decode additional data and various processing data, offer the user that it is served, and submit all direct and computerized data to about each object (for example size, distance, position) to the user.Continuously or report is provided when requiring.

2. computer image system can be discerned the integral body and the details of picture, and the position of each details in the picture, and can be on request transmitting picture with form such as notice, literal, language, fax or as the integral body of the detail analysis that has different pictures, all depend on the needs.

3. receive the user who reports from system and should prepare to receive and handle the data that received, so that can work.The image if the user uses a computer in order to carry out specific operation, he can utilize the accurate motion of this system and direction etc.

4. can be as required in statu quo be transferred to Any user to the picture of space storage, can be directly or from space memories to supervision screen or an appropriate device, comprise the purpose that manifests for 3-D and/or transmit with recognition data.Sum up

1. should suitably make computer image system prevent dazzling light, the flash of light of any kind of as much as possible, in any case and also comprise laser, and prevent any other possible physical damnification in any place.

2. computer image system is compatible, and will and require work according to user's needs.

3. computer image system comprises that system is by its software program that works 56 (Fig. 1), electronics and general circuit, shooting angle according to dimensional standard and storage shape is regulated size and the software program of data is amplified/dwindles in photography/shooting, be used for 3-D and manifest and multimedia software, and in requisition for any other software program.

The design and by any method, the calculating of drawing a picture in the picture that receives, data, the space memories is sent to Any user as requested same as before with any out of Memory relevant with the reception picture stored or data or map in any one system storage, robot for example, comprising data (data are only arranged) and/or input or space multistory picture, the user can extract data according to its requirement with according to system design.

5. above-mentioned all operations will can adopt Attached Processor if desired by A/55 and B/55 (Fig. 1) processor, and processing and calculation element execution.Their will use a computer device, parts, one or more electronic circuit and similar circuit, and any combination required and that be fit to the computer picture purpose.

Work allocation among them is such: each parts, partly, electronic circuit, system etc. carry out its task separately under the situation of non-interference, and in they middle chief coordinator and compatibilities of carrying out, for example, No. 1 processor is by parts, No. 2 processors of electronic circuit untill further notice, it has finished its that part in operational zone A, No. 2 processors can continue to carry out that part of No. 2 processors oneself etc. in composition operation district A.

Industrial applicibility

1. computer picture can be used for:

A. as a viewer (monitor, analyser, demoder, indicator etc.).

B. as one by any way (all, partly, the form by any classification or classification etc.) collect data and store the data in viewer in any database or the register.

2. because the multiple possibility that computer picture uses, can adopt several standard computer picture systems, wherein adopt every kind of modular system, so that specific service is provided and carries out particular task, and may be used on (on the shelter) ＂ on the ＂ blindage, and, may be used on ＂ on the ＂ blindage for special duty, and for specific (special) requirements, design is suitable for the computer image system of special requirement.

3. be each computer image system specified value and certain capabilities.

4. in the design phase, can make computer image system adapt to special needs and requirement, and can make its part as whole user as deviser or fabricator's Any user, or as a separative element of its service-user.

Claims

1. computer stereo vision system comprises:

A. a pair of fixed range M that is in, photography/shooting angle with coordination and amplification/minification and be identical, alignment, the video camera coordinated, at fixed range M, at any photo distance, from (0:0) to (Y:X) has the optical parallel visual field of coordinating, aliging for video camera produces the video camera in the identical sight line visual field at Liang Tai, receive the picture of described video camera by the input memory storage in the computing machine, convert computerese to;

B. on the basis of step a, by available device according to coordinate and space counter with the speed of making a flim and/or any other speed each the picture coupling formerly that exists in the storage registration of the space of the pixel of received picture and system, when not finding coupling, system upgrade and proceed to have imported the coupling of two pictures;

C. on the basis of step a and b, for the unrecognized zone of in two video cameras, seeing simultaneously or any mobile zone takes place, the various features of this system looks picture, color for example, and carry out distance calculation according to the difference in the pixel counts of video camera from each picture that delegation begins to gather to this match point, these distances that can be enough calculate are calculated the data the size of a point in the space of being represented by a pixel, and the size and the distance of zoning in view of the above;

D. on the basis of step b and c, in the match point of the picture of unidentified zone and storage, because the change of distance between the distance between this zone and the surrounding environment and two video cameras, exist unmatched point around in this zone, except to the approximately uniform distance of region point, the point of these depth directions produces an outline line and/or separatrix, for zone at similarity distance, the color of zone and/or regional movement/between moving or the change of tone, permission detects by available device, and is each zone definitions framework or profile;

E. can calculate by enough available devices in order to collect, coupling, the additional data that definition is set and inferred, the picture that receives from video camera is carried out colour filter and is input to the time interval moving identification register, duplicating with after this after the Fixed Time Interval check, and according to step b and d, the framework in the zone of having detected or profile are mated with the basic configuration register dimensionally, from the described data of these registers in time calculated example such as zone/shape size and move/movement velocity and additional data, the described data permission of the part of described data of the part of having imported and calculating/detection in advance contrasts with the register that for example is used for respect to " very " of other key element setting definition and deduction shown, these key elements comprise, lifeless object for example, swing, biological, for example, if the size in zone from a kind of size become another kind of size and envelope and/or its inside have and move and/or motion etc.;

F. obtain described key element from data, feature, definition and deduction, arrange by particular order, the register that allows shape is provided by the device contrast that provides detects, mates and discern these zones;

G. Unidentified area stores under one's name temporarily, the area stores of having discerned its oneself under one's name, according to the type in this shape/zone with reference to the coordinate data in each scanning of first and annex point, so that follow the tracks of and observe place in envelope and/or the shape inside always, position and/or motion and/or move in any variation.

2. computer image system according to claim 1, wherein said a pair of and/or a plurality of video camera can be contained in the unique box or separately assembling, this video camera is identical with video camera, comprise ccd video camera and comprise that wherein combination is useful on the data-switching that the picture of being gathered by this video camera is received and becomes the video camera of the device of numerical data, and comprise following one or more devices:

A) be suitable at friction speed and any light, for example IR, visible light and at any lighting condition, for example insufficient light carries out the colored device of taking by light amplification,

B) amplification or reduction means comprise and looking in the distance or microscope equipment;

Whether c) optical imagery comprises desirable resolution, is straight line, convexity or concave.

3. according to claim 1 and 2 described computer image systems, the described main device that wherein is used to collect the data that detected and/or calculate, feature definition, that draw and/or observe allows directly to detect described data by described device, according to classification according to the key element of one or more system sequence arrangements of setting up in advance, detection case can be: determine, fall between and possible, main device or inner or outside, they comprise following one or more devices:

A. be used to receive shooting/input store of observed picture;

B. be used for a video camera or two video cameras, the space memories of 3-D performance, space memories is collected by amplifying/dwindling, all fixing and/or other style of shooting receives pictures save as the profile picture represented to these pictures and they are brought in constant renewal in level and vertical peripheral coordinate;

C. registration form that is the known as memory shape of form, comprise the relevant title that can be defined and/or provide, and/or separately picture, map, symbol and/or the textual data of (for example object, plant, animal, background, symbol, phenomenon or anything) each shape of identification, store, be adjusted to key element by specific and known sequence, these pictures and map are in the shooting standard that depends on geomery and shooting distance;

D. a register is used for a picture that carries out obtaining after the colour filter is moved detection discontinuously, and in order to detect and to calculate such as moving, the content of motion and speed and so on and testing at set intervals;

E. the basic configuration register is preferably stored in the black/white mode and is resembled for example known like this geometric configuration such as rectangle, ladder;

F. one comprises the down database of column register: 1. register that is used for color separation; 2. one is used to store these zones and the register of the profile in the zones of identification in the past; One such as " very " the register of table the table, these shows one by one/one internally to arrange at another, are used for setting at data, definition and conclusion defining and reaching a conclusion; 4. register that is used to reproduce the tables of data of shape; 5. register that is used for key element;

G. auxiliary data register: these auxiliary datas comprise: 1. resemble from the space counter, from such internal datas such as compass; 2. resemble from video camera, for example amplification/reduction means, from the user, such external data such as translational speed for example; 3. additional ancillary data.

4. according to the described computer image system of claim 1 to 3, wherein said data be can be defined by can from picture, from calculating, detection, feature, from definition and infer, from the space counter, from resembling any situation of the data that printer, velograph and/or the system supplymentary device such as compass, telescope, stadimeter (servicing unit hereinafter referred to as) receive, these data are according to the speed of computer picture requirement and hope and computerese (numeral), with following any one, or its combination definition, following these comprise:

A. size, (distance for example, width, highly, the degree of depth, size, ratio and angle), color (color number percent grade for example, chromatic dispersion, spaced point, hot spot), framework (profile), motion, move, speed, (for example northern with direction, south, east, the west, upwards, downwards, forward, backward, right, left) and coordinate system, or regional or regional shape or the regional a part of shape of video camera, or the relevant position of shape and angle are (for example with respect to the zone, shape, video camera);

B. feature, phenomenon, attribute, heat detection, radiation, speech, sense of smell and the sense of taste;

C. define and infer (the flowing of for example interim name, rectangle, ladder, object, automobile, swing, flame, water, biology, four leg walkings, growth, special-purpose vehicles front, time variation).

5. computer image system according to claim 4, comprising the zone that is used for respect to Unidentified shape, feature, the identification of the described data of definition and inference and the device of calculating, and/or certain motion betides in whole zone/shape in unidentified shape, and/or in the envelope, and/or its inside, and this shape sees in the picture or the part of picture in, and theirs is whole, their part, their combination combines with the key element data, and undertaken: system operation and according to the size of storage shape by following array apparatus, the size of distance and shooting angle shape that coupling is seen, distance, amplify/dwindle software program with shooting angle, electronic circuit and rule and be used for transmitting, 3-D manifests the software program with the multimedia form.

6. according to claim 4 and 5 described computer image systems, data wherein, feature, definition and infer will with have all data or its a part of register/memory coupling, compare and mate with described device and software program, wherein discern according to the classification of one or more predetermined system orders by key element, the identification situation can be: determine, or fall between, or it is possible, described key element allows with respect to known or from the information of existing photo, with respect to known region in advance, the described data of shape or its part are come identified region, shape or its part.

7. according to the described computer image system of claim 1 to 6, wherein one or more in computing machine, machine element, parts, electronic circuit, a processor, a computerized data handling system and this type of device, the described device of one or more formations in them, or their any combination; Described device is known, new and software program compatibility; Described device is used for the data-switching that obtains from the picture of taking is become numerical data; Described device is used for some coupling and the colored identification of definition that the same relatively point of seeing and/or shooting picture produce; Described device is used for computed range and size, is used for recognition feature, definition and deduction; Described device is used for matching size and coupling shooting angle; Described device is used for that the compartment of terrain detect to be moved, mobile type, motion; Described device is used for handling " very " table; Described device is handled and is reproduced shape; Described device is used for the crucial data aggregation of forming; Described device is used for comparison and coupling; Described device is used to store different kinds of information; Described device is used for receiving, sends and draw data; Described device comprises the protection of power supply, data protection, described device, system and similar device.

8. according to any one described computer image system in the claim 1 to 7, wherein before identification, during this time and reception of having collected afterwards and canned data, comprise such as with respect to the zone, the calculating that shape and so on has been collected, given data, feature, definition by immediately or blink after by normal mode with message form, and/or send forward by three-dimensional mode and/or 3-D form with the multimedia form, this system preserves by suitable interface arrangement and/or provides and/or can and/or automatically use such as robot when request, device, blind person's and so on user.

9. computer stereo vision method comprises:

D. on the basis of step b and c, in the match point of the picture of unidentified zone and storage, because the change of distance between the distance between this zone and the surrounding environment and two video cameras, exist unmatched point around in this zone, except to the approximately uniform distance of region point, the point of these depth directions produces an outline line and/or separatrix, for in the color of zone, zone and/or the regional movement of similarity distance/between moving or the change of tone, permission detects by available device, and is each zone definitions framework or profile;

E. can calculate by enough available devices in order to collect, coupling, the additional data that definition is set and inferred, the picture that receives from video camera is carried out colour filter and is input to the time interval moving identification register, duplicating with after this after the Fixed Time Interval check, and according to step b and d, the framework in the zone of having detected or profile are mated with the basic configuration register dimensionally, from the described data of these registers in time calculated example such as zone/shape size and move/movement velocity and additional data, the described data of the part of described data of the part of having imported and calculating/detection in advance allow and for example are used for setting definition and the register of the true ＂ table of ＂ of inferring contrasts with respect to other key element, these key elements comprise, lifeless object for example, swing, biological, for example, if the size in zone from a kind of size become another kind of size and envelope and/or its inside have and move and/or motion etc.;

F. obtain described key element from data, feature, definition and deduction, arrange, allow to detect, mate and discern these zones by the register of the device contrast shape of storing that provides by particular order;

G. Unidentified area stores under one's name temporarily, the area stores of having discerned its oneself under one's name, according to the type in this shape/zone with reference to the coordinate data in each scanning of first and annex point so as to follow the tracks of always and observe place, position and/or motion in envelope and/or the shape inside and/or move in any variation.

10. computer picture method according to claim 9, wherein said a pair of and/or a plurality of video camera can be contained in the unique box or separately assembling, this video camera is identical with video camera, comprise ccd video camera and comprise that wherein combination is useful on the data-switching that the picture of being gathered by this video camera is received and becomes the video camera of the device of numerical data, and comprise following one or more devices:

11. according to claim 9 and 10 described computer picture methods, wherein be used to collect the data that detected and/or calculate, definition, the described main device of the feature that draws and/or observe allows directly to detect described data by described device, according to classification according to the key element of one or more system sequence arrangements of setting up in advance, detection case can be: determine, fall between, with possible, main device or inner or outside, they comprise following one or more devices:

A. be used to receive shooting/input store of observed picture;

F. one comprises the down database of column register: 1. register that is used for color separation; 2. one is used to store these zones and the register of the profile in the zones of identification in the past; 3. the register of a table such as the true ＂ of ＂ table, these shows one by one/one internally to arrange at another, are used for setting at data, definition and conclusion defining and reaching a conclusion; 4. register that is used to reproduce the tables of data of shape; 5. register that is used for key element;

12. according to the described computer picture method of claim 9 to 10, wherein said data be can be defined by can from picture, from calculating, detection, feature, from definition and infer, from the space counter, from resembling any situation of the data that printer, velograph and/or the system supplymentary device such as compass, telescope, stadimeter (servicing unit hereinafter referred to as) receive, these data are according to the speed of computer picture requirement and hope and computerese (numeral), with following any one, or its combination definition, following these comprise:

13. computer picture method according to claim 12, comprising the zone that is used for respect to Unidentified shape, feature, the identification of the described data of definition and inference and the device of calculating, and/or certain motion betides in whole zone/shape in unidentified shape, and/or in the envelope, and/or its inside, and this shape sees in the picture or the part of picture in, and theirs is whole, their part, their combination combines with the key element data and is undertaken by following array apparatus: system operation and according to the size of storage shape, the size of distance and shooting angle shape that coupling is seen, distance, amplify/dwindle software program with shooting angle, electronic circuit and rule and be used for transmitting, 3-D manifests the software program with the multimedia form.

14. according to claim 12 and 13 described computer picture methods, data wherein, feature, definition and infer will with have all data or its a part of register/memory coupling, compare and mate with described device and software program, wherein discern according to the classification of one or more predetermined system orders by key element, the identification situation can be: determine, or fall between, or it is possible, described key element allows with respect to known or from the information of existing photo, with respect to known region in advance, the described data of shape or its part are come identified region, shape or its part.

15. according to the described computer picture method of claim 9 to 14, wherein one or more in computing machine, machine element, parts, electronic circuit, a processor, a computerized data handling system and this type of device, the described device of one or more formations in them, or their any combination; Described device is known, new and software program compatibility; Described device is used for the data-switching that obtains from the picture of taking is become numerical data; Described device is used for some coupling and the colored identification of definition that the same relatively point of seeing and/or shooting picture produce; Described device is used for computed range and size, is used for recognition feature, definition and deduction; Described device is used for matching size and coupling shooting angle; Described device is used for that the compartment of terrain detect to be moved, mobile type, motion; Described device is used for handling " very " table; Described device is handled and is reproduced shape; Described device is used for the crucial data aggregation of forming; Described device is used for comparison and coupling; Described device is used to store different kinds of information; Described device is used for receiving, sends and draw data; Described device comprises the protection of power supply, data protection, described device, system and similar device.

16. according to any one described computer picture method in the claim 9 to 15, wherein before identification, during this time and reception of having collected afterwards and canned data, comprise such as with respect to the zone, the calculating that shape and so on has been collected, given data, feature, definition by immediately or blink after by normal mode with message form, and/or send forward by three-dimensional mode and/or 3-D form with the multimedia form, this system preserves by suitable interface arrangement and/or provides and/or can and/or automatically use such as robot when request, device, blind person's and so on user.

17. computer image system and method, this consist essentially of as described, reference, that explain, explanation, that provide or hint, at length and in the above in the claim, or any improvement in the accompanying drawing or its any combination.