ES2209655B1 - SYSTEM FOR MEASURING DISTANCES AND SPEEDS BY NON-RADIANT ELECTROOPTIC PROCEDURES. - Google Patents

Abstract

System for measuring distances and speeds by non-radiating electro-optical procedures. The purpose of these systems is to measure distances to moving objects and their travel speeds. They are composed of two cameras mounted on a rigid base and electronic subsets that analyze stereo images in real time, applying multiresolution techniques to accelerate the search for stereo correspondences. They can use high resolution image sensors to measure with high accuracy. Its functional principles are applicable in wide ranges of distances and speeds. Its reconfigurable electronic structure, implemented in FPGAs, provides very low power consumption systems and allows the resulting data to be used in external processors or easily integrated into low-cost systems, depending on the accuracy, use and presentation required of the data. Being non-radiant passive systems, they are undetectable at radio levels and usable in very diverse environments.

Description

System for measuring distances and speeds by non-radiating electro-optical procedures.

Technical sector

Artificial Vision Systems. CCD sensors and CMOS Programmable devices type FPGA, CPLD and DSP. Process automation by stereo image processing with multiple resolution Automatic distance meters and speeds.

State of the art

Speed is a parameter of interest in Numerous activities and applications. The different ranges of speeds, the precision with which the measurement is desired, the size of mobiles, the distance they are from the observation point and the environment in which they move, among other possible considerations, may require special features of the equipment capable of determining or measuring the speed of mobile phones in observation. That is why there are numerous systems and solutions, in many specific cases, to determine the speed at which move moving or moving objects.

The definition of speed as the ratio between the space traveled and the time interval used to doing so implies that the speed measurement is done in a way indirect, setting a time and measuring the space traveled in him or setting a space and timing the time in which travel. Electronic systems usually adopt the first alternative, because it is easier to establish the reference of time than that of space. The measure of the space traveled is does measuring distances at the beginning of a time interval and at the end of it, by various procedures among which they count the emission of a radio signal and the reception of the same, evaluating the delay of the reflected signal. The relationship of delay with propagation speed and wavelength of the signal, provide information on the difference of measured distances, that is, of the space traveled.

Some speed measurement systems, for example, the Doppler radars used by traffic police in many countries, they are based on procedures that make use of certain bands of the radioelectric space, to determine the distances from mobile phones to the measuring system in two instants of time very close that set the time interval of reference, thus obtaining what can be considered a speed snapshot. Although the procedure can give precise measures, the emitted radiation so that it can be considered permanent is detectable by certain receiving systems and receiving the reflected wave can be interfered, so that such systems they can be cause and object of radio interference, sometimes  caused from the same vehicle whose speed is intended to be measured and occasionally by other radiant systems.

Other systems with similar purpose use frequencies close to the visible spectrum, such as the laser of infrared, less susceptible to interference, but in exchange for demand greater directivity of the signal by its propagation optics. Directivity requires that these systems point to mobile object, so they do not have the capacity to automatic mobile detection with speeding, that is, they must be activated by an agent that points the radiation towards the alleged mobile offender, which is cause for rejection by groups that claim potential risks to living beings, by the thermal effects of infrared and laser.

Any of those systems requires a signal reflected, which worsens if the mobile is not metallic or if partially absorbs the emitted signal, and also requires a minimum size on mobile. Such conditions may limit the use and applications of such systems. They are, in fact, systems specific to measure the speed of vehicles and their principles They are not applicable in other measurement environments.

Such inconveniences and potential effects can be avoided by using systems based on the analysis of stereo images captured by CCD sensors - Charge Coupled Devices - or CMOS - Complementary Metal Oxide Semi-conductors - which, by electro-optical procedures, allow digitally captured images to be processed and obtained information present in dynamic scenes.

Electro-optical systems use the spectrum visible, taking advantage of the light reflected by moving objects or in rest and, consequently, are passive, or non-radiant, systems that neither cause radio interference nor emit signals that can be considered potentially harmful. Being systems liabilities are not detectable at radio level or susceptible if neutralized or deactivated by radio frequencies, except Design with that specific purpose. They are currently marketed CMOS sensors whose resolution or pitch between pixels is 4.5x4.5 microns, provided that these resolutions become less than 3x3 microns in 2004. Its characteristics Functional are similar to those of digital cameras used domestically, so they detect movement or presence of any visible object, whether metallic or not. its reduced power consumption and small size allows them to operate from continuous mode and in very variable environments.

Electro-optical systems require that their elements are sensitive enough to detect light reflected in the environment and an optical resolution related to object size, conditions that image sensors type CMOS or CCD available in the market comply sufficiently, so they can be used as detection elements in systems whose structure and operation allows the measurement of distances in Image sequences Being able to fine tune the interval between them, that is, set the rate of images per second, is determines the time interval that is applied when calculating the space / time ratio that gives the speed measurement.

The problem in these systems lies in the computational complexity of artificial vision algorithms that, in rapid sequences, can limit the ability to complete the operations necessary for calculating distances in what is called real time . On the other hand, the accuracy of these systems depends on several parameters, as justified later in this report, one of them being the resolution of the image sensors used.

Explanation of the invention.

The first object of this invention is to measure distances to specific points of a mobile in intervals of selectable time, determined by the rate of images in a stereo sequence and, secondly, analyzing the values of distances measured at these intervals, derive spaces routes and calculate speeds.

\hbox{2\textperthousand}

en aplicaciones de metrología óptica basada en visión artificial. Sin embargo, el aumento de resolución conlleva un aumento en el número de datos a procesar electrónicamente y un mayor tiempo en la respuesta de los algoritmos. Por ello, para poder obtener datos de forma automática y en tiempo real es necesario aplicar algoritmos a nivel de circuitería o hardware, con velocidades de procesamiento mucho mayores que la de aquellos basados en software, cuya respuesta es lenta frente a la tasa de la secuencia de imágenes estéreo, que debe ser relativamente alta para poder medir una amplia gama de velocidades y poder hacerlo con mayor precisión.The current high resolution CMOS and CCD sensors allow remote resolution with errors below

 \ hbox {2 \ textperthousand} 

in applications of optical metrology based on artificial vision. However, the increase in resolution entails an increase in the number of data to be processed electronically and more time in the response of the algorithms. Therefore, to obtain data automatically and in real time it is necessary to apply algorithms at the circuitry or hardware level , with processing speeds much higher than those based on software , whose response is slow compared to the sequence rate of stereo images, which must be relatively high to be able to measure a wide range of speeds and be able to do it with greater precision.

The solution offered by this invention is to divide the operation of automatic search of stereo correspondences of those specific points that are considered of interest, to obtain the so-called stereo disparities from which the distance data are derived, applying for this purpose artificial vision techniques with multiple resolutions of the type described in the Spanish patent with publication number 2 152 176. The low resolution processes shorten the search time by detecting specific regions and points of the contours of the objects at the same time that the image containing them is received. , while high resolution search processes are located in small regions around the points preselected by the previous processes, where there is usually a high contrast between the object and the background of the scene. Thus, parallel processing in low and high resolution allows to obtain results in a short time and with high precision.

Pre-processing of stereo images in low resolution can be done in reconfigurable electronic systems that, after quickly extracting specific information from mobile phones, send it to a processor that performs accurate and fast calculations of the stereo correspondence of two specific points P_ {d} and P_ {i} in high resolution images and, based on them, obtains the distances and speeds with whose values you can control other elements that can be connected to the system, either to record or store measurement data and images captured, control auxiliary cameras and to configure the measurement system adapting it to the measurement environment and
mobile phones

The availability of portable processors with sufficient calculation capacity for the processes required in this type of systems allows, on the other hand, that any of These systems can be installed and used in environments diverse.

Description of the drawings

To complement the description, in order to help a better understanding of the features of the invention and as an integral part of this report, a set of drawings or figures where, with an illustrative and non-limiting nature, The following has been represented:

Figure 1: Scheme of principle for measuring distances by stereo procedures, using two cameras with parallel optical axes and coplanar focal planes.

\hbox{P _{d} (x _{d} ,
y _{d} )}

y P_{i}(x_{i},y_{i}) de los planos focales.Figure 2: Related to Figure 1 to show in a three-dimensional space the relative situation of the focal planes of the two cameras, parallel to the plane Z = 0 at distance -f, where f is the focal length of the lenses. It also shows the parallelism of the optical axes, in the same plane as the Z axis used to measure distance to objects represented in the figure by a point P (x, y, z) in the space that, simultaneously, is projected at the points

 \ hbox {P d (x d,
and d) 

and P_ {i} (x_ {i}, y_ {i}) of the focal planes.

\hbox{P(x,y,z)}

al origen de coordenadas.Figures 3a, 3b and 3c: They are related to Figure 2 and show the projections of a point P (x, y, z) of the three-dimensional space in the planes X = 0, Y = 0 and Z = -f, relating the projections with the focal length of the lenses and the images of the point P (x, y, z) in the focal planes of the two lenses, identified at the points P_ {d} (x_ {d}, y_ {d}) and P_ {i} (x_ {i}, y_ {i}), whose coordinates in the respective focal planes are used to measure the coordinates and distance of the point

 \ hbox {P (x, y, z)} 

to the origin of coordinates.

Figures 4a, 4b and 4c: Present possible preferential provisions of the elements that make up the system measurement, which can be positioned or slid along the common support to get the baseline length to be the necessary to measure the desired range of distances and speeds.

Figure 5: Present the block diagram of a preferred architecture implemented in FPGAs or CPLDs for the electronic processing of the images of a channel, as well as the elements common to both channels, using a processor external for the calculation, registration and presentation of the measures of distances and speeds.

Description of the invention

The use of stereo images to determine distances to objects makes use of the disparity observed in the position of the objects in the pair of stereo images. For a better understanding of the following, the drawing in Figure 1 is attached, which shows the simplified geometry of a stereo system that uses two identical lenses arranged so that their optical axes are parallel and their corresponding focal planes are coplanar. The separation between the optical axes is determined by the position of the lenses, whose centers are located at the left and right ends, I and D, of a line of length b, the base of a triangle whose point P is opposite vertex , within the optical field of both lenses. The height of the point P on the baseline is considered as distance L to the baseline b. Figure 1 shows two other triangles similar to those formed by the line L with points I and D. These triangles are those that form lines f, which represent the focal distances of the lenses located in I and D, with the points P_ {i} and P_ {d} that correspond to the projections of point P on the focal planes of the lenses, where the images are captured. Stereo disparity is the difference between the lengths d_ {i} and d_ {d}, measured with respect to their respective origins in each focal plane. The disparity is a data that can be deduced by measuring the position of P_ {i} and P_ {d} in the images of the focal planes and the distance L can be expressed as L = \ frac {bf} {d_ {i} -d_ {d}} \ eqnum {(1)}

The distances d_ {i} and d_ {d} are measured in pixels , discrete elements of fixed size and located at regular and constant intervals called step or jump between pixels . The pixels constitute the focal plane where the image that the lenses project on the CMOS or CCD sensors is formed, transforming the photons into electrical signals that, in turn, translate into images. Known the number of pixels and the jump between them, that is, the resolution normally expressed in microns, the conversion to the metric units used for L, b and f is immediate.

Assuming that both the L and the distances the position of point P can be determined with the precision that described below, the speed can be determined by measuring the variation of the position of the point P with respect to the stereo system between successive images of a sequence, that is, the distance traveled by point P. If the time intervals between these images are known and stable, speed determination It is a simple process that will be performed by the external processor to which send the preprocessed data.

The operation of the system consists of segmenting, discriminating or isolating the object of interest, or parts of it, within the captured scene and applying a series of algorithms to calculate the successive positions of the P point of the mobile. This way you can calculate the distances | P_ {t} -P_ {t + 1} | traveled in time intervals | t-t + 1 | known, as a previous step to speed calculation, expressed as V_ {i} = \ frac {| P_ {t} -P_ {t + 1} |} {| t-t + 1 |} = \ frac {| P_ {t} -P_ {t + 1} |} { I} = T | P_ {t} -P_ {t-1} | \ eqnum {(2)} where I represents the time interval between the images captured in the moments te t + 1, and T represents the image rate, in images / second, or abbreviated ips, which can be graduated in the current sensors to obtain rates higher than 70 ips .

Given the vector nature of speed, the expression (2) is the result of properly considering the three components or coordinates P_ {ix}, P_ {iy}, P_ {iz} that they will have the points P in a three-dimensional space. For a better understanding of the above, the drawing in Figure 2 is attached, where point P (x, y, z) is represented, the focal planes where points P_ {d} (x_ {d}, y_ {d}) are projected and P_ {i} (x_ {i}, y_ {i}) and the X, Y, Z axes. In this Figure 2 it can be seen that the optical axes EO_ {d} and EO_ {i} of the focal length lenses f, are parallel to each other, determining a plane in the space that contains the Z axis, parallel to the axes Optical Both the Z axis and the two optical axes are contained in the plane Y = 0. The focal planes are contained in the Z plane = -f and, the points where normal optical axes are projected to they are considered their respective coordinate centers.

\hbox{P(x,y,z)}

en los planos Z = - f, X=0 e Y=0, y permiten deducir las expresiones que, a partir de las coordenadas (x_{d},y_{d}) y (x_{i},y_{i}) en los planos focales, obtienen las coordenadas (x,y,z) que determinan la posición espacial del punto P(x,y,z).From Figure 2, Figures 3a, 3b and 3c are obtained, which respectively represent the projections of the point

 \ hbox {P (x, y, z)} 

in the planes Z = - f, X = 0 and Y = 0, and allow to deduce the expressions that, from the coordinates (x_ {d}, y_ {d}) and (x_ {i}, y_ {i} ) in the focal planes, obtain the coordinates (x, y, z) that determine the spatial position of the point P (x, y, z).

Figure 3a shows the plane Z = - f, which it contains the two focal planes and the points P_ {d} (x_ {d}, y_ {d}) and P_ {i} (x_ {i}, y_ {i}) projected on them, where it is observed that y_ {d} = y_ {i}.

       \ newpage
    

Figure 3b shows the projection in the plane X = 0 and from it it can be deduced that \ frac {y_ {d}} {f} = \ frac {y} {z} \ qquad \ text {or} \ qquad \ frac {y_ {i}} {f} = \ frac {y} {z} , \ qquad \ text {since} \ qquad y_ {i} = y_ {d} \ eqnum {(3)}

Figure 3c shows the projection in the plane Y = 0 and from it it can be deduced that \ frac {\ frac {b} {2} -x} {z} = \ frac {x_ {d}} {f} \ qquad \ text {and, likewise} \ qquad \ frac {\ frac {b} {2} + x} {z} = \ frac {x_ {i}} {f} \ eqnum {(4)} and by adding the two expressions in (4) you get \ frac {b} {z} = \ frac {x_ {d} + x_ {i}} {f} \ qquad \ text {or its equivalent} \ qquad z = \ frac {bf} {x_ {d} + x_ {i}} \ eqnum {(5)} which coincides with the expression (1), where the sign of the respective d_ {x} has been considered.

Deduced the z coordinate, you can get the y of the expressions in (3), resulting y = \ frac {zy_ {i}} {f} = \ frac {zy_ {d}} {f} = \ frac {by_ {i}} {x_ {d} + x_ {i}} = \ frac {by_ {d}} {x_ {d} + x_ {i}} \ eqnum {(6)} and finally, subtracting the first expression in (4) from the second, you get x \ frac {2x} {z} = \ frac {x_ {i} -x_ {d}} {f} \ qquad \ text {from which it is extracted} \ qquad x = \ frac {b (x_ {i} - x_ {d})} {2 (x_ {i} + x_ {d})} \ eqnum {(7)}

If in the expressions (5), (6) and (7) the negative sense of X_ {d} is considered, the expressions for the three coordinates in the space of the point P (x, y, z) are obtained x = \ frac {b (x_ {i} -x_ {d})} {2 (x_ {i} + x_ {d})} \ qquad \ qquad y = \ frac {by_ {i}} {x_ {i } -x_ {d}} \ qquad \ qquad z = \ frac {bf} {x_ {i} -x_ {d}} \ eqnum {(8)} where x_ {i} -x_ {d} is equivalent to the disparity d_ {i} -d_ {d} contained in (1) and z equivalent to L.

From the expressions in (8) the expression of the distance D_ {P} from the point P (x, y, z) can be deduced to the origin of coordinates such as D_ {P} = \ sqrt {x ^ {2} + y ^ {2} + z2}} = \ frac {b} {2 (x_ {i} -x_ {d})} \ sqrt {( x_ {i} + x_ {d}) 2 + 4y_ {i} {} 2} + 4f 2} \ eqnum {(9)} and the space traveled by the mobile or distance D {\ Delta t} = | P_ {t} -P_ {t + 1} |, determined by D _ {\ Delta t} = \ sqrt {(x_ {t + 1} -x_ {t}) 2 + (y_ {t + 1} -y_ {t}) ^ 2 + (z_ {t +1} - z_ {t}) 2} \ eqnum {(10)}

The calculation of expressions (9) or (10) does not supposes an important computational load, neither is the speed calculation in (2), once known D Δt = | P_ {t} -P_ {t + 1} |, so the problem is fundamentally reduced to automatic search for correspondence of points P_ {d} (x_ {d}, y_ {d}) and P_ {i} (x_ {i}, y_ {i}), to be performed on each pair of stereo images and on the same points of the contour of the mobile, to measure the displacement or space traveled by said point that, for the purpose of calculation, equivalent to mobile. Therefore, it should be chosen in a high contrast region, which always occurs in the contour and, preferably, at both ends where there are less shadows. However, automatic search for correspondence of a point P_ {d} in the right focal plane with another P_ {i} in the plane focal left, is a process that can be long and that It is wise to optimize in a rational way in time and cost.

Due to the high resolution of the available sensors, the images of any scene usually contain excessive information, a high number of pixels whose exploration to perform the search for correspondence may require a proportionately excessive time. Taking advantage of the redundancy of data and the similarity of the brightness of contiguous pixels in an image, the search process can be accelerated by reducing the number of pixels to be examined, either by applying sampling or by lowering the resolution of the images by fusion of groups of contiguous pixels. which are averaged in one, so that the resolution and the number of data to be processed can be exponentially reduced. The technique, described in the Spanish patent with publication number 2 152 176, consists of making groups of 2x2 pixels, which can be imagined located in the vertices of a square, averaging its value of gray or brightness in a single element that we call a pixel . If new averages with groups of 2x2 pixels are made, the brightness or gray levels of the higher order pixels that are generated are equivalent to averages of groups of 4x4 pixels of the original image, which results in a spatial reduction of data of 2 2N factor, N being the number of averages made. The value of N is contextual and is related to the size of the object in the image, its texture and differential characteristics that allow it to be segmented or isolated from the rest of the image. A differential feature is movement. When comparing two successive images in sufficiently short time intervals, the background of the image remains fixed as the lighting changes are usually negligible. That is why you can easily distinguish which regions of the images have changed and associate the change to the different positions of a mobile in the images. For the total process to be efficient, it is necessary that the generation of the reduced resolution images be simultaneous or with negligible delay in relation to the images delivered by the sensors and, similarly, when working in a region of an image With reduced resolution, the corresponding coordinates of that region are easily deduced in the maximum resolution image. This type of parallelism is perfectly achievable with FPGA or CPLD devices, since its resources allow them to be configured to operate in this way.

The invention described consists of a series of algorithms whose implementation in high speed hardware allows to extract or segregate the regions corresponding to the mobile and identify in those regions points of its contour that can be easily identified in images of the sequence. The advantages that this implies are, firstly, to facilitate the search for correspondence of a point in each pair of images and, secondly, to facilitate the identification of the same point in the following pair of images.

The algorithms begin with one that allows obtaining an image of the mobile-free scene background, saving the background image in a FIFO memory - First In First Out - from where it is read in the same order in which it is written. The reading process is synchronous with the input of the pixels of an image of equal resolution, comparing both images pixel to pixel, which allows to extract and mark those pixels of the incoming image whose brightness differs from the background image above a threshold , which means associating them with a mobile region. While extracting and marking the corresponding mobile pixels, another algorithm updates the background image in a selective way that delays the incorporation of brightness from mobile regions in the background, while quickly absorbing small changes in brightness that, due to changes of ambient lighting, may slightly affect the brightness of the background regions. In this way an updated background is obtained with each incoming image, rewriting it in the FIFO, and a mechanism is enabled that, in case the mobile stops, after a few images it incorporates it to the updated background, replacing the remaining background occluded by him. In parallel with the processes of the previous algorithms, counters record the evolution of the coordinates (x, y) of the incoming image's pixels, which allows to register the coordinates of those corresponding to regions of the mobile and to know which have been the minimum and maximum coordinates of the maximum corresponding to the mobile, using the marks assigned in the previous step in the registration process. Since the minimum and maximum coordinates correspond to contour points and the algorithms are applied in parallel in each of the images of the pair, the association of minimum and maximum points in both images is done in real time and is equivalent to a correspondence association which cannot be considered definitive, since the processes have been carried out in low resolution images. However, the association exists and serves as a reference in an external process where, starting from the obtained Min / Max coordinates and by a simple conversion of low resolution coordinates to the corresponding high resolution coordinates, an optimal coincidence of regions in the regions will be sought. High-resolution images that will be the one that provides the pixel-level correspondence coordinates, in a short process that runs while the low-resolution subsystem is processing the next pair of images. In this way, the total process is carried out in real time, being the delay in presenting the distance or speed data less than the duration of an image.

The fine correspondence process is carried out in an external processor in whose memory the pair of high resolution images have been stored at the time they were being generated, those of low resolution and the binary marks that identify regions of the mobile, as well as the coordinates of the contour points from which those two will be chosen that, depending on the ambient lighting, have greater contrast because they are free of shadows. The process of searching for correspondence at the pixel level is performed with a software algorithm whose duration is quite short, since the search region is limited to the size that would suppose the decompression of the low resolution image pixel that has been used, expanding in two pixels on each side to absorb possible errors in the values of the low resolution coordinates that have been transferred. Thus, with an illustrative but not limiting nature, if the low-resolution images that were used were the result of two compressions, each pixel would correspond to 4x4 pixels. In this case, the search region is limited to 5x5 pixels, the central one being the one where the specific point of correspondence has been set. In high resolution, the 5x5 pixels correspond to a window of 20x20 pixels, where the central region of 3x3 pixels, that is, 12x12 pixels, is moved to find the maximum coincidence position between the two images of the pair, whose central coordinates will be the of P_ {d} (x_ {d}, y_ {d}) and P_ {i} (x_ {i}, y_ {i}).

On the other hand, in every measurement system it is necessary to know the precision and accuracy with which it is capable of measuring. Deriving equation (1) it is easy to deduce that \ frac {dL} {d | d_ {i} -d_ {d} |} = \ frac {L ^ {2}} {bf}, \ qquad \ text {equivalent to} \ qquad \ Delta L = \ frac { L2} {bf} \ cdot \ Delta | d_ {i} -d_ {d} | = \ frac {L2} {bf} \ cdot \ Delta d \ eqnum {(11)} where \ DeltaL represents the distance resolution, that is, the minimum discernible change in the measured distance L, corresponding to the value of \ Minimum deltad, or minimum value of the disparity d_ {i} -d_ {d}. The value of \ Deltad is the step between pixels. Equation (11) indicates that the distance resolution is so much better, or that the discernible distances are smaller, the greater the focal length f and the baseline b, and that it grows or decreases proportionally with the resolution of the sensor, \ Deltad , determined by the number of pixels per unit length, while it worsens proportionally to the square of the distance L. Based on this data, after selecting the most convenient sensor and knowing its resolution \ Deltad, the design and configuration of a system Speed measurement can be done contextually, relating the range of speeds that you want to measure and the distance of the mobiles with the length of the baseline b that the system must have, since the commercially available lenses allow you to select them within a relatively wide range of qualities and focal length values f.

While \ Deltad will determine the best resolution what can be expected from the system, the accuracy of the measurements will be related to the repeatability or variation of the values which are obtained by performing the same measure several times in some certain conditions The values obtained when making the measures usually have a normal statistical distribution, represented by the variance of said distribution and indicative of System accuracy On the other hand, the accuracy of measures depends on systematic errors due to errors of calibration, mismatches or defects in the system configuration.  Both precision and accuracy can be determined and optimize when the system is installed and ready to to size.

Description of a preferred embodiment

Since the baseline is an important parameter in any equation related to the measure of distances by stereo images, a rigid and resistant structure is required, preferably aluminum, containing electronic circuitry and that allows the placement of the cameras on a stable base, whose length, thickness and strength must be consistent with the use of that the system be destined.

On the rigid support indicated in Figures 4a, 4b and 4c outline possible arrangements of the cameras D e I, of the printed circuits that perform the processing of images, identified as Subsystem D and Subsystem I, and of the printed circuit of the adapter that concentrates the data of both channels to send them to the external system through a cable identified as Cable S.

Printed circuits are linked together by cables identified as Cable D and Cable I according to the channel to which are associated. Depending on the length suitable for the baseline, channel adapter elements and subsystems of each channel are preferably concentrated in a single printed circuit board located approximately in the center of the baseline, as detailed in Figure 4a, but may be separated, if the length of the baseline is relatively large and the quality of the signals requires that the processing subsystems at approximately equal distances of cameras and channel adapter.

It is also possible to fix and connect the cameras as detailed in Figure 4b, directly on the subsystems that can slide and be fixed along the rigid support so that the distance b between the optical axes is the most convenient and, if the length of the line base is not excessively large, in a single printed circuit the cameras and all electronic circuitry can be mounted, as shown in Figure 4c, where, in order to have greater versatility and allow different lengths of baseline, more are available of a location or receptacle for the cameras of each channel, which is also applicable in Figures 4a and 4b. Thus, with an illustrative but not limiting nature, the camera D can be in the location D1 or D2, and the camera I in the I1 or I2, which allows to have four different baseline lengths, if the distances D1- are different D2 and I1-I2. The cameras, apart from their electrical connection, are mechanically attached to their supports with special screws or adjustable mechanical elements that allow a fine adjustment of the orientation of their focal planes, to achieve the parallelism of the axes
Optical

Due to the dynamic characteristics of the scenes, the system must operate with high image processing speed. Therefore, the preferable embodiment is based on hardware circuits based on FPGA, CPLD or DSP type devices, since the application of software algorithms for image capture would, in general, slower responses and would be inadequate to obtain the precise data in short time intervals For a better understanding of the operation of the circuitry and its functional blocks, Figure 5 is attached, which shows the processing subsystem of one of the two channels, identical to the other in structure and functionality, as well as the set of elements common to the two channels The function of the processing blocks and signals is described at
continuation:

Master oscillator

Common to both channels, so that both act synchronized by him. The stereo pairs are captured in them moments of time, otherwise it would not be possible to obtain instant correspondence of the points P_ {i} (x_ {i}, y_ {i}) and P_ {d} (x_ {d}, y_ {d}) which are observed in the image sequences. Of its coordinates the spaces covered in the time intervals that are deducted are deducted be fixed. These intervals are derived from the master clock which, applied in equation (2), they allow to calculate speeds.

External processor

\hbox{P _{d} (x _{d} ,y _{d} )}

en alta resolución. El programa software, por la ecuación (10) obtiene los espacios recorridos por el móvil y, por la ecuación (2), calcula, registra y muestra las distancias y velocidades medidas y, opcionalmente, una serie de datos adicionales como la hora y condiciones en que se realizan las medidas, las imágenes de las que se obtienen y datos como varianzas, distancias al móvil en cada par estéreo según la ecuación (9) y otras afines que, según el interés del usuario, pueden obtenerse por software a partir de los datos registrados en las memorias del procesador.It is the common element that, through an operator and with a specific software for calculation and presentation of speeds, configures the two channels of the system by means of output instructions that are decoded or interpreted in the channels. In addition, it stores the images captured by the two sensors and others of low resolution with signals that serve as a complement to find the correspondence and coordinates of the points P_ {i} (x_ {i}, and {i}) and

 \ hbox {P d (x d, y d)} 

in high resolution. The software program, by equation (10) obtains the spaces covered by the mobile and, by equation (2), calculates, records and displays the distances and speeds measured and, optionally, a series of additional data such as time and conditions in which the measurements are made, the images from which they are obtained and data such as variances, distances to the mobile in each stereo pair according to equation (9) and other related ones that, according to the user's interest, can be obtained by software from the data recorded in the processor memories.

Channel adapter

Common element that distributes instructions to the two channels and temporarily stores the data coming from them, sequencing or multiplexing its transmission to the processor. It contains a FIFO memory - First In First Out - for each channel, which allows it to transmit at a faster rate than it receives, making it at alternate bursts between both channels when the FIFOs contain a number of data that exceeds a configurable preset value from the processor, leaving the two FIFOs empty at the end of the processing of a stereo pair. The adapter inserts default headers at the start of each FIFO burst, to indicate from which channel the data that follows follows. It also acts as an interface to adapt the electrical levels of the signals emitted / received by the channels and those received / emitted by the processor.

Lens and Sensor

They constitute a camera that captures the images of the scenes. The camera outputs are the signals generated by the CCD or CMOS image sensor, adapted in characteristics and electrical levels for transmission. Illustratively and not limitatively, in Figure 5, they concentrate on 8 signals from a bus identified as Im (7..0), which contains the pixel brightness levels and even the idle times between pixels. For greater clarity of Figure 5, no other signals are always shown that always supply these sensors, whose functions are the validation of the data Im (7..0) by a signal known as pixel clock, horizontal or line synchronism and the vertical or image synchronism. These signals are related to the frequency and phase of the Master Oscillator described above which, with the circuitry necessary for it, controls the rate of images and synchronisms.

Command decoder

Receive the processor instructions that control the system Decode the instructions received and, by means of a protocol and conversion to interpretable signals by sensors, configure them to provide images with a variable number of lines and columns of pixels, adapts the Sensitivity of the sensors to the light conditions of the scenes, regulates the rate of images supplied by adjusting delays between successive lines and successive images and graduates the pixel gray levels over thresholds or black levels variables, to compensate for possible differences in characteristics of the right and left channel sensors.

Channel input / output interface

Multiplex the data sent to the processor and separate the formatting and image rate instructions that configure sensor and multiresolution generator, of those related to image processing modules.

Multi-Resolution Generator

Make successive averages of 4 to 1, generating 1/2 2N resolution images through the Mx bus (7..0). The value of N is part of an instruction issued by the processor that is interpreted in the Sensor Command Decoder. Low resolution images are used to accelerate the process of locating mobiles in scenes. Since in certain applications it is not necessary to use all the information provided by the 8 to 10 bits that encode the brightness of the pixels, by other instructions you can order the truncation of the least significant bits that are considered unnecessary, both from the image of high resolution as of the low resolution image obtained at the generator output.

Threshold decoder

Receive specific instructions from the processor and  extract from them the threshold values used in the blocks  identified as background and mask extractors.

Background extractor and background FIFO

They work in parallel. The extractor receives Low resolution images Mx (7..0) and transfer them to a External FIFO, max to max, after performing an update fast brightness of the pixels that are below a brightness difference threshold and a slow update with those who overcome it. The extractor makes a comparison, max at max, between the image you are receiving and the same resolution extracted from FIFO, which is the background image Updated with the previous image. The incoming pixels whose difference with the corresponding pixels of the background image exceeds the established threshold, they are considered to belong to a mobile and are updated in a small percentage of the value of the difference with the background image, while the pixels that are around below the threshold they are updated in a high percentage. After each correction, the pixels are forwarded to the FIFO that looks like this updated with a brightness value almost equal to that of the image incoming in the corresponding maximum pixels, which may be slightly changed due to the effect of ambient light, while the pixels that contain high differences with the threshold are approximate in value to the brightness of the incoming pixel, but slowly, to keep the differences with the threshold in the next picture. In this way, if a mobile were to stay still, after about 15 or 20 update images would integrate into the background image, since it would cease to be mobile at stand still, gradually replacing the real fund that is being occluded by him. In scenes where the mobile evolves, the slow correction effect on the background image is a wake of the mobile that modifies the value of the fund very slightly, keeping the difference with the threshold when making comparisons with the following incoming image and allowing to extract the mobile from the background and Determine your trajectory. Since the comparison is made with resolution images lower than those generated by the sensors, that is, with a number of pixels that will be 1/4, 1/16 or 1/2 N of the incoming pixel to the Multi-Resolution Generator, it is possible to perform all comparison and reload operations of FIFO without time being a critical factor.

The creation of the first fund is done in less than one second when starting the operation of the system, eliminating the mobiles that may exist in the first images, if any. With rates of 25 images / second, 20 images are sufficient for Have a useful background. The threshold value of the differences of brightness, related to the brightness of the scenes, is reset automatically in the processor by an algorithm that gets the average brightness of the images you receive, generating instructions from threshold update when it has changed. The channels separate the instructions and, through their Decoders thresholds, set the background Extractor comparators.

Msk mask extractor

It acts by comparing the background image and the low resolution image incoming to it. It uses another threshold similar to the one mentioned above to generate an image that only contains information in the pixels corresponding to the mobile region, while the rest are null. In this way, a binary mask image corresponding to the low resolution image is obtained, encoded with a value of 1 or 0 for the mobile regions and the opposite ones, 0 or 1, for the background. The Msk bits of the binary mask are sent to the processor as the header or footer of the brightness of the pixels of the low resolution image Mx (7..0). This identifies the pixels that correspond to mobile, facilitating the extraction of coordinates from the mobile region in high resolution images, in relation to the coordinates of the mobile in low resolution.

Min / max

Functional block responsible for recording the maximum and minimum coordinates of the mobile region extracted from the binary mask. Points P_ {i} (x_ {i}, y_ {i}) and P_ {d} (x_ {d}, y_ {d}) of the high-resolution image can be located in an ordinate y_ {i} = y_ {d} equal or very close to that corresponding to Ymin or Ymax , or in the ordinate corresponding to points Xmin or Xmax , which are obtained after making the correspondence from low to high resolution. The Xmin , Xmax , Ymin and Ymax coordinates identify significant points of the contour of the mobile that contrast with the image background, so they are useful references to track and, to that end, are sent to the processor after completing the sending of the data contained in the low resolution FIFO.

FIFO low resolution

Temporarily stores the pixels of the low resolution image Mx (7..0) used in the background and mask extraction processes, in addition to the Msk bit that identifies the mobile region pixels. Since the number of data in this FIFO is relatively low, they are periodically interleaved between the pixels Im (7..0) of the high resolution image for transmission to the channel adapter. With this low resolution image and the corresponding one from the other channel, both with information on the mobile position and the extreme coordinates of it Xmin , Xmax , Ymin and Ymax , the processor estimates the coordinates of the points P_ {i } (x_ {i}, y_ {i}) and P_ {d} (x_ {d}, y_ {d}) corresponding in the high-resolution images stored in memory y, looking for the minimum differences in processes of Comparison of brightness of small regions around such points, selectively makes the fast, precise and definitive search of the coordinates x_ {i} and X_ {d}, to calculate from them the distances traveled in the predetermined time intervals to get the speed measurements.

Claims (11)

1. System for measuring distances and speeds by non-radiating electro-optical procedures, characterized by using multiresolution techniques to analyze stereo images and accelerate the process of searching stereo maps. 2. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claim 1, characterized by its ability to perform the following functions: First. Obtain significant points in contours of mobile regions in stereo image sequences captured by two cameras arranged on a rigid support, performing the following operations or steps in both stereo channels simultaneously:

to-

Reduce the resolution by one or more successive compressions of 2x2 elements to 1.

b-

Compare resolution images reduced with others of equal resolution that contain the background of the scenes, updating said background with the incoming image based on a threshold of brightness differences between both images.

C-

Simultaneously with the previous step, extract the mobile regions by difference of brightness between the elements of incoming and existing images as background.

d-

In parallel with c), extract a binary mask that identifies regions mobile and static for the value of one bit, assigned in reference at a threshold of brightness.

and-

Add to the brightness code of each reduced resolution image element corresponding bit of the binary image extracted in step d), as a header bit or stand, and temporarily store the set in a memory FIFO

F-

Record the minimum coordinates and highs of the regions that identify mobiles in the Binary images, while these are being generated.

Second. Receive from an external processor instructions with information for:

to-

Configure the sensors and define the type of images they supply.

b-

Configure the operating mode of the multi-resolution generator that reduces image resolution that deliver the sensors.

C-

Configure image extractors background and binary mask.

Third. Receive simultaneously from both channels, in an element common to them provided with two FIFOs of reconfigurable lengths and widths, one for each channel, in progressive and with periodic sequences from each channel, the Following data:

to-

The pixels of high resolution images generated by sensors

b-

The elements contained in the FIFOs of step e) previously aforementioned.

C-

The minimum and maximum coordinates recorded in step f) previously aforementioned.

Quarter. Transmit to an external processor the data contained in the two FIFOs of the common element to both channels, in fragments taken alternately from the two FIFOs and transmitted in high-speed bursts before the sensors begin to generate a new pair of images, in order that the processor apply a series of software algorithms for the precise search of the correspondence of specific points in the contour of the mobile region and calculate the distance and speed values. 3. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 and 2, characterized by having in each of its channels a reconfigurable generator that supplies images of reduced resolution by N successive compressions of 2x2 elements to 1, N being an integer sent as part of external instructions that configure the generator to maintain or reduce the number of bits that encode the brightness of the elements of the incoming images to it and the outgoing ones with lower resolution. 4. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 to 3, characterized by having an image background extractor in each channel, implemented in reconfigurable hardware devices that, by instructions of an external processor, update of immediately the value of a threshold of difference between the brightness of corresponding elements of an image called background and an incoming image with the same resolution, which allows them to be compared as the elements of the latter enter and, taking as reference the threshold , consider the elements of the incoming images as belonging to a mobile or static region and update the background image with the brightness of the incoming image in a selective way that delays the update of mobile regions while applying possible changes almost immediately brightness in static regions, saving results of the updates in a progressive and orderly way element by element in external FIFO memories, from which they will be read in the same order to compare the elements with those of new incoming images, detect new mobile regions and update the background image in the manner described. 5. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 to 4, characterized by having in each of its channels an image extractor or binary masks corresponding to the mobile region that is extracted, assigning binary values 1 or 0 to the elements of the mobile region and the opposites, 0 or 1, to the elements that correspond to regions of the image background not occluded by the mobile regions. 6. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 to 5, characterized in that in each of its channels of registers the minimum and maximum coordinates Xmin, Xmax, Ymin and Ymax of the binarized mobile regions , updating the records automatically based on the value of the bit and with the progression of the coordinates of the elements of the binary image of the mobile. 7. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 to 6, characterized in that in each of its channels a FIFO memory composed of static memory elements of the same FPGA or CPLD in which they configure the subsystems of each channel, the width of said FIFOs being sufficient to temporarily store the encoded values of the brightness of the reduced resolution elements, to which are added as header or footer the binary values corresponding to the mask that identifies them as belonging to mobile region or static region. 8. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 to 7, characterized by having in each of its channels an interface of inputs / outputs whose functions are to receive instructions from an external processor and multiplex in its outputs periodically and encoded the signals from the high resolution image sensors, the data read from the FIFOs that contain the reduced resolution image elements with the mask bits and, finally, the minimum and maximum coordinates contained in logs loaded when generating binary masks. 9. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 to 8, characterized by having an element common to its two channels provided with two FIFOs of reconfigurable lengths and widths, one for each channel, which allow receiving data simultaneously from both channels and transmit them in high-speed alternating bursts to an external processor that thus receives all the data coming from the channels before starting a new pair of images on the sensors, applying a series of algorithms to determine the stereo correspondences of two of the points Xmin, Xmax, Ymin and Ymax that, depending on the ambient light, present greater contrast because they are more free of shadows and, with a specific software program, present the measures of distances and speeds derived from it in the form convenient to the user of the system. 10. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1 and 2, characterized by having a pair of cameras with identical lenses and reconfigurable identical sensors, fixed on a rigid support prepared for said cameras or the elements that support them can be positioned or slid so that the distance b between the two cameras can be set at the user's discretion. 11. System for measuring distances and speeds by non-radiating electro-optical procedures, according to claims 1, 2 and 10, characterized by its ability to adjust the orientation of the cameras with adjustable mechanical elements, so that their optical axes are parallel and contained in the same plane, regardless of the position of the cameras along the rigid support that keeps them at distance b convenient to measure the ranges of distances and desired speeds.