DE60305124T2 - Multifunctional integrated optical system with a CMOS or CCD matrix - Google Patents

Abstract

The invention describes a system with a multifunctional integrated visual sensor using a CMOS or CCD technology matrix having a sensitive area divided into sub-areas dedicated to a series of specific functions. <IMAGE>

Description

The The present invention relates to an optical system, in particular for use on motor vehicles, for detecting environmental parameters, such as fogging the motor vehicle windshield or the presence of raindrops on the windshield or states a dim lighting while driving through a tunnel, under a bridge or due to a dusk or the presence of light rain or fog or the meeting to another vehicle, the system also for monitoring the scene in front of the vehicle ("front side monitoring") is to be used, for example, the presence of a curve in such a time to grasp in advance that the movement of the adaptive low-beam before the curve begins, or to a lateral movement of the vehicle towards road markings ("road lane warning").

One Optical system of the type defined in the preamble of claim 1 is known, is known from WO A-0 053 466. Furthermore, one is Device for detecting the scenery in front of the vehicle as well as for Capturing different states, such as for example, a presence of raindrops on the windshield, a twilight state and driving through a tunnel, also known from DE-A-197 04 818.

The The aim of the present invention is to carry out a relatively simple and reliable System that efficiently covers all or part of the above Perform functions can. Another goal is better than current systems for motor vehicles to be a package integration of different sensors for disposal is provided.

in view of To achieve the goal, the invention relates to an optical System as defined in claim 1.

The The invention has the particular object of an optical system of the Type referred to above using a monochromatic, linear or logarithmic VGA-CMOS matrix such in a motor vehicle to integrate that, for example, close to an interior rearview mirror The motor vehicle is placed to perform several functions below perform: a rain detection, a fog detection or fog detection, a fog detection, a twilight detection, a detection of driving in a tunnel, a detection of a meeting to another vehicle, a front side monitor.

Further preferred and advantageous features of the invention are in the dependent claims 2-26 Are defined.

Obviously be output signals from a visual matrix system in the direction directed to an electronic system that images in relation to the various subregions (through "windowing" in the case of CMOS cameras) and She then processed.

Further Characteristic features and advantages of the invention are intended to be apparent the following description with reference to the accompanying drawings which are merely non-limiting Example are provided, wherein:

1 Fig. 12 is a schematic view of a first embodiment of the matrix sensor according to the invention, which implements all functions discussed above;

2 . 3 are schematic views showing the operating principles of the sensor that detects a windshield fog or a Windshield fitting;

4 is a schematic view with respect to the configuration of the sensor for a twilight / tunneling function;

5 is a general diagram showing active fog detection;

6 . 7 show a second and a third embodiment of the system according to the invention, which implements a smaller number of functions;

7A . 7B show a schematic split view and a top view of the sensor associated with the system according to the invention;

8th shows a perspective view of an example of an embodiment of the system according to the invention;

9 a variant of 8th shows;

10 shows the possible embodiments of the optical system for fog detection;

11 . 12A . 12B show three different perspective views of the arrangement containing the sensor array according to the invention, with the associated protective window; and

13 Figure 3 is a perspective view of a prism used in the rain detection system according to the invention.

1 In the accompanying drawings, a preferred embodiment relating to the division of the sensitive area of the CMOS matrix of the sensor according to the invention into sub-areas intended for one or more functions. This embodiment takes into account all the above-mentioned functions: a front-side monitoring, a hitting a vehicle, rain, nebulization, fog (active and passive mode), twilight and tunnel.

1 shows the sensitive area of a VGA matrix with an indication of the functions performed by each sub-area and their features.

• 1. Die Neigung der Matrix, wenn sie an dem inneren Rückspiegel des Motorfahrzeugs angebracht ist, hängt von der Richtung der optischen Achse zum Durchführen einer Vorderseitenüberwachungsfunktion einer Szenerie ab;
• 2. Für jede Funktion ist die Größe der Bereiche eine Funktion der Blickfelder und der erforderlichen Auflösung;
• 3. Die Positionen der Bereiche hängen ab von der Richtung der optischen Achse jeder Funktion und von der Notwendigkeit zur Trennung von Bereichen, in welchen Pixel nicht verwendet werden;
• 4. Ein einzelner Bereich kann für mehrere Funktionen bestimmt sein; oder ein Teil des Bereichs, der für eine Funktion bestimmt ist, kann auch für eine andere Funktion bestimmt sein.

The division of the different sub-areas takes into account some basic criteria:

• 1. The inclination of the matrix, when mounted on the interior rearview mirror of the motor vehicle, depends on the direction of the optical axis for performing a front-side monitoring function of a scene;
• 2. For each function, the size of the areas is a function of the fields of view and the resolution required;
• 3. The positions of the regions depend on the direction of the optical axis of each function and the need to separate regions in which pixels are not used;
• 4. A single area may be dedicated to multiple functions; or a part of the area intended for a function may also be intended for another function.

Possible measurement techniques to be used to develop the functions to which in 1 Reference will now be made.

Windshield fogging

passive System: On an outer surface of the windshield is a reference image, such as a grid, which is on the CMOS matrix is focused. The sharpness of the picture depends on the extent of Nebulization or the fitting of the inner surface of the windshield. The critical points of this technique are the following: the sensitivity across from Nebulization dimensions, which can not be seen by human eyes; the dependence the signal from an ambient exposure.

Active system: - as it is in the 2 . 3 is shown, an infrared transmitter E sends a bundle on the inner surface of the glass of a windshield P of the motor vehicle with an angle of incidence of about 45 °. When the surface is fogged, the bundle is condensed by condensed droplets ( 3 ) backscattered and detected by the CMOS matrix M. If there is no fitting ( 2 ), the aforementioned phenomenon does not occur. The optical system may shape the beam emitted by emitter E (typically an LED) to focus on a comfortable portion of the windshield.

rain

The CMOS matrix captures the image of drops on the outer surface of the Windscreen lie. A quantitative analysis can be done at one single picture, for example, by considering the spectrum with respect to a spatial Frequency or by comparing successive pictures be performed with statistical methods. To the dependence from external exposure conditions to eliminate, which affect a drop contrast, and on a uniform and temporally constant way the subject windshield area illuminate an LED source at a radiation near the Infrared range and a bandpass filter tuned to an LED wavelength used.

Dusk / tunnel

• – kleiner fester Winkel A in Fahrtrichtung (beispielsweise 10°C);
• – größerer fester Winkel B (beispielsweise 40°), der nach oben gerichtet ist, um eine Messung einer durchschnittlichen Belichtung um eine aktuelle Fahrzeugposition zu erreichen.

Two matrix areas are determined for illumination measurements taken from two different directions (see 4 ):

• - small fixed angle A in the direction of travel (for example 10 ° C);
• A larger fixed angle B (for example 40 °) pointing upwards in order to obtain a measurement of an average exposure around an actual vehicle position.

fog

passive Sensor: Passive fog detection is done by scene capture performed in the area the for a front side monitor is determined, and by a subsequent analysis of an image sharpness. she allows the Fog bank to record in advance in terms of an active system which is a limited one Action area has.

Active sensor: the visibility sensor has a transmitter module (LED or laser diode for infrared radiation) and a receiver module (CMOS camera). The two fields of view are partly overlapping. In the case of fog causes the drip concentration in the overlapping area, a backscatter of the beam, which is detected by the sensor ( 5 ).

Meet up a vehicle

The CMOS camera forms some parts of the street scenes in front of the vehicle. The matrix of 1 contains two areas for a "meeting on a vehicle", each intended for the detection of the dipped headlights of vehicles driving in the opposite direction, and the taillights of vehicles driving in the same direction. For the two ranges, well-suited high-pass and low-pass filters are used for distinguishing lowered dipped-beam headlights from taillights. An alternative is to use the area designed for frontside monitoring in a color matrix or in a monochromatic matrix by laying the necessary optical filters on a pixel plane, albeit only in the matrix area or subareas developed for frontside surveillance or are.

Front monitoring the scenery

The main part of the matrix of 1 is used to form the road ahead of the vehicle and to implement functions such as lane warning, adaptive low beam, and vehicle hitting.

This in 6 The embodiment shown is a simplified embodiment with respect to that in FIG 1 and contains only sub-areas for front monitoring, tunnel, fog (active and passive technique) and twilight functions.

7 shows a further embodiment, which also envisages a rain / misting function in addition.

CMOS matrix

• – VGA-Format: dieses Format ermöglicht eine Aufteilung der Matrix in sieben verwendete Bereiche und in eine Trennung von Bereichen, in welchen Pixel nicht verwendet werden;
• – Pixelgröße: eine sehr kleine Pixelgröße impliziert eine höhere Schwierigkeit beim Ausführen von bestimmten optischen Systemen für jede Funktion/jeden Bereich;
• – monochromatisch oder farbig: die Verwendung einer Farbmatrix würde den Einsatz von optischen Filtern vermeiden, die für die Funktion "Treffen auf ein Fahrzeug" erforderlich sind, jedoch zum Nachteil einer Raumauflösungsfähigkeit;
• – lineare oder logarithmische Reaktion: einige Funktionen basieren auf der quantitativen Festsetzung eines Parameters, für welchen es besser wäre, eine lineare Reaktion des Sensors und eine konstante Integrationszeit zu haben; insoweit der dynamische Bereich betroffen ist, ist es jedoch vorzuziehen, einen Sensor mit logarithmischer Reaktion zu verwenden.
• – Dynamikbereich: Vorderseitenüberwachungsfunktionen erfordern, dass der Sensor einen Belichtungsbereich oberhalb von 80 dB abdecken kann, da typische Szenerien eine Situation enthalten können, die von einer Dämmerung zu direkten Sonnenstrahlen variiert;
• – Empfindlichkeit: dies ist ein wichtiger Parameter für Vorderseitenüberwachungsfunktionen während eines Fahrens in der Nacht mit einem niedrigen Beleuchtungspegel (beispielsweise gilt für Dämmerung = 1 Lux) oder für eine Nebelfunktion, wenn die Rückstreustrahlung, die gesammelt wird, sehr gering ist (im Bereich von nW);
• – spektrale Reaktion: der Sensor sollte eine gute Reaktion im Bereich von 800–900 nm haben (der für aktive Funktionen verwendet wird – wo das Windschutzscheibenglas eines standardmäßigen Motorfahrzeugs eine Transmittanz von etwa 30 % hat); für eine "Dämmerung"-Funktion solle der Sensor eine Reaktion möglichst gleich derjenigen von menschlichen Augen haben (fotometrische Reaktion);
• – Framerate: Framerate ist kein kritischer Parameter, solange schnelle Erfassungen mittels einer Fenstergebung durchzuführen sind, und darauf folgend dann digitale Filtertechniken zu implementieren sind (beispielsweise in dem Fall der Funktion eines Treffens auf ein Fahrzeug, in welchem es nützlich wäre, die modulierte Strahlung, die von mit Wechselspannung versorgten Lichtquellen mit Frequenzen von 50- 60 Hz kommt, zu beschneiden).

As far as a visual CMOS matrix is concerned, the main requirements that it should fulfill for the multifunctional integration proposed in the previous paragraphs are as follows:

• - VGA format: this format allows to divide the matrix into seven areas used and to separate areas where pixels are not used;
• Pixel size: a very small pixel size implies a higher difficulty in running certain optical systems for each function / area;
• Monochromatic or colored: the use of a color matrix would avoid the use of optical filters required for the on-vehicle function, but to the detriment of space resolution capability;
• Linear or logarithmic reaction: some functions are based on the quantitative determination of a parameter for which it would be better to have a linear sensor response and a constant integration time; however, as far as the dynamic range is concerned, it is preferable to use a logarithmic sensor.
• - Dynamic range: Front monitor functions require the sensor to cover an exposure range above 80 dB, as typical scenes may contain a situation that varies from dusk to direct sun;
• Sensitivity: this is an important parameter for front monitor functions during night driving with a low level of illumination (for example, for dusk = 1 lux) or for a fog function when the backscatter radiation being collected is very low (in the range of nW );
• Spectral response: the sensor should have a good response in the range of 800-900 nm (which is used for active functions - where the windshield glass of a standard motor vehicle has a transmittance of about 30%); for a "twilight" function, the sensor should have a response as close as possible to that of human eyes (photometric reaction);
• Frame rate: frame rate is not a critical parameter as long as fast captures are to be performed by windowing and then digital filtering techniques are to be implemented (for example, in the case of meeting a vehicle in which it would be useful to have the modulated radiation, which comes from AC powered light sources with frequencies of 50-60 Hz comes to crop).

at a practical example of an embodiment of the invention is a logarithmic monochromatic VGA CMOS camera used been executed with a CMOS technology of 0.35 microns. It integrates the matrix of active pixels, gain levels, a 10-bit A / D converter and the interface for a microprocessor. The Photodiode array continuously converts radiation into voltage (without charge integration, as is typical with linear CMOS), and therefore you can Pixels are read at any time.

The current generated by the photodiodes has a linear dependence on the intensity of the incident radiation. The supply circuit of the photodiode has a logarithmic property, so a pixel voltage output is proportional to the logarithm of the incident radiation.

• – die optischen Systeme sollten derart platziert sein, dass eines neben dem anderen ist. Dies begrenzt die Größe von einzelnen Komponenten. Bei der Bewertung der Grenzen sollten mögliche mechanische Lösungen für Montage- und Packungsoperationen auch berücksichtigt werden;
• – die Nähe der Bereiche resultiert in Störungen zwischen den Signalen der einzelnen Funktionen aufgrund der teilweisen Überlagerung von Bildebenen. Bezüglich dieser Probleme sollte ein Trennungssystem für die einzelnen Bereiche entwickelt werden.

With respect to a sensor package, the latter is in 7A (in lateral section) and in 7B (which shows a view from above). The package has a base 10 on, on which the sensor chip 11 attached, and an optical protection window 12 , which consists of a substrate of transparent material, with flat and parallel surfaces. The CMOS matrix is associated with optical components designed to optimize the signal collected by the integrated sensor for each of the environmental monitoring functions; in the case of rain, vehicle encounter and observer functions, the optical components should also perform scene imaging. For each function, the field of view and the particular sensor area determine the focal length of the optical component to the intermediate position. Once the focal length has been adjusted, the exposure level requirements limit the minimum dimension of the lens. The conditions imposed by the system geometry should continue to be considered. In fact, each of the adjacent areas into which a matrix is divided is associated with an independent optical system. At a development level, this type of integration implies two additional conditions:

• The optical systems should be placed in such a way that one is next to the other. This limits the size of individual components. When evaluating the limits, possible mechanical solutions for assembly and packaging operations should also be considered;
• - The proximity of the areas results in interference between the signals of the individual functions due to the partial overlay of image planes. With regard to these problems, a separation system for each area should be developed.

A Another feature to consider is the orientation the optical axis for each Function. It is obvious that it is necessary to fulfill all requirements, solutions to change to find the direction of the optical axis. Used for this purpose the invention, as will become apparent hereinafter, prisms and optical fibers.

There for all Functions the distance of the target point over a 10 times the focal length is the distance between the optical system and the sensor (pixel) be nearly the same as the focal length, what "rain", "twilight" - and "meeting on a vehicle" - (detection of Taillights) functions under 5 mm. The distance is a strict requirement, and in particular then, when arranging prisms between the lens and the sensor are.

8th shows a perspective view of an embodiment of the system according to the invention. The with a protective window 12 equipped CMOS matrix 11 is a lens 13 assigned to front side monitoring. As already stated, the object is 13 together with the sensor 11 in the front part (ie in the direction of travel) of the rearview mirror of the motor vehicle (not shown for reasons of clarity). In one example of an embodiment, the lens has 13 a negative lens as the first element to cover a very wide field of view (35 ° -45 °).

Then there is the problem in terms of isolating a front monitor function. A possible solution contains the use of a partition with a rectangular opening. The opening has the same shape and size as the front panel areas.

considered Now you have "twilight" and "fog" functions it's the problem the fact that the optical axis, for example for a Angle of 60 ° in Reference to directions of movement should be tilted upwards. A Capture is of the type no rendering.

For these two functions, the invention solves the problem of tilting the optical axis by using conventional optical fibers such as the optical fiber 14 in 8th , Such an optical fiber makes it possible to transmit the signal along a path, which may also be curved. The components are still very cheap. The optical fiber is preferably associated with a lens, such as a ball lens or a grin lens (gradient of an index).

8th also shows a lens 15 for a rain detection and an emitter 16 to which a condenser lens 17 and an optical fiber 18 are assigned for a fog detection.

9 shows a variant of 8th , where corresponding elements have the same reference numerals.

10 shows the different possibilities of assigning a GRIN lens 19A or a ball lens 19B to an optical fiber 18 or leaving the fiber end without a lens, as in 20 is shown. The opposite end of the fiber 18 can with a GRIN lens 21 be provided, or a micro-optical system 22 while still being able to read the end without a lens sen, as is in 23 is shown. The condenser lens 17 is an optical bandpass filter 24 assigned.

One step after integrating "front-face", "fog" and "dusk" functions through a lens and optical fibers is the additional integration of a "rain" feature. The latter function requires optical imaging components, ie lenses and prisms. The field of view of a front monitor function has an optical axis which is inclined downwards by a few degrees. For Twilight, Rain, and Fog functions, the slope is 60 degrees up. In fact, for the latter three functions there is a wide tolerance. A variation of 5-6 degrees contains no significant change as far as functionality is concerned. One possible optical solution is to maintain a matrix plane orthogonal to the optical axis of a front-face monitoring function and to change the direction of the optical axis for the other three functions by means of prisms. The tolerance range for an alignment makes it possible to select a prism that deflects the optical axis by 60 degrees with respect to the perpendicular to the image plane (matrix plane). The considered prism (Littrow prism) is in the 12A and 12B With 20 designated. 11 also shows the shaded area 31 and the end parts of optical fibers 14 and 18 which are fitted into holes in the optical window 12 are inserted.

13 shows a perspective view of the prism 30 , The prism 30 is made of glass or transparent plastic with its surface BC covered by a single-layer or multi-layer coating to obtain a surface which reflects toward the inside of the prism and absorbs or reflects toward the outside. The triangular surfaces ABD of the prism 30 should be covered with a monolayer or multilayer coating to obtain a surface which absorbs toward the inside of the prism and absorbs or reflects toward the outside.

The prism 30 should be like in 12 be aligned with its surface AD parallel to the sensor plane; furthermore, the surface CD should not rest on any optical element for reasons which will become apparent hereinafter. In front of the area AC of the prism 30 should the imaging lens 15 which should have such a back focal length to focus beyond the prism. The light that comes out of the lens hits the surface AC and enters the prism 30 ; if the angle of inclination to AC is less than a critical angle (which for a prism-air passage is 27.9 ° with a refractive index of the prism of 1.5), light is reflected entirely on the surface CD; this results in the need for air as an optical medium outside the area CD. The bundle may come out of the prism 30 via CD towards the sensor.

For the optical isolation of the bundle coming out of the prism 30 should have a rectangular opening with the same surface as the section of the radiation beam coming out of the prism 30 or a larger surface in the optical protection window 12 be made. The inner wall of the opening should be covered with an absorbent coating to avoid interference with the signal from a front monitoring function. If necessary, all interior walls can be covered with the coating in a case of interference with the signals from the optical fibers.

These Type of optical system (lens plus prism) can also work together with the lens for a front-side monitoring function be used. The fact that the two optical systems not having the same axis is beneficial in that respect as the size affected is.

Claims (26)

Optical system using a CCD or CMOS matrix ( 11 ) having a sensitive area and a plurality of optical devices ( 13 . 14 . 15 . 18 ) with different directions and / or fields of view and / or modes of optical separation, characterized in that the sensitive area of the matrix ( 11 ) is divided into a plurality of separate sub-areas developed for different specific functions, wherein a part of the sub-areas is intended for scene monitoring and a part of the sub-areas is intended for detection of environmental parameters, the division by the plurality of optical devices is reached. System according to claim 1, characterized in that it in the front part of an inner rearview mirror of a motor vehicle is installed in the direction of travel to perform functions among the following: one Rain detection, windshield fogging detection, fog detection, a twilight detection, a tunnel detection, a detection of a meeting on a vehicle, a surveillance the scene in front of the vehicle, such as a lane warning, an adaptive low beam, a meeting on a vehicle. System according to claim 1, characterized in that the matrix is a linear or logarithmic, monochromatic or colored VGA CMOS matrix is. System according to claim 1, characterized in that the matrix changes its sensitive area into specific subregions has split that for one Front monitoring function such as lane warning, passive fog detection, a twilight detection, a tunnel detection and active fog detection. System according to claim 4, characterized in that the sensitive area of the matrix also has a specific sub-area for one Rain and nebulization detection or detection of rain and has fine rain. System according to claim 5, characterized in that the sensitive area of the matrix continues to have an extra specific sub-area for has a detection of a meeting on a vehicle. System according to claim 6, characterized in that it for a sub-range is provided for active rain detection is determined, by means of an emitter. System according to claim 7, characterized in that the area that for a rain function is determined, even for an active windshield fogging function is determined. System according to claim 8, characterized in that a twilight function is performed by a specific subrange of the CMOS matrix. System according to claim 9, characterized in that that a tunnel function by using part of the area carried out that will be for one Front monitoring function is determined. System according to claim 10, characterized that a fog function with both a certain sub-area, an active technique for one Local fog detection using an emitter is used as well with a passive technique for one Fog bank detection in another sub-area corresponding to that the for a front side monitor determined or contained therein is performed. System according to claim 11, characterized in that that a function for a meeting on a vehicle using two distinct subregions, or a subsection that is responsible for a Front monitoring is determined, in a colored matrix or in a monochromatic matrix by means of an optical filter that has a discretization level on the pixel level is arranged in the subarea of the matrix, which is for a Vorderseitenüberwachung is determined, performed becomes. System according to claim 1, characterized that the matrix sensor is made of glass or transparent plastic Protective window has, which also serves as a support for one or more optical Fibers works. System according to claim 13, characterized that the optical fibers have proximal ends fitted in holes are made in the protective window. System according to claim 13, characterized that there is a device for optical isolation between the area, the for a front side monitor and those responsible for rain, fog, fog and and twilight functions are determined based on a partial coverage of the surface of the Matrix protection window on the side towards the matrix with a layer of absorbent or reflective material. System according to claim 13, characterized that there is a device for optical isolation of the area, the for one Rain function is determined by having the influence of other functions, where the isolation is based on: 1) partial coverage of prism faces with a layer of absorbent or reflective material and 2) a hole made in the optical window, and covering or coating the inner walls of the hole. System according to claim 13, characterized that subsection that is for a rain function is determined, the optical signal from an optical System receives, in a row a prism with optical isolation, a filter and a lens having an optical axis orthogonal to the windshield having. System according to claim 13, characterized that subsection that is for a windshield fogging function is determined, the optical Signal from an optical system that receives a prism with optical Isolation, a filter and a lens with an optical axis orthogonal to the windshield. System according to claim 13, characterized that subsection that is for a twilight function is determined, the optical signal is received by an optical fiber. A system according to claim 13, characterized in that the sub-area intended for a tunneling function receives the optical signal through a lens which is also suitable for a front side monitoring function is determined. System according to claim 13, characterized that subsection that is for a fog function is determined based on an active technique the optical signal is received by an optical system, the a ball lens or a grin lens or no lens together with one end of an optical fiber on the output side having. System according to claim 13, characterized that subsection that is for a fog function is determined based on a passive technique the optical signal is received by a lens, which is also for a Vorderseitenüberwachungsfunktion is determined. System according to claim 13, characterized that the two subsections that make up a function of a meeting are determined by vehicles, the optical signal through filters together received with a lens. System according to claim 13, characterized that at the variant of a function of a meeting of vehicles based on the application of a subsection, which is responsible for a front page monitoring in a colored matrix or in a monochromatic matrix the optical signal is collected by the same lens, that for one Front monitoring function is determined. System according to claim 13, characterized that subsection that is for a front-side monitoring function is determined, the optical signal through a lens with an optical axis, which is shifted with respect to a matrix center. System according to claim 1, characterized that some subareas for unused pixels are reversed as an additional separation between used sub-areas are necessary.