CN1798958A

CN1798958A - Stereo optical module and stereo camera

Info

Publication number: CN1798958A
Application number: CN 200480014907
Authority: CN
Inventors: 三由贵史; 小坂明生; 岩城秀和; 荒井和彦
Original assignee: Olympus Corp
Current assignee: Olympus Corp
Priority date: 2003-05-29
Filing date: 2004-05-28
Publication date: 2006-07-05
Anticipated expiration: 2024-05-28
Also published as: CN100554878C

Abstract

Constituting members of a stereo optical module are disposed in a space region of at least either a circuit disposition space (27a) located at a position substantially behind a front lens unit (20a), which position is above a primary deflection mirror (21a) and a secondary deflection mirror (22a), or a circuit disposition space (27b) located at a position substantially behind a front lens unit (20b), which position is below a primary deflection mirror (21b) and a secondary deflection mirror (22b).

Description

Stereo optical module and stereo camera

Technical field

The present invention relates in a kind of moving object that can be installed in such as vehicle, boats and ships, aircraft or robot, or be installed in stereo optical module and stereo camera in the non-moving object such as the rig camera system.

Background technology

Routinely, the stereo optical module that employing links to each other with the imaging optical system of imaging device receives the light from same object at two-phase position spaced place, and each light is imported in the optical system of imaging device, make it possible to carry out the imaging (stereoscopic shooting) of stereo-picture, and proposed this stereo optical module of various structures.

For example, a kind of stereo optical module has been proposed, it has graphic pattern projection portion, predetermined pattern is projected to (for example referring to the flat 2000-236332 communique of Japanese Patent Application Laid-Open) on the object, perhaps proposed a kind of stereo optical module, its center pillar lens or wide-angle transform lens are removably disposed in (for example referring to the flat 9-327042 communique of Japanese Patent Application Laid-Open) on its front surface.In addition, a kind of endoscope that utilizes the image guiding to obtain binocular vision (for example referring to the flat 4-016812 communique of Japanese Patent Application Laid-Open) has been proposed.In addition, open the mirror structure that discloses a kind of stereo optical module in the clear 53-112426 communique for example Japanese utility model application spy.In addition, a kind of three order stereographic adaptor (for example referring to Japanese Patent Application Laid-Open 2001-305682 communique) have been proposed.

In addition, in traditional Raindrop sensor that is proposed by Denso company, for example shown in Figure 26, light is projected to the glass blocks 74 near glass pane 73 from the diode (light projection LED) 75 of emission infrared light or visible light.This light is assembled by light projection lens 76, and by adopting catoptron 77 etc., with the reflected light lead-in light receiver lens 78 of this light of being assembled on glass outer surface.Then, assemble this lead-in light, and detect this light through assembling by optical receiving sensor 79 such as photodiode by optical receiver lens 78.At this moment, when raindrop have been attached on the glass pane 73, change, therefore can measure the raindrop amount thereby light is scattered light quantity.

Yet, in above-mentioned communique (being that the flat 2000-236332 communique of Japanese Patent Application Laid-Open is to Japanese Patent Application Laid-Open 2001-305682 communique), in order to take the wide angle picture of distant objects, device for projecting or mirror size can become bigger, and are difficult to constitute small and light stereo optical module.

In addition, in traditional Raindrop sensor, Raindrop sensor must be placed near glass surface, and this causes on space or the cost limited.And then, during barrier in Raindrop sensor is the driver visual field or when the stereo optical module that detects the barrier that is positioned at the vehicle front also is installed, there is the problem in the visual field of blocking stereo optical module or Raindrop sensor problem to the installation site of stereo optical module except Raindrop sensor.

Summary of the invention

The purpose of this invention is to provide a kind of stereo optical module, it is small and light, and can take distant objects with wide visual angle.

In addition, another object of the present invention provides a kind of stereo camera with the Raindrop sensor that needn't consider its installation site etc.

In order to realize this purpose, comprise according to the stereo optical module of first aspect present invention:

First light guide section is used to generate first subject image of seeing from first viewpoint of relative object;

Second light guide section is used to generate second subject image of seeing from second viewpoint of described first viewpoint of being different from of relative object;

The first polarisation mechanism, on the first half surface portions, form the whole or most image of described first subject image, described first subject image is to be obtained by the light beam that enters from described first light guide section, described the first half surface portions are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first light guide section and described second light guide section; And

The second polarisation mechanism, on the second half surface portions, form the whole or most image of described second subject image, described second subject image is to be obtained by the light beam that enters from described second light guide section, described the second half surface elements are positioned on the second direction relative with described first direction, be described datum line between described first direction and the described second direction

Wherein a member places at least one of first space region and second space region, described first space region is positioned at the described first light guide section back and substantially on the incident direction to the described first polarisation mechanism perpendicular to described datum line and described light beam, and along this direction, the described second polarisation mechanism departs from the described first polarisation mechanism, described second space region is positioned at the described second light guide section back and substantially on the incident direction to the described second polarisation mechanism perpendicular to described datum line and described light beam, and along this direction, the described first polarisation mechanism departs from the described second polarisation mechanism.

In addition, to achieve these goals, comprise according to the stereo optical module of second aspect present invention:

The first polarisation mechanism, on the image-forming component that is positioned at the first half surface portions, form the whole or most image of described first subject image, described first subject image is to be obtained by the light beam that enters from described first light guide section, described the first half surface portions are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first light guide section and described second light guide section;

The second polarisation mechanism, on the image-forming component that is positioned at the second half surface portions, form the whole or most image of described second subject image, described second subject image is to be obtained by the light beam that enters from described second light guide section, described the second half surface elements are positioned on the second direction relative with described first direction, are described datum line between described first direction and the described second direction; And

Package is protected the mechanism in the described stereo optical module,

Wherein a member places at least one of first space region and second space region, described first space region is substantially in the described first light guide section back and be clipped between described first polarisation mechanism and the described package, and described second space region is substantially in the described second light guide section back and be clipped between described second polarisation mechanism and the described package.

In addition, to achieve these goals, comprise according to the stereo optical module of third aspect present invention:

Package is protected the mechanism in the stereo optical module from the outside,

Wherein a member places at least one of first space region and second space region, described first space region is positioned at the described first light guide section back substantially and is surrounded by the described first polarisation mechanism, the described second polarisation mechanism and described package, and described second space region is positioned at the described second light guide section back substantially and is surrounded by the described first polarisation mechanism, the described second polarisation mechanism and described package.

Further, in order to realize above-mentioned purpose of the present invention, comprise according to the stereo optical module of fourth aspect present invention:

First optical lens is used to generate first subject image of seeing from first viewpoint of relative object;

Second optical lens is used to generate second subject image of seeing from second viewpoint of described first viewpoint of being different from of relative object;

The first polarisation mechanism, comprise first reflecting element of the light that reflection enters from described first optical lens and second reflecting element that the light by described first reflecting element reflection is reflected again, on the image-forming component that is positioned at the first half surface elements, to form the whole or most image of described first subject image, described the first half surface elements are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first optical lens and described second optical lens;

The second polarisation mechanism, comprise the 3rd reflecting element of the light that reflection enters from described second optical lens and the 4th reflecting element that the light by described the 3rd reflecting element reflection is reflected again, on the image-forming component that is positioned at the second half surface portions, to form the whole or most image of described second subject image, described the second half surface portions are positioned on the second direction relative with described first direction, are described datum line between described first direction and the described second direction; And

The 3rd optical lens forms the image by the light of described second reflecting element and the reflection of described the 4th reflecting element on described image-forming component,

Wherein a member places at least one of first space region and second space region, described first space region is present on the described first direction of described relatively first reflecting element or described second reflecting element, and described second space region is present on the described second direction of described relatively the 3rd reflecting element or described the 4th reflecting element.

In addition, to achieve these goals, comprise according to the stereo optical module of fifth aspect present invention:

First shading piece is positioned near described first optical lens, and prevents to transmit the segment beam that enters described first optical lens;

Second shading piece is positioned near described second optical lens, and prevents to transmit the segment beam that enters described second optical lens;

The first polarisation mechanism, comprise first reflecting element of the light that reflection enters from described first optical lens and second reflecting element that the light by described first reflecting element reflection is reflected again, on the image-forming component that is positioned at the first half surface portions, to form the whole or most image of described first subject image that enters described first optical lens, described the first half surface portions are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first optical lens and described second optical lens;

The second polarisation mechanism, comprise the 3rd reflecting element of the light that reflection enters from described second optical lens and the 4th reflecting element that the light by described the 3rd reflecting element reflection is reflected again, on the image-forming component that is positioned at the second half surface portions, to form the whole or most image of described second subject image that has entered described second optical lens, described the second half surface portions are positioned on the second direction relative with described first direction, are described datum line between described first direction and the described second direction; And

Wherein a member places at least one of first space region and second space region, described first regional location is in the described first shading piece back and on the described first direction of described relatively first reflecting element or described second reflecting element, and described second regional location is in the described second shading piece back and on the described second direction of described relatively the 3rd reflecting element or described the 4th reflecting element.

According to this first aspect to the five aspects, owing to the member of on the vacant space of described stereo optical module, arranging such as image-forming component, thereby can the shooting of wide visual angle be positioned at distant objects, and and then can reduce size and weight.

And, to achieve these goals, placing in the vehicle and comprise according to sixth aspect present invention by the stereo camera that glass pane is observed the place ahead:

Stereo optical system is used to generate the subject image of seeing from a plurality of viewpoints of same relatively object;

Image-forming component, the described subject image of seeing from described a plurality of viewpoints based on being produced by described stereo optical system produces stereoscopic image data; And

The water droplet checkout equipment based on the described stereoscopic image data that is produced by described image-forming component, detects the water droplet amount attached to described glass surface.

Further, to achieve these goals, placing vehicle and comprise according to seventh aspect present invention by the stereo camera that glass pane is observed the place ahead:

The attachment judgment device carry out distance calculation according to each view data from the described stereoscopic image data of described image-forming component output, and there is attachment in judgement on described glass surface when the distance of target object falls in the preset range.

In addition, to achieve these goals, placing vehicle and comprise according to eighth aspect present invention by the stereo camera that glass pane is observed the place ahead:

Image-forming component, the described subject image of seeing from described a plurality of viewpoints based on being produced by described stereo optical system produces stereoscopic image data;

Image evaluation device, have the first image evaluation sequence and the second image evaluation sequence to assess based on the described stereoscopic image data carries out image of exporting from described image-forming component, the water droplet of the described first image evaluation Sequence Detection on described glass surface to be carrying out image evaluation, the described second image evaluation Sequence Detection at the object of described glass pane front to carry out image evaluation; And

Switching device shifter is used to switch the described first image evaluation sequence of described image evaluation device and the described second image evaluation sequence to operate.

Further, to achieve these goals, placing vehicle and comprise according to ninth aspect present invention by the stereo camera that glass pane is observed the place ahead:

Image-forming component, the described subject image of seeing from described a plurality of viewpoints based on being produced by described stereo optical system produces view data; And

Image evaluation device, has the first image evaluation/calculating part and second image evaluation/calculating part to assess based on the described stereoscopic image data carries out image of exporting from described image-forming component, described first image evaluation/calculating part detects water droplet on described glass surface carrying out image evaluation, and described second image evaluation/calculating part detects object in described glass pane front to carry out image evaluation.

In addition, in order to achieve the above object, placing vehicle and comprise according to tenth aspect present invention by the stereo camera that glass pane is observed the place ahead:

Luminaire, the described glass surface in the zone that overlaps with first visual field of first viewpoint in a plurality of viewpoints of the described stereo optical system of rayed and described glass pane; And

And, in order to achieve the above object, the tenth the placing vehicle and comprise on the one hand according to the present invention by the stereo camera that glass pane is observed the place ahead:

The moisture content checkout equipment, in the described stereoscopic image data of described image-forming component output, be arranged on the surface of the described glass pane in first visual field of first viewpoint in a plurality of viewpoints of described stereo optical system and the zone that described glass pane overlaps, and when based on being adhered to color information in the corresponding view data of material that moisture content changes output signal when detecting pre-color information with color with what detect.

According to the 6th to the tenth one side, because detect raindrop by Flame Image Process, thus do not need sensor special, thus the space saved.

Description of drawings

Fig. 1 shows the block diagram according to the basic structure of the stereo camera system of first embodiment of the invention;

Fig. 2 shows the block diagram of the circuit structure of the stereo optical module among first embodiment;

Fig. 3 A shows the vertical view of the structure of the stereo optical system in the stereo optical module;

Fig. 3 B is the outside drawing of stereo optical module;

Fig. 4 A shows the cut-open view of the structure of stereo optical module;

Fig. 4 B shows the vertical view of the circuit arrangement in stereo optical module;

Fig. 5 shows the view of the example of stereo-picture;

Fig. 6 shows the block diagram of the circuit structure of the central control unit in the stereo optical module in first embodiment of the invention;

Fig. 7 shows the view that is installed in the example on the vehicle according to the stereo camera system of first embodiment;

Fig. 8 A shows the view of first modification of visual field shadow shield;

Fig. 8 B is the view that second modification of visual field shadow shield is shown;

Fig. 9 shows the block diagram of the modification of central control unit;

Figure 10 shows the block diagram of the circuit structure of the stereo optical module in the second embodiment of the present invention;

Figure 11 shows the block diagram according to the circuit structure of the central control unit of second embodiment of the invention;

Figure 12 shows the block diagram of the circuit structure of the stereo optical module in third embodiment of the invention;

Figure 13 shows the view of Content of Communication example of the video format of employing such as DV form or NTSC standard;

Figure 14 is the process flow diagram of the main operation sequence in the stereo camera in fourth embodiment of the invention;

Figure 15 shows the process flow diagram of sensor checking sequence;

Figure 16 is the process flow diagram of calibration data correction sequence;

Figure 17 drives the process flow diagram of supporting sequence;

Figure 18 shows the view of the demonstration example of separation between vehicles warning;

Figure 19 is the demonstration example of road surface recognition result;

Figure 20 is a demonstration example of hiding warning;

Figure 21 is lane identification result's a demonstration example;

Figure 22 is the process flow diagram of modification that the process flow diagram of main operation sequence is shown;

Figure 23 is the process flow diagram that first modification of sensor checking sequence is shown;

Figure 24 shows the process flow diagram of second modification of sensor checking sequence;

Figure 25 shows the block diagram of the structure of the stereo camera system in fifth embodiment of the invention;

Figure 26 is the block diagram of traditional Raindrop sensor;

Figure 27 shows the view as the stereo-picture example of sixth embodiment of the invention when detecting raindrop;

Figure 28 is the view that crossover region is shown;

Figure 29 is the view that schematically shows traditional corresponding point retrieval;

Figure 30 is the view that schematically shows the retrieval of the corresponding point when detecting raindrop in the 6th embodiment;

Figure 31 show when raindrop attached to the zone beyond the crossover region in the time the view of stereo-picture example;

Figure 32 shows the view that detects the technology of raindrop in crossover region zone in addition;

Figure 33 shows the view that adopts the light beam projectoscope that raindrop are detected; And

Figure 34 illustrates the view that raindrop detect the example of coating.

Embodiment

Below with reference to accompanying drawings to describing according to embodiments of the invention.

[first embodiment]

Fig. 1 shows the block diagram according to the basic structure of the stereo camera system of first embodiment of the invention.Here, this stereo camera system is assumed to be the system in the moving object that can be installed in such as vehicle, boats and ships, aircraft or robot.Describe with reference to Fig. 1 to 9 pair of example that is installed in this system in the vehicle below.

That is, as shown in Figure 1, this camera chain is made of stereo optical module 1, communication line 2 and central control unit 3 basically.1 pair of object of stereo optical module carries out imaging with the acquisition stereo-picture, and the stereo-picture that is obtained is proofreaied and correct.Communication line 2 is the communication lines between stereo optical module 1 and the central control unit 3.Central control unit 3 is based on from the stereo-picture of stereo optical module 1 by communication line 2 input the object as the photographic subjects of stereo optical module 1 being assessed.Particularly, central control unit 3 obtains the range image of description after a while based on the stereo-picture of input, and the range image that this obtained is assessed.

To describe these structures in detail below.

Stereo optical module 1 at first is described.Fig. 2 is the block diagram that shows in detail this stereo optical module inside.Promptly, as shown in Figure 2, stereo optical module 1 is by stereo optical system 4, image-forming component 5, image-forming component driving circuit 6, control information storer 7, image output processing circuit 8, module (M) side communication interface 9, constitute as the Raindrop sensor 10 and the temperature sensor 11 of water droplet checkout equipment or attachment checkout equipment.

As shown in Figure 3A, stereo optical system 4 comprises right visual field shadow shield 19a and left visual field shadow shield 19b, right front lens unit 20a and left front lens unit 20b, as the right elementary polariscope 21a of first and second reflecting elements and left elementary polariscope 21b, as right secondary polariscope 22a and left secondary polariscope 22b, rear lens unit 23 and the low-pass filter 24 of third and fourth reflecting element, stereo optical system 4 forms image according to the unshowned object on image-forming component 5.

Here, shown in Fig. 3 B, about the outward appearance of stereo optical module 1, stereo optical module 1 is covered by the package 26 that is formed with visual field shadow shield opening 25a and 25b on it, and light enters stereo optical system 4 from these visuals field shadow shield opening 25a and 25b.Here, package 26 is assemblies of protecting the interior mechanism of stereo optical module from the outside, and it is the covering with the function of blocking mechanism in the external beam function, dust reduction capability, support etc.

Further, shown in Fig. 3 A and 3B, for send from the not shown object that is positioned at fixed range, enter the light beam of stereo optical system 4 from visual field shadow shield opening 25a and 25b, prevent that by visual

field shadow shield

19a and 19b this light beam of part from seeing through, and the residue light beam enters right front lens unit 20a and left front lens unit 20b.Here, visual

field shadow shield

19a and 19b are as the aperture diaphragm in the visual field that dwindles each front lens unit 20a and 20b.In first embodiment, shown in Fig. 4 A, visual field shadow shield 19a blocks the upper half of front lens unit 20a, and visual field shadow shield 19b blocks the bottom half of front lens unit 20b.

The effective optical axis (following will be called optical axis) that should be noted that each front lens unit 20a with visual

field shadow shield

19a and 19b and 20b with light beam central authorities (i.e. central optical axis when not being provided with visual

field shadow shield

19a and 19b) coupling from object.In addition, shown in Fig. 4 A, the optical axis of right front lens unit 20a and left front lens unit 20b not at grade, they are bent change (twisted).

Reflect by

elementary polariscope

21a and 21b from light beam object, that entered by front lens unit 20a and 20b.Each

elementary polariscope

21a and 21b have can each front lens unit 20a of reflecting ﹠ transmitting and the size of the light beam of 20b.That is, each

elementary polariscope

21a and 21b have the size that equals substantially or be slightly larger than the zone that is not blocked of front lens unit 20a.In addition, each elementary polariscope 21a becomes ground, several years angle to tilt to place with miter angle and at vertical direction towards the image-forming component side with 21b in the horizontal direction.By elementary polariscope 21a of placement like this and 21b, enter each

secondary polariscope

22a and 22b by each

elementary polariscope

21a and 21b beam reflected.

Secondary polariscope 22a is basically perpendicular to elementary polariscope 21a in the horizontal direction with 22b and becomes ground, several years angle to place with 21b inclination placement and at vertical direction towards the image-forming component side.By secondary polariscope 22a of placement like this and 22b, enter rear lens unit 23 by each

secondary polariscope

22a and 22b beam reflected.Shown in Fig. 4 B, as from top finding, secondary polariscope 22a and the 22b placement that is perpendicular to one another.That is to say, reflect by secondary polariscope 22a from the incident beam of elementary polariscope 21a, and enter along the following direction deflection of rear lens unit 23.On the other hand, reflect by secondary polariscope 22b from the incident beam of elementary polariscope 21b, and enter along the last direction deflection of rear lens unit 23.

Here, the first polarisation mechanism comprises elementary polariscope 21a and secondary polariscope 22a, it has the function that the light beam that will enter front lens unit 20a guides to the first half surface portions, the first half surface portions the bottom half of pixel spare 5, this bottom half is positioned on the first direction perpendicular to datum line and light beam incident direction, and datum line connects the middle body of front lens unit 20a and the middle body of front lens unit 20b.

And, the second polarisation mechanism comprises elementary polariscope 21b and secondary polariscope 22b, it has the function that the light beam that will enter front lens unit 20b guides to the second half surface portions, the second half surface portions the upper half of pixel spare 5, this upper half is positioned on the second direction relative with first direction, is datum line between first direction and the second direction.

So light beams deflected enters low-pass filter 24 by rear lens unit 23, and removes its high frequency noise components by low-pass filter.Then, the light beam that enters by right front lens unit 20a is imaged on the bottom half of image-forming component 5, and the light beam that enters by left front lens unit 20b is imaged on the upper half of image-forming component 5.That is to say by visual

field shadow shield

19a and 19b being limited in of the visual field not had the ground of overlapping on the image-forming component and form epigraph and hypograph.So this stereo-picture with side-looking difference as shown in Figure 5 is aligned and is formed on the vertical direction of image-forming component.So, can obtain stereo-picture by stereo optical module of independent employing.

Here, because the placement of ground, each secondary polariscope 22a angle of relative 5 one-tenth several years of image-forming component with the 22b in the vertical direction, so each image is formed with object space.Therefore, in order on image-forming component 5, to be formed into the image of irradiating light beam effectively, preferably shown in Fig. 4 A, tilt to place image-forming component 5.

By the function of the image-forming component driving circuit 6 shown in the block diagram of Fig. 2, the stereo-picture that so is formed on the image-forming component 5 is exported to image output processing circuit 8 in proper order as simulating signal.Here, the inside of image output processing circuit 8 comprises Analog to Digital Converter section 12, C-Y converter section 13, compression handling part 14, revises (rectification) handling part 15, γ conversion process portion 16, shade (shade) treatment for correcting portion 17 and frame buffer memory 18.

Further, shown in Fig. 4 A and 4B, image output processing circuit 8, Raindrop sensor 10 and temperature sensor 11 are placed among the circuit arrangement space 27a and 27b as vacant space, promptly in first space region and second space region of the rear surface side of the secondary polariscope 22a of visual field shadow shield 19a and 19b back and 22b.Shown in Fig. 4 A and 4B, in right optical system, circuit arrangement space 27a is positioned at elementary polariscope 21a and above the secondary polariscope 22a, and in left optical system, circuit arrangement space 27b is positioned at below elementary polariscope 21b and the secondary polariscope 22b.As mentioned above, because the first half of front lens unit, right side 20a is blocked by visual field shadow shield 19a, and the Lower Half of front lens unit, left side 20b is blocked by visual field shadow shield 19b, so optical element can not be placed on these visuals field shadow shield 19a and 19b back, thereby vacant space is provided.Therefore, when circuit (for example image output processing circuit 8 and other circuit) when being arranged in such vacant space, can effectively utilize the vacant space in the stereo optical module, and can not hinder the incident optical transmission, this has reduced the stereo optical module size.Here, can be arranged on circuit in these vacant spaces and for example comprise the electric signal output circuit of exporting electric signal and image-forming component driving circuit, peripheral circuit, temperature sensor, Raindrop sensor, illuminance transducer, gps antenna, ETC card system and other circuit such as the number storage (control information storer) of storage correction data, and above-mentioned image output processing circuit is also included within these circuit.And, the power circuit or the mechanical parts of unshowned stereo optical module can be set in this vacant space.As this mechanical parts, can provide for example unshowned adjusting mechanism that is used for mechanically adjusting stereo optical system.In addition, the unshowned ND filter configuration that adjustment can be entered the light quantity of stereo optical system is an insert type, and above-mentioned vacant space can be used as the retraction space of this ND wave filter.

Below with the each several part of the image output processing circuit 8 shown in the key diagram 2.In image output processing circuit 8, the stereo-picture of simulating signal is converted to the digital stereoscopic image data of numerical data by Analog to Digital Converter section 12.The front surface that it should be noted that image-forming component 5 is provided with unshowned optical colour filter, and by this optical colour filter, image-forming component 5 outputs are corresponding to the stereoscopic image data of each color R, G and B.In C-Y converter section 13, the elementary chrominance signal of R, G and B is converted into luminance signal and carrier chrominance signal.This is with the pre-service of these conversion of signals for motion JPEG (motion-JPEG) signal in compression handling part 14.Then, with these signal storage in frame buffer memory 18.

The stereoscopic image data that is stored in this frame buffer memory 18 is inputed to shadow correction handling part 17.Shadow correction handling part 17 carries out shadow correction based on the shadow correction data of storage in the control information storer 7 to be handled.Then, gamma transformation handling part 16 carries out gamma transformation to be handled, so that image I/O characteristic is suitable, and correcting process portion 15 carries out correcting process based on the calibration data that is stored in the control information storer 7.

It should be noted that correcting process is image to be handled the processing of the distortion that for example causes owing to following reason with correcting image: the right image that in stereo optical system 4, produces and the displacement of the nuclear line (epipolar line) of left image or the distortion of lens by the mechanical shift of optical element etc.In this processing, for example detect position corresponding to the point in the right image of the unique point of in left image, selecting, detect the displacement of distance one straight line (nuclear line) (these corresponding point should be positioned on this straight line in principle), and carry out anamorphose corresponding to this displacement after testing.With preferred coordinate right image and the left image (hereinafter referred to as correction image) that has passed through correcting process like this carried out normalization, then these images are write in the frame buffer memory 18 once more with the state of homeotropic alignment as shown in Figure 5.

In addition, for the calibration data that is stored in the control information storer 7, storage is used to consider many groups calibration data of the thermal deformation of stereo optical module etc.That is to say, to input to image output processing circuit 8 by M side communication interface 9 by temperature sensor 11 detected temperature values, and calibration data in the control information storer 7 is selected to be stored in carrying out correcting process based on this input temp by correcting process portion 15, thereby realizes not being subjected to the processing of influence of temperature change.

Then, the digital stereoscopic image data that write once more in the frame buffer memory 18 are inputed to compression handling part 14, by adopting it is carried out data compression, export it to communication line 2 by M side communication interface 9 then such as the data compression scheme of motion JPEG.In addition, Raindrop sensor 10 also is connected with M side communication interface 9 with temperature sensor 11, makes the output of these sensors can be output to order wire 2 or image output processing circuit 8.

The following describes communication line 2.In first embodiment, communication line 2 has a plurality of usb communication lines, by these usb communication line transmission correction image data and the required data-signal of control stereo optical module.These order wires are connected with the M side communication interface 9 of stereo optical module 1 and unit (u) the side communication interface 28 of central control unit 3 respectively.

Here, with motion jpeg format compression correction image, and send it to the usb signal line, also transmit information the environmental information (such as by Raindrop sensor 10 detected Raindrop sensor information or by temperature sensor 11 detected temperature sensor information or object luminosity information) for example by the usb communication line.In addition, by communication line 2 this information is sent to central control unit 3.Here, because compress, so the quantity of information can reduce communication time the and can increase the quantity of information of the data of sending outside the view data/receive by compression handling part 14 stereoscopic image data.Here, communication line 2 (being the usb communication line) is sent to central control unit 3 with above-mentioned digital data form from stereo optical module 1 with correction image data, Raindrop sensor information, temperature sensor information and other data.Here, stereo optical module 1 comprises that temperature sensor, Raindrop sensor, control information storer and other generation tentation datas are with the device of response from the control command of central control unit 3.Communication line 2 is bi-directional digital communication circuits, can by this communication line 2 various control commands be sent to stereo optical module 1 from central control unit 3.Although omitted the detailed description to control command in first embodiment, these control commands for example are: the power supply control command of stereo optical module 1 (being used for the order of switching operation modes and energy saver mode), be used for that temperature sensor information or Raindrop sensor information read out to the request signal of central control unit side or the information that is used for being stored in control information storer 7 reads out to the request signal of central control unit 3 sides.Thus, can carry out mutual data communication effectively by the communication line that adopts single system.

In addition, as the modification of first embodiment, can transmit the correction image data by analog communication line.For example, the output of image-forming component 5 is converted to the analog picture signal that meets such as the NTSC form, then it is exported to communication line 2 by the M side communication interface 9 on the module side.In this case, in stereo optical module, do not need to compress handling part 14.As the feature of this modification, communication line 2 must have two systems, i.e. analog communication line and digital line, and need exceptional space in the part of placing communication line.In addition, exist the shortcoming that for example needs the exceptional space that communications connector occupies, yet advantage is also arranged, promptly improved image data transmission speed.

In addition, when by analog communication line transmission correction image, can transmit the control data except that view data by the zone (superimposed in blank signal (blanking signal)) that employing is used for character broadcast in this zone information.

Below central control unit 3 will be described.As shown in Figure 6, central control unit 3 carries out the environment judging unit of object identification etc. and sequence controller (following will be called the environment judging unit) 33 by u side communication interface 28, image input device 29, frame memory 30, EPROM 31, range image computing unit 32 and based on range image.In addition, central control unit 3 also has image output device 34, points to input equipment 36, audio output apparatus 37, audio input device 40 and pilot lamp output device 41.Foregoing circuit or equipment are connected to each other by bus, and they can transmit data each other.

In addition, image output device 34 is connected with sensing input equipment 36 and demonstration input equipment 35.In addition, audio output apparatus 37 is connected with loudspeaker 38, and audio input device 40 is connected with microphone 39.In addition, pilot lamp output device 41 is connected with pilot lamp 42.

The correction image that inputs to central control unit 3 by communication line 2 is temporarily stored in the frame memory 30 via u side communication interface 28 and image input device 29, is input to range image computing unit 32 then.

Here, be according to the data of motion Joint Photographic Experts Group compression owing to input to the correction image of central control unit 3, carry out decompress(ion) and recovery by 29 pairs of these packed datas of image input device.

Range image computing unit 32 has stereo-picture montage portion 43 (it is right image of montage and left image from stereo-picture) and form matching part 46.That is, the correction image that inputs to range image computing unit 32 by 43 pairs in stereo-picture montage portion is carried out montage so that left image 44 and right image 45 to be provided.In addition, in form matching part 46, to carrying out the form coupling, and detect the displacement of each corresponding point through the left image 44 of montage and right image 45, i.e. parallax amount, thus produce anaglyph 47.Then, in range image generating unit 48, this parallax amount is converted to range information, and generates range images based on this range information and the calibration data 49 that is stored among the EPROM 31.Here, " range image " speech refers to have the image at the range information of each pixel of the subject image of being caught.In addition, the range information correction data of this calibration data 49 for selecting according to raindrop information of importing by communication line 2 or temperature sensor information.

Afterwards, the range image that is generated is sent to environment judging unit 33.

Environment judging unit 33 is similar to the opertaing device or the microcomputer of general purpose personal computer, have CPU (central processing unit) (CPU) 50, primary memory (RAM) 51, digital signal processing unit (DSP) 52, hard disk (HDD) 53 and other equipment of linking to each other with communication bus, and environment judging unit 33 is configured to realize assessing the sequence of range image and controls with form of software recognition object or other various sequences.

Promptly, in the time of in range image input environment judging unit 33, CPU 50 launches and carries out the object identification handling procedure that is stored among the HDD 53 in RAM 51, and detects the processing of barrier (for example pedestrian on the road, vehicle, junk etc.) or recognition feature environmental information (for example white line on the road, guard rail, sign etc.) based on the range image of two-dimentional correction image that obtains in the stereo optical module 1 or acquisition in range image computing unit 32.And, CPU 50 for example shows that recognition result, control audio output device 37 send beep sound from loudspeaker 38 and notify the close or control pilot lamp output device 41 of vehicle to open pilot lamp 42 by adopting by control image output device 34 showing based on this identification result on the input equipment 35, what cause that operator (for example driver) notes supports relevant control with driving.In addition, relatively, under having from the situation of operator by the order of manual operation part (be similar to and show input equipment 35) or voice-input device (being similar to microphone 39), obtain this order by pointing to input equipment 36 or audio input device 40, and carry out control corresponding to this command context.

As shown in Figure 7, when above-mentioned stereo camera system is installed in the vehicle, be connected to having of communication line 2: radar module 54 is equipped with radar equipment on it; Vehicle speed sensor module 55, it detects car speed; Auto navigation module 56 is notified to driver etc. with vehicle location; Engine control module 57, its Control Engine; Side camara module 58, its monitoring vehicle both sides; Interior monitoring module 59, its monitoring vehicle inside; Braking sensor module 61, its control brake device; Back camara module 62, the monitoring vehicle rear side; Load detecting module 63 detects the charge capacity in the vehicle; Back ultrasonic radar module 64 is equipped with ultrasonic radar equipment on it; With other modules, and stereo optical module 1 and central control unit 3, thereby various vehicle-mounted module can communicate with one another.Should be noted that conventional module can be used as above-mentioned module.

For as the unit of a this in-vehicle network part or the communication between the module, for example can adopt bluetooth, IEEE 802.11 or IEEE 802.1x as wireless communication technology.And, can also adopt cable network such as USB, IEEE 1394 or Ethernet, host-host protocol, real-time messages function (RIC/SIP user) and other such as the voice data of IP-based voice technology (VoIP), and can make up LAN (Local Area Network) in the car, wherein give each equipment with IP address assignment by IPv6 etc.In addition, can also adopt the communication bus that meets such as the open standard standard of optical network standards in the vehicle " MOST " or " CAN " (not being subjected to the influence of internal car noise etc.).

As mentioned above, this first embodiment has following certain effects.

That is, in this first embodiment, the desired Flame Image Process of stereo optical module (such as correction, γ correction, shadow correction or be separated into luminance signal and colour difference signal etc.) is carried out in stereo optical module.As a result, reduce the circuit load of central control unit side, and can simplify the specification of central control unit side, thereby improved the versatility of this system.

Further,, Digital Image Data is transmitted owing to being compressed then, and can be to come the transmission of digital view data near the frame rate under the analogue communication situation.

In addition, owing in stereo optical module 1, be provided with temperature sensor 11, so the temperature can be based on correcting process the time is suitably carried out correction.Therefore, it is just enough good that the region of search of the form that carries out in the central control unit 3 coupling is restricted to the nuclear line, thereby realized high speed processing.

It should be noted that certainly and each structure among this first embodiment is carried out variants and modifications in a lot of modes.

For example, can adopt a plurality of above-mentioned stereo optical modules 1.In this case, video camera ID can be added from the correction image that each stereo optical module obtains, make that can clearly discern which stereo optical module has corresponding correction image.

In addition, although comprise the treatment circuit of the processing that execution such as γ conversion process, C-Y conversion process or compression are handled in the stereo optical module 1 in first embodiment, but these functions and nonessential, can carry out with generic video video camera or digital stillcamera in the treatment circuit of the processing that is equal to mutually of Flame Image Process just enough.

In addition, the control information storer 7 of stereo optical module 1 can be stored control information that correcting process uses or shadow correction information and how invest the control information that obtains on the vehicle by calibrating each stereo optical module.Thus, can remove the EPROM 31 in the central control unit, or reduce memory capacity.Further, the information from central control unit 3 that obtains by calibration can be sent to stereo optical module 1 etc., and this information can be stored in the stereo optical module 1.In addition, used calibration data also can be stored in the control information storer 7 of stereo optical module 1 in the range image computing unit 32 of central control unit 3, makes when range image calculates this calibration data to be sent to central control unit 3 from stereo optical module 1.

Promptly, in this modification, be stored in part control information (calibration data) in the control information storer 7 and be used for the Flame Image Process and the calculating of stereo optical module, and any other control information is sent to central control unit 3 by order wire 2, make it can be used for proofreading and correct, for example in the distance calculation of central control unit side.

And, in central control unit 3, can adopt application-specific IC (ASIC) to connect and realize various treatment circuits, and these circuit can be made of the reconfigurable processor that for example dynamically rewrites by the hardware lead.

In addition, central control unit 3 not only can be carried out the control to stereo camera system, and can information, control information or the montage district information from temperature sensor or Raindrop sensor be sent.Need not to point out that this not only can be applied to stereo optical module 1, communication line 2 and central control unit 3, also can be applied to the communication between each car inner module.

Further, each communication line 2, stereo optical module 1 and central control unit 3 can be electrically insulated from each other by photoelectrical coupler, thereby circuit is not subjected to the influence of various spike noises that act on lead etc.

In addition, the form as the video that transmits by communication line 2 except that the NTSC form, can also adopt PAL form or SECAM form, as long as it is a kind of analog format.And can by adopt various compressed formats (for example, be used for the compression scheme of rest image the JPEG scheme, be used for the MPEG scheme of the compression scheme of moving image or the digital video under the situation of digital format (DV) standard) transmit.

In addition, for input or demonstration, can adopt GUI or speech interface by the user interface design instrument that is used for in-vehicle device (such as the automobile UI kit of for example producing) configuration by company of Microsoft's (registered trademark).

In addition, although each visual

field shadow shield

19a and 19b have and can cover half shape of each

front lens unit

20a and 20b in Fig. 4 A, but the visual field shadow shield 19 with square aperture part 25 can be placed on the position of the optical axis that departs from front lens unit 20, the four limit D that perhaps excise lens itself come limits vision, thereby obtain to be equal to the effect when placing visual field shadow shield as illustrated in fig. 8, further to limit effectively.

And then, as shown in Figure 9, central control unit 3 can be constituted as and have stereo optical system correcting device 65 (calibration data 66 that it calculates stereo optical system) or Vehicular system correcting device 67 (it carries out Vehicular system calibration, position relation and computing system calibration data 68 between this calibration measurement vehicle and the stereo optical module 1).

In addition, although the control information of storage is to be used for that shadow correction is handled or the calibration data of correcting process in control information storer 7, certainly add the calibration data of the position relation of the information in image clips district or vehicle.In addition, control information storer 7 can be placed on central control unit 3 sides.Further, can adopt and make it possible to the structure that suitably reads and write by communication line 2.

[second embodiment]

Second embodiment according to stereo camera system of the present invention will be described below.It should be noted that with identical structure among the indication of similar label and first embodiment, thereby omit explanation it.

Promptly, in this second embodiment, as shown in figure 10, treatment circuit and control information storer 7 such as correcting process portion 15, γ conversion process portion 16, shadow correction handling part 17 and other circuit are not installed in the stereo optical module 1, but as shown in figure 11 they are installed in the central control unit 3.

As a result, can reduce the circuit scale of each circuit in the stereo optical module.That is, the heat that has suppressed to produce in stereo optical module can suppress because the calibration data fluctuation that the thermal deformation of the optical system in stereo optical module etc. causes.In addition, owing to image-forming component 5 can effectively cool off because of the minimizing of other heater elements, thus can suppress the generation of thermonoise, thus export better stereo-picture to communication line 2.

It should be noted that with first embodiment similarly, certainly multiple mode is carried out variants and modifications to each structure among this second embodiment.

[the 3rd embodiment]

The following describes the 3rd embodiment according to stereo camera system of the present invention.It should be noted that with identical structure among the indication of similar label and first and second embodiment, thereby omit explanation it.That is, as shown in Figure 12, by with the range image computing unit from shown in Fig. 6 according to removing the central control unit obtain in the 3rd embodiment the central control unit of first embodiment.

That is, in the 3rd embodiment, as shown in figure 12, in stereo optical module, carry out from beginning processing to three-dimensional matching treatment.Therefore, in stereo optical module 1, also comprise range image computing unit 32.In this case, export two dimensional image and the range image that is used for object identification to communication line 2.The difference of the 3rd embodiment and first embodiment is that view data is based on NTSC to send in the M side communication interface 9 of modeling scheme from stereo optical module.

As shown in figure 13, alignment and rearrange two dimensional image and range image in NTSC type picture for example, and it is exported as vision signal.So, can range image and two dimensional image be sent to central control unit 3 with synchronous regime.Further, when in the signaling zone that for example is being used for text broadcasting to which having indicated in right image and the left image be used and as the information of benchmark image output, carried out coding and when it is sent by temperature sensor 11 detected temperature or by Raindrop sensor 10 detected information, can carry out and the communicating by letter of image synchronization.

After so having carried out communication, central control unit 3 is decoded through image transmitted by adopting 28 pairs of u side communication interfaces, thereby makes these images be treated to two dimensional image, range image and sensor information, thereby realizes various identifications processing.Particularly, central control unit 3 is based on by the range image data of u side communication interface 28 inputs the not shown object in environment judging unit 33 being assessed.

It should be noted that, can adopt a plurality of stereo optical modules, and for example when adopting two stereo optical modules, can be with in each of epigraph and hypograph in the sign indicating number of the video camera ID of each three-dimensional module of expression and the range image adding picture, and these images can with the single audio video synchronization that will be sent to central control unit 3.

In addition, in central control unit 3, can connect by the hardware lead of special-purpose ASIC and realize various treatment circuits, and they can be made of the reconfigurable processor that can dynamically rewrite.

In addition,, be used for the two dimensional image and the range image of object identification, can export them to digital communication line as numerical data with first embodiment is similar by adopting the output of NTSC pattern analog signal line although in the 3rd embodiment.In this case, can come these images are carried out data compression process such as the data compression scheme of motion JPEG or MPEG by employing, and export treated image to communication line 2 similarly with first embodiment then.

When being installed in the vehicle as shown in Figure 7 particularly, stereo optical module 1, central control unit 3, mode of operation detection module and other modules can be connected with communication line 2, thereby make the work that can be fitted to each other of various vehicle inner modules.Here, as the mode of operation detection module, vehicle speed sensor module 55, engine control module 57, braking sensor module 61 and other modules shown in Fig. 7 are for example arranged.Further, as shown in Figure 7, radar module 54, auto navigation module 56, side camara module 58, interior monitoring module 59, back camara module 62, load detecting module 63, back ultrasonic radar module 64 and other modules as the environmental monitoring module are arranged.For as the unit of a this vehicle network part or the communication between the module, can adopt bluetooth, IEEE 802.11 or IEEE 802.1x etc. as wireless communication technology.And, can also adopt cable network, voice data transmission agreement, real-time messages function (RIC/SIP user) and other technology such as IP-based voice technology (VoIP) such as USB, IEEE 1394 or Ethernet, and can make up LAN (Local Area Network) in the car, wherein worker's P address is dispensed to each equipment by IPv6 etc.In addition, can also adopt corresponding to communication bus such as the open standard standard of optical network standards in the vehicle " MOST " or " CAN " (not being subjected to the influence of internal car noise etc.).

In addition, communication line 2, stereo optical module 1 and central control unit 3 can be electrically insulated from each other by photoelectrical coupler, thereby circuit is not subjected to the influence of various spike noises that put on lead etc.

Further, for input or demonstration, can adopt GUI or speech interface by the user interface design instrument that is used for in-vehicle device (such as the automobile UI kit of for example producing) configuration by company of Microsoft's (registered trademark).

Promptly, in the 3rd embodiment, not only the desired Flame Image Process of stereo optical module (for example to correction, γ correction, the shadow correction of the stereoscopic image data that obtains in stereo optical module or be separated into luminance signal and colour difference signal) is carried out in stereo optical module, and the calculating of range image is also carried out in stereo optical module.Therefore, compare, reduced the circuit load of central control unit side, and can further simplify the specification of central control unit side, thereby improved the versatility of central control unit with first embodiment.

[the 4th embodiment]

Below a fourth embodiment in accordance with the invention will be described.In the 4th embodiment, the explanation of the sequence of operation (carrying out this sequence calculates to carry out range image better) about stereo camera system will be provided.It should be noted that here with the trigger pip that is used for using in the vehicle as example.In addition, be the sequence of operation shown in the 4th embodiment according to the stereo camera system of first embodiment.Need not to point out that this sequence can be applied in the structure according to the second or the 3rd embodiment equally.

At first with reference to the main operation sequence of Figure 14 explanation in stereo camera system.That is, make door key be in open mode or with ignition switch (ignition key) when switching to out state, be in dormant central control unit 3 and recover (step S1) and be activated (step S2) when detecting by operator such as the driver.Activating simultaneously, central control unit 3 starts the image output processing circuit 8 and the image-forming component 5 (step S3 and S4) of stereo optical module 1.Here, will activate stereo optical module 1 desired " activation command " and be sent to stereo optical module 1 from central control unit 3 by communication line 2.Stereo optical module 1 switches to operator scheme with unshowned internal power circuit from energy saver mode in response to this " activation command ", and starts image output processing circuit 8 and image-forming component 5 subsequently.So, the beginning imaging.In addition, simultaneously, central control unit 3 starts the sensor checking sequence (step S5) shown in Figure 15.The details of this sensor checking sequence will be described after a while.

When exporting no problem environmental information data 100 to correcting process portion 15 as the result of sensor checking sequence, the beginning correcting process.That is, correcting process portion 15 is based on these environmental information data 100 or be stored in calibration data 101 in the control information storer 7 etc. and carry out correcting process (step S7).Further, the correction image that will carry out this correcting process by communication line 2 provides to central control unit 3.It should be noted that, at central control unit 3, if environment judging unit 33 calibration data 101 that inspection is used for this correcting process and is stored in frame memory 30 based on correction image whether suitable (step S8) for example is and this calibration data 101 existing problems then this correction image is deleted from frame memory 30 to prevent to use this image.In addition, carry out the calibration data treatment for correcting (step S9) that will illustrate after a while, and will re-writing in the control information storer 7 in the stereo optical module 1 by the corrected calibration data of this treatment for correcting, thereby the calibration data of updated stored 101 (step S10) by communication line 2.In addition, in the correcting process at step S7 place, adopt this calibration data 101 to carry out correcting process through upgrading.So, always carry out correcting process with the correct calibration data with up-to-date.

If judge that based on the correction image of having carried out correcting process this calibration data is no problem, then the range image computing unit 32 in the central control unit 3 carries out form matching treatment (step S12) based on the correction image of input, generate range image, and export the range image that this generated to environment judging unit 33 (step S13, the second image evaluation sequence).The driving auxiliary (step S14) that environment judging unit 33 in the central control unit 3 will be described after a while by adopting this range image to carry out.

Here, when door key was opened or ignition switch switched to off status, the judgement at step S15 place skipped to "Yes" to write down current calibration data (step S16), and central control unit 3 enters sleep state, and stereo optical module 1 also stops its operation.Here, " operation stop command " that will be used to stop the operation of stereo optical module 1 is sent to stereo optical module 1 by communication line 2 from central control unit 3.Stereo optical module 1 is carried out scheduled operation in response to this " operation stop command ", enters energy saver mode then.

Sensor checking sequence below with reference to Figure 15 explanation step S5 place in Figure 14.Here, it should be noted that will provide explanation hypothesis stereo optical module 1 invest on the automobile rearview mirror and imaging carried out in car body the place ahead by front screen.

In this sensor is checked, central control unit 3 at first drips (step S21) by gloomy sediment (dull deposit)/dew that known technology detects on the front screen, and judges whether gloomy sediment is arranged on the front screen of vehicle or reveal droplet (a step S22).If judge to have gloomy sediment in this judgement, then central control unit 3 is opened unshowned defroster and is removed this gloomy sediment (step S23), and is showing that carrying out gloomy sediment warning on the input equipment 35 shows (step S24).And when the effect by defroster removed gloomy sediment, central control unit 3 judged in the judgement of step S22 and do not have gloomy sediment, and closed gloomy sediment warning and show (step S25).

Then, central control unit 3 detects raindrop (step S26, the first image evaluation sequence) by adopting through communication line 2 from the output that Raindrop sensor 10 sends, and judges whether raindrop (step S27) are arranged on front screen.If judging in this judgement has raindrop, then the unshowned rain brush of central control unit 3 operations begins to remove raindrop (step S28), and is showing that the warning of raining on the input equipment 35 shows (step S29).In addition, when the effect by rain brush removed raindrop, central control unit 3 is judged in the judgement of step S27 did not have raindrop, and closed the warning demonstration (step S30) of raining.

Then, central control unit 3 detects the environment temperature (step S31) of stereo optical module 1 by adopting temperature sensor 11, and judges that the temperature that detected is whether in the scope that is suitable for stereo optical module 1 operation (step S32).If judge that in this judgement the temperature that is detected does not fall in the described proper range, then the unshowned air conditioner of central control unit 3 operations is regulated temperature (step S33), and is showing that input equipment 35 enterprising trip temperature warnings show (step S34).In addition, in the time of in the operative temperature by air conditioner falls into the proper temperature scope, central control unit 3 judges that in the judgement of step S32 temperature falls in the proper temperature scope, and the closing temperature warning shows (step S35).

In central control unit 3 operations, carry out these sensor inspections.That is, central control unit 3 judges whether to have imported stop signal (step S36), and finishes these sensor inspections when stop signal has been imported in judgement.On the other hand, when judgement is not imported stop signal, it will export correcting process portion 15 (step S37) in the stereo optical module 1 such as the environmental information data 100 gloomy sediment information, Raindrop sensor information and the temperature sensor information to by communication line 2, and program is back to step S21 to continue the sensor inspection.Here, in the 4th embodiment,, also can switch by manual operation by the user although switch and carry out the first image evaluation sequence and the second image evaluation sequence with time-sharing format.

It should be noted that, obvious from the process flow diagram that sensor is checked, only when gloomy sediment, reveal to drip, output environment information data 100 just when raindrop and temperature fall into the predetermined condition scope in, even therefore caught the correcting process that stereo-picture does not start these environmental information data 100 of employing yet by image-forming component 5.Therefore, in this case, adopt to show that preferably input equipment 35 warns demonstration, to notify the driver not have executable operations to be owing to do not obtain predetermined condition rather than because mistake can not executable operations.

Figure 16 shows the process flow diagram of the calibration data correction sequence at step S9 place among Figure 14.

Promptly, as shown in figure 16, environment judging unit 33 at first judges by temperature sensor 11 environmental baselines such as temperature information detected and that obtain by communication line 2 whether change (step S41), and for example when the thermal expansion that causes owing to temperature change causes calibration data to change, according to the temperature stepping (temperaturebracket) of previous preparation in EPROM 31 or HDD 53, from calibration data table 102, read calibration data 101 corresponding to this temperature, and by communication line 2 data of reading are write in the control information storer 7 of stereo optical module 1 (step S47), thereby upgrade calibration data 101.

Alternatively, if judge that in step S41 environmental baseline does not change, the subject image that then obtains to have known form (in this example, the stereo-picture that provides from stereo optical module 1 by communication line 2 is provided) (step S42), whether the condition of adhering to such as the attachment position of stereo optical module 1 changed judge (step S43), and when the condition of adhering to changes, according to this variation calibration data is proofreaied and correct (step S44), and in the calibration data 103 adding calibration data table 102 with calibrated and renewal, and it is write in the control information storer 7 of stereo optical module 1, thereby the calibration data 101 that adopts in the correcting process is upgraded by communication line 2.

Carry out this correction and can often carry out correcting process or form coupling with kilter.

Figure 17 is the process flow diagram that the example of sequence is supported in driving that step S14 place among Figure 14 is shown.

Promptly, when environment judging unit 33 enters this driving support sequence, it adopts normal image (sub-picture that is gone out by 43 montages of stereo-picture montage portion) 104 and range image 105 to come to distribute based on scope as input and carries out staging treating (step S52), and carries out guided vehicle identification (step S54 to S57), road surface identification (step S58 to S59), barrier identification (step S61 to S70), lane identification (step S71 to S73) etc. subsequently.

In guided vehicle identification, from normal image 104, extract by detecting the guided vehicle that identifies from vehicle laser light reflected bundle by the radar module 54 that adopts laser radar etc., and the segmentation (segmentation) of itself and range image 105 is complementary discerns guided vehicle zone (step S54), and extract this identified region and such as the information of vehicle width of guided vehicle etc., thereby extract distance (step S55) apart from this guided vehicle, then, inspection is apart from the distance (step S56) of guided vehicle, and ought (for example be not suitable for apart from the distance of guided vehicle, too short) time, showing that by image output device 34 image that carries out on the input equipment 35 as shown in figure 18 shows, thereby carrying out separation between vehicles warning (step S57).

In road surface detects, in range image 105, find the plane (step S58) that from range image 105, to discern in the position of guaranteeing to have road surface, and the scope on this plane is identified as the plane that has with the road surface same level.Further, this road surface is set to reference surface (step S59).When in each identification is handled, adopting the data 106 of this reference surface, its can be used for to object identification, to the identification of guided vehicle, to the identification in track etc.At this moment, as shown in figure 19, can show the plane that shows on the input equipment 35 through identification by image output device 34.

In barrier identification,, and its distance, size etc. are identified as barrier for example to detecting detected plane upper process from road surface and discerned (step S61) by the object that staging treating identified of range image 105.In view of the above, extract the distance (step S62) of barrier, whether to current route is that collision course is judged (step S63), and is showing that by image output device 34 hiding warning on the input equipment 35 urges the operator to hide (step S64) under the situation that is collision course.Then, whether judgement is the distance (step S65) that automobile is stopped by operator's (being the driver) instinct brake operating in the distance that step S62 extracts, and if automobile can stop safely then urge the operator to adopt detent (step S66) showing to warn on the input equipment 35 by image output device 34.On the other hand, under the situation of the distance that automobile can not stop safely, adopt detent by image output device 34 from showing input equipment 35 indication operators, and accelerator pedal unclamps (step S67) automatically.Further,, if possible hide operation (step S68) here,, take safety measures, for example showing that by image output device 34 input equipment 35 urgencies turn to (step S69) then as emergency running.

If collision is inevitable, then carry out the layout in advance (step S70) of passive security equipment (for example air bag).Especially, when the communication system outside the automobile can be operated, then will report etc. take place such as the accident report of current location, accident and be configured to automatically perform.

In lane identification, discern the white line that is positioned on the reference plane according to the brightness value of normal image 104, or the guard rail of reference plane level altitude is given prominence in identification from range image 105, thereby discerned track (step S71).So, but identified running region, check whether automobile travels (step S72) in the track, and when automobile during, in demonstration input equipment 35, carry out the demonstration (step S73) of track warning etc. as shown in figure 21 by image output device 34 run-off-road.

It should be noted that, provided the explanation that on the screen of the demonstration input equipment 35 that links to each other with central control unit 3 shown in Figure 6, shows the example of warning, as the warning in the 4th embodiment, although the method that can provide voice warning from loudspeaker 38, open pilot lamp 42 or adopt the seat that for example vibrates the driver to cause that the operator notes provides warning as not shown but natural modification.This is very effective when the emergency of similar step S64, S66 and S67 is very high, shifts from the place ahead under the preferred situation at operator's eyes, and is especially true.

As mentioned above, according to the 4th embodiment, can when suitably selecting according to environmental information and proofreading and correct calibration data, carry out three-dimensional coupling.

It should be noted that certainly to come each structure among the 4th embodiment is carried out variants and modifications in a lot of modes.

For example, can proofread and correct calibration data by from a plurality of calibration data table of having carried out the table conversion, selecting in advance based on temperature etc.

In addition, countermeasure when sensor is pinpointed the problems in the sensor inspection is not limited to the defroster, rain brush and the air conditioner that illustrate among the 4th embodiment, and can adopt any other method, for example adopt gloomy sediment to forbid that coating or hot-wire well heater replace defroster, adopt super-hydrophobic glass coating or fan blower to replace rain brush, perhaps adopt peltiert equipment or refrigerating fan to replace air conditioner.

In addition, can change the sequence of operation process flow diagram shown in Figure 14 in a lot of modes.For example, when gloomy sediment/dew in the sensor inspection drip/when raindrop/temperature does not fall in the predetermined condition scope, can output environment information data 100, and when the environmental information condition in the environmental information data 100 is not suitable for (step S87: not) and can not carry out correcting process etc. the time, can come stop distance image output processing and can show that display process can not be warned (step S88) on the input equipment 35 by the activated sensors checking sequence by image output device 34, thereby with true notifying operation person (for example driver).

In addition, although in sensor checking sequence as shown in figure 15, continue to use gloomy deposit removing device or rain brush, but when the sensor information indicating status is good (step S105, S111 and S117) can close defroster, rain brush or air conditioner as shown in figure 23.Further, output environment information data (temperature sensor information etc.) (step S120) periodically.

In addition, provided for all explanations of the example of processing serial execution in sensor checking sequence shown in Figure 15, but can check each environmental baseline (step S122, S128 and S134) as shown in figure 24 simultaneously or carry out the operation (step S123, S129 and S135) that disabled status is measured in elimination.In addition, can automatically perform by system and eliminate the operation of measuring disabled status, and the message that indication can not be handled can be shown to the operator.

In addition, although in the 4th embodiment, gloomy sediment, raindrop and thermometer are shown the typical environment condition, also can adopt conversion, illuminance information, fallen leaves or birds droppings, the stain at problem, daytime and night in mist, the image-forming component or the mascot that for example hangs in the automobile to glass pane stop or such as the information of the phase position of sailing the vehicle that comes as environmental information.

In addition, the 4th embodiment can also be used for adopting the stereo camera of a plurality of image-forming components.

[the 5th embodiment]

To illustrate according to a fifth embodiment of the invention below.To the 5th embodiment be described for system with a plurality of optical modules.The structure of each optical module is based on first to fourth embodiment.

That is, as shown in figure 25, a plurality of stereo optical module 1a are connected with communication line 2 with 71b with 1b and monocular optical module 71a.What be connected to communication line 2 has central control unit 3, mode of operation detection module 72, an environmental monitoring module 60 etc.

Here, as the monocular optical module, as shown in Figure 7 side camara module 58 or back camara module 62 are arranged.In addition,, as shown in Figure 7 engine control module 57 or braking sensor module 61 are arranged as mode of operation detection module 72, or unshowned steering angle detection module, transmitting, monitoring module etc.In addition, as environmental monitoring module 60, as shown in Figure 7 radar module 54, vehicle speed sensor module 55, auto navigation module 56, interior monitoring module 59, load detecting module 63, back ultrasonic radar module 64 etc. are arranged.

Here, when being added in the image information that obtains in each optical module such as the video camera ID of each optical module or the information of the calibration data in each optical module, and send/receive and when judging, can judge which calibration data should be used for handling the information in each optical module by 3 pairs of information that obtained of central control unit.

Further, communication line 2 is single system communication lines, and can adopt this communication line 2 to communicate effectively between each unit that comprises central control unit 3 and stereo optical module 1a.Promptly, in this communication, sequence control is carried out to total system in central control unit 3 concentrated areas, and central control unit 3 will comprise that the various control commands of data request command are sent to each unit, and each unit is back to central control unit 3 with the data of request etc. simultaneously.

In addition, as shown in figure 25, can reduce the car inside conductor by sharing this order wire 2.Therefore, can obtain to influence the effect that descends such as weight minimizing, noise.

[the 6th embodiment]

To illustrate according to a sixth embodiment of the invention below.The 6th embodiment is that Raindrop sensor 10 is installed in the embodiment in the stereo optical module.

For example, as shown in Figure 26, traditional Raindrop sensor will project glass blocks 74 near glass pane 73 from the infrared light of light emitting diode (light projection LED) 75 or visible light.Assemble by 76 pairs of these light of light projection lens, and the reflected light lead-in light receiver lens 78 of light on glass outer surface that will assemble by catoptron 77 etc.Then, assemble, and should detect by the light through assembling by 79 pairs of optical receiving sensors such as photodiode by 78 pairs of these lead-in lights of optical receiver lens.At this moment, when raindrop are attached on the glass pane 73, thereby, therefore can measure the raindrop amount because light is scattered light quantity and changes.

Yet in this tradition Raindrop sensor, this Raindrop sensor must tightly invest glass surface, and this causes on space or the cost limited.And then, when Raindrop sensor has blocked driver's the visual field or when the stereo optical module that detects the barrier be positioned at the vehicle front also is installed except Raindrop sensor, there is the problem in the visual field of blocking stereo optical module or Raindrop sensor problem to the installation site of stereo optical module 1.

Therefore, the Raindrop sensor according to the 6th embodiment detects raindrop by adopting the image of being caught by stereo optical module 1.

In described stereo optical module 1 in conjunction with previous embodiment, adopted such stereo optical system, it vertically separates from a side to opposite side long right anaglyph and left anaglyph and they is formed on as shown in Figure 5 the image-forming component, thereby obtains long stereo-picture on as shown in figure 27 this horizontal direction.Here, according to the measurement in space principle, usually remote object has little parallax, and closer object has big parallax.Promptly, in described optical system in conjunction with previous embodiment, owing to can guarantee the bigger disparity range measured, so compared with prior art, consider base length, can in having the zone of stereo-picture, handle the distance of very near distance (for example for ten centimetres and above base length, tens centimetres to tens centimetres closely) to infinity.

Figure 28 shows the view of angle of visibility of the stereo-picture of two viewpoints in the right side with the stereo optical module 1 that uses among the 6th embodiment and a left side.That is, in the 6th embodiment, place stereo optical module 1 as follows: at least one glass pane of vehicle detects target as raindrop and enters crossover region 83, and the angle of visibility of right viewpoint and left viewpoint overlaps in crossover region 83.For example, it is such distance range (dash area among the figure) that the raindrop in the example of Figure 27 detect the hunting zone, and wherein glass pane is arranged in crossover region that Figure 28 paints 83.

Particularly, when considering vehicle planted agent's time spent, owing to remote (several meters to tens or hundreds of rice) are carried out during cognitive disorders thing etc., so the parallax amount of this moment is less.Therefore, when carrying out the form coupling, usually, the search width of the corresponding point on the nuclear line is shortened to improve processing speed.It should be noted that under in-plant situation the search width increases, it is more time-consuming therefore to search for corresponding point.

Yet, when only in closely (especially, being similar to the 6th embodiment, tens centimetres the zone at glass pane place), carrying out the form coupling, can guarantee bigger parallax amount.As extreme example, raindrop only shown in Figure 27 detect just to search in the hunting zone and can detect raindrop.

With reference to Figure 29 and 30, provide this more detailed description, come in the situation of computed range image in that the remote object in is closely carried out continuous range observation, when the reference point from picture end begins to search for, must in whole image mobile search form, carry out form and mate.Search needs the time, may cause critical defect in using in range observation is subjected to the vehicle of time restriction especially etc.

Therefore, in the 6th embodiment, first image evaluation sequence (detecting water droplet or raindrop) and the second image evaluation sequence (detecting the object such as barrier or vehicle) are switched as pattern.That is, in the pattern that detects water droplet or raindrop, as shown in figure 29, in corresponding to the in-plant scope in the benchmark image 80, carry out the form coupling.On the other hand, in the pattern that detects such as the object of barrier or vehicle, the restriction hunting zone, and carry out corresponding to for example 3 meters to 100 meters the search of parallax amount.Carry out this processing by central control unit 3 as image evaluation device.

In addition, when often detecting raindrop particularly, hunting zone in searching image 81 sides can be restricted to the hunting zone (dash area among the figure) in restricted distance zone corresponding to the surface that comprises glass pane 73 shown in Figure 30 narrower zone of form shape or inclination angle coupling (for example, with).That is, can compensate the value of hunting zone as follows: will be defined as the hunting zone in the raindrop hunting zone of right image left end and left image right-hand member, and can carry out calculating based on the first image evaluation sequence.So the restriction hunting zone can be realized with the detection to raindrop of more speed based on the form coupling.

In addition, as shown in figure 30, when the hunting zone tilts along with the inclination of glass pane 73, because the upper end of picture has closer distance, and the picture lower end has the distance far away than the upper end, so can reduce shown in dash area among Figure 30 when the picture upper end side being searched compensation rate to the hunting zone of picture lower end search, carry out the form coupling.

As mentioned above, in the 6th embodiment, mate the function that realizes as the raindrop sensor and can detect raindrop on glass or stain by being limited to in-plant form, and need not in central control unit 3 or stereo optical module 1, to add new equipment.

Further, by in having the scope of little parallax, mating the range observation of carrying out conventional obstacles, and in having the scope of big parallax, mate the detection of carrying out raindrop concurrently.Alternatively, can when carrying out, realize the raindrop measuring ability by carry out this coupling with time-sharing format to the range observation of barrier.

It should be noted that certainly and each structure among the 6th embodiment is carried out variants and modifications in a lot of modes.

For example, the calculating part such as the object of barrier or vehicle of the water droplet on the detection window glass surface or the calculating part of raindrop and detection window glass front can constitute a calculating part, and the second image evaluation/calculating part such as the object of barrier or vehicle of the first image evaluation/calculating part of water droplet on the detection window glass surface or raindrop and detection window glass front can be set respectively in central control unit 3.In this case, first calculating part and second calculating part are constituted and can switch according to each situation.

Further, although in the 6th embodiment, provided the explanation of the example that captured images is handled passively, but also can adopt aggressive mode: have the light of specific wavelength to glass pane 73 emission with the fixed cycle, and detect raindrop based on light from the raindrop scattering with Passive Mode.In this case, can be well enough on time series, switch raindrop and detect and range observation, and light beam is projected one of the right visual field of stereo optical module and left visual field.In addition, under the situation of parallel processing, can in the zone except that crossover region (it is the periphery in the visual field shown in Figure 31 or 32), take piece image well enough as two dimensional image, and the light beam (such as infrared light) of light-beam transmitter 82 (luminaire shown in Figure 33) projected on the zone outside the crossover region, thereby detect raindrop.Even it should be noted that in remote also can not overlap, can outwards open convergent angle and set viewpoint.In addition,, infrared light only can be projected this within sweep of the eye, thereby detect raindrop owing in measurement in space, do not adopt crossover region zone in addition.

In addition, as shown in figure 34, can on corresponding to the glass surface in the zone beyond the crossover region, adopt raindrop to detect coating 84, and for example on glass pane 73 surfaces, provide a scattering surface so that ground glass to be provided well enough, thereby improve perviousness at the raindrop attachment portion.Further, for example can construct the raindrop checkout equipment that detects raindrop by being stained with the coloring board that is used for the humidity judgement.

In addition, can on window, be stained with hydrophobic plate and hydrophilic plate further clearly to judge having/not having of raindrop.

And, by being installed, the close-up shot with positive electric power (positivepower) handles more shooting at close range in the part of the visual field of visual field shadow shield, focal zone can be restricted to closely, or the scope between glass surface and stereo camera system at least, with by to the detection of two dimensional image, adopt infrared light to take, extraction, the color detection of shape waited detect raindrop.When carrying out these classes detections in the zone that is being not used in range observation, can realize detection by adopting single image-forming component, and can influence range observation sharply raindrop.

Further, under the situation of projection infrared light, can realize more effective measurement by removing the infrared ray filter that places on the image-forming component of taking crossover region zone in addition before or substituting this part with bandpass filter.

In addition, although in the 6th embodiment, interior light projection or the raindrop of adopting in zone beyond the crossover region detect coating, if but the picture periphery is not used in range observation, and then above-mentioned raindrop pick-up unit is certainly at constructing corresponding to the peripheral glass pane of locating of the picture of this part.

It should be noted that, in each previous embodiment, can construct so-called general focusing stereo optical system (its have near the glass pane before the pentrution preset distance (for example, 50 to 300 meters) scope), and focusing is not set in stereo optical system.In this case, even focusing is not provided, the barrier that also can focus on the raindrop that are attached on the glass pane or attachment and front substantially is on both, thereby discerns the shape of attachment and barrier simultaneously.

Although based on embodiment the present invention has been described above, the present invention is not limited to previous embodiment, and certainly carries out various modification or application within the scope of the present invention.Although stereo camera system in the vehicle has been described in the aforementioned embodiment, even in such as the moving object of robot, also can adopt the present invention under the situation of execution range observation.

In addition, previous embodiment comprises the invention in each stage, can obtain diversified invention by a plurality of disclosed constitutive requirements of appropriate combination.For example, even discloseder constitutive requirements among the deletion embodiment, still can solve the technical matters of describing in " technical matters that invention will solve " part, and in the time can obtaining the effect of describing in " invention effect " part, also can provide the structure of wherein deleting this structure important document as of the present invention.

Claims

1, a kind of stereo optical module is characterized in that, comprising:

2, a kind of stereo optical module is characterized in that comprising:

The second polarisation mechanism, on the image-forming component that is positioned at the second half surface portions, form the whole or most image of described second subject image, described second subject image is to be obtained by the light beam that enters from described second light guide section, described the second half surface portions are positioned on the second direction relative with described first direction, are described datum line between described first direction and the described second direction; And

Package is protected the mechanism in the described stereo optical module,

3, a kind of stereo optical module is characterized in that, comprising:

The first polarisation mechanism, on the image-forming component that is positioned at the first half surface portions, form the whole or most image of described first subject image, described first subject image is to be obtained by the light beam that enters from described first light guide section, described the first half surface elements are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first light guide section and described second light guide section;

Package is protected the mechanism in the described stereo optical module from the outside,

4, a kind of stereo optical module is characterized in that, comprising:

The first polarisation mechanism, comprise first reflecting element of the light that reflection enters from described first optical lens and second reflecting element that the light by described first reflecting element reflection is reflected again, on the image-forming component that is positioned at the first half surface portions, to form the whole or most image of described first subject image, described the first half surface portions are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first optical lens and described second optical lens;

Wherein a member places at least one of first space region and second space region, described first space region is present on the described first direction to described first reflecting element or described second reflecting element, and described second space region is present on the described second direction to described the 3rd reflecting element or described the 4th reflecting element.

5, a kind of stereo optical module is characterized in that, comprising:

The first polarisation mechanism, comprise first reflecting element of the light that reflection enters from described first optical lens and second reflecting element that the light by described first reflecting element reflection is reflected again, on the image-forming component that is positioned at the first half surface portions, to form the whole or most image of described first subject image that enters described first optical lens, described the first half surperficial branches are positioned on the first direction perpendicular to the incident direction of datum line and light beam, and described datum line connects the center separately of described first optical lens and described second optical lens;

The second polarisation mechanism, comprise the 3rd reflecting element of the light that reflection enters from described second optical lens and the 4th reflecting element that the light by described the 3rd reflecting element reflection is reflected again, on the image-forming component that is positioned at the second half surface portions, to form the whole or most image of described second subject image that enters described second optical lens, described the second half surface portions are positioned on the second direction relative with described first direction, are described datum line between described first direction and the described second direction; And

According to each the described stereo optical module in the claim 1 to 3, it is characterized in that 6, described first light guide section comprises first optical lens, and described second light guide section comprises second optical lens.

7, stereo optical module according to claim 1 is characterized in that, the described first polarisation mechanism comprises first reflecting element and second reflecting element, and the described second polarisation mechanism comprises the 3rd reflecting element and the 4th reflecting element.

8, stereo optical module according to claim 1 is characterized in that, described member is the peripheral circuit of described image-forming component.

9, stereo optical module according to claim 8 is characterized in that, the peripheral circuit of described image-forming component is the driving circuit of described image-forming component.

10, stereo optical module according to claim 8 is characterized in that, the peripheral circuit of described image-forming component is for carrying out the image processing circuit of Flame Image Process to the picture signal of exporting from described image-forming component.

11, stereo optical module according to claim 8 is characterized in that, the peripheral circuit of described image-forming component is used to proofread and correct from the number storage of the correction data of the picture signal of described image-forming component output for storage.

12, stereo optical module according to claim 1 is characterized in that, described member is a temperature sensor.

13, stereo optical module according to claim 1 is characterized in that, described member is a Raindrop sensor.

14, stereo optical module according to claim 1 is characterized in that, described member is an illuminance transducer.

15, stereo optical module according to claim 1 is characterized in that, described member is a gps antenna.

16, stereo optical module according to claim 1 is characterized in that, described member is an ETC card system.

17, stereo optical module according to claim 1 is characterized in that, described member is the electric signal output circuit of output electric signal.

18, a kind of stereo camera places on the vehicle and by glass pane and observes the place ahead, and this stereo camera is characterised in that, comprising:

19, stereo camera according to claim 18, it is characterized in that, place described stereo optical system in the following manner: described glass pane is placed in the zone, this district or described in the visual field of second viewpoint that is different from described first viewpoint in a plurality of viewpoints of the visual field of first viewpoint in a plurality of viewpoints of stereo optical system and described stereo optical system overlap mutually.

20, a kind of stereo camera places in the vehicle and by glass pane and observes the place ahead, and this stereo camera is characterised in that, comprising:

21, stereo camera according to claim 20 is characterized in that, described predetermined distance range is the distance range corresponding to the relation of the position between described stereo camera and the described glass pane.

22, stereo camera according to claim 21 is characterized in that, described predetermined distance range with the visual field change in location of the corresponding described stereo camera of shape or inclination angle of described glass pane.

23, stereo camera according to claim 20 is characterized in that, described attachment judgment device is for detecting the water droplet checkout equipment of the water droplet that is attached to described glass surface.

24, a kind of stereo camera places in the vehicle and by glass pane and observes the place ahead, and this stereo camera is characterised in that, comprising:

Image-forming component, the described subject image of seeing from a plurality of viewpoints based on being produced by described stereo optical system produces stereoscopic image data;

25, stereo camera according to claim 24 is characterized in that, the described first image evaluation sequence is calculated the sequence that is attached to the water droplet of described glass surface with detection for carrying out in the first range search scope, and

The described second image evaluation sequence is for carrying out the sequence of calculating with the object that detects described glass pane front in the second distance hunting zone that is different from the described first range search scope.

26, stereo camera according to claim 24, it is characterized in that the described first image evaluation sequence is calculated near carrying out between the predetermined image zone of the described first viewpoint side near the predetermined image zone of the second viewpoint side that is different from described first viewpoint in a plurality of viewpoints described in the described stereo optical system with respect to the image of seeing from described second viewpoint in the view data of seeing with respect to first viewpoint from a plurality of viewpoints of described stereo optical system.

27, stereo camera according to claim 24, it is characterized in that, when calculating between the view data that the view data of seeing in first viewpoint from a plurality of viewpoints of described stereo optical system and second viewpoint that is different from described first viewpoint from described a plurality of viewpoints of described stereo optical system are seen, the described first image evaluation sequence provides compensation corresponding to the distance of determining according to the position of described stereo camera and described glass pane to calculate.

28, stereo camera according to claim 27 is characterized in that, described compensation changes with the position, the visual field with the corresponding described stereo camera of shape or inclination angle of described glass pane.

29, stereo camera according to claim 25 is characterized in that, the described first range search scope changes with the position, the visual field with the corresponding described stereo camera of shape or inclination angle of described glass pane.

30, a kind of stereo camera places in the vehicle and by glass pane and observes the place ahead, and this stereo camera is characterised in that, comprising:

31, stereo camera according to claim 30 is characterized in that, described first image evaluation/calculating part calculates the water droplet that is attached to described glass surface with detection in the first range search scope, thereby realizes image evaluation, and

Described second image evaluation/calculating part calculates in the second distance hunting zone that is different from the described first range search scope with the object of detection in described glass pane front, thereby realizes image evaluation.

32, a kind of stereo camera places in the vehicle and by glass pane and observes the place ahead, and this stereo camera is characterised in that, comprising:

Luminaire, the glass surface in the zone that overlaps with first visual field of first viewpoint in a plurality of viewpoints of the described stereo optical system of rayed and described glass pane; And

33, stereo camera according to claim 32 is characterized in that, described luminaire is with the surface of described glass pane in the zone of described first visual field of rayed and described glass pane overlapping, and described zone is the periphery in described first visual field.

34, stereo camera according to claim 32 is characterized in that also comprising:

Object detecting apparatus detects the object of described glass pane front based on the described stereoscopic image data from the output of described image-forming component; And

Control circuit activates luminaire when the described water droplet checkout equipment of operation, and when the described object detecting apparatus of operation inactive described luminaire.

35, a kind of stereo camera places in the vehicle and by glass pane and observes the place ahead, and this stereo camera is characterised in that, comprising:

36, stereo camera according to claim 18 is characterized in that, described stereo optical system is constituted as general focus type.