US20170006276A1

US20170006276A1 - Wide angle imaging system for providing an image of the surroundings of a vehicle, in particular a motor vehicle

Info

Publication number: US20170006276A1
Application number: US15/248,294
Authority: US
Inventors: Christoph Arndt; Urs Christen; Heike Kleinschmidt
Original assignee: Ford Global Technologies LLC
Current assignee: Ford Global Technologies LLC
Priority date: 2009-02-02
Filing date: 2016-08-26
Publication date: 2017-01-05
Also published as: US20100194889A1; CN101881923A; US9469249B2; DE102009000550B4; CN101881923B; DE102009000550A1

Abstract

An imaging system includes a digital camera having a sensor (such as a charge coupled device), a first lens directing a first image onto a first region of the sensor, a second lens directing a second image onto a second region of the sensor, and a third lens directing a third image onto a third region of the sensor. A display screen displays to a driver of the vehicle the first image, and a processing unit performs stereoscopic image analysis on data originating from the second and third regions. A fourth lens may be used to direct a fourth image onto a fourth region of the sensor, and the processing unit performs calculations on data from the fourth region for the detection of movement of the vehicle.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional claiming priority on U.S. application Ser. No. 12/695,554, filed on Jan. 28, 2010, which claims foreign priority benefits under 35 U.S.C. §119(a)-(d) to DE 10 2009 000 550.1, filed on Feb. 2, 2009, which is hereby incorporated by reference in its entirety.

BACKGROUND

1. Technical Field
The invention relates to an imaging system for providing an image of the surroundings of a motor vehicle, such that this image can be displayed to the driver and simultaneously used for further evaluations by image analysis.
2. Background Art
U.S. Pat. No. 6,922,292 B2 discloses a wide angle imaging system for providing an image of the surroundings of a vehicle (for example a reversing image display device). In this case, a camera inserted into the front side or rear side of the vehicle is used together with a lens arrangement downstream thereof, in order to present the driver with a view of the surroundings imaged by the camera on a display screen in the vehicle's interior. The image is prepared by refracting and diffracting lenses together with distortion compensation by way of correction. The imaging device used has a non-uniform array of electro-optical pixels, and the display screen is provided with a uniform pixilation.
Another wide angle imaging system is disclosed in DE 10 2004 053 416 A1 which discloses a stereoscopic distance measurement system for use in a motor vehicle which has the following elements arranged in the specified sequence in the beam path of a virtual beam: a) a digital camera having an imaging optics, and having an aperture solid angle range that delimits the virtual beam emanating from the camera; b) at least one deflecting mirror element entirely or partially deflecting the virtual beam emanating from the camera; and c) at least one image field divider element having at least one divider mirror element deflecting at least a first component beam of the virtual beam emanating from the camera, the first component beam overlapping at least a second component beam of the virtual beam emanating from the camera in a restricted spatial region. The corresponding CCD element is split up into two separate regions lying one above another, of which a lower region is used for the driver display. This region and an upper region (invisible to the driver) are used together for a stereoscopic evaluation, both regions having the same resolution.

SUMMARY

In a disclosed embodiment of the invention, an imaging system for a motor vehicle comprises a digital camera having a sensor, a first lens directing a first image onto a first region of the sensor, and a second lens directing a second image onto a second region of the sensor. A display screen displays to a driver of the vehicle the first image and a processing unit processes data originating from the second region. The lens system is configured with at least two separate lenses in order to direct separate images of the surroundings on the imaging sensor as two optically different regions, of which a first region is associated with the driver display, and a second region is associated with the data processing unit for the purpose of evaluation. It is therefore possible to adapt the regions to the different requirements, and to optimize them. Thus, the lenses can be optimized independently of one another for the respective use of the regions. Thus, the region for the driver display can be a large central region of the virtual image, directed on the sensor, of the surroundings in high resolution. Thus, the driver can be provided with a central, more effectively resolved image, and the other region can be optimized for digital image processing, such as detection of relative movement between the vehicle and objects exterior to the vehicle.
The lower image segment, which is preferably used for this purpose, of the image of the surroundings on the sensor permits this in a favorable way, since the observed region comprises the substratum of the road that, in turn, includes structures (lane markings, etc.) which permit simple determination of the path and/or a deviation therefrom. The evaluation of this image region also permits the determination of the vehicle's longitudinal and transverse speeds.
In another embodiment, the digital camera further comprises a third lens directing a third image onto a third region of the sensor, and the processing unit processes data originating from the second and third regions of the sensor and performing stereoscopic image evaluation. When the lens system is configured with three independent lenses directing three separate images of the surroundings onto the imaging sensor in three different regions, of which the second and third regions are associated with the data processing unit for the purpose of stereoscopic evaluation, the second and third lenses and image regions can be optimized for stereoscopic evaluation, in particular as concerns their distortion and spacing from one another. The spacing between the lenses is important for the stereoscopic evaluation with the aid of sequentially stored images of the surroundings or their data from the sensor.
When the lens system is configured even with a further lens in order to direct the surroundings on the imaging sensor with the aid of an additional (fourth) optical region, and this additional region is associated with the data processing unit for the purpose of evaluations intended to detect movement, it is further possible to provide a direction detection unit based on an image processing unit in addition to the driver display and the stereoscopic evaluation of the determination of distance. The processing unit may detects movement and/or direction of the vehicle by analyzing lane markings, curbs, or other structures appearing in the fourth image.
In particular, the left and right hand lateral edge regions are suitable for the stereoscopic evaluation, and the lower edge region is suitable for detecting relative movement. The middle region important to the driver therefore remains in association with the driver display.
In the disclosed embodiment of the invention, the first region lies in the middle region of the image, for which the driver usually has the most interest, and so said region is associated and used with the driver display on a display screen. Additionally, information of interest to the driver such as, for example, data on path and distance etc, can be overlaid onto the visual display. Again, this region can be used for the image calibration and even detection of collision with an object, without the main function of the driver display being influenced.
The second region preferably lies at the middle lower edge. This region has also, if appropriate, already been used in the first region, or been overlapped therewith. In this region of the image, the road is included directly in front of or behind the vehicle so that the structures present there (compare above) can be used to evaluate movement, that is to say to detect the path of the vehicle by image processing.
The third and fourth regions preferably lie respectively at the left and right hand lateral edge outside the middle region. As already described above, these regions can be used for the stereoscopic evaluation. Moreover, they are also suitable for monitoring the dead angle as well as for determining a rotation (angle and angular velocity) of the vehicle by means of image processing methods.
It may therefore be advantageous for the lens system and/or respective lens to comprise prisms and/or mirrors for increasing the effective spacing of the optical regions for the purpose of evaluation, and for reducing the distortions that occur. An increase in the resolution of the stereoscopy may also be achieved.
For evaluation of the data supplied by the sensor, the data processing unit may have a buffer for the sensor data such that the data can be processed sequentially, for example during the image processing for the purpose of detecting movement and path or for stereoscopic determination of distances.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details, features and advantages of the invention emerge from the following description of exemplary embodiments with the aid of the single drawing, in which:

FIG. 1 is an overview schematic of a vehicle with an embodiment of a wide angle imaging system;

FIG. 2 is a schematic of the division of an imaging sensor;

FIG. 3 is a schematic of the design of an embodiment of a lens system; and

FIG. 4 is a schematic of the design of an alternative embodiment of a lens system.

DETAILED DESCRIPTION

FIG. 1 is a schematic of a vehicle 1 having an imaging system 2 for providing an image of the vehicle surroundings U. The imaging system is able to display this image to the driver on a screen and to simultaneously digitally process the image for further evaluations.
The imaging system 2 comprises a camera 3 for imaging the surroundings U, here in particular the road S in front of or behind the vehicle 1. The camera 3 comprises a lens system 4 and a sensor 5, the lens system 4 directing multiple images of the surroundings onto the sensor 5. Sensor 5 is preferably an electronic optical sensor and may be a charge coupled device (CCD) or a complimentary metal-oxide semiconductor (CMOS) image sensor, as is well known in the field of artificial vision.
The imaging system 2 further comprises a visual display screen 6 for displaying to the driver of the vehicle the surroundings, as imaged by the camera 3 and directed onto the sensor 5 by the lens system 4. The driver is thereby enabled when driving backward to search the surroundings for objects, etc., visible on the display screen in a known way.
The imaging system 2 additionally comprises a data processing unit 7 for processing the electronic data originating from the sensor 5 for the purpose of further evaluations. The data processing unit 7 is “switched” logically between the sensor 5 and display 6, and can be embodied, with appropriate suitability, by a microprocessor, which is present in any case in the onboard electronics.
As seen in FIG. 3, the lens system 4 is configured with four lenses 4 a, 4 b, 4 c, 4 d in order to direct images of the surroundings onto the imaging sensor 5 as four optically different regions 5 a, 5 b, 5 c, 5 d (compare FIG. 2). The first region 5 a is associated with the visual display screen 6. The other regions 5 b, 5 c, 5 d are, by contrast, associated with the data processing unit 7 for digital image evaluation purposes.
Thus, the lenses 4 a, 4 b, 4 c, 4 d can be optimized independently of one another depending on the respective use of each particular region. Characteristics that may be optimized may, for example, relate to their distortion, resolution, etc. Thus, the region for the driver's visual display can be a large middle region allowing for an image of the surroundings in high resolution. The driver can thereby be provided with a central, more effectively resolved image, and it is possible for the other regions to be optimized for detection of direction and movement and/or for stereoscopic evaluation by image processing.
As seen in FIG. 2, a vehicle 11 located behind the driver's own vehicle is shown with little or no distortion in region 5 a. The roadway directly behind the driver's own vehicle is illustrated in region 5 b with height distortion for the purpose of detecting obstacles by processing unit 7. The region behind the driver's own vehicle is illustrated in regions 5 c and 5 d with width distortion, with in each case the same resolution for the purpose of stereoscopic evaluation performed by processing unit 7.
A quasi-stereoscopic evaluation is being carried out here, by means of a single camera with the aid of different camera positions (on the basis of the movement of the vehicle). The greater this inter-axis spacing between the respective central axes of the lenses, the more effective or accurate is the subsequent evaluation. The differences between the respective images on the two regions of the sensor can be used to determine angular differences between the observed objects, and to calculate a pixel displacement from them. The mutual inter-axis spacing between the two regions is known per se, and is used for the further evaluation by the processing unit.
It is also possible to combine two of more of the lenses so as, as illustrated here in the exemplary embodiment, to combine the two central lenses 4 a and 4 b into a combined central lens 8 a/b that consists of an upper lens portion 4 a and a lower lens portion 4 b. The two lens portions 4 a, 4 b have different optical characteristics to produce the two different images in regions 5 a and 5 b respectively.
The first region 5 a is located in this embodiment in the middle region of the sensor 5 which is usually of most interest for the driver, and so is associated with the driver's visual display on the display screen 6. The display of this image on display screen 6 can be overlaid with information of interest to the driver such as, for example, vehicle path and distance data, etc. This region 5 a can also be used for the image calibration, and even for detection of collision with objects, without the main function of the driver display being influenced. The data of region 5 a can also be processed, analyzed, and/or conditioned by the data processing unit 7.
The second region 5 b lies at the middle lower edge, below the first region 5 a. This region may, if appropriate, also already be used for the driver display together with the first region 5 a.
The region 5 b of may include the road directly in front of or behind the vehicle, and so the structures present there (compare above) may be used to evaluate movement, that is to say to detect the path of the vehicle, by image processing. The lower section of the image of the surroundings, which is preferably used for this purpose, on the sensor 5 a permits this in a favorable way, since the observed region comprises the surface of the road which, in turn, includes structure (lane markings, lines, curbs, etc.) that allow a determination of the desired vehicle path or a deviation therefrom, and/or the determination of the longitudinal and transverse speeds.
The third and fourth regions 5 c, 5 d lie respectively at the left and right lateral edge outboard of the middle regions 5 a and 5 b. As already described above, the regions 5 c, 5 d may be used for stereoscopic evaluation, performed by processing unit 7. Furthermore, they are also suitable for monitoring a blind spot as well as for determining a rotation (angle and angular velocity) of the vehicle by means of image processing methods.
As best seen in FIGS. 3 and 4, the lens system is configured with independent lenses 4 a, 4 b, 4 c, 4 d in order to direct multiple images of the surroundings onto the three or four optically different regions 5 a, 5 b, 5 c, 5 d of the imaging sensor 5.
The inter-axis spacing A between the respective central axes of lenses 4 c and 4 d and between the respective regions 5 c and 5 d is significant for the stereoscopic evaluation with the aid of sequentially stored images of the surroundings, or their data from the sensor. A quasi-stereoscopic evaluation may be carried out here by means of a single camera with the aid of different camera points-of-view (on the basis of the movement of the vehicle). The greater this inter-axis spacing A, the more effective or more accurate is the subsequent evaluation. The spacing between the two regions 5 c, 5 d of the image of the surroundings U is used in this case to determine angular differences between the objects observed in the regions, and to calculate a pixel displacement from them. The spacing A between the two regions is known per se and is used for the further evaluation by processing unit 7.
Referring now to FIG. 4, each lens 4′c, 4′d comprises a pair of mirrors 9 c, 10 c and 9 d, 10 d respectively. The mirror pairs 9 c, 10 c and 9 d, 10 d are optically aligned to direct and focus the images onto regions 5 c and 5 d respectively. Other focusing lenses and/or prisms (not shown) may also be employed along with the mirrors to provide the desired optical characteristics. The outboard placement of mirrors 9 c, 9 d results in an increase in the effective spacing A between the optical regions for the purpose of stereoscopic evaluation. An increase in the resolution of the stereoscopy may be achieved thereby.

Claims

What is claimed is:

1. An imaging system comprising:

an electro-optical sensor;

a first lens directing an image onto the sensor;

a display screen displaying the image;

second and third inter-axially spaced lenses directing respective second and third images onto the sensor on opposite sides of the image, the second and third lenses producing purposeful width distortion of the respective images adapted for stereoscopic analysis; and

a processing unit performing stereoscopic analysis on the second and third images.

2. The imaging system of claim 1, further comprising a fourth lens inter-axially spaced from the first, the second, and the third lenses to direct a fourth image onto a fourth region of the sensor located adjacent a third edge of the sensor, the fourth image exhibiting purposeful height distortion.

3. The imaging system of claim 2, wherein the processing unit performs calculations on data from the fourth region to detect lateral movement of a host vehicle.

4. The imaging system of claim 3, wherein the processing unit detects lateral movement of the vehicle by analyzing lane markings appearing in the fourth image.

5. The imaging system of claim 1, wherein at least one of the lenses comprises at least two mirrors optically aligned with one another.

6. The imaging system of claim 1, further comprising a data buffer associated with the processing unit.

7. An imaging system for a motor vehicle comprising:

an electro-optical sensor;

a first lens directing a first image onto a central region of the sensor;

a display screen for displaying the first image to a driver of the vehicle;

a second lens inter-axially spaced from the first lens to direct a second image onto a second region of the sensor located adjacent a first lateral edge of the sensor;

a third lens inter-axially spaced from the first lens and the second lens to direct a third image onto a third region of the sensor located adjacent a second lateral edge of the sensor, the second lateral edge positioned opposite from the first lateral edge, the second and third images having substantially equal amounts of purposeful width distortion adapted for stereoscopic image analysis; and

a processing unit performing stereoscopic image analysis on data originating from the second and third regions.

8. The imaging system of claim 7, further comprising a fourth lens inter-axially spaced from the lens, the second lens, and the third lens to direct a fourth image onto the sensor, the fourth image exhibiting height distortion.

9. The imaging system of claim 8, wherein the processing unit performs analysis on the fourth image to detect lateral movement of a host vehicle.

10. The imaging system of claim 9, wherein the processing unit detects lateral movement of the vehicle by analyzing lane markings appearing in the fourth image.

11. The imaging system of claim 7, further comprising a data buffer associated with the processing unit.

12. Apparatus comprising:

a first lens directing a first image onto a first region of an electro-optical sensor; and

second and third lenses spaced inter-axially from one another and located on opposite sides of the first lens and directing respective second and third images having substantially equal amounts of purposeful width distortion adapted for stereoscopic evaluation onto respective second and third regions of the sensor.

13. The apparatus of claim 12, further comprising a processing unit performing stereoscopic analysis on the second and third images.

14. The apparatus of claim 13, further comprising a data buffer associated with the processing unit.

15. The apparatus of claim 12, further comprising a display screen for displaying the first image to a driver of a host vehicle.

16. The apparatus of claim 12, wherein at least one of the lenses comprises at least two mirrors optically aligned with one another.

17. The apparatus of claim 12, further comprising a fourth lens inter-axially spaced from the first lens, the second lens, and the third lens to direct a fourth image onto the sensor, the fourth image exhibiting purposeful height distortion.

18. The apparatus of claim 17, further comprising a processing unit performing analysis on the fourth image to detect lateral movement of a host vehicle.

19. The apparatus of claim 18, wherein the processing unit detects lateral movement of the vehicle by analyzing lane markings appearing in the fourth image.