WO2019091692A1

WO2019091692A1 - Arrangement for optical sensors with effect on the viewing angle

Info

Publication number: WO2019091692A1
Application number: PCT/EP2018/077731
Authority: WO
Inventors: Alexander Stohr
Original assignee: Zf Friedrichshafen Ag
Priority date: 2017-11-09
Filing date: 2018-10-11
Publication date: 2019-05-16
Also published as: DE102017219916B4; DE102017219916A1

Abstract

The invention relates to an image sensor adjusting device (10) for an imaging system (20), to an imaging system (20) for rotating an image sensor (1), to an imaging system (20) with a rotated image sensor (1), to the arrangement of an image sensor (1) in a housing (24) of an imaging system (20), to a computer program product, to an imaging system (20) comprising a computer (40), to the use of an imaging system (20) according to the invention, to a method for increasing the viewing field (22) of an imaging system (20), and to a camera of a vehicle assistance system of a vehicle (30) which can be operated in an automated manner.

Description

Arrangement for optical sensors with effect on the viewing angle

The invention relates to an image sensor positioning device for an imaging system according to claim 1, an imaging system according to claim 4, an imaging system according to claim 6, an arrangement of an image sensor in a housing of an imaging system according to claim 7, a computer program product according to claim 8, an imaging system with a computer according to claim 9, a use of an imaging system according to the invention according to claim 10, a method for enlarging a field of view of an imaging system according to claim 11 and a camera of a driver assistance system of an automatically operable vehicle according to claim 14.

Optical sensors, for example image sensors, with wide-angle or fisheye fields of view lead to a circularly limited imaging of a detected environment at the level of the sensor. By means of a corresponding combination of optics and camera can be cut by an image sensor, the entire circle, a section of the circle or a complete rectangle of the circular area of an image of a lens of the optics, a so-called lens image. Each variant leads to a typical pixel density. For the purposes of the invention, the horizontal pixel density is the quotient of the number of pixels in width on the sensor and the width of the detected area of the imaged world. Accordingly, the vertical pixel density is the quotient of the number of pixels in height on the sensor and the height of the detected area of the imaged world. Width, height and length denote the respective extent in the direction of the width, height and length of an object. The choice of each variant is in each case a certain compromise between the detected area and the system-related achieved angular resolution in terms of pixel density.

The object of the invention is based on optionally improving the detected area and / or the pixel density achieved compared to an identical pairing of sensor and optics and a viewing area. The object is achieved with an image sensor positioning device according to the invention having the features of claim 1, an imaging system having the features of claim 4, an imaging system having the features of claim 6, an arrangement of an image sensor having the features of claim 7, a computer program product having the features of Claim 8, an imaging system with a computer having features of claim 9, a use of an imaging system according to the invention with the features of claim 10, a method for enlarging a field of view with the features of claim 11 and a camera of a driver assistance system of an automated operable vehicle with the features of claim 14.

Further developments and advantageous embodiments of the invention are specified in the subclaims.

The image sensor positioning device according to the invention for an imaging system, which has a housing and an image sensor arranged in the housing, has an actuator which is designed to set the image sensor in rotation with a constant orientation of the housing by a rotation angle.

An image sensor is a device for receiving images of light by electrical or mechanical means. An example of an image sensor is an active pixel sensor based on CMOS technology, that is complementary metal-oxide semiconductor. An Active Pixel Sensor is a semiconductor detector for light measurement in which each pixel includes an amplifier circuit for signal readout. In digital photography, rectangular active pixel sensors are preferably used with a screen diagonal of 27 mm to 28 mm. Other formats used have a screen diagonal of 21, 633 mm, the so-called Four-Thirds sensors, or a screen size of 43.267 mm, the so-called full size, which corresponds to the small format of roll films. Image sensors preferably have a rectangular format. An image sensor adjustment device is a mechanical and / or electronic device for offsetting an image sensor from a first position to a second position.

An imaging system projects a light image onto a photosensitive medium using optical techniques. The photosensitive medium may be a roll film from which photographic images are used to form permanent photographs, e.g. Slide, film or paper image. However, the photosensitive medium can also be an image sensor which uses a digital method to convert the photograph into electronic data and to store it permanently.

An imaging system comprises, in addition to the photosensitive medium, a housing and, advantageously, a collecting optical system for producing an optical image of an object, for example an objective which may have an array of several lenses. A lens produces an image of a detected object on a receiving surface of the photosensitive medium. A receiving surface of the image sensor is the surface on which the photosensitive elements of the image sensor are arranged. For example, a camera with an objective is an imaging system.

An actuator is a mechanical and / or electronic element for relaying an adjusting movement of the image sensor adjusting device.

With the image sensor adjusting device according to the invention, a recording surface of the image sensor can detect different regions of a lens image of the imaging system by the rotation of the image sensor while maintaining the alignment of the housing of the imaging system. In particular, for fixed, permanently installed, fixedly mounted imaging systems, for example a camera of a driver assistance system installed behind a windshield of a vehicle, objects that would be outside the field of view of the imaging system can be detected at the same resolution with the same resolution. In a further embodiment of the invention, the actuator is designed to move the image sensor in an image plane. In this way, objects that would be outside the field of view of the imaging system can be detected at the same resolution. Preferably, the actuator is configured to combine rotation and displacement of the image sensor.

Advantageously, the actuator is a motor, preferably an electric motor, more preferably a miniaturized electric motor, and the image sensor setting means has a control adapted to drive the actuator to perform the rotation by the rotation angle.

A further embodiment of the invention provides that the actuator is a mechanical actuator and is coupled to a user interface.

With the invention, an advantageous fixation of the image sensor can also be achieved without an actuator, for example by a static arrangement of the image sensor.

A motor is a machine that converts energy to mechanical work, and preferably includes a shaft, that is, a rotary motion and / or torque transmitting machine element that is rotated by the motor to drive a device. An electric motor is a motor that converts electrical energy into mechanical work. Miniaturization refers to a reduction of structures while retaining the function and preferably the shape. A miniaturized electric motor can advantageously be easily placed in the housing of the imaging system to rotate the image sensor. The electrical energy can be obtained by the miniaturized electric motor from a battery of the imaging system.

With the control, a predetermined rotation angle, which receives the image sensor adjustment means, for example via an input interface, in a signal, preferably an electrical signal, to be converted to drive the motor. An interface is a device between at least two functional units at which an exchange of logical quantities, eg data or physical quantities, eg electrical signals, takes place, either only unidirectionally or bidirectionally. The exchange can be analog or digital. The exchange can be wireless or wired. An interface may exist between software and software, hardware and hardware, and software and hardware, and hardware and software.

The rotation angle can also be set manually via the input interface.

In a particularly advantageous embodiment of the invention, the image sensor positioning device is designed to rotate the image sensor in such a way that after rotation, a diagonal of the image sensor is arranged horizontally. For a rectangular image sensor, the length of the diagonal results from the root of the sum of the square of the length and the square of the width of the image sensor. Therefore, the diagonal is longer than the length of the image sensor and longer than the width of the image sensor

Image sensor. This expands the horizontal angle of view and thus the field of view.

The field of view is the object sided volume, which can be detected, that is, with the imaging system can be sufficiently sharply imaged. The angle of view is the angle in the object-side space which is limited by the edges of the image sensor and depends on a focal length of a lens of the imaging system. The horizontal angle of view results from the horizontal image sensor width. The vertical angle of view results from the vertical image sensor height.

The imaging system according to the invention comprises a housing, an image sensor arranged in the housing and an image sensor setting device, wherein the image sensor positioning device is designed to rotate the image sensor while the housing is kept in the same orientation to enlarge a field of view of the imaging system. This advantageously allows the horizontal and vertical image of the imaging system and thus the field of view of the imaging system can be increased.

In a further embodiment of the invention, the image sensor setting device is designed to move the image sensor in an image plane. Preferably, the image sensor setting means is adapted to combine rotation and displacement of the image sensor.

The image sensor adjusting device is preferably an image sensor adjusting device according to the invention.

A further imaging system according to the invention has a housing and an image sensor arranged in the housing, wherein a receiving surface of the

Image sensor is rotated about an axis of rotation, preferably parallel to an optical axis of the imaging system, by a rotation angle.

The optical axis is the symmetry axis of the imaging system.

With the imaging system according to the invention, edge regions of the lens image can advantageously be imaged.

Preferably, an imaging system according to the invention comprises a lens or a system of lenses. In this case, a lens is advantageously a wide-angle or a fisheye lens with regard to the size of the field of view.

In the inventive arrangement of an image sensor in a housing of an imaging system, a receiving surface of the image sensor is rotated about an axis of rotation, preferably parallel to an optical axis of the imaging system, by an angle of rotation to increase a field of view of the imaging system.

The computer program product according to the invention is designed to be loaded into a memory of a computer and has software code sections with which an image sensor arranged in a housing can be rotated by rotation constant alignment of the housing, resulting distortion of an image of an imaging system can be corrected when the computer program product is running on the computer.

Computer program products typically include a sequence of instructions that cause the hardware, when the program is loaded, to perform a particular procedure that results in a particular result. When the program concerned is used on a computer, the computer program causes a technical effect, namely an equalization of an image, which has arisen due to a rotation of the image sensor with constant alignment of the housing.

In this case, a computer is a device for processing data that can process data by means of programmable calculation rules.

A memory is a medium for backing up data.

Software is a collective term for programs and related data. The complement to the software is hardware. Hardware refers to the mechanical and electronic alignment of a data processing system.

The technical problem of correcting a distortion of an image of an imaging system resulting from a rotation of an image sensor arranged in a housing while maintaining the alignment of the housing is technically solved by the computer program, for example by means of the software code sections mapping in a 3D space, in which distortions can be compensated.

According to the invention, an imaging system according to the invention has a computer on which the computer program product according to the invention runs. Thus, distortion can already be corrected by the imaging system. The computer advantageously has an image processor with multi-core processor architecture. A processor is a programmable calculator. For example, a central processing unit or a graphics processing unit, which can execute multiple bills in parallel, are each a processor.

According to the invention, an imaging system according to the invention for an automatically operable vehicle is used as the camera of a driver assistance system.

An automatically operable vehicle is a vehicle having technical equipment capable of handling a driver task, including longitudinal and lateral guidance, of the respective vehicle after activation of a corresponding automatic driving function, in particular a fully or fully automated driving function according to standard SAEJ3016, with a vehicle control device can control. A partially automated vehicle can take over certain driving tasks. A fully automated vehicle replaces the driver. A pure assistance system assists the driver in carrying out a driving task.

A driver assistance system, also known as the Advanced Driver Assistance System, is a system that supports the driver in coping with driving tasks.

A camera of a driver assistance system is permanently installed in the vehicle and can not be rotated. Therefore, a rotation of the image sensor with a constant orientation of the camera or an already rotated image sensor is advantageous in order to achieve a new distribution of the field of view.

With the method according to the invention for enlarging a field of view of an imaging system with a housing and an image sensor arranged in the housing, the image sensor is rotated as a function of a rotation angle while maintaining the housing in the same orientation.

Preferably, for carrying out the method according to the invention

Image sensor adjustment direction used. In a further embodiment of the method, the imaging system is an imaging system according to the invention.

The camera according to the invention of a driver assistance system of an automatically operable vehicle has an image sensor which is rotated by the method according to the invention. The distribution of the field of view achieved by rotation of the image sensor is in many cases asymmetrical. Thereby, a preference can be formed, e.g. in use for signs or traffic lights, which are often strongly right, especially for countries with right-hand traffic, or for countries with left-hand traffic strongly left of the vehicle.

The invention will be described in detail with reference to the following figures. 1 shows an exemplary embodiment of an image sensor, an exemplary embodiment of a rotated image sensor, an embodiment of an imaging system with an exemplary embodiment of an image sensor adjustment device,

An embodiment of an image sensor setting device, an embodiment of an automated operable vehicle with an embodiment of an imaging system,

a respective combination of image sensor and lens with a course of the usable number of pixels for the same image size,

a schematic representation of the usable number of pixels for the respective combination of FIGS. 5A-5D, a respective combination of image sensor and lens with a course of the usable number of pixels for the same image sensor size, a schematic illustration of the usable number of pixels for the respective combination of FIGS. 7A-7D, and 9A-9C an embodiment for use of an image sensor adjustment device according to the invention or an imaging system according to the invention for real scenarios.

In the figures, like reference numerals designate like or functionally similar reference parts. In each of the figures, the relevant reference parts are numbered.

FIG. 1A shows an image sensor 1 with a receiving surface 4. With regard to a radiation incidence into an imaging system 20, different filters, for example blocking filters for infrared or ultraviolet light, may be arranged in front of the receiving surface 4. Behind the filters, a microlens structure layer may be arranged to concentrate the incident light. This is followed in the known from the prior art image sensors 1 a color filter. In an image sensor 1, which is designed for front side illumination, that is, a front-side exposure, then follow the photodiodes as photosensitive components.

The image sensor 1 has a rectangular format and has an image sensor width 7 and an image sensor height 8 and a diagonal 5. In one known from the prior art arrangement of the image sensor 1 in an imaging system 20, the image sensor width 7 extends horizontally.

FIG. 1B shows the image sensor 1, which is rotated by a rotation angle 2 about a rotation axis 3 which is perpendicular to the reception surface 4. The image sensor 1 in Fig.l B is rotated so that the diagonal 5 is arranged horizontally. As a result, a horizontal angle of view 6 and thus a field of view 22 are increased. Since the diagonal 5 is longer than the image sensor width 7, in comparison with FIG. 1A, the horizontal image angle of the image sensor 1 in FIG. 1B is greater.

FIG. 2 shows an imaging system 20 with a housing 24. The housing 24 has a rear wall surface 25. Adjacent to the rear wall surface 25 is the

Image sensor 1 is arranged. The imaging system 20 is constructed rotationally symmetrical to an optical axis 27. The optical axis 27 is the optical axis of a lens 21. The image sensor 1 is in operative connection with an image sensor unit. tion 10, which has an actuator 11. The image sensor actuator 10 is a miniaturized electric motor, and the actuator 11 is a shaft of this motor that rotates the image sensor 1. The lens 21 of the imaging system 20 has a focal length 9 and a focal point 23. A field of view 22 of the imaging system 20 is limited by the horizontal angle of view 6.

3 shows the image sensor setting device 10 in detail. The actuator 11 is driven by a controller 12. The controller 12 receives a predetermined rotation angle 2 from an input interface 13. The input interface 13 may be a mechanical device, such as a rotary knob operatively connected to the actuator 11 and manually rotatable to adjust the angle of rotation 2, or a digital device , via which a value for the angle of rotation 2 is input. The controller 12 converts the input received via the input interface 13 into a signal with which the actuator 11 is driven.

4 shows an automated operation vehicle 30 having an imaging system 20. The imaging system 20 is a camera of a driver assistance system. This camera is fixedly mounted on a windshield 31 of the vehicle 30 and has an image sensor 1 which can be rotated with the image sensor actuator 10. The camera has a computer with a memory 41 on which, in addition to, for example, reducing noise, image scaling and gamma correction, a computer program product runs to correct a distortion caused by the rotation of the image sensor 1 when the computer program product is loaded into the memory 41.

In FIG. 5A, the receiving surface 4 of the image sensor 1 completely covers a lens image 26. In Fig. 5B, the lens image 26 is only partially covered by the receiving surface 4 of the image sensor. In Fig. 5C, the receiving surface 4 of the image sensor is completely disposed within the lens image 26. Fig. 5D differs from Fig. 5C in that the receiving surface 4 of the image sensor 1 is rotated. From Fig. 5A to Fig. 5C, the size of the receiving surface 4 and thus of the

Image sensor 1 smaller. In Fig. 5D, the size of the receiving surface 4 and the

Image sensor 1 is the same as in Fig. 5C. The lens image 26 remains in Figs. 5A to 5D same size. By reducing the size of the image sensor 1, the pixel density decreases from FIG. 5A to FIG. 5C.

The usable number of pixels associated with the respective figure of FIGS. 5A to 5D is shown in the respective figure of FIGS. 6A to 6D.

FIGS. 7A to 7D each have the same size of the image sensor 1. The size of the lens image 26 is different. As in FIGS. 5A to 5C, the pixel density and thus the resolution are reduced in FIGS. 7A to 7C.

The usable number of pixels associated with the respective figure of FIGS. 7A to 7D is shown in the respective figure of FIGS. 8A to 8D.

Fig. 9A shows an initial scenario. The pedestrians arranged on the left in the image section and the traffic light arranged on the right in the image section are difficult to recognize because they lie in an edge region of the recording format. The area of the traffic light color in the real world has a size of 8 pixels x pixels on the receiving surface 4 of the image sensor 1. In Fig. 9B, the opening angle of the lens is increased as compared with Fig. 9A. The pedestrians arranged on the left in the image section and the traffic light arranged on the right in the image section are now recognizable, but the resolution is reduced. The area of the traffic light color in the real world now has a size of 6 pixels x pixels on the receiving surface 4 of the image sensor 1. In FIG. 9C, the image sensor 1 is rotated in such a way that in the left-hand edge area the pedestrian and in the right-hand edge area the traffic light is respectively detected better than in FIG. 9A. The surface of the traffic light color in the real world has on the 4 receiving surface

Image sensor 1 now again a size of 8 pixels x pixels. Thus, the objects are better recognizable with constant resolution relative to Fig. 9A. Reference symbol image sensor

angle of rotation

axis of rotation

receiving surface

diagonal

angle of view

Image sensor width

Image sensor height

focal length

Image sensor setting device

actuator

control

Input interface

imaging system

lens

field of view

focus

casing

Rear wall

lens image

optical axis

vehicle

Windshield

computer

Storage

Claims

claims

An image sensor positioning device (10) for an imaging system (20), which has a housing (24) and an image sensor (1) arranged in the housing (24)

- An actuator (11) which is designed to enable the image sensor (1) with a constant alignment of the housing (24) in a rotation about a rotation angle (2).

Second image sensor adjusting device (10) according to claim 1, characterized in that

- The actuator (11) is a motor, preferably an electric motor, very particularly preferably a miniaturized electric motor, and

- The image sensor adjusting means (10) comprises a controller (12) which is designed to control the actuator (11) for carrying out the rotation about the rotation angle (2).

3. image sensor adjusting device (10) according to claim 1 or 2, characterized in that the image sensor adjusting device (10) is adapted to rotate the image sensor (1) such that after rotation, a diagonal (5) of the image sensor (1) is arranged horizontally ,

4. imaging system (20) with

a housing (24),

- One in the housing (24) arranged image sensor (1) and

- An image sensor adjusting device (10), wherein

- The image sensor positioning device (10) is designed to rotate the image sensor (1) while maintaining the alignment of the housing (24) for enlarging a field of view (22) of the imaging system (20).

5. An imaging system (20) according to claim 4, characterized in that the image sensor adjusting device (10) is an image sensor adjusting device (10) according to one of claims 1 to 3.

6. imaging system (20) with

- A housing (24) and

an image sensor (1) arranged in the housing (24), - Wherein a receiving surface (4) of the image sensor (1) about an axis of rotation (3) is rotated by a rotation angle (2).

7. Arrangement of an image sensor (1) in a housing (24) of an imaging system (20),

- Wherein a receiving surface (4) of the image sensor (1) about an axis of rotation (3) is rotated by a rotation angle (2)

for enlarging a field of view (22) of the imaging system (20).

A computer program product configured to be loaded into a memory (41) of a computer (40) and having software code portions adapted to receive an image sensor (1) disposed in a housing (24) while maintaining the same orientation Enclosure (24), resulting distortion of an image of an imaging system (20) is corrected when the computer program product is running on the computer (40).

The imaging system (20) of any one of claims 4 to 6 including a computer (40) on which runs the computer program product of claim 8.

10. Use of an imaging system (20) according to any one of claims 4 to 6 or 9 for an automated operable vehicle (30) as a camera of a driver assistance system.

11. A method for magnifying a field of view (22) of an imaging system (20) having a housing (24) and one in the housing (24) arranged

Image sensor (1), wherein the image sensor (1) in dependence of a rotation angle (2) is rotated while maintaining the alignment of the housing (24).

12. The method according to claim 11, characterized in that for carrying out the method, an image sensor adjusting device (10) according to one of claims 1 to 3 is used.

13. The method according to claim 11, characterized in that the imaging system (20) is an imaging system (20) according to any one of claims 4, 5 or 9.

14. Camera of a driver assistance system of an automatically operable vehicle (30), wherein an image sensor (1) of the camera is rotated by a method according to one of claims 11 to 13.