DE19539642A1 - Visualisation method for monitoring system in vehicle - Google Patents

Abstract

The method involves visualising a not directly visible space particularly in a vehicle (10). An image of the space is picked-up by a video camera (1) and transferred to a screen connected through a data line (2). The method uses a camera lens with an extremely wide-angle objective, and a semiconductor image sensor with pixels arranged in columns and rows. An allocation function associates pixel values describing luminance or chrominance on the actual image plain. The allocation function is implemented in semiconductor logic, and is performed in real time, based on the necessary correction due to the geometric distortion of the image by the wide-angle lens.

Description

The invention relates to a method for visualization a surveillance room that is not immediately visible especially in the case of a vehicle of the genus An saying 1. Furthermore, the invention relates to a front direction to visualize a not immediately visible surveillance room.

The problem of visualizing a not immediate Visible surveillance room is widely known. A common example is the invisibility of the back space of vehicles when parking, which leads to development development of active perception systems based on In led infrared light and / or ultrasound.

An alternative are video cameras that also already used on commercial vehicles, cf. Fig in FZ ACE steering wheel 5/91 on p. 60. But also in Passenger vehicles would use such cameras desirable because of the increasingly scarce parking space For example, driving in cities with centimeter precision  demands.

In the case of a passenger car, for example the rounded and retracted rear geometries are more modern Vehicle designs require wide angle lenses for this. However, these have the disadvantage that they are the driver from a certain object angle a difficult to interpret deliver a clear picture. They can also have blind spots do not completely eliminate.

With regard to the aforementioned problems in a vehicle A solution is known from FR 2 673 499 A1. There are inside the vehicle envelope contour - namely behind glazing in the area of the right and left Taillights - motorized around a vertical axis Nuably swiveling television cameras used, so position-dependent drawing files by swiveling the Cameras with a total visible rear area on a distance to display visual screens in the area of the dashboard len. This way, camera lenses can be used with relative low viewing angle. For that, however a very high mechanical and control technology wall for pivoting the cameras as well as the circumstance in Be bought that the cameras scream over soiling of said glazing again and again "go blind". The total space requirement of both came ras including the means for pivoting them into not insignificant within the vehicle shell.

A similar solution aimed at a vehicle that only works with a single video camera the preliminary application P 43 36 288.5. There becomes Back room observation a video camera with relatively little Obstacles in terms of fuzzylogic focus  pivoted.

In contrast, it is an object of the invention, both a Process for visualizing a not immediately visible surveillance room especially when driving stuff and a device for visualizing a not to propose an immediately visible surveillance room, which equally overcome the aforementioned disadvantages.

The first task is through a generic Ver drive with the characteristic features according to claim 1 solved.

This procedure allows only a single video camera in a fixed working position with only one lens in ver binding with a picture that takes up little space use umbrella.

For this purpose, the method sees the use of a camera object tivs previously unusually large object angle. At the same time the procedure takes into account the fact that Weitwin Kel lenses with an object angle of z. B. 160 ° he captured object space on the image plane of a sensor chip map recorded.

Listed here means that according to the optical Physics of such a lens on the image plane Change pixels in the radial and tangential direction are different because light beams pass through the object tiv with respect to their directions of propagation Experienced. Furthermore, each light beam pixel has his own image height in the image plane. M.a.W. is the Image scale depending on the image height. It is below the image height is the distance of the light beam image point in the image plane from that in the image plane  understand the origin of the coordinates. In the present In the case only lenses are used that are in high Dimensions are rotationally symmetrical, so that the tangential Distortion compared to the resolution of the image sensor is negligibly small in the image plane. As a result the image points in the image plane are decisive Lich in the radial direction according to the index voltage curve is shifted non-linearly, causing the image distortion in the image plane on the image sensor and therefore for a quick cognitive in formation of an observer is unsuitable.

According to the method, interpolation-free pixels are used distribution a coordinate-image transformation of the object actually created on the image sensor of a camera image in a cognitively better by an observer perceptible display image on a screen causes.

This transformation takes place in the course of signal transmission and data format conversion between camera and screen. According to the method, in the video camera, in addition to a lens with said large object angle, the image sensor is a semiconductor image sensor with a first defined number of pixels (image field parameters of the sensor), which is defined in rows and columns, and a screen with a second one defined as a screen , number of pixels (image field parameters of the screen) arranged in a defined manner in rows and columns. Corresponding to the object space mapping in the real image plane on the image sensor, each pixel from said first number of sensor pixels will describe the brightness or brightness, color saturation and color type however (e.g. in the form of RGB values) F _V and in the display image plane on the screen each pixel from said second number of screen pixels is assigned a corresponding value F _M. Then the value F _M of each pixel currently to be written on the screen is derived from the value F _V of that sensor pixel to which the screen pixel points due to a geometric mapping transformation, this mapping transformation in real time by means of a the transformation rule is hard-wired semiconductor logic and the image transformation on the basis of the image field parameters of both the image sensor and the screen requires at least one category of the object-specific geometrical-optical distortion of the real object space image (on the image sensor) from the desired object space representation (on the screen) corrected by changing the assignment of sensor pixels to screen pixels.

This process is characterized by process steps according to the Features of the dependent claims 2 to 8 advantageous trainable. So with different procedures accordingly different types or progresses levels of coordinate image transformation up to for the quasi-synthesis of vertical and horizontal images be realized. It is advantageous that in the case several transformation steps in the shortest possible time possible time are executable because they do not follow each other different, but at the same time in the manner of a total re sponse per image content.

Based on at least one electronically generated and only in this form existing coordinate transfer the image according to the invention performs  the visualization of a not immediately visible baren surveillance room in one for one Driver cognitively light and effortless ver workable form of presentation.

Therefore, the method according to claim 10 can be special advantageous in a passenger motor vehicle use Find. The method allows optical detection solution z. B. the delimiting the rear driving space Rear lines of a passenger car with no blind spot in terms of obscuring objects through the stern bumpers.

The second task is accomplished by a device according to the independent claim 11 solved.

The device is essentially functionally based to a semiconductor circuit implemented as an ASIC (preferably with two functional areas or assembled from two less complex ASICs) and a single video camera in a fixed operating position. This has the advantages that the device works without a time delay that affects the reaction, d. H. allows an observer real-time assessment and its image signal processing device as essential Licher component inexpensively and with low ab measurements can be mass produced. Furthermore possible light the representation of a very robust overall systems without moving parts and a space saving integration especially of the visible part z. B. in the area already with displays and controls elements overloaded dashboards.

Advantageous further developments of the device are  according to the teaching of dependent claims 12 to 20 possible Lich.

The device can e.g. B. Means of Guarantee a defined fixed position of the camera during its Operation include. For example, the camera by means of an extension mechanism from a fixed rest position in a fixed operating position the one where it then remains during its operation. In a vehicle, it may include means that the latter z. B. when engaging reverse gear effect automatically.

One embodiment of the device and the Procedures are illustrated in the figure drawing light, which is explained below. Thereby the Illustration using the practical example of a per no limitation of the invention mean since these - with appropriate constructive Adaptation to the respective application - also is applicable in other areas.

Show it:

Figure 1 is a schematic illustration of the Anord voltage of a device according to the invention in a passenger vehicle for the purpose of a view of the vehicle rear space.

FIG. 2 shows a schematic illustration of the fixed operating position of the video camera in the device according to FIG. 1;

Fig. 3 is an illustration of the method according to thus visible rear area by the example of a sonen motor vehicle;

Fig. 4 is a schematic functional block diagram of the electronic components of the device;

Fig. 5 is a schematic illustration to the invention for visualizing OF INVENTION method;

6 shows the distortion illustration of the first step of the method of visualization Streckenent.

Fig. 7 is an illustration of the second step of the visualization method for generation of a winkelentzerrten base image;

Fig. 8, the illustration of the third step of the visualization method of generation from a perspective corrected image of the basic image.

Referring to FIG. 1 of a vehicle 10 at is integrated into a play of drive or swing unit 1.1, a video camera 1 at the rear. The configuration shown illustrates the fixed operating position II of the video camera, which looks at play at an angle of attack Φ to the vertical of 45 ° down into the rear area behind the vehicle.

The camera is connected via a data cable 2 to an image signal processing device 3 which contains a logical real-time image transformation module 4 . The image signal processing device 3 controls a screen 5 . Furthermore, the image signal processing device 3 via a line 6 with a not shown, for. B. the engagement of the reverse gear detecting switch or sensor z. B. are connected to the transmission of the vehicle. This can, for. B. when engaging the reverse gear, the image signal processing device 3 is switched on and an automatic movement of the video camera into its operating position II can be controlled. The switching on of the screen 5 can also be triggered thereby.

In Fig. 2, the position of the video camera 1 in relation to the vehicle 10 is illustrated both in a resting position I and in its operating position II. Accordingly, in the operating position II there is an angle of attack Φ of, for example, 45 ° between the vertical 11 and the optical axis 12 of the video camera 1 . The extendable or swiveling unit 1.1 and the data cable 2 are only indicated symbolically. A vehicle-compatible device can in any case include all means within the scope of the invention which the video camera spend in a protected idle position when not in use and in a defined operating position for use.

Without restricting generality, the video camera can also be mounted in a fixed position in the example of a motor vehicle, and z. B. Means may be provided to control and actuate a movable lens cover so that it releases the lens only in the event of an operation. Accordingly, z. B. when inserting the Rückgangsgan ges the image signal processing device 3 is turned on and the release of the lens of the video camera can be effected.

According to Fig. 3 with a lens with object angle Θ = viewable rear space 160 ° by way of example illustrates behind the rear face 13 of a passenger car as a chamber body 14 having a certain pulp te B, a room depth T, the vehicle side has a height H1 and fahrzeugabseitig a height H2 ; the case H1 = H2 is illustrated. Due to the large object angle of the camera and its supply in a storage position out of the vehicle envelope contour II, only an extraordinarily small, marked by hatching ge made part 15 of the body 14 is not visible.

According to FIG. 4, the electronics of the device substantially comprises the video camera 1 preferably with a color image pickup sensor 1.2, an analog-Digi talwandler 1.3 with a downstream parallel-serial converter 1.4 with, for example, twisted pair or glass fiber output, further, the Data line 2 leads out, for example, as a twisted pair or in glass fiber technology, and the image signal processing device 3 which controls the screen 5 .

Depending on the version, parts 1.3 and 1.4 can be combined functionally and physically integrated.

The image signal processing device 3 comprises a corresponding serial / parallel converter 3.1 with a corresponding twisted pair or glass fiber input, which here controls two cascaded ASICs 4.1 and 4.2, for example.

Depending on the technology used, these two ASICs can also be implemented on a chip and are the essential components of the real-time image transformation module 4 . The image signal processing device 3 further comprises a RAM 4.3 and a Mikrocom computer 4.4 . The microcomputer and RAM are connected to one another and to the transformation module 4 . The ASIC 4.2 has a digital RGB video output which controls the screen 5 in a manner known per se.

The function of the device is as follows. The image of the three-dimensional rear space falling on the CCD sensor 1.2 according to the rear space body 14 is converted into a two-dimensional charge image in this sensor, which is emitted, for example, 50 times per second in a half-frame manner. The image signal obtained in this way is generally available in analog form and is then converted into a serial data signal. For this purpose, the converter 1.3 first performs an analog / digital conversion, and the parallel-serial converter 1.4 converts the parallel digital signal thus obtained into a serial data stream, which is transmitted in the form of a fast digital two-wire signal or by light to the image signal processing device 3 and there is converted back into a parallel data stream in the serial / parallel converter 3.1 .

Using the RAM 4.3 as image memory between two consecutive (half) images under timing control by the microcomputer 4.4 . the resulting parallel "picture words" are loaded into the ASIC 4.1 and then subjected to an incremental re-sorting, for example in accordance with the procedural rules outlined below, for the purpose of changing the picture geometry.

The rule of this resort is exactly on the unwanted distortion of the camera lens in the sense an image distortion and perspective correction Voted. It can also under special conditions a rectangular ratio of pixel cells of the screen be taken into account. In this regard, the ASIC The area can also have at least two different Koor dinate image transformation types or levels correspond appropriate and correspondingly different range "Wiring schemes" causing selective activation be realizable and it can then be dependent different from a selection signal Reordering can be activated.

It is essential that the microcomputer is there prefers a pure control function without its own Direct participation in the signal processing of the picture can realize content.

For example, fifty screen color fields with 564 pixels horizontal and 224 pixels vertical image width per second are generated in this way. At the same time clocks the microcomputer 4.4 which he testified image words is adapted from the ASIC 4.1 by the ASIC 4.2 latter which as in the ASIC 4.1 that it converts this image words into a digital RGB video output signal, which thus directly a accordingly controllable screen 5 is feedable.

This configuration and mode of operation of the real-time image transformation module 4 enables the transformation of each pixel in a very short time, for example within a maximum of 150 nanoseconds, since the transformation does not come about as usual via a microcomputer, but rather due to the respective image deviations of the lens used Fixed wiring or programming of ASICs from the usual visual impression.

The following more detailed description of the method using the example of the rear area visualization in a motor vehicle is first preceded by FIG. 5 to illustrate the deviation from an optically realistic image mentioned at the beginning. In the following, each individual pixel of an image - whether real or fictional - is assigned a data record F, for example in the manner of a vector, which clearly describes the brightness, the color saturation and the color type of the respective pixel, in the following simplifies it Called "F value" at the pixel location. The respective picture can thus be understood as a set of all F with coordinates running through according to the size, shape and (directional) resolution of the respective picture.

According to each individual pixel of the real image plane generated by the wide-angle lens 1.5 in the real image plane - ie on the surface 1.2.1 of the image sensor 1.2 of the camera 1 - a correspondingly recorded data record F _{V is} assigned, which brightness, color saturation and Describes the color type of the pixel clearly, referred to below as the "F _V value" at the pixel location (y, z) on the surface 1.2.1 of the image sensor 1.2 . So that the real, ver recorded image can be described as a set of all F _V with continuous pixel coordinates y and z.

This real, recorded picture is called DELTA net. With regard to this wide-angle illustration, the following now applies the inadequacy mentioned at the beginning that in the Image plane the pixels in radial and tangential Direction are changed because light beams pass through through the lens with respect to their direction of propagation experienced distractions. Furthermore everyone has light bundle pixel its own image height in the image level. M.a.W. the image scale depends on the image height. The Ab is below the picture height the light beam image point in the image plane from the coordinate origin lying in the image plane to understand. Because in the present case only lenses used, which are highly rotationally symmetrical trical, the tangential distortion is counter above the resolution of the image sensor in the image plane negligibly small. As a result, the Pixels in the image plane largely in radial Direction according to the distortion curve of the object tivs shifted non-linearly causing the image to DELTA the image sensor has a distortion and therefore for quick cognitive information of an observer aft is unsuitable.

Starting from this real, optically unusable figure, FIG. 6 illustrates the first part of the process, namely that of "electronic route equalization" for the purpose of obtaining a non-real, but only electronically existing, route-corrected image.

This first sub-step is equivalent to or  causes the "synthesis" of a single project center projection for all straight lines through object and image point.

Accordingly, the lower part of FIG. 6 corresponds to the image conditions already explained in connection with FIG. 5 in the camera. The upper part of FIG. 6 illustrates the result achieved by the first sub-step of the method.

Accordingly, a distortion-free image E is formed by means of a distance equalization function E, which preferably comprises two individual functions E 1 and E 2 for two coordinate directions, from the recorded sensor image DELTA, which exactly corresponds to the image of a theoretically distortion-free lens with object angle Θ z. B. 160 °, which in turn - vice versa - speaks for image E the existence of a single projection center 1.10 for all straight lines 1.9 'from the object space by means of ent.

The "F value"(brightness; brightness, color saturation, color type; RGB values) of a pixel in the image plane of the rectified image E with the coordinates (y ′, z ′) is denoted by F _E (y ′, z ′) . Thus, the rectified image E can be described as the total set of all F _E with pixel coordinates y 'and z'.

Seen from image E - ie in the reverse direction - there is the two-axis transformation

E = E₁, E₂ (2)

the relationship between the coordinates of mutually corresponding F values in the images DELTA and E in accordance with the transformation regulation

y = E₁ (y ′, z ′; k₁,... k _n ),
z = E₂ (y ′, z ′; k₁,... k _n ) (2.1)

In E all parameters k₁,. . . k _n the real sensor range lens - image sensor - digitizer up to and including 1.3 such as B.

• - Radial and tangential distortion of the lens;
• - Point of penetration of the optical axis of the lens through the real image plane of the image sensor;
• - Inclination of the optical axis of the lens Lich the real image plane of the image sensor;
• - line scanning grid of the digitizer,
• - and the same

linked via individual mathematical functions for E₁ and E₂. The parameters k 1,. . . k _{n of} the sensor path can be determined. The physics of the lens descriptive functions E₁ and E₂ are unambiguous and continuously differentiable. With regard to the defined sensor pixel order, they can nevertheless also be present as functions defined only at discrete locations, for example comparable to a table.

The geometric link between the stretched image E and the originally distorted real image DELTA on the image sensor is given by:

(points to)
F _E (y ′, z ′) - - - E₁, E₂ → F _V (y, z), or (2.2)

The direct representation of the rectified image E can be achieved with lenses with object angles Θ of z. B. less than 160 ° can be quite interesting. In the other case, the resulting image E does not really exist as such, but only purely electronically.

The pointing function from F _E to F _V means that on a (in this transformation step) real or fictional screen for the image E, a gapless pixel grid is written from pixels that are horizontally adjacent to one another and thereby form lines, or from vertically successive lines and in each case the F value of the E pixel used is that value which has the DELTA source point which is fixed in the image plane of the distorted real image DELTA via the transformation E 1, E 2. Ultimately, this means that even on a real or fictitious screen, there can already be no "omitted pixels" for the image E , since the transformation ultimately results in the regular "pixel grid" of the image E on such a screen from a selection of the route equalization transformation Pixels of the real image DELTA are assembled on the image sensor 1.2, so the latter do not necessarily have to be adjacent to one another or have to be connected in succession.

The result is this coordinate image transformation straightforward.

However, for a wide-angle lens with an object angle Θ of z. B. 160 ° or even greater the visual visual impression of the rectified image E not yet human viewing habits. This gives rise to striking effects and, in particular, a rough grid in the peripheral zones. Furthermore, objects that are at different depth positions in the object space are reproduced with such large object angles in the distance-rectified image E with incorrect size ratios to one another. For this reason, the rectified image E for lenses with very large object angles Θ of z. B. 160 ° or even less suitable as an information image on a screen for a vehicle driver.

As illustrated in FIG. 7, a further transformation W, which in turn comprises two direction-dependent components W 1 and W 2, can be used to generate a further angle-corrected basic image W from the previously stretched image E ;

W = W₁, W₂ (3)

The "F value"(brightness; brightness, color saturation, color type; RGB values) of a pixel in the image plane of the angle-corrected image W with the coordinates (α, β) is designated as F _W (α, β).

The angle-corrected image W can thus be described as the total of all F _W with pixel coordinates α and β. There is a right-angled coordinate system with the axes α and β and each F-value of the image W with the coordinates (α, β) shows the transformation rule

y ′ = W₁ (α, β) = f _* tan (α)
z ′ = W₂ (α, β) = f _* tan (β) (3.1.1)

with f = image width
to the coordinate location (y ′, z ′) of the F value of the corresponding pixel F _E (y ′, z ′) in the rectified image E or via the coordinate image transformation rule

y = E₁ [W₁ (α, β), W₂ (α, β); k₁,. . . k _n )
z = E₂ [W₁ (α, β), W₂ (α, β); k₁,. . . k _n ] (3.1.2.)

to the coordinate location (y, z) of the F value of the corresponding pixel F _V (y, z) in the originally recorded real image DELTA.

Overall, the transforma carried out up to that point tion can be described as follows:

The above total transformation G₁, G₂ is in all mean not straightforward. However, she does fol gendes:

• - All object space lines that run vertically or horizontally in an object space plane that is parallel to the y / z plane are mapped vertically and horizontally true to the straight line. The amount of this straight line of object space is designated G _SW .
• - Furthermore, those straight lines of the object space are mapped in a straight line that run in the planes that span the individual straight lines that form the set G _SW with the projection center 1.10 ;
• - The image W corresponds to human viewing habits. It is suitable for display on a screen for the visualization of a space that is not immediately visible, for example a vehicle rear space for a vehicle driver. In the case of a car, the free space required for staggered rear parking can thus be monitored relatively well from the bumper as shown in FIG. 3.

Starting from this transformation step with the result of an angle-corrected image W , an electronically perspective-corrected image K can be obtained by a method step illustrated in FIG. 8, which - alternatively - a vertical image (vertical viewing direction) or a horizontal image (horizontal viewing direction) can be. The perspective is understood as the viewing direction of the lens.

The same applies to this transformation

K = K₁, K₂ (4)

The "F value"(brightness; brightness, color saturation, color type; RGB values) of a pixel of the perspective-corrected angular equalization image of the K with the coordinates (α ′, β ′) thus obtained is given by F _K (α ′, β ′ ) designated. The perspective-corrected angle-corrected image K can thus be described as the total amount of all F _K with pixel coordinates α ′ and β ′.

Seen from the image K - ie in the reverse direction - the transformation K = K₁, K₂ indicates the relationship of the coordinates of the same F value in the images W and K.

The image K has a right-angled coordinate system with the axes α ′ and β, and each F-value of K with the coordinates (α ′, β ′) shows by virtue of the transformation rule

regulation on the coordinate location (α, β) of the F value of F _W (α, β) in the image W or via the coordinate image transformation

y = E₁ [W₁ [K₁ (α ′, β ′), K₂ (α ′, β ′)],
W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ],
z = E₂ [W₁ [K₁ (α ′, β ′), K₂ (α ′, β ′)],
W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ] (4.1.2)

to the coordinate location (y, z) of the F value of the corresponding pixel F _V (y, z) in the originally recorded real image DELTA.

Overall, the entirety of all Transforma can steps up to here are described as follows:

The transformation K = K₁, K₂ makes within the overall transformation G₁, G₂ according to (4.2) ultimately the effect of an "electronic pan" of the camera including the image sensor about an axis through the projection center orthogonal to the clamping plane of the angle of attack illustrated in FIG. 2 Φ. In the exemplary embodiment according to FIG. 2, the video camera is, for example, in a fixed operating position in the rear cover of a car, its optical axis having the angle of attack Φ, which is 45 ° by way of example.

.. 8 effected with regard to Figures 7 and the Trans formation (4) - (4.1.1) with an unchanged fixed camera angle Φ with a rotation of the image K in relation to the image W orthogonal to an axis through the center of projection to 1.10 Clamping plane of the angle of incidence sicht visible from FIG. 2 such that the new fictitious optical axis of the image K points vertically downward, with the result that the fictitious plane of the image K then extends horizontally. W is generated from K in the reverse direction.

The angle equalization according to process step (3) - (3.1.1) / (3.1.2) - (3.2.) - (3.3) applied to this fictitious image W thus results in a vertical image with a fictitious optical axis vertically downwards in the overall transformation . That is, although the camera and its image sensor are not positioned vertically downwards at all, it allows the process to synthetically prepare a vertical image from the image actually obtained in the y / z coordinates by prior to the transformation according to (3) - (3.1.1) / (3.1.2) - (3.2.) - (3.3) the angles α and β of the transformation K = K₁, K₂ according to (4) - (4.1.1) are subjected.

The total transformation G₁, G₂ according to (4) - (4.1.1) / (4.1.2) - (4.2) - (4.3) is generally not a straight line  faithful. Overall, however, it does the following:

• - All straight lines of the object space, which run with reference to FIGS. 2 and 3, for example on a flat vehicle surface or in parallel planes parallel or perpendicular to the longitudinal direction of the vehicle, are depicted true to the vertical and horizontal. The amount of this straight line of object space is designated G _SW .
• - Furthermore, those object space lines are also shown true to the straight line that run in the planes that span the individual straight lines that form the set G _SW with the projection center 1.10 .
• - The image K conveys on a screen electronically the parallax-free viewer spotting "view vertically down" example, in a vehicle rear space for a vehicle driver in the embodiment according to FIGS . 1 to 3.
• - Since the image W offers a cognitively good spatial overview in the manner of a panoramic image, but less good overview of the distances to obstacles, is optionally provided as part of the visualization process or in the case of a device performing this, when approaching / from obstacles to switch between the panoramic image W and the parallax-free vertical image K.
• - By a special non-linear scaling of the coordinate axis β ', a varied image K is obtained , which has distance fidelity with respect to the distances on the road surface in the longitudinal direction of the vehicle or in the clamping direction of the angle Φ. This is therefore a further training of the overall transformation G₁, G₂ according to (4) - (4.1.1) / (4.1.2) - (4.3)

The alternative representation of a correspondingly angular distorted and perspective corrected horizontal image is also possible.

For this purpose, the value F _K of each pixel (α ′, β ′) of the (horizontal) image K corrected by perspective is derived from the value F _{W of} that pixel (α, β) of the (perspective non-corrected) image W to which the pixel (α ′, β ′) by the transformation

K = K₁, K₂ (5)

according to the transformation regulation

shows.

The value F _K can also be derived from the F value of the corresponding pixel F _V (y, z) in the real image DELTA originally recorded, to which the pixel (α ′, β ′) by the coordinate image transformation

y = E₁ [W₁ [K₁ (α ′, β ′), K₂ (α ′, β ′)],
W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ],
z = E₂ [W₁ [K₁ (α ′, β ′), K₂ (α ′, β ′)],
W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ] (5.1.2)

shows according to

From this it can be seen that in a work positioning of the video camera with a viewing angle of 45 ° downwards, particularly simple conditions with regard to the alternative derivation of the two images of the K result.

With regard to FIG. 7 and FIG. 8, such a transformation (5) - (5.1.1) results in a rotation of the plane of the image K about an axis through the projection center 1.10 orthogonal to the clamping plane of the in FIG Fig. 2 illustrated clear angle of attack Φ so that the new fictitious optical axis of the image K horizontally away from the vehicle (towards th) shows with the result that the fictitious plane of the image K then extends vertically. W is generated from K in the reverse direction.

The angle equalization according to process step (3) - (3.1.1) / (3.1.2) - (3.2.) - (3.3) applied to such fictional image W provides a horizontal image with a fictional one optical axis horizontally away from the vehicle. Although the Camera and its image sensor not at all with viewing direction are positioned horizontally away from the vehicle the method based on that in the y / z coordinates a real image received a horizontal image on a Display monitor by according to the transformation (3) - (3.1.1) / (3.1.2) - (3.2.) - (3.3) the angles α and β of the Transformation K = K₁, K₂ according to (5) - (5.1.1) subjected will.

The aforementioned partial transformations themselves understandable no limitation of the invention, son selected step examples for how a general coordinate image transform according to the invention tion

A = A₁, A₂ (1)

with orthogonal transformation components

y = A₁ (a ′, b ′) and
z = A₂ (a ′, b ′) (1.1)

as an "over-all step" of the real location

(points to)
F _M (a ′, b ′) - - - A₁, A₂ → F _V (y, z) (1.2)
F _M (a ′, b ′) = F _V [A₁ (a ′, b ′), A₂ (a ′, b ′)] (1.3)

can be designed in detail, whereby in this general description of the method F _M the "F value" (at least brightness in a black / white system, but preferably also color saturation and color type or RGB value) of a pixel in the a ' / b'-image plane on the screen and F _V the "F value" (at least brightness in a black / white system, but preferably also color saturation and color type or RGB value) of a pixel in the y / z DELTA plane on the sensor chip corresponds to the camera and the transformations N₁ and A₂ indicate which coordinates (y, z) on the image sensor in the camera the F-value to be written on the screen at the coordinates (a ′, b ′) on the screen The overall transformations G₁, G₂ indicated under (2.3), (3.3) and (4.3) are exemplary special cases of the general transformation A₁, A₂ mentioned above.

Such an allocation transform to be hardwired mation is usually beyond the specific un desired image distortion also the image field parameters of the image sensor and the screen (image aspect ratios or horizontal and vertical in frame number of useful pixels) and in practice advantageously be designed so that they at least a category of the majority of objective-specific specific geometric-optical distortions of the real  available object room image of the desired Ob display space optimally corrected, the Kate with different levels of corrections can choose.

In any case, the inherent procedural advantage remains effective that the compensation of mapping insufficient possibilities by pixel reallocation so-called "indefinite Pixels "on the screen and all that occurs Avoids consequential problems and therefore no effort for electronic means of filling or interpolation "worthless" pixels must be driven.

It goes without saying that the subject of Invention neither in terms of the method nor related Lich leaves the device if, for example a simplified black and white transmission technology and the F-values in this case then to brightness level values to reduce.

The method thus opens a path to devices to visualize a not immediately visible barer surveillance room with work in real time the image signal processing devices with gerin working quickly, small in size and effort inexpensive to implement. Corresponding Vorrich are suitable for. B. also for the inspection of Passways in passenger terminals, from lower lying the entrance rooms at a POS terminal, etc., where in other cases, of course, other image rectifications than necessary or appropriate for a vehicle could be.

Claims (20)

1. A method for visualizing a surveillance space that cannot be viewed directly, in particular in a vehicle, in which method a video camera assigned to the surveillance space and, separated from it, a screen arranged in an observation space is used, a signal transmission taking place between the video camera and the screen and the surveillance space ver is optically imaged on an image sensor using the lens of the video camera,
characterized,

• - That in the camera as an lens an extreme Weitwin lens and as an image sensor a semiconductor sensor with a first defined, defined in rows and columns th number of pixels (image field parameters of the sensor) and as a screen such with a second defined , number of pixels (image field parameters of the screen) defined in rows and columns is used;
• - That, according to the object space mapping DELTA in the real image plane on the image sensor, each DELTA pixel (y, z) corresponding to the first number of sensor pixels describes at least the brightness, but preferably also the color type, brightness and color saturation or RGB values Value F _V and a corresponding value F _{M is} assigned to each screen image pixel (a ′, b ′) from said second number of pixels in the display image plane on the screen,
• - That the value F _M of each screen image pixel currently to be derived is derived from the value F _V of the sensor pixel to which the screen image pixel by transformation A = A₁, A₂ with the transformation rule = A₁ (a ′, b ′ ) and
  z = A₂ (a ′, b ′) according to F _M (a ′, b ′) - - - A₁, A₂ → F _V (y, z),
  F _M (a ′, b ′) = F _V [A₁ (a ′, b ′), A₂ (a ′, b ′)], and
• - That this assignment takes place in real time by means of a semiconductor logic which is hard-wired according to the transformation specification and the image transformation on the basis of the image field parameters of both the image sensor and the screen has at least one category of object-specific geometrical optical distortion of the real object space image (on the image sensor) of the corrected the desired object space representation (on the screen) by changing the assignment of sensor pixels to screen pixels.

2. The method according to claim 1,
characterized,

• - That the aforementioned image transformation comprises at least one line rectification step such that the value F _E of each pixel (y ', z') of the line rectified image E is derived from the value F _{V of} that DELTA pixel (y, z) to which the E pixels by the transformation E = E₁, E₂ according to the transformation rule y = E₁ (y ′, z ′; k₁,... K _n ),
  z = E₂ (y ′, z ′; k₁,... k _n ) according to F _E (y ′, z ′) - - - E₁, E₂ → F _V (y, z), where with k₁,. . . k _n as parameters of the real sensor path lens - image sensor - digitizer ( 1.5 , 1.2 , 1.3 ) and E₁ and E₂ as unambiguously and continuously differentiable lens description functions, which are defined in terms of the defined sensor pixel order at least at corresponding discrete locations or value available

3. The method according to claim 2,
characterized,

• - That the image transformation further comprises at least one angular equalization step such that the value F _W of each pixel (α, β) of an angularly equalized image W is derived from the value F _V of the DELTA pixel (y, z) to which the pixel the image W by the coordinate image transformation with the partial transformation W = W₁, W₂ according to the coordinate biald transformation rule = E₁ [W₁ (α, β), W₂ (α, β); k₁,. . . k _n ]
  z = E₂ [W₁ (α, β), W₂ (α, β); k₁,. . . k _n ] shows according to F _W (α, β) - - - W₁, W₂ → F _E (y ′, z ′) - - - E₁, E₂ → F _V (y, z),
  in which

4. The method according to claim 3,
characterized,

• - That the video camera is operated with respect to the vertical at an angle of incidence Φ and the image transformation includes a step for perspective transformation into a vertical image such that the value F _K of each pixel (α ', β') of the perspective corrected (vertical -) Image K is derived from the value F _{W of} that pixel (α, β) of the (perspective non-corrected image of the W to which the pixel (α ', β') by the transformation K = K₁, K₂ according to the transformation rule shows, or is derived from the F value of that pixel F _V (y, z) in the originally recorded real image DELTA to which the pixel (α ′, β ′) by virtue of the coordinate image transformation = E₁ [W₁ [K₁ ( α ′, β ′), K₂ (α ′, β ′)],
  W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ],
  z = E₂ [W₁ [K₁ (α ′, β ′), K₂ (α ′, β ′)],
  W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ] shows according to

5. The method according to claim 3,
characterized,

• - That the video camera is operated with respect to the vertical at an angle Φ and the image transformation includes a step for perspective transformation into a horizontal image such that the value F _K of each pixel (α ′, β ′) of the perspective-corrected (horizontal) Image K is derived from the value F _{W of} that pixel (α, β) of the (perspective non-corrected image W on which the pixel (α ′, β ′) by the transformation K = K₁, K₂ according to the transformation regulation shows, or is derived from the F-value of that pixel F _V (y, z) in the originally recorded real image DELTA to which the pixel (a ′, b ′) by virtue of the coordinate image transformationy = E₁ [W₁ [K₁ ( α ′, β ′), K₂ (α ′, β ′)],
  W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ],
  z = E₂ [W₁ [K₁ (α ′, β ′), K₂ (α ′, β ′)],
  W₂ [K₁ (α ′, β ′), K₂ (α ′, β ′)]; k₁,. . . k _n ] shows according to

6. The method according to claim 3,
characterized,

• - That the image W is displayed.

7. The method according to claim 3,
characterized,

• - That the image W only exists fictitiously as an (electronic) intermediate image and as such is further processed before being displayed on the screen in at least one further process step.

8. The method according to claim 4,
characterized,

• - That the execution of the transformation K is optionally provided and can be called up optionally from the observation room near the screen.

9. The method according to claim 5,
characterized,

• - That the execution of the transformation K is optionally provided and can be called up optionally from the observation room near the screen.

10. Use of the method according to claim 1 with a vehicle. 11. Device for the visualization of a non-immediately visible surveillance room, with a video camera assigned to the surveillance room and detached therefrom a screen arranged in an observation room, a signal transmission path being arranged between the video camera and screen and the surveillance room by means of the lens of the video camera on an image sensor is optically mapped,
characterized,

• - That the video camera includes a device for digitized transmission of the image information over the signal transmission path and an orientation of its optical axis (viewing direction) inclined relative to the vertical by an angle of attack,,
• - That the lens of the video camera is one of extreme wide-angledness and the image sensor is a semiconductor image sensor with a first defined number of pixels (image field parameters of the sensor) arranged in rows and columns and as a screen one with two number of pixels defined (defined in rows and columns) (image field parameters of the screen) is provided,
• - That an image signal processing device is provided in the signal transmission path between the video camera and the screen, which includes a microcomputer and a RAM area and associated at least one application-specific semiconductor circuit function (ASIC) within which, under time control by the microcomputer, a geometric equalization of the on Image sensor generated object space image in a screen image closer to a desired representation by a pixel-wise incremental, hard-wired to the imaging properties of the lens hard-wired change in the assignment of corresponding pixels on the image sensor and the screen in the sense of a re-sorting is possible.

12. The device according to claim 11,
characterized,

• - That in the RAM area for the purpose of incremental re-sorting of pixels at least part of a full sensor image, for example a half image, can be buffered.

13. The apparatus of claim 11,
characterized,

• - That it comprises two functionally separate application-specific semiconductor circuit functions (ASIC areas 4.1 and 4.2 ), one of which the said resorting and the other the generation of a digital RGB video output signal for the screen.

14. The apparatus according to claim 11,
characterized,

• - That in the re-sorting ASIC area at least two different transformation stages corresponding and accordingly different re-sorting effecting "wiring schemes" are selectively activated and depending on a selection signal one of them is effective.

15. The apparatus according to claim 11,
characterized,

• - That the microcomputer only fulfills control functions and no processing function bezüg Lich image content and geometry.

16. The apparatus of claim 11,
characterized,

• - That it includes means which at least cover the lens of the video camera when not in use and are usable for the use of the latter in a defined position Ar.

17. The apparatus of claim 11,
characterized,

• - That it includes means which, when not in use, bring the video camera into a protected rest position and for the same use bring it into a defined operating position.

18. The apparatus according to claim 17,
characterized,

• - That the video camera is at least pivotally arranged such that it is in the rest state within the envelope of the object in which the observation room is located.

19. The device according to claim 18,
characterized,

• - That the object is a vehicle.

20. The device according to claim 19,
characterized,

• - That it is arranged in the rear of the vehicle and comprises means which cause the video camera to be automatically moved into its position of use when the reverse gear is engaged.