WO2008037392A1

WO2008037392A1 - Method for determining the colour components of light

Info

Publication number: WO2008037392A1
Application number: PCT/EP2007/008188
Authority: WO
Inventors: Gregor Fischer
Original assignee: Fachhochschule Köln
Priority date: 2006-09-27
Filing date: 2007-09-20
Publication date: 2008-04-03
Also published as: DE102006054306A1

Abstract

The invention relates to a method for determining the colour components of the light from a light source by means of an image sensor, wherein the image sensor can be used to record an image, which is subdivided into image units, of a scene which is illuminated using the light source, wherein numerical image information is stored for each image unit for a plurality of colour channels, in which a first image is recorded and at least one second image which comprises only the linearly polarized light components of the same scene is recorded, wherein numerical difference information is formed by forming the difference between the numerical image information of each colour channel of at least some of the image units of a first image and at least one second image. The invention also relates to a photographic camera or film camera which has a lens and an image sensor for recording an individual image or a number of images and comprises at least two image sensors, wherein, before and/or during image recording, a first image sensor can be illuminated with unpolarized light and a second image sensor can be illuminated with linearly polarized light or both image sensors can be illuminated with differently linearly polarized light, wherein at least one item of difference information, which can be used to correct the image information of an image sensor, in particular for white balancing, can be formed from the image information of the two sensors.

Description

Method for determining the color components of light

The invention relates to a method for determining the color components of the light of at least one light source by means of at least one image sensor, wherein the at least one image sensor, a divided into image units (pixels) image of a lit with the at least one light source motif is receivable, wherein each image unit for several Color channels a numerical image information is stored.

The invention further relates to a photographic or film camera or an additional module for such a camera, in particular with at least one lens and at least one image sensor for taking a single image or a series of images. The invention also relates to a unit for a photographic or film camera for providing color information about the illuminating light.

When taking photos or filming, the problem is known that the reproduction of a recorded subject in terms of color largely depends on the color components of the light used or the color temperature of the light is matched to the film material used or the spectral sensitivity of an image sensor in the field of digital photography or film production. In this case, furthermore, the light used can not only originate from a light source, but possibly also from several, which leads to mixed light situations.

In classical photography, it is known that a photographer by means of a measuring device, the color temperature and thus the spectral color distribution, therefore Thus, the color components of the light used measures to then, taking into account the film material used by using different filters to achieve an optimal white representation without color distortion in objects in the subject, which actually have a white color in reality. Furthermore, it is achieved that neutral gray areas in the visual representation also appear neutral gray.

The problem is solved in digital photography or film production in that used cameras usually perform a so-called white balance, which often does not lead to satisfactory but sufficient photographic or film results.

It is known that in digital photo or film cameras preset color temperature values can be selected, which are then used for a correction of the actual image recordings. Likewise, it is known to perform an automatic white balance, in which by means of an image processing algorithm from an image on the basis of various criteria, the expected color temperature or the composition of the light of one or more light sources is essentially estimated only. The information thus obtained regarding the color components of the light are then used to carry out a correction of the recorded images.

Essentially, it trusts in the presence of sufficiently neutral image contents, which consequently leads to estimates for the color components of the light used, which in this known method depend not only on the illumination but also strongly on the image content.

As a result, very simple correction values are often obtained for each color channel (for example, RGB, red-green-blue) over which the color channels of a recorded image are globally balanced over the entire image section. Local variable lighting distributions, For example, with different light sources (such as in mixed light situations, especially in yellow sunlight and blue skylight) can be so inadequately balanced and can only be understood as a compromise.

The object of the invention is to determine the color components of the light of at least one light source with sufficient accuracy, in particular independent of the content of a recorded image and to obtain reliable correction values representing the color components of the light used at least one light source, in particular the possibility to offer a white balance not only globally but eg also locally, for example, pixel by pixel to perform, so that mixed light situations are successfully corrected.

The object is achieved by a method for determining the color components of the light of at least one light source by means of at least one image sensor, wherein with the at least one image sensor, a picture of a subject illuminated with the at least one light source can be picked up in image units, in particular in at least one image unit for each image unit for a plurality of color channels, a numerical image information is stored and wherein according to the invention a first image is taken and at least a second image is taken which comprises only or at least substantially only the linearly polarized light components of the same motif, whereby by differentiating between the numerical image information each Color channel of at least a portion of the image units of a first and at least one second image numerical difference information is formed.

Furthermore, we achieved the object by a photo or film camera or an additional module for a photo or film camera, in particular with at least one lens and at least one image sensor for taking a single image or a series of images wherein at least two image sensors are provided, wherein before and / or during imaging, a first image sensor with unpolarized and a second image sensor with linearly polarized light or both Image sensors with different linearly polarized light can be illuminated is / are, from the image information of both sensors at least a difference information can be formed, with which the image information of an image sensor can be corrected, in particular for a white balance. The object is also achieved by a unit for a photographic or film camera in which a single image sensor is provided, wherein before and / or during an image acquisition a first image with unpolarized and a second image with linearly polarized light or both images with different linear polarized light can be received, wherein from the image information of both images at least a difference information can be formed, with which the image information of an image sensor of the camera can be corrected, in particular for a white balance.

The invention initially assumes that at least one image sensor is used, by means of which a picture subdivided into image units (i.e., pixels per line and column by column) can be recorded in a known manner. This may be the image sensor which is already in use in a recording device, e.g. a camera is provided to take a picture, or one or more separate image sensors, which are used only for carrying out the method. One or more separate image sensors may be incorporated into a camera as a dedicated unit for performing the method, or may be an external unit or external add-on module, which may be e.g. can be connected to the hot shoe of a camera.

Such an image sensor can provide an image resolution equivalent to the resolution used to capture photos or movies, such as many megapixels or a significantly reduced resolution, or even just one pixel. Thus, an image taken with such an image sensor can have a resolution of one pixel to many megapixels. An image recorded with the at least one sensor thus has one or more image units, which are also referred to as pixels, wherein numerical image information is stored for each image unit for a plurality of color channels on the basis of a color space system. Such a known color space system can be, for example, the RGB color space in which the primary colors are red, green and blue. Each of these colors forms a color channel in the sense of the invention.

In order to store the color of an image unit of an image, for each image unit, based on the aforementioned example of the selected color space, information about the proportionate distribution of the colors in the respective color channel, e.g. stored in red, green and blue. In the RGB color space, at least three numerical values are thus stored for each image unit or pixel.

The method thus determines the color values (e.g., RGB) of the light color as one or more sensors receive them due to their specific spectral sensitivities.

The present description understands an image sensor in the sense of the method as an image sensor which receives a plurality of color signals (usually 3-dimensional). Whether an image is captured by a color filter mosaic of adjacent pixels or superimposed color pixels (e.g., Foveon) with only one shot, or multiple shots of changing color filters, is immaterial to the process.

Important for the method is that the spectral sensitivities of the sensors involved in the process match as well as possible. This applies both to the sensors involved in the determination of the light color (measuring sensors) and also to the main sensor of a camera (image sensor) whose image in a preferred embodiment is to be neutralized to this light color. The resolution of the measuring sensors can be chosen to be significantly lower than that of the image sensor, because the lighting within a scene changes only slightly and then not abruptly. In addition, a certain degree of local averaging (which in fact corresponds to a lower resolution) does not detract from the method, because in the electronic evaluation at least for a global white balance a certain averaging of the illumination is made.

However, it is recommended, at least in the case of a polarization evaluation with a liquid crystal array, to provide sufficient spatial resolution in order to adapt the phase shift, for example, to the location-dependent inclination angle or also to the color-dependent wavelength in the case of a color mosaic.

Depending on the motif and logic of the electronic evaluation, it may be useful to record and evaluate different exposure levels of the scene. This makes it possible to use otherwise over- and under-steered areas that would not be meaningfully evaluated.

In practice, the image information is obtained by, for example, using a CMOS or CCD sensor for image acquisition, which has corresponding filters for the respective colors of the color channels, so that for each physically represented pixel of this image sensor, each color can be separately recorded and stored , In a stored image, which is for example subdivided into the same number of image units as the image sensor physically has, three numerical image information can thus be stored for each image unit based on the RGB color space, each of the three image information being assigned to one of the three color channels is. If a different color space is used, one of three different number of color channels can also be taken into account. The invention uses to solve the problem the fact that it comes to bodies that are present in a recorded motif, both to a reflection of the light on the body surfaces as well as to a remission, ie a return of the light from the inside of the body.

It should be noted that remitted radiation components from the body are completely unpolarized and determine the individual body color due to a wavelength-dependent absorption.

In contrast, interfaces reflect the spectral light components of one or more light sources almost independently of wavelength. The reflected light is at least partly to completely linearly polarized, depending on the angle at which the reflection from the surface of a body is observed. The highest degree of polarization is present when the reflection takes place at Brewster's angle.

The core idea of the invention is based on evaluating the at least partially linearly polarized light of a recorded motif, since this light, as mentioned above, is wavelength-independent and thus represents the color components of the at least one light source used. A corruption by the color of absorbed body can thus be excluded.

In order thus to obtain a suppression of the remitted light components from a motif, it may be provided according to a possible aspect of the invention that first as above a first image is taken, which comprises all polarization components or polarization directions of the light and thus both the remitted light components of a or more bodies in a subject, as well as the portions of light reflected from the one or more bodies.

On the other hand, in the at least one second image, only the linearly polarized light components of the same motif are detected, after which, subsequently, the numerical image information of each of them, which is thus present in the two images, is detected Color channel of at least a part, optionally of all image units of an image, the difference is formed so that in the difference information then contained to each of the color channels of each image unit used for the process, the remission component is substantially or completely eliminated and essentially only the reflection component remains on surfaces, which as previously mentioned wavelength-independent and thus indicates the color distribution or spectral composition of the light used for lighting.

According to another aspect of the invention, it may be provided that even in the case of a first image, the recording takes place in such a way that a first image essentially comprises only linearly polarized light components. Otherwise, the procedural procedure as described above. In this case, it is important that the polarization directions of the linearly polarized light, which is detected in the first and a second image, is different, since otherwise no difference remains in the difference formation. Particularly preferably, the polarization directions in the first and a second image can be arranged at 90 degrees to each other. This has the advantage that there are reflections in a given image at a given position of the polarization, which are completely extinguished and, in the case of a second image with a polarization direction set perpendicular thereto, precisely these reflections are transmitted through the polarizer and captured in the image. All unpolarized remission components in the light are captured in both images at least substantially equally.

After subtraction of two images with each detected linearly polarized light, preferably less than 90 degrees polarization difference between the images, the reflected light component, in particular that which was determined by the choice of the polarization direction, thus dominate in the differential image.

Regardless of whether two of the captured images of one of the images have detected unpolarized light or have both detected linearly polarized light, Thus, for each of the image units of the two images used for the method, difference information is present for each image unit of the two images per color channel. With reference to the RGB color space, three difference information thus result for each image unit to which a difference has been formed, namely one for the color channel red, one for the color channel green and one for the color channel blue.

Since this difference information is available for each of the image units under consideration, according to the invention it is possible, based on this obtained difference information, to correct in an image, in particular an effective white balance locally in a recorded image, i. if necessary, even for each pixel or each image unit of an image. In this case, it may be provided to evaluate only the amount of the difference information, that is to say without taking account of the sign, for correction.

If the procedure is performed with an image sensor that is also used to capture images on a camera, then difference information can be at the same resolution as a captured image. It can then be made a pixel-by-pixel correction of a captured image. If separate image sensors are used, the resolution with which difference information is present can be obtained, for example. be significantly smaller than the resolution of a main image sensor.

It can then be provided that the difference information for a correction of an image from a main sensor is assigned in regions to the image units of a recorded image. If, for example, only one image unit is used with image sensors for carrying out the method, then the difference information can be used globally for correction for all image units of a recorded image. When using a main image sensor of a camera for carrying out the method, advantageously the first image taken in accordance with the method can also be used to correct it after the subtraction.

Regardless of the physical or utilized resolution of the at least one image sensor used to perform the method, it may be provided to weight and / or average the obtained difference information, e.g. so as to use the weighted / averaged difference information for area or global correction of images or the image units (pixels) of these images.

According to the invention, e.g. only an image sensor is used (e.g., in addition to a main image sensor or alone) to sequentially acquire a first and at least a second image. It is also possible to use at least two image sensors, wherein at least one second image is always recorded with a first image sensor and at least one second image sensor with at least one second image sensor at the same time.

In this case, each image sensor on a camera for carrying out the method can have its own imaging optics, or the image sensors use a common optical system and a subsequent beam splitting in order to illuminate each image sensor with a light beam. If necessary. For this, the optics can be used, which produces a picture of the subject on a main picture sensor of a camera.

In order to record the at least one second image, if necessary also the first image with only or at least substantially linearly polarized light components, it can be provided that this takes place using a linear polarizer, for example a polarization filter or a polarization cube.

It may be provided here that, for example, a known photographic or film camera with at least one objective and at least one (main) Image sensor is used to capture a single image or a series of images. Thus, with such a camera, an image can first be recorded in a customary manner, in particular which receives the light without any polarization-influencing measures (or with a first polarization direction) and then again of the same motif, but now with polarization-influencing measures or in another polarization direction at least a second image is taken, due to the use of the polarizer essentially only linearly polarized light in a second image and possibly also the first image the way to the image sensor finds.

The method according to the invention can thus be carried out with cameras already available on the market only with the aid of a polarizer.

According to the invention, however, provision can also be made for using cameras or additional modules for cameras which, in addition to a main sensor, also have two separate image sensors for carrying out the method, wherein at least one of the image sensors (after passage of the light of a motif through a polarizer) illuminates with linearly polarized light if necessary, both image sensors will also be illuminated with differently linearly polarized light.

In all embodiments in which a first image is taken without using a polarizer and a second image using a linear polarizer, it may be particularly preferred that the transmission loss of the polarizer is compensated when taking a second image compared to the first image , For example, a modified, in particular higher amplification in the signal path of an image sensor can be used for this purpose, or it can also be provided to apply to the numerical image information obtained in the second image a multiplier which compensates for the transmission loss. In this way, the first image and the at least one recorded second image can be calibrated to each other, in particular so that in the second image, substantially the same or similar signal levels are obtained in relation to the transmitted linear light component as in the first image thereof Motive.

Thus, the difference formation ensures that the light components based on remission are eliminated and that the spectrally independent reflection components dominate in the difference image thus formed.

In this case, in the differential image those reflection components will dominate whose polarization direction are aligned parallel to the linear transmission direction of the polarization filter or polarizer which was used in the second image recording. Since these components are maximally transmitted in the recording due to the action of the linear polarization filter in the second image and the second recording is calibrated or normalized with respect to the signal level, in particular of the remitted components, the information content remains at the difference formation in the image content of the difference image. which is parallel to the orientation of the polarizer used, so the reflection component.

If polarizers are used for both images, a calibration of the images may possibly be omitted, since the transmission loss has the same effect on both images.

It is possible that body surfaces are arranged in different directions in a photographically detected motif, so that reflections of the light used for the illumination and thus of the light source used in particular also take place at different angles. Because of the different orientations of the body surfaces in a scene, there will therefore be reflections at many different angles in a captured image, so that, on the one hand, the intensity of the reflection and, on the other hand, the degree of polarization can be distributed over a recorded image.

However, in each recorded motif, relative to the selected direction of the linear polarization, there will always be motif areas from which a reflection of the light emanates and whose difference information can be evaluated, in particular in order to carry out a correction of an image, which may have been taken later.

Since the most reliable information about the color components of the light used at least one light source can be obtained from those reflections in which the highest degree of polarization and / or the highest reflection intensity is present, it can be provided according to an aspect of the invention that from the difference information each (Used) image unit those difference information of image units are selected, which have the highest difference amount at least in one of the color channels or in all of the color channels. This selected difference information can then be used to correct images.

It can be concluded from this difference information with the highest difference that a high degree of polarization is present in the reflection detected in this imaging unit, so that the difference information obtained reflects very reliable information about the color component of the reflected light, at least that light which was reflected locally in the image recording on the relevant body, which was imaged in the considered image unit.

In another embodiment of the invention, it may also be provided that a plurality of second images are taken, wherein in each of the second images, the light is recorded with a linear polarization of another polarization direction, in particular for each of a first and one of the second images Difference information are formed. Also here For example, the first image can be recorded with unpolarized light or else with linearly polarized light, as explained above.

This method aspect is based on the consideration that, as mentioned above, bodies with different surface orientations can be recorded in a photographed subject, whereby reflections can occur at different angles and thus also with differently oriented linear polarizations. By taking a series of at least two second images at different polarization directions (e.g., 45 degrees to each other), it is thus possible to ensure that reflections at different polarization angles can be discriminated in the difference formation from two images. Thus, difference information can each be formed into a pair of two images, namely a first (unpolarized or polarized) and one of the second polarized images or even between two of the second polarized images, depending on which pair for the difference formation, and thus which Polarization direction was used, reflections from different areas of a recorded motif in each case can dominate the difference image or the difference information to the image units.

Here, according to one aspect of the invention, it may be provided that the difference information is compared to at least a portion of the possible pairs, preferably to each pair of images, thus a comparison of the obtained difference images (to any image pairing, if applicable) takes place and the difference information of one Pair or the difference image of a pair is selected in which there is at least in one of the color channels, optionally in all color channels, the highest difference amount. Accordingly, in the case of these difference information, the reflection component will be highest, so that the difference image with these highest difference components permits a reliable determination of the color components of the light used. This difference image or the difference information of each color channel to each selected image units of this difference image can therefore be used to perform white balance on later captured images, for example per pixel or unit of image or area or globally.

According to another aspect of the invention, it may be provided to compare the difference information present from all pairings to each image unit and to select for each image unit those difference information having the highest amount in at least one of the color channels. Thus, according to this aspect, a complete difference image is not selected from the different pairs for further processing, but always from all possible pairings, the difference information for a particular image unit representing the strongest reflection component and therefore, as previously mentioned, the highest numerical value. This results in a resulting difference image, which can be composed in each image unit of difference information resulting from different of the original pairings of the recorded images.

Regardless of the aforementioned embodiments, according to the invention difference information exists for each of the image units of a first and a second image under consideration, or else between two second images. In this case, there are a number of difference information items corresponding to the number of color channels used in each color space considered, that is, for example, three difference information per image unit of an image in the RGB color space.

According to the invention, it may now be provided that the difference information for at least one image unit may be used for several or all image units directly or after a conversion in order to correct the image information of an image which has also been taken later, in particular by the correction a white balance is performed. For example, a difference image, ie all difference information of second subtracted image information for each image unit can be logarithmized. It is possible to perform global averaging over all the difference information (in particular selectively for each color channel). Or a histogram is created with regard to the frequency of the difference information that occurs. The mean value or the most frequent difference information can then form correction values for each color channel, with which, for example, globally a recorded image can be corrected with regard to the white value.

If, for example, an image is recorded with a specific number of image units, each image unit being able to be associated, for example, with one column and one line of the image, it may be provided that there is difference information on each individual image unit of a recorded image due to prior difference image formation the correction can be made. There is thus the possibility of pixelwise, i. per image unit of a recorded image to perform the white balance.

In another embodiment, it may also be provided that the difference information is weighted so as to form, for example, common difference information for all image units, which can then be used to correct for later recorded images. It then takes place in the later recorded images in each individual image unit a correction, for example, using the same difference information.

Due to the fact already mentioned above, that in some image areas of a recorded image the reflection components can sometimes be more and sometimes less powerful, it can also be provided that the difference information present for an image unit is weighted with the difference information of neighboring image units. Thus, for each image unit considered, a new resulting difference information can be formed which takes into account how the difference information is in neighboring image units. It can be achieved so that even in image units that at have a recording only a small amount of reflection, in the future recorded images correct difference information available to perform a later comparison can.

In order to carry out the method with a camera or film camera or an additional module to a camera, it may be provided that before and / or during an image acquisition, the light collected by the lens in a unpolarized and a linearly polarized portion or two differently oriented linearly polarized Shares is split, wherein the two portions are each received by an image sensor and wherein one of the image sensors that a first image and the other image sensor receives a second image.

Here, from the differential information obtained from the images of the two image sensors, the numerical image information recorded with the first image sensor or those image information of another image sensor provided for the photo or film recording can be corrected.

It can be provided that a photographic or film camera or additional module according to the invention, in particular with at least one lens and at least one image sensor for taking a single image or a series of images is used, which comprises two image sensors, wherein before and / or during an image acquisition a first image sensor with unpolarized and a second image sensor with linearly polarized light can be illuminated or the two image sensors can be illuminated with differently linearly polarized light. In this case, at least one difference information can be formed from the image information of the two sensors, with which the image information of an image sensor can be corrected, in particular for the abovementioned white balance.

It can be provided that the first and the second image sensor are provided separately from an image sensor in the camera, which is used to pick up the image sensor Single frame or the series of images is provided, so separately to a main sensor. By way of example, the first and the second image sensor may have a lower resolution than the main image sensor which is provided for recording the single image or the series of images.

This is based on the consideration that it is not absolutely necessary for each image unit of a recorded image to have individual difference information for carrying out a correction, but that it is possibly sufficient to provide one or only a few difference information items by the first and second image sensors which are global or, for example, are applied in regions for correction in those image units of a recorded image, in particular those associated with the global or regionally detected difference information.

In a camera or additional module according to the invention, which can be used with the method, the first and second image sensor can be assigned a single lens or a separate lens, so that with this arrangement of lens and the first and second image sensor substantially same scene is detected as with the main image sensor and the lens, which is used for the actual capture of the desired scene image.

Alternatively, it can also be provided that the first and the second image sensor are acted upon by light which is coupled out of the main beam path from the main objective and a main image sensor, which is provided for recording the single image or the series of images , Such coupled-out light can be split, for example, after decoupling into an unpolarized and a linearly polarized component or two differently linearly polarized components, wherein these two split components are in each case again supplied to the first and the second image sensor for carrying out the method. It can also be provided a camera having a main image sensor and another image sensor with its own optics (lens) for receiving the same motif, for example, the additional image sensor has a lower physical resolution than the main image sensor. This additional image sensor can be integrated in the camera or in an external additional module. In the case of an image recording, for example with the main image sensor, the first image according to the invention can be recorded, which, for example, captures all unpolarized light components or all light components of a first polarization direction.

With the second separate image sensor only linearly polarized light of the same subject or another polarization direction can be recorded. For this purpose, a polarizer can be provided at least in the optics of the second sensor. Since the second separate image sensor has a lower resolution, it may be provided to select only a part of the image units for carrying out the method from the first image of the main sensor, in particular an area in the image, e.g. a central area corresponding in number and arrangement of the image units to the image units of the second separate image sensor.

Thus, a difference image from all image units of the image can be created by the second image sensor and the selected image units of the image of the first main sensor. The difference information thus obtained can be used, if necessary after a weighting, averaging, interpolation or summary, to correct the image of the first main sensor, in particular globally.

In another embodiment, it may also be provided to provide a unit for a photographic or film camera, in which only a single image sensor is provided, wherein before and / or during an image acquisition, a first image with unpolarized and a second image with linearly polarized light or both images with different linearly polarized light are absorbable, wherein from the Image information of both images at least a difference information can be formed, with which the image information of an image sensor of the camera can be corrected, in particular for a white balance. As described above, such a unit can be integrated in a camera or arranged externally, for example attached to the camera via the hot shoe of a camera, in particular via a communication in the hot shoe and the additional module communication is to the camera information for a white balance to convey.

In the case of the abovementioned camera designs or embodiments of the units / additional modules, the formation of a difference image can take place before or during image acquisition with a main image sensor via the first and second additional image sensors, whereby the obtained difference information can be used to correct the image units of that image. which were recorded with the main sensor.

Unless otherwise stated, a polarizer means in the sense of the invention any type of polarization-influencing element, e.g. a simple polarizing filter or an array of polarizing filters and / or lambda / 2 or lambda / 4 plates.

The polarization-influencing element may also be formed as a liquid crystal device with subsequent polarization filter, in particular wherein the degree of polarization influence is adjustable by different voltages on the liquid crystal array. Likewise, the polarization-influencing element may be formed as a polarization filter array, which has adjacently arranged surface areas that allow differently polarized light to pass or the polarization-influencing element can be formed as a birefringent element, in particular in front of which a mask, in particular a line mask, is arranged. Embodiments of the invention and introductory remarks are explained with reference to the following figures. Show it:

Figure 1: the description of reflection and remission

FIG. 2 shows the effect of different polar filter positions on reflection component and remission component FIG. 3: a diagram of the method for determining the illuminating component

light color

Figure 4: different polarization directions by multiple filter-sensor combination Figure 5: different polarization directions through rotatable filter unit with coupled (left) and fixed (right) sensor unit Figure 6: different polarization directions by birefringent

Material with line mask Figure 7 different polarization directions through Polfilterarray on a sensor surface Figure 8 different polarization directions by voltage-controlled liquid crystal Figure 9 a color channel and pixel by pixel subtraction of 2 differently polarized sub-images (R ₁ G ₁ B) ₁ and (R ₁ G ₁ B) ₂ figure 10 is an exemplary representation of two output images and one

difference image

Figure 11 shows the structure of a simple sensor module Figure 12, the combination of 2 sensor modules Figure 13 shows the structure of a simple sensor module with birefringent

Material and line mask

Figure 14 shows the structure of a simple sensor module with liquid crystal Figure 15 is a sensor module with liquid crystal and beam splitting Figure 16 is a sensor module with Polfilterarray FIG. 17 shows the realization of a polarizer array with a 2-stage structure with birefringent material

The aim of the new method is to obtain the light color of the illumination within the scene to be recorded in order to use this information to create a secure automatic white balance in electronic cameras. In contrast to the prior art, the method has metrological character and exploits the degree of polarization of light fractions reflected on body surfaces.

The reflectance on glossy or even matt surfaces is, to a good approximation, wavelength-independent. The reflected light components of any surface are therefore spectrally composed as the illuminating light and thus achromatic. The degree of polarization of these reflected light components is established by the Fresnel equations (reflection law) and can be evaluated by at least 2 images to be recorded under different polarization directions

The remitted light components of body surfaces characterize its color and are almost unpolarized by multiple scattering effects within the body.

FIG. 1 shows these relationships. The directed reflection on the left-hand side, as well as the diffuse reflection in the middle, generates at least partially polarized light. On the other hand, in the remission the light is unpolarized.

FIG. 2 shows the effect of different polarization filter positions. In the left-hand illustration it becomes clear that both reflected and remitted portions can pass the polarization filter. When aligning the polarizing filter according to the right-hand illustration, only the one linearly polarized light component of the remitted light can pass through the filter. The difference formation of images taken under different polarization directions detects the degree of polarization of the achromatic reflection components and makes the colorful unpolarized remission components disappear. One can thus obtain spatially resolved information of the illuminating light color within the scene as received by an image sensor.

Both the intensity of the reflected light component and its degree of polarization are decisively dependent on the orientation of the body surface viewed to the light source and camera and its surface roughness, but not on the remitted body color. In the case of 3-dimensional objects, this results in a locally variable useful signal level.

The ratios of the sensor and illumination-specific light color values are precisely the information needed for an accurate white balance of an image recorded with this sensor.

The practical implementation of the method is shown schematically in FIG. According to this figure, an observed scene is optically imaged and several images are generated. The imaging is thereby divided into polarization filtering, i. the splitting of the incident light into proportions of differently oriented polarizations. The images thus obtained are recorded and a difference is formed. The difference image obtained in each case can be evaluated.

In order to determine the light color, the optical image with regard to the resolution and the freedom of geometrical distortion need to meet significantly lower requirements than the optics of the main sensor of a camera. However, it is preferable to ensure that the image section of the sub-images for difference formation match as well as possible. Edge offset, for example, between the partial images or a pronounced lateral chromatic aberration can lead to interference signals in the differential image, which can falsify the measurement result. How are these effects by optical accuracy or corrections or only compensated by an electronic correction in the image evaluation is a question of the desired system design.

The viewed image section for determining the light color does not necessarily have to match the image section of the main sensor. Thus it may be desirable, e.g. To extend the field of view by means of a wide-angle or fish-eye optics in order to make the determination of the illuminating light color as robust as possible.

When using only a simple measuring cell (1-pixel sensor), it is possible to dispense with a sharp (focusing) optical image and to use a directional characteristic by mechanical means (such as the light meter without dome).

The taking of multiple images of the same image section may be e.g. by multiple exposures of a sensor in a temporally short sequence or e.g. by beam splitting into different optical paths with multiple sensors or e.g. by staggered pixel raster between the sub-images on a sensor surface, in particular similar to the color filter array done in simple color sensors.

The advantage of using a single sensor is the constancy of the image detail and the spectral transmission characteristic. However, due to the time difference between the images, fast moving objects can lead to the mentioned falsifications of the measurement result.

The advantage of beam splitting with multiple sensors is the simultaneous recording of the difference images, which leads to advantages for fast moving objects. However, in this case, preference should be given to the same image sections and the same spectral characteristic between the sensor paths. As a rule, these specifications can be met by electronic calibration functions between the sensors. For beam splitting, for example, optical beam splitters (mirrors or cubes) and / or optical gratings are suitable. It is preferable to ensure that any changes in polarization caused by the beam splitting largely independent of the wavelength of the light. A single sensor with offset pixel raster for different partial images can preferably combine the addressed points in one concept, namely the acquisition of the partial images at one time and the same image section between the partial images.

The polarization filtering can be done by polarizing filters, dielectric interfaces or birefringent material. At beams incident on a dielectric interface under the Brewster angle α _B , only the polarization component oscillating perpendicularly to the plane of incidence is reflected.

In the case of optically anisotropic materials, the light propagation is dependent on direction and polarization. Thus, when the optic axis of the crystal is at an angle, a beam which is incident vertically falls into a so-called ordinary ray without deflection and an extraordinary ray with deflection. Both beam components are polarized perpendicular to each other. In addition, due to different deflection of the beam components, a double image results, which corresponds to the generation of multiple images (see previous section) and is therefore preferred for the realization of the recording of images with different polarization.

The polarization filtering with different orientations can be done mechanically or electronically. Mechanically, different polarization directions can be realized, for example by a multiple implementation of the image acquisition unit with polarizing filter and sensor under different orientations. This is shown for example in FIG. 4. On the other hand, FIG. 5 shows a rotatable filter unit with a simple sensor. A simple (or multiple) image acquisition unit with birefringent material is shown in FIG and FIG. 7 shows an image recording unit on which a polarizing filter array of different polarization directions is applied.

In the rotatable version of the sensor can be connected depending on the structure with the polarizing filter and thus rotated together.

When using a birefringent material, a sensor is sufficient to evaluate a polarization direction. However, it is preferable to use a mask on the input side, in particular a line mask, in order to spatially separate the resulting double image of the perpendicular polarization directions and to assign them to separate pixel regions of the sensor.

By masking the birefringent material polarized light components are shaded on the affected sensor areas, which act depending on the mask structure as a free to be designed Polfilterarray (s.u.) Vertical polarization directions. In the simplest case of an exemplarily mentioned line mask, a line structure of parallel and perpendicularly arranged polarization directions results, in the case of a mask in the structure of an exemplary checkerboard pattern corresponding to a checkerboard pattern of perpendicular polarization directions.

In the variant with polarizer array, the polarized pixels may be scattered incompletely. It is preferable to combine a polarizing filter cell with an agglomeration of RGB pixels of the color filter mosaic in order to obtain the polarized RGB color information as locally as possible in a coherent manner.

In addition, when the polarizer array is combined with the main image sensor, the brightness loss of the polarizer can be compensated by a flat-field calibration in conjunction with an interpolation from the non-polarized environment. For an electronic version of different polarization directions, for example, a liquid crystal can be combined with a downstream polarization filter. It can thus realize the rotation of the polarization direction without moving parts. FIG. 8 shows an example of such an arrangement.

A liquid crystal has birefringent behavior depending on the arrangement of the crystal molecules. An electric field (external applied voltage) allows the orientation of the crystal molecules within the liquid to change, thereby gradually controlling the birefringence. The liquid crystal acts as a switchable phase shifter with preferential direction in principle like a λ / 4 or λ / 2 plate. According to this function, therefore, a liquid crystal can have a linearly polarized light component of suitable orientation, e.g. rotate by exactly 90 ° through a λ / 2 delay, or circularly polarize through a λ / 4 delay (evenly distributed in all directions of vibration such as unpolarized light) or pass unchanged (0 °) without delay.

By subtraction between a pair of images of these 3 cases, the light color between 2 mutually perpendicular polarization directions can be evaluated.

Since the phase shift by a fixed wavelength component only works exactly for a specific wavelength, it is recommended to set the control voltage of the liquid crystal in a color-dependent manner (eg R, G and B). Otherwise, the polarization components are transmitted differently depending on the direction as well as on the wavelength, which can lead to measurement errors in the determined light color. Similar incorrect measurements can be caused by oblique incidence of light. Here, it is preferable to adhere to a fixed phase relationship and, accordingly, to adapt the activation of the LC to the angle of inclination in a location-dependent manner. An exact phase position can be determined and set by a suitable calibration specification (flat-field calibration for the polarization direction under consideration) in a color-dependent and location-dependent manner. The difference of the filtered under different polarization directions images takes place in an electronic processing unit on the basis of intensity linear color signals R, G, B. For reasons of accuracy, it is useful when using different sensors sub-images to use sensor-corrected color signals in which parasitic influences such as dark current, sensitivity noise (Fixed Pattern Noise) and false pixels are already corrected. The difference image is obtained by color channel and pixel-by-pixel subtraction of 2 differently polarized partial images (R, G, B) i and (R ₁ G ₁ B) ₂ according to FIG. 9.

The difference image diff is generally signed. Since the further evaluation of the light color involves the color ratios between the R, G and B components of the differential image, this circumstance is negligible because, given a sufficient signal level, the color values of a pixel in the difference image are either all positive at the same time or all negative at the same time (Exception: understeer areas with superimposed dominant signal noise). Without limiting the generality, therefore, the amount of the difference image can also be taken and evaluated to the light color.

In FIG. 10, a first image 1 is shown by way of example above, which is subdivided into individual image units in three columns and three rows. Each of the nine image units here comprises image information for three color channels in this example, which correspond to the colors red, green and blue, ie the RGB color space. By way of example only, three numbers are shown here in each image unit, which correspond in proportion to the proportion of the respective colors in the considered image unit, which were created when a motif was taken.

It should be noted here that the division of an image selected here into nine image units is to be understood only as an example and, of course, significantly more image units in an image can be used in today's conventional image sensor resolutions or, if necessary, even less. The former applies in particular when the image sensor which is to be used to take a picture is also provided with the purpose of determining the determination according to the invention of the color components of the at least one light source used.

If, on the other hand, it is provided to use separate image sensors in addition to an image sensor which is used to take a photograph, which only serve to carry out the method according to the invention, then one can make do with a significantly lower resolution, if appropriate, if only regional or differential information is formed should be used for the global or only partial correction of the image units of a recorded image.

With reference to the example given here, the image 1 is therefore intended to represent a first image within the meaning of the invention in which completely unpolarized light has been recorded or the light of a motif which has arrived at the first image sensor without any polarization influence. Alternatively, light of a first linear polarization direction can be recorded.

In contrast, Figure 2 shows a photograph with the same or an equivalent image sensor in which a polarizing filter was used to detect the image, or another polarization direction was chosen with the polarizer.

The size of the numerical image information listed here in each color channel RGB is preferably adjusted (unpolarized light / polarized light in one image) by calibration / normalization to the numerical values which have been achieved in FIG. This can be compensated in particular that it comes through the use of a polarizer or a polarizing filter to a transmission loss. Between pictures 1 and 2, according to the invention, a difference is formed such that a difference occurs in each of the nine picture units with respect to each color channel R, G and B. In the example chosen here, which is not to be understood as limiting, it can be stated that the color channel R for red is slightly dominated in the difference image and an equal distribution is present in the color channels G and B, ie for green and blue.

It can therefore be concluded that in the light source used for lighting a slight red component dominates, as it occurs for example in thermal light sources or, for example, in light generated by the late evening sun.

Scenes that are photographed or recorded with this illumination will thus have a slight reddish cast due to this illumination, so that a white balance can be performed on the basis of the knowledge of the difference information present here, namely, for example, by the fact that the image information belonging to one Image are obtained, calibrated or normalized based on the differential information available here.

If an image is taken with an image sensor having the same resolution, ie, nine image units in this simplified example, it can basically be provided that an individual correction takes place for each image unit with the difference information present for the relevant image unit.

In another embodiment, it can also be provided that the difference information from the difference image is weighted and summarized, for example regionally or globally, to a plurality of difference information per area or only to a difference information for the entire image, so that, for example, in a global summary only There is a single triplet of difference information for the color channels R, G and B, which are used to correct all image units of a captured image.

It should be pointed out that the example given here is merely highly simplified and should not be interpreted as limiting, but merely as illustrative in the number of image units used here or in the order of the numerical numerical values shown.

To maximize the useful signal level, it is useful to subtract pairs of images from mutually perpendicular polarization directions. One of the polarized images can be replaced by a suitably normalized unpolarized image. The useful signal level is then halved, but the difference information remains qualitatively preserved.

^ES 9 ' ^lt: Iwφohπaert = ⁷ 0 ° + ⁷ 90 °

Thus follows: Diff = I ₀ . - I ₉₀ . = I ₀ . - (I _unparmed - I _0. ) = L {l _v - I _unpolansιen / 2)

The level adjustment of the unpolarized recording should preferably compensate for the transmittance of the polarization filter. This normalization is preferably determined by a calibration such that unpolarized light components of the scene disappear due to the difference formation.

The use of an unpolarized recording for difference formation has the advantage that it can serve for different pairs of images, and thus relatively few image recordings allow as many different polarization directions to be evaluated.

The evaluation of different polarization directions makes the method largely independent of the position of illuminating light sources, so that a sufficient useful signal is available for the evaluation in any direction of the illumination. As a rule, two pairs of images (eg 0790 ° and 457135 °) are sufficient to make the process sufficiently robust. Alternatively, in the case of a rotatable polarizer and a control loop would be conceivable, the polarization direction to a max. User signal level in the difference image controls. In this case, the evaluation of a single differential image for the determination of the dominant light color is sufficiently robust.

In order for the unpolarized remission components to disappear as a result of the subtraction of the pairs of images, the best possible match of the location information between the images is important. Depending on the optical design and manufacturing tolerances, it may be necessary to electronically correct (e.g., by shifting or scaling algorithms) any pixel offset between the images or even a transverse chromatic aberration within an image prior to subtraction.

An electronic evaluation, which serves to optimize the accuracy and robustness of the light color recovery, in a preferred embodiment, which can be combined with any other designs, e.g. proceed according to the following general scheme:

1. union of different difference images to maximize the Nutzsignalpegels

2. Formation of relative light color components:

Diff _R + Diffo + Diff _B

Diffa g = DWR + Diffo + Diffs

3. It can preferably be provided to _identify or mask invalid regions, for example in the form of a metrological location-dependent weighting function W _meS s (xy).

4. It may furthermore be preferred to create a picture-content-oriented location-dependent weighting function (confidence measure) W _SZ ene (x, y) _> which evaluates the image content in terms of relevance and reliability for the determination of the illuminating light color

5. Histogram calculation (distribution of frequencies) h (r, g) taking into account the metrological (see 3.) and the image content-oriented (see 4.) weighting functions

6. Determination of the mean dominant light color (r, g) _g ι _O bai from the histogram distribution (see 5.)

Experience shows that real complex scenes like local problem cases

• Noise / understeer

• Reflecting surfaces (directional reflection)

• Several different light sources (for example sun / sky)

may lead to erroneous measurements of the illuminating light color. In order to hide these in a preferred embodiment for the evaluation of error-prone regions, weighting functions W (x, y) according to 3 and 4 can be created, which rate the reliable image areas higher and the less trusted image sites lower.

a) Combination of different difference pictures

There is the union of different difference pictures. In this case, differential images can be calculated from all possible combinations of image pairs from the images Ij recorded under different polarization directions in a possible embodiment:

Diffjk ( ^χ > y) = ij ( ^χ > y) - ¹ Ic ( ^χ . Y)

The final difference image Diff (x, y) can be composed, depending on location, from the maximum difference signal, which has the maximum local value:

Diff (x, y) = maχ (\ Diff _jk (x, y) \) Because these are RGB color images, it is a good idea to make the maximum decision depending on one of the three color channels (eg green channel). If one or more color channels are overdriven (clipping), a different difference image combination can be used (see c) Creation of a measuring technique weighting function).

b) formation of relative light color components

There is the formation of relative light color components. The color of the light depends on the spectral composition of the lighting, regardless of its intensity (brightness). The light color can therefore be represented by a combination of only 2 color components. Since the evaluation is based on intensity-linear sensor signals R, G, B, a histogram analysis also makes sense on linear color components. Assuming equal noise on the color signals, in one possible implementation, this noise may be eliminated by an averaging process, such as histogram analysis (see T). The linear relative color value shares

r = diff _R

Diff _R + Diff _G + Diff _B Diff _o

8 =

DiJf _R + Dijgr _G + DiJ? _B

meet the given conditions of linearity and intensity invariance.

Alternatively, intensity invariant nonlinear representations such as a = Ig (R / G) b = Ig (B ² Z (R - G) or other ratio combinations are possible.) Due to the nonlinear transformation (logarithm), such a representation should be preferred only for low noise component can be used. Otherwise, an averaging would be in the form of a change in the mean.

c) Creation of a metrological weighting function

There is the creation of a metrological weighting function.

The aim of the metrological weighting function is, not or insecure evaluable

Detect image areas and mark them by a weight function (i.d.R between 0 and 1).

Criteria can be:

• Overdriven areas (clipping of at least one color channel in the source images)

• Very noisy image areas with too low signal-to-noise ratio (SNR)

For example, in one possible implementation, the SNR criterion is local filtering with a median filter according to median (Diff).

SNR =

Diff - median (Diff) | proven.

d) Creation of a picture-content-oriented weighting function

The creation of a picture-content-oriented weighting function continues. The goal of this image-content-oriented weighting function is the local weighting of

• critical image locations with a low weighting factor and

• Particularly reliable appearing image areas with a high weighting factor.

However, due to special constellations of the object surface texture and the scene, image areas which can be evaluated by measurement technology can lead to erroneous measurements of the light color. It should be noted that the electronic evaluation of complex scenes, the dominant light colors (ie the Light sources or primary radiator) and not secondarily radiating object colors to find. As critical to be evaluated image locations are therefore

• shaded image areas (no direct illumination and thus no dominant light source, also often very noisy (see c))

• glossy (= reflective) object surfaces (mirror images of even colored objects within the scene that do not contain the light color)

• classify very rough or three-dimensional high-frequency object surfaces by multiple reflections leading to colored false light colors in the difference image (for example trees, grass, colored velvet or wool etc.), to name but a few examples. On the other hand, picture sites are like

• shiny object surfaces, as long as they are reflections (= mirror images) of light sources,

• gray objects (which can be very reliably evaluated both in the difference image and in the original image for the light color) very safe (robust with high accuracy of the light color) and therefore preferable to evaluate.

The following local criteria can be used as criteria for the above classification of the image contents:

• Brightness original image

• Brightness difference image

• Chroma difference image

• Brightness degree of polarization

• Chroma degree of polarization

The polarization degree image P (x, y) can be calculated from an image pair Ij (xy) and _lk (x, y) and is called

Are defined. It is determined separately for the 3 colors and has correspondingly 3 color components Pr ₁ Pg, Pb.

According to the above definition, the _degree of polarization represents the ratio of the useful signal Diff _Jk = I _} - I _k to the brightness I ₁ + I _k . As a result, it acts as a measure of the evaluability of an object surface, regardless of the brightness of the illumination. Particularly high degrees of polarization achieve glossy surfaces and dark, dull surfaces. Thus, it can be used as an essential indicator for the detection of mirror images (critical cases see above).

In addition, the color composition of the degree of polarization produces a direct connection with the object color. Considering the inversion of the degree of polarization

the counter contains a mixed color of light and object color (under the illuminating light color), the denominator only the light color. The reciprocal degree of polarization therefore represents the locally white-balanced object color. For neutral objects, a combination of three identical color values of the reciprocal polarization degree and thus also the degree of polarization itself results. Neutral objects can therefore be localized within the scene as gray objects in the polarization degree image.

The degree of polarization, like the difference signal, is generally dependent on the angle of incidence of the light rays to the surface of the object. At the so-called Brewster angle, the degree of polarization is maximal. Since the angle of incidence depends essentially on the aperture angle between the camera orientation and the direction of illumination, the average degree of polarization is a global measure of the evaluability and thus of the applicability of the method in general. As a brightness measure H (x, y), for example, a linear combination of the 3 color values can be used for the original, difference image and polarization degree image

H = (R + G + B) / 3 can be used meaningfully.

As two-dimensional chrominance information (C1, C2), intensity-invariant representations as in b) formation of relative color components (s.o) are suitable.

Neutral object regions can be determined, for example, from the color values of the polarization degree image P _n P ₉ , P _b as follows:

AGrey = J (P _r - 1/3) ² + (P _g - \ / 3f + (P _b - 1/3) ²

(ΔGrey s ~ ι _S Λ i_ • j. [ThresholdGrey

GrayObject = e ^vy

To localize the (positive and negative) cases in the image, the above criteria can be mathematically linked together. The result is a feature function that expresses the fulfillment of the relevant situation (for example mirror image much / little).

The image-content-related weighting function W (x, y) can then be created by an essentially logical combination (AND, OR can be implemented by multiplication or addition in the case of continuous features) of the individual feature functions.

A combinatorics can be for example (see above): W _scene = (GrayObject OR (mirroring AND very bright)) AND

NOT (Dark OR (Reflection ANDNOT Very Light) OR Colorful Lettering Image)

e) histogram calculation

There is also a histogram calculation using e.g. the histogram of the light color taking into account the two weighting functions from c) and d) pixel by pixel (index n) according to

lKr "g,) _n + W _mess (x, y) - W _scene (x, y) for r, ≤ r (x, y) _{n + ι} <r, + ΔΓ Λ h (r" gX _{+ i} = lg , ≤ g (x, y) _{n +} ₁ K g _{1 +} Ag

[Kr ₁ , g X can be calculated otherwise.

f) Determination of the mean dominant light color

Further, as the final step in this embodiment, the determination of the average dominant light color, e.g. results from an arithmetic mean of the histogram distribution:

For the realization of the described white balance method, there are in principle 2 preferred alternatives which do not limit the method or the devices mentioned.

1. The execution as a separate unit (either as an accessory to a

Camera or integrated as an independent sensor module in the housing) or 2. the integration into a camera, being essential

Camera function elements (e.g., optics or sensor) for the white balance method.

The construction of an exemplary sensor unit of the described method is shown in FIG. 11.

Lens and beam splitter mirror 2 images are designed, which are recorded simultaneously via a polarizer-sensor combination. The polarizing filters are rotated in their orientation by 90 ° from each other. The unit thus provides 2 mutually perpendicular polarized images, from which the difference and further the light color can be obtained.

In order to make the evaluation robust against arbitrary directions of illumination, preferably 2 additional polarization directions can be recorded and evaluated offset by 45 °. For this purpose, an additional sensor module according to FIG. 12 is used rotated by 45 ° about the optical axis.

Coincident image contents are only necessary for the subtraction. The parallax between the two sensor modules is less critical because the difference images are processed substantially independently of each other.

The same basic function of the sensor module can be realized with a simple optical path using a birefringent material with an upstream line mask according to FIG. 13 or a liquid crystal according to FIG.

The sensor module with liquid crystal comes out like the module with birefringent material without beam splitter and a single sensor. With the liquid crystal module, moreover, the two images required for the subtraction can only be recorded one behind the other, which is disadvantageous in the case of moving objects. Even the simple sensor modules with birefringence or with liquid crystal should be doubled by 45 °, either in a separate duplicate or in combination with a beam splitter.

Combining the liquid crystal design with beam splitting results in a complex sensor module which is shown schematically in FIG. 15 and offers special possibilities: 2 images each required for the subtraction can be recorded simultaneously and 4 polarization directions can be evaluated with only one sensor module. For this purpose, the liquid crystals can be switched alternately without and with λ / 4 phase shift (corresponds to the unpolarized state).

Comparable properties such as the latter variant with liquid crystal and beam splitting offer a variant with a polarizing filter array on the sensor surface, which is much easier to implement from the optical setup and with respect to calibration. This is shown in the figures 16. For a reliable function, it is sufficient to apply 2 polarization directions with 45 ° rotation. By subtraction of the polarized colors with the unpolarized environment, all directions in the 45 ° grid can be evaluated. The local normalization factor between the polarized and unpolarized color values can be determined by a flat-field calibration with a polarized white image.

Likewise, the polarizing filter array can be realized by a combination of two masks, in particular with a suitable anti-aliasing filter. Figure 17 shows a polarizing filter array which can be realized by birefringent material in combination with an upstream mask. To evaluate the 0790 ° and parallel 457135 ° polarization directions, for example, a multi-layer structure similar to the arrangement of an optical anti-aliasing filter can be used. The 1st stage (upper mask and 1st layer of the birefringent material) is used for filtering the 0790 ° calibration and the 2nd stage (45 ° polarization rotator, intermediate mask and 2nd layer of the birefringent material) filtering 457135 ° -Direction. When the difference is the Arrangement of the polarization directions by the 2-stage pre-filtering in the sensor plane to consider.

In general, in one embodiment it may be provided to form a polarizing filter array by embedding dichroic crystals in a support and aligning the crystals in the desired direction by means of a static electric field during curing. For a wearer, it may be e.g. to trade the color filters of a sensor.

The measuring device can be installed in a separate housing similar to an external flash unit and connected to the hot shoe of a camera and thus forms an additional module or external unit. The contacts provide power and communication interfaces that allow the light color information to be passed to the camera.

Alternatively, the units can also be integrated directly into the camera body and used to measure the light color. In this case, it must be ensured by a suitable positioning in the housing that the optics can not be concealed accidentally and the image section of the sensor is not unnecessarily limited by the camera lens.

Within electronic cameras, existing beam paths (main beam path, viewfinder beam path, autofocus) can be expanded by beam splitting and adapted to the described method. The sensor modules in the previous section can be transferred to this situation, possibly can be dispensed with the measuring optics.

The advantage of this procedure is the matching of the image section between the measuring and image sensor, which is advantageous for a local white balance.

Another advantage is the possibility of using the main sensor as an unpolarized partial image for subtraction. This reduces the number of necessary measurement images (then sense 2 polarization direction with a rotation of 45 °).

Also advantageous is the measurement of the light color in the camera beam path, whereby the color matching between the measuring device and the main sensor can be omitted.

When integrated into the camera beam path, the introduced measuring device could also take over the exposure control in addition to the control of the white balance, the exposure sensor could then be omitted.

Another possibility is the use of the image sensor as a measuring device as shown in Figure 14 or 15 with upstream liquid crystal and polarizing filter. However, the loss of light due to the absorption of the polarizer in the main beam path leads to a reduction in the sensitivity of the camera system by a factor of 2-3. In addition, only 2 polarization directions could be evaluated.

In principle, the method of birefringence could also be realized in conjunction with the main sensor, but the line mask would cause both light and resolution losses.

A particularly interesting option is to accommodate a distributed polarizing filter array on the high-resolution image sensor. Because the illumination information changes only relatively slowly, the polarizing filter regions on the sensor surface can be distributed in a stillly patchy manner in order to disturb the image information as little as possible through the polarization regions.

For a safe function it is sufficient to apply 2 polarization directions with 45 ° rotation. By subtraction of the polarized colors with the unpolarized environment, all directions in the 45 ° grid can be evaluated. The local normalization factor between the polarized and unpolarized color values can be determined by a flat-field calibration with a polarized white image.

One difficulty with all variants of camera integration arises when using a polarizing filter in front of the camera lens: the degree of polarization of reflected components, on which the new measurement method is based, is lost, and the useful signal disappears, so that the process is inconclusive. In the amateur sector (compact and viewfinder cameras), this restriction would probably be acceptable, because here almost never worked with polarization filter.

With regard to all embodiments, it should be noted that the technical features mentioned in connection with one embodiment can be used not only in the specific embodiment, but also in the other embodiments. All disclosed technical features of this invention description are to be classified as essential to the invention and arbitrarily combined or used alone.

Claims

claims

1. A method for determining the color components of the light of at least one light source by means of at least one image sensor, wherein the at least one image sensor is subdivided into at least one image unit image of a lit with the at least one light source motif, each image unit for a plurality of color channels a numeric Image information is stored, characterized in that a first image is taken and at least a second image which comprises only or at least substantially only the linearly polarized light components of the same motif, wherein by differentiating between the numerical image information of each color channel of at least a portion of the image units of a first and at least one second image, a numerical difference information is formed.

2. The method according to claim 1, characterized in that a first image is taken, which comprises all polarization components of the light.

3. The method according to claim 1, characterized in that a first image is taken, which comprises only linearly polarized light components, wherein the polarization directions in the first and a second image differ from each other, in particular are arranged at 90 degrees to each other.

4. The method according to any one of the preceding claims, characterized in that an image which comprises substantially only linearly polarized light components, using a linear polarizer, in particular a polarizing filter, is recorded.

5. The method according to claim 4, characterized in that the

Transmission loss of the polarizer is compensated in a second image, in particular by applying a gain in the signal path of an image sensor or by applying a multiplier in the numerical image information of a second image.

6. The method according to any one of the preceding claims, characterized in that from the difference information of each image unit those difference information are selected which has the highest amount at least in one of the color channels.

7. Method according to claim 1, characterized in that a plurality of second images are taken, wherein in each of the second images the light is recorded with a linear polarization of another polarization direction, wherein for each pair a first and one of the second images and / or from two second images difference information are formed.

8. The method according to claim 7, characterized in that the

Difference information is compared to each pair of images and those difference information of a pair are selected, which have the highest amount at least in one of the color channels.

9. The method according to claim 7, characterized in that the

Difference information to each image unit are compared and are selected for each image unit that difference information having the highest amount in at least one of the color channels

10.A method according to one of the preceding claims, characterized in that the difference information of at least a part of the image units are weighted to form a resulting difference information.

11.A method according to one of the preceding claims, characterized in that the difference information of an image unit are weighted with the difference information of neighboring image units, in particular line and or column by column to form new resulting difference information for the considered image unit.

12. The method according to any one of the preceding claims, characterized in that at least the difference information to an image unit are used directly or after a conversion to correct the image information of a recorded image, in particular wherein a white balance is performed by the correction.

13. The method according to any one of the preceding claims, characterized in that with a single image sensor, in particular the main image sensor of a camera, successively the first and at least one second image are taken.

14. The method according to any one of the preceding claims, characterized in that in particular in a photographic or film camera before and / or during an image acquisition, the light collected by a lens light is split into an unpolarized and linearly polarized portion or two different linearly polarized components, wherein the two portions are each taken by an image sensor, wherein one of the image sensors that a first image and the other image sensor receives a second image.

15. The method according to any one of the preceding claims, characterized in that in particular in a photographic or film camera before and / or during an image acquisition, the light collected by a lens light is split into an unpolarized and linearly polarized component or two different linearly polarized components, wherein the two parts of different image units of the same image sensor with one group of image units receiving a first image and another group of image units taking a second image.

16. The method according to claim 15, characterized in that the

Splitting takes place in different proportions by a polarizing filter array, which is arranged in front of an image sensor and which has juxtaposed surface areas which allow differently polarized light to pass.

17. The method according to any one of the preceding claims, characterized in that from the obtained difference information, the numerical image information of the first or another provided for the Fotooder film recording image sensor can be corrected.

18. A photographic or film camera or additional module for a photographic or film camera, in particular with at least one lens and at least one image sensor for receiving a single image or a series of images, characterized in that at least two image sensors are provided, wherein before and / or during a Imaging a first image sensor with unpolarized and a second image sensor with linearly polarized light or both image sensors with different linearly polarized light is illuminable is / are, from the image information of both sensors at least a difference information can be formed, with the image information of an image sensor can be corrected, in particular for a white balance.

19. A camera or additional module according to claim 18, characterized in that the first image sensor is formed by a main image sensor for recording images and the second image sensor by a separate image sensor, in particular with its own optics and in particular a lower resolution than the main image sensor.

20. A camera or additional module according to claim 18, characterized in that the first and the second image sensor are provided separately from a main image sensor, which is provided for receiving the single image or the series of images.

21. A camera or additional module according to claim 20, characterized in that the first and second image sensor have a low resolution than the main image sensor, which is provided for receiving the single image or the series of images.

22. A camera or additional module according to any one of claims 18 to 21, characterized in that the first and second image sensor is associated with a separate lens.

23. A camera or additional module according to any one of claims 18 to 22, characterized in that the first and second image sensor is acted upon by light, which is coupled out of the beam path between a lens and a main image sensor, for receiving the single image or the series of images is provided.

24. unit for a photographic or film camera, characterized in that a single image sensor is provided, wherein before and / or during an image recording a first image with unpolarized and a second image with linearly polarized light or both images with different linearly polarized light recordable are, from the image information of both images at least a difference information can be formed, with which the image information of an image sensor of the camera can be corrected, in particular for a white balance.

25. Unit according to claim 24, characterized in that it is integrated in a camera or attached to the camera via the hot shoe of a camera, in particular via a communication in the hot shoe and the additional module communication is feasible to the camera information for a white balance to convey.

26. Unit according to one of claims 24 or 25, characterized in that a polarization-influencing element is arranged in front of the image sensor.

27. Unit according to claim 26, characterized in that the polarization-influencing element is formed as a liquid crystal device with subsequent polarization filter, in particular wherein the degree of polarization influence by different voltages on the liquid crystal device is adjustable.

28 unit according to claim 26, characterized in that the polarization-influencing element is formed as a polarizing filter array, which has juxtaposed surface areas that allow differently polarized light to pass.

29 unit according to claim 26, characterized in that the polarization-influencing element is formed as a birefringent element, in front of which a mask, in particular a line mask is arranged.