DE102021215048A1 - Arrangement and method for distinguishing between a background and a foreground object of a scene - Google Patents

Abstract

The present invention relates first to a method for distinguishing between a background (02) and at least one foreground object (01) of a scene to be imaged. According to the method, key light is used for the background (02) with a spatial and temporal variation. In individual periods of time, the punched light is present locally only in sections, in each case in at least one section of the background (02). The sections of the background (02) in which the key light is present in the individual time sections change over time. At least one image of the scene made up of several image sections is obtained. Furthermore, an at least one-channel key signal is derived by determining a distribution of the key light imaged in the at least one image. To determine a first channel of the key signal, a directly imaged component (06) of the key light is determined from the individual image sections, which correspond locally to those sections of the background in which the key light is present in the respective time sections. The invention also relates to an arrangement for imaging a scene comprising at least one foreground object (01) and one background (02).

Description

The present invention relates first to a method for distinguishing between a background and at least one foreground object of a scene to be imaged, so that the imaged background can be exchanged in an image depicting the scene, whereby the imaged foreground object appears in front of a different background. The method according to the invention thus represents a punching method for the production of individual images and/or moving images. The invention also relates to an arrangement for imaging a scene comprising at least one foreground object and one background.

In television and film productions, there is often a need to release certain image content, in particular foreground objects, from a background. For this are u. a. Various chroma keying techniques, for example based on blue and green screens, are known. Methods are also known in which narrow-band backlighting is used for precise optical differentiation, for which purpose sodium vapor lamps are used, for example.

The U.S. 2018/0332239 A1 shows a method for replacing a background in images. Images are captured using infrared light and visible light. In particular, objects are illuminated with visible light, while the background is illuminated with infrared light.

The WO 01/06766 A1 teaches a method for image data processing in which a separation of object data of an image data set representing an image foreground from a data set representing an image background takes place. A background to be captured by a camera is changed at a frequency tuned to a capture frequency of the capturing camera. The background is changed, for example, by switching between two states.

The U.S. 3,133,814 teaches a method of producing a color image from a monochrome original image of a plurality of objects. In this process, the objects are painted on a translucent sheet of material and illuminated with diffuse light.

From the WO 2012/038009 A1 a method for distinguishing between background and foreground of a scenery is known, in which the background shows any image encoded with a coding such that the image is recognizable for a direct observer of the background. In a signal generated by an image sensor, the background is distinguished from the uncoded foreground by the coding.

The DE 10 2007 041 719 A1 shows a method for generating an augmented reality in a room. A digital video projector is used to project images from a sequence of images onto part of the surfaces delimiting the space. The images are spatially and/or temporally modulated in luminance and/or chrominance. A camera synchronized with the video projector records at least part of the room.

The US 10,313,607 B2 teaches a method of distinguishing between background and foreground of scenery in images. The background shows an arbitrary image encoded with an encoding such that the arbitrary image is visible to a direct viewer of the background. The background is distinguished from the foreground by a signal which is generated by an image sensor of the electronic camera which records the images by means of the coding. The background displays the arbitrary image using only colors from one or at least two regions of a visible spectrum. The light emanating from the background passes through a corresponding color filter before hitting the image sensor.

The DE 60 2004 010 464 T2 shows a method for acquiring an image from a scene containing a primary object and a secondary object. An image of the primary object and a key signal to distinguish the primary object from the secondary object are to be provided. The primary object is illuminated by key light of a selected color and temporal variation. At least two images of the scene are obtained, with the temporal variation of the key light being such that the amplitude of the key light is different in the first image than in the second image. The key signal is derived by comparing the different amounts of key color in each image.

Based on the prior art, the object of the present invention is to provide a flexibly adaptable and expandable solution for improved differentiation between a background and at least one foreground object of a scene to be imaged.

Said object is achieved by a method according to appended claim 1 and by an arrangement according to appended independent claim 10.

The method according to the invention is used for the preferably gradual differentiation between a background and at least one foreground object of a scene to be imaged, so that the method is a punching method, with the quantitative components originating from the background and the foreground object in the imaging process preferably being determined. The scene is to be depicted by at least one image in which the at least one foreground object can be distinguished from the background in order to release the at least one foreground object in the image and/or remove the background depicted and/or exchange it for image content representing a different background , which is preferably done gradually, for example to take into account semi-transparent areas of the foreground object. The image of the at least one foreground object represents the image information to be used for further image processing. The at least one foreground object can be formed by a human actor and/or furniture or other scenery elements, for example. Since the background in the image is to be removed or exchanged, it is not relevant for further image processing, so it is preferably implemented in accordance with the technical requirements resulting from the recording and differentiation. Since the at least one foreground object is preferably differentiated gradually from the background, in particular transitions between the foreground object and the background or also partially transparent areas of the foreground can be quantitatively detected, so that the background can be replaced or removed at the transitions and the partially transparent areas according to the quantitative proportion of the background takes place, which is comparable to an image shutter or an image mixing process.

The method provides that punch light is used for the background or in the background, in that the background is illuminated from the front or behind with the punch light and/or the background or parts of the background emit the punch light. Thus, the key light is reflected and/or transmitted and/or emitted by the background, so that the key light is present in the background in any case. The punched light is used for the background with a spatial and temporal variation, with the punched light only being present locally in individual time sections in at least one section of the background, so that the sections of the background in which the punched light is present in the individual time sections change in time. The background is thus divided into different sections, as a result of which there is a spatial division of the background. The sections can be formed by spatial sections of the background. If the background is formed by a surface, the sections are preferably formed by surface sections. The key light is not permanently present in all of the sections of the background. Rather, the key light is only present in individual sections of the background in the individual time sections. The keying light is preferably used at least once in each of the sections of the background over a period of time, which is formed by a defined number of time sections. This means that the background emits the key light in its entirety or is illuminated by the key light over this period of time. The at least one foreground object is preferably not directly illuminated by the key light or preferably in such a way that a position of the background at a location of an image conversion results from a direction of the key light and a different distance via a parallax.

In a further step of the method, at least one image of the scene composed of several image sections is obtained, which in particular depicts the key light used in the background and at least one contour and transparency of the at least one foreground object. A camera is preferably used for this purpose, which has at least one electronic image converter, but preferably at least two electronic image converters. The image sections are preferably obtained one after the other, which is preferably done by converting them with the image converter one after the other. The individual image sections preferably correspond to individual sensor elements or groups of individual sensor elements of the image converter. Alternatively, preferably, the image sections can be defined with a movable diaphragm.

In a further step of the method, an at least one-channel key signal is derived for the preferably gradual differentiation of the at least one foreground object from the background by determining a distribution of the key light imaged in the at least one image. The key signal or a channel of the key signal is preferably a mask signal or a transparency or alpha signal. In a simple case, the key signal or one of the channels of the key signal is a binary signal which assigns the points of the image either to the background or to the at least one foreground object. However, it is particularly preferably a multilevel signal in the sense of a grayscale image or a transparency or alpha channel. It is preferably a signal with at least 256 levels or an analog signal. The determination of the distribution of the key light imaged in the at least one image can be done optically, for example by spectral filtering of the image in imaging optics. However, the determination of the distribution of the key light imaged in the at least one image can also be carried out by electronic image processing of the image converted by the image converter.

To determine a first channel of the key signal, a directly imaged portion of the key light is determined from the individual image sections, which correspond locally to those sections of the background in which the key light is present in the respective time sections. The determination of the key light is therefore limited to the individual image sections; namely in each case on the respective of the image sections which is assigned to that section of the background which is currently being illuminated by the punch light or which emits the punch light. The determination of the key light can thus be restricted to that part of the key light which was imaged directly onto the image without reflection, possibly with partial transmission, but without diffraction and without scattering from the background. This part of the key light used in the background was either not covered by the foreground object or has transmitted a transparent or partially transparent section of the foreground object without diffraction or scattering. The local correspondence of the individual image sections to the individual sections of the background is given when the respective image section depicts the respective segment of the background or would depict this segment of the background if this was not prevented by the foreground object. The keying signal preferably includes at least one further channel, for the determination of which an indirectly imaged portion of the keying light is also determined.

A particular advantage of the method according to the invention is that it enables the directly imaged portion of the key light to be determined with little effort, which makes it possible to differentiate from an indirectly imaged portion of the key light, which leads to an improved result when removing or replacing the background in the image .

The method is preferably carried out for at least two of the images. The at least two images are preferably an image sequence or moving image.

The spatial correspondence of the image section used to determine the key color in each case to that of the sections of the background in which the key light is present in the respective time segments leads to the confocal detection of the key light. In particular, a beam path of a light beam forming the punched light for illuminating the sections of the background in a time-changing manner and a beam path for imaging the background in the individual image sections are preferably confocal.

In preferred embodiments, the background is formed by a surface, which is preferably a surface of a wall, a screen, a stage or studio curtain, a projection screen or a two-dimensional display. The sections are correspondingly formed by surface sections which are preferably triangular, square, polygonal or circular. The sections are preferably formed by rectangles or by lines. Correspondingly, the image sections from which the directly imaged portion of the key light is determined are preferably each formed by an image line, so that the image is converted line by line. The image lines correspond locally to the sections of the background formed by lines. In further preferred embodiments, the sections of the background and the corresponding image sections are not unchangeable but are continuously adapted to the foreground object and, if necessary, to the background. The size and/or the shape of the sections of the background and the corresponding image sections can be adjusted.

In preferred embodiments, a first electronic image converter, which operates according to a rolling shutter method, is used to determine the first channel of the keying signal. Accordingly, the displayed image exposes the first image converter in sections, preferably in rows or columns. The rolling shutter method can be implemented by a corresponding electronic readout of the first image converter or also mechanically, for example with a mechanically driven slit diaphragm or pinhole diaphragm. The first image converter is exposed line-by-line or column-by-column by the imaged image in synchronism with the illumination or emission of the background taking place line-by-line or column-by-column. The first image converter is preferably formed by a CMOS image converter, which works according to the rolling shutter method. In one embodiment, the first image converter is also used to convert an image of the at least one foreground object. In preferred embodiments, a second image converter is used to convert an image of the at least one foreground object, for which purpose a separation in an imaging beam path preferably takes place in order to distinguish the imaged background from the at least one imaged foreground object. This separation in The imaging beam path takes place optically, preferably spectrally and/or in relation to a polarization direction and/or in relation to a luminance. In principle, a separation can also take place in the electronically converted image.

In preferred embodiments, the key light is used for the background by illuminating the background with the key light. To achieve the spatial and temporal variation of the punched light, a controllable projector or light scanner is preferably used for illumination. The projector or scanner emits the punch light. A laser projector is particularly preferably used for this purpose, so that a laser beam generated by the laser projector forms the punched light and illuminates the background. The laser beam is preferably deflected line by line. Accordingly, the laser beam is moved like a scanner over the background.

In further preferred embodiments, the key light is used for the background in that at least parts of the background emit the key light. For this purpose, the background preferably comprises a light source which has a multiplicity of locally distributed individual light sources. The individual light sources can be controlled individually and are distributed over the sections of the background. The light source is preferably formed by an at least two-dimensional display, for example in the form of a screen, such as an LED screen. Correspondingly, the background preferably comprises such a two-dimensional display, which preferably emits the punched light line by line.

The key light preferably has a property in which it differs from illumination light for illuminating the foreground object. The key light preferably has a key color in which the key light preferably differs at least largely from the illumination light and the foreground object. The punched light preferably has a spectrum with one or more spectral lines and/or spectral bands. A wavelength of the at least one spectral line or a mean wavelength of the at least one spectral band is preferably 532 nm, 440 nm or 640 nm, although other mean wavelengths can of course also be used. The at least one spectral band has a bandwidth which is preferably at most 30 nm and particularly preferably at most 5 nm. The spectrum of the punched light preferably has only one spectral line or spectral band. Accordingly, the key light is monochrome. The spectrum of the illumination light for the foreground object preferably has no or only a negligible overlap with the spectrum of the key light. This can be ensured, for example, by using special filters. In further preferred embodiments, the spectrum of the key light has several of the spectral lines or spectral bands in order to obtain a wavelength-dependent key signal, for example, when the foreground object is partially transparent. The spectrum of the punched light particularly preferably has at least three of the spectral lines or spectral bands, with the punched light in total preferably representing white light. The three spectral lines or spectral bands are preferably formed by red, green and blue light. By weighting the three spectral lines or spectral bands, other colors of the punched light can also be generated if required.

In preferred embodiments, the key light is used in different regions of the background with different spectral compositions in order to adapt to the background, to the foreground object and/or to image content to be inserted. For example, the spectrum of the punched light can each have two spectral lines or spectral bands, which have different wavelengths in the different regions of the background.

In a preferred embodiment, the punched light has a spectral profile ranging from a first wavelength to a second wavelength, which runs in a direction of extension of the background. The spectral progression preferably extends across the visible spectrum, so that the background is illuminated with the punched light like a rainbow. The spectral profile preferably runs vertically, so that it runs from bottom to top in the background. However, the spectral course can also run horizontally or in another direction. A white-light laser with a prism or with a grating is preferably used to illuminate the background with the punched light having the spectral profile. A spectral filter with a spectral profile is preferably arranged in front of the first image converter, which corresponds to the spectral profile of the key light in the background, so that only the directly imaged key light spectrally matches the filter and other light is eliminated. In this respect, there is no need to operate the first image converter using a rolling shutter method. A third image converter is preferably also used, but without spectral filtering, which is operated according to a global shutter method.

In another preferred embodiment, the different regions of the background form a pattern, wherein in a first set of the background regions, the spectrum of the punch light has a spectral line with a first wavelength or a spectral band with a first mean wavelength, while in a second set of the background regions the spectrum of the punch light has a spectral line with a second wavelength or a spectral band with a second mean wavelength. Overall, this preferably leads to a homogeneous use of the punch light in the background, so that the background is illuminated homogeneously with the punch light or the punch light emits homogeneously. For this purpose, the use of the punched light in the regions of the first set and the use of the punched light in the regions of the second set alternate locally, so that a structured illumination or a structured glow is achieved. The regions of the first set and the regions of the second set preferably form a regular structure. The structure allows the associated region of the background to be identified, whereby local alignments of normals on the foreground object can be determined from the indirectly imaged key light. The first wavelength or the first mean wavelength and the second wavelength or the second mean wavelength have a spectral distance from one another which is preferably less than 100 nm and more preferably less than 30 nm. The key light is therefore perceived by humans as monochrome. Alternatively, the key light in the first set of regions and the key light in the second set of regions preferably have colors that are metameric but have different spectral compositions. Preferably, the regions of the first set and/or the regions of the second set have different sizes in different portions of the background to achieve different levels of detail; for example in the form of a fractal design. For example, if a portion of the foreground object is reflected, such as an actor's jacket, the background can only be identified in detail to a limited extent. But if a rough part of the pattern formed by the regions of the two sets can be identified, then the background replacement can be done reasonably well.

In further preferred embodiments, the key light differs in other/additional properties in particular from the foreground object and the illumination light for the foreground object. For this purpose, the punched light has a polarization and/or the punched light is used as a pulse sequence. Alternatively or additionally, the key light differs in its luminance or luminous intensity from the illumination light or from the light reflected from the foreground object by a factor which is preferably at least two and more preferably at least ten. In a further preferred embodiment, the key light differs in that position on which it is imaged in the image by an alignment inclined to a direction of the image. The parallax is preferably chosen so that there is a relative angle between 5 degrees and 40 degrees; preferably of 10 ±5 degrees between the directions of the key light and the illumination light. The other properties that characterize the key light allow a more precise differentiation of the foreground object from the background. The punched light that differs in other/additional properties is preferably used in the embodiment described above, according to which the punched light is used in a structured manner in the background.

If the key light has a key color, the at least one image is preferably first spectrally divided and/or filtered to determine the distribution of the key light depicted in the at least one image, whereby the at least one image is divided into a foreground channel and a background channel comprising the separated key light becomes. In this way, a distribution of the key light imaged in the at least one image is determined before its electronic conversion. If the punched light has a characterizing polarization or the punched light is used as a pulse sequence, polarization filtering preferably takes place before the electronic conversion or temporal filtering during or after the electronic conversion of the image. A polarization beam splitter is preferably used for polarization filtering or polarization splitting.

The spectral splitting of the image can be done electronically in the image that has already been converted. However, the at least one image is preferably optically spectrally divided before the electronic image conversion, as a result of which the at least one image is divided into an optical foreground channel and an optical background channel comprising the separated key light. Furthermore, to distinguish the key light, an opto-mechanical element configured to separate a spatial pattern can be used as a spatial demodulator. This is preferably at least one galvanometric mirror in combination with a slit diaphragm or a perforated diaphragm, which compensate for line illumination or point illumination. Alternatively, two-dimensionally moveable micromechanical mirrors (MEMS) are preferably used to compensate for scanning point illumination, which is preferably done in combination with a reflecting pinhole to achieve the separation. With these scanning methods, the local Chen acquisition of the image preferably uses one or more single-channel transducers in a high temporal resolution. Alternatively, the image is preferably generated by a further rescanning with the mentioned optomechanical element or with a further optomechanical element on the image converter. Alternatively, a spatial light modulator (SLM), such as based on liquid crystals on silicon (Liquid Crystal on Silicon, LCoS) or a system with digitally controllable micromirrors (Digital Micromirror Device, DMD) is preferably used to spatially separate to reach. In this case, it may be sufficient to have the spatial light modulator imaged onto the image converter without rescanning being necessary.

The first image converter is preferably used for image conversion of the background channel. A second image converter is preferably used for image conversion of the foreground channel. A beam splitter is preferably used for the optical splitting of the image. A dichroic beam splitter is preferably used for spectral optical division of the image. Partial beam paths of the beam splitter each form the optical foreground channel and the optical background channel. The first image converter is accordingly arranged in the optical background channel. The second image converter is correspondingly arranged in the optical foreground channel.

In preferred embodiments, the punching signal has at least two channels. To determine a second channel of the key signal, the key light imaged in all of the image sections of the at least one image is determined in the individual time sections. Thus, in contrast to the determination of the first channel, the determination of the imaged key light in the individual time segments is not restricted to the individual image segments. This means that the indirectly imaged key light, which is created in particular by reflection, refraction, diffraction and/or scattering on the at least one foreground object, is also detected, since reflection, refraction, diffraction or scattering result in the the key light used for the sections of the background is also imaged in other than the locally corresponding image section. However, since the key light imaged in all of the image sections of the at least one image is determined in the individual time segments, the key light reflected, diffracted and/or scattered on the foreground object, i. H. captures the indirectly imaged key light. A total of the directly imaged key light and the indirectly imaged key light is thus recorded. A comparison with the first channel, which detects the directly transmitted or directly imaged key light, allows conclusions to be drawn about the indirectly imaged key light. By using the second channel of the key light, better results can be achieved when determining a preferably finely graded alpha channel and finally when cutting out delicate edges of the depicted foreground object and/or in partially transparent areas of the depicted foreground object. A third image converter, which operates according to a global shutter method, is preferably used to determine the second channel of the keying signal. Alternatively, the second channel of the keying signal is preferably determined using the first image converter, which is operated image-by-image alternately according to a rolling shutter method and a global shutter method. The first image converter or the third image converter or a fourth image converter can also be configured such that the key light imaged in all of the image sections of one of several image regions of the at least one image is determined in the individual time sections. For example, one of the image converters can use the global shutter method to capture indirect, particularly scattered, background light originating in an upper area of the background, while another of the image converters can capture indirect, particularly scattered, background light originating from a lower area of the background. In the course of post-processing, for example, scattered light components can be replaced with blue for a sky and green-brown for a landscape. Like the first channel of the key signal, the second channel of the key signal is preferably determined from the optical background channel. The second channel is color independent in a simple case, i. H. the spectral properties of reflection, diffraction or scattering, which led to the indirectly imaged key light, are not taken into account. In this sense, the second channel is monochrome. However, the second channel preferably also describes a dependency on the color, as a result of which the spectral properties of the reflection, diffraction or scattering, which led to the indirectly imaged key light, are taken into account. To this end, the second channel can include a number of sub-channels for different colors or an additional definition of the color in the color space or an additional definition of the spectral properties.

The determination of the second channel of the key signal, ie in particular the determination of the indirectly imaged key light, is preferably carried out for the embodiment described above, in which the structured key light is used, since this enables further analysis. Local alignments of normals on the foreground object are preferably determined from the indirectly imaged key light. The determined from the second channel of the key light indirectly abge The key light formed, with or without information about the normals, is preferably used as a basis for emulating automated indirect background information. For example, the blue of a sky can be taken into account for such indirect lighting. If information about the normals is available, specular reflections are preferably also taken into account.

In preferred embodiments, the key light present in the foreground channel or the key light present on the at least one foreground object is determined in order to determine an optical interaction between the at least one foreground object and the background. The determined optical interaction is taken into account when replacing the displayed background, as a result of which the image of the foreground object can appear even more realistic in front of the replaced background. In addition, the sections of the background from which the key light present on the at least one foreground object originates are preferably determined. The optical interactions between the at least one foreground object and the background can also be assigned to a local area of the background, which is preferably taken into account when replacing the background. The indirectly imaged key light can be taken into account when replacing the imaged background with an adjusted effort. For example, the directly imaged key light determined for deriving the first channel can be determined with a much higher resolution, whereas the indirectly imaged key light is determined with a coarse resolution or only for selected regions with a fine resolution.

In preferred embodiments, the at least one-channel punching signal is applied to the at least one image or to the at least one image depicting the scene or at least the at least one foreground object in order to remove the background of the scene depicted in this at least one image and thus the at least one foreground object free, which is preferably done gradually, for which the punch signal preferably includes a multi-level alpha channel. The keying signal is preferably applied to the at least one image in order to replace the displayed background of the scene in the at least one image with other image content, which then represents a background or an environment in the image for the at least one foreground object. The two-channel key signal is preferably applied to the at least one image in order to remove the directly imaged key light and the indirectly imaged key light in the at least one image and to replace the imaged background of the scene with other image content, which is then used in the image for the at least one Foreground object to represent a background or environment. As already described above, the keying signal is preferably multi-level, so that the removal or replacement is gradual, i. H. done softly. In this case, the optical interactions between the foreground object and the background of the scene are preferably also taken into account, so that these interactions are also present between the foreground object and the image content representing a different background.

The arrangement according to the invention serves to record at least one image of a scene comprising at least one foreground object and one background. The arrangement enables a preferably gradual differentiation between the background and the at least one foreground object.

The arrangement initially comprises a device for generating punched light, which is preferably formed by a controllable projector, a laser projector, a light scanner, a light source with a large number of locally distributed individual light sources, a two-dimensional display or the like. The projector or scanner preferably includes a microelectromechanical system (MEMS) for beam steering. Alternatively or additionally, the projector or scanner can also include a liquid crystal on silicon (LCoS) or a digital micromirror device (DMD). The device for generating punched light is configured to use the punched light for the background with a spatial and temporal variation, wherein the punched light is locally only partially present in individual time segments in each case in at least one section of the background, and wherein the sections of the background, in which the punched light is present in the individual time periods, change over time.

The arrangement also includes an image recording and image processing device for obtaining at least one image of the scene made up of a plurality of image sections. The image recording and image processing device is configured to derive an at least one-channel key signal for distinguishing the at least one foreground object from the background by determining a distribution of the key light imaged in the at least one image, wherein a directly imaged portion of the Key light is determined from the individual image sections, which correspond locally with those of the sections of the background, in wel chen the key light is present in the respective time periods.

The arrangement is preferably configured to carry out the method according to the invention or one of the described preferred embodiments of the method according to the invention. The arrangement preferably also has features which are described in connection with the method according to the invention. In particular, the arrangement preferably includes the described first image converter and preferably also the second image converter and/or the third image converter. The image recording and image processing device preferably includes a camera in which the two or three image converters and the beam splitter are combined. Alternatively, the image recording and image processing device preferably comprises a plurality of image conversion modules, each with one of the image converters, so that greater adaptability is provided. The camera or the image conversion modules preferably include a lens, it also being possible for the lens to be separate.

The image recording and image processing device preferably also includes a computer which is configured in particular for image processing. In particular, the computer is configured to derive the punch signal. The computer is also preferably configured to replace the imaged background of the scene with other image content. These computer configurations are preferably defined by software plug-ins for existing software solutions for image and/or video processing.

Further details and developments of the invention result from the following description of a preferred embodiment of the invention, with reference to the drawing.

The only figure shows a preferred embodiment of an arrangement according to the invention for recording at least one image (not shown) of a scene comprising a foreground object 01 and a background 02. In a simple case, the background 02 is formed by a wall. The arrangement includes a laser projector 03 for generating a laser beam 04, with which the background 02 is scanned like a scanner, i. H. is illuminated line by line, for example from top to bottom, which is repeated periodically. The laser beam 04 requires a period of time to illuminate a line (not shown) of the background 02. Within each of these time intervals, the laser beam 04 is only within the respective line on the background 02. The laser beam 04 forms a punched light, which preferably has a spectral line at, for example, 532 nm, 440 nm or 640 nm. The punched light formed by the laser beam 02 is reflected by the background 02, with a punched light beam 06 shown as an example resulting from diffusely reflected punched light 07 being imaged directly.

The arrangement includes a camera, which is represented by a beam splitter 08, a first image converter 09 and a second image converter 11, with further components of the camera, such as a lens, not being shown. The beam splitter 08 is dichromatic, so that it spectrally splits an imaging beam path directed towards it, as a result of which the imaged key light of the background 02 is directed onto the first image converter 09, which is shown as an example for the directly imaged key light beam 06. In particular, an image of the foreground object 01 is directed onto the second image converter 11 .

The first image converter 09 is formed, for example, by a CMOS image converter, which operates according to a rolling shutter method, so that it is read out line by line, which here takes place conjugated synchronously with the illumination of the background 02 performed line by line. The illumination of the background 02 with the laser projector 03 and the conversion of the image of the background 02 with the first image converter 09 thus take place confocally line by line. As a result, the directly imaged punched light, which is illustrated by way of example by the directly imaged punched light beam 06, is converted with the first image converter 09. The directly imaged punched light has not experienced any reflection, diffraction or scattering except from the directly illuminated line of the background 02. In particular, the directly imaged key light has not experienced any reflection, diffraction or scattering by the foreground object 01. A first channel of a key signal in the form of a finely graded alpha channel is derived from the image of the directly imaged key light converted with the first image converter 09.

The second image converter 11 is used to convert the image of the foreground object 01. The keying signal is applied to the converted image in order to gradually release the image of the foreground object 01, so that in the next step a desired image content for imaging a different background (not shown) is in place the imaging of the background 02 located in the scene occurs.

Reference List

01

foreground object

02

background

03

laser projector

04

laser beam

05

-

06

directly imaged punch light beam

07

diffusely reflected key light

08

beam splitter

09

first image converter

10

-

11

second imager

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• US 2018/0332239 A1 [0003]
• WO 0106766 A1 [0004]
• US 3133814 [0005]
• WO 2012/038009 A1 [0006]
• DE 102007041719 A1 [0007]
• US 10313607 B2 [0008]
• DE 602004010464 T2 [0009]

Claims (10)

Method for distinguishing between a background (02) and at least one foreground object (01) of a scene to be imaged, comprising the following steps: - Use of punched light for the background (02) with a local and temporal variation, with the punched light only being present locally in individual time segments in at least one section of the background (02), and with the sections of the background (02) in where the punched light is present in the individual periods of time, change over time; - Obtaining at least one image of the scene made up of a plurality of image sections; and - Deriving an at least one-channel key signal for distinguishing the at least one foreground object (01) from the background (02) by determining a distribution of the key light imaged in the at least one image, with a directly imaged portion (06) of the Key light is determined from the individual image sections, which correspond locally to those of the sections of the background in which the key light is present in the respective time sections. procedure after claim 1 , characterized in that the sections of the background (02) in which the punched light is present in the individual time sections are each formed by at least one line; and that the image sections from which the directly imaged portion (06) of the key light is determined are each formed by an image line. procedure after claim 2 , characterized in that a first image converter (09), which operates according to a rolling shutter method, is used to determine the first channel of the punching signal. Procedure according to one of Claims 1 until 3 , characterized in that the punch light is used for the background (02) by illuminating the background (02) with a laser beam (04) deflected line by line from a laser projector (03), the laser beam (04) forming the punch light. Procedure according to one of Claims 1 until 4 , characterized in that the use of the key light for the background (02) takes place in that a two-dimensional display located in the background (02) emits the key light line by line. Procedure according to one of Claims 1 until 5 , characterized in that to determine the distribution of the key light imaged in the at least one image, the at least one image is first divided spectrally, as a result of which the at least one image is divided into a foreground channel and a background channel comprising the separated key light. Procedure according to one of Claims 1 until 6 , characterized in that in order to determine a second channel of the key signal, the key light imaged in all of the image sections of the at least one image is determined in the individual time sections. Procedure according to one of Claims 1 until 7 , characterized in that the punched light has a spectrum with one or more spectral lines and/or spectral bands. procedure after claim 8 , characterized in that the spectral bands have a bandwidth which is at most 5 nm. Arrangement for imaging a scene comprising at least one foreground object (01) and one background (02), comprising the following components: - a device (03) for generating punched light, which is configured to use the punched light for the background (02) with a spatial and temporal variation, wherein the punched light in individual periods of time locally only in sections in each case in at least one section of the background ( 02) is present, and wherein the sections of the background (02) in which the punched light is present in the individual time sections change over time; and - an image recording and image processing device (08, 09, 11) for acquiring at least one image of the scene composed of a plurality of image sections; wherein the image recording and image processing device (08, 09, 11) is configured to derive an at least one-channel key signal for distinguishing the at least one foreground object (01) from the background (02) by determining a distribution of the key light imaged in the at least one image, wherein to determine a first channel of the key signal, a directly imaged component of the key light is determined from the individual image sections, which correspond locally to those of the sections of the background (02) in which the key light is present in the respective time sections.

Images