WO2013185787A1

WO2013185787A1 - Apparatus and method for compositing an image from a number of visual objects

Info

Publication number: WO2013185787A1
Application number: PCT/EP2012/002533
Authority: WO
Inventors: Matthias KUNTER; Sebastian Knorr; Benjamin WELLE; Ralf MÜLLER-ZIMMERMANN; Ronald KLUTH
Original assignee: Imcube Labs Gmbh
Priority date: 2012-06-15
Filing date: 2012-06-15
Publication date: 2013-12-19

Abstract

The present invention provides an apparatus (100) for compositing an image from a number of visual objects. The apparatus (100) comprises a color value assigning unit (10) for assigning object color values to first pixels of the image based on the number of visual objects, a visual highlighting unit (30) for visually highlighting second pixels of the image that have not been assigned an object color value, and an image displaying unit (50) for displaying the image with the visually highlighted second pixels to a user. Holes, even of small size, are thereby easily detectable by the user.

Description

Apparatus and method for compositing an image from a number of visual objects

FIELD OF THE INVENTION

The present invention relates to an apparatus for compositing an image from a number of visual objects. The present invention further relates to a corresponding method and computer program. BACKGROUND OF THE INVENTION

In today's movie production processes, images are often composited from a number of visual objects. One example are classical compositing methods, in which visual objects from separate sources, e.g., elements from live-action shootings and elements created by means of two-dimensional (2D) and/or three-dimensional (3D) computer graphics (CG) techniques, are combined into single images to create the illusion that all those elements were part of the same scene. Another example are so-called 2D to 3D conversion methods, in which images are first segmented into separate visual objects and the pixels of the visual objects are assigned depth values. "Virtual" stereoscopic 3D images are then generated by shifting pixels of the visual objects in accordance with their depth values. In both cases, the compositing of an image from a number of visual objects comprises a step of assigning object color values to first pixels of the image based on the number of visual objects. This object color value assigning step may, however, also lead to holes, i.e., second pixels of the image that have not been assigned an object color value. The occurrence of such holes is in general undesirable and should therefore be corrected before the image is finalized. However, depending on the size and/or appearance of the holes, they may be difficult to detect by a user. SUMMARY OF THE INVENTION

It is an object of the present invention to provide an apparatus for compositing an image from a number of visual objects, wherein holes in the image can more easily be detected by a user. It is a further object of the present invention to provide a corresponding method and computer program. In a first aspect of the present invention, an apparatus for compositing an image from a number of visual objects is presented, wherein the apparatus comprises:

a color value assigning unit for assigning object color values to first pixels of the image based on the number of visual objects;

a visual highlighting unit for visually highlighting second pixels of the image that have not been assigned an object color value, and;

an image displaying unit for displaying the image with the visually highlighted second pixels to a user.

The invention is based on the idea that by visually highlighting second pixels of the image, i.e., pixels that have not been assigned an object color value, holes, even of small size, are easily detectable by a user in an image composited from a number of visual objects when the image with the visually highlighted second pixels is displayed to the user.

It is preferred that the apparatus further comprises an opacity value providing unit for providing opacity values associated with the first pixels of the image, that the visual highlighting unit is adapted to further visually highlight third pixels of the image that have been assigned an object color value but that have an associated opacity value indicating a reduction in opacity, and that the image displaying unit is adapted to display the image with the further visually highlighted third pixels to the user.

This allows the user to easily detect not only holes but also (semi-)transparent regions in the image. It is further preferred that the visual highlighting unit is adapted to temporally vary a background color value of the second pixels and/or the third pixels over at least two different highlight color values, and that the image displaying unit is adapted to display the image with the background color value of the second pixels and/or the third pixels being temporally varied over the at least two different highlight color values.

In particular, it is preferred that the temporal variation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values consists of a temporal alternation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values. By doing so, i.e., by visually highlighting the second pixels and/or the third pixels by means of a "background color blinking", holes and/or (semi-)transparent regions, even of small size, are particularly easily detectable by the user in the image.

It is further preferred that the visual highlighting unit is adapted to perform an operation of lowering the opacity values associated with the third pixels. Lowering the opacity values associated with the third pixels makes it easier for the user to detect (semi-)transparent regions, in particular, those of high opacity, in the image.

It is preferred that the visual highlighting unit is adapted to perform an operation of converting the object color values assigned to the first pixels to grayscale values.

In this case, since the image composited from the number of visual objects is displayed by the image displaying unit in grayscale, the holes and/or (semi-)transparent regions that are visually highlighted by means of the "background color blinking" effect may even be more easily detectable by the user in the image.

It is further preferred that the visual highlighting unit is adapted to perform an operation of modifying a background color variation frequency with which the background color value of the second pixels and/or the third pixels is temporally varied over the at least two different highlight color values.

It is preferred that the visual highlighting unit is adapted to perform an operation of modifying the at least two different highlight color values. The two last-mentioned features inter alia allow adapting the visual highlighting of the second pixels and/or the third pixels to the particular circumstances, e.g., the preferences of the user.

It is further preferred that the apparatus further comprises a user input unit for allowing the user to control the operation performed by the visual highlighting unit.

In particular, it is preferred that the user input unit is adapted to allow the user to control at least one of: an amount of lowering the opacity values associated with the third pixels; whether or not the object color values assigned to the first pixels are converted to grayscale values; the background color variation frequency, and; the at least two different highlight color values.

It is further preferred that the at least two different highlight color values consist of just two different highlight color values.

It is preferred that the number of visual objects are visual objects that are segmented in a source image, and that the apparatus is adapted to composite the image by spatially shifting the number of visual objects based on depth values assigned to the number of visual objects.

In a second aspect of the present invention, a method for compositing an image from a number of visual objects is presented, wherein the method comprises:

assigning object color values to first pixels of the image based on the number of visual objects;

visually highlighting second pixels of the image that have not been assigned an object color value, and;

displaying the image with the visually highlighted second pixels to a user.

In a third aspect of the present invention, a computer program for compositing an image from a number of visual objects is presented, the computer program comprising program code means for causing an apparatus as defined in claim 1 to carry out the steps of the method as defined in claim 13, when the computer program is run on a computer controlling the apparatus. It shall be understood that the apparatus of claim 1 , the method of claim 13 and the computer program of claim 14 have similar and/or identical preferred embodiments, in particular, as defined in the dependent claims.

It shall be understood that a preferred embodiment of the invention can also be any combination of the dependent claims with the respective independent claim.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. In the following drawings:

Fig. 1 shows schematically and exemplarily an embodiment of an apparatus for compositing an image from a number of visual objects,

Fig. 2 shows an example of an image composited from a number of visual objects by means of the apparatus shown in Fig. 1 ,

Fig. 3 shows an example of a visually highlighted image, and

Fig. 4 shows a flowchart exemplarily illustrating an embodiment of a method for compositing an image from a number of visual objects.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 shows schematically and exemplarily an embodiment of an apparatus 100 for compositing an image from a number of visual objects. The apparatus 100, in this embodiment, is an apparatus for converting 2D images into "virtual" stereoscopic 3D images (a.k.a. "2D to 3D conversion"). The apparatus 100 comprises a color value assigning unit 10 for assigning object color values to first pixels of the image based on the number of visual objects. This is described in more detail with further reference to Fig. 2, which shows an example of an image 4 composited from a number of visual objects VO, by means of the apparatus 100 shown in Fig. 1. Fig. 2 shows a source image 1 , in this example, a natural 2D image (a.k.a. "monoscopic image") from a 2D image sequence captured by a movie camera. The source image 1 of this example shows the projection of a simple 3D scene consisting mainly of a house, a flagpole, and a tree located on a ground plane with a sunny sky located in the background. In order to convert the source image 1 into a stereoscopic 3D image, the source image 1 has first been segmented into a number of visual objects VOi. Here, the subscript "i" is used as an index that denotes a given visual object, e.g., a first visual object is denoted by VO, , a second visual object is denoted by V0₂, etc. The segmentation of the source image 1 is represented, in this example, by a so-called segmentation mask 2, e.g., a grayscale image, in which pixels belonging to a given visual object, for example, VO^ are assigned a grayscale value that differs from a grayscale value assigned to pixels belonging to another visual object, for example, V0 . This is visualized in the figure by different shadings which shall represent different grayscale values in the segmentation mask 2. The figure thus shows that the source image 1 has been segmented, in this example, into five visual objects VOi, i-e., the ground plane VO^ the house V0₂, the flagpole V0₃, the tree V0₄, and the sunny sky in the background V0₅. Moreover, the pixels belonging to the visual objects VO, have been assigned depth values. The depth values indicate how far the captured 3D scene has been away from the capturing movie camera, i.e., they indicate for each pixel belonging to a visual object VO, the distance from the capturing movie camera to the location of the corresponding 3D scene point that projected into the source image 1 at the respective pixel. The depth values are represented, in this example, by a so-called depth map 3, e.g., a grayscale image, in which darker grayscale values indicate larger depth values and lighter grayscale values indicate smaller depth values. Again, this is visualized in the figure by different shadings which shall represent different grayscale values in the depth map 3. The figure thus shows that the pixels belonging to the sunny sky in the background have been assigned a very dark grayscale value indicating that the part of the 3D scene corresponding to the visual object V0₅ has very large depth values, i.e., it has been located at a very large distance from the capturing movie camera. The pixels belonging to the house, the flagpole, and the tree have been assigned medium grayscale values indicating that the parts of the 3D scene corresponding to the visual objects V0₂, V0₃, and V0 have me- dium depth values, i.e., they have been located at a medium distance from the capturing movie camera. Finally, the pixels belonging to the ground plane have been assigned grayscale values that are comparably light at the lower border of the depth map 3 but become darker and darker in the direction towards the upper border of the depth map 3. This indicates that the part of the 3D scene corresponding to the visual object VO_! ex- tends continuously into depth with the regions that are projected at the lower border of the source image 1 having depth values that are smaller than the depth values of the regions that are projected at a higher position in the source image 1.

The apparatus 100 is adapted to generate a "virtual" stereoscopic 3D image by compositing from the number of visual objects VO, an image 4 which together with the source image 1 constitutes an image pair that appears as if it had been captured by a "virtual" stereoscopic movie camera. In more detail: The image 4 is composited from the number of visual objects VO, such that the 3D scene is shown in this image with a perspective that slightly differs from the perspective of the source image 1. This implicates that the projection of a given 3D scene point in the image 4 is spatially shifted slightly with respect to the corresponding projection in the source image 1 , wherein the magnitude of the spatial shift depends inter alia on the distance at which the 3D scene point has been located from the capturing movie camera. Please also note that the term "virtual" is used here to indicate that the stereoscopic 3D image is not actually captured by a stereoscopic movie camera, but that it is generated by means of image processing techniques from the source image 1. The "virtual" stereoscopic 3D image may be viewed, just like a "real" stereoscopic 3D image, by means of any suitable viewing technique that visually separates the source image 1 from the image 4 such that a viewer perceives with each of his two eyes only one of the two images, respectively. This may be achieved, for example, by a suitable multiplexing technique based on, e.g., an anaglyphic color multiplexing, a spatial multiplexing, a polarization multiplexing, a temporal multiplexing, or a combination of such techniques.

The compositing of the image 4 from the number of visual objects VO, is performed, in this embodiment, by calculating for each of the pixels of the visual objects VO, in the source image 1 a pixel shift in dependence of the depth values assigned to them. The pixel shifts indicate where the 3D scene points corresponding to the pixels in the source image 1 project in the image 4, i.e., they represent the difference in perspective between the source image 1 and the image 4 (as described above). In a basic realization of the apparatus 100, the object color assigning unit 10 may then assign object color values to first pixels of the image 4 based on the number of visual objects VO, by simply copying the object color values of each of the pixels of the visual objects VO, in the source image 1 to a respective first pixel in the image 4 that is spatially shifted with respect to the corresponding pixel in the source image 1 by its calculated pixel shift. (Please note that this may require a quantization of the pixel shifts to integer values.) In this embodiment, however, a more complex approach is used which is able to efficiently "resample" the image 4 in case of non-integer pixel shifts, i.e., pixel shifts that do not fit into the discrete pixel raster of the image 4. The resampling may be based on techniques as described, for example, in the textbook "Digital Image Warping" by G. Wolberg, IEEE Society Press Monograph, 1990, the contents of which are incorporated herein by reference. Moreover, a layer-based compositing is utilized, wherein the different visual objects VO, are managed in separate layers (not shown in Fig. 2) and wherein the object color assigning unit 10 assigns the object color values to the first pixels of the image 4 successively in a layer- by-layer fashion. The layers may thereby be selectively activated resp. de-activated by a user. The apparatus 100, in this embodiment, further comprises an opacity value providing unit 20 for providing opacity values associated with the first pixels of the image 4. The opacity values (a.k.a. "alpha channel") range logically from 0% opacity up to 100% opacity and may be stored, for example, in a so-called opacity map (not shown in Fig. 2), e.g., a grayscale image, in which darker grayscale values indicate smaller opacity values and lighter grayscale values indicate larger opacity values. An opacity value of 0% indicates that a pixel is completely transparent while an opacity value of 100% indicates that the pixel is completely opaque. In this embodiment, the opacity values are used to indicate, for example, a reduction in opacity of first pixels of the image 4 that are assigned object color values from border pixels of a visual object VO, in the source image 1 with a non- integer pixel shift. As another example of first pixels of the image 4 with reduced opacity, the case may be considered, in which e.g. a user reduced the opacity of a visual object VOi already during segmentation, because the visual object VOj itself is not fully opaque (e.g., glass, smoke, or fast moving visual elements that create so-called "motion blur").

When the image 4 is composited from the number of visual objects VOi in the above- described manner, discontinuities between the depth values of different visual objects VOj may lead to holes, i.e., second pixels of the image 4 that have not been assigned an object color value. These holes (a.k.a. "disocclusions" or "gaps") correspond to parts of the captured 3D scene that are occluded, and therefore not visible, in the source image 1 , but that are not occluded, and therefore visible, in the image 4 due to the difference in perspective. The missing object color values in the holes may be suitably filled-in, e.g., by assigning to the second pixels object color values based on the more distant visual object bordering on the hole. This process is called "in-painting" or "image completion" and may be performed either automatically or manually, e.g., by creating so-called "clean plates". However, depending on the resampling and/or the utilized "in-painting" process, small holes, such as holes comprising only single or few pixels, may still occur in the image 4. Moreover, in addition to holes resulting from depth discontinuities, second pixels of the image 4 that have not been assigned an object color value can also result in the case where one or more of the layers have not been activated by the user in the above- described layer-based processing.

Fig. 2 illustrates the occurrence of holes during the compositing of the image 4 from the number of visual objects VO,. Here, it is assumed that the difference in perspective between the source image 1 and the image 4 has let to disocclusions at the left border of the house V0₂ and of the tree V0₄ (visualized by the stippled lines). These gaps have been painted-in with suitable object color values from the ground plane VO,, resp. from the sunny sky in the background V0₅, but small groups of second pixels that have not been assigned an object color value nonetheless occurred in the image 4, e.g., due to resampling errors (visualized in the figure by the black spots). In addition, it is assumed that the layer used to manage the flagpole V0₃ has accidentally not been activated by the user resulting in the bigger hole shown on the right side of the image 4. Depending on the actual processing, e.g., the initialization of the image 4, these holes may appear in the image 4 as black pixels or as pixels of a random color or the like.

It should be apparent that the occurrence of holes in the image 4 is in general undesirable in an apparatus 100 for compositing an image 4 from a number of visual objects VO,, here, an apparatus for converting 2D images into "virtual" stereoscopic 3D images, and should therefore be corrected, for example, by repeating the layer-based processing with all layers activated or by manually panting-in small holes with suitable object color values, before the image 4 is finalized. However, depending on the size and/or appearance of the holes, they may be difficult to detect by a user. For this reason, the apparatus 100 comprises a visual highlighting unit 30 for visually highlighting the second pixels of the image 4, i.e., the pixels of the image 4 that have not been assigned an object color value, and an image displaying unit 50 for displaying the image 4 with the visually highlighted second pixels to a user. Moreover, in this embodiment, the visual highlighting unit 30 is adapted to further visually highlight third pixels of the image 4, i.e., pixels of the image 4 that have been assigned an object color value but that have an associated opacity value indicating a reduction in opacity (also called "(semi- )transparent regions" in this specification), and the image displaying unit 50 is adapted to display the image 4 with the further visually highlighted third pixels to the user. Such (semi-)transparent regions are not explicitly shown in Fig. 2, but may occur due to the reasons described above. It is noted, however, that in other embodiments, a visual highlighting of the third pixels does not have to be provided.

Here, the visual highlighting unit 30 is adapted to temporally vary a background color value of the second pixels and/or the third pixels over at least two different highlight color values, and the image displaying unit 50 is adapted to display the image 4 with the background color value of the second pixels and/or the third pixels being temporally varied over the at least two different highlight color values.

In some embodiments, the temporal variation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values may consist, for example, of random or quasi-random changes of the background color value. In other words, in such embodiments, a repetition of the at least two different highlight color values is not provided for. In this embodiment, however, the temporal variation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values consists of a temporal alternation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values. By doing so, i.e., by visually highlighting the second pixels and/or the third pixels by means of a "background color blinking", holes and/or (semi- )transparent regions, even of small size, are particularly easily detectable by the user in the image 4.

In the layer-based compositing utilized here, this "background color blinking" effect can be realized, for example, by positioning a blinking background color layer behind the layers used for managing the different visual objects VOj, such that the visual objects VO, are composited over the blinking background color layer. A user who views the image 4 can then easily detect holes and/or (semi-)transparent regions by virtue of the colors that "shine through" these not fully opaque areas from the blinking background color layer. In this exemplary realization, the composited color values of the second pixels, i.e., the pixels that have not been assigned an object color value ("holes"), correspond to the temporally alternating background color value, and the composited color values of the third pixels, i.e., the pixels that have been assigned an object color value but that have an associated opacity value indicating a reduction in opacity ("(semi-)transparent regions"), correspond to a mixture of their object color values and the temporally alternating background color value. In more detail: For a given third pixel, the percentage of its object color value in the mixture corresponds to its opacity and the percentage of the temporally alternating background color value corresponds to its reduction in opacity (i.e., its transparency) as indicated by its associated opacity value. For example, if a given third pixel has an associated opacity value of 75%, its composited color value corresponds to a mixture of only 30% of its object color value and of 100% - 30% = 70% of the temporally alternating background color value. If the third pixel has an associated opacity value of 75%, its composited color value corresponds to a mixture of 75% of its object color value and of only 100% - 75% = 25% of the temporally alternating background color value. In other words: The more opaque a given third pixels is, the higher is the percentage of its object color value in the mixture and the lower is the percentage of the temporally alternating background color value.

In this embodiment, the visual highlighting unit 30 is therefore adapted to perform an operation of lowering the opacity values associated with the third pixels, in particular, down to 0% opacity. If we stay with the above-given example, this means that, in one realization, if the third pixel has an associated opacity value of 30%, the opacity value may be lowered to any value ranging from 0% up to 30%. If the third pixel has an associated opacity value of 75%, the opacity value may be lowered to any value ranging from 0% up to 75%. In another realization, it may only be possible to lower the opacity values associated with the third pixels to just 0% opacity. By lowering the opacity values asso- ciated with the third pixels, the percentage of the temporally alternating background color value in the composited color values of the third pixels can be increased. This makes it easier for the user to detect (semi-)transparent regions, in particular, those of high opacity, in the image 4.

The visual highlighting unit 30 is adapted, in this embodiment, to further perform an operation of converting the object color values assigned to the first pixels to grayscale values. In this case, since the image 4 composited from the number of visual objects VO, is displayed by the image displaying unit 50 in grayscale, the holes and/or (semi- )transparent regions that are visually highlighted by means of the "background color blinking" effect may even be more easily detectable by the user in the image 4. In this embodiment, the visual highlighting unit 30 is also adapted to perform an operation of modifying a background color variation frequency with which the background color value of the second pixels and/or the third pixels is temporally varied over the at least two different highlight color values, and/or to perform an operation of modifying the at least two different highlight color values.

To this end, the apparatus 100 comprises, in this embodiment, a user input unit 40 for allowing the user to control the operation performed by the visual highlighting unit 30. In more detail, the user input unit 40 is adapted to allow the user to control an amount of lowering the opacity values associated with the third pixels. In one realization, this may be realized by weighting the opacity values associated with the third pixels by a common weighting factor in the range from 0 up to 1 , wherein a common weighting factor of 0 results in a lowering of the opacity values associated with the third pixels down to 0% opacity whereas a common weighting factor of 1 does not lead to a change in the opacity values associated with the third pixels. The control of the common weighting factor by the user may be realized, e.g., by a slide control, a control dial, a text box that allows entering a number between 0 and 1 , or another suitable user interface element provided by the user input unit 40. In another realization, the user input unit 40 may allow the user to choose between only two different "opacity modes" for visually highlighting the third pixels: (1 ) A first mode in which the opacity values associated with the third pixels are lowered to 0% opacity (corresponding to an amount of lowering of 100%), and; (2) a second mode in which the opacity values associated with the third pixels are not lowered (corresponding to an amount of lowering of 0%). In this case, the user input unit 40 may provide the user with e.g. a button, a check box, or another suitable user interface element for switching between the two "opacity modes". Of course, other realizations are also possible and can be understood by those skilled in the art in practicing the claimed invention.

The user input unit 40 is further adapted, in this embodiment, to allow the user to control whether or not the object color values assigned to the first pixels are converted to grayscale values. Again, this may be realized by providing the user with e.g. a button, a check box, or another suitable user interface element for switching between the two modes.

Yet further, the user input unit 40 is adapted, in this embodiment, to allow the user to control the background color variation frequency, i.e., the frequency with which the back- ground color value of the second pixels and/or the third pixels is temporally varied over the at least two different highlight color values. A suitable user interface element that is provided by the user input unit 40 could be, e.g., a slide control, a control dial, or a text box that allows entering a number indicating the background color variation frequency. Suitable values for the background color variation frequency are, e.g., values in the range of 0.25 Hz to 4 Hz.

In this embodiment, the user input unit 40 is also adapted to allow the user to control the at least two different highlight color values. In one example, the at least two different highlight color values consist of just two different highlight color values that can be selected by the user by means of, e.g., a color picker, one or more slide controls, one or more text boxes that allow entering (a) number(s) indicating the two different highlight color values, or other suitable user interface elements provided by the user input unit 40. In other examples, however, the at least two different highlight color values may consist of more than two different highlight color values, e.g., they may consist of three or four different highlight color values or they may be varied over a whole range of different highlight color values, such as in accordance with a linear color gradient, with the colors of a color wheel, or the like. The visual highlighting of the second pixels and/or the third pixels is described in more detail with further reference to Fig. 3, which shows an example of a visually highlighted image, here, the image 4 described with reference to Fig. 2 above.

Fig. 3 shows a timeline on which equidistant points in time are marked as t₀, , t₈. In this example, it is again assumed that the at least two different highlight color values, over which the visual highlighting unit 40 temporally varies the background color value of the second pixels and/or the third pixels, i.e., the holes and/or the (semi-)transparent regions, consist of just two different highlight color values, e.g., blue and green, black and white, or any other suitable color combination, that may have been selected by the user with the aid of the user input unit 40. The figure then shows a clipping from the upper right corner of the image 4 including the bigger hole shown on the right side of the image 4, which resulted from the fact that the layer used to manage the flagpole V0₃ had accidentally not been activated by the user. (Please note that Fig. 3 shows only a clipping of the image 4 in order to make the figure more comprehensible and easier to view. In practice, the whole image 4 would preferably be displayed to the user.) As can be seen from the figure, the image 4 is displayed to the user at the marked points in time t₀, t,, t₈ with the background color value of the "flagpole hole" being temporally alternated between the two different highlight color values (visualized in the figure by different shadings which shall represent the two different highlight color values). In the following, a method for compositing an image 4 from a number of visual objects VOi, here, a method for converting 2D images into "virtual" stereoscopic 3D images, will be described with reference to a flowchart shown in Fig. 4.

In step 101 , object color values are assigned to first pixels of the image 4 based on the number of visual objects VO,. Then, in step 103, second pixels of the image 4 that have not been assigned an object color value are visually highlighted, and in step 105, the image 4 with the visually highlighted second pixels is displayed to a user. The steps 101 , 103, and 105 of the method may respectively be performed by the color value assigning unit 10, the visual highlighting unit 30, and the image displaying unit 50 of the apparatus 100 for compositing an image 4 from a number of visual objects VO, described with reference to Fig. 1 above.

The method may further comprise a step 102, in which opacity values associated with the first pixels of the image 4 are provided. This step may be performed by the optional opacity value providing unit 20 of the apparatus 100 for compositing an image 4 from a number of visual objects VOi described with reference to Fig. 1 above. In this case, the step 103 may comprise further visually highlighting third pixels of the image 4 that have been assigned an object color value but that have an associated opacity value indicating a reduction in opacity. The step 105 then comprises displaying the image 4 with the further visually highlighted third pixels to the user. The further features of the method for compositing an image 4 from a number of visual objects VO| can be understood by those skilled in the art in practicing the claimed invention from a study of the corresponding features of the apparatus 100 for compositing an image 4 from a number of visual objects VOj described with reference to Fig. 1 above. In particular, it is understood that the method may comprise a step 104 that allows the user to control the operation performed for visually highlighting the second pixels and/or the third pixels, as is described above.

In the above-described embodiments, the apparatus 100 resp. the method for compositing an image 4 from a number of visual objects VOi is an apparatus resp. method for converting 2D images into "virtual" stereoscopic 3D images. Such "virtual" stereoscopic 3D images can also be subsets of so-called multi-view 3D images that can be displayed, e.g., on autostereoscopic multi-view 3D displays, on stereoscopic 3D displays providing head-motion parallax, or the like. Moreover, in other embodiments, the apparatus 100 resp. the method for compositing an image 4 from a number of visual objects VO, can also be something else, e.g., a classical compositing apparatus resp. method, in which visual objects from separate sources, e.g., elements from live-action shootings and elements created by means of 2D and/or 3D CG techniques, are combined into single images to create the illusion that all those elements were part of the same scene.

The present invention is also applicable when an image is generated completely by means of 2D and/or 3D CG techniques. The phrase "compositing an image from a number of visual objects", as used in this specification and in the appended claims, shall therefore be understood to also encompass the case where the visual objects consist of only 2D and/or 3D CG objects and the image is generated by means of 2D and/or 3D rendering techniques, such as ray tracing.

It is to be understood that the second pixels, i.e., the pixels of the image 4 that have not been assigned an object color value, may be visually highlighted in a way that may differ from the way in which the third pixels, i.e., the pixels of the image 4 that have been assigned an object color value but that have an associated opacity value indicating a reduction in opacity, are visually highlighted. In particular, the second pixels may be visually highlighted by means of at least two highlight color values that may differ - partly or completely - from at least two different highlight color values that are used to visually highlight the third pixels.

Likewise, it is to be understood that also among the second pixels and/or among the third pixels, not all pixels have to be visually highlighted in the same way. For example, a first subset of the second pixels may be visually highlighted by means of at least two highlight color values that may differ - partly or completely - from at least two different highlight color values that are used to visually highlight a second subset of the second pixels. Thus, the second pixels may be visually highlighted e.g. by a spatially alternating and temporally varying background color pattern. It may even be possible that e.g. the second pixels are visually highlighted by a spatially random color arrangement that temporally varies for each of the second pixels over at least two different highlight color values. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Operations like the assigning of object color values to first pixels of the image 4 based on the number of visual objects VOi, the visual highlighting of second pixels of the image 4 that have not been assigned an object color value, et cetera, performed by one or several units, can be performed by any number of units or devices. These operations and/or the control of the apparatus 100 for compositing an image 4 from a number of visual objects VOj in accordance with the method for compositing an image 4 from a number of visual objects VOj can be implemented at least in part as program code means of a computer program and/or as dedicated hardware.

A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Any reference signs in the claims should not be construed as limiting the scope.

Claims

1 . Apparatus (100) for compositing an image (4) from a number of visual objects (VOi), wherein the apparatus (100) comprises:

a color value assigning unit (10) for assigning object color values to first pixels of the image (4) based on the number of visual objects (VOj);

a visual highlighting unit (30) for visually highlighting second pixels of the image (4) that have not been assigned an object color value, and;

an image displaying unit (50) for displaying the image (4) with the visually highlighted second pixels to a user.

2. The apparatus (100) according to claim 1 , further comprising an opacity value providing unit (20) for providing opacity values associated with the first pixels of the image (4), wherein the visual highlighting unit (30) is adapted to further visually highlight third pixels of the image (4) that have been assigned an object color value but that have an associated opacity value indicating a reduction in opacity, and wherein the image display- ing unit (50) is adapted to display the image (4) with the further visually highlighted third pixels to the user.

3. The apparatus (100) according to claim 1 or 2, wherein the visual highlighting unit (30) is adapted to temporally vary a background color value of the second pixels and/or the third pixels over at least two different highlight color values, and wherein the image displaying unit (50) is adapted to display the image (4) with the background color value of the second pixels and/or the third pixels being temporally varied over the at least two different highlight color values.

4. The apparatus (100) according to claim 3, wherein the temporal variation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values consists of a temporal alternation of the background color value of the second pixels and/or the third pixels over the at least two different highlight color values.

5. The apparatus (100) according to any of claims 2 to 4, wherein the visual highlighting unit (30) is adapted to perform an operation of lowering the opacity values associated with the third pixels.

6. The apparatus (100) according to any of claims 1 to 5, wherein the visual highlighting unit (30) is adapted to perform an operation of converting the object color values assigned to the first pixels to grayscale values.

7. The apparatus (100) according to any of claims 3 to 6, wherein the visual highlight- ing unit (30) is adapted to perform an operation of modifying a background color variation frequency with which the background color value of the second pixels and/or the third pixels is temporally varied over the at least two different highlight color values.

8. The apparatus (100) according to any of claims 3 to 7, wherein the visual highlighting unit (30) is adapted to perform an operation of modifying the at least two different highlight color values.

9. The apparatus (100) according to any of claims 3 to 8, further comprising a user input unit (40) for allowing the user to control the operation performed by the visual highlighting unit (30).

10. The apparatus (100) according to claim 9, wherein the user input unit (40) is adapted to allow the user to control at least one of: an amount of lowering the opacity values associated with the third pixels; whether or not the object color values assigned to the first pixels are converted to grayscale values; the background color variation frequency, and; the at least two different highlight color values.

1 1. The apparatus (100) according to any of claims 3 to 10, wherein the at least two different highlight color values consist of just two different highlight color values.

12. The apparatus (100) according to any of claims 1 to 11 , wherein the number of visual objects (VO,) are visual objects that are segmented in a source image (1 ), and wherein the apparatus (100) is adapted to composite the image (4) by spatially shifting the number of visual objects (VOj) based on depth values assigned to the number of visual objects (VOj).

13. Method for compositing an image (4) from a number of visual objects (VO,), wherein the method comprises:

assigning object color values to first pixels of the image (4) based on the number of visual objects (VO,); visually highlighting second pixels of the image (4) that have not been assigned an object color value, and;

displaying the image (4) with the visually highlighted second pixels to a user.

14. Computer program for compositing an image (4) from a number of visual objects (VOi), the computer program comprising program code means for causing an apparatus as defined in claim 1 to carry out the steps of the method as defined in claim 13, when the computer program is run on a computer controlling the apparatus.