DE102016013766A1 - Image compensation for a masking, direct-view augmented reality system - Google Patents

Abstract

Described is an image compensation for a masking, direct view augmented reality system. In one or more embodiments, an augmented reality device includes an emissions display layer for presenting emission graphics to a user's eye, and an attenuation display layer for presenting attenuation graphics between the emission display layer and a real world scene to keep light of the real world scene away from the emission graphics. A bright area compensation module complements an attenuation graph based on an attribute of an observer's eye, such as a pupil size, to produce an expanded attenuation graph that keeps additional light off to compensate for an unintended bright area. A dark area compensation module conceals an unintended dark area with a replica graphic in the emission display layer that reproduces the appearance of the real world scene in the unintended dark area. A camera provides the light data used to generate the replica artwork.

Description

background

The augmented reality is extremely promising for providing information that is important to a person's current environment. In augmented reality, visual images are added to a scene of an environment being viewed by a person. The visual images are synthetic graphic images (hereinafter also referred to as "synthetic graphics") generated by a computer and added to a real world scene. For example, a synthetic graphic may be superimposed on or superimposed on a real world scene. In addition, a synthetic graphic can be inserted into the real world scene in such a way that the inserted visual image of the synthetic graphic in the field of vision seems to be partly behind a real world object at a certain depth. With a more sophisticated insertion of a synthetic graphic, the synthetic graphics may appear to be integrated into the real world scene, allowing for greater realism or even annotations that do not block important visual aspects, but clearly indicate which visible object is annotated.

Synthetic graphics can be helpful, fun, entertaining, and more. For example, an augmented reality system may display directional arrows and textual instructions down roads or corridors to allow a person to move in a desired direction. In another example, recommended tolerances or detailed installation instructions for a motor vehicle component are displayed to assist a car mechanic. In yet another example, an augmented reality system may project the results of a planned structural retrofit or new furniture set up into a real world view of the interior of a building. Synthetic graphics can also be used for entertainment, such as displaying funny comments or recent social media communications over the minds of friends and employees. In another example, an electronic "buddy" who acts as a playmate, tutor, or teacher may be employed in a child's environment. This augmented reality has the potential to positively impact many areas of life by allowing the addition of synthetic graphics in real world scenes.

An augmented reality system can be configured in a variety of ways. For example, an augmented reality system may be configured as eyeglasses that must be placed in front of a user's eyes as swimming goggles that cover the eyes, as another type of head mounted display, as a display screen, as a video projector, or as one any combination of these. These systems also include functionality for creating the synthetic visual presentation graphic in conjunction with the real world scene.

Augmented reality systems are typically built as one of two major types, namely, browsing systems that support direct view, or as camera-mediated systems that support indirect viewing. In systems of the former type, a person looks through a display screen presenting a synthetic graphic and can also directly view the real world scene located behind the display screen. In systems of the latter type, a person looks at a display screen to see both the synthetic graphics and the real world scene presented in common on the display screen. For both types of augmented reality systems, the trend is to compose a synthetic graph with the real world scene to create a composite view. Disadvantageously, a number of technical hurdles prevent the development of the full potential of augmented reality in both the direct view approach and the camera switching approach.

In an augmented reality system with camera switching, a person's view is usually limited to a single-layer display screen that presents the world as captured using a camera, but the display screen is physically opaque to the real world from the perspective of the eye is the person looking at the display screen. In other words, the person's view is indirect because the person does not receive light that comes from the real world. Instead, there is a camera with an image sensor in the real world scene, and two or more miniature display screens show the real world scene by accessing data from an image sensor of the camera. A camera-augmented augmented reality system is analogous to seeing through a "view finder" of a modern camcorder in which an LCD (liquid crystal display) LCD presents a view of what the camcorder's image sensor is currently "seeing" while illuminating - or focus information superimposed on the recorded scene.

Accordingly, camera-enhanced augmented reality systems allow for accurate compositing because the combining of the real world scene and the synthetic graphics are performed electronically and then displayed together on a single-layer display screen. Disadvantageously worsen camera-mediated systems provide a person's view of the real-world environment due to camera resolution and latency limitations due to image processing components and limitations of the display screen itself. A camera-augmented reality system also becomes dark and completely opaque when a power failure occurs in the system.

In contrast, in a direct-view augmented reality system, a person's eyes are exposed to light rays originating from real world objects. A direct view system includes a display screen having one or more display layers adapted for insertion between the eyes of a person and a real world scene being viewed directly. An example of a display layer of a display screen through which at least partially passes is an emission display layer. An emission display layer may be primarily transparent to the light originating from real world objects, and the emission display layer may also selectively present images that are sufficiently opaque that they are perceptible to a human eye. Therefore, augmented reality based systems use advanced optics integrated into one or more display layers to create an illusion in which a transparent display is generated at a fixed distance in front of the viewer superimposed as a real world scene. Since real world objects are viewed directly, the real world is viewed with minimal latency.

Disadvantageously, augmented reality systems based on direct view also have a number of disadvantages, such as a narrow field of view. In addition, synthetic graphics added to a real world scene are subject to resolution limitations of the display subsystem, which may include multiple display layers of a display screen, along with associated processing capabilities. In addition, as synthetic graphics are superimposed by a direct-view augmented reality system, a person's ability to appropriately focus on synthetic graphics may be compromised due to anatomical features of the human eye. Presenting synthetic graphics such that the eye and brain are caused to perceive synthetic graphics at a desired depth in the real world view is difficult. This is especially true when a person's eye and brain try to emit light from a distance of a few centimeters in front of the eye with light from real world objects that are tens, hundreds, or even more meters away from the person combine.

To facilitate accurate composition of near-surface synthetic graphics and remote real-world objects, a free-form optics prismatic lens can be used as the display layer of a display screen in a direct view augmented reality system. A prismatic lens with freeform optics presents synthetic graphics with appropriate focus characteristics that assist a person in perceiving the displayed synthetic graphics without blurring and at a desired depth in the field view. An integrated low weight eye tracking can be used to estimate where a person's eye is currently focusing which object to determine the depth of focus.

However, a problem that remains difficult to solve remains the introduction of masking synthetic graphics in direct-view augmented reality systems. A synthetic graphic that is added to a real world scene obscures a person's view of one or more real world objects. However, although the obscuring synthetic graphics blocks the view of some real world objects, the obscuring synthetic graphic is desirable because the graphic provides information to the person, such as a description or other comment of an object in the real world scene. In contrast, distracting image artifacts are undesirable. It is also problematic in direct view augmented reality systems that such systems produce distracting image artifacts in conjunction with the presentation of obscuring synthetic graphics. An example of such an image artifact is a dark field around a displayed synthetic graphic. Such image artifacts result from multiple show through display layers, where the light comes from the real world scene or from the interaction of multiple layers with incident light.

A comparatively simple solution, at least from the optical perspective, for introducing masking synthetic graphics into a real-world scene uses a conventional stack of LCD layers. The stack of LCD layers can both attenuate the real world and provide foreground synthetic graphics along with appropriate focus hints. Disadvantageously, such devices have low spatial resolution and require computationally expensive optimization processing to display the synthetic graphics. In addition, in order for the stacked LCD layers to achieve angular resolution, the resolution of the LCD layers is increased. However, LCD layers of a sufficiently high resolution may result in the transmitted view of the real world to blur diffraction effects. The use of three such LCD layers stacked together further aggravates this problem. In addition, when a conventional stack of LCD layers is used to dampen a person's view of the environment to aid in viewing a synthetic graphic, a dark courtyard is visible around the synthetic graphic. Alternative optical systems have been proposed, but these alternative optical systems are expensive and complex. Other alternatives affect other desirable aspects of an augmented reality system, such as good contrast.

Summary

Described is an image compensation for a masking, direct view augmented reality system. An augmented reality device includes a display screen having two display layers that are selectively transparent. If the two display layers are both transparent, a user can look through the two layers and directly look at a real world scene. However, if one or both of the layers display a synthetic graphic, the displayed graphic obscures at least a portion of the user's view of the real world scene, which can produce unwanted image artifacts. Two types of distracting and unwanted image artifacts are unwanted bright areas and unwanted dark areas.

In one aspect described, an augmented-reality facility based on direct view compensates for an unintended bright area. In a further described aspect, a direct view augmented reality facility compensates for an unintended dark area. In both aspects described, an augmented reality device includes a display screen having an emission display layer and a damping display layer to present synthetic graphics. In operation, the emissions display layer presents an emissions graphic that is displayed to an eye of the user and includes some informational content. The attenuation display layer is positioned between the emission display layer and the real world scene and presents an attenuation graph that attenuates (eg, substantially keeps away) light originating from the real world scene. The emission graph or the attenuation graph may be prepared or presented using intensity maps or matte, respectively indicating areas of the emission display layer or the attenuation display layer to be activated instead of being transparent to light.

An undesirable bright region is visible when light originating from the real world scene fades (eg, faint) at least part of an emission graphic presented by the emission display layer. For example, an inner light halo at the boundary of the emission graph may evolve depending on the size of a corresponding attenuation graph that is responsible for keeping away from incident light. To compensate for the unwanted bright area, a bright area compensation module supplements the mute graph to expand the size of the mute graph. In operation, the bright area compensation module supplements the attenuation graph based on an attribute of a user's eye, such as the radius of the pupil of the eye. A convolution filter can be set and applied using the eye attribute to produce the expanded attenuation graph. The Enhanced Reality feature displays the expanded attenuation graph from the user's perspective behind the emissions graph. The expanded attenuation graph display compensates for the unintended bright area that would otherwise be visible to the user.

An undesirable dark area is visible when an attenuation graph keeps too much light off, which is the case, for example, in a zone that is not covered by a corresponding emission graph from the perspective of the user's eye. For example, an outer dark halo at the boundary of the emission graph may develop when a corresponding attenuation graph extends beyond the emission graph. To compensate for the unwanted dark area, a dark area compensation module lightens the dark area using the emission display layer. In operation, the dark area compensation module uses a camera to obtain data representing light coming from the real world scene. This light data is used to create a replica graphic that reproduces the appearance of the real world scene for the unintended dark area. Furthermore, the creation of the replica graphic can be adjusted by defocusing an attenuation map for the enhanced attenuation graph to accommodate lens blurring that results from the attenuation graph being out of focus for the user's eye. The augmented reality facility uses the emission display layer to present the replica graph in a region that bounds an edge of the emission graph where the expanding attenuation graph extends beyond an edge of the emission graph. The display of the replica graphic compensates for the unintended dark area by reproducing the appearance of the real world scene in a zone of the intended dark area that would otherwise be visible to the user.

The present summary, in a simplified form, provides an introductory selection of concepts, which are further described below in the Detailed Description. The present summary is not intended to identify per se essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

Brief description of the drawing

The detailed description will be presented on the basis of the accompanying figures.

1 FIG. 10 illustrates an environment for exemplary embodiments operable to implement image compensation for a hidden direct-view augmented reality system. FIG.

2 shows an example of simulated images of a real world scene and a masking synthetic graphic, which are assembled in a composite image view.

3 shows an example of an augmented reality device with a see-through emission display layer and a damping display layer.

4A and 4B show a first visual composition problem that occurs when an emission display layer is operated without a damping display layer.

5A and 5B show a second visual composition problem that occurs with a direct implementation of a damping display layer.

6 shows an example of an image compensation module including a bright area compensation module and a dark area compensation module.

7 shows exemplary aspects for displaying an emissions graph and an attenuation graph.

8th FIG. 12 schematically illustrates an exemplary approach to supplementing a sharp attenuation graph to produce an expanded attenuation graph.

9 FIG. 10 shows an example of multiple physical properties associated with using a radius of a pupil to determine a factor to supplement an attenuation graph.

10 shows a problem that arises from the fact that an attenuation graph is increased such that an undesirable dark area is visible.

11 shows exemplary aspects for adjusting an attenuation attitude to cope with an out-of-focus attenuation plot.

12A and 12B show exemplary aspects for generating a replica graphic using a replica mask and light detected by a camera.

13 FIG. 12 is a schematic diagram of an augmented reality device that supports a composite image view. FIG.

14 shows an exemplary result of image compensation for a masking graphic by an image augmentation module of an augmented reality device.

15 FIG. 10 is a flowchart illustrating an exemplary procedure according to one or more exemplary embodiments.

16A and 16B Figure 2 shows two examples of simulated composite image views resulting from misalignment.

17A and 17B Figure 2 shows two examples of simulated composite image views that result in overemphasizing the effects of image blur.

18A and 18B Figure 2 shows two examples of simulated composite image views that result in under-emphasizing the effects of image blur.

19A and 19B Figure 2 shows two examples of simulated composite image views that result from a user focusing at a depth different than that of the background.

20 shows an example of a simulated composite image view that results from a user focusing on the depth of the background.

21 FIG. 12 shows an exemplary system or apparatus that may be used for embodiments of image compensation for a hidden direct-view augmented reality system.

detailed description

Overview

Compared to augmented reality systems based on a camera-mediated indirect view, augmented-reality augmented reality systems have lower latency and provide a more authentic interaction with the real world as well as a more natural experience of the real world view. Disadvantageously, implementing a direct view augmented reality system presents a different set of problems. For example, displaying synthetic graphics such that the eye and brain are allowed to interpret the synthetic content at an intended depth in the real-world view is difficult. This is particularly because the eye and brain are trying to focus on beams of light that come from many meters away from the eye and beams of light that come from a distance of a few millimeters, and to combine them. These distances are different from each other by orders of magnitude, the different distances making the creation of a comparatively fluid, non-blurred composite view problematic. Furthermore, light rays from real world objects may interact with the light rays of the synthetic graphics such that they interfere with a user's ability to see one or the other as image artifacts are created. For example, areas that are darker or lighter than desired or intended may be visually visible to a user in an image view composing the synthetic graphics with the real world objects.

There is disclosed herein a technology for a direct view augmented reality system that allows image composition for a viewing user. The direct view augmented reality system includes an emission display layer and a damping display layer placed between the eyes of a user and a real world scene. The technology advantageously compensates for unwanted image artifacts resulting from an interaction between the user's eyes, the real world objects, and the display layers of a direct view augmented reality system. Existing systems address unwanted image artifacts by using three or more stacked LCD layers, but this is computationally expensive and very likely blurs the transmitted view of the real world scene due to diffraction effects. However, the technology can compensate for the unwanted image artifacts using less than three LCD layers by adjusting what is displayed at each layer.

Instead of using three or more stacked LCD layers, a display panel for an augmented reality system may use a single display layer, such as an emission display layer. However, when a display layer having only one emission display layer is used, a light background of a real world scene may fade the appearance of a synthetic graphic presented by the emission display layer. To counteract this bleaching effect, a damping display layer is incorporated as a second layer between the emission display layer and the real world scene. The emission display layer presents an emission graphic, such as an alert or an instruction icon, to add information to the real world scene. The attenuation display layer presents an out-of-focus occluding graph (hereinafter referred to as "attenuation graph") behind the emission graph to prevent light from the real world scene from bleaching the emission graph. Unfortunately, the resulting combination of emission graphics and the same sized attenuation graph may develop image artifacts when viewed through an user's eye. For example, the combination makes an inadvertent light area and unintended dark area visible to the user. For example, an inner Hellhof and an outer Dark Court appear around a boundary zone of the emission graph.

To address the visual effects of image artifacts resulting from the introduction of a masking synthetic graphic into a direct view augmented reality display, devices and techniques for compensating such image artifacts are described. The use of the augmented reality device disclosed herein including a display screen with two display layers, as well as one or more associated techniques, advantageously enhances the appearance of the unwanted light areas or unwanted dark areas. In exemplary implementations, an out-of-focus hiding attenuation graph supports the view of an emissions graph added to a real world scene. The out-of-focus attenuation map may be implemented at a comparatively low resolution, which lower resolution advantageously reduces the diffraction effects that may otherwise result in blurring the direct view of the real world viewed through the transparent display layers.

In some embodiments, the disclosed technology includes an image compensation module employing techniques for enhancing areas having undesirable luminance levels. The image compensation module includes a bright area compensation module or a dark area compensation module. The bright area compensation module and the dark area compensation module may be used separately or synergistically with each other. In general, the bright area compensation module modifies a transparency layer used for a damping display layer to prevent unwanted light from passing through the display. This modification is achieved by means of a calculation which takes into account the pupil diameter of the user. The dark area compensation module prevents the background from appearing too dark by adding sufficient light to the foreground to compensate for the muted background. This implies the use of a front-facing camera to estimate the light coming in from the background. The resulting camera-based foreground image, referred to herein as a replica graphic, is presented along with the emission graphic in an emission display layer to compensate for the unwanted dark area. However, the part of the background is viewed directly by the user.

In order to compensate for an undesired bright area, the bright area compensation module supplements (for example, extends) the attenuation graph, in particular such that it extends beyond an edge of the emission graph. In order to determine how far the boundary of the attenuation graph needs to be extended beyond the edge of the emission graph, a value indicating an attribute of the user's eye is used. For example, the radius of the pupil of the eye is used to set a filter that adjusts the size of the expanded attenuation graph. Due to the natural focusing properties of the eye along with the relative positioning of the emission and attenuation display layers, the expanded attenuation graph appears to be out of focus. The extended attenuation graph still attenuates more light from the environment of the border of the emission graph compared to a non-expanded attenuation graph. As a result, an unintended bright area (for example, an inner bright court) is compensated for by considerably reducing the visibility of the unwanted bright area. However, the appearance of an undesirable dark area (for example, an outer dark courtyard) may become more prominent.

In order to compensate for an undesired dark area, regardless of its cause, the dark area compensation module may cover the dark area with pixels that at least replicate the brightness level of the real world scene in the vicinity of the dark area. If the dark area corresponds to an outer dark courtyard, then there is a zone between the edge of the emission graph and a boundary of the attenuation graph, which may be an extended attenuation graph. An outwardly oriented camera detects light originating from the real world scene, at least in the environment corresponding to that zone. The dark area compensation module converts the detected light into pixels that replicate the appearance of the real world scene in the unintended dark area to produce a replica graphic. The dark area compensation module adds the replica pixels of the replica graphic to those of the emission graphics and presents both in the emission display layer to mask the unwanted dark area. Therefore, an unintended dark area (for example, an outer dark courtyard) is compensated for by significantly lightening the unwanted dark area based on the appearance of the real world scene in the background.

In the following discussion, after some exemplary terminology has been described, an exemplary environment in which the techniques described herein may be used will be described. Exemplary embodiments, systems, and techniques are then described, followed by a section in which procedures and processes of an exemplary embodiment are explained. The procedures and processes can be performed in the example environment and in systems as well as in other environments. However, the performance of the exemplary procedures is not limited to the exemplary environment, nor are the example environment and systems limited to the performance of the example procedures.

The drawing, in particular 1 to 14 , contain a mixture of schematic diagrams as well as simulated pictures. The schematic diagrams and the simulated images are intended to depict aspects of embodiments such that the relevant terminology terms and principles are easily understood. The simulated images should also be visual examples of the results of applying the principles described. A comparison of schematic diagrams with simulated images can be done by the schematic diagrams of 1 along with the simulated pictures of 2 to be viewed as. 2 . 4A . 5A and 14 contain both a schematic diagram and a simulated image. 15 concerns exemplary procedures. 16A to 20 show additional simulated images to facilitate understanding of the accuracy and error analysis section, followed by a description of the exemplary procedures. A final section describes 21 an exemplary computing device that can implement the described techniques.

Examples of the terminology

Exemplary descriptions or explanations of certain terms used herein follow below. Each term is applicable to one or more, but not necessarily all embodiments presented herein. Some terms are also illustrated using one or more examples.

A "real world scene" refers to a view of a current environment that a user is looking at in real time or near real time. Real world scenes are in contrast to computer generated images. Examples of a real world scene may include a celebration with friends, a football game, a mountain view, a children's dance performance, and the like. A real world scene can be viewed directly or indirectly.

A "direct view" of a real world scene refers to a situation or environment in which the eyes of a user receive light originating from a real world object (such as reflection or broadcast). However, the light rays may pass through one or more transparent, semi-transparent, or partially opaque layers, such as an LCD layer. A direct view is in contrast to an indirect or camera-mediated view in which a person looks at a real world scene using a display screen to which images are fed from a camera sensor.

An "image artifact" in an still or moving image refers to an unintentional or undesirable aspect that is not a natural part of the image. The image artifact is a perceptible artificial aspect of an image that may be more introduced due to a lens, a recording mechanism, a reproduction technique, as a result of added synthetic graphics, human perception, a combination thereof, and the like. Examples of image artifacts may include an inadvertent light area, an unintended dark area, a lens flare effect, a blur, a pixelation, or a combination thereof.

An "unintended bright area" refers to a zone of an image that is brighter than intended or desired. A visually visible bright area, which is actually not present, has no purpose or does not indicate exactly a brighter zone of a real world scene, may be unintentional. For example, if one zone of an image is brighter than another zone of the image, but corresponding real-world zones do not differ in brightness, the brighter zone of the image may be an unintended bright region. A zone that is so bright that the zone appears bleached or difficult to see may clearly be an inadvertent bright area.

An "unintended dark area" refers to a zone of an image that is darker than intended or desired. A visually visible dark area that is actually non-existent, has no purpose or does not accurately indicate a darker zone of a real world scene may be unintentional. For example, if one zone of an image is darker than another zone of the image, but the corresponding real world zones are not different in darkness, the darker zone of the image may be an undesirable dark area. A zone that is so dark that the zone may be black, dark gray, shaded, or hard to see may also be an unintended dark area.

An "emissions display layer" refers to a layer of augmented reality equipment that can present an emissions graphic to a user's eye. Of two or more layers, an emission display layer may be positioned closest to the user's eye. In a direct view augmented reality facility, the emission display layer is transparent to incident light originating from a real world scene. An emission display layer is configured to emit light or a color from multiple pixel locations to form patterns, shapes, or objects in the form of one or more emission graphics recognizable to a user. The emitted light can be colored.

An "attenuation display layer" refers to a layer of augmented reality device that can present a attenuation graph that keeps light from a real world object away from an emission graph. Of two or more layers, an attenuation display layer is positioned between an emission display layer and a real world scene. In a direct view augmented reality device, the attenuation display layer may be transparent to incident light originating from a real world scene, or substantially opaque at different times or in different zones. An attenuation display layer is configured to make a plurality of pixel locations substantially opaque to the view of a pixel To support emission graphics by keeping light out, otherwise the emission graphics would fail due to the formation of an inadvertent bright area. If an attenuation display layer forms an opaque zone that is not behind active pixels of an emission graph, the opaque zone may result in an unintended dark area.

A "synthetic graphic image or" synthetic graphic "refers to a diagram, image, symbol, and the like that is visually perceptible and computer generated. A synthetic graphic image can be computer-generated by being made up of vectors, bitmaps, textures, three-dimensional grids, and the like. Additionally or alternatively, a synthetic graphical image may be processed, modified or stored by a computer and used in conjunction with other images. Examples of synthetic graphical images may include an emission graphic, a mute graphic, a replica graphic, or a combination thereof.

An "emissions graph" refers to a chart, image, symbol and the like that is generated on the computer and presented on or through an emissions display layer. Examples of an emissions graph may include a notification or alert, navigation directions or icons, textual comments or instructions, an animated cartoon character, information from a social media feed, or combinations thereof. An emission graph can be upright or in motion. An emission graph can also be called a foreground graph here because the emission graph in the foreground is displayed in comparison to the real world scene that serves as the background. However, an emissions graph may also be rendered such that the graph blends or integrates with the background, for example, by appearing to be partially behind an object of the real world scene. An emission graphic may be bleached by a backlight, or rendered unintentionally too bright if the emission graphics are not supported by a snubber image that keeps the backlight off.

An "attenuation graph" refers to a pattern or shape presented at or through a damping display layer. An attenuation graph may be associated with an emission graph that is intended to support an attenuation graph. For example, an attenuation graph may be formed to have a substantially similar size and shape to an associated emission graph. The attenuation graph is substantially opaque, such that the light originating from a real world object and kept away by the attenuation graph can prevent an unintended bright area from forming over at least a portion of an associated emission graph.

A "replica graphic" refers to a graphic presented on or through an emissions display layer. A replica artwork is created to reproduce the look of a real world scene. For example, a replica graphic can simulate the brightness level or the coloration in a particular image area. A replica graphic can be used to hide an unintended dark area. Data that assigns values to the pixels of a replica graph can be obtained from a camera that targets the real world scene.

A "camera" refers to a device or component that includes a sensor that can detect light. The light sensor may convert detected light into data representing the appearance of the source or origin of the light, such as a real world scene. The "look of a real world scene" refers to how a real world scene looks visually. The appearance of an area of a real world scene may generally relate to a brightness level or coloring details for specific pixels or groups of pixels, both of which may be used to conceal an unintended dark area.

An "attribute of the eye" refers to a particular aspect of the eye of a user of an augmented reality device. The attribute may capture a change quantity or an aspect as a function of time, may include a snapshot of a changing quantity, may include a constant property, and the like. Examples may include a value indicating the size of an eye, the direction of a fixed eye or the movement of the eye, or the depth of focus of the eye.

A "value indicating the size of an eye" denotes a determinable measure of a particular aspect of the eye. A value indicating, for example, the size of the eye may relate to the radius of the pupil of an eye, or equally its diameter. A "direction of a fixed glance or a movement" refers to the circumstance where, under what angle and on which object an eye looks, what it tracks or where it looks. A "focal depth" refers to a distance or plane below which one or both eyes clearly reflect observable details, such as a person on the other side of a Table or a mountain on the other side of a valley.

The use of the word "or" as used herein may, unless explicitly stated otherwise, signify an "inclusive or" or a term that permits inclusion or application of one or more objects represented by the word "or (For example, the term "A or B" may be interpreted to mean only "A", "B", or both "A" and "B", or is relevant thereto).

Exemplary environment

1 shows an environment 100 for exemplary embodiments that may be operated to employ the techniques described herein relating to image compensation for a masking, direct view augmented reality system. As illustrated, the exemplary environment includes 100 a real world scene 102 , an emission chart 104 and a composite picture view 106 , The environment 100 further includes an augmented reality facility 108 , a user 110 and a computing device 112 , The computing device 112 includes a processing system 114 and a memory 116 , The memory 116 includes an image compensation module 118 , Exemplary implementations for the computing device 112 will be discussed later in this section as well 21 described.

In exemplary embodiments, the augmented reality facility is 108 implemented as a walkthrough and direct view based augmented reality facility. The user 110 primarily looks directly at the real world scene 102 by receiving light originating from real-world objects (for example, due to aura or reflection). The augmented reality facility 108 can at least one emission graph 104 to the real world scene 102 Add. A user's eyes and brain combine the real world scene 102 and the emission graph 104 to a seemingly composite picture view 106 ,

The emission graph 104 is a computer-generated synthetic content that provides information to the user 110 provides. The emission graph 104 may include text, at least one character, at least one symbol, at least one design, at least one object, or a combination thereof. Although the emission graph 104 computer generated and in relation to the current real world scene 102 is synthetic, can the emission graph 104 Include photos, videos or parts thereof of other real world objects in addition to content created on the computer. In other words, in addition to computer-generated images derived from vectors, three-dimensional wireframes with texture surfaces, fonts, and the like, a displayed emission graph 104 also include images taken using a camera sensor. In parts of the present description, for purposes of simplicity, the real world scene will be described 102 called background or wallpaper while the emission graph 104 is called foreground or foreground picture. However, the described principles are also applicable in situations where the emission graph 104 is inserted at a certain depth of the scene under consideration and therefore appears to be in the background.

The computing device 112 can be implemented as any suitable type of computing device. The computing device 112 can with the view section of the augmented reality facility 108 integrated or separated from it. In some embodiments, where there is separate integration, the augmented reality facility may 108 and the computing device 112 communicatively wired or wirelessly coupled. Furthermore, the computing device 112 be implemented as a locally available device, such as a mobile phone or game console, or as a remote device available, such as an Internet server or vehicle computer. Implementations of the computing device 112 As such, they may range from relatively large resource devices with significant memory and processor resources (e.g., cell phones or remote servers) to relatively low resource devices with limited storage or processing resources (e.g., extended-reality augmented glasses).

The computing device 112 is illustrated as including a variety of hardware components, examples of which are a processing system 114 , an example of a computer-readable storage medium used as storage 116 and more. Other hardware components are also as described below 21 are included. The processing system 114 provides functionality for performing operations by executing instructions stored in the memory 116 are stored, ready. Exemplary implementations include an image compensation module 118 in the computing device 112 or operated there such that it is part of a larger program application (not shown) that can provide augmented reality capabilities to a user. The image compensation module 118 provides functionality to implement of schemes and techniques for image compensation for a masking, direct view augmented reality system as described herein. The image compensation module may be implemented 118 at least as part of a software package running on and specifically configuring one or more processors, the processors physically hosting the processing system 114 can realize; as a hardware device, as an application specific integrated circuit (ASIC) or as a computing device 112 can be realized, or using a combination of software, hardware, firmware or a fixed logic circuit, as any combination thereof, and the like. Examples of hardware and associated logic are set forth below in particular 21 described.

2 generally indicates 200 an example of simulated pictures of a real world scene 102 and a masking synthetic graphic leading to a compensated picture view 106 are composed. The pictures of 1 are shown as schematic diagrams. In contrast, the pictures are from 2 shown as simulated pictures. The user 110 Look through the augmented reality setup 108 on the real world scene 102 containing an added emission graph 104 so as to have the composite image view 106 to produce. The composite picture view 106 from 2 represents a composite image view that does not appear to have an inadvertent light area or an unintended dark area. The exemplary real world scene 102 For the simulated pictures is a beach scene with sand, sea and an interesting rock formation. The exemplary emission graph 104 for the simulated images is the status symbol of a ringer (ringer), how it is produced, for example, for the iPhone ^® from Apple Inc. of Cupertino, California.

Systems and techniques

The present section describes exemplary details for systems and techniques for image compensation for a masking, direct view augmented reality system, according to one or more embodiments.

3 shows an environment 300 with an exemplary augmented reality facility 108 , which is a see-through emission display layer 306 and a damping display layer 308 includes. The environment 300 further includes an eye 302 a user (not shown) and a real world object 304 a real world scene 102 (please refer 1 ). The augmented reality facility 108 is between the eye 302 and the real world object 304 positioned. The emission display layer 306 is between the eye 302 and the damping display layer 308 positioned. The damping display layer 308 is between the emission display layer 306 and the real world object 304 positioned.

Several beams of light 314 are also in 3 shown. The multiple beams of light 314 originate (due to propagation or reflection) from the real world object 304 , The rays of light 314 pass through transparent portions of the emission display layer 306 and the damping display layer 308 and should be from the eye 302 be detected. In zones that do not currently display a graph, the emission display layer may 306 and the damping display layer 308 transparent to the light rays 314 be made. Such transparent portions are detected by the intensity map and the damping layers shown in FIG 7 ff, respectively. For the eye 302 a graphic appears on the emission display layer 306 is present, as if it were at a certain distance from the user as a virtual image 312 an emission advertisement.

In exemplary embodiments, the attenuator display layer 308 a lower resolution than the emission display layer 306 on. Although the indicator for the emission display layer 306 from the eye 302 at a focally distant focal distance as it is in the emission display layer 306 is designed, becomes the attenuation display layer 308 out of focus due to the positioning of the attenuation indicating layer 308 behind the emission display layer 306 seen in conjunction with the pupil diameter of the user's eye. Will the damping display layer 308 used to select the alpha channel or the location for the emission graph 104 directly display, the resulting image formed on the retina of the watching user through the attenuation display layer 308 Being out of focus, even muted when the view of the world is in focus. This can be simulated by holding a finger close to the eye while focusing on a distant object. The edge of the masking finger appears out of focus, while the faraway object that is in focus is crisp. The state out of focus is explained below 5A and 5B The situation can be improved with techniques based on 9 to 12B are described.

The camera 310 is shown as having the augmented reality facility 108 is integrated. The camera 310 can be from the augmented reality facility 108 disconnected and coupled to it wirelessly or wired. The camera 310 includes one or more sensors, such as a CCD (Charge-Coupled Device CDD) device or a CMOS image sensor (Complementary Metal-Oxide Semiconductor CMOS). During operation, the camera detects 310 light rays 314 that of the real world object 304 come from, and produces data that the light rays 314 represent. As will be described below, the data may be used to indicate an unintended dark area as viewed from the perspective of the eye 302 due to an attenuation graph attached to the attenuation display layer 308 is presented, compensate and the corresponding light rays 314 from the eye 302 keep.

4A and 4B generally indicate 400A and 400B a first visual composition problem. The first visual composition problem arises in an augmented reality device that has an emission display layer 306 but works without a damping display layer. As a result, the emission chart becomes 104 bleached by the backlight from the real world scene. 4A shows the emission display layer 306 in a cut-away side view, showing the emission graph 104 is displayed therein. 4A also shows two exemplary composite image views 402 that show an unwanted image artifact. In particular, the emission graph 104 shown so that it is excessively bright and therefore difficult to see because the backlight is the emission graph 104 fades. The exemplary schematic composite image view 402 is on the left while the exemplary simulated composite image view 402 is on the right side. 4B provides an enlarged view of the simulated composite image view 402 as in 4B shown is the faded appearance of the emission graph 104 to see clearly. Advantageously, the inclusion of the attenuation graph in an attenuation display layer can account for much of the blurred appearance of the emission graph 104 , as in 5A and 5B shown, mitigate.

5A and 5B show 500A and 500B a second visual composition problem. The second visual composition problem arises in an augmented reality facility that involves a direct implementation of an emissions display layer 306 and a damping display layer 308 is working. Such a situation produces an unintended bright area 502 and an unintended dark area 506 , These unwanted image artifacts are in two exemplary composite image views 508 shown. 5A also shows the emission display layer 306 and the damping display layer 308 in a cutaway side view. The emission display layer 306 shows an emission graph 104 while the damping display layer 308 a damping graph 504 displays. In this direct implementation, the attenuation graph is 504 the same size as the emission chart 104 displayed.

The attenuation graph 504 keeps much of the backlight from the emission graph 104 remote. This prevents the backlight from the emission graph 104 completely fade, what the in 4A and 4B is the givenness shown as the first visual composition problem. The disadvantage is with the direct inclusion of the attenuation graph 504 a second problem arose. In particular, areas with undesirable appearance, such as with an unintentional bright area 502 and an unintended dark area 506 , as in 5A and 5B shown is produced. An exemplary schematic composite image view 508 is shown on the left, while an exemplary simulated composite image view 508 shown on the right. 5B shows an enlarged view of the simulated composite image view 508 , As in 5B is shown, the unintentional light and dark areas are clearly visible. As shown, the unintentional bright area develops 502 and the unintentional dark area 506 in the vicinity of the emission graphics 104 , The unintentional bright area 502 manifests as inner Hellhof, while the unintentional dark area 506 manifested as an outer dark court.

These two farms (for the areas 502 and 506 Examples are) follow at least a portion of a border of the emission graph 104 , The courtyards are image artifacts resulting from the out-of-focus attenuation display layer 308 result. The inner edge of the emission graph 104 may appear too bright because the background is not completely out of the out of focus attenuation display layer 308 is darkened. In addition, beyond the limit of emission graphics 104 In addition, the background may appear too dark because the fuzzy attenuation layer obscures a portion of the background that is not to be attenuated. Although certain implementations described herein have an inadvertent bright area 502 or an unintended dark area 506 in the vicinity of an emission graph 104 or manifested as court, the principles described herein are applicable to unintentional light and dark areas with other causes, locations or shapes.

6 shows an example of an image compensation module 118 , The image compensation module 118 includes a synthetic graphics module 602 , a bright area compensation module 604 , a dark area compensation module 606 and a display module 608 , One or more of the mentioned Modules may be implemented, at least in part, in hardware, such as a processing system and a computer-readable storage medium as described 21 will be described. In exemplary embodiments, the synthetic graphics module generates 602 an emission graph 104 and an initial version of a damping graph 504 based on the emission graph 104 , The bright area compensation module 604 compensates for an unintentional bright area 502 by adding the initial version of the attenuation graph 504 to produce an expanded attenuation graph 804 (please refer 8th ). The supplement is based on an attribute of the user's eye, such as the radius of the pupil of the eye. 7 to 9 relate to implementations of the Synthetic Graphics Module 602 and the bright area compensation module 604 ,

The dark area compensation module 606 compensates for an unintended dark area 506 by adding pixels to the emission display layer 306 which replicates the light passing through the attenuation indicating layer 308 is kept away, but is not covered by pixels at the emission display layer 306 are displayed. For example, replicating pixels that reproduce the light may be displayed by an enhanced version of the attenuation graph 504 or the expanded attenuation graph 804 is kept away. The light reproduction can be the imitation of the brightness level or the appearance of the real world scene 102 imply. The out-of-focus property of the attenuation graph 504 or 804 can be accomplished by defocusing or blurring an attenuation layer, that of the attenuation graph 504 or 804 matches before creating a replica layer that defines a zone in which replica pixels are created. The display module 608 can the emission display layer 306 and the damping display layer 308 so control or regulate that the emission graph 104 , the advanced attenuation graph 804 and pixels of a replica graphic are displayed appropriately. 10 to 13 pertain to implementations of the dark area compensation module 606 and the display module 608 ,

7 generally indicates 700 exemplary aspects for displaying an emissions graph 104 and a damping graph. An example of an intensity map 702 for the emission layer is in the upper half of 4 shown while an example of a sharp damping position 704 is shown in the lower half. The intensity map or map 702 is black or dark in places where there are no pixels in the emission display layer 306 (please refer 3 ) should be displayed or colored. The Synthetic Graphics Module 602 therefore makes the intensity map 702 except the location of a zone of the graphic 104 that according to the intended emission graph 104 is colored, black. The sharp damping position 704 corresponds to the intensity map or map 702 the emission layer and is the emission graph 104 based on the size and shape of the emission graph 104 assigned.

The sharp damping position 704 , which serves as a transparent map, is white in areas where light from the background is to be transmitted, is black in opaque areas where no light is to be transmitted from the background, and is shaded gray in partially transparent areas. The white sections of the sharp damping layer 704 specify sections that should remain transparent to incident light rays. The design of the attenuation characteristics of the foreground graphics may be provided along with the design of the emission intensity characteristics of the foreground graphics, or the design of the attenuation characteristics may be automatically established based on the emission intensity characteristics of the foreground graphics. Both may be specified using field arrays of image pixel values or via a vector-based description, such as one that can be created with a drawing program. The exact design of the transparency and emission layers is provided by the synthetic graphics module 602 which is used to create the graph, where any combination of foreground intensity and background transparency may be implemented. In the upper half of 7 is a border 706 the emission graph 104 together with the intensity map 702 the emission layer shown. The edge 706 is also in 8th shown.

8th schematically shows an exemplary approach 800 to supplement a damping graph 504 , The solution 800 includes the bright area compensation module 604 that a supplement 810 performs. In the upper left corner is the emission graph 104 such that they represent the edge 706 having. The Synthetic Graphics Module 602 uses the size of the emission graph 104 to set up the size of the attenuation graph 504 in the upper middle of 8th is shown. The sizes of the emission graphics 104 and the attenuation graph 504 For example, they may be substantially the same. In other words, to an extent, bearing in mind the relative resolution capabilities of the emission display layer 306 and the damping display layer 308 is still acceptable, can the attenuation graph 504 be designed so that they are initially the same size as the emission graphics 104 having. in the For the purposes of the present, the term "attenuation graph" or "attenuation graph 504 "At least an initial version of a damping graph, for example, before the supplement 810 , several versions of a damping graph, for example, before and after the addition 810 , denote the general concept of a damping graph and the like.

The Synthetic Graphics Module 602 can the attenuation graph 504 or 804 at the damping display layer 308 such that it is substantially opaque to light rays, forming or using them. The attenuation graph 504 or 804 For example, it can be formed using a rich dark color such as black or gray. The schematic diagrams of the attenuation graphics 504 and 804 are shown with a crosshatch pattern to provide visual differentiation over the schematic diagram of the emission graph 104 to enable. A sharp border 802 the attenuation graph 504 and the expanded attenuation graph 804 is in the upper middle section of 8th or the upper right corner of 8th displayed.

In certain exemplary embodiments, the light area compensation module supplements 604 the attenuation graph 504 such that the expanded attenuation graph 804 keeps more background light color away from it through the display layers around the edges 706 the emission graph 104 to get around. The complement 810 the attenuation graph is in the upper right corner of 8th shown. The bright area compensation module 604 adds (for example, enlarges or expands) the attenuation graph 504 by the limit 802 over the edge 706 the emission graph 104 is extended out. The extended attenuation graph 804 includes compared to the emission graph 104 a zone 806 that are not along with the smaller emission graph 104 extends. The zone 806 is in 8th not necessarily drawn to scale. Although the attenuation graph 504 Moreover, it is shown in several different directions around the border 802 the attenuation graph 504 may also be supplemented by one or more pages or other sections of a damping graph 504 to a varying extent, including entirely a missing supplement.

The two different versions of a damping layer for the damping display layer 308 are in the lower half of 8th shown. The dark portion of the damping layer indicates which portion of the damping display layer 308 should be activated to activate the incident light rays. The white portions of the damping layer indicate sections that are to remain transparent to incident light rays. The Synthetic Graphics Module 602 creates the attenuation graph 504 such that these are a sharp border 802 having. Although a displayed attenuation graph 504 or 804 for the user's eye out of focus and therefore out of focus, the bright area compensation module operates 604 at the sharp border 802 , As shown by the schematic diagrams of the damping layers in the lower half of 8th is shown, the supplement may 810 Alternatively, this can be done by limiting 802 according to the attenuation graph 504 in the sharp damping position 704 is moved so that it extends the attenuation graph. The bright area compensation module 604 can the supplement 810 therefore at the sharp damping position 704 make to the extended damping position 808 to produce the expanded attenuation graphic 804 which is a sharp limit 802 maintains. The extended attenuation graph 804 can also be referred to here as a supplemental attenuation graph, and it can be the extended attenuation 808 Here also be referred to as a supplemental damping position.

Exemplary operations of the bright area compensation module 604 have been qualitatively described above. Exemplary operations of the bright area compensation module 604 are explained below in quantitative terms by means of a computationally more rigorous description. To facilitate understanding by simplifying the mathematical description, the emission display layer is referred to as the foreground, while the real world scene is referred to as the background.

The foreground graph has a spatially varying non-pre-multiplied intensity I _F , that of the emission graph 104 and a foreground transparency value T _F corresponding to the sharp attenuation position 704 corresponds, on. Given the background real world intensity I _B , the composite color I _C can be calculated using equation (1): I _C = I _F × (1-T _F ) + T _F × I _B (1)

In a case where a user's eye is focused on the depth of the foreground graphics display and the background is at the same depth, the effective transparency of the background attenuation element is calculated as T _B resulting from a convolution of the attenuation pixels t _B relative to the size of the Pupil using equation (2) gives: T _B (x, y)> = Σ x + k / i = x - kΣ y + k / j = y - kt _B (i, j) · W (i - x, j - x) (2)

Here, W (i, j) is a kernel representing the convolution through the eye of the attenuation graph, where the core radius k is expressed in pixel coordinates of the attenuation display layer.

For calculating W (X, Y) in the coordinate system of the attenuation display layer, the real world attenuation function corresponding to the physical aperture of the eye E (X, Y) is projected onto the attenuation display layer and converted into pixel coordinates. A simplifying assumption is to regard the pupil as round and the viewing direction as perpendicular to the display, in which case W (i, j) takes the form of a circular top hat function with the value 1 within the radius of the circle and Otherwise assumes 0. Alternatively, in a non-perpendicular viewing direction, the pupil may be approximated using an ellipse. In a vertical viewing direction and a round pupil, the radius R _{att of} the top hat function in physical units depends on the pupil radius of the eye R _eye . In particular, the radius R _{att of} the pupil radius of the eye R _eye depends on how the scaling is scaled by the ratio of the distances (i) between the eye and the virtual image and (ii) between the attenuation display and the virtual image. The radius R _{att of} the top hat function is calculated from equation (3): R _att = R _eye * (Z _display - Z _att ) / (Z _display + Z _eye ). (3)

The variables of equation (3) will be described below. The size of the pupil of an eye (eg, the pupil radius R _{eye of} the eye) may be determined using an eye-facing camera, such as a camera that is also used for eye-tracking. Looking at the other three variables shows 9 generally included 900 Examples of multiple variables, namely P and Z. These variables are associated with an augmented reality facility that is a measurable property of a pupil 902 used to determine a factor to supplement an attenuation graph. The variable Z _att represents a distance between the emission _{display layer} 306 and the damping display layer 308 The variable Z _eye represents a distance between the pupil 902 of the eye and the emission display layer 306 The variable Z _display represents a distance between the attenuation _{display layer} 308 and the virtual picture 312 the emission display. The variables P _e and P _a represent centers or centers of the emission display layer 306 or the attenuation display layer 308 when, from the real world object 304 originating light rays 314 pass through the layers.

Assuming that the goal of T _{B is} a strong approximation to T _F , the problem can be formulated as boundary-conditional optimization to maximize the array of t _B values with an L1 or L2 norm, the constraints of the above Equation (2) apply. This constrained optimization can be represented by Equation (4): T _F (x, y)> = Σ x + k / i = x - kΣ y + k / j = y - kt _B (i, j) · W (i - x, j - x) (4)

However, the constraints may not be compatible with low latency real-time computations, particularly for a low-resource device such as a portable or other mobile computing device. Instead, the variable t _B may be formulated as a minimum value for all pixels for a min-kernel M centered on x, y with a domain greater than or equal to the convolution kernel W (i, j).

A conservative set of t _B values can then be calculated using Equation (5): t _B (x, y) = M · T _F = ∧ _{i, j for all M! = 0 centered on x, y} T _F (i, j) (5)

It is evident that this satisfies equation (4) since t _B (i, j) is less than or equal to T _F (x, y) with respect to the kernel. The convolution kernel W (i, j) is normalized and does not accept negative weights. This relationship holds as long as the originally fuzzy core fits entirely into M's min-core domain. A min core can be chosen for optimal efficiency. However, a disadvantage of choosing a kernel that is too large is that such a too large kernel leads to a larger dark area around the foreground image for which compensation is subsequently to be calculated.

For circular camera apertures (for example, human pupils and certain cameras), a circular min filter may be used which is slightly larger than the convolution diameter of the convolution kernel W. Alternatively, a quadratic min filter may be used whose half width is greater than or equal to the half width k of the W filter. A one-dimensional (1-D) -min filter can be computed comparatively efficiently given a spatially invariant filter width that uses only three comparisons per pixel regardless of the filter width. A two-dimensional (2-D) box-min filter is separable, allowing one pass for horizontal filtering and a second pass for vertical filtering. A circular filter can be calculated as a combination of a set of 1-D filters, the cost of which is proportional to the filter height. For a fast circular filter, the min filter can be calculated by having a series of filter convolutions on each of the source scan lines for the core is combined. The total costs are linear in the radius of the filter core.

10 generally indicates 1000 a problem that may arise from enlarging an attenuation graph such that an unintended dark area 506 remains visible, becomes visible or becomes more visible. For the in 10 shown image view is a compensation for an undesirable bright area (for example, an inner Hellhof) already from the bright area compensation module 604 (please refer 6 ) has been implemented. However, it still remains an unintended dark area 506 (for example, an outer dark courtyard). The unintentional dark area 506 In practice, after the compensation of the unwanted bright area, it may be wider or thicker, since the attenuation diagram corresponds to the supplement 810 (please refer 8th ) is extended.

It becomes an exemplary simulated composite image view 1004 shown. The unintentional dark area 506 will be in the vicinity of the border of the emission graph 104 shown. The blur that results from the attenuation graph being out of focus when viewed by a user's eye is due to the blurred border 1002 for the unintended dark area 506 specified. The unintentional dark area 506 can through the dark area compensation module 606 , as in 11 and 12 is shown to be compensated.

11 generally indicates 1100 exemplary aspects of adjusting a damping position by a dark area compensation module 606 to an out of focus attenuation graph 504 or 804 (please refer 5A . 5B and 8th ) to meet. An example of an extended attenuation graph 808 that continues to be a sharp border 802 is in the upper half of 11 shown while an example of a fuzzy cushioning 1104 is shown in the lower half. The damping layer uses white portions to indicate which portions of a damping display layer 308 (please refer 3 ) should remain transparent. After the supplement 810 (please refer 8th ), corresponds to the extended damping position 808 an expanded attenuation graph 804 ,

The extended attenuation graph 804 seems to be out of focus for the user's eye. Accordingly defocused to determine the size and appearance of the unintended dark area 506 who has a blurred border 1002 has (as well as in 10 shown), the dark area compensation module 606 the extended damping position 808 , The dark area compensation module 606 defocuses the sharp border 802 the attenuation graph as they do in the extended attenuation position 808 is shown to a blurred damping position 1104 that the blurred border 1002 has to produce. The blurred cushioning 1104 gets along with the sharp cushioning position 704 (please refer 4 ) used to generate an intensity replica mask of 12 to create.

12A and 12B generally indicate 1200A and 1200B exemplary aspects of creating a replica artwork 1204 by a dark area compensation module 606 using an intensity replica mask. An example of an intensity replica mask 1202 for the emission layer is in the upper half of 12A as an example of a colored intensity replica image 1208 is shown in the lower half. The intensity replica mask 1202 is there white where pixels through the emission display layer 306 be displayed or colored, as by the colored intensity replica image 1208 is shown. Other portions of the emission display layer that are not to be colored for the replica artwork are in the intensity replica mask 1208 and the colored intensity replica image 1208 shown in black. The intensity replica mask 1202 includes a blurred zone 1206 that is a combination of the zone 806 (please refer 8th ) and the blurred border 1002 (please refer 10 and 11 ) corresponds.

The dark area compensation module 606 produces the intensity replica mask 1202 by reversing the fuzzy damping position 1104 (please refer 11 ) and adding (e.g., by qualitatively involving or quantitatively multiplying) the sharp attenuation layer 704 (please refer 7 ). The dark area compensation module 606 also colors the fuzzy zone 1206 and forms a replica artwork 1204 as in the colored intensity replica image 1208 is shown. To the coloring of the blurred zone 1206 to clearly represent the reproduction of the brightness level or the appearance of the background scene is in 12B an enlarged view of the replica graphic 1204 shown. A camera gets data that represents light coming from the real world scene. From these data representing the real world light, the dark area compensation module identifies 606 the dates, the real world scene behind the blurred zone 1206 correspond. Replica pixels are colored based on the light data to reproduce the real world scene blocked by out-of-focus, blurred-appearing and expanded attenuation graphics. Therefore, the dark area compensation module colors 606 the blurred zone 1206 to the colored intensity replica image 1208 that's the replica artwork 1204 produced, produce. The replica artwork 1204 reproduces the appearance of the real world scene around the border around the Emission graphics around based on the data from the camera. The appearance of the general light or brightness level of the real world scene or the actual specific coloration of the real world scene may vary from the replica artwork 1204 be reproduced.

Exemplary operations of the dark area compensation module 606 have been qualitatively described above. Exemplary operations of the dark area compensation module 606 are presented quantitatively below by means of a computationally more rigorous description. To facilitate understanding by simplifying the mathematical description, the emission display layer is referred to as the foreground, while the real world scene is referred to as the background.

Using the circular core-min transparency algorithm described above for the bright area compensation module, the expanded attenuation layer becomes 808 (please refer 8th and 11 ) is supplemented by a circular min-filter. The effect of lens blurring on the supplemented, extended damping position 808 with the sharp border 802 is as a blurred damping position 1104 shown. The blurred cushioning 1104 is used to modulate camera-related incident light from the real world background. To the advanced damping position 808 to producing blurred damping attitude 1104 to defocus, the aforementioned equation (2) is used to calculate the estimated blurred topper position T _B (x, y) using a lens blur kernel for a near circular aperture.

The colored intensity replica picture 1208 , denoted by I _R , can be used to compensate for an outer dark courtyard. The colored intensity replica picture 1208 is calculated according to equation (6): I _R = T _F (1-T _B ) I _B (6)

I _B is in this case the intensity of the background as it is obtained from a camera, T _B is the blur damping layer 1104 , and T _F is a foreground implementation of the sharp attenuation layer 704 , The emission display image I _E , which is a combination of the emission graph 104 and the replica artwork 1204 is calculated using equation (7): I _E = (1-T _F ) I _F + I _R (7)

Here, I _{F is} the intensity of the emission graph 104 ,

13 shows a schematic diagram 1300 an augmented reality device that provides a composite image view (not shown) with a supplemented or expanded attenuation graph 804 and a replica artwork 1204 supported. As shown, the display module causes 608 in that the emission display layer 306 the emission graph 104 and the replica artwork 1204 indicates or indicates this. The display module 608 also causes the attenuation indicating layer 308 the extended attenuation graph 804 indicates or indicates this. The eye of the user sets the emission graph 104 and the replica artwork 1204 over the real world scene 102 together, what about the light-stopping feature of the expanded attenuation graph 804 being affected. This leads to a compensated picture view, which in 14 is shown.

14 generally indicates 1400 an exemplary result of image compensation in a masking graphic by an image compensation module 118 , On the left are the emission display layer 306 and the damping display layer 308 displayed in a cutaway side view. The emission display layer 306 shows an emission graph 104 and a replica graphic 1204 while the damping display layer 308 an extended attenuation graph 804 displays. An example of a schematic composite image view 106 is shown in the middle while an example of a simulated composite image view 106 shown on the right. The emission graph 104 is in every composite image view 106 shown. In the compensated image views, as shown, no unwanted bright area or unwanted dark area is visually apparent.

Having described exemplary details of the image compensation systems, techniques, and schemes for a hidden direct-view augmented reality system, some exemplary procedures for illustrating additional aspects of the techniques will now be considered.

Exemplary procedures

This section describes by reference 15 exemplary procedures associated with image compensation for a masking, direct view augmented reality system in one or more embodiments. Aspects of the procedures may be implemented in hardware, firmware or software or even a combination thereof. The procedures are shown as sets of blocks that specify operations that may be performed by one or more devices, wherein the performance of the operations is not necessarily limited to those sequences, which are shown by the respective blocks or the description, since the operations can also be performed in other orders or in a fully or partially overlapping manner. In at least some embodiments, the procedures may be performed by a suitably designed device, such as an exemplary augmented reality device 108 in cooperation with a computing device 112 pointing to an image compensation module 118 takes recourse (see, for example 1 . 6 and 14 ).

15 is a flowchart 1500 that's six blocks 1502 to 1512 and illustrates an exemplary image compensation procedure for a hidden direct-view augmented reality system according to one or more exemplary embodiments. At block 1502 is an emission graph related. A computing device 112 can, for example, an emissions graph 104 which has information to a user 110 should be presented. An augmented reality facility 108 may be a text message-type alarm notification from a mobile phone, a navigation instruction from a remote cloud server, or a battery indication from a near-end operating system function provided to the user 110 should be presented, received by the emission graph 104 to a composite view, which is also a direct view of a real world scene 102 contains, adds or adds. The emission graph 104 can be provided with a damping layer. If this is not the case, then the damping position can be determined from the emission graph 104 For example, be generated by that intensity values of an intensity map or map 702 can be formed by thresholding or otherwise mapped to transparency values for the damping position, such as the sharp damping position 704 to acquire.

At block 1504 a damping layer is supplemented to produce an extended damping position. A computing device 112 For example, a damping position, such as the sharp damping position 704 Supplement to an advanced damping position 808 to produce. The extended damping position 808 becomes the emission graph 104 assigned. The sharp damping position 704 corresponds to the supplement 810 a damping graph 504 whose size matches that of the emission graph 104 is comparable. The attenuation graph 504 or 804 Furthermore, it is for the removal of light for the corresponding emission graphics 104 responsible. To carry out the supplement 180 can the at least one sharp border 802 the sharp damping position 704 over a border 706 the emission graph 104 beyond on the basis of an attribute of the eye 302 of the user 110 expand. The attribute can be the size of a physical property of the eye 302 be such as a measure of the radius or diameter of a pupil 902 of the eye 302 , which is used to set a filter that is in the sharp damping position 704 Additionally 810 is used to the extended damping position 808 that of an extended attenuation graph 804 corresponds to produce.

At block 1506 the expanded damping position is defocused to produce a blurred damping position. A computing device 112 For example, the extended damping position 808 defocus to get a blurred cushioning 1104 to produce. An eye-lens blur due to an out-of-focus extended attenuation graph 804 to cope with, is the advanced damping position 808 based on an attribute of the eye 302 , such as a physical property thereof, defocused. A convolution kernel, set by a physical property of the eye, can defocus the sharp boundary 802 the extended damping position 808 be applied to a blurred border 1002 the blurred damping position 1104 to produce.

At block 1508 A replica graphic is generated based on the blurred cushioning and emission graphics, the replica graphic serving to reproduce the appearance of the real world scene. A computing device 112 can be a replica artwork 1204 for example, based on the blurred attenuation 1104 and the emission graphics 104 produce. The replica artwork 1204 serves to reproduce the appearance of the real world scene 102 in an area that has the emission graph 104 limited. For this purpose, data for light coming from the real world scene 102 is for modulating (for example, coloring) the pixels of the replica graphic 1204 using an intensity replica mask 1202 used to create a colored intensity replica image 1208 to produce. The intensity replica mask 1202 can by reversing the fuzzy damping position 1104 and modifying the reverse blurred cushioning layer based on a sharp cushioning layer 704 that is the size and shape of the emission graph 104 corresponds to be created.

At block 1510 For example, an attenuation display layer presents an expanded attenuation graph using the expanded attenuation map. A computing device 112 may cause a damping display layer 308 an extended attenuation graph 804 using the extended damping position 808 after the supplement 810 presents. To accomplish the presentation, the attenuation display layer can 308 Pixels of the attenuation display layer 308 activate ( For example, make the pixels opaque), the expanded attenuation graph 804 as they go through the black zone of the extended damping position 808 is defined to correspond to light for the emission graph 104 and the replica artwork 1204 keep.

At block 1512 An emissions display layer presents a user with the emissions graph and the replica graph. A computing device 112 For example, it may cause an emission display layer 306 the user 110 the emission graph 104 and the replica artwork 1204 presents. To accomplish the presentation, the emission display layer can 306 Enable pixels in zones (for example, make the pixels lighter or colored) through an intensity map 702 for the emission graphics 104 and through a colored intensity replica image 1208 for the replica artwork 1204 are shown.

Having described exemplary procedures in accordance with one or more embodiments, an exemplary system and apparatus will now be considered that may be used to implement the various schemes and techniques described herein.

Accuracy and error analysis

In an augmented reality device, the foreground emission light may be presented as being focused at a fixed distance. If so, certain implementations of the present description provide an approximation in which compensating graphics presented by an emissions display layer are presented at a depth and with a resolution determined by the emissions display layer. However, the direct view background is limited by the optical performance of the observer's eye and the glass through which the light passes (for example, including diffraction effects from the attenuation display layer).

To reduce perceptible image artifacts, a replica image (eg, a foreground compensation image) may be as small as possible (e.g., such that a corresponding attenuation map can still eliminate an inner halo when lens blurring occurs) and as low as possible (e.g. perceptible brightness nuance is avoided for an outer dark court). An additional factor that may affect the described implementations is the potential for optical misalignment between attenuation display layer graphics and emissions display layer graphics, which can lead to eye position uncertainty. There may also be misalignment between estimated views of the background scene and light actually coming from the background. The misalignment can occur due to errors in the estimation of pupil size. Careful implementation of the techniques described herein may enable these problems and factors to be properly addressed.

An augmented reality facility may include a front-facing camera that allows for calculation and selection of obscuration by an emission graph in relation to real world objects. This may imply making a depth estimate of the real world and relating an intensity image. Such a device and the functionality associated therewith may be sufficient to develop an estimate of the world's view as viewed by one or both eyes based on an estimated camera model. From this information, depths can be estimated, and images acquired by a camera can be warped to serve as background images for described implementations.

In an optical system, multiple forms of misalignment may develop. For example, the estimated background I _B may shift relative to the true background as viewed by the viewer's eyes. 16A and 16B show 1600A and 1600B two examples of simulated composite image views resulting from misalignment. The composite picture view 1600A Fig. 11 shows a situation where the estimated background is shifted by three pixels in the vertical and horizontal directions relative to the true background. A double image for some of the finer details appears in the courtyard of the foreground graphic and is most visible along the horizon line. However, the results are quite acceptable because the replica graphic is cross-dissolved with the true background as the distance from the foreground emission graphics boundary increases (for example, because the intensity replica mask ramp structure allows the replica graphic to be true) Background is faded to weaken the appearance of the artifacts). This type of misalignment is likely due to a reconstruction error of the camera / geometry detection system or due to head movement of the user. The inherent latency of the camera and graphics system causes the estimated background to be shifted from the ground truth conditions. However, advanced capturing and low latency rendering architectures can at least sometimes use rotary or other movements to minimize these effects.

The composite picture view 1600B Figure 12 shows a second form of misalignment that may occur when the fuzzy attenuation map for the attenuation display layer is offset relative to its estimated position for the emission display layer. This can occur due to a parallax between the emission and attenuation display layers in combination with an eye position estimation error (for example, if the pupil position is estimated inaccurate relative to the display). This results in a dimming of the yard on one side and a lightening of the yard on the other side, as shown by the misalignment of three pixels. The degree of brightness shift depends on the slope of the blurred attenuation graph. For obscuring objects that are more out of focus, the shift is less susceptible to misalignment. Similar artifacts occur in other estimation errors in the masking geometry, including rotation and scaling of the estimated masking geometry as compared to ground truth conditions.

A visible artifact may occur as a result of errors in the estimation of the pupil diameter of the eye (for example, if defocusing is present to blur the obscuring object). 17A and 17B show 1700A and 1700B two examples of simulated composite image views that result from overemphasizing the effects of image blur. In particular, an actual lens blur for an attenuation graph is 10% less than the estimated blur that is for the simulated composite image view 1700A is used, and the actual lens blur for an attenuation graph is 20% less than the estimated blur for the simulated composite image view 1700B is used. A court around the foreground graphic is barely visible with a 10% error, but is more visible with a 20% error.

18A and 18B show 1800A and 1800B two examples of simulated composite image views that result from under-emphasizing the effects of image blur. In particular, the actual lens blur for an attenuation graph is 3% greater than the estimated blur that is for the simulated composite image view 1800A is used, and it is the actual lens blur for a damping graph 10% greater than the estimated blur, which for the simulated composite image view 1800B is used. A court appears around the foreground graphics, barely visible on a 3% error, but much more visible with a 10% error. This implies that an example of a suitable guideline for pupil size estimation accuracy is less than or equal to 3% to properly account for potential pupil size estimation problems.

An additional form of error can occur as a result of the user's eyes being set to different depths. 19A and 19B show 1900A and 1900B Two examples of simulated composite image views resulting from a user focusing in a different depth or plane than the background. In particular, the pictures of 19A and 19B the effect of the adjustment when the user focuses on the foreground image and the background image is blurred. A focused yard is visible around the foreground image in both composite image views. In the simulated composite image view 1900A There is a comparatively weak yard around the foreground graphics, with the replica yard having a crisper level of detail than the rest of the background. In the simulated composite image view 1900B a comparatively more visible courtyard is visible around the foreground graphics. The composite picture view 1900B shows the background that is out of focus when using a lens blur that is twice that of the composite image view 1900A is. As a result, the focus on the farm is stronger than the comparatively blurred background of the composite image view 1900B visible, noticeable.

The effect of this problem can be improved by defocusing the replica graphic (e.g., the estimated background) using depth information and eye depth estimation based on pupil size and eye convergence. For example, defocusing (using, for example, a hardware solution) may be based on adjusting the capturing camera to have a similar depth of field and depth of focus as the human eye, or may be based on a synthesized effect using a depth map Image filtering (for example, rather using a software solution) can be realized. Alternatively, the scene may be captured using a brightfield camera. An appropriate depth of focus can be determined by tracking the user's eyes and using the convergence of the eyes to estimate the depth setting. However, these types of artifacts may be acceptable without setting the depth of focus, if a comparatively low degree blur, such as the one in the simulated composite image view 1900A is shown.

Another problem can occur when the user adjusts to the background, which can cause the foreground to be blurry. 20 shows an example of a simulated composite image view 2000 that results from a user focusing on the depth of the background. As a result, a blurred replica-based halo surrounds the foreground image because the emission display layer is out of focus for the eye. To avoid this effect, a light field foreground optical layer can be implemented. Alternatively, a multi-level emission display layer may be used in which the focal depth of a graph in the foreground may be adjusted by adjusting at which depth level each emission pixel is presented. In a multi-level emission display layer, image compensation pixels of a replica graphic are placed on the depth plane to best match the background depth for each corresponding pixel.

Exemplary system and exemplary device

21 generally indicates 2100 an exemplary system that includes an exemplary computing device 2102 which represents one or more computing systems or one or more computing devices that can implement the various techniques described herein. This is through the inclusion of an image compensation module 118 illustrated, which can perform the operations described above. A computing device 2102 can, for example, as a computing device 112 (please refer 1 ) implemented with an augmented reality facility 180 for a user 110 coupled or integrated. The computing device 2102 may cause one or more enhanced reality device display layers to be presented to the user 110 present synthetic graphic images. A computing device can be implemented 2102 in general, for example, as an end user device (for example, as a smartphone) of a user 110 , as a proprietary device (for example, as a server-side device or as a data center hardware), as an on-chip system, or system-on-a-chip (SOC) system (such as the augmented reality device 108 integrated) or as any other suitable computing device or as such a computing system.

An exemplary implementation works as in 1 shown is the image compensation module 118 at a location (for example, within an enclosure of the augmented reality facility 108 ). The image compensation module 118 can also operate in the cloud (eg, on a network-side computing device) if the transmission latency is sufficiently small, such an exemplary implementation also being described in US Pat 21 is shown. Alternatively, at least a portion of the image compensation module 118 working on both a client-side computing device and a server-side computing device. In such an implementation, the operations performed by the image compensation module 118 described above are distributed across a client-server architecture.

The exemplary computing device 2102 includes, as shown, at least one processing system 2104 , one or more computer-readable media 2106 and one or more I / O interfaces 2108 that can be communicatively coupled with each other. Although not explicitly shown, the computing device may 2102 further include a system bus or other data and command transfer system that couples the various components together. A system may be any bus structure or even a combination of different bus structures, such as a memory bus or a memory controller, a peripheral bus, a universal serial bus, or a processor or local bus employing a wide variety of bus architectures. A variety of other examples are also included, such as control and data lines.

The processing system 2104 provides functionality to perform one or more operations using hardware. Accordingly, the processing system 2104 represented as having one or more hardware elements 2110 which may be more configured as processors, functional blocks, and the like. This may include an implementation in hardware as an application specific integrated circuit (ASIC), as a general purpose processor, or as another logical device formed using, for example, one or more semiconductors. The hardware elements 2110 are not limited by the materials of which they are formed or the processing mechanisms employed therein. For example, the processors may be formed of a semiconductor / semiconductors and / or transistors (eg, electronic integrated circuits (ICs)). In this context, processor executable instructions may also be electronically executable instructions.

The computer-readable storage medium 2106 is shown as having a memory / shelf 2112 having. The storage / storage 2112 provides a storage / storage capacity associated with one or more computer-readable media. The storage / storage component can be included 2112 volatile media (such as a random access memory (RAM)) and / or non-volatile media (such as a read-only memory (ROM), a flash memory, optical disks, or magnetic disks). This may include the storage / storage component 2112 fixed media (for example, RAM, ROM, or a fixed hard disk drive) as well as removable media (for example, a flash memory card, a removable hard disk drive, or an optical disk). The computer-readable media 2106 can be configured in a variety of other ways as described below.

The input / output interface 2108 provides / the input / output interfaces 2108 Provide functionality that allows a user to send commands or information to the computing device 2102 and that also allows information to be presented to the user or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (eg, a mouse or a touch pad), a microphone, a scanner, touch functionality (eg, capacitive, resistive, or with other sensors configured to detect a physical touch), a camera (eg, one that can use visible or invisible wavelengths, such as infrared frequencies, to detect motion as gestures that do not imply touch), an accelerometer, or a combination thereof. Examples of output devices include a display device (eg, an LCD or LED screen, a display layer, a monitor, or a projector), a speaker, a printer, a network card, a haptic vibrator, or a combination thereof. The computing device 2102 can be configured in a variety of ways, as described below, to support local or remote user interaction.

Various techniques have been described herein in the general context of software and hardware elements or program modules. Generally, such modules may include routines, programs, objects, elements, components, data structures, combinations thereof, and the like that perform particular tasks or implement particular abstract data types. The terms "module," "functionality," and "component" generally refer to software, firmware, hardware, or a combination thereof as used herein. The features of the techniques described herein may be platform independent, meaning that the described techniques may be implemented on a variety of off-the-shelf computing platforms having a plurality of processors.

An embodiment of the described modules and techniques may be stored on or transmitted through any form of computer-readable medium. The computer-readable medium 2106 can involve a variety of media, by the computing device 2102 can be accessed. By way of example and not by way of limitation, the computer-readable media may include "computer-readable storage media" and "computer-readable signal media."

"Computer-readable storage media" as used herein refers to media or devices that allow for permanent and / or non-temporary storage of information, as opposed to mere signal transmission, to carrier waves or signals as such. Therefore, computer readable storage media does not include signals or signal carrying media as such. The computer readable storage media include hardware such as volatile and nonvolatile, removable and non-removable media or memory devices for implementation in a process or technology suitable for storage of information such as computer readable instructions, data structures, program modules, logic elements / circuits or others Dates. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory, or other storage technology, such as solid state, CD-ROM, DVD or other optical storage, hard drives, magnetic cassettes, magnetic tapes, magnetic disk drives, or others magnetic storage devices, or other storage device, physical media, or a manufactured product, and a combination thereof with the ability to store the desired information so that a computer can access it.

"Computer-readable signal media" as used herein refers to a signal carrying medium implemented to provide instructions to the computing device hardware 2102 for example, to transmit over a network. Computer readable signal media may typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves, data signals, or another Embody transport mechanism. Computer readable signal media may also include any information distribution media. The term "modulated data signal" refers to a signal in which one or more of its characteristics are set or changed such that information in the signal is encoded. By way of example, and not by way of limitation, computer-readable signal media may include wired media, such as a wired network or a direct-wired connection, as well as wireless media, such as acoustic, radio frequency-based, infrared, and other wireless media.

As described above, hardware elements 2110 and computer-readable media 2106 Modules, programmable device logic, fixed device logic, a combination thereof, and the like, which are implemented in hardware, which in some implementations is used to implement at least some aspects of the techniques described herein, such as when executing one or more of these techniques several instructions or calculation steps. The hardware may include integrated circuit (IC) or on-chip system components, an application specific integrated circuit (ASIC), field programmable gate array (FPGA), complex programmable logic device (CPLD), and other implementations in silicon or other hardware. In this regard, hardware may act as a processing device that performs program tasks defined by instructions and / or logic embodied by the hardware, as well as hardware used to store instructions for execution, such as those described above computer-readable storage media.

Combinations of the foregoing may also be used to implement various of the techniques described herein. Accordingly, software, hardware, or executable modules may be implemented as one or more instructions or logic that may be in any form of computer-readable storage media or by one or more hardware elements 2110 are embodied. The computing device 2102 may be configured to implement certain instructions or functions according to the software and / or hardware modules. Accordingly, the software implementation of a module is that of the computing device 2102 executable, even at least partially realized by hardware, for example, using a computer-readable storage medium or hardware elements 2110 of the processing system 2104 , The instructions or functions are by one or more manufactured products (for example, one or more computing devices 2102 and / or processing systems 2104 ) executable / operable to perform techniques, modules, and examples described herein.

The techniques described herein may be achieved by various configurations of the computing device 2102 and are not intended to limit the specific aspects of the exemplary devices described herein. This functionality can also be implemented in whole or in part using a distributed system, for example by means of a "cloud". 2114 over a platform 2116 as described below.

The cloud 2114 can be a platform 2116 for resources 2118 include or represent. The platform 2116 abstracts underlying hardware functionality (eg one or more servers or at least one data center) and software resources of the cloud 2114 , The resources 2118 may include applications or data or data that may be used while computer processing is performed at least in part on servers that are hosted by the computing device 2102 removed or distributed around them. The resources 2118 may also include services provided over the Internet or through a subscriber network, such as a cell-based or Wi-Fi network.

The platform 2116 can provide resources and functions for connecting the computing device 2102 abstract with other computing devices or services. The platform 2116 may also abstract a scaling of resources to provide a corresponding scaling level for existing resource requirements 2118 that over the platform 2116 are implemented. Accordingly, in one embodiment with interlinked devices, the implementation of a functionality described herein may be particularly useful in the system of FIG 21 or at least in the cloud 2114 together with the computing device 2102 be distributed. The functionality may be partially on the computing device, for example 2102 as well as the platform 2116 that the functionality of the cloud 2114 abstracted, implemented.

concluding remark

While the invention has been described in language specific to structural features and / or methodological acts, it is to be understood that the invention as defined in the appended claims is not necessarily limited to the particular features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementation of the claimed invention.

Claims (20)

At least one computing device operable in a digital media environment to present a synthetic graphical image in conjunction with a direct view of a real world scene such that the visibility of an image artifact is reduced, the device comprising: a synthetic graphics module configured to obtain an emission graphic and a damping graphic; a bright area compensation module configured to supplement the attenuation graph to produce an expanded attenuation graph based on an attribute of a user's eye; and a display module designed to: Instructing an emissions display layer to present the emission graphic to the user's eye; and Instruct an attenuation display layer to present the enhanced attenuation graph to keep light of the real world scene away from the emission graph and prevent the development of an inadvertent bright area on the emission graph. The device of claim 1, wherein the attribute of the user's eye comprises a value indicating a size of the user's eye. The device of claim 2, wherein the value indicative of the size of the user's eye is based on a radius of a pupil of the eye. The apparatus of any one of the preceding claims, wherein the bright area compensation module is further adapted to produce the expanded attenuation graph by applying a convolution kernel set based on the attribute of the user's eye. The device of claim 4, wherein the convolution kernel is further adjusted based on a distance measured from a position of the emission display layer. The apparatus of any one of the preceding claims, wherein the bright area compensation module is further adapted to produce the expanded attenuation graph by applying a minimum filter set based on the attribute of the user's eye. Apparatus as claimed in any one of the preceding claims, wherein the unintentional bright region comprises an inner halo which limits at least a portion of the emission graphic. Device according to one of the preceding claims, wherein: the bright area compensation module is further configured to produce the expanded attenuation graph by supplementing an attenuation pad; and the display module is further configured to instruct the attenuation display layer to present the enhanced attenuation graph based on the supplemental attenuation pad. The device of any one of the preceding claims, further comprising: a dark area compensation module configured to generate a replica graphic to reproduce an appearance of the real world scene in an area where the light is kept away from the direct view of the real world scene by the extended attenuation chart; wherein the display module is further configured to instruct the emission display layer to present to the user's eye the replica artwork in the region. A digital media environment operable system for presenting a synthetic graphical image in conjunction with a direct view of a real world scene such that the visibility of an image artifact is reduced, the system comprising: a camera designed to detect light representing the real world scene; an emission display layer configured to present an emission graphic; an attenuation display layer configured to present an attenuation graph; a dark area compensation module designed to: Determining an unintended dark area in which the attenuation graph keeps light of the real world scene from an user's eye; and Generating a replica graphic for the unintended dark area based on the light representing the real world scene; and a display module configured to cause the emission display layer to present the replica graphic to reproduce an appearance of the real world scene in the unintended dark area. The system of claim 10, wherein: the unintended dark area comprises a fuzzy unintended dark area; and the dark area compensation module is further adapted to: Defocusing a boundary of an attenuation pad to produce a fuzzy cushion; and generating the replica graphic based on the fuzzy cushion. The system of claim 11, wherein the dark area compensation module is further configured to generate the replica graph based on a sharp attenuation map associated with the emission graph. The system of any one of claims 10 to 12, wherein the dark area compensation module is further adapted to produce a color value for each of a plurality of pixels of the replica graphic to reproduce the appearance of the real world scene using digital values derived from the light representing the real world scene as detected by the camera. A system according to any one of claims 10 to 13, wherein: the unintended dark area comprises an outer dark courtyard that delimits at least a portion of the emission graphic; and the display module is further configured to cause the emission display layer to present the replica graphic adjacent the emission graphic to compensate for the outer dark courtyard. The system of any one of claims 10 to 14, further comprising: a light area compensation module adapted to supplement the attenuation graph to expand a boundary of the attenuation graph past an edge of the emission graph to create the unintended dark area between the edge of the emission graph and the expanded boundary of the attenuation graph, wherein: the dark area compensation module is further configured to generate the replica graphic from the edge of the emission graph to the extended boundary of the attenuation graph based on the light representing the real world scene; and the display module is further configured to cause the emission display layer to present the emission graphic. A method implemented by a computing device in a digital media environment for presenting a synthetic graphical image in conjunction with a direct view of a real world scene such that the visibility of an image artifact is reduced, the method comprising: obtaining an emission graph by the computing device; adding a damping layer or mat to produce an expanded damping layer or mat by the computing device; defocusing the extended damping pad or mat by the computing device to produce a fuzzy damping pad or mat; generating, by the computing device, a replica graphic based on the fuzzy matte and the emission graphics, wherein the replica graphic is for reproducing an appearance of the real world scene; causing the computing device to cause the attenuation display layer to present an enhanced attenuation graph using the expanded attenuation map; and causing the computing device to cause an emission display layer to present the emission graphic and the replica graphic to a user. The method of claim 16, wherein: completing comprises producing the expanded attenuation attitude based on an attribute of an eye of the user; and defocusing involves producing the blurred cushioning based on the attribute of the user's eye. A method according to claim 16 or 17, wherein: the emission graphic comprises a notification indication provided by a mobile device; and causing the attenuation display layer to present the augmented attenuation graph, comprising causing the attenuation display layer to present the augmented attenuation graph to keep light of the real world scene from the emission graph and the replica graph, the attenuation display layer being positioned between the emission display layer and the real world scene. The method of any one of claims 16 to 18, further comprising: by the computing device, obtaining data corresponding to light representing the real world scene as detected by a camera, wherein generating comprises generating the replica graphic for reproducing the appearance of the real world scene using the data for determining values for pixels of the replica graphic. The method of any one of claims 16 to 19, wherein: the supplementing comprises supplementing the damping layer to compensate for a potential inner halo; the defocusing comprises defocusing the extended attenuation layer to accommodate lens blurring by an eye of the user; and generating comprises generating the replica graphic for reproducing the appearance of the real world scene to compensate for a potential outer dark courtyard.

Images