KR20140004115A - Morphological anti-aliasing (mlaa) of a re-projection of a two-dimensional image - Google Patents

Abstract

According to the present invention, it is possible to implement morphological anti-aliasing (MLAA) of reprojection of two-dimensional images in a manner that produces better results while using fewer processor resources. Determine one or more discontinuities between each neighboring pixel of the two-dimensional image. Identify one or more predefined patterns formed by one or more discontinuities. The blending amount is calculated for each pixel neighboring the identified predefined pattern. Reprojection is applied to the blend amount and the two-dimensional image for each pixel to generate the reprojected blend amounts. Then, neighboring pixels of the reprojection are mixed according to the reprojected blending amounts.

Description

Morphological Antialiasing of Reprojection of Two-Dimensional Image (MORPHOLOGICAL ANTI-ALIASING (MLAA) OF A RE-PROJECTION OF A TWO-DIMENSIONAL IMAGE)

Cross-reference to related application

This application is filed and filed on January 7, 2011, entitled "DYNAMIC ADJUSTMENT OF PREDETERMINED THREE-DIMENSIONAL RE-PROJECTION SETTINGS BASED ON SCENE CONTENT," ).

This application is filed and filed on January 7, 2011, entitled "SCALING PIXEL DEPTH VALUES OF USER-CONTROLLED VIRTUAL OBJECT IN THREE-DIMENSIONAL SCENE," co-transferred and co-pending application number 12 / 986,827 (Representative No. SCEA10053US00).

This application is filed on January 7, 2011, entitled "MULTI-SAMPLE RESOLVING OF RE-PROJECTION OF TWO-DIMENSIONAL IMAGE," and is hereby assigned and co-pended to Application No. 12 / 986,872 (Agent No. SCEA10055US00). It is about.

The ability to perceive two-dimensional images in three dimensions by many different techniques has been fairly well known over the years. By providing aspects of depth to the two-dimensional image, it potentially creates a sense of presence for any illustrated scene. Introducing this three-dimensional visual representation greatly enhances the observer experience, especially in the area of video games.

There are a number of techniques for three-dimensional rendering of a given image. Most recently, a technique has been proposed for projecting two-dimensional image (s) in three-dimensional space, known as depth-image-based rendering (DIBR). In contrast to this proposal, which often relied on the basic concept of "stereo" video, namely, two separate video streams, one for the left eye and the other for the right eye, often this idea Is based on a more flexible monoscopic video (ie, a single video stream) and associated transmission of associated per-pixel depth information. From this data representation, one or more "virtual" views of the 3D scene can be generated in real time at the receiver side by so-called DIBR techniques. This new approach to 3D image rendering offers several advantages over previous approaches.

There are generally two ways of presenting two separate images to the observer to produce an illusion of depth. In a system commonly used to project 3D images onto a screen, there are two separate synchronous projectors for the left eye image and the right eye image. Images for both eyes are simultaneously projected onto the screen, but there is orthogonal polarization, for example vertical polarization for the left eye image and horizontal polarization for the right eye image. The observer wears special polarized 3D viewing glasses with lenses that are properly polarized for the left and right eyes (eg, vertically polarized for the left eye and horizontally polarized for the right eye). Because of the polarization of the lens and image, the observer perceives only the left eye image with the left eye and only the right eye image with the right eye. The degree of depth illusion is partly a function of the offset between the two pictures on the screen.

In a 3D video system, the left eye picture and the right eye picture are displayed by the video display screen but are not displayed correctly at the same time. Instead, the left eye image and the right eye image are displayed in an alternating manner. The observer wears active shutter glasses that block the left eye when the right eye image is displayed and vice versa.

The experience of 3D video may depend somewhat on the characteristics of human vision. For example, the human eye has a distinct number of light receptors, but a human cannot distinguish any pixels even in the surrounding field of view. More surprisingly, the number of color-sensitive cones in the human retina can vary dramatically between individuals and up to 40. Nevertheless, people seem to perceive colors in the same way, and we essentially use the brain. In addition, the human watch has the ability (super vision) to determine the alignment of objects at a fraction of the cone's width. This explains why spatial aliasing artifacts (ie, visual irregularities) are more pronounced than color errors.

Because of this fact, graphics hardware vendors make great efforts to compensate for aliasing artifacts by adjusting color accuracy to spatial continuity. As with the integration nature of digital cameras, a number of techniques are supported in hardware based on mixing weighted color samples.

Of course, any aliasing artifacts will eventually disappear as the display resolution and sampling rate increase. It can also be handled at low resolutions by computing and averaging multiple samples per pixel. However, for most image rendering algorithms (e.g. ray tracing, rasterization based rendering) this may not be very feasible, resulting in a dramatic increase in overall performance by computing color samples that are discarded through averaging. Can be reduced.

Morphological anti-aliasing (MLAA) is a technique based on the recognition of certain patterns in an image. Once these patterns are found, the colors can be blended around these patterns to achieve an inductive estimation of the most likely predetermined image. MLAA has a set of unique characteristics that distinguish it from other anti-aliasing algorithms. MLAA is completely independent from the rendering pipeline. This represents a single post-processing kernel that can be implemented on the GPU even if the main algorithm runs on the CPU. MLAA, even in non-optimized implementations, is quite fast and handles about 20M pixels per second on a single 3GHz core.

MLAA is an anti-aliasing technique established for two-dimensional images. However, by performing the same MLAA technique used for two-dimensional images for three-dimensional reprojection, additional problems arise.

Embodiments of the invention have been made within this context.

Aliasing refers to the generation of visual distortion artifacts (ie, jagged edges between neighboring pixels) caused by representing a high resolution image at low resolution. Morphological anti-aliasing is the process of mixing such jagged edges that occur between pixel discontinuities of a given image to produce a smoother image for the viewer to see. In general, morphological anti-aliasing for a two-dimensional image occurs in three stages, i.e. 1) when looking for discontinuities between pixels in a given image, 2) pre-defined patterns generated by such discontinuities. And 3) mixing colors in the neighborhood of such predefined patterns to produce a smoother image.

However, morphological anti-aliasing for reprojection of two-dimensional images further creates a set of problems during anti-aliasing of two-dimensional images. For a two-dimensional picture to be reprojected in three dimensions, such a video picture must be presented to the observer so that the placement of two separate video pictures (one for one eye per eye) creates a depth illusion. This added depth dimension makes it difficult to apply the techniques used for two-dimensional morphological antialiasing.

A first possible solution for implementing morphological antialiasing in three dimensions involves performing morphological antialiasing on each two dimensional image after the two dimensional image is reprojected to each viewpoint. Therefore, determination and mixing of pixel discontinuity are performed twice for each two-dimensional pixel so as to be reprojected in three dimensions when reprojection is performed on the left eye and the right eye. In theory, this solution can provide an accurate procedure for morphological antialiasing of three-dimensional reprojection, but in practice, this is very expensive to implement. In addition, performing morphological anti-aliasing more than once for each two-dimensional image to be reprojected in three dimensions significantly degrades the performance of some 3D video applications (eg, video games or video game system processors). . In addition, different edges can be detected between different images, such that one eye can see the mixed edge while the other eye still sees the aliased edge. This is a form of retinal contention, which reduces the reliability of the overall stereoscopic effect and adds some inconvenience to perceived 3D images.

A second solution for implementing morphological antialiasing in three dimensions involves performing morphological antialiasing once for each two-dimensional image before three-dimensional reprojection. This provides a cost effective solution, but this also adds haloing artifacts to the three-dimensional reprojection. Depending on the mixing before reprojection, the foreground pixels may be mixed with the background pixels. During reprojection, the foreground pixel is shifted by a different amount than the background pixel. Sometimes, this leaves a hole between these pixels. The hollowing artifacts refer to the color or shape information of the elements in the scene as seen from the other side of the hole. It is difficult to assign depth values to mixed two-dimensional image pixels during morphological antialiasing, since no single value can represent both sides of the hole. A single value can separate the hole into two holes, reducing the hole size but does not actually solve the problem. Since there is not enough way to determine the pixel depth values of the mixed two-dimensional images, this hauling artifact is a problem that occurs cyclically if morphological antialiasing is done before three-dimensional reprojection.

Embodiments of the present invention utilize other approaches. Instead of blending before reprojection, the amount of blending is calculated before reprojection but no blending is applied to the pixels before reprojection. Instead, reprojection is applied to the calculated blend amount to produce a reprojected blend amount. After reprojection, these reprojected blend amounts are applied to the relevant pixels of the reprojection image. In particular, the discontinuity between each neighboring pixel of the two-dimensional image can be determined. Predefined patterns formed by one or more discontinuities can be identified, and the blending amount can be calculated for each pixel neighboring the predefined patterns. Three-dimensional reprojection can then be applied to the two-dimensional image and the corresponding blend amounts. As a result, the reprojected blend amounts can be applied to neighboring pixels of the three-dimensional reprojection. This technique is advantageous in that it is computationally less concentrated than any of the solutions described above and produces better results than a robust solution.

According to the present invention, there is a problem of reducing the reliability of the overall stereoscopic effect and causing some inconvenience to the perceived 3D image, and a problem of circulating occurring when the morphological anti-aliasing is performed before the halo artifact is three-dimensional reprojection. Can be prevented.

1 is a flowchart illustrating a morphological anti-aliasing (MLAA) method of three-dimensional reprojection of a two-dimensional image in accordance with one embodiment of the present invention;
2 is a block diagram illustrating a morphological antialiasing apparatus for three-dimensional reprojection of a two-dimensional image according to an embodiment of the present invention;
3 is a block diagram illustrating an example of a cell processor implementation of a morphological anti-aliasing device of three-dimensional reprojection of a two-dimensional image in accordance with one embodiment of the present invention;
4 illustrates an example of a non-transitory computer readable storage medium having instructions for implementing morphological antialiasing of three-dimensional reprojection of a two-dimensional image in accordance with an embodiment of the present invention.

1 is a flowchart illustrating a morphological anti-aliasing (MLAA) method of reprojection of a two-dimensional image. The method 100 of the present invention reduces the cost associated with running MLAA more than once for a given image, while the incidence rate of hauling artifacts / aliasing associated with pre-re-projection MLAA Also reduces. This method 100 separates the MLAA process into two separate stages, one to be run before reprojection and the other to be run after reprojection has occurred.

This method 100 can be applied to reprojection of two-dimensional left and right eye images for three-dimensional display. In the left and right eyes, MLAA and reprojection may occur sequentially or simultaneously, depending on the nature of the processing system used. Images 101 may be generated by a computer graphics program based on data for a virtual environment. The virtual environment, eg, a video game environment, can be generated from data indicative of physical properties (eg, size, location, texture, lighting, etc.) for objects within the virtual environment. A view of the environment can be created from a defined viewpoint, sometimes referred to as a virtual camera location. If this viewpoint is known, the field of view can be calculated. The field of view can be considered a three-dimensional shape and can be, for example, a cone, a pyramid, or a pyramid. The graphics software can determine if the virtual objects are in a three-dimensional shape. If within the three-dimensional shape, these objects are in view and may be part of the image from the corresponding viewpoint. Virtual objects outside the field of view may be excluded from the image. Note that two separate viewpoints and corresponding clocks slightly offset from each other may be used to generate left and right eye images for the 3D view of the virtual world.

First, a given two-dimensional image 101 undergoes a series of processing steps before being presented to the viewer as smooth three-dimensional reprojection. First, pixel discontinuity is determined by traversing the two-dimensional image 101 ( 103 ). It is possible to traverse a given image vertically and then horizontally, or vice versa. Pixel discontinuity occurs between neighboring pixels (eg, both vertical and horizontal neighbors) if the neighboring pixels have inconsistent characteristics. For example, such properties may include, but are not limited to, a color or shape profile associated with a given pixel. It is important to note that discontinuities can be defined to include any number of different characteristics between the pixels.

Once pixel discontinuities for a given two-dimensional image have been determined, one can identify the predefined patterns formed by these pixel discontinuities ( 105 ). For example, a discontinuity between the two pixels can be identified by a line separating the two pixels, but the example is not limited thereto. Each pixel can be characterized by up to four different discontinuities (ie, top, bottom, left, right). Adjacent and orthogonal pixel discontinuities may form predefined patterns that characterize changes between the pixels of the two-dimensional image. For example, such predefined patterns may include L shape, U shape, Z shape, but are not limited to this example. An L-shaped pattern is formed when a chain of one or more pixel discontinuities intersects an orthogonal chain of one or more pixel discontinuities. A U-shaped pattern is formed when a chain of one or more pixel discontinuities intersects two orthogonal chains of one or more pixel discontinuities on opposite sides, each orthogonal chain having the same length and pointing in the same direction. A Z-shaped pattern is formed when a chain of one or more pixel discontinuities intersects two orthogonal chains of one or more pixel discontinuities on opposite sides, with each orthogonal chain facing in the opposite direction. These predefined patterns provide a blueprint for calculating the pixel mixing amount.

After predefined patterns formed by the pixel discontinuities have been identified for a given two-dimensional image, as indicated by reference numeral 107 , the blending amounts for the pixels neighboring the identified patterns can be calculated. Depending on the arrangement of neighboring pixels surrounding the predefined pattern, a different blending amount can be selected for each individual pixel. The blend amount refers to the weighted color / shape profile for a given pixel used for smooth transitions between discrete pixels. For example, a pixel that is closer to a predefined pattern may experience a greater amount of mixing than a pixel that is farther away, but is not limited to this example. Various equations based on the identified predefined patterns can be used to determine the blending amount for each pixel of the image. This step ends the first stage of morphological antialiasing of the three-dimensional projection of the two-dimensional image.

Following determination of the blending amount, re-projection is performed, as indicated by reference numeral 109 , before pixel mixing. Reprojection involves mapping one or more two-dimensional images to three-dimensional space. Different views of the same image are presented to each eye, creating a depth illusion. In general, each pixel of a two-dimensional image is assigned a color profile and depth value during reprojection. Then, these values are adjusted for each view (ie, left eye view, right eye view) to generate three-dimensional reprojection. In the method of the present invention, additional information corresponding to the blending amount is assigned to each pixel, and the information is converted into appropriate values for each view (i.e., reprojection of the blending amount for each pixel). Thus, the reprojection is applied to each blend of pixels and the one or more two-dimensional images, thereby producing the reprojected blend amounts and one or more reprojected images for each pixel of the images.

After reprojection of the images and blending amounts, as indicated by reference numeral 111 , the reprojected blending amounts may be applied to the reprojection (each two-dimensional view of the three-dimensional reprojection) to produce output images. Neighboring pixels of the reprojected image (s) are blended according to the reprojected blend amounts, thereby producing one or more output images. Note that if one or more two-dimensional images 101 include left and right eye views of the scene, the output pictures correspond to the reprojected left and right eye images of the scene. The output pictures can be presented on the display as indicated by reference numeral 113 . Note that in the case of three-dimensional stereoscopic left and right eye images, the images can be displayed sequentially or simultaneously depending on the nature of the display. For example, left and right eye images may be displayed sequentially in the case of a 3D television display used with active shutter glasses. Alternatively, the left eye and right eye images may be displayed simultaneously in the case of the dual projection type used with manual 3D view glasses having left and right eye lenses of different colors or different polarizations.

Although the blend amounts were determined prior to reprojection, the image edges are expected to not change significantly during reprojection from two to three dimensions. As such, it is possible to produce a smooth image without being affected by any of the results associated with the two possible solutions described above.

2 shows a block diagram of a computer device that may be used to implement a morphological anti-aliasing (MLAA) method of three-dimensional reprojection of two-dimensional images. The device 200 may generally include a processor module 201 and a memory 205 . The processor module 201 may include one or more processor cores. One example of a processing system using multiple processor modules is a cell processor, examples of which include, for example, Cell. Broadband Engine Architecture , Version 1.0, August 8, 2005, which is incorporated herein by reference. A copy of this document is available online at the URL: http://www.ief.u-psud.fr/~lacas/Computer Architecture / CBE_Architecture_v10.pdf.

The memory 205 may be in the form of an integrated circuit, and may be, for example, RAM, DRAM, ROM, or the like. Memory 205 may also be main memory accessible by all of the processor modules. In some embodiments, processor module 201 may have local memories associated with each core. The program 203 may be stored in the main memory 205 in the form of processor readable instructions that may be executed in the processor modules. Program 203 may be configured to perform morphological anti-aliasing (MLAA) of three-dimensional projection of a two-dimensional image. The program 203 can be written in any suitable processor readable language, for example C, C ++, Java, assembly, Matlab, Fortran, and many other languages. Input data 207 may also be stored in the memory. This input data 207 may include information regarding neighboring pixel discontinuities, identification of predefined patterns, and pixel blend amounts. During execution of program 203 , portions of program code and / or data may be loaded into local storage or memory of processor cores for parallel processing by multiple processor cores.

In addition, device 200 includes well-known support functions such as input / output (I / O) element 211 , power supply (P / S) 213 , clock (CLK) 215 , and cache 217 . 209 may be included. The apparatus 200 may optionally include a mass storage device 219 , such as a disk drive, CD-ROM, tape drive, for storing programs and / or data. The device 200 may optionally include a display unit 221 and a user interface unit 225 to facilitate interaction between the user and the device. For example, display unit 221 displays text, numbers, graphic symbols, or other visual objects as stereoscopic images perceived by 3D view glasses 227 , which may be shutter glasses connected to I / O element 211 . It may be in the form of a 3D ready television set, but is not limited to this example. Alternatively, the display unit 221 may include a 3D projector that simultaneously projects left and right eye images on the screen. In this case, the 3D viewing glasses may be passive glasses having left and right eye lenses of different colors or different polarizations. Stereoscopic images indicate an improvement in depth illusion in two-dimensional images by presenting slightly different images for each eye. The user interface 225 can include a keyboard, mouse, joystick, light pen, or other device that can be used with a graphical user interface (GUI). The apparatus 200 may also include a network interface 223 that allows the device to communicate with other devices via a network such as the Internet.

The components of system 200 , including processor 201 , memory 205 , support function 209 , mass storage device 219 , user interface 225 , network interface 223 , and display 221 , It may be operatively connected to each other via one or more data buses 229. These components may be implemented in hardware, software, or firmware, or some combination of two or more thereof.

There are many additional ways to simplify parallel processing using multiple processors in the device. For example, one can "unroll" processing loops by duplicating code on two or more processor cores and having each processor core implement code for processing different data. Such an implementation may avoid latency associated with establishing a loop. As applied to the present invention, multiple processors may determine discontinuities between pixels for a given image in parallel (eg, one processor performs a horizontal pass and another processor performs a vertical pass). The ability to process data in parallel saves significant processing time, making the system for morphological antialiasing of three-dimensional reprojection of two-dimensional images more efficient and simplified.

In particular, an example of a processing system capable of implementing parallel processing on three or more processors is a cell processor. There are a number of different processor architectures that can be classified as cell processors. For example, FIG. 3 shows one type of cell processor, but is not limited to this example. The cell processor 300 includes a main memory 301 , a single power processor element (PPE) 307 , and eight synergistic processor elements (SPEs) 311 . In the alternative, the cell processor may be configured in any number of SPEs. 3, the memory 301 , the PPE 307 , and the SPE 311 may communicate with each other and the I / O device 315 via a ring type element interconnect bus 317 . The memory 301 includes input data 303 having features in common with the above-described program. At least one of the SPEs 311 is in its local storage LS a morphological anti-aliasing of the three-dimensional reprojection of the two-dimensional picture instruction 313 and / or input data to be processed in parallel, for example as described above. It may include part of. The PPE 307 may include, in its L1 cache, morphological anti-aliasing of three-dimensional reprojection of two-dimensional image instructions 309 having features common to those described above. Further, instructions 305 and data 303 may be stored in memory 301 for access by SPE 311 and PPE 307 as needed. It should be noted that any number of processes included in the method of morphological antialiasing of three-dimensional reprojection of two-dimensional images of the present invention can be parallelized using a cell processor. MLAA has significant parallelization potential, and in multi-core machines, to achieve better load balancing by processing the final output image in idle threads (threads that terminate rendering or those that terminate building part of the acceleration structure). Can be used.

For example, PPE 307 may be a 64-bit Power PC Processor Unit (PPU) with an associated cache. The PPE 307 may include an optional vector multimedia expansion unit. Each SPE 311 includes a Synergy Processor Unit (SPU) and Local Storage (LS). In some implementations, local storage can have a capacity of about 256 kilobytes of memory, for example, for data and programs. SPUs are computation units that are less complex than PPU in that they typically do not perform system management functions. SPUs may have a single instruction multiple data (SIMD) capability and typically process data to perform assigned tasks and initiate any necessary data transfer (to access properties set by the PPE). SPUs enable the system to implement applications that require higher computational unit densities and can effectively use the provided instruction set. A significant number of SPUs in a system managed by the PPE allow for cost effective processing over a wide range of applications. For example, the cell processor may feature an architecture known as cell broadband engine architecture (CBEA). In a CBEA-compatible architecture, multiple PPEs can be combined into PPE groups and multiple SPEs can be combined into SPE groups. For example, a cell processor is shown as having a single SPE group and a single PPE group together with a single SPE and a single PPE. In the alternative, the cell processor may include multiple groups of power processor elements (PPE groups) and multiple groups of synergy processor elements (SPE groups). CBEA-compatible processors are, for example, Cell available online at https://www306.ibm.com/chips/techlib/techlib.nsf/techdocs/1AEEE1270EA277638725706000E61BA/$file/CBEA_01_pub.pdf It is described in detail in the Broadband Engine Architecture , which is incorporated herein by reference.

According to another embodiment, instructions for morphological antialiasing of three-dimensional reprojection of a two-dimensional image may be stored in a computer readable storage medium. For example, FIG. 4 shows an example of a non-transitory computer readable storage medium 400 according to one embodiment of the invention, but is not limited to this example. Storage medium 400 includes computer readable instructions stored in a format that can be retrieved, converted, and executed by a computer processing device. For example, the computer readable storage medium may be a computer readable memory such as RAM or ROM, a computer readable storage disk (eg, a hard disk drive) for a fixed disk drive, or a removable disk drive. However, such examples are not limited. In addition, computer readable storage medium 400 may be a flash memory device, computer readable tape, CD-ROM, DVD-ROM, Blu-ray, HD-DVD, UMD, or other optical storage medium.

Storage medium 400 includes instructions 401 for morphological anti-aliasing of three-dimensional reprojection of a two-dimensional image. Instructions 401 for morphological antialiasing of three-dimensional reprojection of a two-dimensional image may be configured to implement morphological antialiasing according to the methods described above with respect to FIG. 1. In particular, the instructions 401 for morphological anti-aliasing may include determining neighboring pixel discontinuity instructions 403 used to determine discontinuities between neighboring pixels of a given image. The determination of discontinuity can be completed in two stages. Vertical discontinuity between neighboring vertical pixels in one stage may be determined, and horizontal discontinuity between neighboring horizontal pixels in another stage may be determined. Alternatively, vertical and horizontal discontinuities may be determined simultaneously. A discontinuity may be when there is a difference in color profiles between two neighboring pixels, when there is a difference in shape profiles between two neighboring pixels, or in any number of other differences between neighboring pixels in a given image. When, it can happen.

Morphological anti-aliasing instructions 401 may also include identifying predefined pattern instructions 405 that identify one or more predefined patterns formed by discontinuities between pixels. Such predefined patterns may include a U-shaped pattern, a Z-shaped pattern, and an L-shaped pattern as described above.

Morphological anti-aliasing instructions 401 may further include calculating blend amount instructions 407 configured to calculate blend amounts for pixels neighboring predefined patterns formed by discontinuities. The blend amount refers to the weighted color / shape profile for a given pixel used for smooth transitions between discrete pixels. For example, a black pixel neighboring a white pixel converts a black pixel (and possibly other neighboring pixels) into a gray pixel so that the jagged edge feeling caused by discontinuity subsides when perceived by the viewer. Mixing amounts can be produced.

Morphological anti-aliasing instructions 401 may include applying three-dimensional reprojection instructions 409 that apply reprojection to both the two-dimensional image and the corresponding blend amount. Rather than applying the blending amount to the two-dimensional image before reprojection, these instructions three-dimensional reprojection of the blending amounts (ie, converting the blending amounts into corresponding corresponding three-dimensional reprojection values) so that blending can occur in a later step.

Morphological anti-aliasing instructions 401 mix three-dimensional reprojection instructions 411 that blend three-dimensional reprojection of a two-dimensional image according to the reprojected blend values and thereby generate one or more output images. It may further include.

Morphological anti-aliasing instructions 401 may further include display instructions 413 that format the output pictures for display on the display.

Embodiments of the present invention can implement MLAA in a manner that can produce better MLAA results than conventional approaches while reducing the amount of work that must be done by processors implementing MLAA.

Although embodiments have been described for viewing stereoscopic 3D images using passive or active 3D view glasses, embodiments of the present invention are not limited to these embodiments. In particular, embodiments of the present invention may be applied to stereoscopic 3D video techniques that do not rely on head tracking or passive or active 3D view glasses. Examples of such “glasses free” stereoscopic 3D video technologies are sometimes referred to as autostereoscopic techniques or autostereoscopy. Examples of such techniques include, but are not limited to, techniques based on the use of lenticular lenses. A lenticular lens is an array of magnifying lenses designed to magnify different images when viewed from slightly different angles. Different images may be selected to provide a three-dimensional viewing effect when viewing the lenticular lens from different angles. The number of generated images increases in proportion to the number of viewpoints for the screen. The more pictures used in such a system, the more useful embodiments of the present invention for implementing morphological antialiasing in such a system.

More specifically, in a lenticular lens video system, reprojected images of a scene at slightly different viewing angles may be generated from depth information and an initial 2D image for each pixel of the image. By using the reprojection technique, progressively different views of the scene at different viewing angles can be generated from the initial 2D image and depth information. Images representing different views can be divided into strips and displayed in an interlaced manner on an autostereoscopic display having a display screen disposed between the lenticular lens array and the view position. The lenses that make up the lenticular lenses may be cylindrical magnifying lenses that are aligned with the strips, and are generally twice the width of the strips. The viewer perceives different views of the scene according to the angle of view of the screen. Different views may be selected to provide depth welcome in the scene being displayed.

In addition, some embodiments of the present invention may solve the aliasing problem in the case of three-dimensional reprojection of a two-dimensional image and include generating more than one image for reprojection, although embodiments are non-3D of reprojection. More generally applicable. In addition, in some three-dimensional implementations, it may not be necessary to generate more than one image. For example, in the case of a stereoscopic display, it may not be necessary to generate both left and right eye images via reprojection. Alternatively, only one new picture may be generated through reprojection. For example, you can start with color and depth information for each pixel for a left eye image, and create a corresponding right eye image through reprojection (or vice versa), so that in a stereoscopic display Sufficient images may be generated for the display of. This involves generating only a reprojected single picture.

Although the invention has been described in great detail with respect to some preferred versions of the invention, other versions are possible. Accordingly, the spirit and scope of the claims should not be limited to the description of the preferred versions contained herein. Instead, the scope of the invention should be determined with reference to the claims, along with the full scope of equivalents of the claims.

All features disclosed in this specification (including claims, summaries, and drawings) may be replaced by alternative features that serve the same, equivalent, or similar purpose unless explicitly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is a general example of a series of equivalent or similar features. Any feature, whether preferred or not, may be combined with any other feature, whether desired or not. In the following claims, the indefinite article "a" or "an" refers to one or more quantities of items following the phrase, unless expressly stated otherwise. Any element in a claim that does not explicitly mention "means for" to perform a particular function is to be construed as a "means" or "step" clause as specified in US Patent Law (35 USC § 112, ¶ 6). It should not be. Specifically, the use of “steps” in the claims herein is not intended to apply the provisions of US patent law (35 USC § 112, ¶ 6).

Those skilled in the art are interested in paying attention to all papers and documents submitted concurrently with this specification and open for examination by the public with this specification, the contents of which are incorporated herein by reference.

Claims (24)

A method of morphological antialiasing of reprojection of one or more two-dimensional images,
a) determining one or more discontinuities between each neighboring pixel of the one or more two-dimensional images;
b) identifying one or more predefined patterns formed by one or more discontinuities in step a);
c) calculating a blending amount for each pixel neighboring the predefined pattern identified in step b);
d) applying reprojection to the blend amount and the one or more two-dimensional images for each pixel to produce one or more reprojected images and a reprojected blend amount;
e) blending neighboring pixels of the one or more reprojected images according to the reprojected blending amount to produce one or more output images; And
f) displaying the one or more output images. The method of claim 1, wherein the one or more output pictures comprise left and right eye views of a scene, whereby the output picture corresponds to reprojected left and right eye images of the scene. Aliasing method. 2. The method of claim 1, wherein step f) comprises displaying the reprojected left and right eye images on a three-dimensional display. 2. The method of claim 1, wherein discontinuity occurs between neighboring pixels when each pixel has a different color profile. 2. The method of claim 1, wherein discontinuity occurs between neighboring pixels when each pixel has a different shape profile. 2. The method of claim 1, wherein step a) comprises discovering horizontal discontinuities between neighboring horizontal pixels or vertical discontinuities between neighboring vertical pixels. 2. The method of claim 1, wherein step a) comprises finding horizontal discontinuities between neighboring horizontal pixels and finding vertical discontinuities between neighboring vertical pixels. The method of claim 1, wherein f) comprises: dividing the one or more two-dimensional images into strips, interlacing the strips into strips of one or more different two-dimensional images of different views of the scene, and interleaving. Generating a set of raced images and displaying the interlaced images on an autostereoscopic display having a lenticular lens disposed between the display screen and the view position. . The method according to claim 1, wherein the predefined pattern in step b) comprises an L-shaped pattern. The method of claim 1 wherein the predefined pattern in step b) comprises a U-shaped pattern. The method of claim 1, wherein the predefined pattern in step b) comprises a Z-shaped pattern. As a morphological antialiasing device,
A processor;
Memory; And
Computer coded instructions embodied in the memory and executable by the processor,
The computer-coded instructions are configured to implement a morphological anti-aliasing method of reprojection of one or more two-dimensional images,
The method comprises:
a) determining one or more discontinuities between each neighboring pixel of the one or more two-dimensional images;
b) identifying one or more predefined patterns formed by the one or more discontinuities;
c) calculating a blending amount for each pixel neighboring the predefined pattern identified in step b);
d) applying reprojection to the blend amount and the one or more two-dimensional images for each pixel to produce one or more reprojected images and a reprojected blend amount; And
and e) generating at least one output image by mixing neighboring pixels of said at least one reprojected image in accordance with said reprojected blending amount. 13. The morphological anti-aliasing device of claim 12, further comprising a three-dimensional visual display configured to display the one or more output images. 14. The morphological form of claim 13, wherein the one or more two-dimensional images comprise left and right eye views of a scene, whereby the output pictures correspond to reprojected left and right eye images of the scene. Anti-aliasing device. 14. The morphological anti-aliasing device of claim 13, wherein the display is an autostereoscopic display having a lenticular lens disposed between the display screen and the view position. The method of claim 15, wherein dividing the one or more two-dimensional images into strips and interlacing the strips into strips of one or more different two-dimensional images of different views of the scene to produce a set of interlaced images. Morphological anti-aliasing device, characterized in that. 13. The morphological anti-aliasing device of claim 12, wherein discontinuity occurs between neighboring pixels when each pixel has a different color profile. 13. The morphological anti-aliasing device of claim 12, wherein discontinuity occurs between neighboring pixels when each pixel has a different shape profile. 13. The morphological anti-aliasing device of claim 12, wherein step a) comprises first discovering horizontal discontinuities between neighboring horizontal pixels and then finding vertical discontinuities between neighboring vertical pixels. 13. The morphological anti-aliasing device of claim 12, wherein step a) includes first discovering vertical discontinuities between neighboring vertical pixels and then finding horizontal discontinuities between neighboring horizontal pixels. 13. The morphological anti-aliasing device of claim 12, wherein the predefined pattern in step b) comprises an L-shaped pattern. 13. The morphological anti-aliasing device of claim 12, wherein the predefined pattern in step b) comprises a U-shaped pattern. 13. The morphological anti-aliasing device of claim 12, wherein the predefined pattern in step b) comprises a Z-shaped pattern. As a computer program product,
A non-transitory computer readable storage medium having computer readable program code for morphological antialiasing (MLAA) of reprojection of two-dimensional images,
The computer readable program code is embodied in the non-transitory computer readable storage medium, and the computer program product includes:
a) computer readable program code means for determining one or more discontinuities between each neighboring pixel of the two-dimensional image;
b) computer readable program code means for identifying one or more predefined patterns formed by the one or more discontinuities;
c) computer readable program code means for calculating a blending amount for each pixel neighboring the predefined pattern identified in step b);
d) computer readable program code means for applying a reprojection to the blend amount and the two-dimensional image for each pixel to produce a reprojected image and a reprojected blend amount;
e) computer readable program code means for mixing one or more output images by blending neighboring pixels of the reprojected image according to the amount of reprojected blending; And
f) computer readable program code means for displaying said at least one output image.

Images