Classifications

• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
  • G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
  • G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
  • G02B30/26—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type
  • G02B30/27—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type involving lenticular arrays
  • G02B30/29—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type involving lenticular arrays characterised by the geometry of the lenticular array, e.g. slanted arrays, irregular arrays or arrays of varying shape or size
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
  • H04N13/305—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using lenticular lenses, e.g. arrangements of cylindrical lenses
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
  • G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
  • G02B30/26—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type
  • G02B30/27—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type involving lenticular arrays
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2300/00—Aspects of the constitution of display devices
  • G09G2300/04—Structural and physical details of display devices
  • G09G2300/0439—Pixel structures
  • G09G2300/0452—Details of colour pixel setup, e.g. pixel composed of a red, a blue and two green components
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2340/00—Aspects of display data processing
  • G09G2340/14—Solving problems related to the presentation of information to be displayed
  • G09G2340/145—Solving problems related to the presentation of information to be displayed related to small screens

Definitions

• the present invention relates to an autostereoscopic display system, and more specifically to a method of hardware based interdigitation for an autostereoscopic display system using component view-map tiles.
• the StereoGraphics SynthaGramTM is an autostereoscopic imaging product that utilizes an array of (typically slanted) lenticular elements on the display surface as shown in FIG. 1.
• U.S. Patent No. 3,409,351 to Winnek describes the slant, or a process in which the lenticular axis is tipped at an angle to the vertical edge of the display.
• Special software in the StereoGraphics SynthaGramTM arranges multiple- view image information using an interdigitation mapping technique, such that someone viewing the display will see one of the left-eye views with their left eye and one of the right-eye views with their right eye. This results in a stereoscopic viewing experience without the need to use special glasses, mirrors, or other stereo selection devices used at the eye.
• each source-view image is provided, each representing a different viewpoint to be integrated into the autostereoscopic scene.
• These source-view images may be from a photographic source or a computer graphics source, and may have been prepared previously, or in real-time (effectively at the same time that they are being displayed).
• the number of views may be greater or less than nine without loss of generality.
• the StereoGraphics SynthaGramTM algorithm for interdigitation is called Interzig, and the two terms will used interchangeably throughout this disclosure.
• the interdigitation process takes subpixels (a subpixel is the red, green, or blue component of a single pixel) from all of the different source- view images, and arranges the sub- pixels in the resultant interdigitated image. For example, a subpixel from a particular source- view image might be copied to the Interzigged image, if that subpixel will appear under the portion of a micro-lens in the lenticular array corresponding to a particular viewing zone for the person viewing the image.
• the interdigitation process performs the mapping of sub-pixels with an understanding of the optical display properties. These include the physical arrangement of the lenticular array (pitch, slant, offset) as well as the viewing distance between a viewer and the interdigitated image.
• Interzigged subpixel data is passed to the actual subpixels in the display device and passed through a lenticular microlens array, resulting in an autostereoscopic viewing experience.
• the system may create a view-map.
• the view-map is encoded to indicate the specific source- view image used as the source for any given subpixel appearing in the final image, and an example of such a view-map is shown in FIG. 3.
• the interdigitation process thus takes the source view images, such as nine such source-view images, and maps them to a single view-map image, thereby speeding up the interdigitation process. Having a pre-calculated view-map saves the trouble and computation time of repeatedly calculating which source-view image should be used per any given subpixel in the final image.
• the system can determine the intensity of a particular subpixel based on the color component value of a pixel in the appropriate image-map location belonging to the source-view image to which the view-map value points.
• the view-map can be applied efficiently using different implementations including pixel masking, where the view-map spawns a set of binary masks that filter source-view images, and pixel, where fast per-pixel calculations are performed by referencing a view-map stored in the graphics device's texture. Even with high-speed algorithms such as those described above, a performance penalty can result from a significantly large view-map. As described above, the view-map matches the full size of the final image being generated (and may take up an even larger segment of video memory). If the view-map and possibly the binary masks that the view-map generates could be smaller, the system would require less graphics memory, and considerable processing speed benefits could be realized. Processing speed benefits generally result from a decreased time to swap textures in and out of limited texture memory.
• additional textures may be used for visual effects (surface patterns).
• the view-map texture competes for memory space with these other textures. For example, a final image created for a full screen size of 3840x2400 pixels requires an excessively large view-map sized at 4096x4096, if the graphics device has a power-of-2 size limitation.
• the present design seeks to cure the problem of view map performance and time penalties to increase the throughput of the device and increase viewer enjoyment of the resultant autostereoscopic image. It would be advantageous to offer a design that employs a view-map in a more efficient manner within an autostereoscopic display system, particularly a design that offers benefits over those previously available.
• a method for creating a reduced sized view-map of an autostereoscopic source image comprises creating a mathematical model for mapping the autostereoscopic source image into a first view-map, mapping the autostereoscopic source image into the first view-map, wherein the first view-map comprises a source view image value and a plurality of pixel locations comprising an origin pixel, scanning the first view- map for at least one pixel location whose source-view image value closely matches that of the origin pixel, and generating the reduced sized view-map by replicating matched pixel locations within the first view-map.
• FIG. 1 illustrates a display with a lenticular screen attached thereto
• FIG. 2 shows a profile of a lenticule and the subpixels covered by the leticule
• FIG. 3 represents the view-map pattern, on which the interdigitation is based
• FIG. 4 shows an arrangement where view-map tiles are arranged in a grid fashion
• FIG. 5 shows a "brick" arrangement of view-map tiles where a horizontal offset is allowed between rows
• FIG. 6 shows a variation of the tiled view-map implementation.
• FIG. 1 illustrates such a display with the lenticular screen attached.
• Display device 101 is the flat-panel display device, while screen 102 is a lenticular screen comprising a micro-lens array. Screen 102 is placed over the display surface in order to direct different components of the autostereoscopic view to different segments of a user's or viewer's viewing zone.
• FIG. 2 shows a profile of one particular lenticule covering subpixels of the display device 101.
• Lenticule 201 is one lenticule of the micro-lens array.
• the lens element faces out from the screen, but the lenticules may alternately be inward facing.
• Subpixels 202 represent the device subpixels, each of which generates a dot of light. With currently manufactured flat-panel displays, these device dots each carry either red (R), green (G), or blue (B) information.
• R red
• G green
• B blue
• the essential attribute of subpixels 202 here is that they represent dots of light that are discrete and separately addressable, not that they are specifically red, green, and blue; future displays may arrange display dot elements to carry different kinds of color or light information, such as multi-color dot elements, for example.
• Region 203 shows that each lenticule is divided up into nine sub-regions (in this example, we assume that the autostereoscopic image uses nine source-view images), here numbered 0 through 8, corresponding to source-view images that may be represented under the appropriate lenticule sub-regions.
• the middle source view, number 4 preferably has its data positioned directly underneath the middle sub-region of the lenticule.
• the system uses a pre-calculated view-map as a map of the source-view image to be used for any given subpixel.
• This view-map guides the implementation of the interdigitation much more rapidly and efficiently than if the interdigitation process was based on repeated calculations of the particular source- view image to be used.
• FIG. 3 illustrates an example of such a view-map.
• Region 301 shows that the view-map is an array of numbers.
• Single pixel 302 represents a portion of the view- map array that represents a single pixel.
• the pixel includes three subpixels in this arrangement, each subpixel representing a discrete addressable point of light on the display device.
• the subpixel set includes a red (R), green (G), and blue (B) subpixel.
• Red subpixel 303 represents a particular red (R) subpixel's location in the view-map.
• the value contained in this member of the view-map array happens to be 0, indicating that the corresponding device subpixel should obtain its graphical information from the corresponding source- view, #0.
• the value pertaining to red subpixel 303 in this example 0 is likely to be an approximation, a round-off of the actual source- view image value of that particular subpixel location.
• the actual source- view image value may be 0.25, meaning that the corresponding device subpixel should ideally obtain its graphics information from a source- view that is 1 A of the way from source- view #0 to source- vie w #1. Since no source- view exists between source- view #0 and source- view #1, the source-view image value for that sub-pixel location must necessarily be rounded off from its actual value of 0.25 to the integer 0.
• the present design outlines an enhanced, efficient view-map tiling solution, addressing the need for view-maps that are significantly smaller than the final interdigitated image that they map. View-maps, being smaller than full-image size can then be reused over different tiles, or portions, of the final image.
• Generating smaller view-maps that effectively map the required views requires taking the mathematical model used to generate a view-map or a first view map and scanning the view-map for a pixel location whose source-view image value closely matches that of the origin pixel.
• the source- view image value here represents the precise value, not the rounded off integer version of this value that ends up in the view-map.
• the mathematical model used to generate a view-map comprises an array of offset values. Typically nine source- view images are employed, but other numbers of source-view images may be used. With nine source- view images, the final view-map has each mapped subpixel pointing to one of nine source- view images, based on that subpixel's position beneath a lenticular optical element.
• Such a mapping is typically based on approximation of a best position to display the pixel such that when passing through the lenticule and to a viewer it results in an optimal viewing condition. For example, a particular subpixel location's calculation might call for a source- view image value of 4.3278; this value rounds off to 4, and so that subpixel location will receive a value of 4, meaning that source- view image #4 will be used to represent that subpixel in the final interdigitated autostereoscopic image.
• Calculating the source-view image value for any given subpixel location typically entails a mapping calculation, where known values pertaining to the lenticular pitch, slant, position, and other physical attributes are used to determine precisely which portion of a given lenticular element reside over a particular position of a particular subpixel grid.
• mapping calculation There are various methods for performing this calculation, some involving iterative additions of offset values for each new pixel in a given row and each new row of an overall screen; other methods involve multiplying position values by pre-calibrated constant values. What is important is not how the mapping is calculated, but that the view-map ends up representing the positioning of lenticular elements above the full range of subpixel locations on the screen.
• view-map calculations end up being rounded off to reflect the fact that a limited number of actual views are available, which are represented by integer source-view image values in the view-map.
• the system locates a repeating pattern in the nine source-images.
• the system locates a repeating pattern by scanning the source-images for such a repeating pattern, within a full-sized view-map, that closely matches the numeric pattern found near the origin of such a full-sized view-map.
• the origin (very first) subpixel has a source-view image value of 1.3000
• another subpixel in the same row (and representing the same color component as the origin subpixel) has a source- view image value of 1.3000 as well (or something very close to that value)
• the view-map information starting from that second subpixel becomes redundant.
• a copy of the (smaller than full-width) view-map information up to that subpixel location may be reused starting at that subpixel.
• the system uses the accurate source-view image values rather than the rounded off integer values.
• the general algorithm used to find an optimal tile size along a particular axis involves scanning along that axis for the lowest possible error.
• the error would be 1.3000 - 1.3000, or 0.0000.
• error represents the difference between view image values between pixels, such as between adjacent pixels. Normally, an error of zero is not achieved, nor is such perfection necessary. More often, the error values will be fractional values. These fractional error values may then be multiplied by the number of times that the error will be compounded to determine the total error.
• the horizontal pixel size of the final image is 1600, where a tile dimension of 300 is being tested, that tile size may result in an error value of 0.06 per tiling instance (six errors per 100 tiles).
• the total error in this example will actually be approximately (1600/300 x 0.06), or about 0.3.
• the objective is to scan for a tile dimension whose total error, over the full area of the resulting image, is minimized.
• tile 401 is a tile in the upper-left corner of a representation of the image area.
• Region 402 shows that this tile contains a view-map, similar to that in FIG. 3, having its origin at the corner of the tile.
• Tile 403 is a separate tile instance in the image area. Note that the origin of this tile has exactly the same view-map data (the source- view image values all being ⁇ 0, 2, 4, 6, 8, 1, 3, ... ⁇ as is used in tile 401.
• tile 404 is yet another tile instance, appearing directly below another tile. Note that the full set of tiles in FIG. 4 form a grid arrangement. The result is that the system has multiple regions of the screen area, those collective regions forming a grid.
• Determining grid size is a matter of finding the optimal horizontal tile dimension, where optimal indicates a minimized total error along that dimension, and fmding the optimal vertical tile dimension.
• the advantage of a "bricks" tiling arrangement, such as that shown in FIG. 5, with a horizontal offset between rows, is that a smaller total error may result in the vertical dimension if the tile position is not restricted to being directly above or below other tiles.
• tile 501 is a tile in the upper-left corner of a representation of the image area.
• Region 502 represents this tile's view-map, with its origin at the corner of the tile.
• Tile 503 is a separate tile instance in the image area, with the origin having the same view-map data as is used in tile 501.
• Tile 504 is a tile instance in the following row. Note that tile 504 is not directly below any of the tiles in the above row; there is a horizontal offset, resulting in a "brick” like arrangement. Tile 505 shows that when using this "brick" layout, some tiles will result where the view-map origin precedes the overall image origin, i.e. a partial overall image will be presented.
• tile 601 in Figure 6 shows a tile in the upper- left corner of a representation of the image area.
• Region 602 is tile 601 's view-map, a two-dimensional array of numbers starting (in this example) with 0, 2, 4, 6, 8, 1, 3, ...
• Tile 603 is the next tile instance.
• FIG. 6 we use the same view- map data array (once again starting with 0, 2, 4, 6, 8, 1, 3, ...), but for each subpixel the system adds, in this example, a value of 4. Note that if the sum equals or exceeds the total number of source-view images (9), the system subtracts that number to normalize the index value.
• subpixels in this tile 603 use source- view images 4+0, 4+2, 4+4, 4+6-9, 4+8-9, 4+1, 4+3, ...
• Tile 604 is a tile instance in the following row.
• the system uses the same view-map and associated data, but the system applies a different offset factor (in this example, 1) for the next-row implementation of the tile.
• this enhancement entails applying at least one offset factor to all subsequent tiles, reducing the need for processing and producing acceptable overall viewing results.
• Tile 605 is a tile instance offset horizontally from tile 604.
• Tile 605 uses the view-map data, added to the vertical tile offset value (1) that was also used in 604, and added to the horizontal tile offset value (4) used in tile 603.
• a different offset value may be applied to different row instances of the view- map tile.
• the error value minimized is no longer the non-integer source- view value calculated for some subpixel minus the source- view value of the origin subpixel. Instead, the system aims to minimize the difference between the fractional portion of the arbitrary subpixel' s calculated source- view value and the fractional portion of the origin subpixel's source-view value. Changing the optimization to just compare fractional portions of the numbers (which range from 0 to the total number of source-view images, typically 9) can improve the selection of optimization candidates significantly.
• Interdigitated stereoscopic images can appear acceptably seamless if the overall total tiling error is within approximately 5% of the range that would round off to a single source- view image.
• the just described technique of pointing to source-view images using an incremented index value nearly can yield a tiling solution with a total tiling error factor below 5%. Consequently, using tiled view-maps, the present design produces interdigitated stereoscopic images that appear seamless, accompanied by greatly improved processing speed compared to using full- sized view-maps.
• additional performance benefits may be derived from keeping the total number of tiles within a particular maximum limit.
• the processing time advantage in having a smaller view-map size might be outweighed by the amount of time the view-map needs to be applied to complete the display, depending on the computational hardware and implementation.
• the algorithm that finds the optimal tiling scheme can be modified to restrict solutions according to this and other possible restrictions.
• the tiling scheme Before the tiling scheme can be applied, such a scheme needs to be described by a set of numbers that constitutes the tiling specification.
• the objective is to represent the tiling scheme in a relatively simple manner such that the system assembling interdigitated images for autostereoscopic display can apply the view- map, which is smaller than full-screen, accurately to tiled regions of the screen. Such assembling results in a visually seamless effect, and autostereoscopic graphics as good as using a full-screen view-map.
• the tiling specification includes several numerical components.
• the view-map tile has horizontal and vertical pixel dimensions.
• the computer system can assume that additional positionings of the view-map tile should occur following, and immediately adjoining, the previous tile, both horizontally and vertically.
• the tiling specification also needs to specify the horizontal offset per vertical row. If the tiling methodology is to address source- view images based on an increment applied to the value indicated by the view-map, the tiling specification needs to indicate the particular amount of that increment.
• the tiling specification might typically include a separate increment value for the horizontal and vertical tiling dimensions. Tiling error information may be included as part of the tiling specification, for the express purpose of judging whether results are acceptable. For example, if a 5 per cent tiling error is acceptable, the tiling specification may include this value such that a tiling error in excess of five per cent indicates an unacceptable solution.
• the system may apply the tiling scheme programmatically, determining where each instance of the view-map tile overlays the overall interdigitated image being produced.
• the computer or display system implementation can lay out the tiles based on the particular tiling specification that has been calculated. If source-view increment values are used as part of the tiling model (such as was illustrated in FIG. 6), the computer or display system implementation will need to apply these increment values correctly according to the tiling model and specification, which will typically involve recalculating increment values for each tile position, based on horizontal and vertical increment offset values indicated by the tiling specification.
• a computer or display system applies to the current tiling scheme, and that application typically involves obeying rules about the nature of how tiles are to be laid out, subject to the constraints specified for the tiling model being implemented.
• the simple example is a grid tiling (with no view-map increments used).
• the "rules" might be to base the first tile at the screen origin, and base all subsequent tiles (horizontally and vertically) such that they fall adjacent to the previous tiles.
• the tiling scheme and its implementation may additionally need to both specify what the horizontal offset per row is and how the horizontal offset is to apply.
• tiling specifications are highly dependent upon the specific needs addressed and the desired performance or activities.
• a rule-set is established together with a specification that abides by that rule-set, in order to insure that increment values are applied simply and accurately.
• Multiple rule-sets for performing this function may be employed, but one such rule-set may be employed having a horizontal and vertical increment value added for each successive tile that appears horizontally and vertically.
• the present design is generally hardware independent in the sense that it may be run on any general high performance computational architecture or hardware.
• the autostereoscopic interdigitation processing using tiled view-maps disclosed herein may encompass processing on a general purpose or specialized high performance computer known generally to those skilled in the art.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• General Physics & Mathematics (AREA)
• Computer Hardware Design (AREA)
• Theoretical Computer Science (AREA)
• Geometry (AREA)
• Optics & Photonics (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
• Image Processing (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Publications (2)

Family

ID=37053933

Family Applications (1)

Country Status (4)

Cited By (1)

Families Citing this family (22)

Citations (1)

Family Cites Families (18)

• 2006
  • 2006-03-24 US US11/388,352 patent/US7570260B2/en not_active Expired - Fee Related
  • 2006-03-24 EP EP06739507A patent/EP1869660A4/en not_active Withdrawn
  • 2006-03-24 WO PCT/US2006/010758 patent/WO2006104878A2/en not_active Ceased
  • 2006-03-24 JP JP2008504190A patent/JP5070198B2/ja not_active Expired - Fee Related

Patent Citations (1)

Also Published As

Similar Documents

Legal Events