Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/349—Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/20—Image signal generators
  • H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/50—Lighting effects
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T19/00—Manipulating 3D models or images for computer graphics
  • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
  • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
  • G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/172—Processing image signals image signals comprising non-image signal components, e.g. headers or format information
  • H04N13/183—On-screen display [OSD] information, e.g. subtitles or menus
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
  • H04N13/305—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using lenticular lenses, e.g. arrangements of cylindrical lenses
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
  • H04N13/31—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using parallax barriers
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/324—Colour aspects
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/366—Image reproducers using viewer tracking
  • H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/398—Synchronisation thereof; Control thereof
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
  • G06T2219/20—Indexing scheme for editing of 3D models
  • G06T2219/2012—Colour editing, changing, or manipulating; Use of colour codes
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2300/00—Aspects of the constitution of display devices
  • G09G2300/04—Structural and physical details of display devices
  • G09G2300/0439—Pixel structures
  • G09G2300/0452—Details of colour pixel setup, e.g. pixel composed of a red, a blue and two green components
• • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
  • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
  • Y10T—TECHNICAL SUBJECTS COVERED BY FORMER US CLASSIFICATION
  • Y10T29/00—Metal working
  • Y10T29/49—Method of mechanical manufacture
  • Y10T29/49002—Electrical device making
  • Y10T29/49004—Electrical device making including measuring or testing of device or component part

Definitions

• the invention relates to a method for visualizing three-dimensional images on a 3D display device, wherein an image to be visualized is supplied as an input image. Furthermore, the invention relates to an SD display device, in particular a stereoscopic or autostereoscopic display, for the visualization of three-dimensional images.
• autostereoscopic visualization systems that allow one or more viewers, who are in front of an autostereoscopic display, to view a three-dimensional image without visual aids.
• e.g. Parallax barrier systems or lenticular lens systems positioned in front of the display panel. Since one or more observers may be at different angles relative to a direction perpendicular to the display, more than two perspectives must always be generated and fed to the respective left and right eyes in order to provide as natural a three-dimensional image impression as possible for the viewer allow respective viewer.
• These systems are also referred to as multi-viewer systems or multiview systems.
• the present invention is therefore based on the object, a method for the visualization of three-dimensional images on a 3D display device of the type mentioned in such a way and further, that the
• CONFIRMATION COPY Visualization of three-dimensional images is improved with simple constructive means. Furthermore, a corresponding 3D display device is to be specified.
• the above object is solved by the features of claim 1.
• the present method for visualizing three-dimensional images on a 3D display device is characterized in that using the input image at least one feature matrix is determined, wherein the feature matrices define light / dark information, and that from the input image using the Hell / Dark information, a display image for reproduction is generated on the SD display device.
• the SD display device in question in particular a stereoscopic or autostereoscopic display, for the visualization of three-dimensional images, is characterized in that the 3D display device has means which determine at least one feature matrix using an input image supplied, wherein the feature matrices Define bright / dark information, and that the means from the input image using the light / dark information to generate a display image for playback on the 3D display device.
• the input image may comprise two perspectives - partial images - which correspond to a left and a right partial image.
• the sub-images may be rectified, i. be brought to stereo normal form.
• features can be extracted from the input image (s).
• the features can describe local properties. For example, shapes, textures and / or edges can be considered as local properties.
• a feature extraction Sobel feature extraction can be used.
• a feature matrix comprises edge information.
• the human brain essentially uses edges on objects to build up the three-dimensional space image in the brain. Consequently, the edge information greatly facilitates the work of the observer's brain and improves the adaptation to the anatomy of the eye and the downstream information processing in the brain.
• feature extraction can be carried out using a Speed up Robust Features (SURF) feature extraction.
• SURF Speed up Robust Features
• H. Bay, T. Tuytelaars and LV Gool Speeded Up Robust Features
• Computer Vision and Image Understanding 110 (3), 2008, pp. 346-359.
• a feature matrix may include information about prominent pixels.
• a characteristic value can be assigned to a pixel of a perspective of the input image as light / dark information by a feature matrix. Consequently, feature values can be assigned to each pixel of a perspective of the input image as light / dark information.
• the feature value can be added to the subpixels of the associated pixel in the display image and / or multiplied.
• the feature value can be weighted with a scaling factor.
• the scaling factor can be changed interactively by a control unit, preferably with a remote control.
• RGB subpixels In a display device with RGB subpixels, the features of an edge operator used to emphasize the edges in the RGB subpixels can be used.
• a pixel e.g. consisting of the RGB subpixels, can be adapted to the anatomy of the eye as follows:
• R new (i, j): R (i, j) + M (i, j, 1) - s,
• B new (U): B (i, j) + M (ij, 1) - s, where R (i, j), G (i, j) and B (i, j) are the respective colors red, green and define blue.
• M (i, j, 1) is the value of the edge operator or edge information feature matrix for the pixel (i, j).
• s is a freely adjustable scaling parameter. When controlled via a remote control, each viewer can adjust the edge enhancement for their own feelings. This method slightly emphasizes the color values of the stereo image in the edges (but not color distorted) and makes them more easily recognizable for the light / dark receptors.
• R new (i, j): R (iJ) + s * ZM, (i, j) * s "
• G new (i, j): G (ij) + s * ZM, (i, j) * s "
• the light / dark information can be displayed with additionally supplemented light / dark subpixels in the display image.
• the bright / dark subpixels can significantly improve the viewer's spatial image impression by displaying edge information.
• an autostereoscopic display could be characterized by a panel of subpixels and an upstream optical element.
• the subpixels can be both color subpixels such as RGB or CMY and bright / dark subpixels.
• the color information of the subpixels of the perspectives to be displayed can be displayed.
• the light / dark subpixels can contain, for example, grayscale image features that support the impression of the image.
• the human eye has about 1 10 million light / dark receptors and only about 6.5 million color receptors.
• edge information can be displayed via light / dark subpixels, and thus, this image information becomes taken over the much larger number of light / dark receptors. The work of the brain is relieved.
• a pseudoholographic display may e.g. contain at least 10 to 20 times as many subpixels as are present in an input image supplied as a stereo image. This greater number of subpixels makes it possible to represent a larger number of pixels per perspective out of the many perspectives that are synthesized.
• High-definition images and videos of today's generation have about 1920x1080 pixels with 5760 subpixels per line.
• a display may advantageously have at least 76,800x1080 subpixels. It is taken into account that in the autostereoscopic displays, the assignment of perspectives takes place at the subpixel level. A summary of pixels is not relevant there.
• a stereoscopic display or an autostereoscopic display can be used as the 3D display device.
• the input image may have a first perspective and a second perspective, wherein the second perspective is generated by shifting the first perspective by an amount m> 0.
• the supplied 2D image can be used as a left partial image.
• the same 2D image can be used to shift to the right by an amount m> 0.
• a disparity matrix of the following kind can be generated:
• the viewer could let the viewer choose the amount m interactively, e.g. By means of a remote control, the viewer can adjust the "pop-out” or "pop-in” effect at any time.
• the 3D display device may comprise subpixels comprising subpixels for representing a color of a predeterminable color system and light / dark subpixels for displaying feature information.
• subpixels can be advantageously designed as independent elements.
• the subpixels can have the same extent in the horizontal and vertical directions.
• the subpixels are square, whereby a higher resolution can be achieved.
• a round design of the subpixels Thus, the hitherto customary requirement that all subpixels of a pixel together form a square is dropped. Rather, each subpixel is an independent element.
• Each of these subpixels has a color of the selected color system and has the same extent in the horizontal and vertical directions. With the OLED or Nano technology, this is technically easy to implement.
• FIG. 1 shows a block diagram for illustrating the overall system according to an embodiment of the method according to the invention and the 3D display device according to the invention
• FIG. 1 is a flowchart of the overall system of FIG. 1,
• Fig. 3 is a conventional subpixel layout compared to a new one
• FIG. 5 shows the subpixel layout of FIG. 4, wherein a larger number of different perspectives are activated.
• FIG. 1 shows a block diagram for illustrating the overall system according to an exemplary embodiment of the method according to the invention and the 3D display device according to the invention.
• the exemplary embodiment according to FIG. 1 relates to a method for the visualization of three-dimensional images on an autostereoscopic display as a 3D display device, on which a plurality of perspectives, generally more than 100, from a supplied stereo image in any 3D format are interlaced on the display are displayed.
• the display consists of an optical element and an image-forming unit. The multitude of perspectives is generated in such a way that only those pixels of a perspective are generated, which also have to be displayed.
• the imaging unit of the display consists of subpixels emitting a color, eg red, green or blue.
• the autostereoscopic display is capable of receiving a 3D image or an SD image sequence in any format, such as a stereo image or a stereo image sequence. Other formats, such as stereo image including a disparity card, can also be received and processed.
• a received stereo image is first rectified, i. brought to stereo normal form or in the epipolar standard configuration. If this is already the case, then the identical figure arises here.
• the disparity map of the stereo image is calculated. It contains an assignment of the pixels of the left and right field, which are present in both received perspectives. In addition, the left and right occlusions are identified.
• any number of perspectives are synthesized. This is done so that only those subpixels are synthesized, which must be displayed on the display actually. Thus, for 100 perspectives to be displayed, only 1% of the subpixels are calculated from each perspective.
• the information about which perspective is to be displayed on which subpixel is defined in the perspective map P.
• the perspective map is defined and stored in the production of the display by a calibration process between subpixels and optical system. Adjacent subpixels are generally associated with different perspectives. The storage of different subpixels from the different perspectives in the pseudoholographic image B is referred to as mating.
• the autostereoscopic display is characterized by a panel of subpixels and an upstream optical element.
• the subpixels are color subpixels, e.g. RGB or CMY.
• RGB or CMY color subpixels
• the color information of the subpixels of the perspectives to be displayed is displayed.
• a pseudoholographic display according to FIG. 1 has at least 10 to 20 times as many subpixels as are present in the received stereo image. This greater number of subpixels makes it possible to represent a larger number of pixels per perspective out of the many perspectives that are synthesized.
• FIG. 2 shows a flowchart with the individual steps to the overall system from FIG. 1.
• FIG. 1 and FIG. 2 The steps according to FIG. 1 and FIG. 2 are described in more detail below.
• a picture sequence of stereo images is received, decoded and made available in the memory areas I, and I r by a receiving module, for example via an antenna or the Internet.
• a display with a resolution of 19,200 x 10,800 pixels can be considered high-resolution.
• a stereo HD image enlarged tenfold horizontally and vertically.
• a first step the rectification is performed.
• These methods are known from the literature.
• nine prominent points which are distributed uniformly over the image, are searched for by the SURF method.
• the coordinate of each distinctive Point is used as the center of a search block in the right field l r .
• this search block the most similar point in the right field l r is searched.
• SURF Speed up Robust Features
• Sobel Edge Detector method For calculating features, e.g. the SURF (Speed up Robust Features) or the Sobel Edge Detector method are used.
• This method is based on approximating the determinant of the Hesse matrix for each pixel.
• the procedure is as follows.
• NZ the number of rows
• NS the number of columns
• M r (/, y ' , 1): (/, j) ⁇ Dyy (, j) - 0.81 ⁇ (/, j) ⁇ (, j)
• the Sobel operator is just one of a large number of edge operators and is therefore described as an example.
• an edge operator is of particular importance, as it helps to assign more importance to edges than to smooth surfaces. Since an edge is always a regional property, this procedure within a row also allows to take into account the properties of local regions.
• the Sobel-Prewitt operator works with 3x3 matrices that detect the edges in different directions. Basically, here are horizontal, vertical, left and right diagonal edges to distinguish. For their detection, the following 3x3 matrices are used:
• each row i is assigned a row calculation unit i.
• the computational-local field edge (1) to edge (9) is filled from the right-rectified partial image R r as follows:
• ⁇ ⁇ : 2 ⁇ edge (2) + 2 ⁇ edge (5) + 2 ⁇ edge (8)
• FIG. 3 shows on the left a conventional pixel layout with the three subpixels R (red), G (green) and B (blue). With these subpixels, the three perspectives 1, 2 and 3 are operated using a lenticular lens as optical element O.
• FIG. 3 shows a new subpixel layout on the right, wherein the autonomous subpixels according to an embodiment of the 3D display device according to the invention as a autostereoscopic display a square Design have. 9 perspectives can be controlled by the optical element O with 9 subpixels.
• FIG. 4 once again shows a conventional pixel layout on the left.
• Fig. 4 right another embodiment of an SD display device according to the invention is shown as autostereoscopic display.
• a much finer and more detailed subpixel structure is created.
• 144 subpixels are generated in the subpixel layout of the embodiment.
• the subpixels R (red), G (green) and B (blue) are another subpixel W (eg white or yellow) for Presentation of light / dark information added.
• these 144 subpixels 36 perspectives are activated in the illustrated embodiment.
• FIG. 5 shows the subpixel layout of FIG. 4, wherein the 144 individual, independent subpixels are used to drive 144 perspectives.
• the procedure for an autostereoscopic display can be as follows.
• the resolution of the human eye is i.A. between 0.5 'and 1 .5'. Therefore today's displays usually have a dot pitch of 0.2 to 0.3 mm. That is, from a distance of about 1 m, the pixels of the display are no longer visible.
• the lens width of the lens grid used is in the range of 0.2 mm. That is about 125 LPI (lenses per inch).
• the lens structure is no longer recognizable from a viewing distance of about 1 m.
• the number of subpixels behind a lens is on the order of 10 subpixels per lens. That is, the dot pitch of a pseudo-holographic display is on the order of about 0.06 mm. While in conventional displays, e.g. 1920 x 1080 pixels (HD-TV), a pseudo-holographic display presented here consists of at least 19,200 x 1080 pixels.
• the lenticular may consist of lenticular lenses or hexagonal lenses, for example, which in this case have a diameter of 0.2 mm.
• lenticular lenses or hexagonal lenses for example, which in this case have a diameter of 0.2 mm.
• the human brain uses the edge information to a great extent for the generation of the inner space image. Edges give information about the front / back relationship of objects with the existing right and left covers.
• the known subpixels RGB or YMC are particularly advantageously supplemented by light / dark subpixels which indicate the edge information generated in the feature extraction phase by the edge operators.
• Homogeneous surfaces contain no edges.
• the light / dark subpixels do not represent any information in the image.
• Edges are brightened in the light / dark subpixels according to the intensity of the detected edge. As a result, the edges present in the image are highlighted and more easily recognized by the 100 million light / dark receptors.
• the brain has it so easier to create the inner space image. Patterns on homogeneous surfaces are recognized as such in the human brain due to the learning effect and do not affect the spatial impression.
• the geometric arrangement of the light / dark subpixels can be varied according to the invention.
• Fig. 3 Fig. 4 and Fig. 5, various divisions are shown.
• further features for spatial image generation can be added.
• the SURF operator is an example, but not exclusive, called the SURF operator.
• each color or light / dark subpixel is in itself a self-contained light element associated with a particular perspective and has the same extent in the horizontal and vertical directions. This is already taken into account in FIGS. 3, 4 and 5 on the right-hand side. A backward compatibility is still given, so that all 2D images and videos can be displayed easily.
• the production of such an autostereoscopic display is possible based on the OLED technology.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Graphics (AREA)
• Computer Hardware Design (AREA)
• Human Computer Interaction (AREA)
• Architecture (AREA)
• Software Systems (AREA)
• General Engineering & Computer Science (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
• Devices For Indicating Variable Information By Combining Individual Elements (AREA)
• Stereoscopic And Panoramic Photography (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=44149833

Family Applications (1)

Country Status (6)

Families Citing this family (11)

Family Cites Families (51)

• 2010
  • 2010-02-25 DE DE102010009291A patent/DE102010009291A1/de not_active Ceased
• 2011
  • 2011-02-25 KR KR1020127024212A patent/KR101825759B1/ko active IP Right Grant
  • 2011-02-25 WO PCT/DE2011/000187 patent/WO2011103866A2/de active Application Filing
  • 2011-02-25 US US13/580,924 patent/US9324181B2/en not_active Expired - Fee Related
  • 2011-02-25 WO PCT/DE2011/000188 patent/WO2011103867A1/de active Application Filing
  • 2011-02-25 EP EP11720977A patent/EP2540089A2/de not_active Withdrawn
  • 2011-02-25 JP JP2012554215A patent/JP6142985B2/ja active Active
  • 2011-02-25 US US13/580,909 patent/US9396579B2/en not_active Expired - Fee Related
  • 2011-02-25 JP JP2012554214A patent/JP6060329B2/ja active Active
  • 2011-02-25 WO PCT/DE2011/000186 patent/WO2011103865A2/de active Application Filing
  • 2011-02-25 KR KR1020127024214A patent/KR101852209B1/ko active IP Right Grant
• 2016
  • 2016-03-23 US US15/078,596 patent/US10134180B2/en active Active
  • 2016-06-13 US US15/181,282 patent/US10229528B2/en active Active
  • 2016-10-24 JP JP2016208015A patent/JP6278323B2/ja not_active Expired - Fee Related

Also Published As

Similar Documents

Legal Events