Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/20—Image signal generators
  • H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis

Definitions

• the invention relates to a method and apparatus for generating three-dimensional (3D) images from a sequence of two-dimensional images.
• a particular disadvantage of the image sequential transmission in connection with conventional television systems is that the image refresh rate for each eye is reduced to 25 images per second. This results in an unpleasant flicker for the viewer. Although this restriction does not occur in the parallel transmission of the image sequences via their own (left or right) channel. Problems can arise here, however, in the synchronization of the two channels and by the requirements of the receiver, which must receive and process two separate channels simultaneously. This is not possible with customary systems.
• each image is decomposed into individual pixels, which are transmitted digitized.
• each image is decomposed into individual pixels, which are transmitted digitized.
• US Pat. No. 6,108,005 discloses a method for producing synthesized stereo images in which at least two images are generated from an input image, wherein at least one of the images is enlarged, reduced, rotated, shifted or in one relative to the supplied image is changed in such a way that at least parts of the image are shifted relative to other parts of the image compared to corresponding parts in another image.
• the disadvantage here is that it depends largely on the skill of the operator, whether a correct or natural stereoscopic image impression is generated by the viewer by a suitable selection of the above changes.
• the invention has for its object to provide a method and an apparatus of the type mentioned, with the / substantially without intervention by an operator or a viewer, a generation of 3D images with a particularly natural three-dimensional image impression is possible.
• Fig. 1 is a schematic block diagram of a circuit arrangement according to the invention.
• FIG. 2 shows a schematic representation for explaining a deformation by means of ball projection
• FIG. and FIG. 3 shows a flow chart of the method according to the invention.
• the essential components of a device according to the invention and their interconnections are shown schematically in FIG.
• the arrangement comprises an input E, via which the two-dimensional images recorded and digitized by a camera are fed to a first image memory 1 for the intermediate storage of at least one respective current image.
• the supplied images are transferred to a second image memory 2 connected thereto, which is provided for storing a predeterminable number of successive images and for their interpolation.
• a scene analysis device 3 is also connected to the first image memory 1, with which the current image stored in the first image memory 1 is examined with regard to its content in order to match a specific scene type such as "close-up”, “normal recording” (US Pat. Medium recording) or "wide-angle recording” assigned.
• the scene analysis device 3 is connected to an image deformation device 4, with which an image supplied from the first image memory 1 is subjected to image deformation associated with this type in accordance with the scene type detected by the device 3.
• the second image memory 2 is also connected to the device 4, so that an image generated by interpolation of previous images can also be deformed.
• Various patterns for such image deformations and their assignment to at least one scene type are stored in an image deformation memory 5, from which the patterns can be retrieved by the image deformation means 4.
• a phase switch 6 is connected to an output of the device 4, to which the undeformed image from the first image memory 1 and the deformed image generated therefrom with the device 4 can be transmitted.
• These images are then applied to a first and second output AI, A2 of the phase switch 6 and respectively form a first and second sequence of images, which are supplied to a left and right viewing channel BL, BR for a left or right stereo image.
• the one image sequence is thus composed by the unchanged, supplied images and the other image sequence by images produced therefrom which are subjected to deformation (asymmetrical deformation).
• deformation asymmetrical deformation
• a further possibility consists in additionally or alternatively supplying an image, which is interpolated in the second image memory 2, to the device 4 and from this - in a deformed and / or non-deformed form - composing the first and / or the second image sequence.
• the interpolated image is calculated for example by linear spline approximation or a higher-grade or polynomial approximation of all pixels, where ⁇ is a Approximation variable is and refers to the time interval from a current image in which a synthetic (interpolated) image is generated.
• ⁇ is a Approximation variable is and refers to the time interval from a current image in which a synthetic (interpolated) image is generated.
• a first and a second image sequence can thus be generated from a sequence of two-dimensionally recorded and digitized images at the input E, which together allow a three-dimensional view of the originally two-dimensional images if the first and second image sequences belong to a left or right Eye is fed.
• the image deformation can be selected and adjusted according to the image content (scene type) and how the transition between different image deformations is preferably made by scene analysis in real time so that no disturbing transition effects occur.
• the pixels of the new image are continuously stretched horizontally increasingly from top to bottom according to the following formulas:
• tL the number of lines
• PpL the number of pixels per line
• Tiilt is any scaling constant that determines the amount of strain.
• the pixels of the new image are distorted concentrically from the center of the image to the edge according to the following formulas:
• tL is the number of lines
• PpL is the number of pixels per line
• Sphere is any scaling constant that determines the extent of the distortion.
• the pixels of the new image are simultaneously distorted and stretched from top to bottom and concentrically from the image center according to the following formulas:
• Undex (i, j): ((0.5 PpL - j) / 0.5 PpL) (1 - (4 / tL 2 ) (0.5 tL - i) 2 ) •
• Sphere jjtadex (i, j) : ((0.5 tL - i) / 0.5 tL) (1 - (4 / PpL 2 ) (0.5 Ppl - j) 2 ) • Sphere
• tL is the number of lines
• PpL is the number of pixels per line
• Sphere is any scaling constant that determines the extent of the distortion
• Tit is any scaling constant that determines the extent of the strain.
• the second method works with a symmetrical image deformation, in which additionally the actual original image is deformed, that means geometrically distorted. It represents, in a generalized form according to FIG. 2, an image of the actual pixels 0 to PpL of an image plane B on a curved surface F (imaging surface), this image being spaced at distance D from two perspectives for the left and the right.
• te eye AI, A2 is considered. Starting from the viewer, the pixels (for example z (j) or the region M) on the imaging surface F in for the two eyes AI, A2 different way (and x M 'for AI or j "and x M " for A2) projected back onto the image plane B. This gives the brain the impression of looking at the images from two angles.
• the image surface represents an outwardly curved spherical surface.
• a "synthetic" pixel z (i, j) results on a spherical surface curved toward the observer:
• tL is the number of lines
• PpL is the number of pixels per line
• Sphere is any scaling constant that determines the amount of distortion.
• a j-index for a left-hand observer position Ej is mapped to:
• the image surface represents an outwardly curved cylinder surface.
• a "synthetic" image point z (i, j) results on a cylinder surface curved toward the observer:
• PpL is the number of pixels per line
• Sphere is any scaling constant that determines the extent of the distortion.
• three different scene types are defined for this purpose, to which the image is examined. Basically, however, a larger number of scene types can be defined.
• the scene types described here by way of example are the close-up N, the wide-angle shot W and the medium shot (normal shot) M.
• the wide-angle shot is often landscape.
• the best three-dimensional effect is generally achieved with a tilt deformation.
• the Sphere Tilt Deformation generally produces the best three-dimensional impression.
• x N be a rectangular partial image of the current image in the center of the image For example, 60 percent of all pixels of the overall picture XQ.
• ⁇ N 2 : ⁇ (xy - x N ) 2 over i, jex N
• M be a rectangular partial image of the current image in the center of the image with, for example, 40 percent of all pixels of the overall image XQ.
• ⁇ M 2 : ⁇ (xy - X " M) 2 over i, jex M
• a transitional deformation for each transition from an "old" deformation to another "new” deformation, a transitional deformation is defined, which for example is also stored in the image deformation memory 5.
• Such a transitional deformation is formed by a predetermined number K of transition matrices, the values of which are calculated and stored by preferably linear interpolation of the shift values stored for the old and the new deformation for each pixel.
• the transmitted image whose scene type has changed is subjected to a transient function consisting of the transitional deformation defined by the number K of transition matrices and the subsequent new deformation corresponding to the new one determined Scene type is assigned.
• the further results of the scene analysis supplied in this case remain unchanged during the application of the transition function. considered.
• the image currently being transferred with the first transition matrix and then the next image with the second transition matrix, which together form the transitional deformation, are to be applied.
• all transition matrices can be calculated in advance and stored in the image deformation memory 5.
• transition matrices stored for a transition from a first to a second deformation are applied to the transferred image in reverse order in the case of a transition from the second to the first deformation.
• FIG. 3 shows a flow chart of the method according to the invention.
• first in the Image deformation means 4 sets a first status "Current Deformation” to a start deformation as an applied deformation, which is, for example, the cylinder deformation.
• a second status "new deformation” is then set to a standard or default deformation, for example also the cylinder deformation, and then the scene type device 3 is used to set the scene type of the current (supplied ) Image as described above.
• step 16 If the query in the seventh step 16 has also been answered with No, the process continues to the ninth step 18, with which it is queried whether the deformations set with the first and second statuses are the same.
• the current image is subjected to (unaltered) image deformation by the image deformation means 4 and output as an image of the second image sequence.
• the procedure is then with the second step 11 for a next picture repeated.
• the twelfth step 21 is repeated and the now current image of the image memory 1 is deformed, now with the second (next) transition matrix, in order then to be output as the next image of the (second) image sequence.
• the current image of the new image deformation set in accordance with steps 13, 15 or 17 is subjected to completion of the transition deformation, and the count is again increased by the value 1.
• the query following the thirteenth step 22 must then be answered with Yes, so that a fourteenth step 23 is continued, with which the first status "Current Deformation" is set to the new deformation. Subsequently, the process is repeated by returning to the second step 11 with a next input image.
• the apparatus shown in Figure 1 is preferably implemented in a digital image processing system for producing a three-dimensional rendering of two-dimensionally transmitted or stored images.
• the described methods are preferably implemented in the form of one or more computer programs with program code means for carrying out the individual method steps with a computer, in particular a microprocessor unit.
• the methods may also be implemented as a computer program product having program code stored on a machine-readable carrier for carrying out the steps of the methods as it is loaded into the memory of a programmable microprocessor unit that is part of a digital image processing system is.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Computer Graphics (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
• Image Processing (AREA)
• Processing Or Creating Images (AREA)
• Studio Circuits (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=8164692

Family Applications (1)

Country Status (8)

Families Citing this family (49)

Family Cites Families (32)

• 2001
  • 2001-11-24 KR KR10-2004-7007885A patent/KR20040071145A/ko not_active Application Discontinuation
  • 2001-11-24 AU AU2002233198A patent/AU2002233198A1/en not_active Abandoned
  • 2001-11-24 JP JP2003548184A patent/JP5020458B2/ja not_active Expired - Fee Related
  • 2001-11-24 WO PCT/EP2001/013674 patent/WO2003046832A1/de active Application Filing
  • 2001-11-24 EP EP01984759A patent/EP1451775A1/de not_active Withdrawn
  • 2001-11-24 CA CA002472272A patent/CA2472272A1/en not_active Abandoned
  • 2001-11-24 CN CNB018239390A patent/CN1295655C/zh not_active Expired - Fee Related
• 2003
  • 2003-05-29 US US10/447,463 patent/US7321374B2/en not_active Expired - Fee Related
• 2007
  • 2007-12-13 US US11/956,089 patent/US20080106546A1/en not_active Abandoned

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events