WO2013143625A1 - Procédé anti-image fantôme utilisant la suppression binoculaire - Google Patents

Procédé anti-image fantôme utilisant la suppression binoculaire Download PDF

Info

Publication number
WO2013143625A1
WO2013143625A1 PCT/EP2012/073871 EP2012073871W WO2013143625A1 WO 2013143625 A1 WO2013143625 A1 WO 2013143625A1 EP 2012073871 W EP2012073871 W EP 2012073871W WO 2013143625 A1 WO2013143625 A1 WO 2013143625A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing
view
identified regions
image
alpha
Prior art date
Application number
PCT/EP2012/073871
Other languages
English (en)
Inventor
Laurent Blonde
Darya Khaustova
Cedric Thebault
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from EP12162641.0A external-priority patent/EP2600616A3/fr
Application filed by Thomson Licensing filed Critical Thomson Licensing
Priority to US14/360,648 priority Critical patent/US10063830B2/en
Priority to EP12791494.3A priority patent/EP2832099A1/fr
Publication of WO2013143625A1 publication Critical patent/WO2013143625A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues

Definitions

  • the present invention concerns an image processing method and more particularly an antighosting method for attenuating visibility of crosstalk effect using binocular suppression.
  • the invention concerns the 3D displays including 3D TV or 3D Cinema. It relates to content creation or adaptation for 3D Cinema or 3DTV.
  • crosstalk In the absence of a perfect 3D display technology, crosstalk has to be taken into account when studying the human visual perception of 3D stereoscopic content. To various degrees, all modern displays suffer from crosstalk, which can lead to a decrease of both video quality and visual comfort. In the worst case scenario, it can even prevent from proper binocular fusion.
  • the influence of crosstalk on human perception is a complex problem to address as it is dependent on different aspects, including viewing conditions, content and disparity. Further investigations are still required to understand the perception of crosstalk and develop crosstalk reduction mechanisms.
  • the problem to be solved by the invention is to attenuate visibility of crosstalk happening on 3D displays when one image channel is leaking in the other image channel creating an unintended ghost image on or around objects.
  • binocular fusion depends on the spatial frequency relationship between the two images. For fusion to occur, either the low or high frequency or both components must be identical. If the low or high frequency components differ, binocular rivalry will occur and the image that contains the high frequency information in a particular section will dominate the final perception. Therefore we will have suppression of different frequency bands.”
  • the subtraction balances the physically generated crosstalk, and then the crosstalk is hidden in the background raised level.
  • US201 1025832 - ghost-compensation for improved stereoscopic projection - applies the same principle, adding a dependence of ghost coefficients upon different image regions/zones.
  • US201 1025832 discloses a method and system for reducing ghost images in plano-stereoscopic image transmissions. The method comprises the steps of establishing a plurality of expected ghosting profiles associated with a plurality of predetermined regions on a screen, and compensating for leakage in each predetermined region of a projected left and right eye images by removing an amount of ghost images leaking from the projected left eye image into the projected right eye image and vice versa.
  • Figure 1 represents the algorithmic structure which illustrates this method.
  • the system comprises a processor configured to receive the quantity of ghost artifacts and compute ghost compensation quantities for left eye images and right eye images (bloc 504).
  • the processor is further configured to remove an amount of actual image ghost artifacts leaking from a projected left eye image into a projected right eye image and vice versa(bloc 506).
  • the processor is also configured to compute ghost compensation quantities for each of a plurality of zones, each zone corresponding to a region on a screen having an expected ghosting profile associated therewith.
  • the objective of the present invention is to improve antighosting even in the worst case when a white signal leaks on a black portion of an image, and the same 'antighosted' signal is expected to be applied to different stereoscopic display models.
  • the present invention provides a method of processing a source left view and a source right view of a 3D image.
  • the method comprises the steps of identifying at least one ghosted region or ghosting region in the left view and/or in the right view and processing the identified regions in order to create a binocular suppression effect for this identified regions then providing a new 3D image formed by the processed left and/or right views.
  • the step of processing the identified region comprises changing the contrast in order to generate binocular suppression in the identified regions.
  • the step of processing the identified regions consists in filtering the high spatial frequencies in the identified regions.
  • the step of processing the identified regions consists in blurring the identified regions.
  • the step of identifying at least one ghosted region or one ghosting region is performed using a ghost simulation.
  • the blurring of the identified regions is performed according to the following steps :
  • the obtained blended left/right view being the addition of an alpha part of the blurred left/right view and a (1 -alpha) part of the original left/right view .
  • the method comprises furthermore the step of modifying left and right alpha masks, in exchanging disparity information between the left and the right alpha masks taking account of the disparity card, such that the quality of the image in the crosstalk generating region is decreased and the quality of the image in the corresponding region is increased and the step of filtering for filtering left and/or right alpha masks belonging to the same object in the left and/or right view so that double processing is avoided .
  • the method is applied on the 3D content, at the post- production level.
  • the method is applied on the 3D content, as a set- top box or gateway embedded processing.
  • Figure 1 is a block diagram of a used algorithm structure as known from the state of the art represented by the document US201 1025832;
  • Figure 2 is a block diagram of an antighosting algorithm structure
  • Figure 3 is a block diagram of an antighosting algorithm structure comprising managing singularities
  • Figure 4 is a block diagram of an antighosting algorithm performance enhancement by local processing
  • Figure 5a-5d corresponds to a processing illustration.
  • the proposed solution does not entirely rely on physics as previous solutions but on some aspects of binocular perception.
  • crosstalk effect or ghosting on 3D displays is characterized by the visibility of an unintended image mixed partially with an intended image.
  • An example is the leakage of light intended to the right eye into the left eye vision channel of an observer.
  • the idea is not to suppress or cancel the unintended light signal emitted, but to modify the left and right images so that the observer does not see or perceives much less this unintended light signal.
  • Binocular Suppression resulting from a selection done by the visual system when two different images are presented to the eyes. While some configurations lead to binocular rivalry, many configurations simply lead to the suppression of one of the two available images (left or right view), the visual system taking care of the perception of depth coherence thanks to other cues than binocular disparity.
  • blurred patterns are suppressed by sharp ones because of both their lower contrast and their loss of high spatial frequencies. While eye dominance may have influence, only the most contrasted (left or right) view and the one with highest spatial frequency will be seen, while the other one is suppressed or is seen with much less contrast or not seen at all. While alteration of high spatial frequencies (blur) are one way to trigger binocular suppression, other image modifications can as well be used as average level dimming, color alteration (such as desaturation) or change of the local orientation of the image information.
  • the point is to force a binocular choice between the left and right view, with one of the two images of the pair being processed in order to be suppressed, the other image of the pair being processed to be retained in the percept, and avoiding too much difference in order not to trigger rivalry.
  • the alternation of the left and right percepts creates annoyance.
  • the solution proposed is to modify the image information in selected image regions of one of the two (left or right) views of a stereo pair when these regions are source of crosstalk for the other view.
  • This modification of the image information is made so that the modified selected regions become preferably suppressed by the binocular vision system and the corresponding, unmodified regions, in the other view become predominant.
  • the selected regions are blurred.
  • the corresponding image information via a disparity displacement in the other view of the stereo pair can be processed so that this other view is privileged in the binocular vision process and part of the percept.
  • the visual attention is thus reported in this privileged other view .
  • Binocular suppression is a common phenomenon, acting unconsciously in our daily life. For example when an object is close in our field of view and we look far, at a distant scene, parts on the left and on the right of the close object will dim or disappear (be suppressed) to privilege the distant scene information where our conscious attention is directed at. It corresponds to a 'see through' experience.
  • the expected advantage is a better quality of the perceived 3D images, with less annoyance created by ghosting.
  • a first advantage is to address ghosting configurations not addressable with current solutions such as light leaking on very dark parts of the images (physical crosstalk contrast is high in this configuration and cannot be corrected by subtracting light)
  • a second advantage is the better stability of the antighosting processing as a precise model of crosstalk is less needed than in current methods.
  • image regions creating ghosting which can be identified using a crosstalk simulator
  • image regions creating ghosting will appear modified on their right of left view for crosstalk generation regions, with an alteration of their image information, like blurring, average level, contrast change, orientation change or color change, and with possibly local reinforcement of image information for the other (left or right) view in the corresponding regions (i.e. corresponding via a disparity displacement).
  • the invention can take for example the form of a base algorithm, applied on the stereoscopic or more generally 3D content, at the post-production level or as a set-top box or gateway embedded processing.
  • Image processing consists first in a linearization step (101 ,102) to linearize the visual space into a linear space in order to realize subsequent computations in a linear space.
  • a processed version of the left and right image signals is performed (201 ,202).
  • Processing can be for example blurring, average level, contrast change, orientation change or color change or any change proper to generate binocular suppression.
  • the strength of processing, and the horizontal and vertical extent of processing are parameters of the algorithm. Successful tests were performed with a Gaussian filter of horizontal extent of 31 pixels and a vertical extent of 5. Vertical extent can be smaller than horizontal extent as crosstalk acts in the horizontal dimension.
  • the third step (301 ,302) is a performing step to perform a blending operation between the original -left or right- image signal and its corresponding processed left or right image signal.
  • the blending is performed using an alpha mask which will be determined by the following mask processing path, alpha (varying between 0.0 and 1 .0) being a coefficient determining the proportion of processed content for each pixel in the resulting image.
  • alpha varying between 0.0 and 1 .0
  • a weighted average is performed for each pixel of the left and right images.
  • the modified image signal being the addition of an alpha part of the processed image signal and a (1 - alpha) part of the original image signal:
  • a luminance attenuation coefficient may be applied on a local orientation modified signal, further increasing predominance of the other view. Then the formula becomes:
  • the final step in the image processing path (401 , 402) is to apply the adequate "gamma" transform from the linear space used for processing to the display space intended for the modified content thus obtaining left and right modified imges reducing the saliency of the crosstalk effect.
  • the goal of mask processing path is first to identify the pixels of crosstalk regions where processing should be applied and, in an example implementation, to compute a proportion corresponding to the coefficient alpha for blending locally the low-pass filtered image with the original image.
  • the mask is acting as an alpha channel in video processing or computer graphics.
  • the images shall be blurred in the crosstalk generating regions of a given view, so that, there, the other view is selected by the visual system.
  • coefficient alpha shall be high (close to 1 .0) in regions generating crosstalk, and low (close to 0.0) where no crosstalk is generated.
  • Mask transitions shall be smooth to avoid creating artificial edges due only to the blending process.
  • the first step of mask processing 501 , 502 is to calculate the ghost contribution from the left and right images. This is performed using a ghost simulation, as described in cited document US201 10025832 for example.
  • the system crosstalk contrast can be relevant information as a first step for initial mask generation.
  • Second and third steps are part of the "Calculate left-(or-right) alpha mask contribution" blocks 601 , 602 in Figure 2.
  • the image of mask and at the beginning a binary image (levels 0 or 255 only) where 255 represents zones, for a right or left view, which generate a ghost in the other view.
  • These binary images can present artifacts in the form of small spots for example. According to a preferred embodiment of the invention these artifacts should be treated.
  • the second step consists in processing spatially the initial mask.
  • the initial mask is then processed for example with mathematical morphology operations to remove small image elements.
  • This topological output will be still binary (0 or 255) if the structuring elements are binary, or in levels of gray (between 0 and 255) if the structuring elements are themselves in levels of gray.
  • the resulting mask can then be low-passfiltered to avoid the effects of edge, in particular in the binary case.
  • the third step consists in applying a look up table (LUT) on the obtained images signal to adapt the levels of the processed initial mask (between 0 and 255) to values of the parameter alpha in order to bring back the range of level 0 to 255 to a range of levels 0 to 1 that it should take to perform the intended blending.
  • LUT look up table
  • the LUT table is a table of correspondence between discrete levels of entry and discrete levels of exit.
  • a LUT 1 D mono dimensional, is used which transforms a range [0, 255] into another range [0, 1 ] by not reaching inevitably all the levels and by duplicating some.
  • This calculated alpha mask is then used in the image processing, as described above, to perform the blending operation.
  • Some stereoscopic singularities may appear concerning this algorithm. For example when a thin bright object is presented on a dark background, ghosting will appear on both sides of the object. When applying the algorithm above, both the left and right view of this object will be processed, and no original quality version will be available for the visual system to reconstruct a percept close to the original intended image.
  • Figure 3 indicates a processing structure (dashed lines blocks and arrows) to manage these singularities.
  • This processing structure will exploit as input a disparity image indicating the local displacement of objects in the image plane between the left and right view. This disparity can be available with- or computed from- the left and right eye images.
  • the main processing block 701 is a "disparity control filter" acting of the left and right alpha masks. This filter will perform two operations: Exclude from the left and right alpha mask regions belonging to the same object, to avoid double processing and a resulting blurred percept.
  • the filtered left and right alpha mask 702, 703 is further used in the image processing to perform the blending operation.
  • Creating "Singularities masks" for the left and right views allow a singularity processing step 706 to be performed for these areas after the blending operation is performed.
  • Singularity processing may take various forms. For example:
  • the base algorithm consists in decreasing the quality of a crosstalk generating region in one -left or right - view
  • the idea here is to reinforce or enhance quality in the corresponding region in the other view. In this way, the main percept will be reinforced and the balance between the intended image region and the unintended image region (the processed one) will be more in favor of the intended one in the binocular suppression mechanism.
  • step 801 to exchange the left and right alpha masks, taking into account disparity, and to apply a sharpening filter weighted by this modified alpha mask.
  • Figure 4 indicates a processing structure (dashed lines blocks) to manage these exchanges.
  • the exchange of masks should take disparity into account to relocate the alpha masks' areas on the corresponding regions of the objects in the other view (regions aligned then with the processed areas of the former view when watching 3D).
  • step 802 803 part of the original right or left image signal is thus reinforced by this processing generating view preference before being added to the corresponding processed image signal.
  • a modified version of the left and right images consists in modifying the color of the identified crosstalk regions of the left or right image by de-saturating the color of the identified crosstalk regions and compensating for this color de-saturation in the homologue region of the other -right or left- view.
  • the singularities processing could also take into account the user's dominant eye.
  • FIG. 5a-5d illustrates the processing:
  • Figure 5a illustrates the left and right original views.
  • Figure 5b illustrates the left and right views with crosstalk (simulation) as regions from the other view generate ghosts.
  • arrows shows crosstalk generated by right view regions and on the right view, arrows shows crosstalk generated by left view regions.
  • Figure 5c illustrates the region generating crosstalk which is processed (blurred in this example).
  • Figure 5d illustrates the difference between processed views and original views indicating the improvements due to the process. High frequencies that have been removed by the blurring process are visible on these difference images. As the ghosting attenuation is perceptual it cannot be rendered in this document but only on a 3D display exhibiting crosstalk.
  • the percept is: when a left image region is processed (here blurred), only the corresponding right image region is perceived and vice-versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

La présente invention concerne un procédé de traitement d'une vue de gauche source et d'une vue de droite source d'une image en 3D comprenant les étapes consistant à : identifier au moins une région avec image fantôme ou une région d'image fantôme dans la vue de gauche et/ou dans la vue de droite; traiter les régions identifiées de manière à créer un effet de suppression binoculaire pour ces régions identifiées et fournir une nouvelle image 3D formée par les vues de gauche et de droite traitées.
PCT/EP2012/073871 2011-11-30 2012-11-28 Procédé anti-image fantôme utilisant la suppression binoculaire WO2013143625A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US14/360,648 US10063830B2 (en) 2011-11-30 2012-11-28 Antighosting method using binocular suppression
EP12791494.3A EP2832099A1 (fr) 2012-03-30 2012-11-28 Procédé anti-image fantôme utilisant la suppression binoculaire

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP12162641.0A EP2600616A3 (fr) 2011-11-30 2012-03-30 Procédé « antighosting » utilisant la suppression binoculaire
EP12162641.0 2012-03-30

Publications (1)

Publication Number Publication Date
WO2013143625A1 true WO2013143625A1 (fr) 2013-10-03

Family

ID=47227815

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2012/073871 WO2013143625A1 (fr) 2011-11-30 2012-11-28 Procédé anti-image fantôme utilisant la suppression binoculaire

Country Status (2)

Country Link
EP (1) EP2832099A1 (fr)
WO (1) WO2013143625A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115209122A (zh) * 2022-07-26 2022-10-18 福州大学 一种基于多智能体的立体图像视觉舒适度增强方法及系统

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060268104A1 (en) * 2005-05-26 2006-11-30 Real D Ghost-compensation for improved stereoscopic projection
WO2010019922A1 (fr) 2008-08-15 2010-02-18 Real D Compensation d'images fantômes améliorée pour imagerie stéréoscopique

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060268104A1 (en) * 2005-05-26 2006-11-30 Real D Ghost-compensation for improved stereoscopic projection
US20110025832A1 (en) 2005-05-26 2011-02-03 Reald Inc. Ghost-compensation for improved stereoscopic projection
WO2010019922A1 (fr) 2008-08-15 2010-02-18 Real D Compensation d'images fantômes améliorée pour imagerie stéréoscopique

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"Reducing Crosstalk between Stereoscopic Views", PROCEEDINGS OF SPIE, vol. 2177, pages 92 - 96
BULBUL A ET AL: "A perceptual approach for stereoscopic rendering optimization", COMPUTERS AND GRAPHICS, ELSEVIER, GB, vol. 34, no. 2, 1 April 2010 (2010-04-01), pages 145 - 157, XP026981665, ISSN: 0097-8493, [retrieved on 20091203] *
See also references of EP2832099A1
WINKLER S ET AL: "Stereoscopic image quality compendium", INFORMATION, COMMUNICATIONS AND SIGNAL PROCESSING (ICICS) 2011 8TH INTERNATIONAL CONFERENCE ON, IEEE, 13 December 2011 (2011-12-13), pages 1 - 5, XP032178524, ISBN: 978-1-4577-0029-3, DOI: 10.1109/ICICS.2011.6173571 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115209122A (zh) * 2022-07-26 2022-10-18 福州大学 一种基于多智能体的立体图像视觉舒适度增强方法及系统
CN115209122B (zh) * 2022-07-26 2023-07-07 福州大学 一种基于多智能体的立体图像视觉舒适度增强方法及系统

Also Published As

Publication number Publication date
EP2832099A1 (fr) 2015-02-04

Similar Documents

Publication Publication Date Title
Seuntiëns et al. Perceptual attributes of crosstalk in 3D images
US8625881B2 (en) Enhanced ghost compensation for stereoscopic imagery
JP5615136B2 (ja) 立体画像補正方法、立体表示装置、および立体画像生成装置
EP2259601B1 (fr) Procédé de traitement d'image, dispositif de traitement d'image, et support d'enregistrement
US10063830B2 (en) Antighosting method using binocular suppression
US20110074933A1 (en) Reduction of viewer discomfort for stereoscopic images
CA2727218A1 (fr) Procedes et systemes destines a reduire ou a eliminer les images fantomes percues sur les images stereoscopiques affichees
WO2013054240A1 (fr) Traitement d'une carte de profondeur
WO2012033224A1 (fr) Système de réduction de diaphonie
JP2015162718A (ja) 画像処理方法、画像処理装置及び電子機器
KR20110047137A (ko) 화상처리장치, 화상처리방법, 및 기억매체
Sohn et al. Crosstalk reduction in stereoscopic 3D displays: Disparity adjustment using crosstalk visibility index for crosstalk cancellation
Hong Reduction of spatially non-uniform 3D crosstalk for stereoscopic display using shutter glasses
WO2013143625A1 (fr) Procédé anti-image fantôme utilisant la suppression binoculaire
WO2016063450A1 (fr) Appareil de traitement d'image, procédé de commande associé, appareil d'affichage d'image et support de stockage lisible par ordinateur
Schwarz et al. On predicting visual popping in dynamic scenes
US8786598B2 (en) Pixel-intensity modulation technique for frame-sequential stereo-3D displays
Barkowsky et al. Crosstalk measurement and mitigation for autostereoscopic displays
WO2022091800A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations, et programme
Wolski et al. Dark stereo: improving depth perception under low luminance
WO2013037040A1 (fr) Procédé et appareil de fourniture d'améliorations vidéo pour afficher des images
Lee et al. Crosstalk reduction method in a glasses-free AR 3D HUD
Atkins Advanced methods for controlling dual modulation display systems
Chang et al. A 3D Index $(\Delta {\rm X} _ {{\rm 3D}}) $ for Evaluating Image Quality of Stereoscopic Displays
Andrade Sharpness Enhancement of Stereo Images Using a Depth-Based Per-Pixel Regularization

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12791494

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 14360648

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2012791494

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE