WO2013028201A1 - Model-based stereoscopic and multiview cross-talk reduction - Google Patents

Model-based stereoscopic and multiview cross-talk reduction Download PDF

Info

Publication number
WO2013028201A1
WO2013028201A1 PCT/US2011/049176 US2011049176W WO2013028201A1 WO 2013028201 A1 WO2013028201 A1 WO 2013028201A1 US 2011049176 W US2011049176 W US 2011049176W WO 2013028201 A1 WO2013028201 A1 WO 2013028201A1
Authority
WO
WIPO (PCT)
Prior art keywords
signals
cross
talk
visual
display
Prior art date
Application number
PCT/US2011/049176
Other languages
French (fr)
Inventor
Ramin Samadani
Nelson Liang An Chang
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to KR1020147004419A priority Critical patent/KR101574914B1/en
Priority to JP2014527133A priority patent/JP5859654B2/en
Priority to PCT/US2011/049176 priority patent/WO2013028201A1/en
Priority to US14/237,439 priority patent/US20140192170A1/en
Priority to EP11871208.2A priority patent/EP2749033A4/en
Publication of WO2013028201A1 publication Critical patent/WO2013028201A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/327—Calibration thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/349—Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking

Definitions

  • ⁇ 00011 Stereoscopic and muU ew displays have emerged to provide viewers a more accurate visual reproduction of three-dimensional ("3D") real-world scenes.
  • Such displays may require the use of active glasses, passive glasses or autostereoscopic lenticular arrays to enable viewers to experience a 3D effect from multiple viewpoints.
  • stereoscopic displays direct a separate image view to the left and to the right eye of a viewer. The viewer's brain then compares the different views and creates what the viewer sees as a single 3D image.
  • FIG. 1 illustrates a schematic diagram of an example 3D display system with cross-talk
  • FIG. 2 illustrates a schematic diagram of a system for characterizing and correcting for cross-talk signals in a 3D display
  • FIG. 3 illustrates an example cross-talk reduction module of FIG. 2 in more detail
  • FIG. 4 is a flowchart for reducing and correcting for cross-talk in a 3D display using the cross-talk reduction module of FIG, 3 in accordance with various embodiments;
  • FIG. 5 is a schematic diagram of a forward transformation model for use with the cross-talk reduc tion module of FIG. 3;
  • FIG, 6 illustrates example test signals that m y be used to generate the forward transformation model of FIG. 5.
  • a model-based cross-talk reductio system and method for use with stereoscopic and multiview 3D displays are disclosed.
  • crosstalk occurs when an image signal or view intended for one viewer's eye appears as an unintended signal superimposed to an image signal intended for the other eye.
  • the unintended signal is referred to herein as a cross-talk signal.
  • cross-taik signals that appear in a 3D display are reduced and corrected for by using a forward transformation model and a visual model
  • the forward transformation model characterizes the optical, photometric, and geometric aspects of cross-talk signals that arise when, image signals are input into the display.
  • the visual model takes into account salient visual effects involving spatial discrimination, color, and temporal discrimination s that visual fidelity to the original image signals thai are input into the display is maintained, A non-linear optimization is applied to the input signals to reduce or completely eliminate the cross-talk signals.
  • the 3D display system 100 has a 3D display screen 105 that may be a stereoscopic or multiview display screen, such as, for example, a parallax display, a lenticular-based display, a holographic display, a projector-based display, a light ieid display, and so on.
  • An image acquisition module 110 may have one or more cameras (not shown) to capture multiple image views or si nals for display in the display screen 105.
  • two image views may be captured, one for the viewer ' s left eye 1 15 (a left image "L” 125 ⁇ and another for the viewer's right eye 120 (a right image "R.” 130).
  • the captured images 125-130 are displayed on the display screen 105 and perceived as image 135 in the viewer's left eye 1.15 and image 40 in the viewer's right eye 120.
  • the image acquisition module 1 10 may refer simply to computer generated 3D or mu view graphical information.
  • the images 135- 140 are superimposed with, cross-talk signals.
  • the image 135 fo the viewer's left eye 115 is superimposed with a cross-talk signal 145 and the image 140 for the viewer's right eye 120 is superimposed with a cross-talk signal 150.
  • the presence of the cross-talk signals 145 and 150 in the images perceived by the viewer affect the overall quality of the images, it is also appreciated that unlike ghosting or other subjective visible artifacts, the cross-talk signals are a physical entity and can be objectively measured, characterized, and corrected for.
  • the 3D display system 200 has an image acquisition module 205 for capturing multiple image views or signals for display in the 3D display screen 2.10, such as, for example, a left image "L" 2.15 and a right image “R" 220,
  • a cross-talk reduction module 225 takes the images 215-220 and applies a model-based approach to reduce and correct for cross-talk introduced by the 3D display screen 210.
  • the cross-talk reduction module 225 modifies the images 215-220 into images 230-235 that are then input into the display screen 210.
  • images 240-245 are perceived by the viewer's eyes 250-255 with significantly reduced or non-existent cross-talk.
  • the cross-talk reduction module 225 and the 3D display screen 10 may be implemented in separate devices (as depicted) or integrated into a single device.
  • FIG, 3 illustrates an example cross-talk reduction module of FIG. 2 in more detail.
  • the cross-talk reduction module 300 has a forward transformation model 305, a visual model 3.1 and a cross-talk correction module 315 to reduce and correct for cross-talk signals destined to a 3D display.
  • the cross-talk reductioii module 300 characterizes the cross-taik introduced by the 3D display and generates corresponding cross-taik corrected images, such as a left cross-talk corrected image "Lee' " 355 and a right cross-taik corrected image " c" 360.
  • the forward transformation model 305 characterizes die optical, photometric, and geometric aspeci of direct and cross-talk signals that are introduced by the 3D display. That is, the forward transformation model 305 estimates or models the direct and cross-talk signals by characterizing the forward transformation from image acquisition (e.g., image acquisition module 205) to 3D display (e.g., 3D display 210). This is done by .measuring output signals generated by the 3D display when, using test, signals as an input. As appreciated by one skilled in the art, the forward transformation model 305 can be represented by a mathematical function F(.).
  • test signals may include both left and right test signals jointly, or individual left and right, test signals, in the first case, test image signals l? and fir ar jointly sent to the 3D display to generate left and right output signals, referred to herein as Lp and and estimate the parameters of the forward transformation function F(.). That is:
  • F / represents the forward model used to characterize the left output signal . . ⁇
  • ⁇ R represents the forward model used to characterize the right output signal R?.
  • test image signals Lr and I are separately sent to the 3D display to generate left and right output signals that are measured.
  • Thai is:
  • the l. i and R n signals are the desired output signals at each eye in the absence of cross-talk, while the Rci and L C R signals represent the cross-talk that leaks to the other eye.
  • Ha represents the cross-talk seen at the right eye when only the left image signal is sent to the display
  • /, ⁇ 3 ⁇ 4 represents the cross-talk seen a the left eye when only the right image signal is sent to the display.
  • an additive or other such model may be used to combine the measured responses for each eye, tha t is, to combine the /. ⁇ 3 ⁇ 4 and responses for the left eye into a combined signal LD and to combine the i1 ⁇ 2, and 1 ⁇ 2 ? responses for the right eye into a combined signal 3 ⁇ 4.
  • the combined responses La and ⁇ » may then used to estimate the parameters of the forward transformation function (.), Note thai this transformation function is display-dependent, as its parameters vary depending on the particular 3D display being used (e.g., a lenticular array display, a stereoscopic active glasses display, a light field display., and so on).
  • input image signals e.g., L 320 and R 325
  • cross-corrected image signals e.g., Lee 355 and ⁇ ;e 360
  • L 320 and R 325 input signals are applied to the forward transformation model 305 to characterize the cross-talk introduced by the 3D display with modeled cross-talk output signals L? and and desired signals Lm. and Rm-
  • the visual model 310 determines a visual measure representing how the visual quality of signals displayed in the 3D display is affected by its cross-talk, in one example, the visual mode!
  • the visual model 10 may be any visual model for computing such a visual differences measure.
  • the cross-correction module 315 uses this measure v to modify the input image signals L 320 and R 325 to generate visually modified input signals LM 345 and RM 350. In one embodiment, this is done by varying visual parameters or characteristics such as contrast, brightness, and color of the input signals to generate the visually modified input signals as canonical transformations of the input signals.
  • the visually modified input signals LM 345 and A'.y 350 are then sent as inputs to the forward transformation model 305 to update the visual measure v and determine whether the modifications to the input signals reduced the cross-talk (the smaller the value of v, the lower the cross-talk). This process is repeated until the cross-talk is significantly reduced or completely eliminated, i.e., until it is visually reduced to a viewer. That is, nonlinear optimization is performed to iterate through values of v until v is minimized and the cross-talk is significantly reduced or completely eliminated in output signals Ice 355 and Rcc 360. it is appreciated that the output signals £ «' 355 and R c c 360 ar the same as the visually modified signals La 345 and 3 ⁇ 4 350 when the visual measure v is at its minimum.
  • FIG. 3 It is also appreciated that the various left and right image signals illustrated in FIG. 3 (e.g., inputs L 320 and R 325, outputs /, ⁇ % 355 and Rcc 360) are shown for illustration purposes only. Multiple image views may be input into the cross-talk reduction module 300 (such as, for example, the multiple image views in a mdtiview display) to generate corresponding cross-talk corrected outputs. That is, the cross-talk reduction module 300 may be implemented for any type of 3D display regardless of the number of views it supports.
  • FIG. 4 shows a flowchart, for reducing and correcting for cross-talk in a 3D display using the cross-talk reduction module of FIG. 3 in accordance with various embodiments.
  • the cross-talk introduced in the 3D display is characterized with a plurality of test signals to generate a forward transformation model (400).
  • image signals are input into the model to generate modeled signals (405),
  • modeled signals may he, for example, the /,/- and Rf- i d Lp and 3 ⁇ 4 signals described above,
  • the modeled signals are applied to the visual model to compute a visual measure indicating how the visual quality of signals displayed in the 3D display is affected by its cross-talk (410).
  • the input signals are then modified based on the visual measure (415) and re-applied to the forward transformation model until the visual measure is minimized (420).
  • the modified, cross-talk corrected signals are sent to the 3D display for display (425).
  • the cross-talk corrected signals ate such that crosstalk is visually reduced to a viewer.
  • the modified, cross-talk corrected signals can be saved for later display,
  • the forward transformation model 500 has four main transformations to characterise the photometric. geometric, and optical factors represented in the forward transformation function £ ' (.): 0) a space-varying offset and gain transformation 505; (2) a color correction transformation 510; (3) a geometric correction transformation 555; and (4) a space varying blur transformation 520. Test signals including color patches, grid patterns, horizontal and verticai stripes, and uniform white, black and gray level signals are sent to a 3D display in a dark room to estimate the parameters of FQ,
  • the color correction transforma ion 510 is determined next by fitting between measured, colors and color values. Measured average color values for gray input patches are used to determine one-dimensional look-up tables applied to input color components, and measured average color values for primary R, G, and B inputs are used to determine a color mixing matrix using the known input color values. Computing the fits using the spatially renormaiized colors allows the color correction transformation 510 to fit the data using a small number of parameters,
  • the geometric correction 515 may be determined using, for example, a polynomial mesh transformation model.
  • the .final space-varying blur transformation 520 is required to obtain good results at the edges of the modeled signals. If the blur is not applied, objectionable halo artifacts may remain visible in the modeled signal
  • the parameters of the space-varying blur may be determined by estimating separate blur kernels in the horizontal and vertical directions, it is appreciated, that additional transformations may be used to generate the forward transformation model 500.
  • FIG. 6 illustrates example test signals that may be used to generate the forward transformation model of FIG. 5.
  • Test signal 600 represents a color patch ha ving multiple color squares, such as square 605, and is used for the color correction 510.
  • Test signal 610 is a checkerboard used for the geometric correction 515, and the white and black test signals 615-620 are used for the space- varying gain and offset transfonnation 505.
  • the test signals 625-630 contain horizontal and verticai lines to determine the space-varying blur parameters.
  • test signals may be used to generate the forward transfonnation model described herein. It is also appreciated that the care taken, in including various transformations to generate the forward transformation model enables the cross-talk reduction module of FIG, 3 to reduce and correct for cross-talk io any type of 3D display and for a wide range of input signals, while improving the visual quality of the displayed signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)

Abstract

A method for reducing cross-talk in a 3D display is disclosed. The cross-talk in the 3D display is characterized with a plurality of test signals to generate a forward transformation model. Input image signals are applied to the forward transformation model to generate modeled signals. The modeled signals are applied to a visual model to generate a visual measure. The input signals are modified based on the visual measure.

Description

MODEL-BASED STEREOSCOPIC AND Mli LTIVJEW CROSS-TALK REDUCTION
BACKGROUND
{00011 Stereoscopic and muU ew displays have emerged to provide viewers a more accurate visual reproduction of three-dimensional ("3D") real-world scenes. Such displays may require the use of active glasses, passive glasses or autostereoscopic lenticular arrays to enable viewers to experience a 3D effect from multiple viewpoints. For example, stereoscopic displays direct a separate image view to the left and to the right eye of a viewer. The viewer's brain then compares the different views and creates what the viewer sees as a single 3D image.
(ΘΘΘ2) One significant challenge thai arises in 3D displays is cross-talk between the image views. That is, part of the image views intended for one eye bleeds or leaks through to the other eye resulting in undesired cross-talk signals. These cross-talk signals are superimposed to the image views thereby diminishing the overall quality of the 3D image. There have been various approaches to reduce and correct for cross-talk in 3D displays, but they tend to he limited to a specific type of content (e.g., graphics imagery), to a specific type of 3D display (e.g., those requiring active glasses), or to a small number of views (e.g., two views in case of stereo), in addition to being expensive to implement in hardware or in physics-based approaches.
BRIEF DESCRIPTIO OF THE DRAWINGS
(0ΘΘ3) The present application may be more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which;
|0004 FIG. 1 illustrates a schematic diagram of an example 3D display system with cross-talk;
(0005 j FIG. 2 illustrates a schematic diagram of a system for characterizing and correcting for cross-talk signals in a 3D display;
(0006] FIG. 3 illustrates an example cross-talk reduction module of FIG. 2 in more detail; (0ΘΘ7) FIG. 4 is a flowchart for reducing and correcting for cross-talk in a 3D display using the cross-talk reduction module of FIG, 3 in accordance with various embodiments;
10008] FIG. 5 is a schematic diagram of a forward transformation model for use with the cross-talk reduc tion module of FIG. 3; and
f000 | FIG, 6 illustrates example test signals that m y be used to generate the forward transformation model of FIG. 5.
DETAILED DESCRIPTION
[0010] A model-based cross-talk reductio system and method for use with stereoscopic and multiview 3D displays are disclosed. As generally described herein, crosstalk occurs when an image signal or view intended for one viewer's eye appears as an unintended signal superimposed to an image signal intended for the other eye. The unintended signal is referred to herein as a cross-talk signal.
[001 1 j In various embodiments, cross-taik signals that appear in a 3D display are reduced and corrected for by using a forward transformation model and a visual model The forward transformation model characterizes the optical, photometric, and geometric aspects of cross-talk signals that arise when, image signals are input into the display. The visual model takes into account salient visual effects involving spatial discrimination, color, and temporal discrimination s that visual fidelity to the original image signals thai are input into the display is maintained, A non-linear optimization is applied to the input signals to reduce or completely eliminate the cross-talk signals.
|0θϊ2) It is appreciated that, in the following description, numerous specific details are set forth to provide a thorough understanding of the embodiments. However, it is appreciated that the embodiments may be practiced without limitation to these specific details. In other instances, well known methods and structures may not he described in detail to avoid unnecessarily obscuring the description of the embodiments. Also, the embodiments may be used in combination with each other.
[00.13] Referring now to FIG. Ϊ , a schematic diagram of an example 3D display system with cross-talk is described. The 3D display system 100 has a 3D display screen 105 that may be a stereoscopic or multiview display screen, such as, for example, a parallax display, a lenticular-based display, a holographic display, a projector-based display, a light ieid display, and so on. An image acquisition module 110 may have one or more cameras (not shown) to capture multiple image views or si nals for display in the display screen 105. For example, in case of a stereoscopic display, two image views may be captured, one for the viewer's left eye 1 15 (a left image "L" 125} and another for the viewer's right eye 120 (a right image "R." 130). The captured images 125-130 are displayed on the display screen 105 and perceived as image 135 in the viewer's left eye 1.15 and image 40 in the viewer's right eye 120. Alternately, the image acquisition module 1 10 may refer simply to computer generated 3D or mu view graphical information.
(0014] As a result of cross-talk generated by the display screen 1.05, the images 135- 140 are superimposed with, cross-talk signals. The image 135 fo the viewer's left eye 115 is superimposed with a cross-talk signal 145 and the image 140 for the viewer's right eye 120 is superimposed with a cross-talk signal 150. As appreciated by one skilled in the art, the presence of the cross-talk signals 145 and 150 in the images perceived by the viewer affect the overall quality of the images, it is also appreciated that unlike ghosting or other subjective visible artifacts, the cross-talk signals are a physical entity and can be objectively measured, characterized, and corrected for.
(003.5) Referring now to FIG. 2, a schematic diagram of a system for characterizing and correcting for cross-talk signals in a. 3D display is described. The 3D display system 200 has an image acquisition module 205 for capturing multiple image views or signals for display in the 3D display screen 2.10, such as, for example, a left image "L" 2.15 and a right image "R" 220, A cross-talk reduction module 225 takes the images 215-220 and applies a model-based approach to reduce and correct for cross-talk introduced by the 3D display screen 210. The cross-talk reduction module 225 modifies the images 215-220 into images 230-235 that are then input into the display screen 210. As a result, images 240-245 are perceived by the viewer's eyes 250-255 with significantly reduced or non-existent cross-talk. It is appreciated by one skilled in the art that the cross-talk reduction module 225 and the 3D display screen 10 may be implemented in separate devices (as depicted) or integrated into a single device.
(0016) FIG, 3 illustrates an example cross-talk reduction module of FIG. 2 in more detail. The cross-talk reduction module 300 has a forward transformation model 305, a visual model 3.1 and a cross-talk correction module 315 to reduce and correct for cross-talk signals destined to a 3D display. Given multiple image views or signals to be displayed in the 3D display, such as, for example, a left image signal "L" 320 and a right image signal "R" 325, the cross-talk reductioii module 300 characterizes the cross-taik introduced by the 3D display and generates corresponding cross-taik corrected images, such as a left cross-talk corrected image "Lee'" 355 and a right cross-taik corrected image " c" 360.
|(K)17| The forward transformation model 305 characterizes die optical, photometric, and geometric aspeci of direct and cross-talk signals that are introduced by the 3D display. That is, the forward transformation model 305 estimates or models the direct and cross-talk signals by characterizing the forward transformation from image acquisition (e.g., image acquisition module 205) to 3D display (e.g., 3D display 210). This is done by .measuring output signals generated by the 3D display when, using test, signals as an input. As appreciated by one skilled in the art, the forward transformation model 305 can be represented by a mathematical function F(.).
(0018] hi various embodiments, the test signals may include both left and right test signals jointly, or individual left and right, test signals, in the first case, test image signals l? and fir ar jointly sent to the 3D display to generate left and right output signals, referred to herein as Lp and and estimate the parameters of the forward transformation function F(.). That is:
f, (L!. Ji, )→Li; (Eq. 1 )
FH (Lf , RT)→RF (Eq. 2) where F/, represents the forward model used to characterize the left output signal . .■ and ¥R represents the forward model used to characterize the right output signal R?.
(0019] In the second case, the test image signals Lr and I are separately sent to the 3D display to generate left and right output signals that are measured. Thai is:
Ff (lj M→laL i i (Eq. 3) where Lot and Rci are the output signals thai would be displayed to the viewer's left (L∑,t) and right (Rci ) eyes when only the IT test signal is used as an input. Similarly, Lot and RDR are the output signals that would be displayed to the viewer's left ( <¾) an right (Rm) eyes when only the Rj- test signal is used as an input.
10020] As appreciated by one skilled in the art, the l. i and R n signals are the desired output signals at each eye in the absence of cross-talk, while the Rci and LCR signals represent the cross-talk that leaks to the other eye. For example, Ha, represents the cross-talk seen at the right eye when only the left image signal is sent to the display, while /,<¾ represents the cross-talk seen a the left eye when only the right image signal is sent to the display.
|(K)21 | in one embodiment an additive or other such model may be used to combine the measured responses for each eye, tha t is, to combine the /.· ¾ and responses for the left eye into a combined signal LD and to combine the i½, and ½? responses for the right eye into a combined signal ¾. The combined responses La and ϋ» may then used to estimate the parameters of the forward transformation function (.), Note thai this transformation function is display-dependent, as its parameters vary depending on the particular 3D display being used (e.g., a lenticular array display, a stereoscopic active glasses display, a light field display., and so on).
(0022] Once the forward transformation mode) 305 is generated with the test signals, input image signals (e.g., L 320 and R 325) may be applied to the cross-reduction module 305 to generate cross-corrected image signals (e.g., Lee 355 and {;e 360), First, the L 320 and R 325 input signals are applied to the forward transformation model 305 to characterize the cross-talk introduced by the 3D display with modeled cross-talk output signals L? and and desired signals Lm. and Rm- These signals are then sent to the visual model 310 to determine a visual measure representing how the visual quality of signals displayed in the 3D display is affected by its cross-talk, in one example, the visual mode! 310 computes a measure v of foe visual differences between the desired signals Lot and Rm and the modeled cross-talk output signals LF and A by taking into account visual effects involving spatial discrimination, color, and temporal discrimination, among others. It is appreciated that the visual model 10 may be any visual model for computing such a visual differences measure.
10023 j The cross-correction module 315 uses this measure v to modify the input image signals L 320 and R 325 to generate visually modified input signals LM 345 and RM 350. In one embodiment, this is done by varying visual parameters or characteristics such as contrast, brightness, and color of the input signals to generate the visually modified input signals as canonical transformations of the input signals.
(0024] The visually modified input signals LM 345 and A'.y 350 are then sent as inputs to the forward transformation model 305 to update the visual measure v and determine whether the modifications to the input signals reduced the cross-talk (the smaller the value of v, the lower the cross-talk). This process is repeated until the cross-talk is significantly reduced or completely eliminated, i.e., until it is visually reduced to a viewer. That is, nonlinear optimization is performed to iterate through values of v until v is minimized and the cross-talk is significantly reduced or completely eliminated in output signals Ice 355 and Rcc 360. it is appreciated that the output signals £«' 355 and Rcc 360 ar the same as the visually modified signals La 345 and ¾ 350 when the visual measure v is at its minimum.
(0625) It is also appreciated that the various left and right image signals illustrated in FIG. 3 (e.g., inputs L 320 and R 325, outputs /,<% 355 and Rcc 360) are shown for illustration purposes only. Multiple image views may be input into the cross-talk reduction module 300 (such as, for example, the multiple image views in a mdtiview display) to generate corresponding cross-talk corrected outputs. That is, the cross-talk reduction module 300 may be implemented for any type of 3D display regardless of the number of views it supports.
(0026) Attention is now directed to FIG. 4, which shows a flowchart, for reducing and correcting for cross-talk in a 3D display using the cross-talk reduction module of FIG. 3 in accordance with various embodiments. First, the cross-talk introduced in the 3D display is characterized with a plurality of test signals to generate a forward transformation model (400). Once the forward transformation model is generated, image signals are input into the model to generate modeled signals (405), These modeled signals may he, for example, the /,/- and Rf- i d Lp and ¾ signals described above,
(0027) Next, the modeled signals are applied to the visual model to compute a visual measure indicating how the visual quality of signals displayed in the 3D display is affected by its cross-talk (410). The input signals are then modified based on the visual measure (415) and re-applied to the forward transformation model until the visual measure is minimized (420). Once the visual measure is minimized, the modified, cross-talk corrected signals are sent to the 3D display for display (425). The cross-talk corrected signals ate such that crosstalk is visually reduced to a viewer. Alternatively, as appreciated by one skilled in the art, the modified, cross-talk corrected signals can be saved for later display,
(0028) Referring now to FIG. 5, a schematic diagram of a forward transformation model for use with the cross-talk reduction module of FIG. 3 is described. The forward transformation model 500 has four main transformations to characterise the photometric. geometric, and optical factors represented in the forward transformation function £'(.): 0) a space-varying offset and gain transformation 505; (2) a color correction transformation 510; (3) a geometric correction transformation 555; and (4) a space varying blur transformation 520. Test signals including color patches, grid patterns, horizontal and verticai stripes, and uniform white, black and gray level signals are sent to a 3D display in a dark room to estimate the parameters of FQ,
{002 j in the space- varying offset and gain transformation 505, white and black level signals are sent to the 3D display to determine its white and black responses and generate a gain offset output. Given this gain offset transformation, the color correction transforma ion 510 is determined next by fitting between measured, colors and color values. Measured average color values for gray input patches are used to determine one-dimensional look-up tables applied to input color components, and measured average color values for primary R, G, and B inputs are used to determine a color mixing matrix using the known input color values. Computing the fits using the spatially renormaiized colors allows the color correction transformation 510 to fit the data using a small number of parameters,
[0Θ30} Next, the geometric correction 515 may be determined using, for example, a polynomial mesh transformation model. The .final space-varying blur transformation 520 is required to obtain good results at the edges of the modeled signals. If the blur is not applied, objectionable halo artifacts may remain visible in the modeled signal In one embodiment, the parameters of the space-varying blur may be determined by estimating separate blur kernels in the horizontal and vertical directions, it is appreciated, that additional transformations may be used to generate the forward transformation model 500.
(0Θ3Ι ) FIG. 6 illustrates example test signals that may be used to generate the forward transformation model of FIG. 5. Test signal 600 represents a color patch ha ving multiple color squares, such as square 605, and is used for the color correction 510. Test signal 610 is a checkerboard used for the geometric correction 515, and the white and black test signals 615-620 are used for the space- varying gain and offset transfonnation 505. The test signals 625-630 contain horizontal and verticai lines to determine the space-varying blur parameters.
[ΘΘ32} As appreciated by one skilled in the art, other test signals may be used to generate the forward transfonnation model described herein. It is also appreciated that the care taken, in including various transformations to generate the forward transformation model enables the cross-talk reduction module of FIG, 3 to reduce and correct for cross-talk io any type of 3D display and for a wide range of input signals, while improving the visual quality of the displayed signals.
10033] it is appreciated that the previous description of the disclosed embodiments is provided to enable any person skilled i the art to make or use the present disclosure. Various modifications to these erabodimems will be readily apparent to those skilled in the art, and the geiieric principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not iiUended io be iiraited to the embodiments shown herein but is to he accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

WHAT IS CLAIMED IS:
1. A method for reducing cross-talk in a 3D display, the method comprising:
characterizing the cross-talk in the 3D display with a plurality of test signals to generate a forward transformaiion model;
applying input image signals to the forward transformation model to generate modeled signals;
applying the modeled signals to a visual model to compote a visual measure; and modi lying the input image signals based on the visual measure.
2. The method of claim ! , wherein characterizing the cross-taik in the 3D display comprises inputting the plurality of test signals into the 3D display and measuring a set of output signals.
3. The method of claim 2, further comprising using the set of output signals to generate the forward transformation model,
4. The method of claim 1, wherein the plurality of test signals comprise signals from the group consisting of a color patch test signal, a checkerboard test signal, a white test signal, a black test signal, a horizontal lined test signal, and a vertical lined test signal.
5. The method of clai 1 , wherein the forward transformation model comprises a set of transformations from the group consisting of a space- ary ing offset and gain transfon.nation. a color correction transformation, a geometric correction transformation, and a space varying blur transformation.
6. The method of claim 1 , wherein the modeled signals comprise a set of cross-talk modeled signals and a set of desired signals.
7. The method of claim 1, wherein the visual measure comprises a visual differences measure between the cross-talk modeled signals and the desired signals.
8. The method of claim 1 , wherein modifying the input image signals based on the visual measure comprises generating visually modified input signals.
9. The method of claim 8, wherein generating visually modified input signals comprises varying visual characteristics of the input image signals to generate the visually modified input signals as canonical transformations of the input image signals.
10. The method of claim 8, further comprising minimizing the visual measure.
11. The method of claim. 1.0, wherein minimizing the visual measure comprises applying the visually modified input signals to the forward transformation model to generate a new set of modeled signals and applying the ne set of modeled signals to the visual model to update the visual, measure until it is minimized.
12. A 3D display system, comprising
a 3D display screen; and
a cross-talk reduction module to reduce cross-talk introduced by the 3D display screen, the cross-talk reduction module comprising:
a forward transformation model to model the cross-talk introduced by the niuiiiview display screen and generate modeled signals from input image signals;
a visual model to compute a visual measure; and
a cross-talk correction module to modify the input image signals based on the visual measure.
13. The 3D display system of claim. 12, wherein the forward transformation model comprises a set of transformations from the group consisting of a space-varying offset and gain iransformaiion, a color correction transformation, a geometric correction transformation, and a space varying blur transformation..
14. The 3D display system of claim .13„ wherein the modeled signals comprise a set of cross-talk modeled signals and a set of desired signals.
15. The 3D display system of claim 12, wherein the visual measure comprises a visual differences measure between the cross-talk modeled signals and the desired signals,
16. The 3D display system of claim 12, wherein the cross-talk correction module generates visually modified input signals.
17. The 3D display system of claim 16, wherein the cross-talk correction module generates the visually modified input signals as canonical transformations of the input image signals by varying visual characteristics of ihe input image signals.
18. The 3D display system of claim 17, wherein the visually modified input signals are applied to the forward transformation model to generate a new set of modeled signals and the new set of modeled signals are applied to the visual mode! to update the visual, measure until it is minimized.
19. A cross- talk reduction module for use with a 3D display, ihe cross-talk reduction .module comprising:
a forward transformation model, to model cross-talk introduced by the 3D display and generate modeled signals from input image signals;
a visual model to compute a visual measure; and
a cross-talk correction module to modify the input image signals based, on the visual measure.
20. The cross-talk reduction module of claim. 1.9. wherein, the visual measure is minimized until the cross-talk introduced by the 3D display is visually reduced to a viewer.
PCT/US2011/049176 2011-08-25 2011-08-25 Model-based stereoscopic and multiview cross-talk reduction WO2013028201A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
KR1020147004419A KR101574914B1 (en) 2011-08-25 2011-08-25 Model-based stereoscopic and multiview cross-talk reduction
JP2014527133A JP5859654B2 (en) 2011-08-25 2011-08-25 Model-based crosstalk reduction in stereoscopic and multiview
PCT/US2011/049176 WO2013028201A1 (en) 2011-08-25 2011-08-25 Model-based stereoscopic and multiview cross-talk reduction
US14/237,439 US20140192170A1 (en) 2011-08-25 2011-08-25 Model-Based Stereoscopic and Multiview Cross-Talk Reduction
EP11871208.2A EP2749033A4 (en) 2011-08-25 2011-08-25 Model-based stereoscopic and multiview cross-talk reduction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2011/049176 WO2013028201A1 (en) 2011-08-25 2011-08-25 Model-based stereoscopic and multiview cross-talk reduction

Publications (1)

Publication Number Publication Date
WO2013028201A1 true WO2013028201A1 (en) 2013-02-28

Family

ID=47746736

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/049176 WO2013028201A1 (en) 2011-08-25 2011-08-25 Model-based stereoscopic and multiview cross-talk reduction

Country Status (5)

Country Link
US (1) US20140192170A1 (en)
EP (1) EP2749033A4 (en)
JP (1) JP5859654B2 (en)
KR (1) KR101574914B1 (en)
WO (1) WO2013028201A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3139604A1 (en) * 2015-09-07 2017-03-08 Samsung Electronics Co., Ltd. Method and apparatus for generating images
US11943271B2 (en) 2020-12-17 2024-03-26 Tencent America LLC Reference of neural network model by immersive media for adaptation of media for streaming to heterogenous client end-points

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5696107B2 (en) * 2012-09-11 2015-04-08 株式会社東芝 Image processing apparatus, method, program, and stereoscopic image display apparatus
US9438891B2 (en) * 2014-03-13 2016-09-06 Seiko Epson Corporation Holocam systems and methods
KR102476852B1 (en) * 2015-09-07 2022-12-09 삼성전자주식회사 Method for generating image and apparatus thereof
KR102401168B1 (en) * 2017-10-27 2022-05-24 삼성전자주식회사 Method and apparatus for calibrating parameter of 3d display apparatus
CA3193491A1 (en) * 2020-09-21 2022-03-24 Leia Inc. Multiview display system and method with adaptive background
JPWO2022091800A1 (en) * 2020-10-27 2022-05-05
WO2023152822A1 (en) * 2022-02-09 2023-08-17 ソニーグループ株式会社 Information processing device, information processing method, and program

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070188602A1 (en) * 2005-05-26 2007-08-16 Matt Cowan Projection of stereoscopic images using linearly polarized light
US20090168022A1 (en) * 2007-12-31 2009-07-02 Industrial Technology Research Institute Stereo-image displaying apparatus and method for reducing stereo-image cross-talk
US20100271464A1 (en) * 2009-04-22 2010-10-28 Samsung Electronics Co., Ltd. Apparatus and method for processing image

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2404106A (en) * 2003-07-16 2005-01-19 Sharp Kk Generating a test image for use in assessing display crosstalk.
JP2005353047A (en) * 2004-05-13 2005-12-22 Sanyo Electric Co Ltd Three-dimensional image processing method and three-dimensional image processor
GB2414882A (en) * 2004-06-02 2005-12-07 Sharp Kk Interlacing/deinterlacing by mapping pixels according to a pattern
JP4331224B2 (en) * 2007-03-29 2009-09-16 株式会社東芝 3D image display device and 3D image display method
US9088792B2 (en) * 2007-06-08 2015-07-21 Reald Inc. Stereoscopic flat panel display with synchronized backlight, polarization control panel, and liquid crystal display
US9384535B2 (en) * 2008-06-13 2016-07-05 Imax Corporation Methods and systems for reducing or eliminating perceived ghosting in displayed stereoscopic images
EP2510683A4 (en) * 2009-12-08 2013-12-04 Hewlett Packard Development Co Method for compensating for cross-talk in 3-d display
US8570319B2 (en) * 2010-01-19 2013-10-29 Disney Enterprises, Inc. Perceptually-based compensation of unintended light pollution of images for projection display systems
EP2557559A1 (en) * 2010-04-05 2013-02-13 Sharp Kabushiki Kaisha Three-dimensional image display device, display system, drive method, drive device, display control method, display control device, program, and computer-readable recording medium
US20120062709A1 (en) * 2010-09-09 2012-03-15 Sharp Laboratories Of America, Inc. System for crosstalk reduction
US8878894B2 (en) * 2010-09-15 2014-11-04 Hewlett-Packard Development Company, L.P. Estimating video cross-talk

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070188602A1 (en) * 2005-05-26 2007-08-16 Matt Cowan Projection of stereoscopic images using linearly polarized light
US20090168022A1 (en) * 2007-12-31 2009-07-02 Industrial Technology Research Institute Stereo-image displaying apparatus and method for reducing stereo-image cross-talk
US20100271464A1 (en) * 2009-04-22 2010-10-28 Samsung Electronics Co., Ltd. Apparatus and method for processing image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2749033A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3139604A1 (en) * 2015-09-07 2017-03-08 Samsung Electronics Co., Ltd. Method and apparatus for generating images
CN106507089A (en) * 2015-09-07 2017-03-15 三星电子株式会社 Method and apparatus for generating image
US10008030B2 (en) 2015-09-07 2018-06-26 Samsung Electronics Co., Ltd. Method and apparatus for generating images
CN106507089B (en) * 2015-09-07 2020-12-08 三星电子株式会社 Method and apparatus for generating image
US11943271B2 (en) 2020-12-17 2024-03-26 Tencent America LLC Reference of neural network model by immersive media for adaptation of media for streaming to heterogenous client end-points

Also Published As

Publication number Publication date
JP2014529954A (en) 2014-11-13
EP2749033A1 (en) 2014-07-02
EP2749033A4 (en) 2015-02-25
JP5859654B2 (en) 2016-02-10
US20140192170A1 (en) 2014-07-10
KR20140051333A (en) 2014-04-30
KR101574914B1 (en) 2015-12-04

Similar Documents

Publication Publication Date Title
WO2013028201A1 (en) Model-based stereoscopic and multiview cross-talk reduction
Lambooij et al. Visual discomfort of 3D TV: Assessment methods and modeling
CN102484687B (en) For compensating the method for the crosstalk in 3-D display
TW201333533A (en) Display apparatuses and methods for simulating an autostereoscopic display device
WO2008102366A2 (en) A method and a system for calibrating and/or visualizing a multi image display and for reducing ghosting artifacts
JP2015162718A (en) Image processing method, image processing device and electronic equipment
CN111869202B (en) Method for reducing crosstalk on autostereoscopic displays
US10368048B2 (en) Method for the representation of a three-dimensional scene on an auto-stereoscopic monitor
TWI469624B (en) Method of displaying three-dimensional image
Sanftmann et al. Anaglyph stereo without ghosting
CA2948697A1 (en) Generation of drive values for a display
JP5488482B2 (en) Depth estimation data generation device, depth estimation data generation program, and pseudo-stereoscopic image display device
JP5845780B2 (en) Stereoscopic image generating apparatus and stereoscopic image generating method
Xu et al. Quality of experience for the horizontal pixel parallax adjustment of stereoscopic 3D videos
US9064338B2 (en) Stereoscopic image generation method and stereoscopic image generation system
Li et al. On adjustment of stereo parameters in multiview synthesis for planar 3D displays
Smit et al. Three Extensions to Subtractive Crosstalk Reduction.
JP5780214B2 (en) Depth information generation device, depth information generation method, depth information generation program, pseudo stereoscopic image generation device
Seuntiëns et al. Capturing the added value of three-dimensional television: Viewing experience and naturalness of stereoscopic images
JP5786807B2 (en) Depth information generation device, depth information generation method, depth information generation program, pseudo stereoscopic image generation device
Gunnewiek et al. How to display 3D content realistically
JP2012084961A (en) Depth signal generation device, pseudo stereoscopic image signal generation device, depth signal generation method, pseudo stereoscopic image signal generation method, depth signal generation program, and pseudo stereoscopic image signal generation program
Xing Towards Reliable Stereoscopic 3D Quality Evaluation: Subjective Assessment and Objective Metrics
Doyen et al. Towards a free viewpoint and 3D intensity adjustment on multi-view display
JP6028427B2 (en) Depth information generation device, depth information generation method, depth information generation program, pseudo stereoscopic image generation device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11871208

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2011871208

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2011871208

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 14237439

Country of ref document: US

ENP Entry into the national phase

Ref document number: 20147004419

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2014527133

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE