US20140292748A1 - System and method for providing stereoscopic image by adjusting depth value - Google Patents
System and method for providing stereoscopic image by adjusting depth value Download PDFInfo
- Publication number
- US20140292748A1 US20140292748A1 US14/230,834 US201414230834A US2014292748A1 US 20140292748 A1 US20140292748 A1 US 20140292748A1 US 201414230834 A US201414230834 A US 201414230834A US 2014292748 A1 US2014292748 A1 US 2014292748A1
- Authority
- US
- United States
- Prior art keywords
- depth value
- stereoscopic image
- display device
- depth
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/128—Adjusting depth or disparity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G06T7/0075—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Definitions
- the present invention relates to technology for estimating a depth value of an object from a first stereoscopic image, and providing a second stereoscopic image in which the depth value is adjusted based on a display device.
- a depth value of the stereoscopic image may vary based on a size of a display device or a viewing distance from the display device.
- a director or a producer may set the depth value in consideration of the display device such that the stereoscopic image may provide a designed cubic effect using a predetermined display device.
- An aspect of the present invention provides a system and method for providing a stereoscopic image realizing a designed cubic effect without restrictions on a type of a display device.
- Another aspect of the present invention also provides a system and method for generating a depth map from a left image and a right image of a stereoscopic image and adjusting a depth image based on a size and a viewing distance of a display device.
- a system for providing a stereoscopic image including a depth value estimator to estimate a depth value of an object included in a first stereoscopic image from the stereoscopic image, a depth value adjusting unit to adjust the depth value in consideration of a display device, a stereoscopic image processing unit to process the first stereoscopic image to be a second stereoscopic image based on the adjusted depth value, and a stereoscopic image provider to provide the second stereoscopic image to the display device.
- a method of providing a stereoscopic image including estimating a depth value of an object included in a first stereoscopic image from the stereoscopic image, adjusting the depth value in consideration of a display device, processing the first stereoscopic image to be a second stereoscopic image based on the adjusted depth value, and providing the second stereoscopic image to the display device.
- FIG. 1 is a block diagram illustrating a configuration of a system for providing a stereoscopic image according to an embodiment of the present invention
- FIG. 2 is a diagram illustrating depth value estimation according to an embodiment of the present invention
- FIGS. 3A through 3C are diagrams illustrating depth value adjustment according to an embodiment of the present invention.
- FIGS. 4A through 4C are diagrams illustrating stereoscopic image processing according to an embodiment of the present invention.
- FIG. 5 is a flowchart illustrating a method of providing a stereoscopic image according to an embodiment of the present invention.
- FIG. 1 is a block diagram illustrating a configuration of a system 100 for providing a stereoscopic image according to an embodiment of the present invention.
- the system 100 for providing a stereoscopic image includes a depth value estimator 110 , a depth value adjusting unit 120 , a stereoscopic image processing unit 130 , and a stereoscopic image provider 140 .
- the depth value estimator 110 may estimate a depth value of an object included in a first stereoscopic image from the first stereoscopic image.
- the first stereoscopic image may be captured using two cameras, and be separated into a left image captured at a left side of the object and a right image captured at a right side of the object.
- the depth value estimator 110 may estimate the depth value by recognizing a matching object between the left image and the right image.
- the depth value estimator 110 may estimate a zero depth value of a reference object included in a zero screen among objects.
- the zero screen may be a point at which focuses of the two cameras match each other in the first stereoscopic image. Since the focuses of the two cameras match each other, the first stereoscopic image and the reference object may be seen at the same distance.
- the depth value estimator 110 may estimate a negative depth value of an object visible over a short distance relative to the reference object.
- the negative depth value is less than the zero depth value.
- the object included in the left image of the first stereoscopic image may be seen at a position horizontally different from a position of an object included in the right image which is paired with the left image in the first stereoscopic image.
- the object included in the left image of the first stereoscopic image corresponding to the object visible over a short distance when compared to a right side, may be positioned unevenly to a right side in the left image.
- the depth value estimator 110 may set a relatively low depth value for the object.
- the depth value estimator 110 may estimate a positive depth value of an object visible over a long distance relative to the reference object. For example, the object visible over a long distance relative to the reference object, when compared to the right image, may be positioned unevenly to the left side in the left image. Also, an image of the object visible over a long distance relative to the reference object may be focused at a longer distance as compared to the first stereoscopic image. Thus, the depth value estimator 110 may set a relatively high depth value for the object.
- the depth value adjusting unit 120 may adjust the depth value in consideration of a size of a display device.
- the depth value may decrease as the first stereoscopic image is downscaled.
- the depth value may increase as the first stereoscopic image is upscaled.
- the depth value adjusting unit 120 may adjust the depth image in consideration of a viewing distance from the display device. When the viewing distance from the display device is relatively short, the depth value of the first stereoscopic image may increase. When the viewing distance of the display device is relatively long, the depth value of the first stereoscopic image may decrease.
- the depth value adjusting unit 120 may adjust, to be the estimated depth value, the depth value adjusted based on the size of display device and the viewing distance from the display device.
- the depth value adjusting unit 120 may maintain a depth value of the reference object included in the zero screen among the objects included in the first stereoscopic image rather than adjusting the depth value of the reference object.
- the stereoscopic image processing unit 130 may process the first stereoscopic image to be a second stereoscopic image using the adjusted depth value, through a depth image-based rendering (DIBR).
- DIBR depth image-based rendering
- the DIBR may refer to a technology for producing a stereoscopic image from a piece of two-dimensional (2D) image or a plurality of 2D images, using a depth map.
- the stereoscopic image processing unit 130 may generate the depth map in which the depth value is set for each pixel included in the first stereoscopic image.
- the stereoscopic image processing unit 130 may calculate a disparity of the object using the depth map, thereby processing the second stereoscopic image.
- the stereoscopic image provider 140 may provide the second stereoscopic image to the display device.
- the display device may include a screen in a theater, a television, a projector, a laptop computer, and a smart phone.
- FIG. 2 is a diagram illustrating depth value estimation according to an embodiment of the present invention.
- a system for providing a stereoscopic image may estimate each depth value of an object 210 , an object 220 , and a reference object 230 included in a first stereoscopic image 200 .
- the first stereoscopic image 200 may be produced by generating a composite of a left image and a right image, respectively captured at different sides of each of the object 210 , the object 220 , and the reference object 230 using two cameras.
- the left image and the right image included in the first stereoscopic image 200 may be recognized through a left eye and a right eye of a human body.
- the first stereoscopic image 200 may provide a cubic effect varying based on the depth value.
- the system for providing a stereoscopic image may estimate a zero depth value of the reference object 230 included in a zero screen.
- the zero screen may refer to a portion of which a pixel value is not changed in a process of compositing the left image and the right image included in the first stereoscopic image 200 . Since the reference object 230 displayed on the same position of the left image and the right image, the reference object 230 may be viewed at the same perspective distance as the first stereoscopic image 200 .
- the system for providing a stereoscopic image may estimate a negative depth value of the object 210 visible over a short distance relative to the reference object 230 .
- the object 210 may be positioned unevenly to a right side in the left image when compared to the right image.
- an image of the object 210 may be obtained at a shorter distance as compared to the first stereoscopic image 200 .
- the system for providing a stereoscopic image may set a depth value less than the depth value of the reference object 230 for the object 210 .
- the system for providing a stereoscopic image may estimate a positive depth value of the object 220 visible over a long distance, relative to the reference object 230 .
- the object 220 may be positioned unevenly to a left side in the left image when compared to the right image. When the left image and the right image are recognized through the left eye and the right eye, an image of the object 220 may be obtained at a longer distance as compared to the first stereoscopic image 200 .
- the system for providing a stereoscopic image may set a depth value greater than the depth value of the reference object 230 for the object 220 .
- FIGS. 3A through 3C are diagrams illustrating depth value adjustment according to an embodiment of the present invention.
- a system for providing a stereoscopic image may estimate a depth value of an object included in a first stereoscopic image, and adjust the depth image in consideration of a display device.
- the first stereoscopic image of FIG. 3B when the first stereoscopic image of FIG. 3B is projected onto a display device having a size of 13 inches and a viewing distance of 10 meters (m), the first stereoscopic image may be displayed as shown in FIG. 3A
- the system for providing a stereoscopic image may adjust the depth image in consideration of a size of a display device.
- the depth value may increase as the first stereoscopic image is upscaled.
- the system for providing a stereoscopic image may adjust the depth image in consideration of a viewing distance from the display device.
- the depth value of the first stereoscopic image may increase.
- the depth value of the first stereoscopic image may decrease.
- the system for providing a stereoscopic image may adjust, to be the estimated depth value, the depth value adjusted based on the size of display device and the viewing distance from the display device.
- the first stereoscopic image of FIG. 3B when the first stereoscopic image of FIG. 3B is projected on a display device having a size of five inches and a viewing distance of 1 m, the first stereoscopic image may be displayed as shown in FIG. 3C .
- the depth value may decrease as the first stereoscopic image is downscaled.
- the depth value of the first stereoscopic image may decrease.
- the system for providing a stereoscopic image may adjust, to be the estimated depth value, the depth value adjusted based on the size of display device and the viewing distance from the display device.
- FIGS. 4A through 4C are diagrams illustrating examples of stereoscopic image processing according to an embodiment of the present invention.
- the system for providing a stereoscopic image may estimate depth images of objects in a first stereoscopic image.
- the first stereoscopic image may include a reference object included in a zero screen, an object visible over a short distance relative to the reference object, and an object visible over a long distance relative to the reference object.
- the system for providing a stereoscopic image may generate a depth map by setting a different depth value for each of the objects.
- the system for providing a stereoscopic image may estimate a negative depth value of the object visible over a short distance relative to the reference object included in the zero screen, and estimate a positive depth value of the object visible over a long distance relative to the reference object.
- the depth map of FIG. 4B may include the same number of pixels as a number of pixels included in the first stereoscopic image of FIG. 4A , and allocate a value corresponding to the depth value for each of the pixels. For example, in the depth map of FIG. 4B , the system for providing a stereoscopic image may allocate a value between “0” to “255” for each of the pixels to express the depth values of the objects.
- the system for providing a stereoscopic image may allocate “128” to a pixel corresponding to the reference object.
- the system for providing a stereoscopic image may allocate a value between “0” and “127” to a pixel corresponding to the object.
- the system for providing a stereoscopic image may allocate a value between “129” and “255” to a pixel corresponding to the object.
- the depth map of FIG. 4B may be expressed in a lower brightness as the value allocated to the pixel, that is, the depth value of the object is increased.
- the system for providing a stereoscopic image may adjust the depth in consideration of a display device.
- the depth value may increase as the size of display device increases or the viewing distance from the display device decreases.
- the depth value may decrease as the size of display device decreases or the viewing distance from the display device increases.
- the system for providing a stereoscopic image may adjust, to be the estimated depth value, the depth value adjusted based on the size of display device and the viewing distance from the display device.
- the system for providing a stereoscopic image may process the first stereoscopic image of FIG. 4A to be a second stereoscopic image of FIG. 4C using the depth value adjusted in the depth map of FIG. 4B .
- the system for providing a stereoscopic image may process the second stereoscopic image through a DIBR.
- the DIBR may process the second stereoscopic image through the DIBR.
- the second stereoscopic image having a different viewpoint using the depth map of FIG. 4B and a piece of 2D image or a plurality of 2D images.
- the system for providing a stereoscopic image may calculate a disparity of the object using the depth map of FIG. 4B .
- the disparity may refer to a difference of a pixel included in the first stereoscopic image of FIG. 4A visible through a left eye and a right eye of a human.
- the system for providing a stereoscopic image may subtract the value allocated to the pixel included in the depth map of FIG. 4B from “1”, and multiply a result of the subtracting by a predetermined maximum disparity value.
- the system for providing a stereoscopic image may process the second stereoscopic image of FIG. 4C by relocating the objects included in the first stereoscopic image of FIG. 4A in a left or right direction by the calculated disparity.
- FIG. 5 is a flowchart illustrating a method of providing a stereoscopic image according to an embodiment of the present invention.
- the system for providing a stereoscopic image may estimate a depth value of an object included in a first stereoscopic image from the first stereoscopic image.
- the first stereoscopic image may be captured using two cameras, and be separated into a left image captured at a left side of the object and a right image captured at a right side of the object.
- the system for providing a stereoscopic image may estimate the depth value by recognizing a matching object between the left image and the right image.
- the system for providing a stereoscopic image may estimate a zero depth value of a reference object included in a zero screen among objects.
- the zero screen may be a point at which focuses of the two cameras match in the first stereoscopic image. Since the focuses of the two cameras match, the first stereoscopic image and the reference object may be seen at the same distance.
- the system for providing a stereoscopic image may estimate a negative depth value of an object visible over a short distance relative to the reference object.
- the object may be seen at a horizontally different position in a left image and a right image of the first stereoscopic image.
- the object may be positioned unevenly to the right side in the left image when compared to the right image.
- an image of the object may be focused at a shorter distance as compared to the first stereoscopic image.
- the system for providing a stereoscopic image may set a relatively low depth value for the object.
- the system for providing a stereoscopic image may estimate a positive depth value of an object visible over a long distance relative to the reference object.
- the object may be positioned unevenly to the left side in the left image when compared to the right image. In this case, an image of the object may be focused at a longer distance when compared to the first stereoscopic image.
- the system for providing a stereoscopic image may set a relatively high depth value for the object.
- the system for providing a stereoscopic image may consider a display device.
- the system for providing a stereoscopic image may consider a size of a display device. When the size of the display device is relatively small, the depth value may decrease as the first stereoscopic image is downscaled. In contrast, when the size of the display device is relatively large, the depth value may increase as the first stereoscopic image is upscaled. Also, the system for providing a stereoscopic image may consider a viewing distance from the display device. When the viewing distance is relatively short, the depth value of the first stereoscopic image may increase. In contrast, when the viewing distance is relatively long, the depth value of the first stereoscopic image may decrease.
- the system for providing a stereoscopic image may adjust the depth image based on the size of the display device and the viewing distance from the display device.
- the system for providing a stereoscopic image may maintain a depth value of a reference object included in a zero screen among objects of the first stereoscopic image, rather than adjusting the depth value of the reference object.
- the system for providing a stereoscopic image may process the first stereoscopic image to be a second stereoscopic image through a DIBR.
- the DIBR may refer to a technology for producing a stereoscopic image from a piece of 2D image or a plurality of 2D images using a depth map.
- the system for providing a stereoscopic image may generate the depth map in which the adjusted depth value is set for each pixel included in the first stereoscopic image.
- the system for providing a stereoscopic image may calculate a disparity of the object to process the second stereoscopic image.
- the system for providing a stereoscopic image may provide the processed stereoscopic image using the display device.
- the methods according to the above-described embodiments may be recorded, stored, or fixed in one or more non-transitory computer-readable media that includes program instructions to be implemented by a computer to cause a processor to execute or perform the program instructions.
- the media may also include, alone or in combination with the program instructions, data files, data structures, and the like.
- the program instructions recorded on the media may be those specially designed and constructed, or they may be of the kind well-known and available to those having skill in the computer software arts.
- non-transitory computer-readable media examples include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM discs and DVDs; magneto-optical media such as optical discs; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like.
- program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.
- the described hardware devices may be configured to act as one or more software modules in order to perform the operations and methods described above, or vice versa.
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Processing Or Creating Images (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Controls And Circuits For Display Device (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR10-2013-0035308 | 2013-04-01 | ||
KR1020130035308A KR20140119999A (ko) | 2013-04-01 | 2013-04-01 | 깊이값 조정을 통한 입체영상 제공 시스템 및 방법 |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140292748A1 true US20140292748A1 (en) | 2014-10-02 |
Family
ID=51620330
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/230,834 Abandoned US20140292748A1 (en) | 2013-04-01 | 2014-03-31 | System and method for providing stereoscopic image by adjusting depth value |
Country Status (2)
Country | Link |
---|---|
US (1) | US20140292748A1 (ko) |
KR (1) | KR20140119999A (ko) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019201355A1 (en) * | 2018-04-17 | 2019-10-24 | Shanghaitech University | Light field system occlusion removal |
US11257467B2 (en) | 2017-12-07 | 2022-02-22 | Samsung Electronics Co., Ltd. | Method for controlling depth of object in mirror display system |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070047040A1 (en) * | 2005-08-31 | 2007-03-01 | Samsung Electronics Co., Ltd. | Apparatus and method for controlling depth of three-dimensional image |
US20120287235A1 (en) * | 2011-05-13 | 2012-11-15 | Ahn Mooki | Apparatus and method for processing 3-dimensional image |
US20130106841A1 (en) * | 2011-11-01 | 2013-05-02 | Acer Incorporated | Dynamic depth image adjusting device and method thereof |
US20130147792A1 (en) * | 2011-12-13 | 2013-06-13 | Samsung Electronics Co., Ltd. | Method and apparatus for displaying a 3d image in a mobile terminal |
-
2013
- 2013-04-01 KR KR1020130035308A patent/KR20140119999A/ko not_active Application Discontinuation
-
2014
- 2014-03-31 US US14/230,834 patent/US20140292748A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070047040A1 (en) * | 2005-08-31 | 2007-03-01 | Samsung Electronics Co., Ltd. | Apparatus and method for controlling depth of three-dimensional image |
US20120287235A1 (en) * | 2011-05-13 | 2012-11-15 | Ahn Mooki | Apparatus and method for processing 3-dimensional image |
US20130106841A1 (en) * | 2011-11-01 | 2013-05-02 | Acer Incorporated | Dynamic depth image adjusting device and method thereof |
US20130147792A1 (en) * | 2011-12-13 | 2013-06-13 | Samsung Electronics Co., Ltd. | Method and apparatus for displaying a 3d image in a mobile terminal |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11257467B2 (en) | 2017-12-07 | 2022-02-22 | Samsung Electronics Co., Ltd. | Method for controlling depth of object in mirror display system |
WO2019201355A1 (en) * | 2018-04-17 | 2019-10-24 | Shanghaitech University | Light field system occlusion removal |
US11398019B2 (en) | 2018-04-17 | 2022-07-26 | Shanghaitech University | Light field system occlusion removal |
Also Published As
Publication number | Publication date |
---|---|
KR20140119999A (ko) | 2014-10-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5156837B2 (ja) | 領域ベースのフィルタリングを使用する奥行マップ抽出のためのシステムおよび方法 | |
US9153032B2 (en) | Conversion method and apparatus with depth map generation | |
US10095953B2 (en) | Depth modification for display applications | |
US20130051659A1 (en) | Stereoscopic image processing device and stereoscopic image processing method | |
US20100315488A1 (en) | Conversion device and method converting a two dimensional image to a three dimensional image | |
US8982187B2 (en) | System and method of rendering stereoscopic images | |
JP2016100899A (ja) | 映像補正方法及び装置 | |
US20140002441A1 (en) | Temporally consistent depth estimation from binocular videos | |
CN106504186B (zh) | 一种立体图像重定向方法 | |
US8750600B2 (en) | Apparatus and method for generating three-dimensional (3D) zoom image of stereo camera | |
US20170019654A1 (en) | Image processing apparatus, image processing method, and storage medium | |
US20140085296A1 (en) | Apparatus and method for processing multi-view image | |
US9571810B2 (en) | Method and apparatus of determining perspective model for depth map generation by utilizing region-based analysis and/or temporal smoothing | |
US20130076858A1 (en) | Method and apparatus for converting 2d content into 3d content | |
US20140292748A1 (en) | System and method for providing stereoscopic image by adjusting depth value | |
US20120170841A1 (en) | Image processing apparatus and method | |
US9787980B2 (en) | Auxiliary information map upsampling | |
JP6025740B2 (ja) | エネルギー値を用いたイメージ処理装置およびそのイメージ処理方法、並びにディスプレイ方法 | |
US9197874B1 (en) | System and method for embedding stereo imagery | |
US20130229408A1 (en) | Apparatus and method for efficient viewer-centric depth adjustment based on virtual fronto-parallel planar projection in stereoscopic images | |
KR20140113066A (ko) | 차폐 영역 정보를 기반으로 하는 다시점 영상 생성 방법 및 장치 | |
US9973745B2 (en) | Stereoscopic focus point adjustment | |
US9591290B2 (en) | Stereoscopic video generation | |
US10931927B2 (en) | Method and system for re-projection for multiple-view displays | |
JP2014022867A (ja) | 画像処理装置および方法、並びにプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTIT Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BAN, YUN JI;KIM, HAE DONG;KIM, HYE SUN;AND OTHERS;SIGNING DATES FROM 20140318 TO 20140325;REEL/FRAME:032564/0732 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |