WO2012165692A1 - Stereoscopic image display device with stereoscopic digital information display function and method of displaying stereoscopic digital information in stereoscopic image - Google Patents

Stereoscopic image display device with stereoscopic digital information display function and method of displaying stereoscopic digital information in stereoscopic image Download PDF

Info

Publication number
WO2012165692A1
WO2012165692A1 PCT/KR2011/004601 KR2011004601W WO2012165692A1 WO 2012165692 A1 WO2012165692 A1 WO 2012165692A1 KR 2011004601 W KR2011004601 W KR 2011004601W WO 2012165692 A1 WO2012165692 A1 WO 2012165692A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
stereoscopic
eye image
right eye
left eye
Prior art date
Application number
PCT/KR2011/004601
Other languages
English (en)
French (fr)
Inventor
Myoung-Han YOU
Original Assignee
Postmedia Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Postmedia Co., Ltd. filed Critical Postmedia Co., Ltd.
Publication of WO2012165692A1 publication Critical patent/WO2012165692A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/172—Processing image signals image signals comprising non-image signal components, e.g. headers or format information
    • H04N13/178—Metadata, e.g. disparity information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Definitions

  • the present invention relates to a stereoscopic image display device, and more particularly, to a stereoscopic image display device with a stereoscopic digital information display function in which a stereoscopic image is displayed in real time through a stereoscopic image converter and a method of displaying stereoscopic digital information in which a virtual stereoscopic object is synchronized and augmented at the position of a certain image.
  • Stereoscopic images are generally displayed using a polarization method or a shutter glasses method.
  • a stereoscopic image is implemented by displaying left and right images on an odd-number line and an even-number line, respectively, within one frame through interlace scanning as illustrated in FIG. 1.
  • a stereoscopic image is implemented by displaying left and right images by progressive scanning through interlace scanning for each frame as illustrated in FIG. 1.
  • a previously created image is displayed as the stereoscopic image, but it is not displayed in real time as the stereoscopic image.
  • a real-time stereoscopic image may be implemented using on left and right images captured by two cameras.
  • AR augmented reality
  • augmented reality may be applied to the stereoscopic image.
  • reality avirtualobjectisoverlaidonanactualenvironment.
  • a user can be provided with supplemental information as virtual reality while seeing an actual environment with a sense of realism.
  • information for example, a telephone number
  • a shop near it is represented through a stereoscopic image by the augmented reality.
  • the virtual stereoscopic object In order to apply the augmented reality, it is necessary to augment a virtual stereoscopic object on the left and right images.
  • the virtual stereoscopic object minutely vibrates, and thus the left and right images are not synchronized. At this time, if synchronization is not made, the stereoscopic effect of the virtual stereoscopic object deteriorates.
  • an image needs be processed to be compatible with a stereoscopic display device.
  • an image is generally processed by software.
  • an overhead occurs in the process of processing the image.
  • RBFT rect base feature tracking
  • the stereoscopic image display device includes a left eye camera that captures a left eye image in real time, a right eye camera that captures a right eye image in real time, a set top box that receives the left eye image and the right eye image, tracks and detects predetermined images from the left eye image and the right eye image, detects a marker ID by detecting and comparing feature points of the detected predetermined images, generates a virtual stereoscopic object corresponding to the marker ID, synchronizes the position of the virtual stereoscopic object, augments the virtual stereoscopic object on the left eye image and the right eye image, and outputs the left eye image and the right eye image on which the virtual stereoscopic object is augmented, a stereoscopic image converter that generates a stereoscopic image from the left eye image and the right eye image output from the set top box and then output the stereoscopic image, and a stereoscopic image
  • the predetermined image detected by the set top box may be a square box.
  • the set top box may binarize the left eye image and the right eye image, detect the predetermined images from the binarized left eye image and the binarized right eye image, calculate three-dimensional coordinates of the detected predetermined images, and detect the feature points inside the predetermined images whose coordinates are calculated.
  • the position of the virtual stereoscopic object may be synchronized such that transformation matrixes
  • Another aspect of the present invention provides a method of displaying stereoscopic digital information in a stereoscopic image.
  • the method includes receiving a left eye image and a right eye image, tracking and detecting predetermined images from the left eye image and the right eye image, detecting a marker ID by detecting and comparing feature points of the detected predetermined images, generating a virtual stereoscopic object corresponding to the marker ID and synchronizing the position of the virtual stereoscopic object, augmenting the synchronized virtual stereoscopic object on the left eye image and the right eye image, and converting the left eye image and the right eye image into a stereoscopic image and outputting the stereoscopic image.
  • the predetermined image may be a square box.
  • the method may further includes, in order to track and detect the predetermined image and detect the feature point of the predetermined image, binarizing the left eye image and the right eye image, detecting the predetermined images from the binarized left eye image and the binarized right eye image, and calculating three-dimensional coordinates of the detected predetermined images and detecting the feature points inside the predetermined images whose coordinates are calculated.
  • the position of the virtual stereoscopic object may be synchronized such that transformation matrixes
  • the virtual stereoscopic object augmented on the left and right images can be reliably displayed by synchronization. Further, the vibration problem of the virtual stereoscopic object is solved, and thus the virtual stereoscopic object can be reliably recognized. Furthermore, the real time image can be augmented and displayed at a high speed as the stereoscopic image through the stereoscopic image converter.
  • FIG. 1 is a diagram illustrating a method of displaying a stereoscopic image by a polarization method.
  • FIG. 2 is a diagram illustrating a method of displaying a stereoscopic image by a shutter glasses method.
  • FIG. 3 is a block diagram illustrating a stereoscopic image display device with a stereoscopic digital information display function according to an embodiment of the present invention.
  • FIG. 4 is a flowchart illustrating a method of displaying stereoscopic digital information in a stereoscopic image which is performed by the device illustrated in FIG. 3.
  • FIG. 5 is a diagram illustrating synchronization and conversion to a stereoscopic image illustrated in FIG. 4.
  • FIG. 6 is a diagram for explaining a process of tracking and detecting a specific image and detecting a feature point of the specific image according to an embodiment of the present invention.
  • a stereoscopic image display device with a stereoscopic digital display function has a configuration for capturing a stereoscopic image in real time using a left eye camera 10 that captures a left eye image in real time and a right eye camera 12 that captures a right eye image in real time.
  • the left eye camera 10 and the right eye camera 12 provide a set top box 20 with the left eye image and the right eye image, respectively.
  • the set top box 20 receives the left eye image and the right eye image from the left eye camera 10 and the right eye camera 12 and outputs a left eye image and a right eye image on which a virtual stereoscopic object is augmented.
  • the set top box 20 tracks and detects previously agreed specific images from the left eye image and the right eye image, detects a marker ID by detecting and comparing feature points of the detected specific images, generates a virtual stereoscopic object corresponding to the marker ID, synchronizes the position of the virtual stereoscopic object, augments the virtual stereoscopic object on the left eye image and the right eye image, and outputs the left eye image and the right eye image on which the virtual stereoscopic object is augmented.
  • a stereoscopic image converter 30 generates one stereoscopic image from the left eye image and the right eye image output from the set top box 20 and then output the stereoscopic image to a stereoscopic display unit 40.
  • the stereoscopic display unit 40 displays the stereoscopic image provided from the stereoscopic image converter 30.
  • the virtual stereoscopic object is synchronized with the left eye image and the right eye image through the set top box 20, and conversion into the stereoscopic image is performed by the stereoscopic image converter 30.
  • an image conversion process may be performed as illustrated in FIG.5.
  • the set top box 20 sets a square box as a marker for determining a feature point, that is, a previously agreed specific image. Markers of various shapes may be used depending on producers.
  • the set top box 20 loads data of a marker (a square box) to recognize (step S10) and then acquires the left eye image and the right eye image from the left eye camera 10 and the right eye camera 12 (step S12).
  • a marker a square box
  • the set top box 20 tracks and detects the square boxes from the left eye image and the right eye image (step S14).
  • the square box is applied to the left eye image and the right eye image, and the set top box 20 detects the feature point only when the square box is detected. As a result, the processing speed is improved, and a vibration phenomenon is reduced.
  • the set top box 20 binarizes an original image and then detects the square box (step S16). At this time, since the original image is binarized, the recognition rate of the square box is improved. If a monochrome image is black-and-white processed by binarization, the recognition rate can be further improved.
  • the set top box 20 extracts a square box image (step S18). In this process, the set top box 20 calculates three-dimensional coordinates of the square box and determines the feature point of the specific image inside the square box (step S20).
  • the set top box 20 detects a marker ID corresponding to the feature point (step S22).
  • the set top box 20 performs an operation for synchronizing a virtual stereoscopic object corresponding to the marker ID (step S24).
  • the set top box 20 performs left rendering (step S26) and right rendering (step S28).
  • the left rendering refers to rendering the augmented virtual stereoscopic object onto the left eye image
  • the right rendering refers to rendering the augmented virtual stereoscopic object onto the right eye image.
  • FIG. 5 (1) illustrates the left eye image and the right eye image acquired in step S12, and (2) illustrates the images synchronized in step S14 to step S28.
  • the synchronization of the virtual object is made such that transformation matrixes representing the position at which the virtual stereoscopic object is to be augmented on the left eye image and the right eye image are obtained, and then a process of changing rotational components of the transformation matrix of either image is performed for synchronization.
  • the transformation matrixes representing the position at which the virtual stereoscopic object is to be augmented on the left eye image and the right eye image may be obtained as follows:
  • the synchronization may be made by changing rotational components of the transformation matrix of the right eye image to those of the left eye as follows:
  • R represents the right
  • L represents the left
  • xpos represents an x coordinate
  • ypox represents a y coordinates
  • zpos represents a z coordinate.
  • L'11, L'12, L'13, L'21, L'22, L'23, L'31, L'32, L'33, R'11, R'12, R'13, R'21, R'22, R'23, R'31, R'32, R'33 are rotational components, and (0,0,0,1) are components representing the position.
  • the stereoscopic image converter 30 After synchronizing the virtual stereoscopic object with the left eye image and the right eye image through the set top box 20 as described above, the stereoscopic image converter 30 converts the synchronized images into a stereoscopic image illustrated in (3) of FIG. 5 and provides the stereoscopic display unit 40 with the converted stereoscopic image.
  • the left eye image and the right eye image captured by the two cameras can be displayed as the stereoscopic image, and the virtual stereoscopic object augmented on the stereoscopic image can be three-dimensionally displayed.
  • the virtual object augmented on the left and right eye images can be reliably displayed by synchronization.
  • the vibration phenomenon of the virtual stereoscopic object is prevented, and thus the virtual stereoscopic object can be reliably recognized.
  • the real-time image can be augmented and displayed at a high speed as the stereoscopic image by the stereoscopic image converter.
  • the position and depth of a touching hand change depending on the position of the marker, and thus the virtual stereoscopic object with the improved stereoscopic effect can be provided.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Library & Information Science (AREA)
  • Processing Or Creating Images (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
PCT/KR2011/004601 2011-05-30 2011-06-23 Stereoscopic image display device with stereoscopic digital information display function and method of displaying stereoscopic digital information in stereoscopic image WO2012165692A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020110051471A KR101242551B1 (ko) 2011-05-30 2011-05-30 입체 디지털 정보 표시 기능을 갖는 입체 영상 디스플레이 장치 및 입체 영상에서 입체 디지털 정보 표시 방법
KR10-2011-0051471 2011-05-30

Publications (1)

Publication Number Publication Date
WO2012165692A1 true WO2012165692A1 (en) 2012-12-06

Family

ID=47259525

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2011/004601 WO2012165692A1 (en) 2011-05-30 2011-06-23 Stereoscopic image display device with stereoscopic digital information display function and method of displaying stereoscopic digital information in stereoscopic image

Country Status (2)

Country Link
KR (1) KR101242551B1 (ko)
WO (1) WO2012165692A1 (ko)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060065159A (ko) * 2004-12-10 2006-06-14 (주)제니텀 엔터테인먼트 컴퓨팅 3차원 마커 인지 방법 및 이를 이용한 증강현실 및복합현실 제공 방법 및 장치
KR20090041804A (ko) * 2007-10-24 2009-04-29 중앙대학교 산학협력단 3차원 가상 신발 착용 시스템 및 이의 운용방법
KR20090087545A (ko) * 2008-02-13 2009-08-18 광주과학기술원 심플 프레임 마커를 이용하는 증강현실 시스템 및 그 방법,상기 방법을 구현하는 프로그램이 기록된 기록매체
KR20090087807A (ko) * 2008-02-13 2009-08-18 세종대학교산학협력단 증강 현실을 구현하는 방법 및 이를 이용하는 장치

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20050083352A (ko) * 2004-02-23 2005-08-26 주식회사 넥스페이스 휴대용 단말장치에서 스테레오 카메라를 이용하여 파노라믹 영상과 3차원 영상을 획득 및 디스플레이를 할 수 있는 장치 및 그 방법.

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060065159A (ko) * 2004-12-10 2006-06-14 (주)제니텀 엔터테인먼트 컴퓨팅 3차원 마커 인지 방법 및 이를 이용한 증강현실 및복합현실 제공 방법 및 장치
KR20090041804A (ko) * 2007-10-24 2009-04-29 중앙대학교 산학협력단 3차원 가상 신발 착용 시스템 및 이의 운용방법
KR20090087545A (ko) * 2008-02-13 2009-08-18 광주과학기술원 심플 프레임 마커를 이용하는 증강현실 시스템 및 그 방법,상기 방법을 구현하는 프로그램이 기록된 기록매체
KR20090087807A (ko) * 2008-02-13 2009-08-18 세종대학교산학협력단 증강 현실을 구현하는 방법 및 이를 이용하는 장치

Also Published As

Publication number Publication date
KR20120133014A (ko) 2012-12-10
KR101242551B1 (ko) 2013-03-19

Similar Documents

Publication Publication Date Title
WO2018012888A1 (en) Method and system for interactive transmission of panoramic video
WO2018174535A1 (en) System and method for depth map
WO2018182321A1 (en) Method and apparatus for rendering timed text and graphics in virtual reality video
WO2011139070A2 (en) Method and apparatus for recognizing location of user
WO2012077922A2 (en) 3 dimensional (3d) display system of responding to user motion and user interface for the 3d display system
WO2011136407A1 (ko) 스테레오 카메라를 이용한 영상인식장치 및 방법
CN106898049A (zh) 一种用于混合现实设备的空间匹配方法及系统
WO2011105671A1 (ko) 증강 현실을 이용한 사용자 설명서 제공 시스템 및 방법
WO2011152634A2 (ko) 모니터 기반 증강현실 시스템
EP3590090A1 (en) Method and apparatus for processing omni-directional image
WO2013025011A1 (ko) 공간 제스처 인식을 위한 신체 트래킹 방법 및 시스템
WO2015008932A1 (ko) 증강현실에서의 원격 협업을 위한 디지로그 공간 생성기 및 그를 이용한 디지로그 공간 생성 방법
WO2012133962A1 (ko) 스테레오 카메라를 이용한 3차원 동작 인식장치 및 인식방법
WO2011136405A1 (ko) 3ｄ 카메라를 이용한 영상인식장치 및 방법
WO2018110810A1 (ko) Vr 컨텐츠 생성 시스템
WO2023163376A1 (ko) 가상협업 비대면 실시간 원격 실험 시스템
WO2017213335A1 (ko) 실시간 영상 합성 방법
WO2012165692A1 (en) Stereoscopic image display device with stereoscopic digital information display function and method of displaying stereoscopic digital information in stereoscopic image
WO2019103193A1 (ko) 분산 가상 카메라를 이용한 게임 내 360 vr 영상 획득 시스템 및 방법
WO2018230971A1 (en) Method and apparatus for processing omni-directional image
WO2022265347A1 (en) Three-dimensional scene recreation using depth fusion
WO2019103192A1 (ko) 가상 카메라를 이용한 게임 내 360 vr 영상 획득 장치 및 방법
WO2017204598A1 (ko) 촬영된 영상에 관한 데이터 프로토콜을 설정하는 단말기 및 방법
WO2020184890A1 (ko) 2차원 카메라를 이용하여 객체 제어를 지원하는 방법, 시스템 및 비일시성의 컴퓨터 판독 가능 기록 매체
WO2020130211A1 (ko) 관절 모델 정합 장치 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11867087

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11867087

Country of ref document: EP

Kind code of ref document: A1