WO2009035199A1 - Machine de correction de maintien en studio virtuel - Google Patents

Machine de correction de maintien en studio virtuel Download PDF

Info

Publication number
WO2009035199A1
WO2009035199A1 PCT/KR2008/002304 KR2008002304W WO2009035199A1 WO 2009035199 A1 WO2009035199 A1 WO 2009035199A1 KR 2008002304 W KR2008002304 W KR 2008002304W WO 2009035199 A1 WO2009035199 A1 WO 2009035199A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
virtual
virtual studio
unit
background
Prior art date
Application number
PCT/KR2008/002304
Other languages
English (en)
Inventor
Young-Dae Kim
Original Assignee
Young-Dae Kim
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020080036592A external-priority patent/KR100956454B1/ko
Application filed by Young-Dae Kim filed Critical Young-Dae Kim
Publication of WO2009035199A1 publication Critical patent/WO2009035199A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/2224—Studio circuitry; Studio devices; Studio equipment related to virtual studio applications

Definitions

  • the present invention relates to a posture correction machine, and more particularly, to a virtual studio posture correction machine which enables a user to correct his or her own posture by comparing his or her own motion and a character's motion in a three- dimensional (3D) graphic device.
  • a posture correction machine and more particularly, to a virtual studio posture correction machine which enables a user to correct his or her own posture by comparing his or her own motion and a character's motion in a three- dimensional (3D) graphic device.
  • Portable posture training devices are mainly used in golf swing posture correctors, baseball swing training devices, model walking training devices, acting expression training devices, dance training devices, facial expression correctors for education, or in other similar devices.
  • a golf swing posture corrector captures the swing posture of someone practicing their golf stroke, he or she may play back, analyze and compare the video captured by the golf swing posture corrector to check his or her own posture, and thus may improve his or her golf stroke.
  • it is very important to correct golfer's swing posture because incorrect posture affects the direction a golf ball travels and may cause the golf ball not to reach the full distance of a golf club.
  • incorrect swing posture may cause damages to the waist, neck or legs, which are important parts of the human body.
  • Korean Patent Laid-Open No. 2003-0082203 entitled 'portable posture training device capable of recording video' describes the following technology.
  • a portable system includes a liquid crystal display (LCD) display panel 10 disposed on a front side thereof, a hinged support 20 disposed on a rear side thereof, and a built-in digital camera 30 disposed above the panel 10.
  • the portable system has an instant recording function to record video content output from an external video player, such as a television, a video player, an external compact disc (CD) player, a digital versatile disc (DVD) player, a cable broadcasting apparatus, a computer, a cable television or a satellite broadcasting apparatus, for a required period of time and to utilize the recorded video as education teaching materials.
  • an external video player such as a television, a video player, an external compact disc (CD) player, a digital versatile disc (DVD) player, a cable broadcasting apparatus, a computer, a cable television or a satellite broadcasting apparatus, for a required period of time and to utilize the recorded video as education teaching materials.
  • the portable system is able to select and record video content output from the external video player, and to capture a user's figure using the digital camera 30, so that a user can correct his or her own posture by comparing it with the recorded video.
  • the present invention provides a virtual studio posture correcting device, which causes motion data regarding motion of various professionals to be displayed as a three-dimensional (3D) character, synthesizes a user's posture with the character in 3D space and shows a result of synthesizing in real time, so that the user is able to correct his or her own posture by comparing it to the character's posture.
  • 3D three-dimensional
  • a virtual studio posture correction machine including a camera which photographs a subject; a chroma key unit which systematically removes certain image components from an image output from the camera and outputs only a subject image; a three-dimensional (3D) acceleration graphic unit which forms a 3D virtual studio image and synthesizes the 3D virtual studio image with the subject image acquired by the chroma key unit; and a display unit which displays an image synthesized by the 3D acceleration graphic unit.
  • the 3D acceleration graphic unit may include a 3D image generator which generates a 3D virtual studio using information regarding a character 3D database (DB) and a background 3D DB, the 3D image generator having the character 3D DB, which is constructed using motion capture data of a professional dancer and which contains various dance moves matching the music, and the background 3D DB, which is constructed so as to enable a person who desires to dance or learn dancing to select a favorite background and dance move; and a rendering unit which performs video texturing on the subject image acquired by the chroma key unit, and synthesizes the textured image with the 3D virtual studio, which is generated by the 3D image generator and includes a character image and background image, and which outputs a synthesized image to the display unit.
  • DB character 3D database
  • background 3D DB which is constructed using motion capture data of a professional dancer and which contains various dance moves matching the music
  • the background 3D DB which is constructed so as to enable a person who desires to dance or learn dancing to select
  • the 3D image generator may generate a 3D virtual studio using information regarding a character 3D DB and a background 3D DB.
  • the 3D image generator may have the character 3D DB, which is constructed using motion capture data of a professional, such as a professional dancer, a sportsman, a model or an actor, and the background 3D DB, which is changeable according to user selections.
  • the virtual studio posture correction machine may further include at least one virtual camera, a position of which is adjusted according to preset track information; a screen control through which the at least one virtual camera is selectable; and a joystick which changes position information of the camera and at least one virtual camera and which adjusts a screen of the 3D virtual studio. If a character image and background image selected using the screen control and joystick is combined with the subject image in 3D space, the rendering unit may perform 3D rendering on information regarding movement and arrangement of a combined image in response to a signal input by a user using the joystick, may synthesize the 3D virtual studio with the subject image, and may output a synthesized image in real time.
  • a virtual space synthesis technology for showing an actual video of a subject input to a graphic generator by a real camera together with three-dimensional (3D) characters dancing in 3D space is used, so a user who actually dances appears to be in virtual space. Additionally, the position and angles of a plurality of virtual cameras on virtual space may be adjusted, and accordingly the user can learn to dance or improve his or her dance moves while viewing his or her own posture together with the character's posture. Video effects or characters are used as background dancers, so the present invention is equally applicable to a machine for learning to dance or dancing or singing, or a machine for recording video or real-time Internet broadcasting.
  • dance moves of a singer of a song selected by a user may be performed on a screen by a 3D character, and a user's posture may also be displayed using a chroma key, so it is possible to learn the dance moves of a singer easily.
  • a user may select golf swing data of a professional player whose stroke he or she desires to learn, and may view a simulation to compare his or her own posture to a professional player's posture on a screen, and it is thus possible to learn more effectively.
  • FIG. 1 illustrates a dance training device employing a general dance recording and analyzing apparatus
  • FIG. 2 is a block diagram showing main function blocks of a three-dimensional (3D) virtual posture correcting device according to an exemplary embodiment of the present invention.
  • FIG. 3 illustrates a situation in which a 3D virtual posture correcting device according to an exemplary embodiment of the present invention is applied to a dance training device.
  • a three-dimensional (3D) virtual posture correcting device is configured based on a virtual studio apparatus described in Korean Patent Application No. 2005-0011164 entitled '3D virtual studio apparatus using video texturing' by DARIM VISION CO. LTD.
  • FIG. 2 illustrates main function blocks of the 3D virtual posture correcting device according to the exemplary embodiment of the present invention.
  • a virtual overlay technology is used in which video is processed as a single subject in a 3D rendering engine rather than background images overlapping foreground images in a conventional mixer, in order to create a virtual overlay.
  • An acceleration graphic unit 200 includes a 3D image generator 210 and a rendering unit 220.
  • the 3D image generator 210 may generate 3D data capable of being output in real time, using prepared moving images, for example a character 3D database (DB) 214 or a background 3D DB 216, which are processed by a 3D data processor 212, by launching a program for designating 3D subjects, namely an editing program such as 3D Studio Max or MAYA.
  • DB character 3D database
  • DB background 3D DB
  • 3D data processor 212 by launching a program for designating 3D subjects, namely an editing program such as 3D Studio Max or MAYA.
  • the character 3D DB 214 is constructed using motion capture data of a professional dancer.
  • the character 3D DB 214 contains various dance moves matching the music.
  • the character 3D DB 214 is constructed enabling a user to select various characters which may be viewed from different angles, so that the user can select a desired posture and learn dancing. Additionally, a plurality of characters may be displayed as background dancers which are dimensionally synthesized on a virtual screen.
  • the background 3D DB 216 is constructed so that a user who desires to dance or learns dancing may select a desired background, character and dance move. Since the selected background contains pre-stored information on the movement of a virtual camera, preset special effects or the movement of the virtual camera may be performed automatically for each moment when the user dances. For example, if the user selects a background of a nightclub, the lighting and virtual camera may move automatically and the effect for dancing together with a plurality of background dancers may be provided automatically when the user dances. In this situation, the user may set the background differently from the basic settings by editing the background.
  • the background 3D DB 216 may store graphic data representing a large-scale dance studio or a virtual real world (for example, a natural setting such as Niagara Falls) or a fantastic setting such as outer space even when the 3D virtual posture correcting device of the present exemplary embodiment is mounted in a narrow space.
  • a virtual real world for example, a natural setting such as Niagara Falls
  • a fantastic setting such as outer space even when the 3D virtual posture correcting device of the present exemplary embodiment is mounted in a narrow space.
  • the rendering unit 220 receives the 3D data generated by the 3D image generator
  • the rendering unit 220 performs video texturing on a surface of a 3D subject in real time, and outputs a 3D image. Additionally, the rendering unit 220 performs 3D rendering on the 3D image together with 3D camera images captured by a plurality of real cameras 240 to 24N, and outputs images in real time using the virtual overlay technology.
  • the virtual overlay technology enables the 3D camera images representing subjects captured by the plurality of real cameras 240 to 24N and the 3D image having the 3D data generated by the 3D image generator 210 to be input simultaneously to the rendering unit 220, and processed as an analog image 230, so that the analog image 230 may be displayed.
  • the main technique of this exemplary embodiment of the present invention is that the selected background DB and character DB are displayed together with subjects in virtual space, so there is no need to limit information regarding the DBs.
  • the character 3D DB 214 and background 3D DB 216 may contain data for model gait training or data for correcting sports posture, such as a person's golf stroke, or may contain data regarding characters representing dancers or stages matching the music to be applied to a karaoke.
  • FIG. 3 illustrates a set graphic device in which all units to the left of a virtual device dividing line 300 are placed in a virtual space.
  • a chroma key unit 280 causes only the subject 270 in the captured image to appear to be in any location of a 3D graphic virtual space 320 having the same size as a screen of a virtual camera 310.
  • Such a chroma key function is a technique for removing only a background from a subject displayed together with the background captured by a real camera and for making the background transparent. Since the chroma key function is known to those skilled in the art, detailed description thereof is omitted.
  • the acceleration graphic unit 200 replaces the background which has been made transparent by the chroma key unit 280 with the 3D graphic virtual space 320 and background 340, which are formed by synthesizing the character 3D DB 214 with the background 3D DB 216, so that a virtual screen may be generated, using the virtual camera 310 included in the 3D image generator 210.
  • the rendering unit 220 synthesizes video information regarding the subject 270 with the 3D graphic virtual space 320 and background 340, so that a synthesized image is input as texture of the 3D data, and then displays a final output image 250, so that the subject 270 appears to dance together with a character 330 in the virtual space.
  • the virtual camera 310 may operate as a Pan/ Tilt/Zoom (PTZ) camera. For example, when a person who sings a song or dances is automatically tracked, the virtual camera 310 may combine the person with a certain background and characters and may cause a combined image to automatically appear on the entire screen. Additionally, in the case of a low-cost system, manual or stationary tracking may be performed in place of automatic tracking.
  • PTZ Pan/ Tilt/Zoom
  • the virtual camera 310 may be controlled by a screen control or a joystick or a predefined track in a 3D data device.
  • the screen control forms a screen through which a specific virtual camera is able to be selected from among a plurality of virtual cameras and preset background images (background screens or character images).
  • the screen control may be implemented by a touch screen. In this situation, if a user adjusts a camera selected by touching the touch screen, adjustment data obtained by adjusting the camera is transmitted to a virtual camera 310, so that the angle of the virtual camera 310 or the like may be adjusted.
  • a joystick is additionally required.
  • the position of background image and subject image selected using the screen control may be precisely controlled using track information of the virtual camera selected according to the adjusted position of the joystick. Accordingly, the user is able to control the position of images to be combined at a desired camera angle in 3D space.
  • a plurality of virtual cameras may be disposed in virtual space, so it is possible to form diverse screens as if a plurality of real cameras operate.
  • a background 340 is formed according to position information of the virtual camera 310 in virtual space and the background 340 is synthesized with video information regarding the dancing subject 270 acquired in real time by the chroma key unit 280.
  • the 3D character 330 selected from among 3D characters is then added to the image of the subject 270 synthesized with the background 340, so that a composite screen may be displayed on a monitor 250.
  • the user may freely adjust the position of the virtual camera 310, so that the effect may appear as if the plurality of real cameras 240 to 24N move, and superior virtual studio video may be generated.
  • a virtual image including background 3D data forming a background in virtual space and 3D data of a continuously dancing character is synthesized with video of a subject captured by a plurality of real cameras.
  • the video of the subject is obtained by removing a background using the chroma key function.
  • the subject then appears to be in a location viewed by a virtual camera which moves in a certain direction according to a specific track in 3D virtual space, so that a composite screen may be displayed in real time. Therefore, a dancing person may view his or her own posture on a monitor as if he or she is dancing together with virtual characters in a virtual dance studio.
  • the dancing person feels as if he or she is dancing together with professional dancers in a large-scale broadcasting studio or concert hall, and as if a plurality of cameras are moving and photographing him or her.
  • it is possible to automatically generate a music video by which people can dance or learn dancing.
  • the dance training device has been described as an exemplary embodiment of the present invention, the present invention is also applicable to the entertainment field in which dynamic motions, such as sports posture, a model's gait, or an actor's form, are very important.
  • a dance composition of a singer or background dancers suitable for a selected song is stored in a database. If a user selects the song, the dance composition is displayed, so that the user may view characters which dance to the song sung by the user. Additionally, the moves of the user singing the song is displayed after the chroma key function is executed, so it is possible for the user to compare his or her own dance moves to the original moves of the singer using the characters displayed on a screen.
  • the virtual posture correcting device is connected to a golf swing posture training device, golf swing data of a professional player selected by a user may be displayed on a screen as a 3D character, so that the user may view a simulation to compare his or her own posture to the posture of the selected professional player through the screen, and it is possible to accurately correct the posture.
  • the present invention is applicable to an entertainment apparatus such as a posture correcting device or a dance training device, using virtual studio technology, and in particular, may desirably be applied to a golf swing posture training device and a karaoke system which are able to provide a 3D screen.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente invention concerne une machine de correction de maintien en studio virtuel au moyen d'un personnage et d'une technologie graphique en temps réel en trois dimensions. La machine mélange et affiche des images capturées conjointement avec un personnage ou un arrière-plan dans des images tridimensionnelles au moyen d'un réglage des effets, de sorte qu'une personne qui danse puisse danser ou apprendre à danser tout en observant son maintien conjointement avec le maintien du personnage à l'aide de la machine.
PCT/KR2008/002304 2007-09-15 2008-04-23 Machine de correction de maintien en studio virtuel WO2009035199A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR20070093909 2007-09-15
KR10-2007-0093909 2007-09-15
KR10-2008-0036592 2008-04-21
KR1020080036592A KR100956454B1 (ko) 2007-09-15 2008-04-21 가상 스튜디오 자세 교정 장치

Publications (1)

Publication Number Publication Date
WO2009035199A1 true WO2009035199A1 (fr) 2009-03-19

Family

ID=40452179

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2008/002304 WO2009035199A1 (fr) 2007-09-15 2008-04-23 Machine de correction de maintien en studio virtuel

Country Status (1)

Country Link
WO (1) WO2009035199A1 (fr)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102118568A (zh) * 2009-12-30 2011-07-06 新奥特（北京）视频技术有限公司 一种用于体育比赛的图形生成系统
CN102118567A (zh) * 2009-12-30 2011-07-06 新奥特（北京）视频技术有限公司 一种分离模式的虚拟体育系统
CN103413353A (zh) * 2013-07-31 2013-11-27 天脉聚源(北京)传媒科技有限公司 一种资源展示方法、装置和终端
CN103959805A (zh) * 2013-10-24 2014-07-30 华为终端有限公司 一种显示图像的方法和装置
US8989521B1 (en) 2011-11-23 2015-03-24 Google Inc. Determination of dance steps based on media content
JP2015192299A (ja) * 2014-03-28 2015-11-02 株式会社エクシング 楽曲演奏装置及びプログラム
CN105873208A (zh) * 2016-06-06 2016-08-17 杭州同步科技有限公司 一种虚拟演播室系统中的命令传递方法
CN106210453A (zh) * 2016-08-09 2016-12-07 安徽喜悦信息科技有限公司 一种智能虚拟演播室系统
CN107509007A (zh) * 2017-08-09 2017-12-22 南京理工大学 一种虚拟演播室同步信号分离系统
CN108259780A (zh) * 2018-04-17 2018-07-06 北京艾沃次世代文化传媒有限公司 用于虚拟片场抗干扰特效视频同步显示方法
CN108540731A (zh) * 2018-04-17 2018-09-14 北京艾沃次世代文化传媒有限公司 实拍摄影机与虚拟场景实时同步显示方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR970058060A (ko) * 1995-12-14 1997-07-31 김광호 3차원 입체영상 합성방법
JPH09247532A (ja) * 1996-03-01 1997-09-19 Aba Internatl:Kk 画像合成方法および装置
KR20050105308A (ko) * 2004-04-28 2005-11-04 주식회사 다림비젼 삼차원 가상 스투디오를 이용한 가라오케
KR20060090367A (ko) * 2005-02-07 2006-08-10 주식회사 다림비젼 비디오 텍스투어링을 이용한 삼차원 가상 스튜디오 장치

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR970058060A (ko) * 1995-12-14 1997-07-31 김광호 3차원 입체영상 합성방법
JPH09247532A (ja) * 1996-03-01 1997-09-19 Aba Internatl:Kk 画像合成方法および装置
KR20050105308A (ko) * 2004-04-28 2005-11-04 주식회사 다림비젼 삼차원 가상 스투디오를 이용한 가라오케
KR20060090367A (ko) * 2005-02-07 2006-08-10 주식회사 다림비젼 비디오 텍스투어링을 이용한 삼차원 가상 스튜디오 장치

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102118567A (zh) * 2009-12-30 2011-07-06 新奥特（北京）视频技术有限公司 一种分离模式的虚拟体育系统
CN102118568B (zh) * 2009-12-30 2015-02-18 新奥特（北京）视频技术有限公司 一种用于体育比赛的图形生成系统
CN102118567B (zh) * 2009-12-30 2015-06-17 新奥特（北京）视频技术有限公司 一种分离模式的虚拟体育系统
CN102118568A (zh) * 2009-12-30 2011-07-06 新奥特（北京）视频技术有限公司 一种用于体育比赛的图形生成系统
US8989521B1 (en) 2011-11-23 2015-03-24 Google Inc. Determination of dance steps based on media content
CN103413353B (zh) * 2013-07-31 2016-09-14 天脉聚源(北京)传媒科技有限公司 一种资源展示方法、装置和终端
CN103413353A (zh) * 2013-07-31 2013-11-27 天脉聚源(北京)传媒科技有限公司 一种资源展示方法、装置和终端
CN103959805A (zh) * 2013-10-24 2014-07-30 华为终端有限公司 一种显示图像的方法和装置
WO2015058388A1 (fr) * 2013-10-24 2015-04-30 华为终端有限公司 Procédé et dispositif d'affichage d'une image
US10283005B2 (en) 2013-10-24 2019-05-07 Huawei Device Co., Ltd. Image display method and apparatus
JP2015192299A (ja) * 2014-03-28 2015-11-02 株式会社エクシング 楽曲演奏装置及びプログラム
CN105873208A (zh) * 2016-06-06 2016-08-17 杭州同步科技有限公司 一种虚拟演播室系统中的命令传递方法
CN106210453A (zh) * 2016-08-09 2016-12-07 安徽喜悦信息科技有限公司 一种智能虚拟演播室系统
CN107509007A (zh) * 2017-08-09 2017-12-22 南京理工大学 一种虚拟演播室同步信号分离系统
CN108259780A (zh) * 2018-04-17 2018-07-06 北京艾沃次世代文化传媒有限公司 用于虚拟片场抗干扰特效视频同步显示方法
CN108540731A (zh) * 2018-04-17 2018-09-14 北京艾沃次世代文化传媒有限公司 实拍摄影机与虚拟场景实时同步显示方法

Similar Documents

Publication Publication Date Title
WO2009035199A1 (fr) Machine de correction de maintien en studio virtuel
US11132533B2 (en) Systems and methods for creating target motion, capturing motion, analyzing motion, and improving motion
US10821347B2 (en) Virtual reality sports training systems and methods
KR102077108B1 (ko) 콘텐츠 체험 서비스 제공 장치 및 그 방법
EP0669758B1 (fr) Processeur d'images variant dans le temps et dispositif d'affichage
KR101304111B1 (ko) 댄싱 가라오케 시스템
US20070122786A1 (en) Video karaoke system
US10049496B2 (en) Multiple perspective video system and method
US20170036106A1 (en) Method and System for Portraying a Portal with User-Selectable Icons on a Large Format Display System
KR100956454B1 (ko) 가상 스튜디오 자세 교정 장치
US20200244891A1 (en) Method to configure a virtual camera path
JP3526897B2 (ja) 画像表示装置
CN111930223A (zh) 用于观看计算机生成的环境并与其互动的可移动显示器
US20230353717A1 (en) Image processing system, image processing method, and storage medium
JP7387286B2 (ja) 情報処理装置、情報処理方法、及びプログラム
KR20190074911A (ko) 실감형 영상 콘텐츠 제공 방법 및 이를 이용한 서버
TWM433876U (en) Fitness equipment combined with geographic information system
JP3629558B2 (ja) 画像練習装置
WO2023026529A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
US20240114181A1 (en) Information processing device, information processing method, and program
JP2015126826A (ja) 情報処理装置及びプログラム
KR100684401B1 (ko) 가상현실 기반의 골프학습 장치, 그 방법 및 그 기록매체
WO2022226745A1 (fr) Procédé photographique, appareil de commande, appareil photographique et support de stockage
JP7403581B2 (ja) システム及びデバイス
JP2023169697A (ja) 情報処理装置、情報処理方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08741539

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25/06/10)

122 Ep: pct application non-entry in european phase

Ref document number: 08741539

Country of ref document: EP

Kind code of ref document: A1