KR20090028673A - Virtual studio posture correction machine - Google Patents

Abstract

A virtual studio posture correction apparatus is provided to characterize motion data in which the motions of various experts are stored, synchronize the posture of a user with the motions in real time within a 3D space, and then show the synthesized posture image of the user. Cameras(240-24N) photographs a subject(270) which has a background(260) and dances. A chroma-key device(280) performs the chroma-key for the photographed picture, and then positions the real subject at an arbitrary point of a 3D graphic virtual space(320) having the size of a camera screen. An acceleration graphic device(200) fills the background processed by the chroma-key device with a background obtained by synchronizing character 3D data and 3D background data through a virtual camera prepared in a 3D image generator, in order to form a virtual image screen.

Description

Virtual Studio Posture Correction Machine

The present invention relates to a posture correcting apparatus, and more particularly, to a virtual studio posture correcting apparatus capable of correcting posture while comparing a character and his / her movement in a 3D apparatus.

Exercisers who practice their own motion or posture in general, use their cameras to take pictures of themselves, and then repeatedly check and compare them to improve their skills.

In particular, as a portable posture exerciser, a golf swing brace, a baseball swing exerciser, a walking exerciser of a model, an expression exerciser of a performer, a dancer's dance exerciser, an educational facial expression brace, and the like are widely used.

For example, the golf swing posture corrector is designed to enhance the golf skills by visually confirming the shortcomings of the exercisers by reproducing, analyzing, and comparing the golfer's swing shape or posture. In particular, golfers' posture correction is very important in golf. Because, if the posture is incorrect, not only greatly affects the direction of the golf ball, but also the reach of the golf ball falls short of the parish distance of the golf club. In addition, an incorrect swing posture causes excessive swinging, which causes damage to the waist, neck, and legs, which are important parts of the human body.

Like this, it is easy to record dynamic movements that occur momentarily in the entertainment field where the dynamic movements such as golf training, model walking, performer's acting, and dancer's dancing are very important. You need a device that can be easily practiced repeatedly.

As an example of such a portable posture corrector, a technique of JP 2003-0082203 (name, portable posture exerciser capable of recording images) is introduced.

Referring to FIG. 1, a portable system having an LCD display panel 10 at a front side, a hinged pedestal 20 at a rear side thereof, and a digital built-in camera 30 disposed at an upper portion of the panel is a TV, a video player, and an external type. It is equipped with the function of real-time recording which records the video contents of external video playback devices such as CD player, DVD player, cable broadcasting, computer, cable TV and satellite broadcasting for the necessary time and uses them as educational materials.

Looking at the function, by selecting and recording the image of the external video playback device and taking a picture of the user by using the digital built-in camera 30, to compare the recorded video and his own image to correct the posture Device.

However, the conventional dance exerciser as described above is not only difficult to find or store the image that the user needs, but even if stored, it is difficult to compare and correct the motion of the image in real time.

In particular, learning and dancing requires a variety of complex postures, so there is a limit to learning it online. In particular, even if enrolled in a dance academy offline, it is not easy to learn the dance movements in detail, and it is not easy to understand even if you see each movement repeatedly.

The present invention is to solve the above problems, characterizes the motion data that stores the movements of various experts, and compares the posture of the character and their posture by showing their posture synthesized in three-dimensional space in real time The purpose is to provide a virtual studio posture correction device that can be corrected while.

A virtual studio posture correcting apparatus using 3D real-time character and graphics technology to achieve the above object comprises a camera for photographing a subject; A chroma key device that systematically removes specific components of an image from an image output from the camera and outputs only an actual subject image; A 3D accelerated graphics device for forming a 3D virtual studio image and synthesizing the image with a subject image obtained by the chroma key device; And a display device for outputting the synthesized image from the 3D accelerated graphic device, wherein the 3D accelerated graphic device is made by using motion capture data of a professional dance dancer, and a character composed of DBs for various dances to music. 3D DB, and a background 3D DB constructed as a DB so that a person who dances or learns can select their preferred background and the content of the dance.The 3D virtual studio is based on the information of the character 3D DB and the background 3D DB. 3D image generating device for generating a; And video texturing an image of a subject acquired through the chroma key device, and synthesizing it with a 3D virtual studio including a character image and a background image formed by the 3D image generating apparatus, and outputting the image to the display apparatus. To render; characterized in that it comprises a.

According to another embodiment of the present invention, the 3D image generating apparatus is not only a professional dancer, but also a character 3D DB composed of a DB made by using motion capture data of various professionals such as athletes, models, and performers, and a user's selection. It may include a background 3D DB that can be changed according to, and may be provided to create a 3D virtual studio based on the information of the character 3D DB and the background 3D DB.

The virtual studio posture correcting apparatus according to the present invention configured as described above comprises at least one virtual camera whose position is adjusted according to preset track information; A screen control to select the virtual camera; And a joystick for adjusting the screen information of the 3D virtual studio by changing the position information of the camera and the virtual camera, wherein the screen image and the background image and the image of the subject are selected using the screen control and the joystick. By combining in the space, it is possible to synthesize the images of the 3D virtual studio and the subject in real time by 3D rendering the information obtained by moving and placing the combined image into a signal input through the user's joystick using the renderer. Do.

The display device may be connected to a karaoke system to correspond to a user's selection, and may output data stored in the 3D accelerated graphic device, or may be connected to a golf practice system and selected by a user from data stored in the 3D accelerated graphic device. It is also possible to output the swing data of the professional athlete with the swing posture of the user.

The present invention uses the virtual space synthesis technology to input a real video of a subject through a real camera into a graphic generator and show it with characters dancing in a three-dimensional space, so that the actual dancing person appears to be in a virtual space. In addition, by adjusting the tracks of various virtual cameras composed of various positions and angles in which the scene of the virtual space is placed in the virtual space, the dancer can learn to dance or enjoy the dance even more while watching his movement and the movement of the character together. It can be. In particular, the use of video effects or dancing by characters can be used as a back dancer, so that not only can you learn to dance, but you can also make fun dancing and singing equipment and video recording or real-time Internet broadcasting equipment.

For example, when applied to the karaoke system, the dance movement of the singer who sings the song selected according to the user's selection is output on the screen as a 3D character and is output along with the chroma keyed user's appearance, so that the dance movement of the singer can be easily learned. Can be.

In addition, when applied to the golf swing practice device, the user can select the swing data of the professional player that you want to learn, you can check the simulation compared to the swing motion of the user through the screen, it is possible to improve the learning effect.

First, the present invention basically uses the configuration of the virtual studio apparatus described in the application No. 2005-11164 filed by the applicant (name: 3D virtual studio apparatus using video texturing).

FIG. 2 is a diagram showing the main functional block diagram of the 3D virtual posture correcting apparatus of the present invention, which uses a virtual overlay technique. Virtual overlay is a technology that realizes virtual overlay by processing video as a single object in the 3D rendering engine, rather than superimposing the background and front picture on the camera in the existing mixer.

The accelerated graphic device 200 includes a 3D image generating device 210 and a renderer 220.

The 3D image generating apparatus 210 is an editing program such as 3D Studio Max or MAYA, which is a program for designing a 3D object. The 3D image generating apparatus 210 processes 3D data of a previously prepared moving image, for example, the character 3D data 214 or the background 3D data 216. The device 212 can be used to form 3D data that can be output in real time.

According to the present invention, the information input to the character 3D data 214 may be variously recorded according to a use purpose.

According to an embodiment using the present invention as a dance exerciser, the character 3D data 214 is made by using motion capture data of a professional dance dancer, and provides various dances as a database for music. In particular, the DB can be built so that the user can select a variety of characters and at the same time view the characters from various angles so that the user can select and learn a desired posture. In addition, a plurality of characters may be composed of a back dancer character dancing in a three-dimensional synthesis in the virtual screen.

The background 3D data 216 constructs a database so that a person who dances or learns can select his or her favorite background, character, and content of the dance. Since the movement of the virtual camera is pre-stored in the selected background, the dancer is configured to automatically generate a special effect or camera movement scheduled for each time during the dance process. For example, when a user selects a background of a night club, lighting and camera moving may be automatically performed during the dancing process, and a plurality of back dancers may automatically perform the dancing effect together. Such a background can also be configured by the user differently from the default setting through editing.

In addition, even in a narrow space in which the present invention is installed, it is also possible to freely store a large dance studio, a virtual real world (natural scenery such as Niagara Falls), or a fantastic space such as outer space as graphic data.

The renderer 220 receives the 3D data output from the 3D image generating apparatus 210 and outputs the 3D image by real-time video texturing on the surface of the 3D object. In addition, the 3D image and the 3D image input from the real cameras 204 ˜ 24N may be 3D rendered and virtually overlayed to output an image in real time. That is, the virtual overlay technology simultaneously inputs a 3D camera image photographing a subject output from the plurality of cameras 240 to 24N and a 3D image formed of 3D data of the 3D image generator 210 to the renderer 220, The analog image 230 is output by processing the 3D image.

In the present invention, since it is a core technology to select the background DB or the character DBs and display them in the virtual space together with the subject, the contents of the DBs need not be particularly limited.

For example, the character 3D data 214 and the 3D background data 216 may be provided as model walking practice data or data for sports posture correction such as golf swing practice, and may be applied to karaoke (karaoke). It is also possible to arrange stage and dancer characters to accompany the song accompaniment.

An embodiment in which the 3D virtual posture correcting apparatus using the video texture as described above is applied to a dance exerciser will be described in detail with reference to FIG. 3.

All components on the left side of the virtual device partition line 300 of FIG. 3 are set graphic devices installed in the virtual space.

That is, the 3D graphic virtual of the camera screen size is taken by photographing the dancing subject 270 having the background 260 from the actual cameras 240 to 24N and chromating the captured screen with the chroma key device 280. Position it at any point in the space 320. The chroma key function is a technique of photographing a subject having a background with an actual camera and deleting only the background to make it blank, and a detailed description thereof will be omitted.

The accelerating graphic apparatus 200 uses the virtual camera 310 provided in the 3D image generating apparatus 210 to set the background of the character 3D data 214 and the 3D background on the background processed by the chroma key apparatus 280. The data 216 is synthesized and filled with the backgrounds 330 and 340 formed to form a virtual video screen. The renderer 220 combines the camera subject video information 270 together with the background 330 and 340 of the virtual camera 310 to be input as a text of 3D data, and displays the final output screen 250 to display the subject. Process as if you are dancing with the character 330 in the virtual space.

Meanwhile, in order to automatically show various motions captured from the virtual camera 310 in a virtual space by a program, the virtual camera 310 is similar to a PTZ (Pan / Tilt / Zoom, horizontal / vertical / zoom) camera. It is also possible to drive. For example, by automatically tracking a person who sings or dances, the background image and the characters may be linked together with the moving object of the song or dance to be automatically displayed on the entire screen. In addition, low-cost systems can be configured as manual or fixed rather than automatic tracking.

Such manipulation of the virtual camera 310 may use a screen control, a joystick, or a predefined track in 3D data device.

For example, the screen controller constituting a screen capable of selecting a specific one from a plurality of virtual cameras and a preset background image (such as a background screen or a character image) may be formed as a touch screen. In this case, when the user adjusts the selected virtual camera by touching the screen, the adjustment data is input to the virtual camera 480 to adjust the angle of the virtual camera and the like.

In order to simplify the adjustment, a joystick may be further provided. When the joystick is used, the image of the virtual background selected by the screen controller and the image of the subject are track information of the virtual camera selected according to the adjustment position of the joystick. Can be finely adjusted using the control of the joystick, the user can adjust the image to be synthesized at the desired camera angle in the 3D space.

Therefore, by using the 3D video taxiing technology and outputting the image of the observer at any position in the 3D space through the virtual camera, the subject is easily positioned in the virtual world using the 3D graphics device and the virtual camera. Can be displayed in different positions as well as can change the screen in various directions in real time, as well as create a moving effect of the camera without the use of expensive camera movement devices.

In addition, since these virtual cameras can be located in a plurality, it is possible to configure various screens as if they are represented by several cell cameras.

When the captured image information of the virtual camera 310 is input to the 3D image generating apparatus 210, the background 340 according to the location information of the virtual camera 310 is created in the virtual space, and the chroma keyed dancing subject is input in real time. It synthesizes with the video information of 270. In addition, the dancing 3D characters 330 of the 3D characters are added to the image of the subject synthesized with the background 340, and the screen is output to the monitor 250.

As described above, when the video texturing technique is used, the user can adjust the position of the virtual camera 301 freely, so that the actual cameras 240 to 24N can be moved, which makes the video screen of a cool virtual studio. You can make

After all, a person who wants to learn to dance can learn to dance at home by mimicking the motion of a 3D character who dances based on the data acquired through motion capture of a dance expert. You can expect the same effect as learning to dance.

As described above, the present invention chroma key a virtual screen composed of background 3D data constituting a background in a virtual space and character 3D data dancing in real time, and video of a subject viewed by a plurality of cameras input in real time. The real screen from which the background is removed can be output in real time to the scene shown at the virtual camera position moving from a specific position to a specific track in a 3D virtual space. Therefore, the dancer can be displayed on the monitor as his scene as if dancing with the virtual characters in the virtual dance studio.

In addition, when the virtual camera 310 is configured to be automatically tracked according to the movement of the person dancing, the person dancing in a three-dimensional space by embodying a wonderful camera movement is like a professional dancer in a large broadcast studio or concert hall. You can also feel like you're dancing with the cameras, and the cameras make you feel like you're shooting with multiple cameras, and you can automatically create a music video that learns and dances.

Although the present invention has been described in detail with the practice of dance as an example, it is also applicable to the field of entertainment in which dynamic movements such as sports training, model walking, acting, etc. are important every week.

For example, when connected to a karaoke system, the choreography of the singer and back dancer suitable for the selected song is stored in a database, and when the user selects the song, the user displays the selected song so that the characters can be matched to the song the user sings. You can visually see the dance. In addition, since the motion of the user who sings the song is output by chromic, it is possible to compare the dancing motion of the current singer with the song and the actual motion of the singer through a character displayed on the screen.

In addition, when used in connection with the golf swing practice system, the golf swing data of the professional player selected by the user can be output as a character on the screen, so that the user's swing motion and the swing motion of the selected professional player can be compared through simulation, Accurate posture correction is possible.

In addition, it is also possible to continuously provide users with a variety of information that changes rapidly according to the fashion when the DB of the motion capture of the expert group such as singer and professional athlete is updated at regular intervals.

While the above has been shown and described with respect to preferred embodiments of the present invention, the present invention is not limited to the specific embodiments described above, it is usually in the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

The present invention is applicable to entertainment devices such as posture correction and dance practice using virtual studio technology, and particularly, it is preferable to apply to a golf swing exerciser that provides a three-dimensional screen and a karaoke system.

1 is a configuration of a dance practice using a typical dance recording and analysis device,

2 is a main functional block diagram of a 3D virtual posture correcting apparatus of the present invention, and

3 is a view showing the application to the dance exerciser using the 3D virtual posture correction device according to the present invention.

<Description of the symbols for the main parts of the drawings>

200; Accelerated Graphics Device 210: 3D Video Generator

220; Render 230; Analog video

240-24N;

300; Virtual device block line

Claims (5)

A camera for photographing a subject; A chroma key device that systematically removes specific components of an image from an image output from the camera and outputs only an actual subject image; A 3D accelerated graphics device for forming a 3D virtual studio image and synthesizing the image with a subject image obtained by the chroma key device; And And a display device for outputting the synthesized image from the 3D accelerated graphic device. The 3D accelerated graphics device, It is made by using the motion capture data of professional dance dancers, and it is constructed as a character 3D DB composed of DBs for various dances, and a DB that allows the dancers or learners to select their preferred background and content of dances. 3D image generating device including a background 3D DB, generating a 3D virtual studio based on the information of the character 3D DB and the background 3D DB; And Rendering the image of the subject obtained through the chroma key device by video texturing and synthesizing it with a 3D virtual studio including a character image and a background image formed by the 3D image generating apparatus and outputting the image to the display device. Virtual studio posture correction device comprising a. A camera for photographing a subject; A chroma key device that systematically removes specific components of an image from an image output from the camera and outputs only an actual subject image; A 3D accelerated graphics device for forming a 3D virtual studio image and synthesizing the image with a subject image obtained by the chroma key device; And And a display device for outputting the synthesized image from the 3D accelerated graphic device. The 3D accelerated graphics device, Character 3D DB consisting of a DB made by using the motion capture data of professionals such as actors, athletes, models, etc., and background 3D DB that can be changed according to the user's selection, including the character 3D DB and background 3D DB 3D image generating device for generating a 3D virtual studio based on the information; And Rendering the image of the subject obtained through the chroma key device by video texturing and synthesizing it with a 3D virtual studio including a character image and a background image formed by the 3D image generating apparatus and outputting the image to the display device. Virtual studio posture correction device comprising a. The method according to claim 1 or 2, At least one virtual camera whose position is adjusted according to preset track information; A screen control to select the virtual camera; And Joystick for adjusting the screen of the 3D virtual studio by changing the location information of the camera and the virtual camera, further comprising: Using the screen control and the joystick, the selected character image and the background image and the image of the subject are combined in a 3D space, and the rendered image is moved and placed into a signal input through the user's joystick. Virtual studio posture correction device, characterized in that by synthesizing the image of the 3D virtual studio and the subject and output in real time by 3D rendering. The method of claim 1, The display apparatus is connected to the karaoke system, corresponding to the user's selection, the virtual studio posture correction device, characterized in that for outputting the data stored in the 3D accelerated graphics device. The method of claim 1, The display apparatus is connected to a golf practice system, virtual studio posture correction device, characterized in that for outputting the swing data of the professional player selected by the user among the data stored in the 3D accelerated graphics device with the user's swing posture.

Images