KR20010081193A - 3D virtual reality motion capture dance game machine by applying to motion capture method - Google Patents

Abstract

PURPOSE: A dancing game device of a three dimensional virtual reality type is provided to improve posture of a user through displaying a correct posture and a posture of the user on one screen. CONSTITUTION: A music manage unit(110) includes a music store unit(160) storing music data, a music engine(170) for detecting a selected music data, and a speaker(180). A motion manage unit(120) includes a motion store unit(190) storing image data, and a motion engine(200) for detecting a selected image data. An image manage unit(130) includes an image input camera(210) for inputting motion of a user, an image process unit(220) simplifying the motion of the user, an image sampling unit(230) sampling characteristics of the image data, and an image synthetic unit(240) synthesizing the sampled image data for displaying on a screen. A dancing game device manage unit(140) includes a dancing game device interface(260) and a dancing game device engine(270). A result manage unit(150) includes an image compare unit(280) comparing the detected image data with the sampled image data, a result analyze unit(290) analyzing a compared result, and a point output unit(300).

Description

3D virtual reality motion capture dance game machine by applying to motion capture method}

The present invention relates to a dance game device of a 3D virtual reality technique using a motion capture function, and in particular, a motion to enable realistic learning by using a virtual reality technique and a motion capture function that captures and represents a human motion itself. The present invention relates to a dance game device of a 3D virtual reality technique using a capture function.

Conventional dance game apparatuses and video learning apparatuses display previously stored contents, music, and images on a screen connected to the apparatus at the request of the apparatus user. After that, the user can recognize the contents, music, and image displayed on the screen by visual, auditory, or the like, and can be made aware by performing the appropriate operation by the user himself.

In addition, motion capture technology that captures human motion as it is, inputs numerical motion data into a computer, and moves a character using the input motion data is widely used in movies, games, and broadcast animations. Examples of broadcasting animations include 'Pangfang' by Korea Broadcasting Corporation, 'Congsilyi' by cultural broadcasting, and 'Lulu Lara' by Seoul Broadcasting.

In addition, the motion capture technology can be easily found on the screens of the dance game apparatus and the video learning apparatus.

However, in the related art, the dance game apparatus and the video learning apparatus have a unidirectional characteristic of previously stored contents, music, and images. Therefore, it is difficult to determine whether the dance, posture, form, etc. of the user of the apparatus are correct. have.

In addition, the conventional dance game device is a monotonous feeling after a certain degree of mastery by installing a footrest on the floor surface and then perform the operation corresponding to the instruction (for example, arrow) displayed on the image on the footrest on the footrest There is a problem to have.

In addition, the conventional dance game device is configured to operate the game using only the lower body, so it is difficult to expect the user's learning effect in terms of posture correction.

Therefore, an object of the present invention is to be stored in the storage unit of the dance game device is standardized to appear on the screen, the screen displaying the correct posture and the screen of the user of the device is displayed on one screen together It is to provide a dance game device of a three-dimensional virtual reality method using a motion capture function that can be improved by the user compared to each other.

In addition, another object of the present invention is to provide a dance game apparatus of a three-dimensional virtual reality technique using a motion capture function that can be learned by learning through all parts of the body, not limited to only one part of the body. .

In addition, another object of the present invention is a dance game device of a three-dimensional virtual reality technique using a motion capture function that is not monotonous and interesting posture correction by allowing learning and learning through all parts of the body. To provide.

1 is a view showing a general configuration of a dance game device of a three-dimensional virtual reality technique using a motion capture function.

2 is a diagram illustrating a general concept of a game information service utilizing a motion capture function.

3 is a diagram illustrating a computer screen configuration to which a dance game device of a 3D virtual reality technique using a motion capture function is applied.

4 is a diagram illustrating a configuration of outputting a score on a computer screen to which a dance game device of a 3D virtual reality technique using a motion capture function is applied.

<Explanation of symbols for the main parts of the drawings>

190: motion storage unit 230: image input camera

210: image synthesizer 280: image comparator

In order to achieve the above object, according to an aspect of a preferred embodiment of the present invention, in the dance game device using the three-dimensional virtual reality technique, the dance, posture, Storing an image including at least one of the forms; Capturing and processing an image of a game user using an image input camera and a motion capture system; Extracting a necessary image from the captured motion picture; Comparing the stored image with the extracted image; Analyzing the compared results; Scoring the analyzed results to facilitate discrimination; The present invention provides a dance game apparatus using a 3D virtual reality technique using a motion capture function, the method comprising displaying the scored analysis result on a screen.

The method may further include classifying and displaying the image on a screen.

In addition, the motion capture system may include at least one of an ultrasonic system, a prosthetic system, a magnetic system, and an optical system.

The ultrasonic method is composed of a sensor for generating a plurality of ultrasonic waves and three ultrasonic receivers (Receiver). Ultrasonic sensors attached to each joint of the performer sequentially generate ultrasonic waves, and calculate the distance from the sensor to the receiving unit using the time taken for the ultrasonic waves to be received by the receiving device. The three-dimensional position of each sensor can be obtained by the triangulation principle using the values calculated at each of three receivers.

The ultrasonic method has a small amount of calculation required for position measurement without receiving much interference from the environment and thus can be processed in real time and has a low cost.

The prosthetic system is a prosthetic motion capture system is composed of a complex consisting of a potentiometer and a slider (Slider) for measuring the joint movement of the actor.

Also, since the system does not have a receiving device, it is an absolute measuring device that is not subjected to any interference or influence of other environments. This requires little initial calibration and no expensive studio environment equipment. The device also has the advantage of being able to acquire motion data at a very high sampling frequency and at a lower cost than other devices.

In general, the prosthetic system is used in combination with existing animation production keyframe and motion control methods to generate keyframes or step motions.

The magnetic system attaches a sensor capable of measuring a magnetic field to each joint of the performer, and calculates a change in the magnetic field measured by each sensor as a spatial change amount when the actor moves near the magnetic field generating device, and measures the movement. That's the way. Each sensor, magnetic field generating device, and main body are connected by cable. Recently, a wireless system is showing a line.

In addition, the advantages of the magnetic system are relatively inexpensive, have a good price performance ratio, enable real-time processing, and are suitable for capturing motion moving in real time within a radius of 1 to 3 m.

The optical system attaches reflective markers reflecting infrared rays to the main segment of the performer, and generates two-dimensional coordinates of the reflected markers using a camera equipped with 3 to 16 infrared filters. Two-dimensional coordinates captured by multiple cameras are analyzed by a dedicated program and used for calculation and animation in three-dimensional space.

Since the optical system has a high sampling frequency, it is useful when capturing very fast movements such as the movement of a sports athlete. The small size and light weight of the marker attached to the performer allows the performer to express the movement freely. It can also process multiple actors at the same time.

In addition, it can capture more space (552.5m) than other systems and is superior to other methods in terms of capture accuracy.

According to another aspect of a preferred embodiment of the present invention, in the dance game device using the three-dimensional virtual reality technique, including at least one of the dance, posture, form of celebrities including at least one of the entertainer, athlete, model Displaying an image on a portion of a screen; And capturing the motion of a game user using an image input camera and a motion capture system to process the image, and then separating and displaying the extracted image on another part of the screen. The present invention provides a screen configuration method of a dance game device using a 3D virtual reality technique.

In addition, in order to simultaneously display images of two or more game users on the screen, by using an image input camera and a motion capture system, the motion of each game user is captured and processed for image extraction. The method may further include displaying images separately on different portions of the screen.

According to another preferred embodiment of the present invention, in the dance game device using the three-dimensional virtual reality technique, in order to enable competition with other game users through a communication network, outside the computer of the first game user (Game) user Capturing and processing an image of a first game user using a mounted image input camera and a motion capture system; Extracting a necessary image from the captured motion picture; Transmitting the extracted first game user's image to a central server; Receiving the transmitted image at a central server; Capturing and processing a motion of a second game user using an image input camera and a motion capture system mounted outside the computer of the second game user; Extracting a necessary image from the captured motion picture; Transmitting the extracted image of the second game user to a central server; Receiving the transmitted image at a central server; Transmitting the received image of the first game user to a computer of a second game user by the central server, and transmitting the received image of the second game user to a computer of a first game user by the central server; Receiving an image of the transmitted second game user at the computer of the first game user; Receiving an image of the transmitted first game user at the computer of the second game user; Displaying the received image of the second game user on a screen of the first game user; The present invention provides a dance game apparatus of a 3D virtual reality technique using a motion capture function, comprising displaying the received image of the first game user on the screen of the second game user.

The method may further include storing an image including at least one of a dance, a posture, and a form of a celebrity including at least one of a celebrity, an athlete, and a model; Comparing the stored image with the extracted image of the first game user; Analyzing the compared results; Scoring the analyzed results to facilitate discrimination; Transmitting the scored analysis result to a central server; Transmitting the analysis result received at the central server to a computer of a second game user; Comparing the stored image with the extracted image of the second game user; Analyzing the compared results; Scoring the analyzed results to facilitate discrimination; Transmitting the scored analysis result to a central server; Transmitting the analysis result received at the central server to a computer of a first game user; Displaying analysis results of the first game user and analysis results of the received second game user on a computer screen of the first game user; And displaying the analysis result of the second game user and the analysis result of the received first game user on the computer screen of the second game user.

According to another aspect of a preferred embodiment of the present invention, in the dance game device using the three-dimensional virtual reality technique, includes at least one of the celebrity dance, posture, form including at least one of the entertainer, athlete, model The storing of the video may include downloading and updating new data in the form of a file through a communication network, updating and receiving new data from a game user, updating the input by a celebrity demonstration, and demonstrating a game user. It may be updated with new video data, including at least one of the methods of updating by input.

Hereinafter, a dance game apparatus of a 3D virtual reality technique using a motion capture function according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the following examples.

FIG. 1 is a diagram illustrating a general configuration of a dance game apparatus of a 3D virtual reality technique using a motion capture function.

Referring to FIG. 1, a music manager 110, a motion manager 120, an image manager 130, a dance game manager 140, and a result manager 150 may be included.

The music manager 110 includes a music storage unit 160, a music engine 170, and a speaker 180.

Prior to describing the apparatus, the term 'engine' as used herein refers to a part of making a decision about how to manipulate and manage data processed by a program by actually operating a specific task.

For example, the Database Engine decides how to access a database to search for information based on a query received from a user, and how the Search Engine on the Internet searches and manages information on the Internet. It refers to the part of the decision or the system itself.

The music storage unit 160 stores sound data for outputting a sound suitable for a user's request of the device, and the music engine 170 stores a music storage unit (3) to retrieve sound data according to the user's request. 160 determines a method of approaching, wherein the speaker is a device that represents the sound data retrieved by the user's request as a sound.

The speaker is classified in the strict sense as a speaker unit and a speaker system, and the speaker system is a kind of set that collects several speaker units, arranges them in a ring, and has terminals for connecting to an amplifier. Hereinafter, the speaker system will be referred to as a speaker.

In addition, the motion manager 120 includes a motion storage 190 and a motion engine 200.

The motion storage unit 190 stores image data for outputting an image suitable for a user's request of the device on the screen, and the motion engine 200 moves the motion to retrieve the image data according to the user's request. Determine how to access the storage 190.

In addition, the image manager 130 may include an image input camera 210, an image processor 220, an image extractor 230, an image synthesizer 240, and an image display unit 250.

The image input camera 210 takes a role of photographing the user's motion and inputting the 3D video data.

In addition, the image processor 220 simplifies the data of the 3D video input from the image input camera 210 to assist in the processing of PATTERN RECOGNITION or MACHINE INTELLIGENCE.

In addition, the image extracting unit 230 is a content-based method of extracting a feature that can represent the content of the image data and index and search based on it. Content-based indexing and retrieval is based on human mental processes that remember and reunite information.

For example, humans remember people through their descriptions of their opponents, such as small stature, dark hair, blue eyes, and big nose. The height, hair color, blue eyes, and big nose are the content or characteristics of the subject.

Similarly, extracting such description content from an image sequence and performing image labeling is called content-based indexing. This method automatically extracts feature information from image data and uses it in the indexing process, which has the advantage of reducing time and manpower consumption.

In addition, the image synthesizing unit 240 synthesizes the image data retrieved by the motion engine 200 and the image data extracted by the image extracting unit 230 to be displayed on the computer screen of the user. .

The synthesis process supports a temporal and spatial relationship, and may have a multi-step structure composed of three elements such as a frame, a shot, and a scene to facilitate a search.

The frame represents one individual image corresponding to one film as a minimum unit constituting a video, and the shut represents an image photographed by one camera. In the shut, the film is continuous without interruption and the scene is continuous. It consists of a series of shuts and shows images taken in succession of one object, such as a user or a specific place.

In addition, the image synthesizing unit 240 may uniquely convert the image input by the image input camera 210 by applying an animation work or the like.

The image display unit 250 displays the image data retrieved by the motion engine 200 and the image data extracted by the image extractor 230 on the computer screen of the user.

In addition, the dance game machine manager 140 includes a dance game machine I / F (Interface) 260 and a dance game machine engine 270.

Prior to describing the device, the term 'I / F (Interface)' as used herein essentially means the boundary between two components, the functional unit, the contact surface, or the shared area, and the boundary between human and machine. Refers to a system, device, standard, or protocol designed to be easy for humans to handle.

In addition, the interface refers to a protocol when connecting hardware or software in relation to a computer. For example, when a computer and a printer are connected, data cannot be exchanged unless their specifications are the same, and output is made by a computer command only when the specifications are the same. At this time, "the same interface is provided between the two devices."

In addition, there are various types of I / F (Interface) such as input / output interface, man machine interface, system call interface, communication protocol, etc. Among them, man machine interface refers to a keyboard or display directly related to a computer and a human using it. .

The dance game machine I / F (Interface) 260 is a protocol for connecting the user's computer system and the dance game machine engine 270, and the dance game machine engine 270 processes data processed by the result manager 150. Make decisions about how to operate and manage.

In addition, the result manager 150 includes an image comparator 280, a result analyzer 290, and a score outputter 300.

The image comparator 280 compares similarities and differences between the image data retrieved by the motion engine 200 and the image data extracted by the image extractor 230, and the result analyzer 290 The compared result is analyzed by predetermined criteria. The criterion may be configured to include a certain error range.

In addition, the score output unit 300 displays a score corresponding to the analysis result of the result analyzer 290 on the user's computer screen. The score displayed on the screen of the computer may be expressed by numbers, letters, symbols, and the like.

2 is a diagram illustrating a general concept of a game information service using a motion capture function.

2, the motion capturer 410, the information service system 420, the data storage unit 430, the image input cameras 440 and 470, the computers 450 and 480, and the speakers 460 and 490. Include.

Parts overlapping with the parts described with reference to FIG. 1 will be omitted.

Video data for each body part, such as a broadcast entertainer or a famous sports player, is input through the motion capturer 41O.

The information service system 420 stores image data input from the motion capturer 410 and sound data corresponding to the image data in the data storage unit 430, and is responsible for authentication process of a game user.

In addition, the data newly stored in the data storage unit is transmitted to the computer of each user through a communication network, and when using a plurality of users through the communication network to interconnect the computer of the plurality of users.

The data storage unit 430 stores image data input through the motion capturer 410 and sound data corresponding to the image data. The data storage unit 430 may be separated and installed by separating the image data and the sound data, respectively, or may be integrated and installed as one.

The computers 450 and 480 display image data stored in the storage unit and image data of a game user input by the image input cameras 440 and 470 on a screen, and sound data corresponding to the image data. Is output through the speakers 460 and 490.

The process described above will be described by taking an example in which images and sounds related to a specific entertainer are stored in the music storage unit 160 and the motion storage unit 190.

First, 3D image data is generated by using a motion capture device of the performer, and music data is generated with sound corresponding to the image.

The music data can be applied in the form of MP3, WAV, MIDI and the like.

Thereafter, the generated image data and sound data are separately stored in each storage unit.

The storing may include converting the character into a character shape so that the actual display of the actor's actual appearance does not cause a problem such as a portrait right.

After that, the format of the virtual stage for the performer's performance is developed and stored in the storage unit.

Through the above steps, a character performing the same action as the performer may be displayed as if performing in a 3D virtual stage.

The displayed screen configuration may be displayed as a full screen configuration of the performer only, a split screen configuration of the performer and the user, and a user full screen configuration.

The game user then wears a motion capturer or sits in front of the image input camera at the user's personal or commercial location.

In addition, the dance game apparatus may be used by connecting a footstep (for example, DDR Pad) for a conventional dance game.

Thereafter, a specific virtual stage format is selected from the storage unit in which the format of the virtual stage is stored.

Thereafter, when a specific actor is selected from the list of performers, the motion of the performer is displayed on the screen, and music is played through the user's computer speaker.

The user then sees the performer's motion displayed on the screen, listens to music, and the user follows the appropriate motion.

Thereafter, after the user's motion is terminated, the actor's motion data stored in the storage unit and the motion data of the user are compared and analyzed, and then a differential score is assigned and displayed on the screen according to the result of the comparative analysis.

In addition, in the above process, a plurality of users may simultaneously play a game through a communication network.

The images of the plurality of users can be simultaneously displayed on the computer screen of each user through user management of the information service system 420.

3 is a diagram illustrating a computer screen configuration to which a dance game device of a 3D virtual reality method using a motion capture function is applied, and FIG. 4 is a computer to which a dance game device of a 3D virtual reality method is used using a motion capture function. It is a figure which showed the structure at the time of a score output on a screen.

Referring to FIG. 3, it can be seen that the screen 510 of the computer may be divided into two parts.

The divided portions 520 and 580 of the divided screens 510 and 570 may display motions of broadcast entertainers and famous sports players stored in the motion storage unit 190, and other portions 530 and 590. May display in real time a 3D moving image of the game user 550 inputted through the image input cameras 210, 440, 470, 540, and 610.

The display contents of the two parts can be exchanged with each other.

In addition, the screen configuration may be divided into two or more parts when a plurality of users participate in the game at the same time through the communication network.

Referring to FIG. 4, when the user 550 recognizes a screen such as 510 and proceeds with the game and ends the game, the user 550 may output the result of the game user 550 as described in FIG. 1. The analysis result is displayed on the screen 570 in the form of a score by comparing and analyzing the motion data stored in 190.

The score may be displayed by a number, a letter, a symbol, or the like.

In addition, as a method of displaying the score, not only a method of displaying the score in the predetermined portion 600 of the screen as shown in FIG. 3 but also a method of displaying the score in a part of the image display areas 530 and 590 of the user. .

In addition, when a plurality of users play a game at the same time through the communication network, it is also possible to display in the order of ranking the users of the highest score.

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

Therefore, the present invention can make the virtual reality technology for three-dimensional character animation advance one step by using motion capture technology to show realistic motion through realistic three-dimensional image.

In addition, it acts as a stimulus for the development of computer vision-based motion capture technology that does not attach a marker or sensor to recognize a human's body naturally without any inconvenience such as attaching a marker or a sensor or restricting a user's activity. can do.

In addition, the stereotyped and displayed on the screen stored in the storage unit of the dance game device, and the screen displaying the correct posture and the screen of the user of the device is displayed on one screen together so that the user The images can be compared and improved.

In addition, it is not limited to only one part of the body can be learned by learning through all parts of the body.

In addition, it can learn through the whole part of the body, and can be learned through the image, so that it is not monotonous, and it is possible to interestingly learn and correct posture.

In addition, the motion capture game is highly dimensional and future-oriented by using three-dimensional virtual reality technology, which is a completely different field from the two-dimensional game method using the conventional scaffolding, and aims for defense, posture correction and medical rehabilitation. It can be used in fields such as exercise and posture correction, broadcasting, film, animation production, game production, educational materials for various sports centers, and educational materials for education.

Claims (8)

In the dance game device using the 3D virtual reality technique, Storing an image including at least one of a dance, a posture, and a form of a celebrity including at least one of a celebrity, an athlete, and a model; Capturing and processing an image of a game user using an image input camera and a motion capture system; Extracting a necessary image from the captured motion picture; Comparing the stored image with the extracted image; Analyzing the compared results; Scoring the analyzed results to facilitate discrimination; Displaying the scored analysis result on a screen Dance game device of the three-dimensional virtual reality method using a motion capture function comprising a. The method of claim 1, Motion capture system, Comprising at least one of an ultrasonic system, a prosthetic system, a magnetic system, an optical system Dance game device of the three-dimensional virtual reality using the motion capture function characterized in that. The method of claim 1, Dividing and displaying the image on a screen; Dance game device of the three-dimensional virtual reality method using a motion capture function comprising a. In the dance game device using the 3D virtual reality technique, Displaying an image including at least one of a dance, a posture, and a form of a celebrity including at least one of a celebrity, an athlete, and a model on a portion of a screen; Capturing the motion of a game user using an image input camera and a motion capture system to process an image, and then separating and displaying the extracted image on another part of the screen Screen configuration method of the dance game device of the three-dimensional virtual reality method using a motion capture function comprising a. The method of claim 4, wherein In order to simultaneously display images of two or more game users on the screen, Using an image input camera and a motion capture system, capturing the motion of each game user and processing the image, and separately displaying the extracted image on different portions of the screen. Screen configuration method of the dance game device of the three-dimensional virtual reality method using a motion capture function comprising a. In the dance game device using the 3D virtual reality technique, In order to enable competition with other game users through the communication network, Capturing and processing an image of a first game user's movement by using an image input camera and a motion capture system mounted outside the computer of the first game user; Extracting a necessary image from the captured motion picture; Transmitting the extracted first game user's image to a central server; Receiving the transmitted image at a central server; Capturing and processing a motion of a second game user using an image input camera and a motion capture system mounted outside the computer of the second game user; Extracting a necessary image from the captured motion picture; Transmitting the extracted image of the second game user to a central server; Receiving the transmitted image at a central server; Transmitting the received image of the first game user to a computer of a second game user by the central server, and transmitting the received image of the second game user to a computer of a first game user by the central server; Receiving an image of the transmitted second game user at the computer of the first game user; Receiving an image of the transmitted first game user at the computer of the second game user; Displaying the received image of the second game user on a screen of the first game user; Displaying the received image of the first game user on a screen of the second game user; Dance game device of the three-dimensional virtual reality method using a motion capture function comprising a. The method of claim 6, Storing an image including at least one of a dance, a posture, and a form of a celebrity including at least one of a celebrity, an athlete, and a model; Comparing the stored image with the extracted image of the first game user; Analyzing the compared results; Scoring the analyzed results to facilitate discrimination; Transmitting the scored analysis result to a central server; Transmitting the analysis result received at the central server to a computer of a second game user; Comparing the stored image with the extracted image of the second game user; Analyzing the compared results; Scoring the analyzed results to facilitate discrimination; Transmitting the scored analysis result to a central server; Transmitting the analysis result received at the central server to a computer of a first game user; Displaying analysis results of the first game user and analysis results of the received second game user on a computer screen of the first game user; Displaying an analysis result of the second game user and an analysis result of the received first game user on a computer screen of the second game user; Dance game device of the three-dimensional virtual reality method using a motion capture function comprising a. In the dance game device using the 3D virtual reality technique, Storing an image including at least one of a dance, posture, and form of a celebrity including at least one of a celebrity, an athlete, and a model may include: At least one method of downloading and updating new data in the form of a file through a communication network, updating and receiving new data from a game user, updating by input by a celebrity demonstration, or updating by input by a game user's demonstration. Update to new video material including Dance game device of the three-dimensional virtual reality using the motion capture function characterized in that.