KR100347960B1 - Video game machine and controlling method of it - Google Patents

Abstract

The present invention relates to a video game apparatus and a method for controlling the same, and intelligently detects a player's motion using an image recognition technology, and determines whether the motion matches a predetermined motion, thereby controlling the game.

To this end, the present invention includes an image input means for capturing the movement of the player to convert to a digital signal and a display means for outputting the image, by generating a video object for instructing the predetermined operation by the specified game program A game that controls the continuation or scoring process of the game by recognizing the movement of the player from the image input signal inputted through the image input means while outputting to the display means, and comparing the operation of the recognized player with a predetermined operation. Control means.

Therefore, the present invention proceeds the game while calculating the score whether or not the motion produced by the player matches the motion of the video object, such as aerobic gymnastics, dance lessons, which require posture correction, as well as entertainment and games Taekwondo, gumdo, golf, etc. will be applied to self-learning and correction of all sports.

Description

Video game machine and controlling method of it}

The present invention relates to a video game apparatus capable of playing a game or guiding a player's motion using an image processing and a motion recognition technology through a camera, and more particularly, to scoring a game score according to a motion produced by a player. The present invention relates to a video game apparatus and a control method thereof, which can be applied as a new concept game apparatus having an open space or as a teaching apparatus for correcting a player's posture by controlling the progress of the game by performing a deduction process.

Conventional game machines and game devices are played by a switch operation, and are configured in such a manner that the game is received by receiving a switch operation state, that is, whether the switch operation matches the operation time, for the item indicated by the player. .

However, these conventional game consoles and game devices are mainly configured to play a game in a one-to-one manner, resulting in an environment where users can enjoy electronic games in a closed space or be immersed in obscene sites on the Internet. Because of the lack of development of open spaces and game devices that allow young people to soundly dissipate young youth, their youths are exposed to extremely dangerous tendencies, such as motorcycle runaways, or overly brutal violent games. Not only did they have a violent mentality, but also caused problems such as imitation crimes.

Therefore, in recent years, game devices have been developed in which a game is played in the form of dancing by operating a game device using a foot switch instead of a hand press switch.

However, these game devices are game devices that operate by stepping on the switch according to the on-screen instructions, which are similar to the dancing action, but are actually similar to the exercise equipment that runs rather than dance because it only works by foot, thus aerobic gymnastics or golf There is a need to develop a specific program or a control device that can self-learn or correct sports that require a specific athletic posture, such as dance training, taekwondo, and kendo, in a more open and healthy environment.

Therefore, the present invention has been made to solve the above problems, the present invention is determined by how the player directs to match the predetermined motion expressed through the display means, calculate the score or proceed the player's motion by proceeding the game It is an object of the present invention to provide a video game apparatus and a control method thereof that can meet the demand for new progressive entertainment according to the capability.

In order to achieve the above object, the present invention comprises: image input means for capturing a movement of a player and converting it into a digital image input signal; Create and store a face photo of the player from the image input signal, and generate and output a video object for indicating the movement of the player according to a specified game program while playing a game according to the movement of the player recognized from the image input signal. Game control means for controlling; The game control means stores a game program for each piece of music or genre, and forms a database by collecting motion instruction codes according to the game program. RAM for each coded and stored; Motion extracting means for extracting contour information on a moving object from the image input signal and extracting a moving object separated from a background image using the extracted contour information, and a body part from the moving object separated by the motion extracting means; The motion recognition means for recognizing the characteristics of movement of each area by setting a predetermined inspection area around the recognized body, and connecting the motion instruction code according to the game program. Instruction motion output means for generating a predetermined pattern to be followed, combining the patterns according to the passage of time, and outputting a moving object and comparing whether the movement produced by the player matches the movement of the moving object, and according to the comparison result. Pro consisting of score calculation means for calculating scores A video game device including a parser unit is provided.

In order to achieve the above object, the present invention also comprises the steps of displaying a background image or game genre and a predetermined game level for supporting the game while waiting for the game to start when a coin is put in the screen display state for game guidance; Creating and storing a face picture of the player from the signal input through the image input means when the game starts, and entering a game execution step with a message output informing of the start of the game; Instruction motion output step of generating a certain pattern to be followed by the player by connecting the designated motion instruction code according to the game program, combining the patterns according to the passage of time, and outputting a video object to the moving object from the image input signal. A motion extraction step of extracting contour information for the moving object and a moving object separated from a background image using the extracted contour information, recognizing a body part from the moving object separated in the motion extraction step, and recognizing the body A motion recognition step of recognizing the characteristics of the motion of each area by setting a predetermined inspection area around the center, and comparing the motion of the recognized player with the motion of the video object according to the comparison result. A score calculation step is performed to score or deduct points It provides a video game control method comprising the step of controlling the game.

1 is a circuit block diagram of a video game apparatus according to an embodiment of the present invention.

2 is a diagram illustrating an external configuration of the video game apparatus of FIG.

3 is a screen display example of the video game apparatus of FIG.

4 is a reference diagram for explaining a motion form and a code example according to the movement of the video object classified according to patterns.

5 is a flowchart illustrating an embodiment of a video game control method according to the present invention.

6 is an operation flowchart showing the game execution step of FIG. 5 in detail;

7A to 7F are reference diagrams for explaining signal processing operations in a motion extracting means, a motion recognizing means, an instruction motion output means, and a score calculating means.

<Explanation of symbols for the main parts of the drawings>

11: camera 12: frame grabber

13a, 13b: monitor 14: three-dimensional video card

15: three-dimensional sound card 16: amplifier

17a, 17b: Speaker 18: CD-ROM driver

19: switch panel 20: central processing unit

21: Memory Card 22: Hard Disk

23a, 23b: lighting device 31: stage device

32: player

DESCRIPTION OF THE EMBODIMENTS Hereinafter, embodiments of the present invention apparatus and method will be described in detail with reference to the accompanying drawings.

1 is a circuit block diagram of a video game apparatus according to an embodiment of the present invention, and FIG. 2 is an external configuration diagram showing one configuration example of a video game apparatus.

As shown in the drawing, the apparatus of the present invention is a stage provided with a square or disc that generates a specific sound according to the step of the player stepping at a position spaced apart from the ground (for example, about 5cm-15cm above the ground). Has a device (31). At this time, the surface of the stage device is suitable to finish with a material that does not slide or break by the dance movement, preferably a material used for the basketball court floor. In addition, the stage device has a resonance so that a light hitting sound can be generated every time the step is performed, and a microphone (not shown) is attached to the floor to play an audio bass sound when the step is performed. May be recognized by the game player itself.

The apparatus of the present invention has an image input means comprising a camera 11 for photographing a player on the stage device and a frame grabber 12 for converting an image pickup signal captured by the camera into a digital signal. In this case, the camera is an image input device for transmitting an input image signal to a PC through a frame grabber connected to a personal computer (PC), and the like, and receives a movement of a player on the stage device. For more active games, the camera can be replaced by a pan / tilt type camera, and the direction can be automatically adjusted according to the movement of the player on the stage device. It shall be further provided with a vertical motor and its motor drive means. In addition, the camera uses a digital or analog CCD camera commonly used for surveillance, a camcorder (resolution 640 × 480 pixels or more) may be used, and in addition, an infrared lens may be attached in consideration of indoor lighting. And the installation angle of the camera is preferably installed diagonally on the top of the head in order to accurately capture the face and movement of the player with a single camera, the angle can be considered about 20 degrees-60 degrees with the vertical axis have. In addition, it is possible to attach two cameras using a multi-frame grabber for a two-player video game, and to implement stereoscopic 3D video games by receiving stereo images from two cameras.

The apparatus of the present invention also has auxiliary lighting devices 23a and 23b in order to allow the camera to capture a clearer image signal. In FIG. 2, the auxiliary lighting device is attached to both sides of the display means, but the installation place may be any place where the player's face and movement can be accurately projected, and the installation angle is left and right illumination. Tilt all the devices about 30 degrees downward, and adjust the left side lighting device to 10 degrees to 20 degrees to the right and the right side lighting device to 10 to 20 degrees to the left so that the player on the stage is focused on. In addition, it is preferable that such lighting device uses a dimmer switch that is automatically adjusted by indoor lighting and ambient brightness.

The apparatus may further include a first monitor 13a for outputting a face photograph of the player and a moving picture of an object for indicating a movement of the player, and the image input means. And a second monitor 13b for outputting a video of the player, and a display means composed of a three-dimensional video card 14 for driving the two monitors under computer control. The two monitors correspond to a display screen display area for indicating a specified movement to the player and a play screen display area for showing the player's movement together with a background image or song lyrics. These two display areas are shown in FIG. 3. One monitor can be split and used as one large outline (13c) on the outside, or two monitors can be installed and used as two small outlines (13d) and 13e on the inside. have. In particular, in the case of a two-player video game, space can be reduced by dividing one monitor for each player. In addition, the display means may use a plurality of monitors at the same time through the screen divider to implement a large screen.

The apparatus of the present invention has a speaker system composed of a three-dimensional sound card 15, an amplifier 16, and a plurality of speakers 17a and 17b. This loudspeaker system is capable of magnificent stereoscopic outputs of more than 100W, allowing a wide range of powerful and sound transmission. In addition, the speaker can be connected to the amplifier to enhance the output as well as to enable the stereoscopic sound through the three-dimensional sound card (15).

In addition, the apparatus of the present invention includes a CD-ROM driver 18 capable of reading a game program from a CD-ROM for storing a game program, a switch panel 19 for controlling the start or selection of a game, a stop, a volume control, and a memory. And a game control means composed of a card 21, a hard disk 22, and a central processing unit (CPU) 20. Such game control means may use a 32-bit PC or a main processor unit (MPU) that is scalable, freely attaches peripheral devices, and is economical for maintenance for high speed 3D video or multimedia.

The game control unit captures the movements associated with the dance with a camera or converts the image into a database, receives a digital image signal from the image input unit, creates and stores a face image of the player, and indicates the movement of the player. The game is controlled by recognizing the movement of the player through the image input means while generating an object according to a specified game program and outputting a moving image. In particular, the game control means stores a game program for each piece of music or genre, collects movement instruction codes according to the game program, forms a database in the ROM, and displays each part of the movement object according to the movement instruction code of the ROM. Coded motion patterns (movement shapes as shown in FIG. 4) and patterns of the motion patterns are coded and stored in a RAM, and a desired game program is read from the ROM, and the codes are stored in the RAM according to a game program. And a processor for generating an instruction object and controlling a game by comparing a video of a player extracted from the digital image signal with the movement instruction object.

In particular, the processor may include: motion extracting means for extracting a moving object from the image input signal, motion recognition means for recognizing a characteristic of motion around a body from a moving object separated from the motion extracting means, and the game program; An instruction motion output means for outputting a video object of a certain pattern to be followed by the player, and a score calculation means for comparing the movement of the player with the movement of the video object to score points or deducting points according to the comparison result; It has a configuration to include.

The motion extracting means is a module for extracting a moving object separated from the background by using contour information extracted by using gray value or boundary detection and contour extraction algorithm of an image from an image input from a camera. Register the extracted value (gray level value or contour information) as background information, and then extract the input image information (gray level value or contour information), and extract only the motion by extracting the extracted image information from the background information. do.

The motion recognizing means recognizes the body part having the strongest value in the motion extraction image separated from the motion extracting means, selects an inspection area around the body, and recognizes the characteristics of the motion for each of the selected areas. Store in a coded pattern.

The instruction motion output means is a module that outputs a predetermined video to the screen output device to instruct a player to perform a certain operation in accordance with music, and reads the movement instruction code data from a database that collects movements by songs or genres, and moves Using a database and a pattern database, an object having a pattern matching the movement instruction code is generated, and the objects are combined over time to output a video.

The score calculating means is a module for calculating a score according to whether an action taken by a player (user) matches an action displayed on a screen, and scores or deductions are processed using an action related code and a maximum response time for each action. .

FIG. 5 is an operation flowchart illustrating an embodiment of a video game control method according to the present invention, and illustrates a game control and score processing process executed by the processor. As shown in the figure, when a coin is put in the screen display state for game guidance as a whole, waiting for a game start key to be selected while displaying a background image or game genre and a predetermined level for supporting a game (S401-). S404); When the game start key is selected in the standby state, searching for the face of the player from the input image creates a face picture, and enters a game execution step while outputting a message indicating the start of the game together with the face picture of the game player (S405-). S406); During the game according to the preset game level, the score obtained at the game level is checked at the end of each game level. If the score acquired at each game level is equal to or higher than the reference score, the game time is checked while the next game level is executed. Thereafter, the ranking of the player is registered, and if the reference score is less than or equal to the end of the game with a message indicating the game interruption (S407-S413).

FIG. 6 is a detailed operation flowchart illustrating a game execution step for each level of FIG. 5 executed by the processor, wherein a motion excluding a background screen is separated from an input image while outputting a video according to a motion instruction code corresponding to the level. (S501 and S502), recognizing a body from the separated image, and selecting recognition regions around the recognized body (S503 and S504), and movement in the selected recognition region. Classifying the classified pattern using a motion database by classifying the pattern into patterns (S505 and S506), scoring the pattern code by comparing the pattern code with a motion instruction code, and outputting a comment corresponding to the scored score (S507-S510) and cumulatively calculating the scores scored, determining whether the time has been given to the level, and instructing the next pattern again. Returning to or ending the operation of the level (S511, S512).

7A to 7F are diagrams for explaining signal processing operations in the motion extracting means, the motion recognition means, the instruction motion output means, and the score calculating means of the processor. Same as

First, in order to implement the present invention, the game control means requires a motion database preparation process of capturing motions associated with a movement posture for a desired dance or learning with a camera or receiving and drawing a picture into a database. After that, the camera is connected to the game control means to recognize the player's face and movement using skin color and motion detection and the difference image calculation, and the movement recognized by the camera is compared with the previously prepared and stored motion database. Calculate the score by the comparison result.

That is, in the game control method according to the present invention, the overall operation is executed by the motion extracting means, the motion recognition means, the instruction motion output means, and the score calculating means of the processor. .

First, when coin input is confirmed and a desired game program is started, the processor of the game control means detects the initial position of the player using the image input signal captured by the camera, and then extracts the face image of the player from the image input signal. Make a face photo and save it temporarily.

Thereafter, the registered background image is removed from the image input signal, and only the moving image is extracted, and the player's body is recognized using the image. Next, the peripheral recognition area is set around the body, the movement in the recognition area is analyzed, and the data is coded by comparing with the model of the database through the classification into a predetermined pattern, and the coded data is calculated with the motion indicating code. The work will calculate the score.

At this time, if the game does not meet the score for passing each level by placing the level at a certain time during the game execution, the game is dropped. The ranking registration may display the face image stored at the initial setting with the score, and if the name is registered at the initial setting, the name, the face, and the score may be displayed.

In particular, in the process of extracting a moving object from the image input signal by the processor, as shown in FIG. 7A, contour information extracted by using a gray value or a boundary detection and contour extraction algorithm of an image from an image input from a camera is used. A module for extracting a moving object separated from a background by using a module. The module extracts a value (gray level value or outline information) extracted from a pure background image without motion as background information and then inputs image information (gray level value or outline). Information), and extracts only the movement by extracting the extracted image information from the background information.

The processor also has a motion recognition means for recognizing the characteristics of the movement around the body from the moving object separated by the motion extraction means. As shown in FIG. 7B, the motion recognizing means recognizes a body part having the strongest value in the motion extraction image separated by the motion extracting means, and then a predetermined region (left arm and left leg) The left area, the right area where the right and right legs are located, the upper area where the head is located, the central area where the body and arms are located, and the lower area where the legs are located) Recognize the characteristics of each of the selected motions and store them in a pattern. For example, in the case of motion detection data in which only the left arm is lifted as shown in FIG. 7C, patterns are classified into "upper left arm", "lower right arm", and "out of head", and each pattern is classified into a database. Compare and convert to code

In addition, the processor has an instruction motion output means for outputting a video object of a certain pattern to be followed by the player according to the game program. This instruction motion output means is a module for outputting a predetermined video to the screen output device for instructing a player (user) a certain operation in accordance with music, as shown in FIG. 7D. For example, as shown in FIG. 7E, read out the movement instruction code data (command code) in the form of "in the head", "in the left arm", "lower right arm", "lower left leg", and "lower right leg". The motion indication code is connected using a motion database, an object having a pattern matching the motion indication code is generated using a pattern database, and the objects are combined according to the passage of time to output a video.

In addition, the processor has a score calculation means for comparing the movement produced by the player with the movement of the video object, the score is added or deducted according to the comparison result. The score calculating means is a module for calculating a score according to whether or not the action taken by the player (user) matches the action displayed on the screen. Compared with the code, it is determined whether or not the match is made, and according to the result of the determination, the deciding point or deduction is determined. At this time, the motion instruction code of the instruction motion output means includes two types of data, that is, an operation related code and a maximum response time for each operation. For example, the player must take the action within 0.8 seconds after the "head left" signal is generated, as shown in FIG. 7F, and the player recognizes the score within 0. seconds after the "left arm" signal is generated. To be recognized as a score, the score calculation means calculates the overall operation as a ratio over time to perform the scoring or deduction process. In addition, a penalty will be deducted if an action different from the specified action is taken.

Therefore, the present invention allows the youth to dance or play as much as they can while listening to the cheerful music of their favorite people in an open environment, so that they can digest the erupting youth soundly, and play extremely dangerous or violent games such as motorcycle runaways. In particular, the present invention recognizes a body part, divides it into areas such as the head, arms, and legs around the body, recognizes movement by area, and then combines the patterned motion code. As the game proceeds in contrast to the motion indication signal, the motion recognition and the evaluation of the movement of the player are more accurate and easier.

In addition, the present invention by using the image processing and the motion recognition technology through the camera can be freely implemented in the actual dance using the hands, feet, and the whole body freely without a switch, just entertainment or games, as well as posture correction learning Aerobic gymnastics, dance lessons, taekwondo, kendo, golf, etc. that all require a self-learning and correction of the exercise has the advantage that can be applied.

Claims (24)

delete delete delete delete delete Image input means for capturing the movement of the player and converting it into a digital image input signal; Create and store a face photo of the player from the image input signal, and generate and output a video object for indicating the movement of the player according to a specified game program while playing a game according to the movement of the player recognized from the image input signal. Game control means for controlling; The game control means stores a game program for each piece of music or genre, and forms a database by collecting motion instruction codes according to the game program. RAM for each coded and stored; Motion extracting means for extracting contour information on a moving object from the image input signal and extracting a moving object separated from a background image using the extracted contour information, and a body part from the moving object separated by the motion extracting means; The motion recognition means for recognizing the characteristics of movement of each area by setting a predetermined inspection area around the recognized body, and connecting the motion instruction code according to the game program. Instruction motion output means for generating a predetermined pattern to be followed, combining the patterns according to the passage of time, and outputting a moving object and comparing whether the movement produced by the player matches the movement of the moving object, and according to the comparison result. Pro consisting of score calculation means for calculating scores And a parser unit. The method of claim 6, wherein the motion extracting means, The video game apparatus, comprising: registering image information extracted from a pure background image without motion as background information, and then extracting only the motion image of the player by removing the registered background information from the input image information. The method of claim 6, wherein the motion recognition means, And classifying and storing the motion characteristics of each area set based on the moving object separated by the motion extracting unit for each pattern, and converting and encoding the motion characteristics of each area into a predetermined pattern. The method of claim 8, wherein the motion recognition means, The upper region where the head is located, the middle region where the body and the arm are located, and the lower region where the leg is located, are divided based on the moving object separated by the movement extracting means. A video game device, characterized in that for setting the inspection area. The method of claim 8, wherein the motion recognition means, And encoding the pattern information classified according to the characteristics of the motion of each region by comparing with the model of the motion database. The method of claim 6, wherein the instruction motion output means, From the database that collects movements by song title or genre according to the game program, it reads the code for movement instruction that lists the predetermined movements that the player should follow according to the passage of time. Connect the read motion instruction code to generate an object of a certain pattern according to the same; And a video object by combining the objects of each pattern over time. The method of claim 11, wherein the movement instruction code, An action-related code for specifying a specific action and data relating to the maximum response time for each action, The score calculating means, The coded data and the motion response time generated from the motion extracting means are compared with the motion related code and the maximum response time of the motion instruction code, and the score is calculated by calculating the ratio according to the motion related codes and the response time. Video game device characterized in that the reduction. delete Displaying a background image, a game genre, and a preset game level for supporting a game while waiting for a game to start when a coin is put in a screen display state for game guidance; Creating and storing a face picture of the player from the signal input through the image input means when the game starts, and entering a game execution step with a message output informing of the start of the game; Instruction motion output step of generating a certain pattern to be followed by the player by connecting the designated motion instruction code according to the game program, combining the patterns according to the passage of time, and outputting a video object to the moving object from the image input signal. A motion extraction step of extracting contour information for the moving object and a moving object separated from a background image using the extracted contour information, recognizing a body part from the moving object separated in the motion extraction step, and recognizing the body A motion recognition step of recognizing the characteristics of the motion of each area by setting a predetermined inspection area around the center, and comparing the motion of the recognized player with the motion of the video object according to the comparison result. A score calculation step is performed to score or deduct points And controlling the game. The method of claim 14, wherein the instruction motion output step, Reading a movement instruction code that lists the movements to be followed by the player according to the passage of time from a database of movements by songs and genres according to a game program; Connecting the read motion instruction code to generate an object having a predetermined pattern corresponding thereto; And combining the objects of each pattern over time to output a video object. The method of claim 15, In the motion instruction code database, a motion related code for designating a specific motion and data relating to a maximum response time for each motion are included and stored. In the score calculation step, the coded data and motion response time generated from the motion extracting means are compared with the motion related code and the maximum response time of the motion instruction code, and the ratios according to whether the motion related codes match and response time Video game control method, characterized in that by adding / subtracting the score. The method of claim 14, wherein the motion extraction step, Registering image information extracted from a pure background image having no motion as background information; And removing only the registered background information from the input image information, thereby extracting only a moving image of the player. The method of claim 14, wherein the motion recognition step, The video game control method of claim 1, wherein the motion characteristics of each area set based on the moving object separated in the motion extraction step are classified and stored for each pattern, and the motion characteristics of each area are converted into a predetermined pattern and coded. . The method of claim 14, wherein the motion recognition step, The upper region where the head is located, the middle region where the body and the arm are located, and the lower region where the leg is located, are divided based on the moving object separated in the movement extraction step. A video game control method comprising setting an inspection area. The method of claim 18, wherein the motion recognition step, The video game control method, characterized in that the pattern information classified according to the characteristics of the motion of each area is compared with the model of the motion database coded. The apparatus of claim 6, further comprising: a stage device installed at a position spaced a predetermined distance from the ground to generate a specific sound according to the player's step; And display means for outputting a face photograph of the player created by using the image input signal, a video object for indicating the movement of the player, or movement of the player detected from the image input signal. Game device. 22. The video game apparatus according to claim 21, wherein a microphone for generating a specific audio signal is provided at a predetermined position of the stage apparatus according to the step that the player presses. The method of claim 21, wherein the image input means, And a horizontal / vertical motor and a motor driving means for moving the camera from side to side or up and down by the control of the game control means, the direction being automatically adjusted according to the movement of the player on the stage device. Video game device. The method of claim 21, wherein the processor unit, And determining whether or not the motion produced by the player is consistent with the motion of the video object for directing the motion, and determining whether to score the game score or to continue the game according to the determination result.