KR20060023313A - Image processing method for bodily sensitive game and game method using same - Google Patents

Abstract

The present invention relates to a method and a game apparatus for photographing a user's behavior using an image camera, extracting an image thereof, and detecting a user's movement to play a game. More specifically, the present invention provides a tactile game method in which a user extracts a user's motion captured by an image camera as a three-dimensional vector and interacts with an object in a game according to the user's motion. The present invention relates to a game-type game in which two or more users cooperatively compete with each other using a single image camera and a network-like game using the Internet.

Game, Experience, Image Processing, Camera

Description

Image processing method for haptic game and game method using same {image processing method for bodily sensitive game and game method using same}

1 is a circuit block diagram of a haptic game device according to the present invention.

Figure 2 is an external configuration example of a haptic game device according to the present invention

3 is a block diagram of image processing according to the present invention;

4 is a detailed configuration diagram of an image image processing pipeline which is an image processing technique according to the present invention;

5 is a reference diagram for explaining an image processing technology image image processing pipeline applied to the present invention.

6 is an illustration of a haptic game operation using image processing according to the present invention.

7 is an illustration of a haptic game operation using image processing according to the present invention.

8 is a detailed illustration of the silhouette of FIG.

a) An example of a silhouette image frame over time of the game

b) Example of comparison judgment between user and game silhouette

c) Detailed diagram of the image frame of the game silhouette

9 is an example of a game immersive game operation screen according to the present invention.

Fig. 10 is an exemplary game immersive game operation using image processing according to the present invention, and an exemplary diagram in the case of two-player play.

11 and 12 are diagrams illustrating an immersive game operation using the Internet according to the present invention.

The present invention relates to a method and a game apparatus for photographing a user's behavior using an image camera, extracting an image thereof, and detecting a user's movement to play a game. More specifically, the present invention provides a tactile game method in which a user extracts a user's motion captured by an image camera as a three-dimensional vector and interacts with an object in a game according to the user's motion. The present invention relates to a game-type game in which two or more users cooperatively compete with each other using a single image camera and a network-like game using the Internet.

Conventionally, a typical device for a user to operate a game includes a keyboard, a mouse, a joystick, and the like. The operation devices are universal devices that cannot utilize the characteristics of each game, and can not fully utilize the characteristics of the game in airplane games, car games, and fighting games. All of these devices are also a passive way of sitting in a chair and enjoying themselves statically, making you tired and tired easily when you sit on the chair for a long time.

Recently, there are many haptic games in line with the high quality of the game system and high consumer demand, and are being developed in the future. A game in which you actually make the inside of a car and look at the monitor and race in it, a game in which a gun-like device is used to trigger and shoot with the hand on the monitor's enemy, a game in which you use the ski footslide to climb a high mountain on the monitor, There are a variety of immersive games, such as a fire truck extinguisher to put a fire extinguisher on the monitor and extinguish the fire.

These immersive games require expensive hardware and large space. In other words, due to the high cost, the game price that the user must bear increases, and the large area causes difficulty in arranging various games. In addition, in the Internet service using a video camera, it remains at the level of a simple video chat, and it can be said that it is narrow in terms of its utilization due to the absence of various contents accessible to the user having the video camera. In addition, a chat site using a video camera has been transformed into a closed environment and obscene video chat and is indiscriminately abused by teenagers.

Brief description of the prior art is as follows.

Patent Application No. 39323, issued on September 14, 1999, uses an image recognition technology to intelligently detect a player's motion, determine whether the motion is consistent with a specified motion, and control the game. The way to take is represented as a 3D instructional character, and on the other side, the camera is illuminated through the camera to compare whether the user is the same or wrong. In the implementation method, the background is pre-registered in the image taken by the camera and the background and the player are separated to face the green screen or the wall behind the player. In the method of determining the motion, the head, body, arms, and legs are coded separately to the 3D instruction character, and the video of the player taken through the camera is also coded by separating the head, body, torso, arms, and legs. Take the way of comparison.

According to the present invention, a "first monitor 13a for outputting a face photograph of the player and a moving picture of an object for indicating the movement of the player, which is created and stored from an image input signal, and said player recognized through said image input means; And display means comprising a second monitor 13b for outputting a moving picture and a three-dimensional video card 14 for driving the two monitors under computer control. A display screen display area and a play screen display area showing a player's movement along with a background image or song lyrics, and these two display areas are shown in FIG. 3 (the drawings attached to the specification of the present invention). The drawing serial numbers of the parts are shown in the corresponding prior art documents). You can use one monitor by splitting the screen (shown as one outer big outline (13c)), or install two monitors separately (indicated by two small outlines (13d) and 13e). It can be done. " As described above, the present invention divides the screen using one monitor or two or more monitors to output a user on one side and a motion instruction object video on the other side.

In addition, the motion extracting means is a module for extracting a moving object separated from a background by using contour information extracted by using a gray value or an edge detection and an outline extraction algorithm of an image from an image input from a camera. Register the value (gray level value or outline information) extracted from the pure background image without any information as background information, and then extract the image information (gray level value or outline information) input thereafter, and extract the extracted image information from the background information. To extract only movement. ” Therefore, the present invention involves registering a pure background image, and extracting only a user image obtained by subtracting background information from the extracted image information using a gray value or a boundary detection algorithm and an outline extraction algorithm.

The invention also has a motion recognition means for recognizing the characteristic of the movement around the body from the moving object separated by the motion extracting means. The motion recognition means as shown in Fig. 7b. After recognizing the body part with the strongest value in the motion extraction image separated from, the predetermined area around the body (the left area where the left and left legs are located, the right area where the right and right legs are located, And the upper region where the head is located, the central region where the torso and arms are located, and the lower region where the legs are located, are divided into inspection regions, and the movement characteristics of each of the selected regions are recognized and stored in a pattern. For example, in the case of motion detection data in which the shape of the movement is lifted only by the left arm, as shown in FIG. 7C, the upper left arm, the lower right arm, Patterns are classified as "in the head" and each pattern is compared to a database and converted into a code. "

However, the above design extracts the user without the background from the image taken by the camera, recognizes the body from the image, and separates and codes the left arm, right arm, left leg, right leg, and head from the body and makes a database. . In this method, the body part of the user is obtained from the image extracted by the camera. In the method, the position of the body is different in the process of recognizing the image of the divided five areas as the parts of each body by dividing the screen into five parts. In other words, when the body shape is incorrect and does not fit into the five partitions, the user's body part may not be properly read and coded, and thus the game may not proceed.

The invention also provides, "The processor has an instruction motion output means for outputting a video object of a certain pattern to be followed by a player according to the game program. The instruction motion output means is a player (user) as shown in Fig. 7D. A module for outputting a predetermined video to the screen output device to instruct a certain operation in accordance with music, and from the database that collects movements by song title or genre, for example, "in the head," "in the left arm," Read the motion instruction code data (command code) in the form of "lower right arm", "lower left leg", "lower right leg", connect the motion indication codes using a motion database, and connect the pattern database. By using the object to create a pattern of the pattern that fits the motion instruction code, and combining the objects over time and moving The outputs "discloses that.

However, the object output as a video, which is used as the instruction motion output means of the present invention, is also divided into the codes of the head, left arm, right arm, left leg, and right leg, which separate the body through a database. The reason why the present invention is divided into codes is that the user input through the camera is divided into body part codes, and the instruction motion output is also made by body part codes to compare the two code values.

Next, Patent Application No. 19350, filed July 6, 2000, relates to the process of performing a dance game by capturing the movement of a player with a CCD camera and analyzing image changes in a predetermined area to determine whether the player's touch is normally touched. A predetermined switch is displayed on the screen and the gamer is starting to touch the switch at an appropriate timing.

According to the present invention, in claim 7, "The space touch motion monitoring means includes a CCD camera for photographing a game player, a reference video memory for storing an initial screen photographed by the CCD camera as a reference screen, and photographing by the CCD camera. A video memory for storing a screen as a current time, a touch area memory for finding a touch area due to an image change by contrasting the screens of the reference video memory and the video memory, and a touch of a player during a dance. A touch area setting memory for storing data about a reference touch area for a touch area, a touch area switching calculator for checking whether a touch is made by comparing a touch area of the touch area memory with a touch area of the touch area setting memory, and the touch Deviation amount of touch of game player from output of area switching calculator The detection, and the piece dance game system using the image sensor characterized by comprising: a deviation detection and integrated to obtain the integrated value of the vehicle "is disclosed.

In the above invention, it is based on the first screen shot by the CCD camera, and if the user is taken on the first screen or another image is taken, the result of the game operation cannot be obtained and the game cannot be played anymore. Looks up.

The above-described invention also provides that " a mark provided with the output of the touch position indicating data memory 5 is generated in the mark memory 6 and outputted to the display control section 7 so that the CRT 8 can be controlled by the control of the display control section 7. A touch display for the player's touch instructing the touch timing is output as shown in Fig. 7. In this case, although a simple arrow may be used as in the conventional dance game system, the color change or the shape change of the touch display is not limited thereto. And a touch signal for the player's touch is provided not only to the mark memory 6 but also to the deviation amount detection and integration unit 15. "

In this way, the above-described design follows the progress in which the touch area is generated and displayed in a specific area through a mark memory made of a mark. That is, a method of detecting a position where a user is turned on by turning on / off a specific area among five predetermined areas according to music in five predetermined areas on the screen.

In addition, the patent application No. 0022091 of August 2, 2000 photographs the motion of the human body in the chroma key screen (outside shadow extraction background) and simulates the motion picture character set as the default dance. It is a dance game of the comparison and scoring method, and the chroma key screen is essential for distinguishing the background from the person in the implementation method, and for each method in the method of determining the motion of the video character and the gamer. The skeleton line is extracted and the motions of the predetermined person are compared to the image having only the gamer whose background photographed by the camera is removed.

In the above invention, the image output monitor 11 and the sound source output speaker 14 are positioned on the upper part of the main body 16, and the solid background screen between the column fixed to the side and the pipe frame 11 connected to the steel frame structure of the square shape ( 18. A monochromatic bottom sheet 17 of the same color as the background screen 18 is adhered to the bottom, and a small digital camera 15 equipped with a wide-angle lens is fixed behind the center of the main body 16. The image output monitor 13 is installed in consideration of the eye height of the standard key and is designated by pressing a direction key in the center of the main body 16 with a finger or touching a virtual space screen on which a user's motion is projected. Is disclosed.

In addition, the above-described method is "a method of analyzing the shape of the user's motion. In the condition that there is a wall obscured behind the user 19 in front of the apparatus and there is no external dynamic obstacle, the primary color floor sheet 17, the background screen ( 18), there is no need to install the pipe frame 11. The principle of user's body extraction necessary for measurement is to store the background screen 71 photographed with a digital camera in memory before the subject appears, and then the color generated when the user appears, The basic shape is extracted by detecting changes in brightness and saturation, and the essential requirement is that there must be no moving objects in front of the camera that cause confusion with the human body of the subject, or the metallic surface where the background reflects the front object. It must have a high gloss finish or reflect the back of the monitor or the back of the user. "It is described.

In other words, the above-described design is stored in the memory of the background screen photographed by the digital camera in advance to detect the change in color, brightness, saturation generated when the user appears.

In addition, in the above-described design, " first, the standard image 75 and the input image 74 are composed of only the shades of 3D characters having a wide thickness, which are produced by the animation keyframe data for motion instruction in the data storage unit 227. Comparing the area of the mismatched area by calculating the result 76. Second, the vertex position of the 3D character human skeletal line 77 corresponding to the standard image is compared with the input image 74 and is not included within the rectangular vertex. The greater the difference in motion, the greater the rate of deviation from the outside, before the two images are compared to each other. "To zoom in, zoom out, and move to fit the position and overall size ratio."

According to the above scheme, the instruction animation keyframe data exists in the size larger than the actual user only in 3D character shadow, and in the method of judging the user's motion by comparing only the user's image extracted from the silhouette and the background internally Therefore, it should be basically bigger than the user, and should be made only in black and white shades, and have a secondary process of enlarging, reducing, and moving.

The present invention aims to improve the existing game apparatus and method as described above, and according to one aspect of the present invention, unlike the conventional game sitting on a chair and enjoying the game statically, the present invention is a keyboard, mouse They are active and immersive, with no joysticks at all, and standing and moving the game to manipulate the game. The game provides a sense of the body utilizing the characteristics of the game and provides an operation method that is specific to each game as a method of progress, giving fun and immersion when the action is appropriate to the game situation.

According to another feature of the present invention, the user is provided to the virtual space where the game proceeds according to the movement of the user and the opponent while looking at each other between the users connected to the image camera using the Internet environment. Here, the user is standing in the game, so in the conventional game for the community between users to chat with the text using the keyboard, this development is to compress the video through the video camera and the voice through the microphone at the same time to the other party through the Internet It provides a virtual environment, such as playing a game with an opponent. The present invention develops a haptic operation method using an image camera and provides a method that can be easily executed through the Internet to establish a healthy and desirable Internet culture for young people.

According to another feature of the present invention, the object in the game may be represented by a person, an object, an animal, an image, or the like, and may independently move and make a sound. In addition, the present invention relates to a haptic game method in which a silhouette image of a human shape is represented as a continuous animation to compare and determine a silhouette image representing various human motions such as a dance motion and a motion of a user photographed through an image camera. will be. In addition, the present invention includes an Internet game service that allows a multi-user to play a game while looking at each other using a video camera without a separate installation on the Internet by utilizing a haptic game operation method using a video camera.

The present invention tracks the user's motion in a high value-added industry such as a haptic game method and device, and proceeds with a proper response according to the user's motion. It provides an opportunity to meet.

The present invention does not separate the user's body, but objectizes a region where motion is captured in an image input through a camera. In the present invention, the object to be objectized can be classified as an object if the movement of the object in the image input through the camera as well as the body part of the person is captured. Therefore, there is no need to separate each part of the body with code, nor does it need a database.

The present invention does not have to face the chroma key screen, blue screen or wall because the processing is performed without separating the user from the background in the image taken by the camera, and the game is not simply through the touch switch, but in game Contains all objects, and objects can also express (move, sound, sound) in a variety of ways. In addition, in a tactile game operation method of an image taken with an image camera, an image filter, time / spatial division, 3D vector, and DB loading are organized into one large module (pipeline), In terms of utilization, it can be said that it is different from the conventional game apparatus and method.

1 is a circuit block diagram of a haptic game device according to the present invention, and FIG. 2 is an overall external view of an external device to which FIG. 1 is applied.

The device is a camera 1, coin inlet 2, CD-ROM 3, game setting switch 4 as an external input device, and memory 5, hard disk 6, CPU as an internal processing device. (7), a video card (8), a sound card (10), and a LAN card (12) exist, and the output device comprises a CRT monitor 9, a speaker 11, and an internet 13.

The memory 5 is divided into various storage spaces, an image memory 5a for storing images photographed through the camera 1, a system image memory 5b for storing images of various objects necessary for a game, Sound memory 5c for storing background music and sound effects, game memory 5d for storing game data, DB, and game execution modules, and data for temporarily storing data or game setting switches during game play. The storage memory 5e is included.

The CPU 7 in charge of all operations of the apparatus includes an image processing operation unit 7a for processing data of the image memory 5a in processing for each function, and a sound processing unit 7b for processing data in the sound memory 5c. And a game processor 7c which processes the game progress with the data of the game memory 5d, and a network processor 7d as an online game, if necessary.

The apparatus photographs the user with the camera 1 and stores the photographed image in the image memory 5a. The image stored in the image memory 5a is transferred to the image processing unit 7a together with the system image memory 5b containing various objects and images of the game. The image processing operation unit 7a will be described in detail with reference to FIGS. 3, 4, and 5. The image calculated by the image processing unit 7a is finally output through the CRT monitor 9 via the video card 8. The camera 1 can be used regardless of various types and models, such as analog, digital, CCD, CMOS cameras, digital camcorders, and also, with the use of a USB image camera used in a general PC, the low price and popularity effect can be seen. Can be.

The performance and function of the camera is also important as the haptic game method of the present invention. As a function, when the camera 1 supports an enlargement / reduction function, various expansions are possible as a haptic game method. The camera 1 should be positioned at a position where the user can capture the center knee portion on the CRT monitor 9 screen when photographing the user and displaying it on the game screen.

In the present apparatus, the user can drive the game through a game start switch next to the coin inlet 2 although it is not shown in the drawing as a method of initially driving the game. When the game start switch is pressed in the standby state, the device immediately changes to the game driving mode so that the user can start the game. Here, the game start switch is activated when the coin is inserted through the coin inlet (2), or by inserting a payment card with a card payment device at the coin inlet (2) position to activate the game start switch. Can be. The game start and payment means such as coin input or card payment can be set in the game setting switch 4 as necessary. Therefore, if the payment means are removed from the setting of the game setting switch 4, the device can drive the game by pressing the game start position without any payment.

When the game is driven, the game processing unit 7c executes the game, and the sound processing unit 7b reads data from the sound memory 5c having various Wavs and MP3s to the speaker 11 through the sound card 10. Output According to the present invention, not only a stereo system including two speakers 11 but also a multi-channel stereo sound system including a 5.1-channel sound card 10 and an amplifier, an amplifier, a plurality of speakers, and the like.

The device of the present invention is configured to easily install and update a game through a CD using the CD-ROM 3. As a method of executing the game of the device, when the game is executed with the game CD inserted in the CD-ROM 3, the data of the game CD is read and stored in the game memory 5d and at the request of the game process 7d. The game CD reads the data of the game CD inserted in the CD-ROM 3 in real time. At this time, since the data is read from the CD-ROM 3 in real time, a CD-ROM 3 device having a high speed is required. In this case, the CD-ROM (3) device is composed of 52x speed or DVD device, and it is preferable to use it as a high speed and high capacity storage device. The above method of executing a game has a disadvantage in that a game CD must always be inserted into the CD-ROM 3. As another method of executing the game, the game CD is inserted into the CD-ROM 3 and installed instead of being executed. There is a way. When the installation program is executed on the game CD, all data in the game is stored in the hard disk 6 through the data storage memory 5e. Since the game once installed is stored semi-permanently on the hard disk 6, the game once installed does not need to insert a CD in the CD-ROM 3. Executing the game executes the game already installed in the hard disk 6, and can drive the haptic game according to the present invention.

In addition, when a part of the game is not upgraded or a part is upgraded or data is required to be updated, the update CD is inserted into the CD-ROM 3 and the update is executed. The data to be updated is stored through the data storage memory 5e. It is stored in the hard disk 6. Since the content once updated is stored in the hard disk 6, it is not necessary to always insert the CD in the CD-ROM 3 by the update once again.

The game setting switch 4 is a component that can change various environments related to the game setting, and is drawn out so that an administrator can easily manage the game. The game setting switch 4 is configured to set the game price, game information, record, game difficulty, etc. without reinstalling the game. In addition, once set data is stored in the hard disk 6 through the data storage memory 5e, it is recorded semi-permanently even when the device is turned off.

The game processing unit 7c separates the network processing unit 7d separately for a game that can be played online or cooperated with each other according to the game characteristics. The network processing unit 7d can play a game online with a user in another place using the Internet 13 through a LAN card 12. The service method of the present development through online is explained with reference to Figs.

The video is captured by the camera 1 and the game is manipulated by a procedural step in the image processing unit 7a of the CPU 7. The quality of the captured video has a great influence on the user's operation of the game. . If the indoor lighting is too low to capture the user properly because the video taken by the camera 1 is too dark or the light is too bright, the image processing unit 7a does not work properly. The operation method is not working properly. In addition, the brightness of a constant light should be maintained, but the brightness of the area photographed by the image camera of the device may change from time to time due to other lighting devices or other game devices around. In order to solve the problem, the camera 1 always takes an image of constant illuminance according to the ambient illuminance, including the function of adjusting the automatic exposure of the camera 1, and the camera 1 does not have the function. Even if the internal processing of the image processing unit 7a to remove the noise of the image taken by the camera (1) it is possible to filter by uniformly adjusting the screen brightness. In addition, an auxiliary lighting device 15 existing next to the CRT monitor 9 may be attached to adjust the illuminance value of the illumination. The auxiliary lighting device 15 may be omitted if the illuminance of the place where the device is installed is sufficiently bright and sufficiently processed by the image processing operation 7a of the present development so as not to affect the haptic game operation.

Image processing method

3 is an image processing method according to the present invention, an operation flowchart showing an operation of controlling an image photographed by an image camera, and FIG. 4 is an operation showing in detail only a step of controlling an image image captured by an image camera. It is a flow chart. 5 is a procedural reference diagram of the image camera image control method screen actually implemented by FIGS. 3 and 4.

FIG. 3 is a flowchart illustrating the internal processing of the image processing operation unit 7a of FIG. 1, wherein the image camera device P1 is a module directly connected to a physical image camera and receives 30 digital images per second from the image camera. Convert to

An image decompressor P2 converts 30 images per second received from the image camera apparatus P1 into one single format, which is unique to each image camera apparatus P1. The image camera device stores and transmits photographed images in various ways depending on the manufacturer, model, and performance. The various image formats of all the devices are changed into a single format of the present invention. Since the image is converted by the Image Decompressor (P2) in this way, it can be supported regardless of what kind of camera is installed in the device.

The image is transferred to an image filter P5, which is an image image processing pipeline P11, through an image camera image extraction P3. At the same time, the resultant image of the image camera image extraction (P3) process is transferred to the renderer (P4). The renderer (P4) outputs the video memory image of the video card and displays it on the screen through the final monitor. The processing of the video taken by the camera and displayed to the user can be done through the Renderer (P4). Here, the method for manipulating the haptic game (to be described later) of the present invention performs all processing in the image image processing pipeline P11 of FIG. Image Image processing processes and converts images captured with an image camera through procedural steps of various techniques. This series of processes is called an image image processing pipeline P11.

The image extracted by the image camera apparatus P11 differs in the quality of the image in proportion to the performance of the camera. In addition, noise is included in the captured image, such as changes in brightness values of consecutive images and changes in brightness values caused by illumination. Therefore, as shown in FIG. 4, the image filter P5, which is the first stage of the image image processing pipeline P11, removes the noise of the image and increases the sharpness to obtain an effective result. Filter 30 images per second.

Referring to FIG. 4, an image processed through the image filter P5 is spatially divided and blocked on a single still image extracted through the spatial division P6. Space division P6 is divided into blocks of 8 pixels horizontally and 8 pixels vertically for each image in 30 images per second as shown in the upper right image of FIG. 5. Each image divided into 8x8 blocks becomes the minimum unit of the processing module of the procedural steps of the image image processing pipeline P11. In addition, the blocks are divided into 8x8 squares, where various sizes of blocks such as 1x1, 4x4, 16x16, and 32x32 are possible. The smaller the block size is, the higher the precision of the haptic operation method is, but the greater the burden on the CPU 7 at the processing speed. Here, a block size of 8x8 shows satisfactory results for execution speed and result.

The spatially divided image is a time-dependent image of the current image frame n (P7a) and the next image frame n + 1 (P7b) of the current image from 30 images stored in the image memory in the time division (P7) step. The difference is detected and difference image detection is performed (P7c). Looking at the two lower images of FIG. 5, it can be seen that the hand moved when comparing the left image P7a with the right image P7b after a certain time. At this time, if only the changed blocks are displayed in the two images divided by the space P6, the difference image P7c, which is a box filled around the hand as shown in the lower right image P7b, can be seen.

3D vector extraction (P8) is performed by searching for blocks that are the same as or similar to the reference time frame through the time-divided difference image blocks based on the result of the difference image detection (P7c). In FIG. 5, the direction in which the hand movement is changed is measured as a three-dimensional vector, and has a two-dimensional x and y values and a z value along the time axis. The three-dimensional vector values x, y, and z can also be used to infer the values of velocity and intensity.

Through the vector extracted through the 3D vector extraction (P8), the connectivity of the coordinate values distributed in the 3D vector is checked to find all connected regions. Object area search (P8a) is performed by area optimization of the connected area. By extracting all the difference images P7c extracted through the spatial division P6 and the time division P7 and connecting the difference images P7c, an area can be made of an object. In FIG. 5, when all P7c are connected, only a hand whose movement is changed may be extracted as one object P8a.

Through the object movement detection (P8b) to extract the object center of gravity through the information obtained through the object area search (P8a) and to detect the object movement information from the change of the center through the final DBization and statistical data (P9) Load into DB.

After each step of the image image processing pipeline P11, an object may be finally detected from a video captured by a camera, and motion may be detected and predicted. For example, when a user moves a hand, only the movement of the hand can be detected and objectified, the direction of the hand movement can be expressed as a three-dimensional vector, and the next user can predict the motion of the hand to take.

Application and Extended Image Processing

6 is an exemplary view of a haptic game to which the image processing method according to the present invention described above is applied. The game screen V1 includes a user 17 and a game object 18 captured by a camera. The game object 18 represents the ball in the shape of a circle in this exemplary diagram, but in the actual game, it can be represented by anything, such as a person, an animal, or an object. The game object 18 may move by itself as seen on the game screen V2 that has flown for a certain time from the game screen V1, and may be animated to change an image. It also has an effect sound and can make a sound through the data in the sound memory 5d.

In the game screen V1, the movement of the user 17 is continuously displayed as a video on the screen at 30 images per second, and then moves, and then, as in the game screen V2, a part of the user 17 (the hand of the user 17) and the game object ( When 18) overlaps, the game object 18 reacts in various ways. When the user 17 and the game object 18 overlap each other, various reactions of the game object 18 may produce a specific sound or change the animation. In addition, in the three-dimensional vector extraction (P8a) step of the user 17, the direction in which the user's hand moves is tracked as a three-dimensional vector to realistically change the motion through the motion vector and the physics of the game object 18. Can cause.

In the portion where the hand of the user 17 and the game object 18 are compared and judged, in the M1 diagram below in FIG. 6, the user 17 uses 30 pieces per second through the time division P7 in FIG. 4. The difference image detection (P7c) is performed as an image. The game object 18 also has a game decision unit T1 that is the same as the user's difference image P7c. The game decision unit T1 can be represented by an 8x8 block like the user's difference image detection P7c. The overlapped block is extracted with respect to the difference image P7c of the hand and the game decision unit T1 of the game object 18.

In this case, the method of comparing and determining the calculated values may include a value obtained by calculating the percentage of the overlapped area between the object area and the game object 18 obtained through the object area search P8a obtained through the difference image P7c of the movement of the user 17. The comparison is made by adjusting two values calculated as a percentage of the number of overlapping frames from 30 frames per second.

As a method of using the difference image P7c in which the process of the object region search P8a is omitted, the following method may be used. A block calculated as a percentage of the overlapped portion of the game decision unit T1 of the game object 18 with respect to the difference image P7c of the movement of the user 17 (as shown in the example of FIG. The two blocks overlap in the 16 game decision units T1 of the object 18 and the 13 difference images P7c of the user (that is, a value of (3/16) * 100 is calculated) and 30 frames per second. It is determined whether the end user 17 and the game object 18 collide with each other by adjusting two values calculated as percentages in the number of overlapping frames. Since the judgment is made at 30 frames per second, the game object 18 is not fixed to a certain area and can move on its own. .

Although the game decision unit T1 has been illustrated as being represented by 8x8 blocks, the game decision unit T1 of the game object 18 is also 8x8 because the image captured by the image camera is space-divided (P6) into 8x8 blocks. It is not fixed to a block. In order to compare and determine the detailed shape of the game object 18, the game decision unit T1 may be divided into various blocks such as 1x1, 4x4, 8x8, and 16x16 blocks regardless of the space division P6. The smaller the block size, the higher the accuracy of the decision between the game object and the user, but the more burden on the CPU. Here, the game decision (T1) also shows that the 8x8 block size satisfies the performance speed and the result value.

Other Game Applications and Comparison Judgments Applying Extended Image Processing

FIG. 7 is an exemplary view of the immersive game using the extended image processing described with reference to FIG. 6, FIG. 8 is for explaining the game method of FIG. 7 in detail, and FIG. 9 is an example of the immersive game screen of FIG. 7. It is also.

The game object of FIG. 7 is represented by a silhouette S1 shaped like a real person. It is a haptic operation method of proceeding a game by determining whether the user 17 and the human-shaped silhouette S1 overlap correctly. In the example diagram of FIG. 7, the user 17 and the silhouette S1 do not overlap precisely, but as shown in the actual game screen reference diagram of FIG. 9, the user 17 moves the camera so that it accurately overlaps the silhouette S9 displayed on the game screen. You have to play the game.

The game silhouette S1 is stored in the memory as a plurality of images as shown in FIG. The game silhouette is made of images that take various actions such as S2, S3, S4, S5, and S6 of FIG. 8 (a), and these images are sequentially displayed on the screen at a predetermined time as time passes, so that the silhouette looks like a video. Will move. The user 17 sees the silhouette moving in the video and manipulates the game according to the maximum motion.

Comparison determination that the operation of the user 17 coincides with the game silhouette can be made in two ways as shown in FIG. 8 (b). The left silhouette image S7 is the same as the above-described game object 18 and is represented as an object taking a human shape. As shown in the above method, the entire silhouette image S7 is set as the game decision unit T1 and the user's difference image ( It is a method judged by the same method as the said method compared with P7c). The other method is that the game judgment part T1 of the silhouette image S8 is set only where the joint or the movement is changed, as in the right silhouette image S8. The game decision unit T1 of the silhouette image S8 is a difference image of the movement of the user 17 as in the method of comparing and determining the hand of the user 17 and the game object 18 as shown in the M1 diagram of FIG. 7. The object area obtained through (P7c) and the area where the silhouette image S8 overlaps with the silhouette image S8 obtained as a percentage, and two values calculated as a percentage of the number of frames overlapped at 30 frames per second. Adjust to compare. In this case, the overlapped portion of the user difference image P7c block and the game decision unit T1 of the silhouette image S8 is calculated as a percentage by using the difference image P7c omitting the process of the object region search P8a. And how many frames were overlapped at 30 frames per second, and the two values calculated as percentages were adjusted to determine whether the end user 17 and the silhouette image S8 matched, and the user 17 took a similar operation of the silhouette S8. Will be judged. Through this determination, each time a silhouette image is added, the silhouette may express various motions by animation, and the motion of the user 17 may be determined. Therefore, by adding the silhouette S7 image, all the actions that a person can take can be expressed in the game, and the game can be conducted by comparing the user 17 with the action. The game decision unit T1 is not displayed on the actual game screen but is processed internally in the internal image image process pipeline P11.

As for the silhouette image, as shown in the diagram of Fig. 8 (c), how many seconds each frame (Frame) should be displayed on the screen, and the game decision unit T1 of each image is equal to the user 17 for several seconds. Include information to determine if you should act. Through the information of the image (Frame), a silhouette is expressed as a motion picture. In addition, through this information, the silhouette continuously displays various motions, and the motion picture of the silhouette can express all the actions that a person can take, such as a dance, a specific motion, and a posture, according to the haptic game method of the present invention. Can be.

In addition, the silhouette image can be continuously updated and expanded by updating only the silhouette image while being installed in the game. In other words, it is possible to expand the game to follow other motions by adding silhouette video data later, with only the basic modules of the game installed for the first time. When the newly added silhouette image update CD is inserted into the CD-ROM 3 and the update is executed, as in the method of updating the game through the CD-ROM 3, the game can execute the silhouette game of the updated contents. It is also possible to update in real time via the Internet, not on an update CD. The silhouette image necessary through the network processing unit 7d is received through the game server in real time through the Internet 13 using the LAN card 12. The silhouette image received here is stored in the hard disk 6 through the data storage memory 5e so that the silhouette image once received on the Internet 13 is permanently stored and can be continuously executed in the game.

Another embodiment

FIG. 10 is a diagram illustrating a method in which two users proceed together on the same screen with one image camera 1 with respect to FIG. 7; a silhouette S10 in which a left user 17 and a left user take action, and a right user 19 and right. The silhouette S11 on which the user will take an action is displayed on the same screen.

The left silhouette S10 and the right silhouette S11 are set to different images, respectively, to animate different motions or use the same silhouette image to allow two users to act on the same motion according to a silhouette. In addition, the two silhouettes can be represented by an animation of a dance movement, in which one background music flows, and the two silhouettes perform a group dance according to the music. At this time, it is possible for a game method in which two users dance one dance while each user moves according to their own silhouettes.

Another embodiment

11 and 12 are exemplified diagrams of a haptic game method in which the haptic game method of FIG. 6 is used to enjoy a game in both directions using the Internet.

On the screen of the monitor 9, the screen E1 projected through the camera 1 of the person and the screen E2 projected by the camera of the counterpart are divided and appear. You will see E3 on your screen and E4 on your screen. The object ball E5 of the game currently exists on his screen E1. If the user E3 pushes the ball E5 with his body, the ball E5 goes to the opponent screen E2. The screen E2 of the other party is reproduced in real time to the other party via the Internet 13 via the LAN card 12 of the network processing unit 7d. The other party's video transmitted in real time may transmit only the image captured by the pure camera 1 that does not include the actual game object E5. The game object E5 is currently displayed on two screens E1 and E2 divided by the monitor 9 within. In addition, the haptic game device does not support chatting using text as a standing game. The chat is a video chat, which is not displayed on the device, but a real time voice chat is performed through a microphone. Therefore, the camera 1 preferably includes a microphone function.

The technical spirit of the present invention has been described above through specific embodiments of the present invention. However, the technical idea of the present invention is not limited to the above embodiment. The technical scope of the present invention is to be determined by reasonable interpretation of the appended claims.

According to the present invention, by using the image camera to look at yourself projected on the screen to move the body while moving the game to manipulate the game according to the characteristics of the game to enhance the immersion and fun to feel. In addition, you can see the diet effect naturally while moving the body lightly, and can give a light exercise effect through healthy sweating. The present invention provides a new concept haptic method that induces more active and active participation that is different from the operation method adopted by existing games. In game shops, haptic games are very popular with the younger generation, and people are playing these haptic games at a high cost. In this high value-added industry, the present invention provides a general user with an opportunity to encounter a new concept of haptic game using an image camera.

Video object tracking using video cameras has been one of the topics of interest in computer vision and many practical applications for many years. In particular, automated monitoring systems in the industry and unmanned monitoring systems widely used in the general society have emerged as a remotely detectable equipment that saves labor costs and is difficult for workers to check in the field. Moreover, due to the rapid development of multimedia related technologies, the technology of video monitoring system is also rapidly developing, and thus the generalization of such surveillance monitoring system is accelerating. As such, domestic technology is still focused on specific industrial purposes, but as the demands of general users increase due to the development of technology and the advancement of the system, interest in this technology will increase. The present invention is to build a unique image image processing pipeline as a specialized technical field such as motion capture and image control, which has been found only in the industrial field, to overcome technical and hardware problems and realize popularization by using a general image camera.

Claims (17)

An external input device including a camera, an output device for outputting images and sounds, images and background music of various objects necessary for games and images captured through the camera, sound effects and game data, a DB, a game execution module, and a game A memory for storing data to be temporarily stored or a value of a game setting switch, an image processing unit for processing an image stored in the memory, an audio processing unit for processing data in the memory, and data of the memory to process the game. An image processing method applied to the CPU image processing unit of a game device including a CPU including a game processing unit for processing, Converting a signal input to the image camera into a digital image of a predetermined frame number; Image Decompressing, which converts an image received from an image camera into a single format, each of which has its own unique image format, Including an image image processing pipeline for processing the image extracted through image camera image extraction, The image processing method according to claim 1, wherein the motion of the object is detected and objectified in the video photographed by the camera, and the direction in which the object moves is expressed as a three-dimensional vector. The method of claim 1, wherein the image processing pipeline An image filtering step to remove noise from the extracted image and increase sharpness to obtain an effective result, Spatially dividing the processed image through spatial filtering to block the image; Time-dividing the spatially divided image into a current image frame n and a next image frame n + 1 of the current image from an image stored in an image memory; Comparing the change of the image with time to detect the difference image, Extracting a 3D vector by searching for a block similar to a reference time frame through a time-divided difference image block based on the result of the difference image detection; Performing a connectivity check of coordinate values distributed in a 3D vector through a vector extracted through 3D vector extraction to find all connected areas, and performing an object area search through area optimization of the connected area, And extracting an object center of gravity based on the information obtained through the object region search and detecting an object movement for detecting object movement information from a change in the center. The method of claim 2, wherein the object area searching step And extracting all the difference images (P7c) extracted through the spatial division (P6) and the time division (P7), and connecting the difference images (P7c) to create an object. The image processing method according to claim 2, further comprising a step of DBizing and statisticalizing data after the object motion detecting step. Displaying the user 17 and the game object 18 taken by the camera on the game screen, the movement of the user 17 is continuously displayed as a video on the screen, the game object to operate to move itself, When the at least part of the user 17 and the game object 18 overlap, the movement of the user is processed by the image processing method of any one of the above claims 1 to 4 to compare and determine a portion where the two overlap. And causing the game object (18) to make various reactions according to the determination result. The method of claim 5, The game screen is connected in real time through the Internet, One of the game screens is divided into a screen E1 taken by the camera of the first user and a screen E2 taken by the camera of the second user, The game object is a haptic game method characterized in that the movement is displayed in real time on the screen of the first user and the screen E2 of the second user. The method of claim 5 or 6, wherein the comparison determination of the portion where the user and the game object overlap Setting a game decision unit (T1) for dividing the game object 18 image into a plurality of blocks, In order to extract the overlapped blocks for the user difference image P7c and the game decision unit T1 of the game object 18, the object region search P8a obtained through the difference image P7c for the user 17 movement. Expected to include comparing the value obtained by calculating the percentage of the overlapping object area and the game object 18 obtained by the step and the two values calculated as a percentage of how many frames overlapped at 30 frames per second Type game method. The method of claim 5 or 6, wherein the comparison determination of the portion where the user and the game object overlap Setting a game decision unit (T1) for dividing the game object 18 image into a plurality of blocks, The block calculated as a percentage of the overlapping portion of the game object 18 with the game decision unit T1 for the difference image P7c of the movement of the user 17 and the number of frames overlapped at 30 frames per second. Determining whether the end user (17) and the game object (18) collide by adjusting two values calculated as percentages. The method of claim 7, wherein The block of the game determination unit (T1) of the game object (18), characterized in that the size is different from the space division (P6) block of the user image, haptic game method. The method of claim 8, The block of the game determination unit (T1) of the game object (18), characterized in that the size is different from the space division (P6) block of the user image, haptic game method. Displaying a silhouette of a person stored in memory as a series of images taking a variety of motions on the screen sequentially according to the time passed so that the silhouette moves like a video, Photographing and displaying that the user 17 sees the silhouette moving in a video and takes the same action, A game-like game method comprising the steps of processing a user (17) image by the image processing method of any one of claims 1 to 4 and comparing and judging overlapping with the silhouette (S1). The method of claim 11, wherein the comparison of the portion where the user and the silhouette (S1) overlap Dividing the silhouette (S1) image into a plurality of blocks, setting the game determination unit (T1) that is not displayed on the actual game screen and is processed internally in the image image process pipeline (P11), Object area search (P8a) obtained through the difference image (P7c) of the movement of the user 17, in order to extract a block overlapped with the difference image (P7c) of the user and the game decision unit (T1) of the silhouette 18 A haptic game comprising comparing a value obtained by calculating a percentage of an overlapped area of an object region and a silhouette (S1) obtained from a target, and calculating a percentage of how many frames overlapped at 30 frames per second. Way. The method of claim 11, wherein the comparison of the portion where the user and the silhouette (S1) overlap Setting a game decision unit (T1) which is not displayed on the actual game screen, which divides the silhouette (S1) image into a plurality of blocks, and is processed internally in the image image process pipeline (P11), For the difference image P7c of the movement of the user 17, the block calculated as a percentage of the overlapped portion with the game decision unit T1 of the silhouette S1 and the number of frames overlapped for 30 frames per second. And adjusting the two values calculated as percentages to determine whether the end user 17 and the silhouette S1 overlap. The method of claim 11, wherein the comparison of the portion where the user and the silhouette (S1) overlap Setting a game decision unit T1 that is not displayed on the actual game screen but internally processed in the image image process pipeline P11 in a portion of the silhouette S1 image, The object area obtained through the object area search (P8a) obtained through the difference image (P7c) of the motion of the user 17 and the area overlapped with the game decision of the silhouette image S8 as a percentage, and a few at 30 frames per second. The game-like game method comprising the step of comparing and judging two values calculated as a percentage of the number overlapped during the frame. The method according to any one of claims 11 to 14, wherein the user is displayed together on the same screen by one image camera as a plurality of names, The silhouette is a game-like game method, characterized in that the silhouette that one user takes an action and the silhouette that the other user takes an action are displayed on the same screen. The method according to any one of claims 12 to 14, The silhouette image includes information about how many seconds should be displayed on the screen, and the game judge includes information determining how many seconds the user 17 should behave the same, According to this information, the silhouette is a haptic game method, characterized in that one motion is represented by a video. The method according to any one of claims 12 to 14, The silhouette image is a game-like game method, characterized in that by adding a silhouette video data is updated to the game to follow a different motion.

Images