KR20170078176A - Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method - Google Patents

Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method Download PDF

Info

Publication number
KR20170078176A
KR20170078176A KR1020150188441A KR20150188441A KR20170078176A KR 20170078176 A KR20170078176 A KR 20170078176A KR 1020150188441 A KR1020150188441 A KR 1020150188441A KR 20150188441 A KR20150188441 A KR 20150188441A KR 20170078176 A KR20170078176 A KR 20170078176A
Authority
KR
South Korea
Prior art keywords
game
user
degree
similarity
joint
Prior art date
Application number
KR1020150188441A
Other languages
Korean (ko)
Inventor
권성근
이우준
Original Assignee
경일대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 경일대학교산학협력단 filed Critical 경일대학교산학협력단
Priority to PCT/KR2015/014440 priority Critical patent/WO2017115887A1/en
Priority to KR1020150188441A priority patent/KR20170078176A/en
Publication of KR20170078176A publication Critical patent/KR20170078176A/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212—Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25—Output arrangements for video game devices
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/45—Controlling the progress of the video game
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/70—Game security or game management aspects
    • H04N5/2257—
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2250/00—Miscellaneous game characteristics
    • A63F2250/30—Miscellaneous game characteristics with a three-dimensional image

Abstract

The present invention relates to a device for providing a game based on motion recognition, a method for the same, and a computer-readable recording medium on which the method is recorded. And displays a guide operation of a demonstrator previously photographed through the display unit, photographs an operation of a user moving along the guide operation through the camera unit, calculates a degree of similarity between the guide operation and a user's operation, And a control unit for controlling the degree of difficulty of the game to be displayed through the display unit in a game in which a plurality of game operations are sequentially performed according to the exercise ability, Apparatus, a method therefor and a computer readable recording medium storing the method Thereby providing a recording medium.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a device for providing a game based on motion recognition, a method therefor, and a computer readable recording medium on which the method is recorded.

More particularly, the present invention relates to a device for providing a game based on a motion recognition by adjusting a degree of difficulty according to a user's exercise ability, The present invention relates to a computer-readable recording medium on which a method is recorded.

As multi-media systems based on video information are developed, there is a growing demand for technologies that can understand the information contained in images and apply them to various fields. Especially, various kinds of entertainment industry based on gesture or motion recognition are being developed.

Korean Public Patent No. 2012-0029738 Published on March 27, 2012 (name: user dynamic engine gesture recognition method and interface, and electric device using the same)

An object of the present invention is to provide a device for providing a game based on motion recognition by adjusting the degree of difficulty according to the exercise ability that can be performed according to a user's operation, a method therefor, and a computer readable recording medium on which the method is recorded.

According to another aspect of the present invention, there is provided an apparatus and method for providing a game based on motion recognition, including a display unit for displaying a screen, a camera unit for capturing an image, Wherein the controller displays the motion of the user, captures the motion of the user moving along the guide motion through the camera unit, calculates the degree of similarity between the motion of the user and the motion of the guide, evaluates the motion capability of the user according to the degree of similarity, And a controller for controlling the degree of difficulty of the game and displaying the game through the display unit in a game in which a plurality of game actions are sequentially performed according to the ability.

Wherein the controller extracts three-dimensional position coordinates of a plurality of joints from an operation of the user photographed through the camera unit, extracts three-dimensional position coordinates of the joints from one of the plurality of joints and one of the joints When two neighboring joints are connected by two line segments, the angle formed by the two line segments is obtained by using the inner product of the vector, and a joint radian value obtained by expressing the obtained angle as a radian value is derived, And the similarity is calculated by comparing the radian value and the joint radian value derived in advance from the guide operation.

Wherein the control unit compares a joint radian value derived from a predetermined number of frames and a mean value or an integral value of a joint radian value derived from the frame with the same number as the predetermined number in an image captured by the user, And the degree of similarity is derived.

Wherein the control unit displays a plurality of game operations sequentially through the display unit, captures an operation of a user following the game operation through the camera unit, and determines whether the similarity of the captured user operation and the game operation is changed according to the degree of difficulty And judges that the player has succeeded in scoring if the predetermined reference value is equal to or greater than a preset reference value.

According to another aspect of the present invention, there is provided a method for providing a game based on motion recognition, comprising the steps of capturing an action of a user moving along a guiding operation of a photographed demonstrator, Calculating a degree of similarity of the game based on the degree of similarity, evaluating the user's ability to exercise according to the degree of similarity, adjusting the degree of difficulty of the game in a game for sequentially following a plurality of game actions according to the exercise ability .

Wherein the step of calculating the degree of similarity comprises the steps of: extracting three-dimensional position coordinates of a plurality of joints from the operation of the user; determining a positional relationship between one of the plurality of joints and one of the plurality of joints A step of obtaining an angle formed by the two line segments using an inner product of a vector when two adjacent joints are connected by two line segments, and deriving a joint radian value expressed by a radian value of the obtained angle And calculating the degree of similarity by comparing the derived joint radian value with the joint radian value derived in advance from the guide operation.

The step of calculating the degree of similarity by comparing the derived joint radian value with the joint radian value derived in advance from the guide operation may include calculating a joint radian value derived from a predetermined number of frames in the image captured by the guide operation, And comparing the average value or the integral value of the joint radian value derived from the frames of the same number as the predetermined number in the photographed image to derive the similarity.

The game providing method includes the steps of capturing an action of a user following a plurality of game actions sequentially displayed; if the similarity of the action of the captured user and the game action is equal to or greater than a preset reference value in accordance with the degree of difficulty, And determining that it is successful.

According to another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon instructions for executing the game providing method.

According to the present invention as described above, the degree of similarity between the operation of the demonstrator and the user's motion is derived through the motion recognition, the user's exercise ability is evaluated through the recognition, and the user-customized game is controlled Thereby providing a user experience (UX: User experience) that allows the user to enjoy the game at an appropriate level of difficulty.

1 is a conceptual diagram of an apparatus for providing a game according to an embodiment of the present invention.
2 is a block diagram illustrating an internal configuration of an apparatus for providing a game according to an embodiment of the present invention.
3 is a flowchart illustrating a method for evaluating exercise capacity according to an embodiment of the present invention.
4 is a view for explaining a characteristic of a human operation according to an embodiment of the present invention.
5 is a flowchart illustrating a method for deriving a radian value of a joint according to an embodiment of the present invention.
6 is a view for explaining a method for deriving a radian value of a joint according to an embodiment of the present invention.
FIG. 7 is a diagram for explaining a method of calculating the degree of similarity according to an embodiment of the present invention.
8 is a flowchart illustrating a method of providing a game according to an embodiment of the present invention.
9 is a diagram for explaining a method of providing a game according to an embodiment of the present invention.

Prior to the detailed description of the present invention, the terms or words used in the present specification and claims should not be construed as limited to ordinary or preliminary meaning, and the inventor may designate his own invention in the best way It should be construed in accordance with the technical idea of the present invention based on the principle that it can be appropriately defined as a concept of a term to describe it. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that, in the drawings, the same components are denoted by the same reference symbols as possible. Further, the detailed description of known functions and configurations that may obscure the gist of the present invention will be omitted. For the same reason, some of the elements in the accompanying drawings are exaggerated, omitted, or schematically shown, and the size of each element does not entirely reflect the actual size.

First, an apparatus for providing a game according to an embodiment of the present invention will be described. FIG. 1 is a conceptual diagram of an apparatus for providing a game according to an embodiment of the present invention, and FIG. 2 is a block diagram for explaining an internal configuration of an apparatus for providing a game according to an embodiment of the present invention.

Referring to FIG. 1, a user device 100, which is an apparatus for providing a game based on a motion recognition according to an embodiment of the present invention, is provided to provide a game suitable for a user's athletic performance. To this end, the user device 100 displays the operation of the demonstrator on the screen, and the user follows the operation of the demonstrator. Here, the operation of the demonstrator may be provided through a previously captured and stored image. At this time, the user device 100 can evaluate the user's exercise ability through the operation of the user following the guide operation by the user. The user device 100 may then provide a game suitable for the user based on the user's athletic performance. Accordingly, the present invention can provide a game suitable for a user's exercise ability.

2, the user apparatus 100 includes a display unit 110, a camera unit 120, a storage unit 130, an input unit 140, and a control unit 150.

The display unit 110 visually provides menus, input data, function setting information, and various other information of the user apparatus 100 to the user. The display unit 110 displays a screen such as a boot screen, a standby screen, a menu screen, and the like of the user device 100. In particular, the display unit 110 performs a function of outputting the operation of the demonstrator according to the embodiment of the present invention to a screen. The display unit 110 may be formed of a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AMOLED), or the like. Meanwhile, the display unit 110 may be implemented as a touch screen. In this case, the display unit 110 includes a touch sensor, and the controller 150 can sense the touch input of the user through the touch sensor. The touch sensor may be constituted by a touch sensing sensor such as a capacitive overlay, a pressure type, a resistive overlay, or an infrared beam, or may be constituted by a pressure sensor . In addition to the above sensors, all kinds of sensor devices capable of sensing contact or pressure of an object can be used as the touch sensor of the present invention. The touch sensor senses the touch input of the user, generates a sensing signal, and transmits the sensing signal to the controller 150. The sensing signal may include coordinate data input by the user. When the user inputs a touch position movement operation, the touch sensor may generate a sensing signal including coordinate data of the touch position movement path and transmit the sensing signal to the controller 150. Particularly, when the display unit 110 is a touch screen, some or all of the functions of the input unit 140 may be performed through the display unit 110.

The camera unit 120 is for capturing an image, particularly for capturing an action of a user. Although not shown, the camera unit 120 includes an infrared light emitting module (IR light), a depth image sensor, and a color image sensor. The color image sensor receives light reflected from a subject and converts the light into an electric signal. The color image sensor can be implemented based on a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The infrared light emitting module emits infrared rays in pixel units. The depth image sensor receives the infrared rays emitted by the infrared light emitting module in pixel units, and measures the distance (or depth) that the infrared ray reflects from the object. Accordingly, the control unit 150 can derive the three-dimensional coordinates of each pixel of the subject of the image photographed through the camera unit 120 according to the embodiment of the present invention. A representative example of the infrared light emitting module (IR light), the depth image sensor, and the color image sensor of the camera unit 120 may be a Kinect sensor. The camera unit 120 may further include an analog-to-digital converter. The camera unit 120 may convert an electric signal output from the image sensor to a digital sequence and output the digital sequence to the controller 150.

The storage unit 130 stores programs and data necessary for the operation of the user apparatus 100, and can be divided into a program area and a data area. The program area may store a program for controlling the overall operation of the user device 100, an operating system (OS) for booting the user device 100, an application program, and the like. The data area is an area where user data generated according to use of the user device 100 is stored. Also, the storage unit 130 may store various kinds of data generated according to the use of the user device 100 by the user. Such data includes an image composed of a plurality of frames, which are images of the operation of the demonstrator, the operation of the user, and the like. Each kind of data stored in the storage unit 130 can be deleted, changed, or added according to a user's operation.

The input unit 140 receives a key operation of the user for controlling the user apparatus 100, generates an input signal, and transmits the input signal to the controller 150. The input unit 140 may include any one of a power key, a numeric key, and a direction key for power on / off, and may be formed of a predetermined function key on one side of the user device 100. When the display unit 110 of the input unit 140 is a touch screen, the function of each kind of keys may be performed by the display unit 110. In the case where all functions can be performed only by the touch screen, It is possible.

The control unit 150 may control the overall operation of the user device 100 and the signal flow between the internal blocks of the user device 100 and may perform a data processing function of processing the data. The controller 150 may be a central processing unit (CPU), an application processor, or the like. The operation of the controller 150 will be described in more detail below.

Although not shown, the user device 100 according to an embodiment of the present invention includes a storage medium inserting unit for inserting an external storage medium such as a memory card to store data, a connection terminal for exchanging data with an external digital device, , And a charging terminal. In addition, the user device 100 may further include units having an additional function such as an audio processing unit for inputting or outputting an audio signal, a voice signal, or the like through a microphone and a speaker, or an MP3 module for reproducing a digital sound source have. Although there are many variations of the portable device according to the convergence trend of the digital device, the user device 100 according to the present invention further includes a unit of the same level as the above-mentioned units It will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention.

Next, a method for providing a game according to an embodiment of the present invention will be described. A method for providing a game according to an embodiment of the present invention evaluates a user's athletic ability and provides a game with difficulty according to the athletic ability. The method for evaluating exercise capacity according to the embodiment of the present invention will be described first. 3 is a flowchart illustrating a method for evaluating exercise capacity according to an embodiment of the present invention.

Referring to FIG. 3, in order to evaluate the user's motor ability, the controller 150 displays the guide operation and the guidance message of the demonstrator through the display unit 110 in step S110, and guides the guide operation. Here, the guide operation is the operation of the demonstrator included in the image photographed through the camera unit 110 according to the embodiment of the present invention. In step S110, the control unit 150 may guide the user to follow the guide operation through the audio signal output.

The user can follow the guide operation displayed through the display unit 110 according to the guide. Then, the control unit 150 photographs the user's operation through the camera unit 120 in step S120. Then, in step S130, the control unit 150 compares the guide operation displayed through the display unit 110 with the operation of the photographed user, and calculates the similarity between the two operations. If the degree of similarity is calculated, the controller 150 evaluates the user's athletic performance according to the degree of similarity in step S140, and determines the degree of difficulty of the game corresponding to the athletic performance.

As described above, the degree of similarity according to the embodiment of the present invention is similar to the guiding operation of the demonstrator. Hereinafter, a method of calculating such similarity will be described in more detail.

4 is a view for explaining a characteristic of a human operation according to an embodiment of the present invention. 5 is a flowchart illustrating a method for deriving a radian value of a joint according to an embodiment of the present invention. 6 is a view for explaining a method for deriving a radian value of a joint according to an embodiment of the present invention. 7 is a diagram for explaining a method of calculating the degree of similarity according to an embodiment of the present invention.

4, the three-dimensional position of the elbow joint 10 is determined by the shoulder joint 21 and the wrist joint 22 is determined by the elbow joint 10 in the three-dimensional position of the joint by the elbow joint 10 Is determined. In other words, all human motion is limited by neighboring joints, and the present invention calculates the degree of similarity of motion using angles observed with these joints.

Referring to FIG. 5, in step S120, the controller 150 photographs a user's operation according to a guide operation. In step S210, the controller 150 may extract the three-dimensional coordinates of each joint of the user from the operation of the user photographed through the camera unit 110. [ For example, referring to FIG. 6, three-dimensional position coordinates of the shoulder joint 21, the elbow joint 10, and the wrist joint 22 can be extracted. More specifically, as described above, the control unit 150 emits infrared rays to the subject in units of pixels through the camera unit 110, receives infrared rays reflected from the subject, Dimensional position coordinates of the object. At this time, since the subject is a person, the controller 150 derives the position data of the main skeleton constituting the human body, that is, the three-dimensional coordinates of the joint, using skeleton features of the human body previously stored.

When the three-dimensional coordinates of the joint are derived, the controller 150 derives two vectors from one joint to two neighboring joints in step S220. For example, when one of the joints is referred to as the elbow joint 10, the vector from the elbow joint 10 to the shoulder joint 21 on the basis of the three-

Figure pat00001
And a vector from the elbow joint 10 to the wrist joint 22
Figure pat00002
Can be obtained. Then, in step S230, the controller 150 calculates the angle between the two line segments when two adjacent joints of the joint are connected to each other by two line segments through the inner product of the two vectors ?) can be obtained. For example, when one of the joints is referred to as an elbow joint 10, the elbow joint 10 and the shoulder joint 21 are connected by a line segment (first line segment), and the elbow joint 10 and the wrist joint 22 are connected by a line segment When connecting with a line segment (second line segment), the angle (?) Formed by the first line segment and the second line segment is expressed as a vector
Figure pat00003
And vector
Figure pat00004
Can be obtained through the inner product of. The controller 150 converts the angle? Obtained in step S240 into a radian value.

As described with reference to FIGS. 5 and 6, according to the present invention, in the image taken by the user, one of the plurality of joints of the user and two joints adjacent to the joint are divided into two segments When connecting, the angle (?) Formed by the two line segments can be obtained by using the dot product of the vector and expressed as a radian value. In the embodiment of the present invention, the expression of the angle (?) Obtained by using any one of the joints and the joints adjacent to the joint with the radian value is referred to as a 'joint radian value'.

No matter what a person does, each joint will have a joint radian value. Therefore, the embodiment of the present invention can express all the actions of a person with the joint radian value of each joint. Thus, according to the embodiment of the present invention, the controller 150 may calculate the similarity by comparing the radian value of the joint derived from the user operation with the radian value of the joint extracted from the guide operation of the demonstrator.

As shown in Table 1, the controller 150 can derive the degree of similarity between the radian values of the joints in mutually corresponding joints in the guide operation of the demonstrator and the user's operation.

joint Similarity ... ... . ... ... . Left shoulder 70% Right shoulder 82% Left elbow 83% Right elbow 42% Center of vertebra 22% Left hip 12% Right hip 62% ... ... . ... ... .

As shown in FIG. 7, assuming that the demonstrator makes a motion by a guide motion, and thus the user follows the motion, the image of the guide motion of the demonstrator with respect to the left elbow joint The joint radian value derived from a predetermined number of frames in the captured image of the joint radian value and the user's motion can be continuously changed according to the motion. Accordingly, the control unit 150 may calculate a joint radian value derived from a predetermined number of frames (for example, 60 frames) of the image of the pilot motion of the demonstrator and a predetermined number of frames of the user's motion The same number of frames as the predetermined number of frames of the image). In this case, when the controller 150 calculates the similarity using the average value of the joint radian values detected for a predetermined number of frames or the integrated value obtained by integrating the joint radian values for a predetermined number of frames, the controller 150 compares the integrated values The degree of similarity can be calculated. In addition, the degree of similarity can be calculated using an algorithm such as a cosine counting method, an Euclidean distance, or a vector inner product.

As described above, when the degree of similarity is calculated, the exercise ability is evaluated according to the degree of similarity. According to the embodiment of the present invention, if the user follows the guide operation properly, the degree of similarity calculated will be high. Therefore, it is judged that the higher the degree of similarity, the higher the exercise capacity, and the lower the similarity, the lower the exercise capacity. Therefore, the present invention provides a game by adjusting the difficulty level of the game according to the exercise ability.

The game according to the embodiment of the present invention is a game in which a plurality of game operations are sequentially displayed on a screen and when the operation of the user who sequentially follows each of the game operations is equal to or greater than a preset degree of similarity to each game operation, If not, progress will be made in such a way that scoring fails. For example, it is assumed that the game operations A, B, and C each have an upper, middle, or lower level of difficulty. Accordingly, in the embodiment of the present invention, a game operation A having a higher degree of difficulty is allocated to a larger number of games in a game, and a lesser C operation with a lower degree of difficulty is assigned to a game with a relatively higher athletic performance. Conversely, in the embodiment of the present invention, as the exercise capacity is relatively low, a game operation A having a high degree of difficulty is assigned less in a single game, and a C operation with a low degree of difficulty can be allocated a lot. For example, if a user is evaluated to have a high athletic ability, if the game operation provided in one game is provided in the order of A, B, A, B, A, C, B, B, C, B, A, C, B, C, and B in this case.

In addition, in the score judgment and the score failure judgment, the range of the degree of similarity that is recognized as the higher the exercise capacity is set narrowly, and the range of the degree of similarity that is recognized as the score is set to be wider as the exercise capacity is lower. For example, in the case of a user having a relatively high exercise capacity, the game is regarded as a score when the degree of similarity to any one game is 90% or more. On the other hand, in the case of a user having a relatively low exercise capacity, % Or more.

Hereinafter, a method of providing a game according to an embodiment of the present invention will be described. FIG. 8 is a flowchart for explaining a method of providing a game according to an embodiment of the present invention, and FIG. 9 is a diagram for explaining a method of providing a game according to an embodiment of the present invention.

As described above, it is assumed that the user's athletic performance is evaluated and the degree of difficulty is determined. In response to the user's request, the controller 150 starts the game in step S310. When the game starts, the controller 150 sequentially displays the game operations through the display unit 110 in step S320. In step S320, the control unit 150 may further display a guidance message to guide the user to follow the game operation. In addition, the control unit 150 may guide the user to follow the guide operation through voice signal output. An example of the game operation is shown in Fig. For example, the game operations shown in FIGS. 9A, 9B, and 9C may be sequentially displayed through the display unit 110 at predetermined time intervals between game operations. The game operation is photographed in advance through the camera unit 120 according to the embodiment of the present invention like the guide operation, and can be performed by the demonstrator.

When the user follows the game operation displayed through the display unit 110, the control unit 150 photographs the user's operation following the game operation in step S330, and calculates the degree of similarity with the game operation. The similarity calculation method is as described with reference to FIG. 4 to FIG.

As a result of the similarity calculation, the controller 150 determines whether the degree of similarity is equal to or greater than a preset reference value in step S340. As a result of the determination, if the similarity is equal to or greater than the reference value, the controller 150 proceeds to step S350 and determines that the goal has succeeded. Otherwise, the controller 150 proceeds to step S360 and determines that the goal has failed.

This game procedure continues until the controller 370 determines that the game is ended in step S370. That is, the steps S320 to S370 are repeated until the game ends. For example, the game operations assigned to one game as shown in Figs. 9A, 9B, and 9C are sequentially displayed on the display unit 110, and whether or not the game is scored according to the determination of similarity. If it is determined in step S370 that the game is terminated, the controller 370 proceeds to step S380 and ends the game.

Meanwhile, according to the embodiment of the present invention, when the game ends, the exercise ability of the user can be re-evaluated by using the degree of similarity calculated according to the user's game operation. Then, you can reset the difficulty of the game through the reevaluated athletic ability.

Meanwhile, a method for providing a game according to an embodiment of the present invention may be implemented in a form of a program readable by various computer means and recorded in a computer-readable recording medium. Here, the recording medium may include program commands, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a recording medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. For example, the recording medium may be a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical medium such as a CD-ROM or a DVD, a magneto-optical medium such as a floppy disk magneto-optical media, and hardware devices that are specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions may include machine language wires such as those produced by a compiler, as well as high-level language wires that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been described with reference to several preferred embodiments, these embodiments are illustrative and not restrictive. It will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit of the invention and the scope of the appended claims.

100: user equipment 110: display unit
120: camera unit 130: storage unit
140: input unit 150:

Claims (9)

A game providing apparatus based on motion recognition,
A display unit for screen display;
A camera unit for capturing an image; And
A guide operation of a pilot photographed in advance through the display unit is displayed,
And a controller for calculating a degree of similarity between the guide operation and a user's operation by photographing an operation of a user moving along the guide operation through the camera unit, evaluating the user's exercise ability according to the degree of similarity, And controlling the degree of difficulty of the game to be displayed through the display unit in a game for sequentially following the game operation. The method according to claim 1,
The control unit
Dimensional coordinates of a plurality of joints from an operation of the user photographed through the camera unit,
Wherein when one of the plurality of joints and two neighboring joints of the one joint are connected by two line segments based on the three-dimensional position coordinate, an angle formed by the two line segments using an inner product of the vector ≪ / RTI >
A joint radian value obtained by expressing the obtained angle as a radian value is derived,
Wherein the degree of similarity is calculated by comparing the derived joint radian value with the joint radian value derived in advance from the guide operation. 3. The method of claim 2,
The control unit
A joint radian value derived from a predetermined number of frames and an average value or an integral value of a joint radian value derived from a frame having the same number as the predetermined number in an image captured by the user, Wherein the game device is a game device. The method according to claim 1,
The control unit
A plurality of game operations are sequentially displayed through the display unit,
A camera unit for capturing an action of a user following the game operation,
And judges that the player has succeeded in scoring if the action of the photographed user and the similarity of the game operation are equal to or greater than a preset reference value in accordance with the degree of difficulty. A method of providing a game based on motion recognition,
Photographing an action of a user moving along a guiding operation of a photographed demonstrator;
Calculating a degree of similarity between the guide operation and a user's operation;
Evaluating the user's exercise ability according to the degree of similarity; And
And adjusting the degree of difficulty of the game in a game in which a plurality of game actions are sequentially performed according to the exercise ability. 6. The method of claim 5,
The step of calculating the degree of similarity
Extracting three-dimensional position coordinates of a plurality of joints from the operation of the user;
Wherein when two joints of one of the plurality of joints and two adjacent joints of the one joint are connected by two line segments based on the three-dimensional position coordinate, an angle formed by the two line segments using an inner product of the vector ;
Deriving a joint radian value representing the obtained angle as a radian value;
And calculating the degree of similarity by comparing the derived joint radian value with the joint radian value derived in advance from the guide operation. The method according to claim 6,
The step of calculating the similarity by comparing the derived joint radian value with the joint radian value derived in advance from the guide operation
A joint radian value derived from a predetermined number of frames and an average value or an integral value of a joint radian value derived from a frame having the same number as the predetermined number in an image captured by the user, Wherein the game is a game. 6. The method of claim 5,
Shooting an action of a user following a plurality of game actions sequentially displayed; And
And determining that the player has succeeded in scoring if the captured action of the user and the similarity of the game action are equal to or greater than preset reference values according to the degree of difficulty. 9. A computer-readable recording medium having recorded thereon instructions for executing a game providing method according to any one of claims 5 to 8.
KR1020150188441A 2015-12-29 2015-12-29 Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method KR20170078176A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/KR2015/014440 WO2017115887A1 (en) 2015-12-29 2015-12-29 Device for providing motion recognition-based game, method for same, and computer-readable recording medium on which said method is recorded
KR1020150188441A KR20170078176A (en) 2015-12-29 2015-12-29 Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150188441A KR20170078176A (en) 2015-12-29 2015-12-29 Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method

Publications (1)

Publication Number Publication Date
KR20170078176A true KR20170078176A (en) 2017-07-07

Family

ID=59224818

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150188441A KR20170078176A (en) 2015-12-29 2015-12-29 Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method

Country Status (2)

Country Link
KR (1) KR20170078176A (en)
WO (1) WO2017115887A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190023628A (en) * 2017-08-29 2019-03-08 안영샘 Apparatus for measuring range of motion, method thereof and computer recordable medium storing program to perform the method
WO2022139199A1 (en) * 2020-12-22 2022-06-30 주식회사 네오펙트 Motion evaluation device and motion evaluation method
KR20220117391A (en) * 2021-02-15 2022-08-24 (주)엘앤에이치랩스 Training system according to personal fitness evaluation
KR102490429B1 (en) * 2022-07-25 2023-01-30 주식회사 컴플렉시온 Control method of electronic device to privide gmae content adjust difficuty according to physical level of user

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107441704A (en) * 2017-08-28 2017-12-08 广西师范大学 A kind of integral type game machine
CN111144217B (en) * 2019-11-28 2022-07-01 重庆邮电大学 Motion evaluation method based on human body three-dimensional joint point detection
CN113938720B (en) * 2020-07-13 2023-11-17 华为技术有限公司 Multi-device cooperation method, electronic device and multi-device cooperation system
KR102511498B1 (en) * 2021-01-29 2023-03-16 경남대학교 산학협력단 Virtual training system and method for apprentices evaluation and computing device for executing the system

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20010081193A (en) * 2000-02-10 2001-08-29 이수원 3D virtual reality motion capture dance game machine by applying to motion capture method
KR100772497B1 (en) * 2006-10-09 2007-11-01 박찬애 Golf clinic system and application method thereof
KR101304111B1 (en) * 2012-03-20 2013-09-05 김영대 A dancing karaoke system
KR101446653B1 (en) * 2012-12-24 2014-10-07 디게이트 주식회사 Health and rehabilitation game apparatus, system and game method
KR20150067922A (en) * 2013-12-10 2015-06-19 서인아 A rhythm game device interworking user behavior

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190023628A (en) * 2017-08-29 2019-03-08 안영샘 Apparatus for measuring range of motion, method thereof and computer recordable medium storing program to perform the method
WO2022139199A1 (en) * 2020-12-22 2022-06-30 주식회사 네오펙트 Motion evaluation device and motion evaluation method
KR20220117391A (en) * 2021-02-15 2022-08-24 (주)엘앤에이치랩스 Training system according to personal fitness evaluation
KR102490429B1 (en) * 2022-07-25 2023-01-30 주식회사 컴플렉시온 Control method of electronic device to privide gmae content adjust difficuty according to physical level of user

Also Published As

Publication number Publication date
WO2017115887A1 (en) 2017-07-06

Similar Documents

Publication Publication Date Title
KR20170078176A (en) Apparatus for presenting game based on action recognition, method thereof and computer recordable medium storing the method
US9293118B2 (en) Client device
WO2021008158A1 (en) Method and apparatus for detecting key points of human body, electronic device and storage medium
CN107943291B (en) Human body action recognition method and device and electronic equipment
US10755438B2 (en) Robust head pose estimation with a depth camera
EP2624217A1 (en) Information presentation device, digital camera, head mount display, projector, information presentation method, and information presentation program
US9392248B2 (en) Dynamic POV composite 3D video system
EP2395454A2 (en) Image generation system, shape recognition method, and information storage medium
US20200265633A1 (en) Image processing apparatus, image processing method, and storage medium
US20170316582A1 (en) Robust Head Pose Estimation with a Depth Camera
WO2019100755A1 (en) Video generation method and device, and electronic apparatus
CN112543343B (en) Live broadcast picture processing method and device based on live broadcast with wheat
US10853966B2 (en) Virtual space moving apparatus and method
WO2019100756A1 (en) Image acquisition method and apparatus, and electronic device
TW201403379A (en) Analyzing human gestural commands
US10694115B2 (en) Method, apparatus, and terminal for presenting panoramic visual content
WO2016056317A1 (en) Information processor and information-processing method
WO2022174594A1 (en) Multi-camera-based bare hand tracking and display method and system, and apparatus
US20130040737A1 (en) Input device, system and method
US10970932B2 (en) Provision of virtual reality content
US9261974B2 (en) Apparatus and method for processing sensory effect of image data
JP4659722B2 (en) Human body specific area extraction / determination device, human body specific area extraction / determination method, human body specific area extraction / determination program
EP3209398B1 (en) Silhouette-based limb finding
JP2016021096A (en) Image processing device, image processing method, and program
EP3206188A1 (en) Method and system for realizing motion-sensing control based on intelligent device, and intelligent device

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E601 Decision to refuse application