WO2016021829A1 - 동작 인식 방법 및 동작 인식 장치 - Google Patents
동작 인식 방법 및 동작 인식 장치 Download PDFInfo
- Publication number
- WO2016021829A1 WO2016021829A1 PCT/KR2015/006475 KR2015006475W WO2016021829A1 WO 2016021829 A1 WO2016021829 A1 WO 2016021829A1 KR 2015006475 W KR2015006475 W KR 2015006475W WO 2016021829 A1 WO2016021829 A1 WO 2016021829A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- function
- motion
- condition
- user
- condition function
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Definitions
- the present invention relates to a motion recognition method and a motion recognition device, and more particularly, to a method and apparatus for recognizing a user's motion accurately and efficiently by quickly determining comparison target information corresponding to the user's motion.
- NUI devices for inputting commands to a computer by a user's voice or operation, etc., rather than an input method such as text, have been developed and spread.
- the current NUI device is only to identify the user's motion by tracking the joint position according to the user's motion, in addition, efforts to interpret the user's voice or the user's movement input through the NUI device have.
- a dynamic time-warping technique has been proposed.
- a technique for verifying whether a current action taken by a user and a predefined operation for the command is consistent is required.
- a dynamic time warping technique has been proposed.
- an input user's motion needs to be scaled in order to compare with a predefined motion, but the user's motion may be different because the user's body size or the degree of detail of the motion may vary. There is also a problem that is recognized as a completely different operation.
- Korean Patent Publication No. 10-2012-0000807 discloses a motion recognition unit for recognizing a plurality of motions, and a motion recognized by the motion recognition unit. And a controller for controlling the digital device, wherein the controller stops controlling the digital device according to the motion recognized by the motion recognition unit when the motion recognition unit recognizes a first motion among the plurality of motions. After that, when a second motion among the plurality of motions is recognized, a control device for controlling the digital device according to the motion recognized by the motion recognition unit is described. That is, although the prior art document describes the contents of determining the control according to the recognized motion by recognizing the motion, it does not mention a method of effectively recognizing the motion, and still does not solve the above-mentioned problem. .
- the background art described above is technical information that the inventors possess for the derivation of the present invention or acquired in the derivation process of the present invention, and is not necessarily a publicly known technique disclosed to the general public before the application of the present invention. .
- One embodiment of the present invention is to accurately recognize the user's motion.
- an embodiment of the present invention is to minimize the operation time for determining the meaning of the user operation by representing the user's motion input through the NUI device in a binary vector.
- a motion recognition device for recognizing a user's motion received through an NUI device, the motion information acquisition unit configured to obtain the user's motion information and the motion information and the preset comparison target information And an operation determiner configured to perform dynamic time warping of the liver.
- acquiring motion information of a user performing dynamic time warping between the motion information and predetermined comparison target information;
- a computer readable recording medium having recorded thereon a program for performing a method comprising the steps of: may be provided.
- the computer the step of obtaining the user's motion information, dynamic time warping between the motion information and the predetermined comparison target information (dynamic time warping)
- a computer program may be provided that includes computer program code for performing a method comprising performing a warping.
- an embodiment of the present invention can accurately recognize the operation of the user.
- minute differences in body size or motion which are inevitably different for each user, do not hinder precisely determining the motion according to the user's intention.
- the operation time for determining the meaning of the user operation can be minimized by representing the user's operation input through the NUI device in a binary vector.
- the sequence of the binary vector can have the same value for two consecutive poses, so that the same length animation can be converted into fewer poses.
- FIG. 1 is a block diagram of a system for recognizing a user's operation through an NUI apparatus according to an embodiment of the present invention.
- FIG. 2 is a block diagram illustrating an operation recognition apparatus according to an embodiment of the present invention.
- FIG. 3 is a block diagram illustrating an operation information acquisition unit according to an embodiment of the present invention.
- FIG. 4 is a flowchart illustrating a motion recognition method according to an embodiment of the present invention.
- a system 10 includes an NUI apparatus 100 and a gesture recognition apparatus 200.
- the NUI device 100 uses all of the commands available for the command of a software program or application that can recognize a user's motion or voice using at least one of an image sensor, a depth sensor, a motion sensor and a voice sensor. Means a device.
- the NUI device 100 may be implemented as, for example, a tablet PC equipped with a touch screen, a smartphone, a RGB camera or a depth camera, a Kinect, or the like. If the NUI device 100 according to the present invention is implemented as a depth camera, for example, the user's motion, and transmit a photographed image frame, or composed of three-dimensional positions of the joint of the user from the photographed image frame The extracted pose information may be transmitted by extracting the pose information.
- the NUI device 100 captures the whole or part of the user's body to determine the user's body part, or to track the three-dimensional position of the joint, or the movement of the joint, to the three-dimensional positions of the user's joints.
- a frame including the pose information may be obtained.
- the NUI apparatus 100 may transmit the pose information or the frame obtained as described above to the motion recognition apparatus 200 through wired / wireless communication means.
- the gesture recognition apparatus 200 analyzes the pose information received from the NUI apparatus 100 (or extracts pose information from the received image frame and analyzes the extracted pose information) to obtain user's motion information. By comparing the motion information and the preset comparison target information, the user's motion in the real world can be accurately recognized and further, a command corresponding to the user's motion can be determined.
- the 'motion information' refers to sequence information when the user poses are arranged in order of time
- the 'comparison target information' refers to information defining the meaning (or command) of the user's motion. 200).
- the gesture recognition apparatus 200 may perform some of the functions performed by the NUI apparatus 100.
- the NUI device 100 is included in the gesture recognition apparatus 200 or the NUI device 100 is included in the gesture recognition apparatus 200 so that the gesture recognition apparatus 200 may be configured to perform the function of the NUI apparatus 100. At least some may be done.
- the gesture recognition apparatus 200 performs at least some of the functions performed by the NUI apparatus 100. For more detailed configuration of the gesture recognition apparatus 200, see FIG. Will be described later.
- system 10 may further include a command processing device 300.
- the command is transmitted to the command processing apparatus 300 through wired / wireless communication means.
- the process conforming to the above instructions can be carried out.
- the command processing device 300 is a device for displaying a cursor on the screen of the electronic device, and the operation of a specific joint as the comparison target information is defined as the movement of the cursor, when the user moves the specific joint
- the movement may be acquired by the NUI device 100 as pose information, and the motion recognition apparatus 200 performs dynamic time warping between the motion information acquired based on the pause information and the comparison target information, and as a result, the motion of the user.
- the command may be determined, and the command processing apparatus 300 may move a process corresponding to the command, that is, a cursor on the screen.
- command processing apparatus 300 is illustrated as being separate from the motion recognition apparatus 200 in FIG. 1 for convenience of description, it is included in the motion recognition apparatus 200 or includes the motion recognition apparatus 200. can do.
- the gesture recognition apparatus 200 includes an information receiver 210, a gesture information acquirer 220, and a gesture determiner 230.
- the information receiver 210 may receive an image frame photographed by the NUI device 100 or may receive pose information extracted from the image frame.
- the series of operations is defined as a 'real operation' of the user for convenience of description.
- the NUI apparatus 100 may acquire an image frame including pose information constituting the actual motion by capturing the actual motion of the user every second and transmit the image frame to the motion recognition apparatus 200, or extract the pose information from the image frame.
- the pause information may be transmitted to the gesture recognition apparatus 200.
- the information receiver 210 may receive an image frame or pose information.
- the information receiver 210 may extract pose information from the image frame.
- the pose information according to the above description may be composed of three-dimensional positions of a user joint, and more specifically, may be defined as follows.
- the joint number Is a set of joints,
- the frame where the pose is recorded Is the 3D position of that joint, Represents the linear velocity of the joint.
- Your actions are entered in a sequence, Discrete differential It can be calculated in the form, where h means the time between two frames.
- the motion information acquisition unit 220 obtains the motion information of the user.
- the motion information acquisition unit 220 may convert each of the series of pause information corresponding to the actual motion of the user into a binary vector, and set the converted binary vector as the motion information of the user.
- the motion information acquisition unit 220 may include a condition setting unit 221 and a vector conversion unit 222.
- the condition setting unit 221 generates a conditional expression that takes each of a series of pause information corresponding to a user's actual motion as a factor.
- condition setting unit 221 may set at least one meta-condition function, and may generate a conditional expression according to determining at least one of the elements constituting the meta-condition function. That is, the meta-condition function may be composed of one or more variables and constants, and the condition setting unit 221 may generate a conditional expression according to determining at least one of the constants.
- condition setting unit 221 may set a plane condition function, which is a function of determining a position of a predetermined joint based on a plane generated based on two or more joints, as a meta condition function. It may be expressed as Equation 2 or Equation 3.
- the plane may be defined according to a method of providing three joints or a method of passing a position of another joint while a vector made of two joints is a normal vector.
- condition setting unit 221 is a meta-conditional function such as the following equation (4), i.e., an angle condition function that can generate a conditional expression for determining whether the angle generated by the four joints is within a specific range. Can be set as
- the specified joint Is the vector to which Is the angle produced by the vector to which end Wow In between, Equation 4 may generate a positive value or a negative value.
- Is And said Positive value or negative value may be determined depending on whether is within a predetermined range.
- the angular condition function can be used to generate various conditional expressions such as whether the elbow angle is extended or bent, and whether the arms are extended or side-by-side by examining the angle between the vectors from both shoulders to the hand.
- condition setting unit 221 may set an access condition function such as Equation 5 as a meta condition function, and the access condition function may generate a condition expression for determining whether a distance between two joints is within a specific range. That is, the condition setting unit 221 may generate a conditional expression that inputs two joints that are subject to the condition, two joints that are the criteria of the condition, and a magnification constant value, and the access condition function is expressed by Equation 5 below. It can be expressed as.
- condition setting unit 221 may set a meta condition function that can generate a conditional expression regarding the speed condition. That is, the condition setting unit 221 may set each of the speed direction condition function as shown in Equation 6 below and the speed magnitude condition function as shown in Equation 7 as a meta condition function.
- Equation (6) is an equation representing a velocity direction condition function capable of generating a conditional expression for determining whether the velocity of an arbitrary joint and the angle formed by the vector connecting two designated joints are within a specified range.
- condition setting unit 221 may generate various conditional expressions such as whether the right hand is moving up and down or left and right, or the foot is moving in the vertical or direction.
- Equation 7 is a velocity magnitude condition function that can generate a conditional expression to determine how large the magnitude of the velocity of any joint compared to the magnitude of the vector connecting the two reference joints specified.
- the function according to Equation 6 may generate an error because it operates regardless of the magnitude of the speed, but when used together with the speed magnitude condition function according to Equation 7, the conditional expression can be set to have meaning only for speeds above the reference value. .
- condition setting unit 221 may generate various condition expressions.
- the vector converter 222 may obtain motion information by converting the motion of the user into a binary vector according to the conditional expression generated by the condition setting unit 221.
- the vector converter 222 may input each of the pose information included in the pose set corresponding to the motion information of the user as a factor of the conditional expression to obtain a binary value corresponding to each pose information.
- a binary vector of an operation consisting of a series of pause information may be generated, and the binary vector may be obtained as motion information.
- the operation determiner 230 may determine the comparison target information that matches the actual operation of the user by comparing the operation information and the comparison target information.
- the operation determining unit 230 may include a storage unit (not shown) for storing the comparison target information, or may communicate with a storage device (not shown), for example, a database, located outside.
- At least one comparison target information exists, and by performing dynamic time warping between each comparison target information and motion information, the comparison target information closest to the user's actual operation among the at least one comparison target information. Can be determined.
- the operation determiner 230 may detect that a command corresponding to the actual operation of the user is input.
- the gesture recognition method according to the exemplary embodiment shown in FIG. 4 includes steps processed in time series by the gesture recognition apparatus 200 shown in FIGS. 2 and 3. Therefore, even if omitted below, the above descriptions of the gesture recognition apparatus 200 illustrated in FIGS. 2 and 3 may also be applied to the gesture recognition method according to the exemplary embodiment shown in FIG. 4.
- the gesture recognition apparatus 200 may receive a user's actual motion (S4000).
- the gesture recognition apparatus 200 may receive a series of image frames corresponding to the actual motion of the user through the NUI apparatus 100.
- the gesture recognition apparatus 200 may respectively receive the image frames.
- the gesture recognition apparatus 200 may receive the gesture of the user.
- the pose information extraction process as described above may be performed by the NUI device 100 so that the gesture recognition apparatus 200 may receive a user's motion by receiving the pose information.
- the gesture recognition apparatus 200 may obtain the gesture information by converting the gesture of the user into a binary vector (S4001).
- the gesture recognition apparatus 200 may generate various conditional expressions for generating a binary vector corresponding to a user's motion by using a metacondition function.
- the gesture recognition apparatus 200 may generate a conditional expression for determining whether “the left hand is in front of the body” or “the right hand is in front of the body” according to Equation 2.
- the gesture recognition apparatus 200 may be configured such that “the left hand is outside the body”, “the right hand is outside the body”, “the left hand is inside the body”, or “the right hand is the body” according to equation (3). Create conditional expressions that can determine whether it is inside, “whether the left hand is higher than the torso", “the right hand is higher than the torso,” “the left hand is higher than the head,” or “the right hand is higher than the head.” .
- the motion recognition apparatus 200 according to equation (4), "the angle made by the left arm elbow is between (0 °, 60 °)", “the angle made by the left arm elbow (60 °, 120 °) ”,“ the angle between the left arm elbow is between (120 °, 180 °) ”,“ the angle between the right arm elbow is between (0 °, 60 °) ”,“ the right arm elbow is Make angle between (60 °, 120 °) ”,“ make angle between right arm elbow (120 °, 180 °) ”,“ angle between left knee (0 °, 60 °) "," The angle between the left knee is between (60 °, 120 °) "," the angle between the left knee is between (120 °, 180 °) "," the angle between the right knee is (0 °, 60 °) ”,“ The angle of the right knee is between (60 °, 120 °) ”,“ The angle of the right knee is between (60 °, 120 °) ”,“ The angle of the right knee is between (120 °,
- the gesture recognition apparatus 200 may determine whether two hands are close to each other, a left hand is near the head, or a right hand is near the head, according to Equation 5. Conditional expressions.
- the motion recognition apparatus 200 "the speed direction of the left hand is the vertical direction", “the speed direction of the right hand is the vertical direction”, “the speed direction of the left hand is the horizontal direction Can generate a conditional expression to determine whether or not "or” the right hand's velocity is horizontal. "
- the gesture recognition apparatus 200 may determine whether the speed of the left hand passes both shoulders in one second or the speed of the right hand in one second, according to Equation 7 below. You can create a conditional expression to determine whether it is past level.
- conditional expression as described above may be generated before or after each step S4000 to S4002 of FIG. 4 is performed, or may be generated even while each step is in progress.
- the pose information according to the user's motion can be substituted into the various conditional expressions generated as described above, thereby generating a binary vector corresponding to the user's motion.
- the motion recognition apparatus 200 may set 30 or more conditional expressions and generate a 30-dimensional or more binary vector corresponding to a user motion.
- the generated binary vector may be set as motion information, and the motion recognition apparatus 200 may perform motion time warping between the motion information and the comparison target information as described above (S4002).
- the difference calculation between the motion information converted into the binary vector and the comparison target information may be simply performed by a logical operation between both binary vectors.
- the gesture recognition apparatus 200 may cause the command processing apparatus 300 to process the command.
- the motion recognition method according to the embodiment described with reference to FIG. 4 may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer.
- Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media.
- Computer readable media may include both computer storage media and communication media.
- Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
- Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.
- the gesture recognition method may be implemented as a computer program (or computer program product) including instructions executable by a computer.
- the computer program includes programmable machine instructions processed by the processor and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language.
- the computer program may also be recorded on tangible computer readable media (eg, memory, hard disks, magnetic / optical media or solid-state drives, etc.).
- the gesture recognition method may be implemented by executing the computer program as described above by the computing device.
- the computing device may include at least a portion of a processor, a memory, a storage device, a high speed interface connected to the memory and a high speed expansion port, and a low speed interface connected to the low speed bus and the storage device.
- a processor may include at least a portion of a processor, a memory, a storage device, a high speed interface connected to the memory and a high speed expansion port, and a low speed interface connected to the low speed bus and the storage device.
- Each of these components are connected to each other using a variety of buses and may be mounted on a common motherboard or otherwise mounted in a suitable manner.
- the processor may process instructions within the computing device, such as to display graphical information for providing a graphical user interface (GUI) on an external input, output device, such as a display connected to a high speed interface. Instructions stored in memory or storage. In other embodiments, multiple processors and / or multiple buses may be used with appropriately multiple memories and memory types.
- the processor may also be implemented as a chipset consisting of chips comprising a plurality of independent analog and / or digital processors.
- the memory also stores information within the computing device.
- the memory may consist of a volatile memory unit or a collection thereof.
- the memory may consist of a nonvolatile memory unit or a collection thereof.
- the memory may also be other forms of computer readable media, such as, for example, magnetic or optical disks.
- the storage device can provide a large amount of storage space to the computing device.
- the storage device may be a computer readable medium or a configuration including such a medium, and may include, for example, devices or other configurations within a storage area network (SAN), and may include a floppy disk device, a hard disk device, an optical disk device, Or a tape device, flash memory, or similar other semiconductor memory device or device array.
- SAN storage area network
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Social Psychology (AREA)
- Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (18)
- 동작 인식 장치에 의해 수행되는, NUI장치를 통하여 수신되는 사용자의 동작을 인식하기 위한 방법에 있어서,(a) 사용자의 동작정보를 획득하는 단계; 및(b) 상기 동작정보 및 기설정된 비교대상정보 간의 동적 타임 와핑(dynamic time warping)을 수행하는 단계를 포함하고,상기 (a) 단계는,(a-1) 상기 사용자의 동작을 구성하되 상기 사용자 관절의 3차원 위치들로 구성되는 하나 이상의 포즈정보에 기초하여 상기 사용자의 동작을 바이너리 벡터로 변환하는 단계; 및(a-2) 상기 변환된 바이너리 벡터를 상기 사용자의 동작정보로서 설정하는 단계를 포함하는, 동작인식방법.
- 제 1 항에 있어서,상기 (a-1) 단계는,메타조건함수를 설정하는 단계;상기 메타조건함수를 구성하는 하나 이상의 요소 중 적어도 하나를 결정함으로써 조건식을 생성하는 단계; 및상기 조건식에 따라 상기 사용자의 동작을 바이너리 벡터로 변환하는 단계를 포함하는, 동작인식방법.
- 제 2 항에 있어서,상기 조건식에 따라 바이너리 벡터로 변환하는 단계는,상기 사용자의 동작을 구성하는 하나 이상의 포즈정보 각각을 상기 조건식에 따라 바이너리 벡터로 변환하되, 상기 변환된 바이너리 벡터의 집합을 동작정보로서 설정하는 단계를 포함하는, 동작인식방법.
- 제 2 항에 있어서,상기 메타조건함수를 설정하는 단계는,상기 메타조건함수로서 평면 조건 함수를 설정하는 단계를 포함하고,상기 평면 조건 함수는 소정 관절의 위치를, 2개 이상의 관절에 기초하여 생성되는 평면을 기준으로 판단하는 함수인, 동작인식방법.
- 제 2 항에 있어서,상기 메타조건함수를 설정하는 단계는,상기 메타조건함수로서 각도 조건 함수를 설정하는 단계를 포함하고,상기 각도 조건 함수는 2개 이상의 관절이 연결된 벡터가 복수 개 존재하면, 상기 복수 개의 벡터의 각도가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작인식방법.
- 제 2 항에 있어서,상기 메타조건함수를 설정하는 단계는,상기 메타조건함수로서 접근 조건 함수를 설정하는 단계를 포함하고,상기 접근 조건 함수는 2개 이상의 관절 간의 거리가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작인식방법.
- 제 2 항에 있어서,상기 메타조건함수를 설정하는 단계는,상기 메타조건함수로서 속도 방향 조건 함수를 설정하는 단계를 포함하고,상기 속도 방향 조건 함수는 소정 관절의 속도 및 2개 이상의 관절이 연결된 벡터가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작인식방법.
- 제 2 항에 있어서,상기 메타조건함수를 설정하는 단계는,상기 메타조건함수로서 속도 크기 조건 함수를 설정하는 단계를 포함하고,상기 속도 크기 조건 함수는 2개 이상의 관절이 연결된 벡터의 속도 및 소정 관절의 속도 간의 차이가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작인식방법.
- NUI장치를 통하여 수신되는 사용자의 동작을 인식하기 위한 동작 인식 장치로서,사용자의 동작정보를 획득하도록 구성되는 동작정보획득부; 및상기 동작정보 및 기설정된 비교대상정보 간의 동적 타임 와핑(dynamic time warping)을 수행하도록 구성되는 동작결정부를 포함하고,상기 동작정보획득부는,상기 사용자의 동작을 구성하되 상기 사용자 관절의 3차원 위치들로 구성되는 하나 이상의 포즈정보에 기초하여 상기 사용자의 동작을 바이너리 벡터로 변환하여, 상기 변환된 바이너리 벡터를 상기 사용자의 동작정보로서 설정하도록 구성되는 벡터변환부를 포함하는, 동작 인식 장치.
- 제 9 항에 있어서,상기 동작정보획득부는,메타조건함수를 설정하여 상기 메타조건함수를 구성하는 하나 이상의 요소들 중 적어도 하나를 결정함으로써 조건식을 생성하도록 구성되는 조건설정부를 더 포함하고,상기 벡터변환부는,상기 조건설정부에 의해 생성된 조건식에 따라 상기 사용자의 동작을 바이너리 벡터로 변환하는, 동작 인식 장치.
- 제 10 항에 있어서,상기 벡터변환부는 추가적으로,상기 사용자의 동작을 구성하는 하나 이상의 포즈정보 각각을 상기 조건식에 따라 바이너리 벡터로 변환하되, 상기 변환된 바이너리 벡터의 집합을 동작정보로서 설정하도록 구성되는, 동작 인식 장치.
- 제 10 항에 있어서,상기 조건설정부는 추가적으로,상기 메타조건함수로서 평면 조건 함수를 설정하도록 구성되며,상기 평면 조건 함수는 소정 관절의 위치를, 2개 이상의 관절에 기초하여 생성되는 평면을 기준으로 판단하는 함수인, 동작 인식 장치.
- 제 10 항에 있어서,상기 조건설정부는 추가적으로,상기 메타조건함수로서 각도 조건 함수를 설정하도록 구성되며,상기 각도 조건 함수는 2개 이상의 관절이 연결된 벡터가 복수 개 존재하면, 상기 복수 개의 벡터의 각도가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작 인식 장치.
- 제 10 항에 있어서,상기 조건설정부는 추가적으로,상기 메타조건함수로서 접근 조건 함수를 설정하도록 구성되며,상기 접근 조건 함수는 2개 이상의 관절 간의 거리가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작 인식 장치.
- 제 10 항에 있어서,상기 조건설정부는 추가적으로,상기 메타조건함수로서 속도 방향 조건 함수를 설정하도록 구성되며,상기 속도 방향 조건 함수는 소정 관절의 속도 및 2개 이상의 관절이 연결된 벡터가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작 인식 장치.
- 제 10 항에 있어서,상기 조건설정부는 추가적으로,상기 메타조건함수로서 속도 크기 조건 함수를 설정하도록 구성되며,상기 속도 크기 조건 함수는 2개 이상의 관절이 연결된 벡터의 속도 및 임의의 관절의 속도 간의 차이가 소정의 범위 이내인지 여부를 판단하는 함수인, 동작 인식 장치.
- 제 1 항 내지 제 8 항 중 어느 한 항에 기재된 방법을 수행하는 프로그램이 기록된 컴퓨터 판독가능한 기록매체.
- 컴퓨터 장치와 결합되어,제 1 항 내지 제 8 항 중 어느 한 항에 기재된 방법을 실행시키기 위하여 매체에 저장된 컴퓨터 프로그램.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017505558A JP2017531226A (ja) | 2014-08-07 | 2015-06-25 | 動作認識方法及び動作認識装置 |
CN201580042182.1A CN106662927A (zh) | 2014-08-07 | 2015-06-25 | 动作识别方法和动作识别装置 |
US15/425,039 US10713479B2 (en) | 2014-08-07 | 2017-02-06 | Motion recognition method and motion recognition device for recognizing motion of user received via NUI device by comparing with preset comparison target information |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR10-2014-0101620 | 2014-08-07 | ||
KR1020140101620A KR101515845B1 (ko) | 2014-08-07 | 2014-08-07 | 동작 인식 방법 및 동작 인식 장치 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/425,039 Continuation US10713479B2 (en) | 2014-08-07 | 2017-02-06 | Motion recognition method and motion recognition device for recognizing motion of user received via NUI device by comparing with preset comparison target information |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016021829A1 true WO2016021829A1 (ko) | 2016-02-11 |
Family
ID=53393302
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2015/006475 WO2016021829A1 (ko) | 2014-08-07 | 2015-06-25 | 동작 인식 방법 및 동작 인식 장치 |
Country Status (5)
Country | Link |
---|---|
US (1) | US10713479B2 (ko) |
JP (1) | JP2017531226A (ko) |
KR (1) | KR101515845B1 (ko) |
CN (1) | CN106662927A (ko) |
WO (1) | WO2016021829A1 (ko) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105320937B (zh) * | 2015-09-25 | 2018-08-14 | 北京理工大学 | 基于Kinect的交警手势识别方法 |
US10764281B1 (en) * | 2017-01-09 | 2020-09-01 | United Services Automobile Association (Usaa) | Systems and methods for authenticating a user using an image capture device |
US11532958B2 (en) | 2021-02-18 | 2022-12-20 | Google Llc | Predictive wireless charging of electronic devices |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003062326A (ja) * | 2001-08-23 | 2003-03-04 | Namco Ltd | 画像生成システム、プログラム及び情報記憶媒体 |
KR20040055310A (ko) * | 2002-12-20 | 2004-06-26 | 한국전자통신연구원 | 고속 마커프리 모션 캡쳐 장치 및 방법 |
KR20100002803A (ko) * | 2008-06-30 | 2010-01-07 | 삼성전자주식회사 | 모션 캡쳐 장치 및 모션 캡쳐 방법 |
JP2010033163A (ja) * | 2008-07-25 | 2010-02-12 | Kddi Corp | 動きデータ検索装置及びコンピュータプログラム |
JP2012066026A (ja) * | 2010-09-27 | 2012-04-05 | Sony Corp | 情報処理装置及び方法、並びにプログラム |
JP2013257656A (ja) * | 2012-06-11 | 2013-12-26 | Kddi Corp | 動き類似度算出装置、動き類似度算出方法およびコンピュータプログラム |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101114449A (zh) * | 2006-07-26 | 2008-01-30 | 大连三曦智能科技有限公司 | 非特定人孤立词的模型训练方法、识别系统及识别方法 |
JP5187280B2 (ja) * | 2009-06-22 | 2013-04-24 | ソニー株式会社 | 操作制御装置および操作制御方法 |
KR101072858B1 (ko) | 2009-10-30 | 2011-10-17 | 화영시스템즈주식회사 | Nui를 위한 제스쳐 인식 장치 및 방법 |
KR20120000807A (ko) | 2010-06-28 | 2012-01-04 | 주식회사 셀런 | 모션 인식을 이용한 디지털 디바이스 제어 장치 및 제어 방법 |
US20110317871A1 (en) | 2010-06-29 | 2011-12-29 | Microsoft Corporation | Skeletal joint recognition and tracking system |
US8897491B2 (en) | 2011-06-06 | 2014-11-25 | Microsoft Corporation | System for finger recognition and tracking |
US8929612B2 (en) | 2011-06-06 | 2015-01-06 | Microsoft Corporation | System for recognizing an open or closed hand |
CN102319130B (zh) * | 2011-07-21 | 2014-03-26 | 山东科技大学 | 用脚趾触发上假肢多自由度运动的控制系统和控制方法 |
US9448636B2 (en) * | 2012-04-18 | 2016-09-20 | Arb Labs Inc. | Identifying gestures using gesture data compressed by PCA, principal joint variable analysis, and compressed feature matrices |
US20140134586A1 (en) * | 2012-11-09 | 2014-05-15 | Orthosensor Inc | Orthopedic tool position and trajectory gui |
CN103345627B (zh) * | 2013-07-23 | 2016-03-30 | 清华大学 | 动作识别方法和装置 |
CN103941869B (zh) * | 2014-04-21 | 2017-07-14 | 云南电网公司普洱供电局 | 一种基于动作元的体感姿势识别方法 |
-
2014
- 2014-08-07 KR KR1020140101620A patent/KR101515845B1/ko active IP Right Grant
-
2015
- 2015-06-25 WO PCT/KR2015/006475 patent/WO2016021829A1/ko active Application Filing
- 2015-06-25 CN CN201580042182.1A patent/CN106662927A/zh active Pending
- 2015-06-25 JP JP2017505558A patent/JP2017531226A/ja active Pending
-
2017
- 2017-02-06 US US15/425,039 patent/US10713479B2/en not_active Expired - Fee Related
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003062326A (ja) * | 2001-08-23 | 2003-03-04 | Namco Ltd | 画像生成システム、プログラム及び情報記憶媒体 |
KR20040055310A (ko) * | 2002-12-20 | 2004-06-26 | 한국전자통신연구원 | 고속 마커프리 모션 캡쳐 장치 및 방법 |
KR20100002803A (ko) * | 2008-06-30 | 2010-01-07 | 삼성전자주식회사 | 모션 캡쳐 장치 및 모션 캡쳐 방법 |
JP2010033163A (ja) * | 2008-07-25 | 2010-02-12 | Kddi Corp | 動きデータ検索装置及びコンピュータプログラム |
JP2012066026A (ja) * | 2010-09-27 | 2012-04-05 | Sony Corp | 情報処理装置及び方法、並びにプログラム |
JP2013257656A (ja) * | 2012-06-11 | 2013-12-26 | Kddi Corp | 動き類似度算出装置、動き類似度算出方法およびコンピュータプログラム |
Also Published As
Publication number | Publication date |
---|---|
KR101515845B1 (ko) | 2015-05-04 |
US10713479B2 (en) | 2020-07-14 |
JP2017531226A (ja) | 2017-10-19 |
CN106662927A (zh) | 2017-05-10 |
US20170147873A1 (en) | 2017-05-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2016017932A1 (ko) | 사용자의 시점을 고려하여 동작인식하는 인터페이스 제공방법 및 제공장치 | |
WO2015137742A1 (en) | Display apparatus and controlling method thereof | |
WO2020017890A1 (en) | System and method for 3d association of detected objects | |
WO2016107231A1 (zh) | 一种3d场景中输入手势的系统和方法 | |
WO2020231005A1 (ko) | 영상 처리 장치 및 그 동작방법 | |
WO2022059969A1 (ko) | 심전도 데이터 분류를 위한 심층 신경망 사전 학습 방법 | |
WO2016021829A1 (ko) | 동작 인식 방법 및 동작 인식 장치 | |
WO2021112382A1 (en) | Apparatus and method for dynamic multi-camera rectification using depth camera | |
EP3983951A1 (en) | Multi-task fusion neural network architecture | |
WO2022014790A1 (en) | Guided backpropagation-gradient updating for image processing task using redundant information from image | |
WO2022139327A1 (en) | Method and apparatus for detecting unsupported utterances in natural language understanding | |
WO2016017931A1 (ko) | Nui 장치를 통하여 사용자와 상호작용하는 인터페이스 제공방법 및 제공장치 | |
WO2021221490A1 (en) | System and method for robust image-query understanding based on contextual features | |
WO2022025458A1 (ko) | 전자 장치 및 그 제어 방법 | |
WO2016021830A1 (ko) | Nui 장치에 의해 수집된 동작정보의 보정 방법 및 장치 | |
WO2015034131A2 (ko) | 공간 제스처 인식 장치 및 방법 | |
EP3186956A1 (en) | Display device and method of controlling therefor | |
KR101412513B1 (ko) | 프레임 그래버 보드를 이용한 로봇팔 제어시스템 및 그 방법 | |
WO2020111844A2 (ko) | 물체 레이블을 활용한 비주얼 슬램에서의 영상 특징점 강화 방법 및 장치 | |
WO2017026834A1 (ko) | 반응형 영상 생성방법 및 생성프로그램 | |
WO2016159613A1 (ko) | 영상 합성을 위한 객체 추적 방법 및 시스템 | |
WO2016085122A1 (ko) | 사용자 패턴 기반의 동작 인식 보정 장치 및 그 방법 | |
WO2022191424A1 (ko) | 전자 장치 및 그 제어 방법 | |
WO2019100547A1 (zh) | 投影控制方法、装置、投影交互系统及存储介质 | |
JP6467994B2 (ja) | 画像処理プログラム、画像処理装置、及び画像処理方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15829106 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2017505558 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12.05.2017) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15829106 Country of ref document: EP Kind code of ref document: A1 |