US9001129B2 - Processing apparatus and method for creating avatar - Google Patents

Processing apparatus and method for creating avatar Download PDF

Info

Publication number
US9001129B2
US9001129B2 US13/276,767 US201113276767A US9001129B2 US 9001129 B2 US9001129 B2 US 9001129B2 US 201113276767 A US201113276767 A US 201113276767A US 9001129 B2 US9001129 B2 US 9001129B2
Authority
US
United States
Prior art keywords
skeleton
function
sizes
target user
avatar
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/276,767
Other versions
US20120127164A1 (en
Inventor
Taehyun RHEE
Inwoo Ha
Dokyoon Kim
Xiaolin Wei
Jinxiang Chai
Huajun Liu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Texas A&M University System
Original Assignee
Samsung Electronics Co Ltd
Texas A&M University System
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd, Texas A&M University System filed Critical Samsung Electronics Co Ltd
Priority to US13/276,767 priority Critical patent/US9001129B2/en
Assigned to TEXAS A&M UNIVERSITY SYSTEM, SAMSUNG ELECTRONICS CO., LTD. reassignment TEXAS A&M UNIVERSITY SYSTEM ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHAI, JINXIANG, HA, INWOO, KIM, DOKYOON, Liu, Huajun, RHEE, TAEHYUN, WEI, XIAOLIN
Publication of US20120127164A1 publication Critical patent/US20120127164A1/en
Application granted granted Critical
Publication of US9001129B2 publication Critical patent/US9001129B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects

Definitions

  • Embodiments of the following description relate to a processing apparatus and method for creating an avatar.
  • various sensors may be necessary to sense a motion of the person. As more sensors are attached to a body of the person, the motion of the person can be realized more accurately.
  • a processing apparatus for creating an avatar including a skeleton database to store statistics of skeleton sizes with respect to reference poses of a plurality of users; a function determination unit to determine a difference function representing a difference between a forward kinematics function, the forward kinematics function regarding joints of the avatar with respect to a reference pose of a target user, and positions of sensors attached to the target user, and to determine a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users; a calculation unit to calculate skeleton sizes of the joints of the avatar, and local coordinates corresponding to the sensors using at least one processor; and a display unit to display the avatar based on the calculated skeleton sizes of the joints and the calculated local coordinates.
  • the skeleton prior function may include a mean, an eigen vector, and an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
  • the calculation unit may calculate the skeleton sizes of the joints of the avatar and the local coordinates corresponding to the sensors, the skeleton sizes and the local coordinates to minimize a sum of the difference function and the skeleton prior function.
  • the processing apparatus may further include a preprocessing unit to define the reference pose and to store information containing the statistics of the skeleton sizes of the reference poses of the plurality of users, in the skeleton database.
  • the processing apparatus may further include an interface unit to receive information on the positions of the sensors attached to the target user from the sensors.
  • the processing apparatus periodically updates the stored skeleton sizes according to changes of the reference poses.
  • the processing apparatus non-periodically updates the stored skeleton sizes according to changes of the reference poses.
  • the sensors of the processing apparatus may be provided in five positions consisting of the forehead, both wrists, a chest, and a waist.
  • a processing apparatus for creating an avatar including a function determination unit to determine a difference function representing a difference between a forward kinematics function, the forward kinematics function regarding joints of the avatar corresponding to a reference pose of a target user, and positions of sensors attached to the target user, and to determine a skeleton prior function based on statistics of skeleton sizes corresponding to reference poses of a plurality of users; a calculation unit to calculate skeleton sizes of the joints of the avatar, and local coordinates corresponding to the sensors using at least one processor; and a display unit to display the avatar based on the calculated skeleton sizes of the joints and the calculated local coordinates.
  • the skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
  • a processing method for creating an avatar including maintaining a skeleton database storing statistics of skeleton sizes with respect to reference poses of a plurality of users; determining a difference function representing a difference between a forward kinematics function regarding joints of an avatar with respect to a reference pose of a target user and positions of sensors attached to the target user; determining a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users; calculating skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function using at least one processor; and displaying the avatar based on the calculated sizes of the joints of the avatar and the calculated local coordinates.
  • the skeleton prior function may include a mean, an eigen vector, and an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
  • the skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
  • the processing method may further include defining the reference poses; and storing information containing the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, in the skeleton database.
  • the processing method may further include receiving information on positions of the sensors attached to the target user from the sensors.
  • a process for creating an avatar including determining a difference function representing a difference between a forward kinematics function regarding joints of the avatar corresponding to a reference pose of a target user and positions of sensors attached to the target user; determining a skeleton prior function based on statistics of skeleton sizes corresponding to reference poses of a plurality of users; calculating skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function using at least one processor; and displaying the avatar based on the calculated sizes of the joints of the avatar and the calculated local coordinates.
  • the skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
  • At least one non-transitory computer readable medium storing computer readable instructions to implement methods of one or more embodiments.
  • FIG. 1 illustrates a block diagram of a processing apparatus for creating an avatar, according to one or more embodiments
  • FIGS. 2A and 2B illustrate diagrams showing a target user taking a reference pose, the target user being attached with sensors, and an avatar created according to the reference pose, according to one or more embodiments;
  • FIG. 3 illustrates a flowchart explaining a processing method for creating an avatar, according to one or more embodiments.
  • FIG. 1 illustrates a block diagram of a processing apparatus for creating an avatar, according to one or more embodiments.
  • the processing apparatus may include a skeleton database 110 , a function determination unit 120 , and a calculation unit 130 . Additionally, the processing apparatus may include sensors 50 , a preprocessing unit 140 , and an interface unit 150 .
  • the sensors 50 may be attached to a body of a target user, that is, an object corresponding to the avatar to be created, so as to sense a pose of the target user.
  • Motion sensors and other functionally similar sensors may be used as the sensors 50 .
  • sensors 50 When excessive sensors 50 are attached to the target user, efficiency of motions of the target user and attaching and detaching efficiency of the sensors may be reduced. Considering this, five sensors may be attached as shown in FIG. 2B according to one or more embodiments. Sensor attachment positions on the body of the target user and a shape of the avatar created according to reference poses will be described in detail with reference to FIGS. 2A and 2B .
  • the skeleton database 110 may store information including statistics of skeleton sizes with respect to (corresponding to or reflecting) reference poses of a plurality of users.
  • the statistics of the skeleton sizes, stored in the skeleton database 110 may be updated periodically or non-periodically according to changes of the reference poses.
  • the reference poses may include a T-shaped pose with both arms laterally stretched, an inverse-L-shaped pose with both arms stretched forward, a boxing pose, a walking pose, a running pose, and the like.
  • various other poses may be defined as the reference poses.
  • the function determination unit 120 may determine a difference function representing a difference between a forward kinematics function regarding respective joints of the avatar with respect to a reference pose of a target user, and positions of sensors attached to the target user. In addition, the function determination unit 120 may determine a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
  • the forward kinematics refer to a kinematics animation method that expresses motions based on position, speed, and acceleration of a 3-dimensional (3D) object, without regard to a force of generating the motions of the 3D object.
  • the forward kinematics method determines the positions of all parts of the 3D object based on joint angles of all joints. Therefore, motions and a final position of a model, for example the avatar, may be determined by setting the joint angles of all the joints in advance. According to the forward kinematics method, optimum positions of all the parts of the 3D object may be found through several attempts of trial and error.
  • the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user may indicate the positions of the joints of the avatar with respect to the reference pose in a 3D space.
  • the calculation unit 130 may calculate the skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors 50 , using the difference function and the skeleton prior function.
  • calculation unit 130 may calculate the skeleton sizes and the local coordinates so that a sum of the difference function and the skeleton prior function is minimized.
  • the calculation unit 130 may perform the calculation using Equation 1 below.
  • the difference function ( ⁇ (s, z; ⁇ tilde over (q) ⁇ t ) ⁇ tilde over (c) ⁇ t ⁇ 2 ) represents the difference between the forward kinematics function ( ⁇ (s, z; ⁇ tilde over (q) ⁇ t )), the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user, and the positions ( ⁇ tilde over (c) ⁇ t ) of the sensors 50 attached to the target user.
  • s denotes the skeleton size of the reference pose of the target user
  • z denotes the local coordinate of the sensor
  • ⁇ tilde over (q) ⁇ t denotes the joint angle with respect to the reference pose of the target user.
  • the skeleton prior function may include a mean e 0 , an eigen vector e n , and an eigen value ⁇ n calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
  • B denotes a number of main eigen vectors among the eigen vectors e n arranged in order of importance.
  • the skeleton prior function may be calculated and stored in a database in advance, and may be used to reduce ambiguity in the skeleton size according to the user.
  • the skeleton prior function may use a principal component analysis (PCA) scheme.
  • PCA principal component analysis
  • the PCA scheme is used in statistics to analyze a set of data. According to the PCA scheme, when each data is mapped to one axis, an axis having a greatest distribution becomes a first coordinate axis, an axis having a second greatest distribution becomes a second coordinate axis, and so forth, in the same manner. Thus, linear conversion of the data is achieved on a new coordinate system.
  • the PCA scheme may be the optimum linear conversion method to preserve a subspace having the greatest distribution. Different from other linear conversion methods, a basis vector of the PCA is not fixed but may be varied according to characteristics of the data. Thus, the PCA may be applied in various manners by disposing a most principal component on each axis.
  • the calculation unit 130 may calculate the skeleton sizes (s) of the joints of the avatar and the local coordinates ( ⁇ tilde over (q) ⁇ t ) corresponding to the sensors for minimizing the sum of the difference function and the skeleton prior function.
  • the preprocessing unit 140 may define the reference pose, and store the information including the statistics of the skeleton sizes with respect to the reference sizes of the plurality of users in the skeleton database 110 .
  • the interface unit 150 may receive information on positions of the sensors 50 attached to the target user, from the sensors 50 .
  • FIGS. 2A and 2B illustrate diagrams showing a target user taking a reference pose, the target user being attached with sensors, and an avatar created according to the reference pose, according to one or more embodiments.
  • FIG. 2A shows the avatar created in the 3D space corresponding to the reference pose and FIG. 2B shows the target user taking the reference pose, being attached with the sensors 210 .
  • the avatar of FIG. 2A denotes the corresponding locations of the sensors 210 of the target user using reference numeral 230 .
  • a position of the avatar created in the 3D space may be determined by the joint angles of the joints corresponding to the reference poses, in accordance with the forward kinematics function.
  • the sensors may be provided at five positions on the target user to minimize the used number of sensors for creating the avatar.
  • the five positions may include a forehead, both wrists, a chest, and a waist. However, the positions and the number of sensors may be adjusted as necessary.
  • FIG. 3 illustrates a flowchart explaining a processing method for creating an avatar, according to one or more embodiments.
  • the processing method maintains a skeleton database storing statistics of skeleton sizes with respect to reference poses of a plurality of users, in operation 310 .
  • information stored in the skeleton database may include the statistics of the skeleton sizes with respect to predetermined reference poses of the plurality of users taking the reference poses.
  • the processing method may define the reference poses, and store the information including the statistics of the skeleton sizes with respect to the reference poses of the plurality of users in the skeleton database.
  • the processing method may receive information on positions of sensors attached to a target user, from the sensors, in operation 320 .
  • the processing method may determine a difference function ( ⁇ (s, z; ⁇ tilde over (q) ⁇ t ) ⁇ tilde over (c) ⁇ t ⁇ 2 ) representing a difference between a forward kinematics function ( ⁇ (s, z; ⁇ tilde over (q) ⁇ t )) regarding joints of the avatar with respect to a reference pose of the target user and positions ( ⁇ tilde over (c) ⁇ t ) of sensors attached to the target user.
  • a difference function ⁇ (s, z; ⁇ tilde over (q) ⁇ t ) ⁇ tilde over (c) ⁇ t ⁇ 2
  • the processing method determines a skeleton prior function
  • the skeleton prior function may include a mean e 0 , an eigen vector e n , and an eigen value ⁇ n calculated by applying the PCA scheme with respect to the statistics of the skeleton sizes.
  • the processing method calculates the skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function.
  • the skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
  • the functions may be understood by referring to the description with respect to FIG. 1 .
  • an avatar with accurate motions may be created in consideration of variation in a figure of a target user, a sensor attachment position, and the like, by using a skeleton prior function based on the statistics of skeleton sizes of references poses of a plurality of users.
  • skeleton prior function containing a mean, an eigen vector, and an eigen value of the skeleton sizes is used, ambiguity in the skeleton size according to the user may be reduced.
  • the methods according to the above-described embodiments may be recorded in non-transitory computer-readable media including computer readable instructions to implement various operations by executing computer readable instructions to control one or more processors, which are part of a general purpose computer, computing device, a computer system, or a network.
  • the media may also have recorded thereon, alone or in combination with the computer readable instructions, data files, data structures, and the like.
  • the computer readable instructions recorded on the media may be those specially designed and constructed for the purposes of embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts.
  • the computer-readable media may also be embodied in at least one application specific integrated circuit (ASIC) or Field Programmable Gate Array (FPGA), which executes (processes like a processor) computer readable instructions.
  • ASIC application specific integrated circuit
  • FPGA Field Programmable Gate Array
  • Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM disks and DVDs; magneto-optical media such as optical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like.
  • the media may be transfer media such as optical lines, metal lines, or waveguides including a carrier wave for transmitting a signal designating the program command and the data construction.
  • Examples of computer readable instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.
  • the described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments, or vice versa.
  • Another example of media may also be a distributed network, so that the computer readable instructions are stored and executed in a distributed fashion.

Abstract

A processing apparatus for creating an avatar is provided. The processing apparatus calculates skeleton sizes of joints of the avatar and local coordinates corresponding to sensors attached to a target user, by minimizing a sum of a difference function and a skeleton prior function, the difference function representing a difference between a forward kinematics function regarding the joints with respect to reference poses of the target user and positions of the sensors, and the skeleton prior function based on statistics of skeleton sizes with respect to reference poses of a plurality of users.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority benefit of U.S. Provisional Application No. 61/394,884 filed on Oct. 20, 2010 in the U.S. Patent and Trademark Office, the disclosure of which is incorporated herein by reference. This application claims the priority benefit of Korean Patent Application No. 10-2010-0109294, filed on Nov. 4, 2010, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference.
BACKGROUND
1. Field
Embodiments of the following description relate to a processing apparatus and method for creating an avatar.
2. Description of the Related Art
In various fields related to a video game, a movie, a computer graphic (CG), and the like, a recent technology has been developed to sense a motion of a target object in a real world and translate the motion into a motion in a 3-dimensional (3D) space.
To realize the motion of the target object, for example a person, in the 3D space, various sensors may be necessary to sense a motion of the person. As more sensors are attached to a body of the person, the motion of the person can be realized more accurately.
However, an increase in the number of sensors may hinder the person from moving smoothly while also complicating attaching and detaching of the sensors. Also, since every person would have a different figure and different sensor attachment positions and directions, consistency of the measurement result may be reduced. Accordingly, there is a demand for a method for generating an accurate pose in consideration of variation in a figure of a user, a sensor attachment position, and the like, while minimizing a number of sensors to be attached to the user.
SUMMARY
According to aspects of one or more embodiments, there is provided a processing apparatus for creating an avatar, including a skeleton database to store statistics of skeleton sizes with respect to reference poses of a plurality of users; a function determination unit to determine a difference function representing a difference between a forward kinematics function, the forward kinematics function regarding joints of the avatar with respect to a reference pose of a target user, and positions of sensors attached to the target user, and to determine a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users; a calculation unit to calculate skeleton sizes of the joints of the avatar, and local coordinates corresponding to the sensors using at least one processor; and a display unit to display the avatar based on the calculated skeleton sizes of the joints and the calculated local coordinates.
The skeleton prior function may include a mean, an eigen vector, and an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
The calculation unit may calculate the skeleton sizes of the joints of the avatar and the local coordinates corresponding to the sensors, the skeleton sizes and the local coordinates to minimize a sum of the difference function and the skeleton prior function.
The processing apparatus may further include a preprocessing unit to define the reference pose and to store information containing the statistics of the skeleton sizes of the reference poses of the plurality of users, in the skeleton database.
The processing apparatus may further include an interface unit to receive information on the positions of the sensors attached to the target user from the sensors.
The processing apparatus periodically updates the stored skeleton sizes according to changes of the reference poses.
The processing apparatus non-periodically updates the stored skeleton sizes according to changes of the reference poses.
The sensors of the processing apparatus may be provided in five positions consisting of the forehead, both wrists, a chest, and a waist.
According to one or more embodiments, there is provided a processing apparatus for creating an avatar including a function determination unit to determine a difference function representing a difference between a forward kinematics function, the forward kinematics function regarding joints of the avatar corresponding to a reference pose of a target user, and positions of sensors attached to the target user, and to determine a skeleton prior function based on statistics of skeleton sizes corresponding to reference poses of a plurality of users; a calculation unit to calculate skeleton sizes of the joints of the avatar, and local coordinates corresponding to the sensors using at least one processor; and a display unit to display the avatar based on the calculated skeleton sizes of the joints and the calculated local coordinates.
The skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
According to one or more embodiments, there is provided a processing method for creating an avatar, including maintaining a skeleton database storing statistics of skeleton sizes with respect to reference poses of a plurality of users; determining a difference function representing a difference between a forward kinematics function regarding joints of an avatar with respect to a reference pose of a target user and positions of sensors attached to the target user; determining a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users; calculating skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function using at least one processor; and displaying the avatar based on the calculated sizes of the joints of the avatar and the calculated local coordinates.
The skeleton prior function may include a mean, an eigen vector, and an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
In the calculating of the skeleton sizes of the joints of the avatar and the local coordinates corresponding to the sensors, the skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
The processing method may further include defining the reference poses; and storing information containing the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, in the skeleton database.
The processing method may further include receiving information on positions of the sensors attached to the target user from the sensors.
According to one or more embodiments, there is provided a process for creating an avatar including determining a difference function representing a difference between a forward kinematics function regarding joints of the avatar corresponding to a reference pose of a target user and positions of sensors attached to the target user; determining a skeleton prior function based on statistics of skeleton sizes corresponding to reference poses of a plurality of users; calculating skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function using at least one processor; and displaying the avatar based on the calculated sizes of the joints of the avatar and the calculated local coordinates.
The skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
According to another aspect of one or more embodiments, there is provided at least one non-transitory computer readable medium storing computer readable instructions to implement methods of one or more embodiments.
BRIEF DESCRIPTION OF THE DRAWINGS
These and/or other aspects will become apparent and more readily appreciated from the following description of embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 illustrates a block diagram of a processing apparatus for creating an avatar, according to one or more embodiments;
FIGS. 2A and 2B illustrate diagrams showing a target user taking a reference pose, the target user being attached with sensors, and an avatar created according to the reference pose, according to one or more embodiments; and
FIG. 3 illustrates a flowchart explaining a processing method for creating an avatar, according to one or more embodiments.
 DETAILED DESCRIPTION
Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. Embodiments are described below to explain the present disclosure by referring to the figures.
FIG. 1 illustrates a block diagram of a processing apparatus for creating an avatar, according to one or more embodiments.
Referring to FIG. 1, the processing apparatus may include a skeleton database 110, a function determination unit 120, and a calculation unit 130. Additionally, the processing apparatus may include sensors 50, a preprocessing unit 140, and an interface unit 150.
The sensors 50 may be attached to a body of a target user, that is, an object corresponding to the avatar to be created, so as to sense a pose of the target user. Motion sensors and other functionally similar sensors may be used as the sensors 50.
When excessive sensors 50 are attached to the target user, efficiency of motions of the target user and attaching and detaching efficiency of the sensors may be reduced. Considering this, five sensors may be attached as shown in FIG. 2B according to one or more embodiments. Sensor attachment positions on the body of the target user and a shape of the avatar created according to reference poses will be described in detail with reference to FIGS. 2A and 2B.
The skeleton database 110 may store information including statistics of skeleton sizes with respect to (corresponding to or reflecting) reference poses of a plurality of users. The statistics of the skeleton sizes, stored in the skeleton database 110, may be updated periodically or non-periodically according to changes of the reference poses.
For example, the reference poses may include a T-shaped pose with both arms laterally stretched, an inverse-L-shaped pose with both arms stretched forward, a boxing pose, a walking pose, a running pose, and the like. Besides the above, various other poses may be defined as the reference poses.
The function determination unit 120 may determine a difference function representing a difference between a forward kinematics function regarding respective joints of the avatar with respect to a reference pose of a target user, and positions of sensors attached to the target user. In addition, the function determination unit 120 may determine a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
The functions determined by the function determination unit 120 will be described in further detail with reference to Equation 1 hereinafter.
Here, the forward kinematics refer to a kinematics animation method that expresses motions based on position, speed, and acceleration of a 3-dimensional (3D) object, without regard to a force of generating the motions of the 3D object.
The forward kinematics method determines the positions of all parts of the 3D object based on joint angles of all joints. Therefore, motions and a final position of a model, for example the avatar, may be determined by setting the joint angles of all the joints in advance. According to the forward kinematics method, optimum positions of all the parts of the 3D object may be found through several attempts of trial and error.
Thus, the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user may indicate the positions of the joints of the avatar with respect to the reference pose in a 3D space.
The calculation unit 130 may calculate the skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors 50, using the difference function and the skeleton prior function.
Also, the calculation unit 130 may calculate the skeleton sizes and the local coordinates so that a sum of the difference function and the skeleton prior function is minimized.
The calculation unit 130 may perform the calculation using Equation 1 below.
argmin s , z , λ n ⁢ f ⁡ ( s , z ; q ~ t ) - c ~ t 2 + s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n [ Equation ⁢ ⁢ 1 ]
wherein, the difference function (∥ƒ(s, z; {tilde over (q)}t)−{tilde over (c)}t∥2) represents the difference between the forward kinematics function (ƒ(s, z; {tilde over (q)}t)), the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user, and the positions ({tilde over (c)}t) of the sensors 50 attached to the target user.
In the forward kinematics function, s denotes the skeleton size of the reference pose of the target user, z denotes the local coordinate of the sensor, and {tilde over (q)}t, denotes the joint angle with respect to the reference pose of the target user.
The skeleton prior function
( s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n )
is calculated based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
The skeleton prior function may include a mean e0, an eigen vector en, and an eigen value λn calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users. B denotes a number of main eigen vectors among the eigen vectors en arranged in order of importance.
The skeleton prior function may be calculated and stored in a database in advance, and may be used to reduce ambiguity in the skeleton size according to the user.
To calculate the mean e0, the eigen vector en, and the eigen value λn from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, the skeleton prior function may use a principal component analysis (PCA) scheme.
The PCA scheme is used in statistics to analyze a set of data. According to the PCA scheme, when each data is mapped to one axis, an axis having a greatest distribution becomes a first coordinate axis, an axis having a second greatest distribution becomes a second coordinate axis, and so forth, in the same manner. Thus, linear conversion of the data is achieved on a new coordinate system.
The PCA scheme may be the optimum linear conversion method to preserve a subspace having the greatest distribution. Different from other linear conversion methods, a basis vector of the PCA is not fixed but may be varied according to characteristics of the data. Thus, the PCA may be applied in various manners by disposing a most principal component on each axis.
As can be understood from Equation 1, by energy minimization, the calculation unit 130 may calculate the skeleton sizes (s) of the joints of the avatar and the local coordinates ({tilde over (q)}t) corresponding to the sensors for minimizing the sum of the difference function and the skeleton prior function.
The preprocessing unit 140 may define the reference pose, and store the information including the statistics of the skeleton sizes with respect to the reference sizes of the plurality of users in the skeleton database 110.
The interface unit 150 may receive information on positions of the sensors 50 attached to the target user, from the sensors 50.
FIGS. 2A and 2B illustrate diagrams showing a target user taking a reference pose, the target user being attached with sensors, and an avatar created according to the reference pose, according to one or more embodiments.
Specifically, FIG. 2A shows the avatar created in the 3D space corresponding to the reference pose and FIG. 2B shows the target user taking the reference pose, being attached with the sensors 210. The avatar of FIG. 2A denotes the corresponding locations of the sensors 210 of the target user using reference numeral 230.
In FIG. 2A, a position of the avatar created in the 3D space may be determined by the joint angles of the joints corresponding to the reference poses, in accordance with the forward kinematics function.
As described above, the sensors may be provided at five positions on the target user to minimize the used number of sensors for creating the avatar. The five positions may include a forehead, both wrists, a chest, and a waist. However, the positions and the number of sensors may be adjusted as necessary.
FIG. 3 illustrates a flowchart explaining a processing method for creating an avatar, according to one or more embodiments.
Referring to FIG. 3, the processing method maintains a skeleton database storing statistics of skeleton sizes with respect to reference poses of a plurality of users, in operation 310.
Here, information stored in the skeleton database may include the statistics of the skeleton sizes with respect to predetermined reference poses of the plurality of users taking the reference poses.
Prior to operation 310, the processing method may define the reference poses, and store the information including the statistics of the skeleton sizes with respect to the reference poses of the plurality of users in the skeleton database.
The processing method may receive information on positions of sensors attached to a target user, from the sensors, in operation 320.
In operation 330, the processing method may determine a difference function (∥ƒ(s, z; {tilde over (q)}t)−{tilde over (c)}t∥2) representing a difference between a forward kinematics function (ƒ(s, z; {tilde over (q)}t)) regarding joints of the avatar with respect to a reference pose of the target user and positions ({tilde over (c)}t) of sensors attached to the target user.
In operation 340, additionally, the processing method determines a skeleton prior function
( s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n )
based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
Here, the skeleton prior function may include a mean e0, an eigen vector en, and an eigen value λn calculated by applying the PCA scheme with respect to the statistics of the skeleton sizes.
In operation 350, the processing method calculates the skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function.
In operation 350, the skeleton sizes and the local coordinates may be calculated to minimize a sum of the difference function and the skeleton prior function.
The functions may be understood by referring to the description with respect to FIG. 1.
According to one or more embodiments, an avatar with accurate motions may be created in consideration of variation in a figure of a target user, a sensor attachment position, and the like, by using a skeleton prior function based on the statistics of skeleton sizes of references poses of a plurality of users.
In addition, since the skeleton prior function containing a mean, an eigen vector, and an eigen value of the skeleton sizes is used, ambiguity in the skeleton size according to the user may be reduced.
The methods according to the above-described embodiments may be recorded in non-transitory computer-readable media including computer readable instructions to implement various operations by executing computer readable instructions to control one or more processors, which are part of a general purpose computer, computing device, a computer system, or a network. The media may also have recorded thereon, alone or in combination with the computer readable instructions, data files, data structures, and the like. The computer readable instructions recorded on the media may be those specially designed and constructed for the purposes of embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. The computer-readable media may also be embodied in at least one application specific integrated circuit (ASIC) or Field Programmable Gate Array (FPGA), which executes (processes like a processor) computer readable instructions. Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM disks and DVDs; magneto-optical media such as optical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. The media may be transfer media such as optical lines, metal lines, or waveguides including a carrier wave for transmitting a signal designating the program command and the data construction. Examples of computer readable instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments, or vice versa. Another example of media may also be a distributed network, so that the computer readable instructions are stored and executed in a distributed fashion.
Although embodiments have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the disclosure, the scope of which is defined in the claims and their equivalents.

Claims (19)

What is claimed is:
1. A processing apparatus for creating an avatar, comprising:
a skeleton database to store statistics of skeleton sizes with respect to reference poses of a plurality of users;
a function determiner which determines a difference function representing a difference between a forward kinematics function, the forward kinematics function regarding joints of the avatar with respect to a reference pose of a target user, and positions of sensors attached to the target user, and to determine a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users,
wherein the function determiner uses at least one processor;
a calculator which calculates skeleton sizes of the joints of the avatar, and local coordinates corresponding to the sensors using at least one processor; and
a display which displays the avatar based on the calculated skeleton sizes of the joints and the calculated local coordinates,
wherein the skeleton database is updated periodically or non-periodically according to changes of the reference poses,
wherein the calculator performs the calculation of skeleton sizes of the joints and local coordinates using the equation
arg ⁢ ⁢ min s , z , λ n ⁢ f ⁡ ( s , z ; q ~ t ) - c ~ t 2 + s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n ,
wherein, the difference function (∥ƒ(s, z; {tilde over (q)}t)−{tilde over (c)}t∥2) represents the difference between the forward kinematics function (ƒ(s, z; {tilde over (q)}t)), the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user, and the positions ({tilde over (c)}t) of the sensors attached to the target user,
wherein s denotes the skeleton size of the reference pose of the target user, z denotes the local coordinate of the sensor, and {tilde over (q)}t denotes the joint angle with respect to the reference pose of the target user, and
wherein e0 denotes a mean, en denotes an eigen vector, λn denotes an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, and B denotes a number of main eigen vectors among the eigen vectors en arranged in order of importance.
2. The processing apparatus of claim 1, wherein the skeleton prior function comprises a mean, an eigen vector, and an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
3. The processing apparatus of claim 1, wherein the calculator calculates the skeleton sizes and the local coordinates to minimize a sum of the difference function and the skeleton prior function.
4. The processing apparatus of claim 1, further comprising a preprocessor which defines the reference pose and which stores information containing the statistics of the skeleton sizes of the reference poses of the plurality of users, in the skeleton database, wherein the preprocessor uses at least one processor.
5. The processing apparatus of claim 1, further comprising an interface which receives information on the positions of the sensors attached to the target user from the sensors, wherein the interface uses at least one processor.
6. A processing method for creating an avatar, comprising:
maintaining a skeleton database storing statistics of skeleton sizes with respect to reference poses of a plurality of users;
determining a difference function representing a difference between a forward kinematics function regarding joints of the avatar with respect to a reference pose of a target user and positions of sensors attached to the target user;
determining a skeleton prior function based on the statistics of the skeleton sizes with respect to the reference poses of the plurality of users;
calculating skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function using at least one processor; and
displaying the avatar based on the calculated sizes of the joints of the avatar and the calculated local coordinates,
wherein the skeleton database is updated periodically or non-periodically according to changes of the reference poses,
wherein the calculating of skeleton sizes of the joints and local coordinates uses the equation
arg ⁢ ⁢ min s , z , λ n ⁢ f ⁡ ( s , z ; q ~ t ) - c ~ t 2 + s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n ,
wherein, the difference function (∥ƒ(s, z; {tilde over (q)}t)−{tilde over (c)}t∥2) represents the difference between the forward kinematics function (ƒ(s, z; {tilde over (q)}t)), the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user, and the positions ({tilde over (c)}t) of the sensors attached to the target user,
wherein s denotes the skeleton size of the reference pose of the target user, z denotes the local coordinate of the sensor, and {tilde over (q)}t denotes the joint angle with respect to the reference pose of the target user, and
wherein e0 denotes a mean, en denotes an eigen vector, λn denotes an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, and B denotes a number of main eigen vectors among the eigen vectors en arranged in order of importance.
7. The processing method of claim 6, wherein the skeleton prior function comprises a mean, an eigen vector, and an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users.
8. The processing method of claim 6, wherein the skeleton sizes and the local coordinates are calculated to minimize a sum of the difference function and the skeleton prior function.
9. The processing method of claim 6, further comprising:
defining the reference poses; and
storing information containing the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, in the skeleton database.
10. The processing method of claim 6, further comprising receiving information on positions of the sensors attached to the target user from the sensors.
11. At least one non-transitory computer readable recording medium storing computer readable instructions to control at least one processor to implement the method of claim 6.
12. The processing apparatus of claim 1, wherein the stored skeleton sizes are updated periodically according to changes of the reference poses.
13. The processing apparatus of claim 1, wherein the stored skeleton sizes are updated non-periodically according to changes of the reference poses.
14. The processing apparatus of claim 1, wherein the sensors are provided at five positions consisting of the forehead, both wrists, a chest, and a waist.
15. A processing apparatus for creating an avatar, comprising:
a function determiner which determines a difference function representing a difference between a forward kinematics function, the forward kinematics function regarding joints of the avatar corresponding to a reference pose of a target user, and positions of sensors attached to the target user, and to determine a skeleton prior function based on statistics of skeleton sizes corresponding to reference poses of a plurality of users,
wherein the function determiner uses at least one processor;
a calculator which calculates skeleton sizes of the joints of the avatar, and local coordinates corresponding to the sensors using at least one processor; and
a display which displays the avatar based on the calculated skeleton sizes of the joints and the calculated local coordinates,
wherein the calculator performs the calculation of skeleton sizes of the joints and local coordinates using the equation
arg ⁢ ⁢ min s , z , λ n ⁢ f ⁡ ( s , z ; q ~ t ) - c ~ t 2 + s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n ,
wherein, the difference function (∥ƒ(s, z; {tilde over (q)}t)−{tilde over (c)}t∥2) represents the difference between the forward kinematics function (ƒ(s, z; {tilde over (q)}t)), the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user, and the positions ({tilde over (c)}t) of the sensors attached to the target user,
wherein s denotes the skeleton size of the reference pose of the target user, z denotes the local coordinate of the sensor, and {tilde over (q)}t denotes the joint angle with respect to the reference pose of the target user, and
wherein e0 denotes a mean, en denotes an eigen vector, λn denotes an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, and B denotes a number of main eigen vectors among the eigen vectors en arranged in order of importance.
16. The processing apparatus of claim 15, wherein the skeleton sizes and the local coordinates are calculated to minimize a sum of the difference function and the skeleton prior function.
17. A processing method for creating an avatar, comprising:
determining a difference function representing a difference between a forward kinematics function regarding joints of the avatar corresponding to a reference pose of a target user and positions of sensors attached to the target user;
determining a skeleton prior function based on statistics of skeleton sizes corresponding to reference poses of a plurality of users;
calculating skeleton sizes of the joints of the avatar and local coordinates corresponding to the sensors, using the difference function and the skeleton prior function using at least one processor; and
displaying the avatar based on the calculated sizes of the joints of the avatar and the calculated local coordinates,
wherein the calculating of skeleton sizes of the joints and local coordinates uses the equation
arg ⁢ ⁢ min s , z , λ n ⁢ f ⁡ ( s , z ; q ~ t ) - c ~ t 2 + s - e 0 - ∑ n = 1 B ⁢ ⁢ λ n ⁢ e n .
wherein, the difference function (∥ƒ(s, z; {tilde over (q)}t)−{tilde over (c)}t∥2) represents the difference between the forward kinematics function (ƒ(s, z; {tilde over (q)}t)), the forward kinematics function regarding the joints of the avatar with respect to the reference pose of the target user, and the positions ({tilde over (c)}t) of the sensors attached to the target user,
wherein s denotes the skeleton size of the reference pose of the target user, z denotes the local coordinate of the sensor, and {tilde over (q)}t denotes the joint angle with respect to the reference pose of the target user, and
wherein e0 denotes a mean, en denotes an eigen vector, λn denotes an eigen value calculated from the statistics of the skeleton sizes with respect to the reference poses of the plurality of users, and B denotes a number of main eigen vectors among the eigen vectors en arranged in order of importance.
18. The processing method of claim 17, wherein the skeleton sizes and the local coordinates are calculated to minimize a sum of the difference function and the skeleton prior function.
19. At least one non-transitory computer readable recording medium storing computer readable instructions to control at least one processor to implement the method of claim 17.
US13/276,767 2010-10-20 2011-10-19 Processing apparatus and method for creating avatar Active 2033-04-07 US9001129B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/276,767 US9001129B2 (en) 2010-10-20 2011-10-19 Processing apparatus and method for creating avatar

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US39488410P 2010-10-20 2010-10-20
KR10-2010-0109294 2010-11-04
KR1020100109294A KR101700377B1 (en) 2010-10-20 2010-11-04 Processing apparatus and method for creating avatar
US13/276,767 US9001129B2 (en) 2010-10-20 2011-10-19 Processing apparatus and method for creating avatar

Publications (2)

Publication Number Publication Date
US20120127164A1 US20120127164A1 (en) 2012-05-24
US9001129B2 true US9001129B2 (en) 2015-04-07

Family

ID=46140790

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/276,767 Active 2033-04-07 US9001129B2 (en) 2010-10-20 2011-10-19 Processing apparatus and method for creating avatar

Country Status (2)

Country Link
US (1) US9001129B2 (en)
KR (1) KR101700377B1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10176655B2 (en) 2016-10-26 2019-01-08 Reavire, Inc. Controlling lockable devices using electronic key
US10444320B2 (en) 2016-10-06 2019-10-15 Reavire, Inc. Locating devices based on antenna coordinates
US10503351B2 (en) 2016-08-23 2019-12-10 Reavire, Inc. Managing virtual content displayed to a user based on mapped user location
US11050758B2 (en) 2016-08-23 2021-06-29 Reavire, Inc. Controlling access to a computer network using measured device location
US11054638B2 (en) 2018-06-13 2021-07-06 Reavire, Inc. Tracking pointing direction of device

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10234941B2 (en) * 2012-10-04 2019-03-19 Microsoft Technology Licensing, Llc Wearable sensor for tracking articulated body-parts
US9990373B2 (en) 2013-02-06 2018-06-05 John A. Fortkort Creation and geospatial placement of avatars based on real-world interactions
US10262462B2 (en) 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
GB2548154A (en) 2016-03-11 2017-09-13 Sony Computer Entertainment Europe Ltd Virtual reality
US20180342109A1 (en) * 2017-05-25 2018-11-29 Thomson Licensing Determining full-body pose for a virtual reality environment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7492268B2 (en) 2004-07-29 2009-02-17 Motiva Llc Human movement measurement system
EP2046029A2 (en) 2007-10-04 2009-04-08 Sony Corporation Image processing apparatus, image processing method, and program
JP2009522559A (en) 2006-01-04 2009-06-11 チエスケー・ビソケー・ウツエニー・テクニツケー・ベー・プラゼ・フアクルタ・ストロイニー Method and apparatus for measuring and / or calibrating the position of an object in space
EP2085817A2 (en) 2008-02-04 2009-08-05 Ricoh Company, Limited Apparatus, method system, and program for image processing
US7661200B2 (en) 2004-03-30 2010-02-16 Commissariat A L'energie Atomique Method and device for determining a person's motions
US20100182260A1 (en) 2008-12-15 2010-07-22 Nintendo Co., Ltd. Computer-readable storage medium having calibration program, coordinate detection apparatus and calibration method
US20100191124A1 (en) 2007-04-17 2010-07-29 Prokoski Francine J System and method for using three dimensional infrared imaging to provide psychological profiles of individuals

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100841925B1 (en) * 2006-09-27 2008-06-30 한국전자통신연구원 System andd method for making digital character motion
KR101221451B1 (en) * 2008-12-22 2013-01-11 한국전자통신연구원 Methodlogy of animatable digital clone creation from multi-view images capturing dynamic performance

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7661200B2 (en) 2004-03-30 2010-02-16 Commissariat A L'energie Atomique Method and device for determining a person's motions
US7492268B2 (en) 2004-07-29 2009-02-17 Motiva Llc Human movement measurement system
JP2009522559A (en) 2006-01-04 2009-06-11 チエスケー・ビソケー・ウツエニー・テクニツケー・ベー・プラゼ・フアクルタ・ストロイニー Method and apparatus for measuring and / or calibrating the position of an object in space
US20090183550A1 (en) 2006-01-04 2009-07-23 Inomech S.R.O. Method and apparatus for measurement and/or calibration of position of an object in space
US20100191124A1 (en) 2007-04-17 2010-07-29 Prokoski Francine J System and method for using three dimensional infrared imaging to provide psychological profiles of individuals
EP2046029A2 (en) 2007-10-04 2009-04-08 Sony Corporation Image processing apparatus, image processing method, and program
EP2085817A2 (en) 2008-02-04 2009-08-05 Ricoh Company, Limited Apparatus, method system, and program for image processing
US20100182260A1 (en) 2008-12-15 2010-07-22 Nintendo Co., Ltd. Computer-readable storage medium having calibration program, coordinate detection apparatus and calibration method

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Huajun Liu, Xiaolin Wei, Inwoo Ha, Jinxiang Chai, Taehyun Rhee, "Realtime Human Motion Control with a Small Number of Intertial Sensors", Feb. 20, 2011, ACM, I3D '11 Symposium on Interactive 3D Graphics and Games, pp. 133-140. *
Jinxiang Chai, Jessica K. Hodgins, "Constraint-based Motion Optimization Using a Statistical Dynamic Model", Jul. 2007, ACM, ACM Transactions on Graphics, vol. 26, No. 3, Article 8. *
Jinxiang Chai, Jessica K. Hodgins, "Performance Animation from Low-dimensional Control Signals", Jul. 2005, ACM, ACM Transactions on Graphics, vol. 24, Issue 3, pp. 686-696. *
Yen-Lin Chen, Jinxiang Chai, "3D Reconstruction of Human Motion and Skeleton from Uncalibrated Monocular Video", Sep. 27, 2009, Springer, Computer Vision-ACCV 2009: 9th Asian Conference on Computer Vision, Xi'an, Sep. 23-27, 2009, Revised Selected Papers, Part I, pp. 71-82. *
Yen-Lin Chen, Jinxiang Chai, "3D Reconstruction of Human Motion and Skeleton from Uncalibrated Monocular Video", Sep. 27, 2009, Springer, Computer Vision—ACCV 2009: 9th Asian Conference on Computer Vision, Xi'an, Sep. 23-27, 2009, Revised Selected Papers, Part I, pp. 71-82. *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10503351B2 (en) 2016-08-23 2019-12-10 Reavire, Inc. Managing virtual content displayed to a user based on mapped user location
US11050758B2 (en) 2016-08-23 2021-06-29 Reavire, Inc. Controlling access to a computer network using measured device location
US11635868B2 (en) 2016-08-23 2023-04-25 Reavire, Inc. Managing virtual content displayed to a user based on mapped user location
US10444320B2 (en) 2016-10-06 2019-10-15 Reavire, Inc. Locating devices based on antenna coordinates
US10176655B2 (en) 2016-10-26 2019-01-08 Reavire, Inc. Controlling lockable devices using electronic key
US11054638B2 (en) 2018-06-13 2021-07-06 Reavire, Inc. Tracking pointing direction of device
US11086124B2 (en) 2018-06-13 2021-08-10 Reavire, Inc. Detecting velocity state of a device
US11867901B2 (en) 2018-06-13 2024-01-09 Reavire, Inc. Motion capture for real-time controller and human pose tracking

Also Published As

Publication number Publication date
US20120127164A1 (en) 2012-05-24
KR101700377B1 (en) 2017-01-26
KR20120041086A (en) 2012-04-30

Similar Documents

Publication Publication Date Title
US9001129B2 (en) Processing apparatus and method for creating avatar
US10832056B1 (en) Visual-inertial positional awareness for autonomous and non-autonomous tracking
US10354396B1 (en) Visual-inertial positional awareness for autonomous and non-autonomous device
US10304244B2 (en) Motion capture and character synthesis
US9435911B2 (en) Visual-based obstacle detection method and apparatus for mobile robot
US11030773B2 (en) Hand tracking based on articulated distance field
US9298257B2 (en) Apparatus and method for controlling avatar using expression control point
JP5012615B2 (en) Information processing apparatus, image processing method, and computer program
US20150235372A1 (en) Computing 3d shape parameters for face animation
CN109255749B (en) Map building optimization in autonomous and non-autonomous platforms
CN104899563A (en) Two-dimensional face key feature point positioning method and system
US20200286286A1 (en) Tracking rigged polygon-mesh models of articulated objects
US9111172B2 (en) Information processing device, information processing method, and program
KR20210058686A (en) Device and method of implementing simultaneous localization and mapping
CN114677572B (en) Object description parameter generation method and deep learning model training method
US9047676B2 (en) Data processing apparatus generating motion of 3D model and method
US11721056B2 (en) Motion model refinement based on contact analysis and optimization
KR20210146264A (en) Method, device and non-transitory computer-readable recording medium for estimating information about golf swing
US9652879B2 (en) Animation of a virtual object
Hirt et al. Preliminary environment mapping for redirected walking
CN116248920A (en) Virtual character live broadcast processing method, device and system
Baik et al. Geometric particle swarm optimization for robust visual ego-motion estimation via particle filtering
Hwang et al. Primitive object grasping for finger motion synthesis
US10565328B2 (en) Method and apparatus for modeling based on particles for efficient constraints processing
KR102413988B1 (en) Method, device and non-transitory computer-readable recording medium for estimating information about golf swing

Legal Events

Date Code Title Description
AS Assignment

Owner name: TEXAS A&M UNIVERSITY SYSTEM, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RHEE, TAEHYUN;HA, INWOO;KIM, DOKYOON;AND OTHERS;REEL/FRAME:027816/0571

Effective date: 20120131

Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RHEE, TAEHYUN;HA, INWOO;KIM, DOKYOON;AND OTHERS;REEL/FRAME:027816/0571

Effective date: 20120131

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8