KR20200097420A - Method and Apparatus for Regeneration Motion in Virtual Object Contact - Google Patents

Abstract

The present invention relates to a method and a device for regenerating when contacting a virtual object so that an interaction between a hand of a user and a virtual object is performed more naturally by using physical particles applied to a virtual hand model as a guide contact point through a physical engine to predict the occurrence of hand interpenetration when contacting the virtual object and according to a prediction result, regenerating so that a motion of the hand does not penetrate a motion at the moment that the virtual object is penetrated.

Description

Method and Apparatus for Regeneration Motion in Virtual Object Contact}

The present embodiment relates to a method and apparatus for regenerating motion upon contact with a virtual object. In more detail, the present invention relates to a method for regenerating motions for preventing mutual penetration of hands when a virtual object is in contact, and an apparatus therefor.

The content described in this section merely provides background information on the present invention and does not constitute prior art.

As technology advances, interest in virtual reality or augmented reality is increasing. Virtual Reality creates virtual images of images, surrounding backgrounds, and objects, whereas Augmented Reality shows the actual state of the real world and only added information as virtual. Both virtual reality and augmented reality must make it as if users who use them are interacting with virtual objects.

In order to make the user feel as if he is interacting with a virtual object, a'computer tactile technology', that is, a haptic technology, is important to allow the user to feel the touch. Although haptic glove technology that can deliver tactile sensation to users has been progressing a lot according to the recent development and research of haptic technology, manipulation of virtual objects in virtual reality and mixed reality spaces does not allow users to accurately determine the depth and real world. Unlike the situation, it is difficult to reproduce the sense of reality because there is no sense due to physical contact.

For this reason, unnatural movements may occur when the user's hand contacts the virtual object. For example, since the user cannot feel the touch, a motion (=hands interpenetration) that enters the virtual object inevitably occurs. This is not only visually unnatural, but also leads to unpredictable physics simulation results when the hand penetrates the object deeply.

In the present embodiment, by using the physical particles applied to the virtual hand model through the physics engine as a guide contact point, the occurrence of hand mutual penetration when the virtual object is in contact is predicted, and according to the prediction result, the hand movement penetrates the virtual object. The purpose is to provide a more natural interaction between the user's hand and the virtual object by recreating and providing the motion at the moment of entry so as not to penetrate.

According to an aspect of the present embodiment, a method for regenerating a motion upon contact with a virtual object includes: forming a plurality of physical particles to be distributed and arranged in a virtual hand model; Detecting whether the physical particles of the virtual hand model are in contact with a virtual object; Calculating a penetration depth of the physical particles in contact with the virtual object to predict whether or not mutual penetration of hands into the virtual object occurs; And regenerating the motion of the virtual hand model according to a prediction result of the predicting process.

According to another aspect of the present embodiment, there is provided an apparatus for regenerating motion upon contact with a virtual object, comprising: a model generator configured to form a plurality of physical particles to be distributed and arranged in a virtual hand model; A touch sensing unit that detects whether the physical particles on the virtual hand model are in contact with a virtual object; An interpenetration prediction unit that calculates a penetration depth of the physical particles in contact with the virtual object and predicts whether or not hand mutual penetration into the virtual object occurs; And a motion control unit for regenerating the motion of the virtual hand model according to a prediction result of the interpenetration predictor.

As described above, according to the present embodiment, by using the physical particles applied to the virtual hand model through the physics engine as a guide contact point, it predicts whether the occurrence of hand mutual penetration when contacting the virtual object is predicted, and according to the prediction result, There is an effect that interaction between the user's hand and the virtual object can be made more natural by providing the regeneration so that the motion at the moment when the movement penetrates the virtual object does not penetrate.

1 is a block diagram schematically showing an operation regeneration apparatus according to the present embodiment.
2 is a diagram showing mesh data of a virtual hand model according to the present embodiment.
3 is a diagram illustrating physical particles formed on a virtual hand model according to the present embodiment.
4 is a diagram illustrating a skeleton structure of a hand for explaining a motion reconstruction method according to the present embodiment.
5 is an exemplary diagram for describing a method of reconfiguring an operation according to the present embodiment.
6 is an exemplary diagram illustrating the effect of the motion reconfiguration method according to the present embodiment.
7 is a flowchart illustrating a method of regenerating an operation upon contact with a virtual object according to the present embodiment.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible even if they are indicated on different drawings. In addition, in describing the present invention, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

In addition, in describing the constituent elements of the present invention, terms such as first, second, A, B, (a) and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the component is not limited by the term. Throughout the specification, when a part'includes' or'includes' a certain element, it means that other elements may be further included rather than excluding other elements unless otherwise stated. . In addition, the'... Terms such as'sub' and'module' mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software.

1 is a block diagram schematically showing an operation regeneration apparatus according to the present embodiment.

The motion regeneration apparatus 100 according to the present embodiment utilizes physical particles applied to the virtual hand model as guide contacts to prevent hand inter-penetration when the virtual object is in contact, so that the movement of the hand can reduce the virtual object. Provides a motion regeneration algorithm that recreates the moment of digging.

1, the motion regeneration apparatus 100 according to the present embodiment includes an input unit 110, a model generation unit 120, a touch detection unit 130, a mutual penetration prediction unit 140, and an operation control unit ( 150). Components included in the motion regeneration apparatus 100 are not necessarily limited thereto.

The input unit 110 collects input information for formation, movement, or transformation of a virtual hand model. The input unit 110 may provide physical quantities such as position, shape, size, mass, speed, magnitude and direction of an applied force, friction coefficient, and elasticity coefficient as input information on the virtual hand model. In addition, the input unit 110 may provide a change in a physical quantity such as a change in a position, a change in shape, a change in speed, etc. to move or deform the virtual hand model.

The input unit 110 according to the present embodiment may be a hand recognition device capable of recognizing the shape or position of an actual hand. As an example, the input unit 110 includes a lip motion sensor, an image sensor such as a camera, a glove attached with various known sensors including an RGBD sensor, and a separate device manufactured to measure the motion of an exoskeleton. (For example, a hand motion capture device), or a method of directly attaching a sensor to a hand, etc. may be used.

The input unit 110 of this embodiment provides input information required to form a virtual hand model. That is, the input unit 110 may recognize the shape of the actual hand and infer the arrangement of the skeleton in the actual hand based on this. Accordingly, the input unit 110 may provide input information for forming the skeleton of the virtual hand model. In addition, the friction coefficient, mass, and the like required to implement the virtual hand model may be provided as preset values.

In addition, the input unit 110 of the present exemplary embodiment may detect a change in the shape and position of an actual hand, and provide input information necessary for moving or deforming the virtual hand model based on the change. In this case, when the connection between the bones and the joints constituting the virtual hand model and the degree of freedom in the joints are set in advance, the input unit 110 recognizes only the angle at which each bone is arranged and the position of the joint or the position of the fingertip in the actual hand. Input information can be provided in a more compact form.

On the other hand, the input unit 110 of the present embodiment can provide the above input information by recognizing actual motion in space through a separate sensor as described above, and simply directly setting physical quantities such as shape and position. You can also provide input information.

The model generation unit 120 forms and controls a virtual hand model based on the information input from the input unit 110.

In the present embodiment, the model generator 120 applies physical particles for physical interaction to the virtual hand model. The model generation unit 120 generates a virtual hand model using a physical engine, and forms a plurality of physical particles to be distributed and arranged on the generated virtual hand model.

In this case, as shown in FIG. 2, if the entire mesh data of the virtual hand model that is deformed in real time is made into one physical particle (physical object), there is a problem that it takes a lot of computation time. In other words, there are about 9000 mesh indexes per hand.If the entire virtual hand model is updated by reflecting the positions of all mesh indices that change in real time, the calculation of the physics engine is overloaded and real-time performance is not guaranteed. There is a problem.

Therefore, in this embodiment, as shown in FIG. 3, when the user performs a hand gesture, the physical particle 300 is generated only in the mesh index that is mainly contacted (ex: the contact frequency exceeds a certain threshold), and a plurality of generated physical particles ( 300) to perform the physical interaction process. In this embodiment, the physical property of the physical particle 300 is defined as a kinematic object, so that various hand movements occurring in the real world can be properly implemented.

The plurality of physical particles 300 of the present embodiment are small particles having an arbitrary shape. In the present embodiment, the physical particles 300 are densely distributed on the fingertip joints, which are mesh indexes that are mainly contacted when performing hand movements, and evenly distributed over the entire area on the palm of the hand. Even if it is used, it is possible to obtain a result of physics interaction similar to that of using the entire mesh data. In this embodiment, algorithms for various operations are calculated by using contact (collision) information between each physical particle 300 and a virtual object. Since there are many, it is possible to distribute the physical particles 300 in an appropriate distribution that does not decrease the calculation speed of the physics engine. This may derive a numerical value through an experiment, and as an example, a total of 130 physical particles 300 may be dispersed and disposed in both hands.

In addition, the plurality of physical particles 300 may be particles of various shapes, but for simplification of calculation, it is preferable that they are spherical particles having a unit size. The plurality of physical particles 300 may have various physical quantities. The magnitude and direction of the force acting on the plurality of physical particles 300 may exist, and may further have physical quantities such as friction coefficient and elasticity coefficient. In addition, a position in which the plurality of physical particles 300 are disposed corresponding to a finger bone of the virtual hand model 310 is also included.

On the other hand, in the present embodiment, the plurality of physical particles 300 operate as a pre-contact point having a constant radius r as a threshold, which means that the surface of the hand directly contacts the virtual object. When the motion is stopped and then updated, it can indirectly detect the collision and correct the motion, resulting in the effect of expressing smooth hand movement.

The touch sensing unit 130 determines whether the physical particle 300 of the virtual hand model has contacted the virtual object. For example, the contact detection unit 130 may use an Axis-Aligned Bounding Box (AABB) collision detection method as a method of determining whether the physical particle 300 and the virtual object are in contact.

In the AABB collision detection method, a bounding box arranged in the same axis direction is covered on all physical objects, and a contact (collision) is determined by checking in real time whether the corresponding bounding boxes between objects overlap. Therefore, in the touch sensing unit 130 of the present embodiment, the bounding box of the physical particle 300 disposed on the virtual hand model 310 and the bounding box of the virtual object to be interacted with are checked in real time, and contact with whether or not they overlap. (Collision) detect.

Previously, it has been described that the contact detection unit 130 uses the AABB collision detection method as a method of determining whether the physical particle 300 and the virtual object are in contact with each other, but is not limited thereto. For example, unlike the AABB collision detection method described above, the bounding box is not fixed in the same axial direction, but the OBB (Object Oriented Bounding Box) collision detection method changes the direction according to the state of the object, and a sphere instead of the bounding box is applied to the physical object. Sphere collision detection method, which determines whether there is contact (collision), and the Convex Hull collision detection method, which determines contact (collision) by placing a convex hull on a physical object instead of a bounding box. Law can be used. That is, any known collision detection method may be used in this embodiment as long as the contact detection unit 130 can determine whether the physical particle 300 and the virtual object are in contact with each other.

The mutual penetration prediction unit 140 uses the physical particle 300 of the virtual hand model as a guide contact point to predict whether hand mutual penetration occurs when the virtual object is in contact.

In this embodiment, when it is determined that the physical particle 300 of the virtual hand model has contacted the virtual object by the touch sensing unit 130, the penetration depth of the contacted physical particle 300 ( Penetration Depth) is calculated to predict the occurrence of hand mutual penetration into the virtual object. Meanwhile, the mutual penetration prediction unit 140 may calculate the penetration depth by using an image collected through a sensor or the like and a physical quantity of the physical particle 300.

The interpenetration prediction unit 140, when the penetration depth does not exceed the radius of the physical particle 300 (=Small Penetration), the penetration depth when the physical particle 300 collides with the virtual object. It is predicted that hand mutual penetration does not occur. The interpenetration prediction unit 140 determines that a predictable physical simulation will occur in which the virtual object is slightly pushed in a direction in which the force is given according to the prediction result, and operates to proceed with this.

When the penetration depth exceeds the radius of the physical particle 300 (=Big Penetration), the mutual penetration prediction unit 140 predicts that hand mutual penetration into the virtual object will occur. The interpenetration predictor 140 determines that an unpredictable physical simulation result and visual interpenetration will occur according to the prediction result, and operates so that processing to prevent this may be performed. In other words, the mutual penetration prediction unit 140 transmits the prediction result of the occurrence of the mutual penetration of the hands to the operation control unit 150, and operates so that the motion regeneration process for preventing the mutual penetration of the hands can be performed.

The motion control unit 150 interlocks with the interpenetration prediction unit 140 and the model generation unit 120, and regenerates the motion of the virtual hand model according to the prediction result of hand interpenetration transmitted from the interpenetration prediction unit 140 Performs the function of In this embodiment, the motion control unit 150 reproduces the motion at the moment when the hand motion penetrates the virtual object.

In this embodiment, the motion control unit 150 exceeds the number of physical particles 300 in contact with the virtual object, the position, and the depth of penetration when the physical particles 300 and the virtual object are in contact with each other. The penetration ratio is calculated, and the operation of the virtual hand model is regenerated using the calculated excess penetration ratio. Meanwhile, the equation for calculating the excess penetration rate by the operation control unit 150 is as follows.

Here, p _n (i) is the physical particle of index i, which is the collision state (collision or inside) in n frames, N is the number of all physical particles currently in the collision state, and P(p _n (i)) is p The position of _n (i) is D (p _n (i)) is the Penetration Depth of p _n (i), and R _n is the rate of over penetration.

In more detail, the motion control unit 150 calculates the angle of change for the hand joint angle between the current frame (n-th frame) and the previous frame (n-th frame) to be regenerated for the virtual hand model, and the calculated The motion of the virtual hand model is regenerated by interpolating the angle of change by the excess penetration rate calculated previously.

Meanwhile, referring to FIG. 4, a hand is schematically composed of a thumb, index finger, middle finger, ring finger, and a little finger and palm (or back of the hand). The hand can be moved by joints, and the joints and joints are made up of bones. Four fingers excluding the thumb are composed of a stun bone, a middle bone 400, and a terminal bone 410, and the thumb is composed of a stunned bone and a terminal bone 410. The palm is composed of the wrist bones and the five hip bones that connect each finger. According to the hierarchical structure of the hand, the penetrating motion in which the hand penetrates the virtual object occurs only in the end bone 410 and the middle bone 400. This is because the stun bone and the palm move with very limited degrees of freedom for a given wrist movement.

Due to this, the motion control unit 150 according to the present embodiment regenerates the movements of the ending bone and the metatarsal bone in the process of regenerating the motion of the virtual hand model, and the metastosis 400 operates according to the hierarchy of the hand. After regenerating first, the motion of the end bone 400 is regenerated. That is, based on the calculated excess penetration rate, the motion control unit 150 sequentially interpolates the angle of change for the angle of the hand joint corresponding to the middle osteotomy in the virtual hand model and the angle of change for the angle of the hand joint corresponding to the end bone. Recreate the motion of the virtual hand model. Meanwhile, an algorithm by which the motion control unit 150 regenerates the motion of the virtual hand model can be confirmed through FIG. 5.

6 is an exemplary diagram illustrating the effect of the motion reconfiguration method according to the present embodiment.

In the case of the motion reconfiguration method according to the present embodiment, since the hand interpenetration motion can be regenerated by continuously utilizing physical particles as a guide contact point, a user can obtain a stable interaction result visually and physically.

In addition, in the case of a method that utilizes indirect collision, such as the motion reconstruction method according to the present embodiment, in a virtual environment where tactile sensation does not exist, human motion may actually change rapidly within one frame. There is an effect that can make the actual hand movement smoother within a predictable range than the direct method using the collision of

Referring to FIG. 6, the result of the motion regeneration algorithm according to the present embodiment that prevents hand mutual penetration can be confirmed. That is, the Wire-Frame Hand is a situation in which the virtual object is mutually infiltrated by the kinematic motion input through the sensing device, but the mesh hand regenerated as a result of the motion regeneration algorithm according to this embodiment is mutually penetrating. Without it, you can see that your hand is in contact with the object surface.

7 is a flowchart illustrating a method of regenerating an operation upon contact with a virtual object according to the present embodiment.

The motion regeneration apparatus 100 is formed so that a plurality of physical particles are distributed and disposed on the virtual hand model (S702). In step S702, the motion regeneration apparatus 100 may generate the physical particle 300 only in the mesh index that the user mainly contacts when performing the hand motion.

In addition, the motion regeneration apparatus 100 may densely distribute the physical particles 300 to the fingertips, which are mesh indexes that are mainly contacted when performing a hand motion, and evenly distribute the physical particles 300 to the palm of the hand.

The motion regeneration apparatus 100 detects whether the physical particle 300 of the virtual hand model generated in step S702 has contacted the virtual object (S704).

When it is determined that the physical particle 300 of the virtual hand model has contacted the virtual object according to the detection result in step S704, the motion regeneration device 100 calculates the penetration depth of the physical particle of the virtual hand model that has contacted the virtual object ( S706).

The motion regeneration apparatus 100 determines whether or not hand mutual penetration into the virtual object occurs based on the penetration depth calculated in step S706 (S708). In step S708, the motion regeneration apparatus 100 may compare the penetration depth calculated in step S706 with the radius of the physical particle 300, and determine whether hand mutual penetration occurs according to the comparison result.

When the penetration depth of step S706 exceeds the radius of the physical particle 300, the motion regeneration device 100 predicts that hand mutual penetration into the virtual object will occur, and performs a motion regeneration process to prevent mutual penetration according to the prediction result. It performs (S710). In step S710, the motion regeneration device 100 is based on the number and position of the physical particles 300 in contact with the virtual object and the depth of penetration when the physical particles 300 and the virtual object are in contact with the excess penetration rate according to hand mutual penetration. And regenerate the motion of the virtual hand model using the calculated excess penetration ratio.

The motion regeneration apparatus 100 operates to generate a predictable physical simulation result when the penetration depth of step S706 does not exceed the radius of the physical particle 300 or when the motion regeneration process of step S710 is completed. (S712).

Here, steps S702 to S712 correspond to the operations of each component of the operation regeneration apparatus 100 described above, and thus further detailed descriptions are omitted.

In FIG. 7, each process is described as sequentially executing, but is not limited thereto. In other words, since it is possible to change and execute the processes illustrated in FIG. 7 or to execute one or more processes in parallel, FIG. 7 is not limited to a time series order.

As described above, the operation regeneration method described in FIG. 7 is a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) that is implemented as a program and can be read using software of a computer. Can be written on.

The above description is merely illustrative of the technical idea of the present embodiment, and those of ordinary skill in the technical field to which the present embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present exemplary embodiments are not intended to limit the technical idea of the present exemplary embodiment, but are illustrative, and the scope of the technical idea of the present exemplary embodiment is not limited by these exemplary embodiments. The scope of protection of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

100: motion regeneration device 110: input unit
120: model generation unit 130: contact detection unit
140: mutual penetration prediction unit 150: motion control unit
300: physics particle 310: virtual hand model

Claims (10)

In the method of regenerating motion upon contact with a virtual object,
Forming a plurality of physical particles to be distributed and arranged in a virtual hand model;
Detecting whether the physical particles of the virtual hand model are in contact with a virtual object;
Calculating a penetration depth of the physical particles in contact with the virtual object to predict whether or not mutual penetration of hands into the virtual object occurs; And
The process of regenerating the motion of the virtual hand model according to the prediction result of the predicting process
A method for regenerating motion upon contact with a virtual object, comprising: a. The method of claim 1,
The forming process,
A virtual object contact, characterized in that among the mesh indexes constituting the virtual hand model, a mesh index whose contact frequency exceeds a certain threshold when a user performs a hand motion is selected, and the physical particles are formed on the selected mesh index. How to regenerate behavior on time. The method of claim 1,
The forming process,
Wherein the physical particles are evenly arranged in a palm area of the virtual hand model and denser than the palm area in a finger area. The method of claim 1,
The predicting process,
Comparing the depth of penetration of the physical particles with the size of the radius of the physical particles, and predicting that mutual penetration of hands into the virtual object occurs when the penetration depth exceeds the size of the radius. How to recreate motion when touching virtual objects. The method of claim 1,
The regenerating process,
Based on the number and location of the physical particles in contact with the virtual object, and the depth of penetration when the physical particles and the virtual object are in contact, the excess penetration rate according to the hand mutual penetration is calculated, and the excess penetration rate is used. A method for regenerating a motion upon contact with a virtual object, characterized in that regenerating the motion of the virtual hand model. The method of claim 5,
The regenerating process,
Computing the angle of change for the angle of the hand joint between the current frame and the previous frame to be regenerated of the virtual hand model, and regenerating the motion of the virtual hand model by interpolating the angle of change by the excess penetration rate. A method of regenerating motion upon contact with a virtual object. The method of claim 6,
The regenerating process,
When contacting a virtual object, characterized in that the angle of change to the angle of the hand joint corresponding to the middle osteotomy in the virtual hand model and the angle of change to the angle of the hand joint corresponding to the end bone are sequentially interpolated based on the excess penetration rate. How to recreate motion. In the apparatus for regenerating motion upon contact with a virtual object,
A model generator configured to form a plurality of physical particles to be distributed and arranged in the virtual hand model;
A touch sensing unit that detects whether the physical particles on the virtual hand model are in contact with a virtual object;
An interpenetration prediction unit that calculates a penetration depth of the physical particles in contact with the virtual object and predicts whether or not hand mutual penetration into the virtual object occurs; And
An operation control unit for regenerating the operation of the virtual hand model according to the prediction result of the interpenetration prediction unit
Motion regeneration device comprising a. The method of claim 8,
The interpenetration prediction unit,
Comparing the depth of penetration of the physical particles with the size of the radius of the physical particles, and predicting that mutual penetration of hands into the virtual object occurs when the penetration depth exceeds the size of the radius. Motion regeneration device. The method of claim 8,
The operation control unit,
Based on the number and location of the physical particles in contact with the virtual object, and the depth of penetration when the physical particles and the virtual object are in contact, the excess penetration rate according to the hand mutual penetration is calculated, and the excess penetration rate is used. Motion regeneration device, characterized in that regenerating the motion of the virtual hand model.

Images