KR20180129140A - Method of user mosion analyzing for virtual reality connection, method and device for virtual reality connection - Google Patents

Abstract

A motion analysis method, and a method and a device for virtual reality connection having the same are disclosed. According to an embodiment of the present invention, a user walking analysis method for virtual reality connection includes the following steps: obtaining position information on a body part of a user; obtaining inertial information on the body part of the user; and determining a walking motion of the user based on the position information and the inertial information.

Description

TECHNICAL FIELD [0001] The present invention relates to a gait analysis method, a gait analysis method,

The following embodiments relate to a gait analysis method, a virtual reality interworking method, and an apparatus for virtual reality interworking.

The virtual reality image reproduces the dynamic change according to the virtual environment controlled by the virtual reality device so that the user can feel the movement of the virtual reality as the motion of the real reality.

The conventional virtual reality apparatus allows the user to experience the visual and auditory experience by operating the controller according to the situation on the screen sitting on the seat. However, recently, in providing the virtual reality to the user, motion through a locomotion device, which allows users to immerse themselves in virtual reality images.

In experiencing the virtual reality, it is an important problem to implement the user 's walking motion on the virtual reality. If there is a difference between the user's walking motion and the operation of the avatar on the virtual reality, there is a problem that the user feels uncomfortable or immersed in the virtual reality. Particularly, since the locomotion device restricts the user to a specific position and provides only a limited action to the user, there is a problem that the user's action is imperfectly implemented in the virtual reality as compared with the case where the user actually performs the walking.

Also, due to the characteristics of the locomotion system for restricting the position of the user, a part of the user's body is hidden by the locomotion device, so that the user's motion can not be accurately caught.

Accordingly, there is a need for a device for realizing movement of many degrees of freedom in accordance with a movement of a user while linking a motion of a user to a virtual reality image while positioning the user within a limited range.

In this regard, Korean Patent Laid-Open Publication No. 10-2015-0065303 discloses an apparatus and method for generating a whole body motion using a wrist trajectory.

An object of an embodiment is to provide a gait analysis method that can analyze a gait operation more accurately by analyzing a gait through inertial information and position information of a user's body.

An object according to an embodiment is to provide a gait analysis method that is close to an actual user's gait direction by setting a gait direction through a waist inclination of a user.

An object according to an embodiment is to provide a virtual reality interworking method similar to a case where a user experiences an actual virtual reality by interlocking an operation of an avatar in a virtual reality similar to a user's actual walking motion.

According to an embodiment of the present invention, there is provided a method for analyzing user gait for virtual reality interworking, comprising: acquiring positional information on the user's body part; Obtaining inertia information for the user body part; And determining the user walking operation based on the position information and the inertia information.

In one aspect of the present invention, the step of acquiring the position information includes: extracting a user image from an observed image; Recognizing a user body part from the user image; And acquiring position information of the recognized body part.

In one aspect, the step of acquiring inertia information comprises: setting a user's body part for obtaining the inertia information; Measuring an acceleration of the set body part; And measuring the rotational acceleration of the set body part.

In one aspect, the step of determining a walking operation of the user includes: determining a walking direction of the user; And determining a walking speed of the user.

In one aspect, the step of determining the walking direction of the user may include the step of determining the walking direction in accordance with the waist tilting direction of the user.

In one aspect, determining the walking direction of the user comprises: setting plane coordinate values of the first portion and the second portion of the user; Calculating a direction vector of a second portion for the first portion; And setting the direction vector in the walking direction.

In one aspect, the step of determining the walking speed of the user includes: calculating a first sensing value according to inertia information of the user's body; Calculating a second sensing value according to the user's body position information; Calculating a third sensing value according to the user's body position information; And calculating the walking speed of the user based on the first sensing value, the second sensing value, and the third sensing value.

In one aspect of the present invention, the step of calculating the first sensing value includes: measuring a moving speed of the user foot; Accumulating and storing the measured moving speed values; And calculating a first sensing value according to an average value (normalize) of the accumulated moving speed values.

In one aspect of the present invention, the step of calculating the second sensing value includes: measuring a waist inclination of the user; And converting the measured waist slope to a predetermined numerical value.

In one aspect, the step of calculating the third sensing value may include: measuring the distance between the user and the knee; And calculating a third sensing value according to the variation of the measured distance.

The step of determining the walking operation may further include a step of determining whether the user is walking based on the inertia information of the body part of the user.

According to an embodiment of the present invention, there is provided a virtual reality interworking method comprising: acquiring position information of a user's body part based on image information; Acquiring inertia information of the user's leg through the inertial sensor; Determining a walking operation of the user based on the position information and the inertia information; And associating the determined gait operation with an avatar of a user in the virtual reality.

Wherein the step of determining the walking operation includes: determining a walking direction of the user based on the positional information; And determining the walking speed of the user based on the position information and the inertia information.

The step of determining the walking direction of the user on the side includes setting plane coordinate values of each of the pelvis and neck based on positional information of the user's pelvis and neck; And calculating a direction vector of the neck with respect to the pelvis.

In one aspect, the step of interlocking with the avatar may include interlocking the moving direction of the avatar according to the calculated direction vector.

In one aspect, the step of determining the walking speed of the user includes: calculating a first sensing value corresponding to a moving speed of the user leg; Calculating a second sensing value according to the user's waist inclination; Calculating a third sensing value according to the distance between the user's knees; And calculating the walking speed of the user based on the first sensing value, the second sensing value, and the third sensing value.

In one aspect, the step of interlocking with the avatar may include: calculating a correction value according to a height of the user; Correcting the calculated walking speed through the correction value; And setting the moving speed of the avatar according to the corrected walking speed.

In one aspect of the present invention, the step of interlocking with the avatar may include: setting a movement limit speed of the avatar; And limiting the moving speed of the avatar to less than or equal to the movement limit speed.

The step of interlocking with the avatar may include a step of stopping the walking operation of the avatar when there is no change in the numerical value measured by the inertial sensor.

The computer readable storage medium according to one embodiment may include instructions for performing any of the methods described above.

A virtual reality interworking apparatus for a walking operation of a user according to an exemplary embodiment includes a position sensing unit for acquiring position information of the user's body part; An inertial sensing unit attached to the user's body for acquiring inertial information of the attached part; A motion analyzer for analyzing the user's walking motion based on the position information and the inertia information; And a controller for interlocking the motion analyzer and the virtual reality.

In one embodiment, the gait analysis method for interworking with virtual reality can analyze the gait operation more precisely by analyzing the gait through the inertia information and the position information of the user's body.

The gait analysis method for interlocking virtual reality according to an embodiment can perform gait analysis close to the actual user's gait direction by setting the gait direction through the slope of the user's waist.

The virtual reality interworking method according to an embodiment can give the user a feeling similar to the actual virtual reality by interlocking the operation of the avatar in the virtual reality similar to the actual walking operation of the user.

The effects of the gait analysis method and the virtual reality interworking method according to the embodiment are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate preferred embodiments of the invention and, together with the description, serve to further the understanding of the technical idea of the invention, It should not be construed as limited.
1 is a schematic diagram of a virtual reality interworking apparatus according to an embodiment.
2 is a schematic diagram illustrating a process of acquiring position information of a position sensing unit according to an exemplary embodiment of the present invention.
3 is a schematic diagram showing a process of acquiring inertia information by the inertial sensing unit according to an embodiment.
4 is a flowchart of a walking motion analysis method according to an embodiment.
FIG. 5 is a flowchart of a step of determining a walking operation according to an embodiment.
6 is a flowchart of a user walking direction determination step according to an embodiment.
Fig. 7 shows a walking direction determination method according to an embodiment.
8 is a flowchart of steps for determining a user's walking speed according to one embodiment.
9 is a flowchart of a first sensing value calculation method according to an embodiment.
10 is a flowchart of a second sensing value calculation method according to an embodiment.
11 is a flowchart of a third sensing value calculation method according to an embodiment.
12 is a flowchart of a virtual reality interworking method according to an embodiment.
13 is a flowchart illustrating a step of linking a walking operation according to an embodiment to an avatar.
FIG. 14 is a diagram showing a method of interlocking in the moving direction of the avatar according to one embodiment.
FIG. 15 is a flowchart of a step of associating an avatar according to an embodiment.
FIG. 16 is a flowchart illustrating a step of associating an avatar according to an embodiment.
17 is a flowchart of a virtual reality interworking method according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments, detailed description of known functions and configurations incorporated herein will be omitted when it may make the best of an understanding clear.

In describing the components of the embodiment, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be " connected, " " coupled, " or " connected. &Quot;

The components included in any one embodiment and the components including common functions will be described using the same names in other embodiments. Unless otherwise stated, the description of any one embodiment may be applied to other embodiments, and a detailed description thereof will be omitted in the overlapping scope.

1 is a schematic diagram of a virtual reality interworking apparatus 1 according to an embodiment.

Referring to FIG. 1, a virtual reality interworking apparatus 1 according to an embodiment can improve the immersion feeling of a user U who experiences a virtual reality by interlocking a walking operation of a user U with a virtual reality . For example, the virtual reality interworking apparatus 1 can give a motion corresponding to the walking motion of the user U to the object set as the user U in the virtual reality. An object in the virtual reality may be referred to as an avatar (A). The virtual reality interworking apparatus 1 analyzes the actual operation of the user U and correlates the movement of the avatar A with the actual operation of the user U by interlocking the walking operation determined according to the analyzed data with the avatar A. [ It can be reproduced similarly to the walking motion.

If the user U experiences a virtual reality, the user U may be located in the locomotion device as in Fig. The locomotion device may allow free movement of the user U while restricting the position of the user U on the base B (base, B). When the user U is walking in the locomotion device, the actual position of the user U may be located in a certain range on the base B, regardless of the walking motion. In this case, the virtual reality interworking apparatus 1 realizes the walking operation of the avatar A in correspondence with the walking operation of the user U, ) Can be reproduced. In other words, the change in the position of the avatar A displayed on the display D can be realized only by the walking operation of the user U, regardless of the actual position of the user U. The virtual reality interworking apparatus 1 according to an embodiment may include a position sensor, an inertial sensor, a motion analyzing unit 130, and a control unit 140.

The position sensing unit 110 may acquire position information on the user body parts. The position sensing unit may set a point of the user's body part and acquire position information of the point point. The position detection unit 110 detects position information on the joint of the user U such as a head, a neck, a shoulder, an elbow, a cuff, a hand, a pelvis, a knee, Can be obtained. The position sensing unit 110 may acquire an input image including the user U and extract a user U image from the input image. The position sensing unit 110 can project infrared rays to the user U and recognize the three-dimensional position information on each body part of the user U based on the infrared rays reflected from the user U. [ The position sensing unit 110 may include, for example, a kinetic sensor including an infrared sensor and a camera. In this case, the position sensing unit 110 may recognize the body part of the user U from the user U image and obtain the position information of the recognized body part. For example, the position sensing unit 110 may recognize a plurality of joints from a user (U) image and obtain coordinate information on the position of each joint.

The inertial sensing unit 120 may acquire inertia information on the user's body part. For example, the inertial sensing unit 120 may be attached to a portion of a user's body part, such as a leg, waist, etc., of a user U to obtain inertial information, Can be measured in real time. The inertial sensing unit 120 can measure the acceleration in the X, Y and Z directions of the body part of the user U and the angular acceleration with respect to the yaw, roll and pitch in real time have. The inertial sensing unit 120 may include an inertia measurement unit (IMU) including, for example, a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis geomagnetic sensor. The inertial sensing unit 120 may acquire inertial information about the body part of the user U in real time so as to obtain information on the inclination, moving direction and speed of the body part according to the user U's walking.

The motion analyzing unit 130 analyzes the walking operation of the user U based on the positional information and the inertia information about the user's body parts obtained from the position sensing unit 110 and the inertial sensing unit 120 can do. Since the position of the user U experiencing the virtual reality is located in the locomotion device, the motion analyzer 130 can determine the position of the user U by using the position information and the inertia information on the body part of the user U, The walking direction and the walking speed of the user U can be determined.

The control unit 140 can link the motion analysis unit 130 and the virtual reality. The control unit 140 controls the movement of the user U in accordance with the actual walking of the user U to the avatar A by interlocking the walking operation of the user U determined through the motion analysis unit 130 with the user U in the virtual reality U, A). ≪ / RTI > In this case, the controller 140 adjusts the moving direction and the moving speed of the avatar A implemented in the virtual reality so that the user U experiencing the virtual reality is uncomfortable, for example, It is possible to alleviate the nausea phenomenon caused by the mismatch between the walking operation and the reproduction operation of the avatar A. [

2 is a schematic diagram showing a process of the position sensing unit 110 acquiring position information according to an embodiment.

Referring to FIG. 2, the position sensing unit 110 according to an embodiment may acquire position information of a body part of the user U. FIG. The position sensing unit 110 can recognize the body part of the user U from the image of the user U experiencing the virtual reality and obtain the three-dimensional position coordinates of the recognized body part. The position sensing unit 110 may divide the body of the user U into individual joints and provide the coordinates of the joints to the motion analyzer 130. For example, as shown in FIG. 2, the position sensing unit 110 sets the center of the base B as an origin, and detects the position of each part of the user's body such as a neck, a pelvis, a knee, and the like can be set as an absolute coordinate value for the origin. According to this method, since the relative positional changes of various body parts of the user U can be confirmed by numerical values, information such as the inclination, positional change, and directionality of each body part can be confirmed quickly and accurately. Hereinafter, for convenience of explanation, the center of the base B is assumed to be the origin.

FIG. 3 is a schematic diagram illustrating a process in which the inertial sensing unit 120 according to an embodiment acquires inertia information.

Referring to FIG. 3, the inertial sensing unit 120 according to an embodiment may acquire inertia information of a body part of the user U. FIG. The inertial sensing unit 120 is attached to the body part of the user U who wants to acquire the inertia information and acquires the change of the inertia information according to the motion of the body part in real time and provides it to the motion analyzer 130 . For example, the inertial sensing unit 120 may be attached to both feet of the user U to measure changes in acceleration and angular acceleration of both feet in real time as the user U makes a walk. As the walking speed of the user U increases, the change in the angle of the foot of the user U increases, so that the change in the angle of the foot can be accurately measured through the inertial sensing unit 120.

FIG. 4 is a flow chart of a walking motion analysis method according to an embodiment, and FIG. 5 is a flowchart of a walking motion determination step according to an embodiment.

4 and 5, a method for analyzing a gait operation according to an exemplary embodiment of the present invention includes a step 410 of acquiring position information on a body part of a user U, a step of acquiring inertia information of a body part of the user U, Step 420 and determining 430 a user U walking operation.

Step 410 may obtain positional information, e.g., three-dimensional absolute coordinate values, of the user (U) body part. Step 410 includes extracting a user (U) image from an observed image, recognizing a user (U) body part from a user (U) image, and acquiring position information of the recognized body part . In step 410, position information of the user's (U) body part can be obtained through the position sensing part 110. [ The user (U) body part may be, for example, the neck, pelvis or knee of the user (U).

Step 420 may obtain inertia information of the user (U) body part. Step 420 may include setting a user (U) body part to obtain inertia information, measuring the acceleration of the set body part, and measuring the rotational acceleration of the set body part. The set body part may be, for example, both legs of the user U.

Step 430 can determine the user U walking operation based on the acquired position information and inertia information. Step 430 may include determining 510 the user U walking direction, determining 520 the user U walking speed, and determining 530 whether the user U is walking .

FIG. 6 is a flowchart of a user U walking direction determination step according to an embodiment, and FIG. 7 illustrates a method of determining a walking direction according to an embodiment.

Step 510 may determine the walking direction according to the waist tilting direction of the user U. In other words, since the waist of the user U is inclined in the walking direction when the user U makes a walk, the walking direction of the user U can be set to the waist tilting direction of the user U.

Referring to FIG. 6, step 510 includes setting (610) an absolute position value of a first portion and a second portion of a user U, setting a plane coordinate value of the first portion and the second portion 620), calculating (630) the direction vector of the second portion for the first portion, and setting (640) the direction vector in the waist tilt direction.

Step 610 can recognize the first and second parts of the body of the user U and set the absolute position values of the first and second parts, i.e., the three-dimensional position coordinate values for the origin . The first and second portions may be various portions of the user (U) body. For example, the first and second portions may be the pelvis and neck of the user U. If the first and second portions are set to the pelvis and neck, accurate positional coordinate measurement may be possible since the problem of the user U being obscured by the locomotion device in various movements does not occur. Hereinafter, for convenience of explanation, the case where the first part and the second part are the pelvis and throat will be exemplarily described.

Step 620 can set the plane coordinate values of the first part and the second part with respect to the XY plane by removing the Z direction coordinates in the three-dimensional position coordinates of the first part and the second part.

Step 630 may yield the direction vector of the second portion for the first portion. In other words, the step 630 may calculate a direction vector having the first part as a 'viewpoint' and the second part as an 'end point'.

Step 640 may set the calculated direction vector to the waist tilt direction of the user U. As a result, the calculated direction vector can be set in the user U walking direction.

Referring to FIG. 7, when the user U performs a walking operation, the waist of the user U may be inclined toward the walking direction. The position sensing unit 110 can set the three-dimensional position coordinates (P1, P2) of the first portion and the second portion of the user U. Subsequently, the position sensing unit 110 replaces the three-dimensional position coordinates of the first and second portions with the plane coordinates (P1 ', P2') and calculates the direction vector D1 of the second portion with respect to the first portion . As a result, the vector direction of D1 can be set to the waist tilt direction of the user U, that is, the walking direction of the user U.

FIG. 8 is a flowchart of a step of determining a user U walking speed according to an embodiment, FIG. 9 is a flowchart of a first sensing value calculating method according to an embodiment, FIG. 10 is a flowchart FIG. 11 is a flowchart illustrating a third sensing value calculation method according to an exemplary embodiment of the present invention. Referring to FIG.

8-11, determining 520 the user U walking speed may include calculating 810 a first sensed value, calculating 820 a second sensed value, A step 830 of calculating a sensing value, and a step 840 of calculating a user U walking speed.

Step 810 may calculate the first sensed value based on the user (U) body inertia information. The first sensing value can be calculated based on, for example, inertia information of the user (U) foot. Step 810 includes a step 910 of measuring the moving speed of the foot of the user U, a step 920 of accumulating and storing the measured moving speeds, and a step of calculating a first sensing value according to the average value of the accumulated moving speed values (930).

Referring to FIG. 9, in step 810, the inertial sensing unit 120 may measure inertia information of both feet of the user U. For example, when the inertial sensing unit 120 is attached to both feet of the user U, the inertial sensing unit 120 may measure the moving speed of both feet of the user U through the amount of change in angle . If the moving speeds of both feet are different from each other, step 910 can select a higher value of the moving speed of both feet as the moving speed. Step 920 can measure the moving speeds of both feet in real time, and accumulate and store the measured values of the moving speed. For example, measured values of the traveling speed can be accumulated in units of n. When the cumulative value of the moving speed exceeds n, the measured value of the moving speed can be updated in the order of measurement. Step 930 can calculate the first sensing value according to the average of the n measured values of the accumulated moving speed. The calculated first sensing value can be updated in real time in accordance with the user U walking.

Step 820 may yield a second sensed value based on the user (U) body position information. The second sensing value may be calculated based on, for example, the user U waist slope. For example, the second sensing value can be calculated by converting the measured user U's waist slope to a predetermined numerical value. In this case, the predetermined numerical value may range from -1 to +1. In other words, the predetermined numerical value can be determined according to the degree of the waist inclination. For example, as the walking speed of the user U increases, the predetermined numerical value converges to the absolute value 1 as the degree of the waist inclination increases. On the other hand, as the walking speed of the user U decreases and the degree of the waist inclination decreases, the preset numerical value can converge to zero.

Referring to FIG. 10, step 820 includes setting (1010) plane coordinates of a first portion and a second portion of the user (U) body, calculating a plane direction vector of the second portion for the first portion A step 1030 of accumulating and storing the direction vectors, and a step 1040 of calculating a second sensing value according to an average value of the accumulated direction vectors. In this case, the first and second portions may be the pelvis and neck of the user U. The direction vector of the second part for the first part can be calculated in the same manner as in Fig.

Step 830 may yield a third sensed value based on the user (U) body position information. The third sensing value may be a correction constant for increasing the reliability of the walking speed of the user. The third sensing value can be calculated based on, for example, the distance between the knees of the user U. [ Step 830 includes a step 1110 of measuring the distance between both knees of the user U, a step 1120 of accumulating and storing the measured amount of change in distance, and a step of calculating a third sensed value according to the average value of the accumulated amount of change Step 1130 may be included.

Referring to FIG. 11, in step 1110, the position sensing unit 110 may measure the distance between both knees of the user U in real time. For example, since the crossing between the knees of the user U increases as the walking speed of the user U increases, the change in the walking speed of the user U can be determined through the change in the distance between the knees.

In step 1120, the measured distance variation can be accumulated and stored. In step 1120, it is possible to select and store only the measured values from which noise has been removed by determining whether the measured distance variation is reliable. The method of determining whether or not to trust the distance variation measured in real time is not limited. For example, in step 1120, the data of the accumulated distance variation amount is compared with the newly measured distance variation amount, and when the difference of the comparison value is out of a certain range, it is determined that the reliability of the newly measured distance variation is insufficient . In this case, the measured value that is determined to be unreliable can be classified and removed as noise. In step 1120, the reliability of the distance variation measured by other information may be determined. For example, if it is determined that the user's walking speed is slowed by information such as a waist inclination, step 1120 may be performed by storing the measured value only when the measured distance variation amount decreases, have.

Step 1120 determines the reliability of the measured data in step 1110 through the accumulated data storage value. If the reliability of the measured data in step 1110 is insufficient, the newly measured data is supplemented with accumulated data, Reliable data can be deduced.

Step 1130 can calculate the third sensing value based on the average value of the accumulated distance variation. Since the third sensing value is calculated based on the average value of the accumulated data, it is possible to prevent the abrupt change in the user's walking speed determined based on the measured data. Also, since the third sensing value uses the accumulated average value of the data updated in real time, high reliability can be assured in determining the walking speed of the actual user.

Step 840 may determine the walking speed of the user U through the first sensing value, the second sensing value, and the third sensing value. Step 840 can calculate a reliable walking speed by correcting the determination value for the user's walking speed through each sensed value. The calculation method for determining the user's walking speed through each sensing value is not limited. For example, in step 840, a first sensed value measured through inertia information is converted into a velocity value, a second sensed value and a third sensed value are converted into a weight value, and the first sensed value is multiplied by the user U) can be calculated. As a result, in determining the walking speed of the user U, the step 840 compensates for the inertia information and the data measured through the position information to obtain a more accurate The user (U) walking speed can be calculated in real time.

Hereinafter, a virtual reality interlocking method according to a user U walking operation according to an embodiment will be described. In describing the virtual reality interworking method, contents overlapping with the above description will be omitted.

12 is a flowchart of a virtual reality interworking method according to an embodiment.

The virtual reality interworking method according to one embodiment is a method of interlocking the walking operation of the user U with the avatar A of the user U in the virtual reality so that the user U experiencing the virtual reality actually walks It can provide a similar feeling. The virtual reality interworking method includes a step 1210 of obtaining positional information of a user's body part based on image information, a step 1220 of acquiring inertia information of a user's leg through an inertial sensor, A step 1230 of determining a walking (U) walking operation, and a step (step 1240) of interlocking the determined walking operation with the avatar A. [

In step 1210, information about the location of the user (U) body part may be obtained in real time. For example, in step 1210, the position sensing unit 110 may acquire three-dimensional position coordinates of the body part of the user U. At step 1220, inertial information of both feet of the user (U) leg, for example, the user U, may be obtained via the inertial sensor.

Step 1230 may determine the walking operation of the user U. For example, in step 1230, the motion analyzing unit 130 analyzes the walking motion of the user U based on the positional information and inertial information about the user U, Can be determined. Step 1230 may include determining the walking direction of the user U based on the positional information and determining the walking speed of the user U based on the positional information and the inertial information.

The step of determining the walking direction of the user U includes the steps of setting the plane coordinate values of each of the pelvis and neck based on the positional information of the user ' s pelvis and neck, and setting the direction vector of the neck with respect to the pelvis . ≪ / RTI > In other words, the waist tilting direction of the user U can be detected through the direction vector of the neck with respect to the pelvis, and the walking direction of the user U can be determined.

The step of determining the walking speed of the user U comprises the steps of calculating a first sensed value according to the moving speed of the user U leg, calculating a second sensed value according to the waist inclination of the user U, Calculating a third sensed value according to the distance between the knees of the user U, and calculating a walking speed of the user U based on the first sensed value, the second sensed value, and the third sensed value . The step of determining the walking speed of the user U may be performed in the same manner as in Figs. 8 to 11. Fig. The first sensing value, the second sensing value, and the third sensing value can be calculated through a cumulative average value of data measured in real time. In this case, it is possible to prevent the determination value of the user's walking speed from rapidly changing. For example, since a user experiencing a virtual reality can not continue to run at a constant speed, when the user's walking speed is determined using only data measured in real time, even if the walking speed of the actual user U is slightly reduced, The moving speed of the avatar A interlocked with the moving object U may be drastically reduced. Therefore, by determining the walking speed of the user through the average value of the accumulated data, it is possible to make the walking speed of the determined user U close to the intended speed of the real user U.

FIG. 13 is a flow chart of a step of interlocking the walking operation according to the embodiment with the avatar A, and FIG. 14 is a diagram showing a method of interlocking the moving direction of the avatar A according to an embodiment.

13, step 1240 can reproduce an action similar to the actual walking of the user U through the avatar A in the virtual reality by interlocking the determined walking user's operation with the avatar A . Step 1240 includes a step 1310 of linking the moving direction of the avatar A to a walking direction of the user U, a step 1320 of linking the moving speed of the avatar A to the traveling speed of the user U, And stopping the walking operation (Step 1330).

Step 1310 can link the moving direction of the avatar A in the user U walking direction. 14, when the user U makes a gait operation, the walking direction of the user U is set to a direction vector D1 based on the position coordinates P1 and P2 of the neck and the pelvis of the user U, Lt; / RTI > In other words, the walking direction of the user U may be the direction with the direction vector D1 as the front. Assuming that D1 is a unit vector with respect to the X axis, the walking direction of the avatar A on the virtual reality may be a unit vector with respect to the x axis on the virtual reality. As a result, the front direction of the avatar A on the virtual reality coincides with the front direction of the user U, thereby matching the moving direction of the user U with the desired direction of the avatar A on the virtual space . In this case, by minimizing the difference between the actual walking direction of the user U and the walking direction of the avatar A, it is possible to solve the discomfort felt by the user U, for example, a nausea.

Step 1320 can link the avatar (A) moving speed to the user (U) walking speed. For example, the step 1320 can match the moving distance of the avatar A on the virtual space according to the determined walking speed of the user U. Therefore, by minimizing the difference between the walking distance of the user U and the moving distance of the avatar A, the user U experiencing the virtual reality can provide a feeling similar to the actual walking.

Step 1330 can stop the walking of the avatar A based on the user U walking operation. For example, in step 1330, if there is no inertia change value measured through the inertial sensor, the walking operation of the avatar A can be immediately stopped. Since the average value of the accumulated sensing values is used in the process of determining the walking speed of the user U, when the user U actually stops the walking, the walking speed of the real user U is set to the real time As shown in FIG. Therefore, when there is no inertia change value, the walking of the actual user U and the walking of the avatar A can be made to coincide with each other by immediately stopping the walking of the avatar A on the virtual reality.

FIG. 15 is a flowchart of a step of associating an avatar A according to an embodiment.

Referring to FIG. 15, step 1240 may correct the moving speed of the avatar A through a correction value according to the elongation of the user U. FIG. Step 1240 includes a step 1510 of calculating a correction value according to the user's extension U, a step 1520 of correcting the user U walking speed through the correction value, and a step of moving the avatar A according to the corrected walking speed And setting a speed (step 1530).

Step 1510 can calculate a correction value according to the elongation of the user U. [ In the case of determining the walking speed according to the waist inclination of the user U, since the plane vector value for the XY plane is used, the deviation of the inclination value according to the height of the user U can be increased. In other words, even when walking at the same speed, the plane vector value of the waist inclination of a person with a large height is larger than that of a person having a small height, so that an error of the waist inclination value due to elongation may occur. Therefore, step 1510 can calculate a correction value for reducing the error of the walking speed of the user U. [ In this case, the correction value may be determined by adjustment of the user U.

Step 1520 can correct the user U walking speed through the correction value. For example, in the case of a person with a large height, it is possible to minimize the error in the walking speed due to the difference in height by correcting the walking speed determined through the correction value to be lower. On the other hand, a person with a small height has a large fluctuation of the waist inclination value according to the walking speed, so that the error of the walking speed can be minimized by increasing the value of the waist inclination value through the correction value.

Step 1530 can set the moving speed of the avatar A in the virtual reality according to the corrected walking speed. As a result, it is possible to reduce the inconvenience of the user U by setting the moving speed of the avatar A in the virtual reality to be similar to the walking speed of the user U regardless of the height difference of the user U .

FIG. 16 is a flowchart of a step of associating an avatar A according to an embodiment.

Referring to FIG. 16, step 1240 may include a step 1610 of setting a movement restriction speed of the avatar A and a step 1620 of limiting the movement speed of the set avatar A to a movement restriction speed or less have.

Step 1610 can set the movement limit speed of the avatar A on the virtual reality. Since the distance on the virtual reality and the actual distance in reality may be different from each other, the maximum moving speed of the avatar A can be limited by the setting of the user U in step 1610.

In step 1620, the moving speed of the avatar A on the virtual reality can be limited to the moving limit speed or less. In this case, the moving speed of the avatar A on the virtual reality can be limited to a limit speed or less so that the user U experiencing the virtual reality does not feel uncomfortable regardless of the determined value of the walking speed.

17 is a flowchart of a virtual reality interworking method according to an embodiment.

The virtual reality interworking method includes a step 1710 of calculating a direction vector, a step 1720 of determining a moving direction of the avatar A, a step 1720 of determining a walking speed of the user U, A step 1750 of determining whether the user U is moved, and a step 1760 of moving the avatar A. In this case,

In step 1710, a direction vector with respect to the waist inclination direction can be calculated through the position information of the neck and the pelvis of the user U acquired through the position sensing unit 110.

In step 1720, the moving direction of the avatar A on the virtual reality can be determined through the calculated direction vector.

In step 1730, position information of both knees of the user U acquired through the position sensing unit 110, inertia information of both legs of the user U acquired through the inertia sensing unit 120, The user can determine the walking speed of the user U through the direction vector for the user U.

In step 1740, the moving speed of the avatar A can be determined through the determined walking speed of the user U. In this case, when the determined moving speed of the user U exceeds the movement restricting speed of the avatar A, the moving speed of the avatar A is set to the movement restricting speed, The moving speed of the avatar A can be set by the determined user U walking speed.

In step 1750, the inertial sensing unit 120 may determine whether the user U is actually moving. If there is no change in the inertial value of the user's foot measured through the inertial sensing unit 120, it is determined that the walking of the user U is stopped and the moving operation of the avatar A can be immediately stopped .

In step 1760, the avatar A can be moved according to the determined moving direction of the avatar A, the moving speed of the avatar A, and whether the avatar A is moved.

Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. For example, it is contemplated that the techniques described may be performed in a different order than the described methods, and / or that components of the described structures, devices, and the like may be combined or combined in other ways than the described methods, Appropriate results can be achieved even if they are replaced or replaced.

Therefore, other implementations, equivalents to other embodiments and the claims are also within the scope of the following claims.

U: User
A: Avatar
1: Virtual reality interlock
110:
120: inertia sensing unit
130: motion analysis unit
140:

Claims (21)

A method for analyzing user gait for virtual reality interworking,
Obtaining location information for the user body part;
Obtaining inertia information for the user body part; And
And determining the user walking operation based on the position information and the inertia information.
The method according to claim 1,
Wherein the acquiring of the position information comprises:
Extracting a user image from an observed image;
Recognizing a user body part from the user image; And
And obtaining position information of the recognized body part.
The method according to claim 1,
Wherein the acquiring of the inertia information comprises:
Setting a user body part for obtaining the inertia information;
Measuring an acceleration of the set body part; And
And measuring the rotational acceleration of the set body part.
The method according to claim 1,
Wherein the step of determining the user ' s walking motion comprises:
Determining a walking direction of the user; And
Determining a walking speed of the user;
5. The method of claim 4,
Wherein the step of determining the walking direction of the user comprises:
And determining the walking direction in accordance with the waist tilting direction of the user.
5. The method of claim 4,
Wherein the step of determining the walking direction of the user comprises:
Setting a plane coordinate value of the first portion and the second portion of the user;
Calculating a direction vector of a second portion for the first portion; And
And setting the direction vector in the walking direction.
5. The method of claim 4,
Wherein the step of determining the user's walking speed comprises:
Calculating a first sensed value according to inertia information of the user's body;
Calculating a second sensing value according to the user's body position information;
Calculating a third sensing value according to the user's body position information; And
And calculating the walking speed of the user based on the first sensing value, the second sensing value, and the third sensing value.
8. The method of claim 7,
Wherein the calculating the first sensing value comprises:
Measuring a moving speed of the user's foot;
Accumulating and storing the measured moving speed values; And
And calculating a first sensing value according to an average value (normalize) of the accumulated moving speed values.
8. The method of claim 7,
Wherein the calculating the second sensing value comprises:
Measuring a waist inclination of the user; And
And converting the measured waist slope to a predetermined numerical value.
8. The method of claim 7,
The step of calculating the third sensing value includes:
Measuring a distance between the user and the knee;
And calculating a third sensing value according to the measured amount of change in distance.
5. The method of claim 4,
The step of determining the walking operation includes:
Further comprising the step of determining whether the user is walking based on inertia information of the body part of the user.
A virtual reality interworking method according to a user walking motion,
Acquiring positional information of the user's body part based on the image information;
Acquiring inertia information of the user's leg through the inertial sensor;
Determining a walking operation of the user based on the position information and the inertia information; And
And linking the determined gait to an avatar of a user in the virtual reality.
13. The method of claim 12,
The step of determining the walking operation includes:
Determining a walking direction of the user based on the position information;
And determining the walking speed of the user based on the positional information and the inertial information.
14. The method of claim 13,
Wherein the step of determining the walking direction of the user comprises:
Setting plane coordinate values of each of the pelvis and neck based on positional information of the user's pelvis and neck; And
And calculating a direction vector of the neck with respect to the pelvis.
15. The method of claim 14,
Wherein the step of interlocking with the avatar comprises:
And linking the direction of movement of the avatar according to the calculated direction vector.
14. The method of claim 13,
Wherein the step of determining the user's walking speed comprises:
Calculating a first sensing value according to a moving speed of the user's leg;
Calculating a second sensing value according to the user's waist inclination;
Calculating a third sensing value according to the distance between the user's knees; And
And calculating the walking speed of the user based on the first sensing value, the second sensing value, and the third sensing value.
17. The method of claim 16,
Wherein the step of interlocking with the avatar comprises:
Calculating a correction value according to a height of the user;
Correcting the calculated walking speed through the correction value;
And setting the moving speed of the avatar according to the corrected walking speed.
13. The method of claim 12,
The step of interworking with the avatar
Setting a movement limit speed of the avatar;
And restricting the moving speed of the avatar to be less than or equal to the movement limit speed.
13. The method of claim 12,
Wherein the step of interlocking with the avatar comprises:
And stopping the walking operation of the avatar when there is no change in the numerical value measured from the inertial sensor.
19. A computer-readable storage medium having stored thereon one or more programs comprising instructions for performing the method of any one of claims 1 to 18.
A virtual reality interworking apparatus for a walking operation of a user,
A position sensing unit for acquiring position information of the user's body part;
An inertial sensing unit attached to the user's body for acquiring inertial information of the attached part;
A motion analyzer for analyzing the user's walking motion based on the position information and the inertia information; And
And a controller for linking the motion analyzer and the virtual reality.

Images