KR101686629B1 - Method for determining location in virtual space indicated by users input regarding information on pressure and apparatus and computer-readable recording medium using the same - Google Patents

Abstract

According to one aspect of the present invention, there is provided a method of determining a position on a virtual space indicated by input of pressure information of a user, comprising the steps of: (a) inputting by a user a plurality of pressure measurement sensors included in a user interface Obtaining pressure information; And (b) determining position information on the virtual space based on the distribution of the pressure information, wherein the arrangement of the plurality of pressure measurement sensors is a 2D plane or a 2D plane that can be approximated as a matrix- And an arrangement in 3D space. Through this, it is possible to precisely determine the position information on the virtual space corresponding to the user's operation through the user interface, and to precisely interact with the virtual space system and the user. Based on the pressure measurement sensor in contact with various body parts of the user The location information on the virtual space corresponding to the various operations of the user can be accurately indicated.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for determining a position on a virtual space indicated by input of pressure information of a user and a computer readable recording medium for executing the method. APPARATUS AND COMPUTER-READABLE RECORDING MEDIUM USING THE SAME}

The present invention relates to a method and apparatus for accurately analyzing a position indicated by a user in a virtual space by measuring a change in a pressure distribution according to a movement of a user and transmitting sensory information about a virtual object located on the virtual space to a user, And a computer-readable recording medium for executing the method.

The existing user interface has changed from a text-based user interface using a keyboard to a user interface using a mouse. Keyboards, mice, and joysticks are popular and dominant devices in traditional computing environments. However, traditional virtual two-dimensional input devices, such as keyboards and mice, are not appropriate because they are unnatural and inconvenient for inputting information about user behavior in a virtual environment where natural, efficient, and powerful and flexible interactions are the primary focus.

Accordingly, various user interfaces for inputting user's operation information into the virtual space have been developed. Various sensor-based user interfaces for virtual space can detect user's body posture and direction, direction of gaze, horse and sound, facial expression, skin reaction and human behavior, You can enter information. The user interface for virtual space should be able to naturally reflect the operation of the actual user in the virtual space. Recently, the analysis and research of human motion have been dealt with in order to develop such a user interface in HCI (human computer interaction) field. That is, in order to reflect the information of the user's operation in the virtual space, accurate analysis of the user's operation by the user interface is required.

Application No. 10-2009-0102228 Three-dimensional space interface device and method

SUMMARY OF THE INVENTION The present invention has been made to solve all the problems described above.

According to the present invention, a plurality of pressure measurement sensors included in a user interface are used to obtain pressure information due to motion, and a position on a virtual space indicated by a motion based on pressure information is determined, And to accurately determine the position on the virtual space.

It is another object of the present invention to provide a user interface capable of finely controlling a virtual object located in a virtual space.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to one aspect of the present invention, there is provided a method of determining a position on a virtual space indicated by input of pressure information of a user, comprising the steps of: (a) inputting by a user a plurality of pressure measurement sensors included in a user interface Obtaining pressure information; And (b) determining position information on the virtual space based on the distribution of the pressure information, wherein the arrangement of the plurality of pressure measurement sensors is a 2D plane or a 2D plane that can be approximated as a matrix- The method comprising arranging in 3D space.

According to another aspect of the present invention, there is provided an apparatus for determining a position on a virtual space indicated by an input of pressure information of a user, comprising: a plurality of pressure measurement sensors included in a user interface, A measuring part to acquire; And an intention analysis unit for determining positional information on the virtual space based on the distribution of the pressure information, wherein the arrangement of the plurality of pressure measurement sensors is a 2D plane or a 3D space that can be approximated by a matrix- Lt; RTI ID = 0.0 > 1, < / RTI >

In addition to this, there is further provided a computer readable recording medium for recording a computer program for executing the method and an apparatus and an apparatus for implementing the present invention.

According to the present invention, the location information on the virtual space corresponding to the user's operation is accurately determined through the user interface, and the user can precisely interact with the virtual space system.

In addition, the position information on the virtual space corresponding to various operations of the user can be accurately indicated based on the pressure measuring sensor in contact with various body parts of the user.

1 is a conceptual diagram illustrating an arrangement of a pressure measurement sensor for estimating a user's intention in a virtual space according to an embodiment of the present invention.
2 is a conceptual diagram illustrating a method of estimating position information on a virtual space indicated by a distribution of pressure information input by a user according to an embodiment of the present invention.
3 shows a modification of the arrangement of the pressure measuring sensor according to the embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

When the user's operation is analyzed in the existing 3D space (or virtual space) and used as input information, the user has performed an unnecessarily large operation to input a specific operation as input information. That is, since it is difficult to analyze the detailed operation of the user in the user interface for the existing virtual space, the user has performed an unnecessarily large operation in order to input the operation information through the user interface in the virtual space.

When the user interface for the virtual space is not aware of the detailed operation of the user, the user's fatigue due to the continuous large operation, the error in the judgment of the user's operation, and the incorrect recognition of the user's operation are utilized in the virtual space It can be a problem. Hereinafter, an embodiment of the present invention discloses a user interface for measuring a pressure generated by a user's motion in contact with a user to solve such a problem, and accurately utilizing the measured pressure for object control of a virtual space. The detailed operation by the user can be accurately recognized through the user interface according to the embodiment of the present invention.

[Preferred Embodiment of the Present Invention]

1 is a conceptual diagram illustrating an arrangement of a pressure measurement sensor for estimating a user's intention in a virtual space according to an embodiment of the present invention.

FIG. 1 illustrates a method for analyzing a user's intention based on information obtained through contact between a user interface 100 and a body part of a user. The body part of the user in contact with the user interface 100 may be a part where a movement by the user occurs. For example, the body part of the user in contact with the user interface may be a part of the user's joint (wrist, elbow, finger joint, etc.).

The user interface 100 may measure the pressure measured through contact with the body part of the user. For example, the user interface 100 may be implemented based on a plurality of pressure measurement sensors 120. The plurality of pressure measurement sensors 120 may be arranged in a matrix form (or a coordinate form) in the user interface 100 in a plurality of directions. Each of the plurality of pressure measurement sensors 120 can sense a pressure generated due to the movement of the user at a position corresponding to each of the plurality of pressure measurement sensors 120.

Hereinafter, in the embodiment of the present invention, the body part of the user in contact with the user interface 100 can be mainly described by assuming a wrist. However, as described above, various body parts as well as the wrist may be in contact with the user interface 100. The movement of the wrist can be independent of the movement of the finger region. In the case of the wrist, since the force is normally applied, the increase in additional fatigue due to the force to be used can be minimized. The user interface 100 that receives information on the movement of the wrist in the virtual space may replace the functions of the existing user interface such as a mouse.

Hereinafter, the user interface 100 includes a measurement unit (not shown) including a plurality of pressure measurement sensors 120 for measuring pressure and an intention analysis unit (not shown) for analyzing the intention of the user by analyzing the distribution of the measured pressure ) Are included. The analyzed user's intention information can be used to control objects in the virtual space. However, the measuring unit and the intent analyzing unit may be separately configured, the measuring unit may be included in the user interface 100, and the intent analyzing unit may be separately managed.

The concrete pressure measurement and user's intention estimation method of the user interface 100 can be performed as follows. A pressure value at a plurality of points in contact with a user's body part (e.g., a wrist) can be measured through a plurality of pressure measurement sensors on the user interface. The intention analysis unit can estimate the position information on the virtual space indicated by the user's operation based on the distribution of the pressure values measured at a plurality of points. Based on the distribution of the pressure values measured at a plurality of points, the position information on the virtual space corresponding to the user's intention can be obtained by various methods.

For example, the intention analysis unit can determine the pressure distribution function for each measurement location and determine the probability distribution function for the pressure at each measurement location based on the pressure distribution function for each measurement location. In addition, the intention analysis unit can determine the x-axis direction pressure estimation point and the y-axis direction pressure estimation point based on the probability distribution function for the pressure at each measurement position. Further, the total pressure estimation value can be determined based on the x-axis direction pressure estimation point and the y-axis direction pressure estimation point. The x-axis direction pressure estimation point, the y-axis direction pressure estimation point, and the total pressure estimation value may be normalized based on the x-axis direction measurement range, the y-axis direction measurement range, and the total pressure measurement range, respectively. According to this, the x-axis direction pressure estimation point, the y-axis direction pressure estimation point and the total pressure estimation value can be converted into the normalized x-axis direction pressure estimation point, the normalized y-axis direction pressure estimation point and the normalized total pressure estimation value.

The normalized x-axis direction pressure estimation point, the normalized y-axis direction pressure estimation point, and the normalized total pressure estimation value can be calculated as position information on the virtual space, for example, by linear transformation. The location information on the virtual space may reflect the intention of the user determined by the distribution of the pressure information inputted by the user. The selection information of the user, that is, the position information on the virtual space reflecting the intention of the user, may be referred to as an x-axis direction selection point, a y-axis direction selection point, and a z-axis selection point. That is, the normalized x-axis direction pressure estimation point, the normalized y-axis direction pressure estimation point, and the normalized total pressure estimation value can be linearly transformed to the x-axis direction selection point, the y-axis direction selection point, and the z-axis selection point. A method of estimating the position information on the virtual space based on the pressure values measured at the plurality of points will be described in detail below.

The user's behavior may be analyzed based on nonlinear transformations as well as the above method. The nonlinear transform function for determining the position information in the virtual space corresponding to the user's intention may be a machine-learned function based on the input values of the collected users.

2 is a flowchart illustrating a method of estimating position information on a virtual space indicated by a distribution of pressure information input by a user according to an embodiment of the present invention.

Referring to FIG. 2, a plurality of pressure measurement sensors may measure (or sense) a pressure value at each of a plurality of pressure measurement sensors (step S200).

 From the pressure values measured at the plurality of pressure measurement sensors, information on the pressure distribution (for example, a pressure distribution function) can be obtained.

The plurality of pressure measurement sensors may be arranged (or arranged) in the form of, for example, a lattice (or matrix, coordinate). When a plurality of pressure measurement sensors are arranged in a matrix form, the position of each of the plurality of pressure measurement sensors may be expressed by (x, y) which is information on the y row and the x column. When a plurality of pressure measurement sensors are arranged in a two-dimensional coordinate form, the position of each of the plurality of pressure measurement sensors may be represented by (x, y) which is integer coordinate information on the x-axis and the y-axis. Hereinafter, for convenience of explanation, the position of each of the plurality of pressure measurement sensors may be expressed based on a two-dimensional coordinate form.

The plurality of pressure measurement sensors can measure the pressure value input by the user. For example, when a user places a palm on the surface of a user interface where a plurality of pressure measurement sensors are located, and applies a predetermined pressure, each of the plurality of pressure measurement sensors measures a pressure value corresponding to a position of the plurality of pressure measurement sensors . The pressure values measured at each of the plurality of pressure measurement sensors can be expressed as f (x, y) which is a pressure distribution function according to x, y. Hereinafter, a method of estimating position information on a virtual space based on pressure values measured by a plurality of pressure measurement sensors located at a plurality of points will be described in an embodiment of the present invention.

A probability distribution function is determined based on the pressure distribution function f (x, y) (step S210).

Equation (1) below represents the probability distribution function p (x, y) generated by dividing the pressure distribution function f (x, y) by the total sum of the pressure distributions.

[Equation 1]

When a pressure value measured by a pressure measuring sensor located at a specific point is divided into a sum of the total pressure distributions, a kind of probability distribution function p (x, y) as shown in equation (1) can be obtained.

Referring to Equation (1), it is assumed that a plurality of pressure measurement sensors are arranged in N rows and M rows. That is, a plurality of N pressure measurement sensors in the x-axis direction and a plurality M of pressure measurement sensors in the y-axis direction may be arranged on the user interface. The position of each of the plurality of pressure measurement sensors may be expressed in coordinates corresponding to one of 1 to N in the x-axis direction and one of 1 to M in the y-axis direction with respect to the coordinate axis.

The x-axis direction pressure estimation point, the y-axis direction pressure estimation point, and the total pressure estimation value may be determined based on the probability distribution function of Equation (1) (step S220).

Equation (2) below shows a method of estimating an x-axis direction pressure estimation point and a y-axis direction pressure estimation point.

&Quot; (2) "

As an example, referring to Equation (2), an x-axis direction pressure estimation point (x _u ) can be determined by averaging the probability distribution function of Equation (1) with respect to the x-axis direction. Similarly, referring to Equation (2), the y-axis direction pressure estimation point (y _u ) can be determined by averaging the probability distribution function of Equation (1) with respect to the y-axis direction.

Next, the total pressure estimation value f _u can be determined based on the x-axis direction pressure estimation point (x _u ) and the y-axis direction pressure estimation point (y _u ). x axial pressure estimating point _(u x) and y-pressure estimating point (y _u) to a value that a cutting point or less in each of _{x-, y-, (x u,} y u) from the total pressure of the The estimated value f _u may be determined based on Equation (3) below.

&Quot; (3) "

That is, the four points total pressure estimate value (f _u) is _{((x -, y -)} , (x - +1, y -), (x -, y - +1), (x - +1, y _- +1)) may be an average of the respective pressure values. The total pressure estimate f _u may then be used to determine the point corresponding to the user's movement in the virtual space with respect to the z-axis. This will be described later.

The x-axis direction pressure estimation point (x _u ) and the y-axis direction pressure estimation point (y _u ) and the total pressure estimation value (f _u ) are respectively the x-axis direction measurement range, the y- (Step S230).

This normalization procedure may be referred to as a measurement range-based conversion procedure. Equation 4 and Equation 5 below show this.

First, the total pressure estimated value f _u can be converted to the normalized total pressure estimated value f _n based on Equation (4) below.

&Quot; (4) "

In Equation (4), f _max is the maximum value that the pressure measurement sensor can measure, and f _min can be the minimum value that the pressure measurement sensor can measure. Through equation (4), the total pressure estimate can be converted to a normalized total pressure estimate based on the total pressure measurement range. f _n may have a value between 0 and 1.

Similarly, the x-axis direction pressure estimation point (x _u ) is normalized based on the x-axis direction measurement range, and the y-axis direction pressure estimation point (y _u ) is normalized based on the y-axis direction measurement range. Equation (5) below expresses this.

&Quot; (5) "

The normalized x-axis direction pressure estimation point (x _n ), the normalized y-axis direction pressure estimation point (y _n ), and the normalized total pressure estimation value f _n have values between 0 and 1, Respectively.

The normalized x-axis direction pressure estimation point (x _n ), the normalized y-axis direction pressure estimation point (y _n ), and the normalized total pressure estimation value (f _n ) To the final position information (x-axis direction selection point, y-axis direction selection point and z-axis selection point) (step S240).

For example, the position information on the virtual space corresponding to the distribution of the pressure information input by the user may be (x _user , y _user , z _user ). It can be assumed that the range of the user x is [x _min , x _max ], the range of the user y is [y _min , y _max ], and the range of the user z is set to [z _min , z _max ]. In this case, the normalized x-axis direction pressure estimation point (x _n ), the normalized y-axis direction pressure estimation point (y _n ) and the normalized total pressure estimation value f _n based on the following Equation (6) (The x-axis direction selection point, the y-axis direction selection point, and the z-axis selection point) on the virtual space corresponding to the user's intention.

&Quot; (6) "

That is, it can be converted into positional information on the virtual space corresponding to the distribution of pressure information measured based on a plurality of pressure measurement sensors located at a plurality of points based on the above-described method.

Of course, the above-mentioned mathematical expression is only described as an example, and the final position on the virtual space will not necessarily be determined by the above equation.

3 shows a modification of the arrangement of the pressure measuring sensor according to the embodiment of the present invention.

The user interface may be implemented in a specific form adapted to a specific area so as to be applied only to a specific area of the user, but the user interface may contact the various areas of the user to estimate the location information on the virtual space corresponding to the user's input .

For example, each of the plurality of pressure measurement sensors of the user interface may be connected via wired or wireless to sense pressure information generated by the user in contact with various body parts of the user. Hereinafter, it is assumed that each of the plurality of pressure measurement sensors is connected via radio.

The user interface having the above structure can be used to estimate position information on a virtual space corresponding to a user's operation by being connected to various parts of the user's body.

For example, the first pressure measurement sensor 305 and the second pressure measurement sensor 310 are joints of the first finger of the five fingers, the third pressure measurement sensor 315 and the fourth pressure measurement sensor 320 The joint of the second finger of the five fingers, the fifth pressure measuring sensor 325 and the sixth pressure measuring sensor 330 are joints of the third finger of the five fingers, the seventh pressure measuring sensor 335, The measurement sensor 340 may be located at the joint of the fourth finger among the five fingers, the ninth pressure measurement sensor 345 and the tenth pressure measurement sensor 350 may be located at the joints of the fifth finger among the five fingers. Each pressure measurement sensor can be connected to a user's body (e.g., a finger) using various types of fasteners.

The first pressure measurement sensor 305 to the tenth pressure measurement sensor 350 connected to each of the plurality of fingers may transmit the distribution of the pressure information generated by the movement of the user's finger to the intention analysis unit of the user interface. The intention analysis unit can estimate the position information on the virtual space corresponding to the motion of each of the plurality of fingers by using the method described in Fig.

For reference, in the example of FIG. 3, although it is not a matrix as a whole as in FIG. 1, it may be approximated to a sensor group including a matrix form of a small portion. For example, the first pressure measurement sensor 305 and the second pressure measurement sensor 310 may be regarded as a small scale matrix, and the third pressure measurement sensor 315, the fourth pressure measurement sensor 320, The measurement sensor 325 and the sixth pressure measurement sensor 330 may be approximated in the form of another small scale matrix or the fifth pressure measurement sensor 325, the sixth pressure measurement sensor 330, (E. G., A matrix of trapezoidal form) of the eighth pressure sensor 335, the eighth pressure sensor 340, the ninth pressure sensor 345 and the tenth pressure sensor 350 may be approximated It can be approximated to the case where each sensor is arranged in various types of matrix. Although only the fingers are assumed in Fig. 3, the pressure measurement may be performed from various body parts other than the finger.

The intention analysis unit can estimate the 3D position information desired by the user based on the information about the measured values of the sensors approximated by arranging the measurement values or partial portions of the sensors arranged in the overall matrix form in a small matrix form. In addition, the relative positional relationship (topology relationship) of the sensor groups disposed at arbitrary positions on the two-dimensional or three-dimensional plane may also be approximated by a matrix of partial portions. That is, even if the sensor group is arranged in three dimensions, if the sensor group is arranged close to a specific plane, the sensor group can be approximated in this way.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

100: User interface
120: Pressure sensor

Claims (11)

A method for determining a position on a virtual space indicated by an input of pressure information of a user,
(a) acquiring pressure information input by the user using a plurality of pressure measurement sensors included in a user interface; And
(b) determining position information on the virtual space based on the distribution of the pressure information,
Wherein the arrangement of the plurality of pressure measurement sensors comprises an arrangement in a 2D plane or 3D space that can be approximated by a matrix arrangement or matrix arrangement,
The step (b)
(b1) determining a pressure distribution function based on pressure information obtained from the plurality of pressure measurement sensors arranged;
(b2) obtaining a probability distribution function obtained by dividing the pressure distribution function by the total sum of the pressure distributions; And
(b3) determining location information on the virtual space based on the probability distribution function. delete The method according to claim 1,
The step (b3)
(b31) when the matrix or an arrangement of a shape approximating the matrix is located in a plane consisting of a first axis and a second axis orthogonal to the first axis, the first axis direction and the second axis Obtaining an average value for the direction;
(b32) estimating a pressure value at an average value in the first axis direction and the second axis direction;
(b33) normalizing a pressure value in the first axial direction and the second axial direction, and an average value in the first axial direction and the second axial direction; And
(b34) converting the normalized values to a range of 3D position information desired by the user. The method of claim 3,
The step (b34)
Wherein the linear transformation is performed on the normalized first axial direction, the normalized second axial direction, and the normalized pressure values. The method of claim 3,
The step (b34)
The input values of the user are collected to obtain a nonlinear transform function for transforming the normalized pressure values, the mean value for the normalized first axial direction, the mean value for the normalized second axial direction, and the normalized pressure values, ≪ / RTI > An apparatus for determining a position on a virtual space indicated by an input of pressure information of a user,
A measuring unit that obtains pressure information input by the user using a plurality of pressure measurement sensors included in a user interface; And
And an intention analysis unit for determining position information on the virtual space based on the distribution of the pressure information,
Wherein the arrangement of the plurality of pressure measurement sensors comprises an arrangement in a 2D plane or 3D space that can be approximated by a matrix arrangement or matrix arrangement,
Wherein the intention analysis unit comprises:
Determining a pressure distribution function based on pressure information obtained from the plurality of pressure measurement sensors arranged to obtain a probability distribution function obtained by dividing the pressure distribution function by a total sum of pressure distributions, And determines location information in the virtual space. delete The method according to claim 6,
Wherein the intention analysis unit comprises:
In order to determine position information on the virtual space based on the probability distribution function, when the matrix or an arrangement of a shape approximating the matrix is located in a plane consisting of a first axis and a second axis orthogonal thereto , Obtains an average value in the first axis direction and the second axis direction from the probability distribution function, estimates a pressure value in an average value in the first axis direction and the second axis direction, Direction, the average value in the second axial direction, and the pressure value in the average value in the first axial direction and the second axial direction, and changes the normalized values so that the 3D position information Lt; RTI ID = 0.0 > 1, < / RTI > 9. The method of claim 8,
Wherein the intention analysis unit comprises:
The method of claim 1, wherein the step of converting the normalized values to a range of 3D position information that the user desires to change comprises: calculating an average value for the normalized first axis direction, an average value for the normalized second axis direction, Lt; RTI ID = 0.0 > 1, < / RTI > 9. The method of claim 8,
Wherein the intention analysis unit comprises:
The method of claim 1, wherein the step of converting the normalized values to a range of 3D position information that the user desires to change comprises: calculating an average value for the normalized first axis direction, an average value for the normalized second axis direction, Collecting input values of a user to obtain a nonlinear transform function for transforming values, and learning by a method of machine learning. A computer-readable recording medium storing a computer program for executing the method according to any one of claims 1 to 5.

Images