KR101338958B1 - system and method for moving virtual object tridimentionally in multi touchable terminal - Google Patents

Abstract

The present invention relates to a system and method for manipulating three-dimensional movement of a virtual object displayed on a screen of a terminal by a multi-touch operation in a multi-touch terminal in which augmented reality is implemented.
In the multi-touch terminal according to the present invention, a three-dimensional motion manipulation system of a virtual object includes a touch sensing unit for sensing a plurality of touch point information and drag gesture information of the touch point, and a screen displaying a virtual object on a screen. A three-dimensional motion manipulation system of a virtual object in a multi-touch terminal, comprising: a posture estimating unit for estimating three-dimensional pose information of a virtual camera and three-dimensional pose information of a virtual object; From the at least one touch point information input through the touch sensing unit generates a projection line passing through the virtual object perpendicular to the screen plane of the virtual camera, and detects the drag gesture information of the plurality of touch points to determine the virtual object. The position movement amount in the two-axis direction and the projection line axis direction parallel to the plane of the virtual camera screen is calculated, and the three-dimensional posture information of the virtual object is changed to reflect the position movement amount in the three axis direction. A motion recognition module for calculating a posture; And a rendering engine configured to render an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object calculated by the motion recognition module.

Description

System and method for 3D motion manipulation of virtual objects in multi-touch terminals {system and method for moving virtual object tridimentionally in multi touchable terminal}

The present invention relates to a system and method for augmented reality in a multi-touch terminal, and more particularly, to a system for manipulating three-dimensional movement of a virtual object displayed on a screen of a terminal by multi-touch in a multi-touch terminal in which augmented reality is implemented. And to a method.

Virtual Reality is a type of user interface technology that creates a specific environment or situation using a computer software program and makes the user interact with the real environment or environment. Augmented Reality is a field of virtual reality. It is a computer graphics technique that synthesizes a virtual object into a real image image to make the virtual object look like an object in the real environment.

Virtual reality targets only virtual spaces and objects, whereas augmented reality can provide virtual users with additional information that is difficult to obtain only by real world by synthesizing virtual content on the basis of real world. In addition, the virtual reality technology displays the virtual environment and the virtual object on the screen of the terminal to allow the user to immerse in the virtual environment, so that the user cannot see the real environment, while in the augmented reality technology, the user can see the real environment. Real and virtual objects are displayed on the screen in a mixed form.

When a virtual reality or augmented reality is implemented in any terminal, the virtual screen is displayed on the terminal screen in computer graphics. In the virtual reality or augmented reality, a virtual camera exists in the virtual space in order to display the virtual environment or the real environment and the virtual object. The relative poses of the virtual camera and the virtual object are defined by a total of 6D information of 3D position information and 3D angle information. That is, when the six-dimensional pose of the virtual camera or the six-dimensional pose of the virtual object is changed, the image displayed on the screen of the terminal is changed.

That is, when the posture (three-dimensional position and three-dimensional angle) of the virtual camera is changed while the virtual background and the posture (three-dimensional position and three-dimensional angle) of the virtual object are fixed, the viewpoint at which the virtual camera looks at the virtual object. The view point changes to change the image displayed on the screen. On the other hand, if the posture of the virtual object is changed while the posture of the virtual camera is fixed, the virtual background is fixed but according to the distance between the virtual object and the virtual camera, the relative position between the virtual object and the virtual camera, and the rotation angle of the virtual object. The image displayed on the screen changes.

In a conventional technique of moving a virtual object displayed on a terminal screen in a three-dimensional direction, a marker is mounted on a user's finger, and a camera mounted on the terminal tracks the movement of the finger marker, and in conjunction with the movement of the tracked finger marker. There is a technique for manipulating three-dimensional movement of a virtual object. However, the conventional three-dimensional motion manipulation technology of the virtual object has a problem in that it is inconvenient to use because the user must attach a separate color marker on the finger.

An object of the present invention devised to solve the above-described problems of the prior art is to operate a three-dimensional movement of a virtual object displayed on a screen screen only by a user operating a touch screen or a touch pad of a terminal without any additional equipment. The present invention provides a three-dimensional motion manipulation system and method for a virtual object in a multi-touch terminal.

In the multi-touch terminal according to an embodiment of the present invention for achieving the above object, the three-dimensional motion control system of the virtual object, a touch sensing unit for detecting a plurality of touch point information and the drag gesture information of the touch point; Claims [1] A three-dimensional motion manipulation system of a virtual object in a multi-touch terminal including a screen for displaying a virtual object, comprising: a posture estimator for estimating three-dimensional attitude information of a virtual camera and three-dimensional attitude information of the virtual object; From the at least one touch point information input through the touch sensing unit generates a projection line passing through the virtual object perpendicular to the screen plane of the virtual camera, and detects the drag gesture information of the plurality of touch points to determine the virtual object. The position movement amount in the two-axis direction and the projection line axis direction parallel to the plane of the virtual camera screen is calculated, and the three-dimensional posture information of the virtual object is changed to reflect the position movement amount in the three axis direction. A motion recognition module for calculating a posture; And a rendering engine for rendering an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object calculated by the motion recognition module.

In the multi-touch terminal according to an embodiment of the present invention, a three-dimensional motion manipulation method of a virtual object includes a touch sensing unit for sensing a plurality of touch point information and drag gesture information of the touch point, and displaying a virtual object on a screen. A method for manipulating three-dimensional motion of a virtual object in a multi-touch terminal including a screen, comprising: a posture estimating step of estimating three-dimensional attitude information of the virtual camera and three-dimensional attitude information of the virtual object; The motion recognition module generates a projection line passing through the virtual object while being perpendicular to the screen plane of the virtual camera from at least one touch point information input through the touch sensing unit, and detecting drag gesture information of the plurality of touch points. The virtual object is calculated in two axis directions parallel to the virtual camera screen plane and in the projection line axis direction, and the virtual object is reflected by reflecting the position movement amount in the three axis direction in the three-dimensional attitude information of the virtual object. A motion recognition step of calculating a changed three-dimensional posture of the; And a rendering engine renders an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object.

As described above, according to the present invention, since the user can manipulate the three-dimensional movement of the virtual object displayed on the terminal screen without any additional equipment, the user's convenience is increased.

1 is a block diagram illustrating a multi-touch terminal to which a three-dimensional motion manipulation system of a virtual object according to the present invention is applied.
2 is a diagram illustrating a process of selecting a virtual object and generating projection lines.
3 is a diagram illustrating a process of manipulating a selected virtual object in a projection line direction.
4 is a flowchart illustrating a three-dimensional motion manipulation method of a virtual object in the multi-touch terminal configured as described above.

Hereinafter, a 3D motion manipulation system and method of a virtual object in a multi-touch terminal according to an embodiment of the present invention will be described in detail.

Here, multi-touch refers to a technology in which a touch screen or a touch pad recognizes multiple touch points at the same time. In the specification of the present invention, multi-touch terminal refers to a terminal in which multi-touch technology is implemented. Multi-touch terminals include iPhones, MacBooks, including smartphones, tablet PCs, and notebooks.

1 is a block diagram illustrating a multi-touch terminal to which a three-dimensional motion manipulation system of a virtual object according to the present invention is applied.

Typically, the multi-touch terminal includes a touch sensing unit 101, a screen 102, a gyroscope sensor 103, and a camera 104. The touch sensing unit 101 and the screen 102 may be integrally configured (touch screen) or may be separately configured (touch pad). The touch sensing unit 101 receives and recognizes a multi-touch gesture of the user. That is, the touch sensing unit 101 detects a press and drag gesture by the user. The screen 102 displays the virtual object according to the present invention on the screen. The gyroscope sensor 103 obtains 3D attitude information of the multi-touch terminal, and the camera 104 obtains image data. The multi-touch terminal may further include a gravity sensor, a geomagnetic sensor, and an acceleration sensor.

The present invention can be applied to virtual reality technology or augmented reality technology, virtual reality is a computer graphics technique consisting of a virtual background and a virtual object, augmented reality is a computer graphics technique consisting of a real background and a virtual object. The present invention estimates the posture information of the virtual camera and the posture information of the virtual object using sensors mounted on the terminal or the photographed image of the camera, generates a projection line between the virtual camera and the virtual object by detecting a user's multi-touch gesture. The position of the virtual object is moved in the three-dimensional direction based on the generated projection line. The present invention can be applied to manipulating three-dimensional rotation of a virtual object in virtual reality or augmented reality.

The virtual object three-dimensional motion manipulation unit 110 according to the present invention, using the data input from at least one of the gyroscope sensor 103 and the camera 104, the three-dimensional pose information of the virtual camera and the three-dimensional pose of the virtual object. The three-dimensional movement (position movement) of the virtual object is calculated based on the posture estimating unit 111 for estimating the information and the user's touch information input from the touch sensing unit 101, and the three-dimensional attitude information of the virtual object. And a motion recognition module 112 for calculating a changed three-dimensional posture of the virtual object by reflecting the three-dimensional motion of the virtual object, and the changed three-dimensional posture information of the virtual object calculated by the motion recognition module 112; The rendering engine 113 renders an image of the virtual object by reflecting the 3D posture information of the virtual camera, and the image of the virtual object rendered by the rendering engine 113. It includes an image combining unit 114 to display in the lean (102).

Posture Estimation (111)

As a first embodiment of the posture estimating unit 111, the posture estimating unit 111 calculates a posture of a terminal using data input from the gyroscope sensor 103, and posture information of the virtual camera based on the posture of the terminal. Estimate

As a second embodiment of the posture estimating unit 111, the posture estimating unit 111 recognizes a specific real object from the photographed image input from the camera 104 and based on the relative posture information between the recognized real object and the camera. The posture information of the virtual camera and the posture information of the virtual object are estimated.

Typically, the posture of the terminal or the camera calculated through the input data of the gyroscope sensor 103 or the camera 104 is estimated as the posture of the virtual camera.

A first embodiment of the posture estimation unit 111 will be described in detail.

The first embodiment of the posture estimation unit 111 may be applied to a virtual reality technology. In the virtual reality environment, all the information about the virtual background and the virtual object are preset and stored on the virtual reality program. Alternatively, the relationship between the posture of the virtual camera and the posture of the virtual object may be preset in the virtual reality program. The posture estimator 111 calculates the posture of the terminal using the data input from the gyroscope sensor 103, estimates the posture information of the virtual camera based on the calculated posture of the terminal, and the virtual reality in the virtual reality environment. The posture information of the virtual object is estimated from the data set on the program.

In a typical virtual reality, a three-dimensional virtual object is located in a three-dimensional virtual environment. The terminal attitude value when the virtual reality program is initially executed is set as a reference attitude, and the initial screen of the virtual reality is displayed on the screen. After that, when the user moves the terminal (3D position movement or 3D rotation), the 3D virtual background and the 3D virtual object rendered and displayed on the screen according to the difference between the initial posture value and the current posture value of the terminal. The image is different. That is, when the posture of the terminal is changed, the appearance of viewing the virtual object at different angles or at different positions according to the posture of the terminal is displayed on the screen.

The posture estimator 111 estimates the posture information of the virtual camera based on the posture information of the terminal, and also estimates the posture information of the virtual object in the virtual environment. The posture estimation unit 111 obtains posture information of the terminal from an output value of the gyroscope sensor 103. The gyroscope sensor 103 outputs six-dimensional attitude information (three-dimensional position information and three-dimensional angle information) in which the current terminal is moved from the reference pose. Here, the reference posture may be determined by the terminal itself or set by the user (for example, the posture of the terminal when the virtual reality program is first executed). The posture estimator 111 further acquires the posture information of the terminal by using at least one output value of the acceleration sensor, the output value of the gravity sensor, and the output value of the geomagnetic sensor together with the output value of the gyroscope sensor 103. The posture information of the virtual camera may be estimated using the posture information of the terminal. A method of acquiring terminal posture information and posture information of a virtual camera using output values of a hardware sensor including a gyroscope sensor 103 is a conventional technique, and a detailed description thereof will be omitted.

A second embodiment of the posture estimation unit 111 will be described in detail.

The second embodiment of the posture estimation 111 may be applied to augmented reality technology. Augmented reality, unlike virtual reality, recognizes a specific real object and augments the virtual object in the real environment based on the recognized real object. For example, a specific real object may be recognized, and the virtual object may be augmented by the position and angle of the recognized real object, or the virtual object may be augmented by a predetermined distance from the recognized real object. In this case, it is preferable that the relation between the posture of the recognized physical object and the posture of the virtual object is set in advance on the AR program.

As described above, the terminal recognizes a specific real object and augments the virtual object in the real environment based on the recognized real object. To this end, feature point information on a real object to be recognized is stored in advance, and a feature point is extracted from a photographed image of the camera, and the extracted feature point and pre-stored feature point information are matched to recognize a real object to be recognized. The posture information of the posture information and the posture information of the augmented virtual object is estimated.

For example, when the real object is recognized, the position of the recognized real object is set to the origin (0,0,0) of the 3D virtual space, and the position information of the virtual camera is obtained by obtaining the position information of the virtual camera with respect to the origin. Estimate. In addition, the posture information of the virtual object is estimated based on the relative position information between the real object and the virtual object preset in the augmented reality program.

Motion recognition module 112

The motion recognition module 112 calculates a three-dimensional movement (position movement) of the virtual object displayed on the screen 102 based on the user's multi-touch gesture detected from the touch sensing unit 101, and the calculated three-dimensional movement is calculated. The 3D posture of the changed virtual object is calculated and output to the rendering engine 113.

The motion recognition module 112 is composed of two embodiments. The first embodiment is named in a sequential manner, and the second embodiment is named in a synchronous manner.

In the sequential method, a user selects a virtual object to be moved to one touch point (first touch point), and then adds another touch point (second touch point) to simultaneously move the first touch point and the second touch point. By adjusting the distance between the first touch point and the second touch point, the selected virtual object is moved three-dimensionally in the virtual environment. Simultaneously, the user selects the virtual object with two touch points (first touch point and second touch point) (selects the virtual object as the center point between the two touch points), and then the first touch point and the second touch point. By moving the position at the same time or by adjusting the distance between the first touch point and the second touch point, the virtual object in the virtual environment to move the three-dimensional position.

(i) sequential

A process of selecting a virtual object and generating a projection line will be described with reference to FIG. 2.

When the user presses any point of the touch sensing unit, the pressing point is designated as the first touch point. The motion recognition module 112 converts the first touch point into the three-dimensional projection point 22 on the virtual camera screen plane (the same plane as the virtual camera screen) 21, and from the three-dimensional projection point 22 to the virtual camera screen. The normal vector 23 is generated in the front direction of the plane. The virtual object where the normal vector 23 meets for the first time is set as the selection virtual object 24. That is, the virtual object nearest to the virtual camera is set as the selected virtual object, and the normal vector 23 is defined as the projection line.

In this way, after selecting the virtual object and generating the projection line, the second touch point is sensed to move the selected virtual object in three dimensions. To move the virtual object in three dimensions means to move to an arbitrary position with respect to three axes perpendicular to each other. Here, it is assumed that two axes are axes passing through the selected virtual object while being parallel to the virtual camera screen plane, and the other axis is perpendicular to the virtual camera screen plane and passing through the selected virtual object.

Manipulate movement in two axes parallel to the virtual camera screen plane and across the selected virtual object

When a drag gesture is performed while the first touch point and the second touch point maintain the same distance and angle at the same time, the motion recognition module 112 performs the drag path along the drag paths of the first and second touch points. The selected virtual object is moved in two axes parallel to the virtual camera screen plane, and the modified three-dimensional posture of the selected virtual object reflecting the movement of the position is calculated and provided to the rendering engine 113. In this way, when the selected virtual object is moved in the two axis direction, the projection point and the projection line are also moved.

Operation in one axis (projection line direction) passing through the selected virtual object while being perpendicular to the virtual camera screen plane

3 is a diagram illustrating a process of moving a selected virtual object in a projection line direction, and FIG. 3A illustrates a process of selecting a virtual object and moving the position in the projection line direction in a sequential manner. b) shows the process of selecting a virtual object and moving it in the direction of the projection line in a simultaneous manner.

As shown in FIG. 3A, when a drag gesture is performed in a direction in which the distance between the first touch point and the second touch point is changed (the first touch point is fixed, the second touch point is fixed). A drag gesture only, although not shown in the drawing, a drag gesture may be performed in a direction in which the first touch point and the second touch point are opposite to each other as shown in FIG. The motion recognition module 112 calculates the distance change amount between the first touch point and the second touch point, moves the selected virtual object in the direction of the projection line to reflect the distance change amount, and reflects the position change. The modified three-dimensional pose of the selected virtual object is calculated and provided to the rendering engine 113. When the selected virtual object is closer to the virtual camera, the size of the selected virtual object displayed on the screen becomes larger (virtual or real background is the same), and when the selected virtual object moves away from the virtual camera, the size of the selected virtual object displayed on the screen becomes smaller. (The background is the same).

(ii) simultaneous

When the user presses any two points as shown in (b) of FIG. 3, the pressing points are designated as the first touch point and the second touch point, respectively. The motion recognition module 112 converts the center points of the first touch point and the second touch point into three-dimensional projection points 22 on the virtual camera screen plane, as shown in FIG. 2, and from the three-dimensional projection points 22. The normal vector 23 is generated in the front direction of the virtual camera screen plane. The virtual object where the normal vector 23 meets for the first time is set as the selection virtual object 24. That is, the virtual object nearest to the virtual camera is set as the selected virtual object, and the normal vector 23 is defined as the projection line.

In this way, after selecting a virtual object and generating projection lines, the selected virtual object is moved in three dimensions. To move the virtual object in three dimensions means to move to an arbitrary position with respect to three axes perpendicular to each other. Here, it is assumed that two axes are axes passing through the selected virtual object while being parallel to the virtual camera screen plane, and the other one axis is an axis (projection line) perpendicular to the virtual camera screen plane and passing through the selected virtual object.

Manipulate movement in two axes parallel to the virtual camera screen plane and across the selected virtual object

When a drag gesture is performed while the first touch point and the second touch point maintain the same distance and angle at the same time, the motion recognition module 112 performs the drag path along the drag paths of the first and second touch points. The selected virtual object is moved in two axes parallel to the virtual camera screen plane, and the modified three-dimensional posture of the selected virtual object reflecting the movement of the position is calculated and provided to the rendering engine 113. When the selected virtual object is moved in two axes, the projection point and the projection line are also moved. As described above, the method of moving the selected virtual object in the two axis directions parallel to the virtual camera screen plane is the same in both the sequential method and the simultaneous method.

Operation in one axis (projection line direction) passing through the selected virtual object while being perpendicular to the virtual camera screen plane

As shown in (b) of FIG. 3, when a drag gesture is performed in a direction in which a distance between the first touch point and the second touch point is changed (the first touch point and the second touch point are opposite to each other) A drag gesture in a direction, although not shown in the drawing, as shown in FIG. 3A, the first touch point may be a drag gesture only while the second touch point is fixed), The motion recognition module 112 calculates the distance change amount between the first touch point and the second touch point, moves the selected virtual object in the direction of the projection line to reflect the distance change amount, and reflects the position change. The modified three-dimensional pose of the selected virtual object is calculated and provided to the rendering engine 113. When the selected virtual object is closer to the virtual camera, the size of the selected virtual object displayed on the screen becomes larger (virtual or real background is the same), and when the selected virtual object moves away from the virtual camera, the size of the selected virtual object displayed on the screen becomes smaller. (The background is the same).

When the selected virtual object is moved in the direction of the projection line, whether in the sequential or simultaneous manner, as shown in FIG. 3 (c), the virtual object is close to the virtual camera along the projection line or is not shown in the drawing. The object moves away from the virtual camera along the projection line.

4 is a flowchart illustrating a three-dimensional motion manipulation method of a virtual object in the multi-touch terminal configured as described above.

The posture estimator estimates the three-dimensional pose of the virtual camera and the three-dimensional pose of the virtual object (S41). The posture estimator estimates the three-dimensional pose of the virtual camera and the virtual object by using the output value of the gyroscope sensor, estimates the three-dimensional pose of the virtual camera and the virtual object by using the captured image of the camera, or outputs the output value of the gyroscope sensor and The 3D pose of the virtual camera and the virtual object may be estimated using the captured image of the camera.

Next, the motion recognition module selects a virtual object by recognizing a user's touch point (S42), and generates a projection line passing through the virtual object while being perpendicular to the screen plane of the virtual camera (S43). The motion recognition module recognizes the virtual object and generates projection lines are divided into sequential and simultaneous methods, and the detailed description of each method is as described above.

Next, the motion recognition module recognizes a user's touch gesture (S44) and calculates a three-dimensional position movement amount with respect to the selected virtual object (S45). When the drag gesture is performed at the same time while the first touch point and the second touch point maintain the same distance and angle, the motion recognition module moves the selected virtual object along the drag path of the first and second touch points. Move in a two-axis direction parallel to the virtual camera screen plane. Meanwhile, when a drag gesture is performed in a direction in which the distance between the first touch point and the second touch point is changed, the motion recognition module calculates a distance change amount between the first touch point and the second touch point, The selected virtual object is moved in the direction of the projection line in consideration of the distance change amount.

Next, the motion recognition module calculates the changed three-dimensional pose by reflecting the three-dimensional position movement amount calculated in the step S45 to the three-dimensional pose information of the selected virtual object estimated in step S41 (S46), and changes the modified three-dimensional pose for the selected virtual object. The dimensional pose is rendered to be displayed on the screen (S47).

The technical spirit of the present invention has been described above with reference to the accompanying drawings, but this is by way of example only for describing the best embodiment of the present invention and not for limiting the present invention. In addition, it is a matter of course that various modifications and variations are possible without departing from the scope of the technical idea of the present invention by anyone having ordinary skill in the art.

101: touch detection unit 102: screen
103: gyroscope sensor 104: camera
110: virtual object three-dimensional motion control unit 111: posture estimation
112: motion recognition module 113: rendering engine
114: image synthesis unit

Claims (16)

In the three-dimensional motion manipulation system of a virtual object in a multi-touch terminal comprising a plurality of touch point information and a touch sensing unit for detecting the drag gesture information of the touch point, and a screen for displaying the virtual object on the screen,
A posture estimation unit for estimating three-dimensional pose information of the virtual camera and three-dimensional pose information of the virtual object;
From the at least one touch point information input through the touch sensing unit generates a projection line passing through the virtual object perpendicular to the screen plane of the virtual camera, and detects the drag gesture information of the plurality of touch points to determine the virtual object. The position movement amount in the two-axis direction and the projection line axis direction parallel to the plane of the virtual camera screen is calculated, and the three-dimensional posture information of the virtual object is changed to reflect the position movement amount in the three axis direction. A motion recognition module for calculating a posture;
And a rendering engine for rendering an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object calculated by the motion recognition module.
The motion recognition module may designate a first touch point of the virtual camera corresponding to the first point when a push gesture is generated at the first point of the touch sensing unit, and the screen plane of the virtual camera may be determined from the first touch point. Generating a projection line passing through the virtual object while being perpendicular to, and designating a second touch point of the virtual camera corresponding to the second point when a push gesture occurs at a second point of the touch sensing unit; 3D motion control system of virtual object in touch terminal. In the three-dimensional motion manipulation system of a virtual object in a multi-touch terminal comprising a plurality of touch point information and a touch sensing unit for detecting the drag gesture information of the touch point, and a screen for displaying the virtual object on the screen,
A posture estimation unit for estimating three-dimensional pose information of the virtual camera and three-dimensional pose information of the virtual object;
From the at least one touch point information input through the touch sensing unit generates a projection line passing through the virtual object perpendicular to the screen plane of the virtual camera, and detects the drag gesture information of the plurality of touch points to determine the virtual object. The position movement amount in the two-axis direction and the projection line axis direction parallel to the plane of the virtual camera screen is calculated, and the three-dimensional posture information of the virtual object is changed to reflect the position movement amount in the three axis direction. A motion recognition module for calculating a posture;
And a rendering engine for rendering an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object calculated by the motion recognition module.
The motion recognition module may designate a first touch point of the virtual camera corresponding to the first point when the push gesture occurs at the first point and the second point of the touch sensing unit, and correspond to the virtual point in response to the second point. A second touch point of the camera is designated, and the projection line passing through the virtual object perpendicular to the screen plane of the virtual camera from the center point of the first touch point and the second touch point is generated. 3D motion control system of virtual object. The method of claim 1 or 2, wherein the posture estimating unit obtains posture information of the multi-touch terminal from an output value of a gyroscope sensor, and estimates a three-dimensional posture of a virtual camera from the acquired posture information of the multi-touch terminal. 3D motion control system of the virtual object in the multi-touch terminal, characterized in that. The method of claim 1 or 2, wherein the posture estimating unit recognizes a specific real object from a photographed image of the camera and obtains position information relative to the recognized real object to estimate a three-dimensional pose of the virtual camera. 3D motion control system of a virtual object in a multi-touch terminal. delete According to claim 1 or 2, The motion recognition module, When the first and the second touch point while the drag gesture is performed at the same time while maintaining the same distance and angle, the drag of the first and second touch points 3D motion manipulation system of the virtual object in the multi-touch terminal, characterized in that for calculating the position movement amount in the two-axis direction parallel to the screen plane of the virtual camera along the path. According to claim 1 or 2, The motion recognition module, When one of the first touch point and the second touch point is fixed and the rest of the drag gesture is performed, the first touch point and the second touch point 3D movement manipulation system of the virtual object in the multi-touch terminal, characterized in that for calculating the position movement amount of the virtual object in the direction of the projection line axis in accordance with the distance change. The method of claim 1, wherein the motion recognition module changes a distance between the first touch point and the second touch point when the first touch point and the second touch point are dragged in opposite directions. And calculating the positional movement amount of the virtual object in the axial direction of the projection line according to the present invention. In the multi-touch terminal three-dimensional movement operation method comprising a touch sensing unit for detecting a plurality of touch point information and the drag gesture information of the touch point, and a screen for displaying the virtual object on the screen,
A posture estimation step of the posture estimating step estimating the three-dimensional pose information of the virtual camera and the three-dimensional pose information of the virtual object;
The motion recognition module generates a projection line passing through the virtual object while being perpendicular to the screen plane of the virtual camera from at least one touch point information input through the touch sensing unit, and detecting drag gesture information of the plurality of touch points. The virtual object is calculated in two axis directions parallel to the virtual camera screen plane and in the projection line axis direction, and the virtual object is reflected by reflecting the position movement amount in the three axis direction in the three-dimensional attitude information of the virtual object. A motion recognition step of calculating a changed three-dimensional posture of the;
A rendering engine renders an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object,
In the motion recognition step, when a push gesture occurs at a first point of the touch sensing unit, a first touch point of the virtual camera is designated corresponding to the first point, and the screen plane of the virtual camera is determined from the first touch point. Generating a projection line passing through the virtual object while being perpendicular to, and designating a second touch point of the virtual camera corresponding to the second point when a push gesture occurs at a second point of the touch sensing unit; 3D motion manipulation method of a virtual object in a touch terminal. In the multi-touch terminal three-dimensional movement operation method comprising a touch sensing unit for detecting a plurality of touch point information and the drag gesture information of the touch point, and a screen for displaying the virtual object on the screen,
A posture estimation step of the posture estimating step estimating the three-dimensional pose information of the virtual camera and the three-dimensional pose information of the virtual object;
The motion recognition module generates a projection line passing through the virtual object while being perpendicular to the screen plane of the virtual camera from at least one touch point information input through the touch sensing unit, and detecting drag gesture information of the plurality of touch points. The virtual object is calculated in two axis directions parallel to the virtual camera screen plane and in the projection line axis direction, and the virtual object is reflected by reflecting the position movement amount in the three axis direction in the three-dimensional attitude information of the virtual object. A motion recognition step of calculating a changed three-dimensional posture of the;
A rendering engine renders an image of the virtual object by reflecting the changed three-dimensional posture of the virtual object,
In the motion recognition step, when a push gesture occurs at a first point and a second point of the touch sensing unit, a first touch point of the virtual camera is designated corresponding to the first point and the virtual point corresponding to the second point. A second touch point of the camera is designated, and the projection line passing through the virtual object perpendicular to the screen plane of the virtual camera from the center point of the first touch point and the second touch point is generated. 3D motion manipulation method of virtual object. The posture estimation step of claim 9 or 10, wherein the attitude estimation step obtains attitude information of the multi-touch terminal from an output value of a gyroscope sensor and estimates a three-dimensional attitude of the virtual camera from the acquired attitude information of the multi-touch terminal. 3D motion control method of the virtual object in the multi-touch terminal, characterized in that. The method of claim 9 or 10, wherein the posture estimating step recognizes a specific real object from a photographed image of the camera and obtains position information relative to the recognized real object to estimate the three-dimensional pose of the virtual camera. 3D motion manipulation method of a virtual object in a multi-touch terminal characterized in that. delete The method of claim 9 or 10, wherein in the motion recognition step, when the first gesture and the second touch point while the drag gesture is performed at the same time while maintaining the same distance and angle, the drag of the first and second touch points And calculating a position movement amount of the virtual object in a two-axis direction parallel to the screen plane of the virtual camera along a path. The method of claim 9 or 10, wherein in the motion recognition step, when one of the first touch point and the second touch point is fixed and the dragging gesture is performed, the first touch point and the second touch point are each different. And calculating a position movement amount of the virtual object in the projection line axial direction according to a distance change. The method of claim 9 or 10, wherein in the movement recognition step, when the first and second touch points are dragged in opposite directions, the distance between the first touch point and the second touch point is changed. And calculating a positional movement amount of the virtual object in the projection line axial direction according to the method.

Images