KR20120106294A - An haptic apparatus and method of controlling thereof - Google Patents

Abstract

PURPOSE: A haptic device and a control method thereof having a grip function are provided to simplify a structure of a haptic device and to miniaturize a haptic device. CONSTITUTION: A grip unit(100) applies external shock to an object. A marker(101) is included in one side of a grip unit. A multiple moving operator(300) approaches the object. A rotation joint unit(102) couples the grip unit with the multiple moving operator and supplies rotation motion to the grip unit. An image photographing unit(200) photographs the grip unit, the marker, and the rotation joint unit. A processor(400) calculates phase information about the grip unit from image information. [Reference numerals] (100) Grip; (101) Marker; (102) Rotation joint unit; (200) Image photographing unit; (300) Multiple moving operator; (400) Processor

Description

Haptic Apparatus and method of controlling Technical Field

The present invention relates to a haptic device, to a method for measuring the position of the grip provided in the haptic device, and to a haptic device supporting the same.

Hereinafter, description will be briefly given of techniques generally used in the technical field of the present invention.

As technology advances, users want more concrete and realistic information through virtual reality. To meet this, recently developed technology is a haptic technology that provides information about virtual touch and force.

Building a simulator using haptic technology requires haptic devices, haptic rendering, and computer graphics technology. Haptic devices are mechanical devices like robotic manipulators designed to interact with humans and virtual environments. The human can give a control command to the virtual environment through the haptic device, and can feel the touch or power coming from the virtual environment according to the control command.

The user mainly manipulates the stylus at the end of the haptic device to give the virtual environment six degrees of freedom (e.g., X, Y, Z position, X, Y, Z phase angle) and simultaneously The force generated through interaction from the virtual environment is felt through the haptic device. The haptic technology is applied to a surgical simulator, a game using a virtual environment, and a remote control technology.

Haptic technology is largely divided into two areas: force feedback and tactile feedback. Force feedback is a technique that uses a mechanical interface to give users a sense of power and movement. For example, shooting a game machine actually shakes the joystick, and force feedback makes it feel at your fingertips when the mouse cursor touches a button on the web. The most common use of tactile feedback is in the medical field. Using haptic technology, a user can directly perform a lesion on a virtual patient while viewing a three-dimensional anatomical structure on a computer screen. In addition, by using a mechanical driver such as a small pin that moves with compressed air or electricity, the tactile sense of touch of the skin tissue is transferred to create an environment almost similar to the actual situation.

When haptic technology is applied to the medical field, it is very important to precisely control the movement of the haptic device because even the minute movement of the haptic device can greatly affect the outcome of the surgery.

In general, a method for measuring the phase of the end of the haptic device is a rotary variable resistor that uses an optical rotary encoder that outputs a square wave according to the degree of rotation, or a voltage in the form of a voltage. There is a way to use.

An optical rotary encoder consists of a light emitting diode and a light receiving transistor that emit light when a power is applied, and a plate that blocks light on the rotation axis is positioned to continuously block and / or release the light emitted from the light emitting diode to form a square wave from the light receiving transistor. (Square wave) means a device that outputs a signal in the form of. At this time, the optical rotary encoder can reversely calculate the rotation angle by counting the square wave signal. In addition, by installing the light emitting diode and the light receiving transistor in the form of crossing two pairs in the optical rotary encoder, the direction of rotation of the haptic device can also be known depending on which signal comes first.

A haptic device using a rotary variable resistor refers to a device having a switch structure in which contact portions of the resistor change according to the degree of rotation of the haptic device. The haptic device increases resistance as power is applied and rotation is increased, and an output voltage value is increased by a voltage divider rule. Therefore, the degree of rotation of the haptic device can be determined by detecting the output voltage value.

These two haptic devices are often referred to as mechanical sensors because they convert mechanically applied rotations into quantified output values. Such a device is widely used because it can quantitatively measure the rotation angle of a multi-degree of freedom actuator such as a haptic device.

However, when using such a mechanical sensor, the use of the haptic device can be cumbersome. For example, free rotation is constrained by power lines and signal lines entering the haptic device, and the behavior of the haptic device may be restricted by power lines and / or signal lines connected to various parts of the haptic device.

In addition, the haptic device is provided with a grip for applying various external forces to the object. In this case, in the case of a surgical simulator in which the type of grip is to be variously changed, an error-free operation may be performed by accurately measuring the position and phase of each grip.

However, in the case of a haptic device using a mechanical sensor, the grip is measured when the surgical scissors are used as the grip, or when the syringe is used as the grip, the grip of the syringe is pulled out. There is a problem that can not pinpoint the location of.

In addition, when using a mechanical sensor, there is a disadvantage in that the power line and the signal line of the mechanical sensor must be connected at every operation.

In this case, the user using the haptic device is a cumbersome task, and since the behavior of the haptic device is restricted due to the power line and the signal line for using the mechanical sensor, there is a disadvantage that it cannot be applied to various operations.

The present invention has been devised to solve the disadvantages of the general technique described above, and an object of the present invention is to provide a method for precisely measuring the position of the grip of the haptic device.

Another object of the present invention is to provide a haptic device for accurately tracking the operation of the grip wirelessly using a vision camera.

It is another object of the present invention to provide a haptic device capable of replacing and attaching various types of grips.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

The present invention discloses various methods for measuring the position of the grip provided in the haptic device and haptic devices supporting the same.

In one embodiment of the present invention, a haptic device using image information includes a grip for applying an external stimulus to a target object, a multi-degree of freedom actuator for accessing a marker and a grip provided at one end of the grip to the target object, and a grip and multiple freedom. It may include a rotary joint unit for coupling the actuator and providing a rotational motion of three degrees of freedom to the grip, and an imaging unit for capturing image information by taking one or more of the grip, marker and the rotary joint.

In this case, the haptic device may further include a processor for controlling the multi-degree of freedom actuator, the rotating joint, the grip and the image capturing unit. The processor may control the image capturing unit to obtain image information when an external stimulus is applied to the target object, and derive phase information about the grip from the image information.

The phase information on the grip may be derived using the three-dimensional rotation angle of the marker derived from the coordinate information of the marker, the coordinate information of the rotating joint part, and the image information.

In addition, the marker may include a light emitting diode (LED) that emits infrared rays, and the image capturing unit may include an infrared camera system capable of detecting infrared rays.

According to another aspect of the present invention, a haptic device using image information includes a grip for applying an external stimulus to a target object, a multi-degree of freedom actuator, a grip, and a multiple freedom for approaching a target object with a marker and a grip provided at one end of the grip. It may include a rotary joint unit for coupling the degree actuator, and provides a rotational motion of three degrees of freedom to the grip, and an image photographing unit for capturing one or more of the grip, the marker and the rotary joint portion to obtain image information. In this case, the image capturing unit may further include an image processing module for processing the image information to derive the phase information of the grip.

The phase information on the grip may be derived using the three-dimensional rotation angle of the marker derived from the coordinate information of the marker, the coordinate information of the rotating joint part, and the image information.

The marker may include a light emitting diode (LED) that emits infrared rays, and the image capturing unit may include an infrared camera system capable of detecting infrared rays.

As another aspect of the present invention, the haptic device using the image information is different from the grip for applying an external stimulus to the target object and the multiple degree of freedom actuator and grip for accessing the marker and the grip provided at one end of the grip to the target object. One or more encoders and grips, markers and one or more of the joints and the joints for acquiring the coordinate information of the rotating joints and markers and the coordinates of the rotating joints, which combine the degrees of freedom actuator and provide three degrees of freedom rotational motion to the grips. It may include an image capture unit for obtaining the image information. In this case, the haptic device may further include a processor for controlling the multi-degree of freedom actuator, the rotating joint, the grip, and the image capturing unit, wherein the processor controls the image capturing unit to obtain image information when an external stimulus is applied to the target object. The phase information on the grip may be derived from the image information.

The phase information on the grip may be derived using the three-dimensional rotation angle of the marker derived from the coordinate information of the marker, the coordinate information of the rotating joint part, and the image information.

According to another aspect of the present invention, a method of controlling a haptic device using image information includes controlling a multiple degree of freedom actuator to apply an external stimulus to an object, and controlling a rotation joint and a grip provided in the multiple degree of freedom actuator. Applying an external stimulus to a target object, acquiring coordinate information of the rotary joint part and coordinate information of the marker, acquiring image information of the markers provided on the rotary joint part and the grip, coordinate information of the rotary joint part, and the marker Computing the phase information of the grip using the coordinate information and the image information of the.

In this case, the grip phase information may include at least one of the coordinate information of the marker, the coordinate information of the rotating joint, the actual distance information of the marker and the rotating joint, the distance information on the image obtained from the marker and the rotating joint, and the phase information of the marker. Can be calculated using.

In addition, the rotary joint may combine the grip and the multiple degree of freedom actuator and provide the grip with three degrees of freedom of rotational motion.

According to another aspect of the present invention, a method of controlling a haptic device using image information includes obtaining coordinate information of a marker included in the haptic device, obtaining image information about the marker, and using coordinate information and image information. Comprising a step of calculating the phase information of the grip is provided with a marker. In this case, the phase information on the grip may be calculated using the 3D rotation angle of the marker derived from the coordinate information of the marker and the image information.

The three-dimensional rotation angle of the marker may be calculated using one or more of coordinate information of the marker, actual distance information of the marker and the rotating joint part, and distance information on the image of the marker and the rotating joint part obtained from the image information.

In this case, the rotary joint unit may combine a grip for applying an external stimulus to the target object and a multi-degree of freedom actuator for controlling the grip to apply the external stimulus to the target object, and provide a three degree of freedom rotational motion to the grip.

In another aspect of the present invention, a haptic device using image information includes a grip for applying an external stimulus to a target object, and a three-dimensional three-dimensional marker and a grip provided at one end of the grip, and a multiple degree of freedom actuator for approaching the target object. Combines grip and multi-degree of freedom actuators and captures encoders, grips, markers, and rotary joints to obtain coordinate information for three-dimensional solid markers and coordinates of rotary joints that provide three-degree-of-freedom rotational motion to grips. The image capturing unit may include an image capturing unit obtaining image information, and an image processing module configured to process the image information to derive the phase information of the grip.

The above aspects of the invention are only some of the preferred embodiments of the invention. Based on the detailed description of the present invention described below, various embodiments reflecting the technical features of the present invention can be derived and understood by those of ordinary skill in the art.

By applying the embodiments of the present invention to the haptic device, the user can enjoy the following effects.

First, since the mechanical sensor is not used to measure the phase of the grip, the structure of the haptic device is simplified, and the weight and size can be reduced.

Second, since the mechanical sensor is not used to measure the phase of the grip, rotation of the grip provided in the haptic device is not constrained by the power line and the signal line.

Third, when changing the type of grip according to the purpose of the haptic device, it is not necessary to release and reinstall the power line and signal line.

Fourth, various applications such as providing a more effective haptic by recognizing the shape of the user's hand and providing a more effective haptic are possible by using the image capturing unit.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a block diagram illustrating one of haptic devices using vision as an embodiment of the present invention.
2 is a diagram illustrating an example of a haptic device as an embodiment of the present invention.
3 is a view showing another example of the haptic device as an embodiment of the present invention.
4 is a view showing still another example of the haptic device as an embodiment of the present invention.
5 is a view for explaining a marker and a feature point as an embodiment of the present invention.
6 is a view showing an example of a three-dimensional marker as an embodiment of the present invention.
7 is a diagram showing an example of a method for calculating three-dimensional phase information of a grip as an embodiment of the present invention.
8 is a diagram illustrating one of methods of controlling a haptic device according to an embodiment of the present invention.

The present invention relates to a haptic device, and discloses various methods for measuring the position of a grip included in the haptic device and haptic devices supporting the same.

The following embodiments are a combination of elements and features of the present invention in a predetermined form. Each component or feature may be considered to be optional unless otherwise stated. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, some of the elements and / or features may be combined to form an embodiment of the present invention.

The order of the operations described in the embodiments of the present invention may be changed. Some configurations or features of certain embodiments may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments.

In the drawings, in order to clearly show the features of each component, some of the components are briefly displayed, and the drawings are created based on the main components. In this process, the shape of some components has been simplified and clearly clarified that each component is characterized by its main function. Therefore, the steps or components necessary for showing the gist of the present invention in the drawings are mainly shown, and the steps or components that may obscure the gist of the present invention are not shown.

In addition, the same reference numerals are attached to the same or similar parts throughout the specification. When parts such as layers, membranes, regions, and plates are 'on top' of another part, this includes not only when the other part is 'just above', but also when there is another part in between. On the contrary, when a part is 'just above' another part, there is no other part in the middle.

Specific terms used in the detailed description of the present invention are provided to aid the understanding of the present invention, and the use of the specific terms may be modified in other forms without departing from the technical spirit of the present invention.

Hereinafter, a haptic device according to the present invention will be described in detail with reference to the accompanying drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

1 is a block diagram illustrating one of haptic devices using vision as an embodiment of the present invention.

Referring to FIG. 1, the haptic device is a marker 101 used to derive position information including a grip 100 for applying an external force to a target object, coordinate information of the grip, and / or phase information. , Connecting the grip 100 and the multiple degree of freedom actuator 300 and the rotary joint 102 to support the rotational movement of the grip, the image capturing unit 200 for measuring the phase of the grip and the movement of the grip and the rotary joint It may include a multiple degree of freedom actuator 300 to support as multiple degrees of freedom. In addition, the haptic device may further include a processor 400 for controlling the operations of the multi-degree of freedom actuator, the rotating joint part and the grip according to the control information of the user, and calculating the phase information of the grip from the image information acquired through the image measuring unit. Can be.

The grip 100 may be changed according to the purpose of the haptic device, and may be any means for applying stimulation such as picking, pressing, cutting, pulling, and the like by directly contacting the target object according to the user's intention. Of course, the grip may be configured as a means for applying an external stimulus to the object without direct contact depending on the application of the haptic device. In embodiments of the present invention, the grip may be used interchangeably with terms such as a handle, a handle, or a stylus.

The marker 101 may be located at a portion of the grip and used to obtain phase information and / or coordinate information of the grip. Detailed description of the marker will be described later.

Rotating joint 102 may engage the grip at one end of the grip to provide the grip with three degrees of freedom. Thus, the rotating joint may include any structure capable of rotating in three degrees of freedom. Also, the rotatable joint can operate as a fastening portion between the grip and the multiple degree of freedom actuator. In the present invention, a gimbal structure can be used as the rotating joint. Gimbal is a rotational support that permits rotation about the front and rear and left and right axes so that equipment or equipment can be placed horizontally and vertically, regardless of the movement of the structure.

The image capturing unit 200 means a means provided at one end of the haptic device to obtain image information on the grip. In this case, the image capturing unit refers to a device which is a means capable of tracking a marker or an arbitrary feature point in the captured image. In an embodiment of the present invention, the image capturing unit is a stereo camera using two lenses using a visual principle, a camera capable of multi-degree of freedom movement, a camera using a plurality of light sources, or a distance using a time of flight (TOF). A camera capable of measuring may be used.

In addition, the image capturing unit may operate automatically or manually to detect the rotating joint connected to the grip. An example of a method of automatically detecting a rotating joint may include a pan / tilt type camera system equipped with a multiple degree of freedom rotating driver, and a method of manually detecting a joint of a handle. For example, the image capturing unit and the rotary joint of the grip may include a system located at the same link unit. The image capturing unit may include an image processing unit or an image processing apparatus capable of processing the captured image information to calculate position information such as phase information and / or coordinate information of the grip.

The multiple degree of freedom actuator 300 is a device having at least two joints to provide force feedback and / or tactile feedback to a user. The user can control the multiple degree of freedom actuator to place the grip in contact with or near the object. In addition, the user may apply a physical external force to the object by controlling the multiple degree of freedom actuator.

The processor 400 performs a function of controlling each component of the haptic device according to the control information of the user so that the user can operate the haptic device. The processor may automatically control the haptic device based on previously input information, and may include an image processor that analyzes and processes image information captured by the image capturing unit. In addition, the processor may further include a graphics card capable of processing graphic information on the surface of the object.

The image processor may acquire the coordinate information and / or the phase information of the grip by converting the image information or performing algorithm processing. However, the image processor may be included in the image photographing unit instead of the controller according to a user's request.

2 is a diagram illustrating an example of a haptic device as an embodiment of the present invention.

In FIG. 2, the multiple degree of freedom actuator 300 shows a case having two joints. One side of the multi-degree of freedom may be provided with the image capturing unit 200, the other side may be provided with a rotary joint (102). At this time, the grip 100 is coupled to the rotating joint portion to apply an external magnetic pole of multiple degrees of freedom to the target object.

The image capturing unit may be provided in a fixed form on one side of the multi-degree of freedom actuator. For example, the multiple degree of freedom can be attached to the link of the actuator to manually continue to shoot the rotary joint. In addition, an image capturing unit may be provided at a link of the multiple degree of freedom actuator to which the rotary joint unit is connected.

Alternatively, the image capturing unit may be provided at one side of the multi-degree of freedom actuator to automatically photograph the rotating joint by moving in response to the operation of the multi-degree of freedom actuator. For example, the image capturing unit may be provided in a multi-degree of freedom actuator in a form similar to a pan-tilt camera to track the movement of the rotary joint provided at the end of the multi-degree of freedom actuator according to the operation of the multi-degree of freedom actuator. Rotating joints can be photographed.

In the embodiments of the present invention, the image capturing unit measures the rotational joint part without photographing the grip or the marker itself provided with the marker as follows. Since the movement of the grip or marker varies according to the user's control, it is difficult to track the movement of the grip or the marker in the image capturing unit. However, the grip or the marker is coupled to the rotary joint and moves around the rotary joint, and since the active radius of the grip does not greatly deviate from the rotary joint, the image capturing unit is image information obtained by photographing the rotary joint, and thus, phase information or coordinate information of the marker. Can be easily obtained. That is, the rotating joint portion can be used as one reference imaging point.

The shape of the grip in FIG. 2 has the form of a stylus. The stylus has a structure in which a pen-like external stimulus can be applied to a target object by pressing or stabbing. Although the grip shown in FIG. 2 has a stylus shape, the grip may be a syringe, tongs, scissors, indenters and / or probes, or the like, depending on the purpose of the haptic device. A marker 101 is provided at one end of the grip so that the grip 101 can be used to measure phase information of the grip when the grip is rotated by three degrees of freedom.

Referring to FIG. 3, the haptic device includes a multiple degree of freedom actuator 300 having one joint. Rotating joint portion may be coupled to one end 310 of the multiple degree of freedom actuator.

The haptic device of FIG. 3 includes a wire drum 320, and the wire drum may transmit a rotational force to the multiple degree of freedom actuator to control the operation of the multiple degree of freedom actuator. In addition, the image capturing unit 200 (not shown) may be fixed to the wire drum to capture a specific still image of the grip and the rotating joint part to acquire phase information and / or coordinate information of the grip and the rotating joint part. Of course, the image capturing unit of FIG. 3 may also acquire image information about the grip by adaptively responding to the movement of the haptic device of FIG. 3.

4 is a view showing still another example of the haptic device as an embodiment of the present invention.

Referring to FIG. 4, the rotation joint portion 102 is provided on one side of the multiple degree of freedom actuator 300 provided in the haptic device, and the grip joint 100 having a stylus shape is provided on the rotation joint portion. In addition, an image capturing unit 200 may be provided at a lower end of the wire drum of the haptic device. In this case, the wire drum may include a motor encoder. The motor encoder can be synchronized with the rotation of the motor by coupling to the rotational axis of the motor to obtain information about the rotational speed, direction of rotation and / or rotational angle of the motor. The user may acquire coordinate information about the initial origin and the current position of the motor shaft. Thus, the coordinate information for the rotating joint can be tracked as a motor encoder. In this case, one or more motor encoders may be provided at one or more joints of the rotary joint part and / or the multiple degree of freedom actuator.

5 is a view for explaining a marker and a feature point as an embodiment of the present invention.

5A is a diagram illustrating examples of markers. The marker has a form in which contrast is clearly displayed during image processing. Referring to FIG. 5 (a), the marker is a black shape of a square and includes a brightly colored inside to clearly show contrast. That is, the user may measure the 3D rotation angle of the specific object to which the marker is attached based on the unique shape of the marker. The markers shown in FIG. 5 (a) are merely examples for describing the present invention, and may be represented by a circle, a triangle, or a polygon rather than a rectangle.

5B is a diagram illustrating an example of a feature point. The feature point is not an artificial marker, but means using a point characteristically different from the surrounding shape. In the case of Fig. 5B, the left and right end points of the human eye, nose and / or mouth are used as feature points as an example of the feature points.

6 is a view showing an example of a three-dimensional marker as an embodiment of the present invention.

Referring to Figure 6 (a), the marker may be implemented in the form of a cube. Markers may be marked on each side, and markers of different shapes may be displayed. 6 (a) shows the basic cube shape, but may have a spherical or tetrahedral polyhedron structure according to the user's requirements.

Referring to FIG. 6 (b), a reference line for determining a three-dimensional rotation angle based on a marker of a cube may be confirmed. That is, the user may know how much the marker is rotated from the reference point based on the image information of the marker photographed by the image capturing unit. Accordingly, the user may acquire phase information of the grip provided with the marker through the marker.

Phase information of three degrees of freedom of the grip and / or marker may be represented by θ, Φ, and г, respectively. The phase information of three degrees of freedom may be expressed as a roll, pitch, and yaw according to the rotation direction. The roll indicates how much the marker has rotated in the horizontal direction, the pitch indicates how much the marker has rotated up and down, and the yaw indicates how much the marker has rotated about the vertical axis.

For example, referring to FIG. 6 (b), when the space is divided into x, y, and z axes based on a hexahedral marker, θ represents a rotation angle in the horizontal direction about the x axis, and Φ is The grip represents the angle of rotation about the y axis, and г represents the angle of rotation about the z axis. In embodiments of the invention the grip and / or marker phase information may be specified as a Roll-Pitch-Yaw (RPY). Hereinafter, a method of calculating the phase information of the grip will be described in detail.

7 is a diagram showing an example of a method for calculating three-dimensional phase information of a grip as an embodiment of the present invention.

The grip may have a phase of three degrees of freedom under control of the haptic device. In the haptic device, by acquiring phase information of the grip, an external stimulus desired by the user can be precisely applied to the target object. The grip phase information may be obtained based on image information obtained by manually or automatically capturing the marker and / or the rotating joint part 102 by the image capturing unit 200.

The processor embedded in the haptic device controls the operation of the haptic device, and may control the image capturing unit to acquire image information of the current state of the marker and / or the rotating joint. The processor may calculate phase information of the marker by calculating a rotation angle of the marker from the captured image information.

In this case, the image capturing unit may perform image processing in advance before delivering the captured image information to the processor. That is, as another embodiment of the present invention, an image processing module (or image processing unit) is provided in the image capturing unit so that the position of the marker in the captured image image can be identified and transmitted to the processor. In addition, only the result value obtained by calculating the phase information of the marker based on the image information may be transmitted to the processor. In this case, the processor may acquire the phase information of the marker and the phase information of the grip. Hereinafter, a method for acquiring the phase information of the marker will be described in detail based on the rotation angle of the roll-pitch-yaw (RPY) described with reference to FIG.

The phase angles of the three degrees of freedom of the grip and / or the marker may be represented by θ, Φ and г, respectively. The method of measuring the phase information of the three degrees of freedom of the grip and / or the marker is a method of obtaining the values of θ, Φ, and г. Explain.

7 (a) and 7 (b), the grip 100 is rotated about the rotary joint 102 and the radius of rotation indicated by the broken lines of FIGS. 7 (a) and 7 (b) by the rotary joint. It can have a sphere of motion within. That is, the grip can be phase shifted in three degrees of freedom by the rotational joint portion. At this time, the marker 101 may be provided at one end portion of the grip, the marker may be provided on the L distance from the rotation joint.

In the image capturing unit 200, the grip is photographed based on the rotational joint part, and a spherical motion range of the grip may appear in a circle in the captured image information. In this case, by measuring the distance between the rotational joint and the marker in the image image it can be calculated L 'that is the distance on the image.

Assuming that the user (or the haptic device) knows the center coordinates of the rotating joint and the center coordinates of the marker, the θ value can be easily obtained by using the method of obtaining the angle between the two vectors. For example, the center coordinates of the rotating joint in space may be represented as a vector U, and the center coordinates of the marker may be represented as a vector V. In this case, the angle θ between the two vectors may be obtained as Equation 1 below.

The value of Φ can be obtained by using the following Equation 2 using L ′, which is the distance on the image between the center of the rotating joint and the marker, and using L, which is a preset actual distance.

The? value can be obtained by measuring the phase of the marker as shown in FIG. 7C when the position of the marker is changed based on when the marker is at the center of the focal point of the image capturing unit. For example, in the case where the image capturing unit is provided in a specific part of the haptic device and operated manually, the left figure of FIG. 7C is a reference axis when the marker is placed on a line coinciding with the focal point of the image capturing unit. It is present. 7C is a diagram showing how much the marker is rotated from the reference axis when the marker is moved.

If the image capturing unit adaptively operates in accordance with the operation of the haptic device (eg, a pan tilt camera, etc.), one point of the marker is taken as a reference point, and how much the marker has moved relative to the reference point during the next shooting. The value of г can be measured based on that.

As such, the user may acquire the phase information of the grip using the phase information of the marker based on the image photographed by the image capturing unit. That is, the user can obtain the value θ by measuring how much the center of the marker is rotated in the horizontal direction, obtain the value of г by measuring how much the pattern of the marker is rotated, and the actual length L of the marker and the rotating joint part. The value of Φ can be obtained from the relationship between the distance L 'and the measured distance L' on the image.

8 is a diagram illustrating one of methods of controlling a haptic device according to an embodiment of the present invention.

The user may operate the haptic device disclosed in the embodiments of the present invention to drive the multiple degree of freedom actuator 300 of the haptic device. That is, the user may control the haptic device to position the multi-degree of freedom actuator of the haptic device to apply an external stimulus to the object (S810).

When the multiple degree of freedom actuator is positioned at a point where a predetermined external stimulus can be applied to the object, the user can drive the rotary joint 102 to apply a phase shift of three degrees of freedom to the grip 100 coupled to the rotary joint. It may be (S820).

The user may apply an external stimulus to the target object by using the grip using the steps S810 and S820. Since the haptic device moves the multi-degree of freedom actuator and the rotating joint under the control of the user, the haptic device may acquire coordinate information of the rotating joint and / or the marker in real time.

In this case, the coordinate information of the rotation joint may be tracked as a motor encoder.

The image capturing unit 200 may acquire image information by capturing the rotating joint part periodically or aperiodically under the control of the user (S840).

In addition, the controller of the haptic device may calculate the phase information of the grip based on the coordinate information obtained in operation S830 and the image information photographed in operation S840 (S850). The method of calculating the three-dimensional phase information of the grip may refer to the description of FIG. 7.

In other embodiments of the present invention, feature points may be used instead of the marker 101. In FIG. 5 (b), a feature point is described using an example of a face of a person. However, when the grip is used, both ends of the grip and the center of the rotating joint may be used as the feature point.

As another embodiment of the present invention, a light emitting diode (LED) capable of emitting infrared rays may be used as a material of the marker 101. In this case, the image capturing unit preferably includes an infrared camera system capable of detecting infrared rays, and may capture infrared images. That is, the image processor may acquire phase information of the grip by detecting a corresponding point in the captured infrared image.

The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention. In addition, claims that do not have an explicit citation in the claims may be combined to form an embodiment or included in a new claim by amendment after the application.

100: grip
101: marker
102: rotating joint
200: video recording unit
300: multiple degree of freedom actuator
400: processor

Claims (17)

In the haptic device using the image information,
A grip for applying an external stimulus to the object;
A marker provided at one end of the grip;
A multiple degree of freedom actuator for accessing the grip to the object;
A rotary joint unit coupling the grip and the multi-degree of freedom actuator to provide a three degree of freedom rotational movement to the grip; And
And an image capturing unit for capturing at least one of the grip, the marker, and the rotating joint part to obtain the image information. The method of claim 1,
Further comprising a processor for controlling the multi-degree of freedom actuator, the rotary joint, the grip and the image capturing unit,
And the processor controls the image capturing unit to obtain the image information when the external stimulus is applied to the target object, and derives phase information about the grip from the image information. The method of claim 2,
The phase information on the grip is derived using coordinate information of the marker, coordinate information of the rotating joint part, and three-dimensional rotation angle of the marker derived from the image information. The method of claim 3,
The marker includes a light emitting diode (LED) emitting infrared light,
The image capturing unit includes an infrared camera system capable of detecting the infrared, haptic device. In the haptic device using the image information,
A grip for applying an external stimulus to the object;
A marker provided at one end of the grip;
A multiple degree of freedom actuator for accessing the grip to the object;
A rotary joint unit coupling the grip and the multi-degree of freedom actuator to provide a three degree of freedom rotational movement to the grip; And
An image capturing unit configured to acquire at least one of the grip, the marker, and the rotating joint part to obtain the image information;
The image capturing unit further comprises an image processing module for processing the image information to derive the phase information of the grip. 6. The method of claim 5,
Phase information about the grip,
The haptic device is derived using the three-dimensional rotation angle of the marker derived from the coordinate information of the marker, the coordinate information of the rotating joint portion and the image information. The method according to claim 6,
The marker includes a light emitting diode (LED) emitting infrared light,
The image capturing unit includes an infrared camera system capable of detecting the infrared, haptic device. In the haptic device using the image information,
A grip for applying an external stimulus to the object;
A marker provided at one end of the grip;
A multiple degree of freedom actuator for accessing the grip to the object;
A rotary joint unit coupling the grip and the multi-degree of freedom actuator to provide a three degree of freedom rotational movement to the grip;
At least one encoder for acquiring coordinate information of the marker and coordinate information of the rotating joint part; And
And an image capturing unit for capturing at least one of the grip, the marker, and the rotating joint part to obtain the image information. The method of claim 8,
Further comprising a processor for controlling the multi-degree of freedom actuator, the rotary joint, the grip and the image capturing unit,
And the processor controls the image capturing unit to obtain the image information when the external stimulus is applied to the target object, and derives phase information about the grip from the image information. The method of claim 9,
The phase information on the grip is derived using coordinate information of the marker, coordinate information of the rotating joint part, and three-dimensional rotation angle of the marker derived from the image information. In the method for controlling the haptic device using the image information,
Controlling the multiple degree of freedom actuator to apply an external stimulus to the object;
Controlling the rotary joint part and the grip provided to the multi-degree of freedom actuator to apply the external stimulus to the target object;
Acquiring coordinate information of the rotary joint part and coordinate information of the marker;
Acquiring image information about the marker provided on the rotary joint part and the grip; And
And calculating phase information of the grip by using coordinate information of the rotating joint part, coordinate information of the marker, and the image information. 12. The method of claim 11,
Phase information of the grip,
At least one of coordinate information of the marker, coordinate information of the rotating joint part, actual distance information of the marker and the rotating joint part, distance information on the image obtained from the marker and the rotating joint part, and phase information of the marker obtained from the image information. Calculated using the haptic device control method. 13. The method of claim 12,
The rotating joint portion,
And coupling the grip with the multiple degree of freedom actuator and providing a three degree of freedom rotational motion to the grip. In the method for controlling the haptic device using the image information,
Acquiring coordinate information of a marker included in the haptic device;
Acquiring image information about the marker; And
Comprising the step of calculating the phase information of the grip equipped with the marker using the coordinate information and the image information,
Phase information for the grip is calculated using the coordinate information of the marker and the three-dimensional rotation angle of the marker derived from the image information, the haptic device control method. The method of claim 14,
The three-dimensional rotation angle of the marker,
And at least one of coordinate information of the marker, actual distance information of the marker and the rotating joint part, and distance information on the image of the marker and the rotating joint part obtained from the image information. 16. The method of claim 15,
The rotating joint portion,
Combining the grip for applying an external stimulus to a target object and a multiple degree of freedom actuator for controlling the grip to apply the external stimulus to the target object,
Providing a rotational motion of three degrees of freedom in the grip. In the haptic device using the image information,
A grip for applying an external stimulus to the object;
A three-dimensional solid marker provided at one end of the grip;
A multiple degree of freedom actuator for accessing the grip to the object;
A rotary joint unit coupling the grip and the multi-degree of freedom actuator to provide a three degree of freedom rotational movement to the grip;
An encoder for acquiring coordinate information of the three-dimensional solid marker and coordinate information of the rotating joint part;
An image capturing unit configured to acquire the image information by capturing the grip, the marker, and the rotating joint part; And
And an image processing module for processing the image information to derive the phase information of the grip.

Images