KR102245898B1 - Method and apparatus of providing tactile feedback - Google Patents

Abstract

A method and apparatus for providing tactile feedback are disclosed. A method of providing tactile feedback according to an embodiment includes the steps of detecting an object, rendering the shape of the object based on the detection result, and virtual rigid body particles based on the rendered shape. ), and analyzing an interaction between the virtual rigid particle and the virtual fluid particle to obtain a focus point at which the force of the virtual fluid acts on the object. Includes.

Description

Method and apparatus for providing tactile feedback {METHOD AND APPARATUS OF PROVIDING TACTILE FEEDBACK}

The following embodiments relate to a method and apparatus for providing tactile feedback.

In a real space, a user interacts with a fluid using a tool, or interacts by directly contacting the body with the fluid. In order to enable the user to interact in a virtual environment as above, physics-based simulation technology that can reflect physical quantities in real time by calculating interactions between objects and fluids in virtual space, and visual feedback based on physics-based simulations. There is a need for rendering technology and haptic rendering technology that provides inverse or tactile feedback.

In particular, in the case of a virtual environment simulation in which interaction is involved, a haptic rendering technology that delivers the result of the interaction through a sense of force or touch is more important.

In order to physically calculate the interaction between an object and a fluid in a virtual environment, simulations based on an integrated particle model that convert objects in a virtual space into particles and compute on individual particles have been continuously proposed.In particular, in the field of haptic rendering technology, the output of 6-degree of freedom Techniques for providing embarrassment feedback to users have been mainly proposed.

This method calculates the fluid force acting on individual rigid particles, calculates the resultant force and the rotation force at the center of gravity, and then provides the corresponding force information to the haptic interface. Here, since haptic feedback is provided by considering a rigid body as a point, it is suitable for a scenario in which a tool is used.

However, for the scenario in which the body and the fluid are in contact, the 6-degree-of-freedom rigid-fluid haptic feedback providing technique that considers the resultant force and the rotational force at the center of gravity is the same as the flow information of the fluid at the surface of the rigid body. It has limitations in delivering information.

Embodiments may provide non-contact tactile feedback by analyzing an interaction between a virtual rigid body and a fluid.

A method of providing tactile feedback according to an embodiment includes the steps of detecting an object, rendering the shape of the object based on the detection result, and virtual rigid body particles based on the rendered shape. ), and analyzing an interaction between the virtual rigid particle and the virtual fluid particle to obtain a focus point at which the force of the virtual fluid acts on the object. Includes.

The method of providing tactile feedback may further include providing tactile feedback to the object based on a focus point.

The providing may include determining a characteristic of a wave based on the concentration point, and providing the tactile feedback by outputting the wave to the object.

The rendering may include rendering the shape as a depth texture for the object based on depth information from a position detecting the object to a surface of the object.

The generating of the virtual rigid particle may include generating a virtual rigid particle corresponding to a part of the surface of the object based on the rendered object.

The step of obtaining the focus point may include generating a force array acting on the virtual rigid particle by performing particle simulation between the virtual rigid particle and the virtual fluid particle, and obtaining the focus point based on the force array It may include the step of.

The generating of the force array includes calculating a force acting between the virtual rigid particle and the virtual fluid particle, calculating a position of the virtual fluid particle based on the force, and the virtual fluid particle And generating the force arrangement based on the position of and the force.

The step of obtaining the focus point based on the force arrangement includes: deriving a maximum force point based on the force arrangement, calculating a virtual focus point based on the maximum force point, and the virtual focus point It may include the step of converting to the concentration point.

The step of deriving a maximum force point based on the force arrangement may include determining a point at which a maximum force acts in the force array as the maximum force point or a point at which a local maximum force acts in the force array is the maximum force. It may include the step of determining the point.

The calculating of the virtual focus point may include calculating at least one of a location of the virtual focus point, a force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point based on the maximum force point. It may include steps.

The apparatus for providing tactile feedback according to an embodiment includes a sensor for detecting an object, rendering a shape of the object based on a detection result, generating a virtual rigid particle based on the rendered shape, and generating a virtual rigid particle and the virtual rigid particle And a processor that analyzes an interaction between fluid particles and obtains a concentration point at which the force of the virtual fluid acts on the object.

The apparatus for providing tactile feedback may further include a transmitter that provides tactile feedback to the object based on a focus point.

The transmitter may provide the tactile feedback by determining a characteristic of a wave based on the concentration point and outputting the wave to the object.

The processor may render the shape as a depth texture for the object based on depth information from a position sensing the object to a surface of the object.

The processor may generate virtual rigid particles corresponding to a part of the surface of the object based on the rendered object.

The processor may generate a force array acting on the virtual rigid particle by performing particle simulation between the virtual rigid particle and the virtual fluid particle, and obtain the focus point based on the force array.

The processor calculates a force acting between the virtual rigid particle and the virtual fluid particle, calculates a position of the virtual fluid particle based on the force, and calculates the position of the virtual fluid particle based on the force and the position of the virtual fluid particle. Can generate force arrays.

The processor may derive a maximum force point based on the force arrangement, calculate a virtual focus point based on the maximum force point, and convert the virtual focus point into the focus point.

The processor may determine a point in the force arrangement where the maximum force acts as the maximum force point, or determine a point in the force array where the local maximum force acts as the maximum force point.

The processor may calculate at least one of a location of the virtual focus point, a force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point based on the maximum force point.

1 is a schematic block diagram of an apparatus for providing tactile feedback according to an exemplary embodiment.
2A shows an example of an implementation of the apparatus for providing tactile feedback shown in FIG. 1.
2B is a photograph of the apparatus for providing tactile feedback shown in FIG. 2A.
3 shows the operation of the processor and the transmitter shown in FIG. 1.
4 shows an example of the transmitter shown in FIG. 1.
5 shows an example of the operation of the virtual input/object controller and the graphic renderer shown in FIG. 3.
6 shows an example of an area rendered by the processor shown in FIG. 1.
7 shows an example of operations of the transmitter, the focus point calculator, and the particle simulator shown in FIG. 3.
8 shows an example of a rigid particle generated by the processor shown in FIG. 1.
9 is a diagram for explaining a focus point on an object.
10 is a graph for explaining an operation of calculating the concentration point by the concentration point calculator of FIG. 3.
11 shows an example of a method of allocating a plurality of focus points.
12 shows an example of an operation of controlling a modulation frequency for a focus point.
13A shows an example of a simulation result of the simulator shown in FIG. 3.
13B shows an example of visualizing the distribution of force acting on a virtual rigid particle.
14 is a flowchart illustrating an operation of the apparatus for providing tactile feedback shown in FIG. 1.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It is to be understood that all changes, equivalents, or substitutes to the embodiments are included in the scope of the rights.

The terms used in the examples are used for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of rights according to the concept of the embodiment, the first component may be named as the second component, and similarly The second component may also be referred to as a first component.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of the reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the embodiments, the detailed description thereof will be omitted.

A module in the present specification may mean hardware capable of performing functions and operations according to each name described in the present specification, or may mean a computer program code capable of performing a specific function and operation. Or, it may mean an electronic recording medium, for example, a processor or a microprocessor in which a computer program code capable of performing a specific function and operation is mounted.

In other words, the module may mean a functional and/or structural combination of hardware for performing the technical idea of the present invention and/or software for driving the hardware.

1 is a schematic block diagram of an apparatus for providing tactile feedback according to an exemplary embodiment.

Referring to FIG. 1, the apparatus 10 for providing tactile feedback may provide tactile (or tactile) feedback to an object. The tactile feedback providing apparatus 10 may provide tactile feedback without contacting an object.

The object may mean an object to provide tactile feedback and/or an object to render a shape. Objects can be composed of rigid bodies. For example, an object may include a person and/or a part of a person's body.

The tactile feedback providing device 10 renders a shape of an object, analyzes an interaction between a rigid body and a fluid based on the rendered shape, and provides tactile feedback to the object. rendering).

The apparatus 10 for providing tactile feedback may analyze an interaction between a virtual rigid body and a virtual fluid through a physics-based simulation in a virtual environment and provide tactile feedback through a non-contact haptic interface.

The tactile feedback providing apparatus 10 may analyze an interaction between a rigid body and a fluid in a virtual environment and provide tactile feedback in real time through a non-contact haptic interface. The tactile feedback providing device 10 may generate rigid particles for a tactile feedback range that can be generated by a haptic interface, and may generate a focus point to provide tactile feedback according to a result of a rigid-fluid interaction. .

The tactile feedback providing device 10 may transmit tactile feedback to a human body without directly contacting the haptic device. The tactile feedback providing device 10 outputs vibration feedback to the body using an ultrasonic transducer array or a laser, so that the user can feel the tactile feedback while freely posing in a haptic rendering space compared to a contact type device. .

Through this, the tactile feedback providing device 10 may provide a higher sense of reality in an immersive virtual reality application including a fluid simulation.

The tactile feedback providing device 10 may be implemented as an IoT device, a machine-type communication device, or a portable electronic device.

Portable electronic devices include a laptop computer, a mobile phone, a smart phone, a tablet PC, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA). ), digital still camera, digital video camera, portable multimedia player (PMP), personal navigation device or portable navigation device (PND), handheld game console, e-book (e-book), it can be implemented as a smart device (smart device). For example, the smart device may be implemented as a smart watch or a smart band.

The tactile feedback providing apparatus 10 includes a sensor 100, a processor 200, and a transmitter 300. The apparatus 10 for providing tactile feedback may further include a memory 400 and a display 500.

The sensor 100 may detect an object. The sensor 100 may detect movement of an object. For example, the sensor 100 may include a camera and a motion sensor. The sensor 100 may output the detected motion of the object to the processor 200.

The processor 200 may process a detection result of the object received by the sensor 100 and/or data stored in the memory 400. The processor 200 may execute computer-readable code (eg, software) stored in the memory 400 and instructions induced by the processor 200.

The “processor 200” may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, desired operations may include code or instructions included in a program.

For example, a data processing device implemented in hardware is a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA).

The processor 200 may generate a virtual rigid particle generated in real time and calculate a fluid force acting on the virtual rigid particle. In addition, the processor 200 may provide tactile feedback by filtering the corresponding arrangement.

The processor 200 converts the surface of the rendered object corresponding to the physical range expressable by the tactile feedback providing device 10 into rigid particles through the surface of the object where the interaction occurs regardless of the shape of the object through 2D-3D projection mapping. Configurable.

The processor 200 is configured to include General-Purpose computing on Graphics Processing Units (GPGPU) to calculate stable tactile feedback and simulate a rigid-fluid interaction in real time.

The processor 200 may render the shape of the object based on the detection result. The processor 200 may render the shape of the object as a depth texture for the object based on depth information from a position detecting the object to the surface of the object.

The processor 200 may provide tactile feedback and/or visual information to the object by processing a result detected by the sensor.

The processor 200 may generate virtual rigid body particles based on the shape of the rendered object. The processor 200 may generate virtual rigid particles corresponding to a part of the surface of the object based on the rendered object.

The processor 200 may analyze an interaction between a virtual rigid particle and a virtual fluid particle to obtain a focal point at which a force of the virtual fluid acts on the object. .

The processor 200 may generate a force array acting on the virtual rigid particle by performing particle simulation between the virtual rigid particle and the virtual fluid particle. Specifically, the processor 200 may calculate a force acting between the virtual rigid particle and the virtual fluid particle. For example, the processor 200 may calculate a force acting between a virtual rigid body and a virtual fluid particle through a physics-based simulation using a particle method such as smoothed particle hydrodynamics.

The processor 200 may calculate the position of the virtual fluid particle based on the force. The processor 200 may generate a force arrangement based on the position of the virtual fluid particle and the calculated force.

The processor 200 may acquire a focus point based on the generated force arrangement. The processor 200 may derive a maximum force point based on the force arrangement. Specifically, the processor 200 may determine a point where the maximum force acts in the force array as the maximum force point, or determine a point in the force array where the local maximum force acts as the maximum force point.

The processor 200 may calculate the virtual focus point based on the derived maximum force point. Specifically, the processor 200 may calculate at least one of a location of a virtual focus point, a force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point based on the maximum force point.

The processor 200 may convert the virtual focus point into a focus point. That is, the focus point may mean a point corresponding to a physical location.

The transmitter 300 may provide tactile feedback to the object based on the acquired focus point. The transmitter 300 may provide tactile feedback by determining a characteristic of a wave based on a concentration point and outputting the wave to an object.

The transmitter 300 may include a transducer. For example, the transmitter 300 may be implemented as an array of a plurality of transducers.

Waves output from the transmitter 300 may include sound waves. For example, the transmitter 300 may output ultrasonic waves to the object.

The memory 400 may store instructions (or programs) executable by the processor 200. For example, the instructions may include instructions for executing an operation of the processor 200 and/or an operation of each component of the processor 200.

The memory 400 may be implemented as a volatile memory device or a nonvolatile memory device.

The volatile memory device may be implemented with dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM).

Nonvolatile memory devices include EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM (MRAM), Conductive Bridging RAM (CBRAM). , FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM)), holographic memory, Molecular Eelectronic Memory Device, or Insulator Resistance Change Memory.

The display 500 may provide visual information to an object. For example, the display 500 may provide a simulation result rendered by the processor 200 to an object.

The display 500 may include a three-dimensional display. For example, the display 500 may include a head mounted display. The display 500 may provide a 3D image to an object by using the rendering result of the processor 200.

FIG. 2A shows an example of an implementation of the tactile feedback providing apparatus shown in FIG. 1, and FIG. 2B shows a photograph of the tactile feedback providing apparatus shown in FIG. 2A.

2A and 2B, the sensor 100 may include a hand tracker. The sensor 100 may detect the user's hand and/or hand movement.

The processor 200 may generate a focus point based on the sensed hand and/or hand movement. The processor 200 may output the generated focus point to the transmitter 300.

The processor 200 may generate visual information to be provided to an object. For example, the processor 200

Transmitter 300 may include a transducer arrangement. The transmitter 300 may provide tactile feedback by outputting ultrasonic waves to the object based on the focus point. The object may include a part of the user's hand.

The display 500 may provide visual information generated by the processor 200 to an object. The display 500 may provide rendered visual information corresponding to tactile feedback. In the example of FIG. 2B, the display 500 may provide an integrated particle simulation as visual information.

3 shows the operation of the processor and the transmitter shown in FIG. 1, and FIG. 4 shows an example of the transmitter shown in FIG. 1.

3 and 4, the processor 200 may include a virtual input/object controller 210, a graphic renderer 230, a focus point calculator 250, and a particle simulator 270.

The virtual input/object controller 210 may change the virtual environment by receiving the detection result of the object from the sensor 100. The virtual input/object controller 210 may reflect the detection result of the sensor 100 in a virtual environment. The detection result of the sensor 100 may include a shape and motion of an object (or a part of the object).

The virtual input/object controller 210 may output a virtual environment in which the detection result of the sensor is reflected to the graphic renderer 230. The virtual input/object controller 210 may be implemented inside the sensor 100 as needed.

The graphic renderer 230 may render the shape of an object. The graphic renderer 230 may render an object and an image to be provided to the focus point calculator. The graphic renderer 230 may output the shape of the rendered object to the focus point calculator 250.

The graphic renderer 230 may render a virtual environment in which the detection result of the sensor is reflected as an image. For example, the graphic renderer 230 may render a depth texture of an object and output it to the focus point calculator 250.

The focus point calculator 250 may generate virtual rigid particles based on the shape of the rendered object. Specifically, the focus point calculator 250 may generate a virtual rigid particle for a location corresponding to the surface of the object. For example, the focus point calculator 250 may generate virtual rigid particles by converting the depth texture into virtual rigid particles.

In addition, the focus point calculator 250 may receive a force array from the particle simulator 270 to calculate the focus point. The concentrated contact calculator 250 may generate a plurality of focus points by filtering the received force array. The focus point calculator 250 may output the generated plurality of focus points to the transmitter 300.

The operation of calculating the focus point by the focus point calculator 250 will be described in detail with reference to FIG. 6.

The particle simulator 270 may perform particle simulation between virtual rigid particles and virtual fluid particles. The particle simulator 270 may generate an array of forces acting on the virtual rigid particle based on the particle simulation result. The particle simulator 270 may output the generated force array to the concentration point calculator 250.

The particle simulator 270 may perform physical simulation on the virtual rigid particle in real time using virtual fluid particle data, virtual rigid particle data, and depth texture.

The virtual fluid particle data may include physical properties of the virtual fluid particle. Physical properties of the virtual fluid particle may include the size, position, force, velocity, mass, and density of the fluid particle. Rigid particle data may include properties of rigid particles. The physical properties of the virtual rigid particle may include the size, position, force, velocity, mass, and density of the forced particle.

The particle simulator 270 may consider a resultant force and a rotational force acting on the center of gravity of the virtual rigid particle in order to analyze the force of the virtual fluid particles acting on the virtual rigid particle.

The particle simulator 270 may analyze the interaction between the virtual rigid body and the virtual fluid using the integrated particle model. For example, you can calculate physics-based forces using smoothed-particle hydrodynamics.

The transmitter 300 may receive a focus point (or a physical focus point) from the processor 200. The transmitter 300 may provide tactile feedback to the object based on the focus point received from the processor 200.

The transmitter 300 may determine the characteristics of the wave to be output based on the concentration point. For example, the transmitter 300 may determine a waveform such as a phase and amplitude of a wave, and output a wave whose characteristics are determined.

There may be a plurality of waves output from the transmitter 300. The transmitter 300 may provide tactile feedback to an object by outputting a wave. For example, the wave output from the transmitter 300 may include ultrasonic waves.

As in the example of FIG. 4, the transmitter 300 may be implemented as a transducer array composed of a plurality of transducers. The transducer may be implemented as a 2D ultrasonic transducer array. The transmitter 300 may transmit acoustic radiation force to a 3D physical point by using a 2D transducer array.

Through this acoustic reflex, the object can feel tactile feedback (or vibration) at a three-dimensional physical point.

The transmitter 300 may perform amplitude modulation of an output waveform to generate acoustic radiation force. For example, the transducer of FIG. 4 may have a range of 15 to 80 cm and may have an angle of 60 degrees.

In addition, as the transducer of FIG. 4 gets further away, the intensity of the sense that the object feels may decrease. Thus, the object can be provided with optimal tactile feedback at a specific distance. For example, the optimal distance may be within 20 cm.

The transmitter 300 may output a wave to the object at a constant feedback rate. For example, the transmitter 300 may provide tactile feedback at a feedback rate of 70 Hz.

를 제한할 수 있다.The transmitter 300 may use the integrated particle simulation to calculate a feedback strength that provides the physical properties and forces of particles in the surface where the interaction occurs every frame. Depending on the simulation integration method, the level of simulation stability may differ. Transmitter 300 is the maximum speed of the virtual particle to provide a stable simulation result.

Can be limited.

Since the maximum force of the virtual particle can be defined by the velocity, the transmitter 300 can determine the feedback strength for each particle corresponding to the maximum force. Here, the maximum force may be determined as in Equation 1.

Here, F _limit means the maximum force, m _particle means the mass of the virtual particle, and c ₀ means a user control constant. For example, the user adjustment constant may be 0.5.

The transmitter 300 may calculate the feedback strength based on the maximum force. The feedback strength based on the maximum force can be expressed as in Equation 2.

Here, I _particle may mean a feedback strength for a _{virtual particle, F particle} may mean a force acting on the virtual particle, and I _min may mean a minimum feedback strength.

Since the feedback strength does not represent an actual virtual force, the transmitter 300 may calculate the feedback strength using the lowest strength capable of detecting a contact.

Hereinafter, operations of the virtual input/object controller 210 and the graphic renderer 230 will be described with reference to FIGS. 5 and 6.

FIG. 5 shows an example of an operation of a virtual input/object controller and a graphic renderer shown in FIG. 3, and FIG. 6 shows an example of an area rendered by the processor shown in FIG. 1.

5 and 6, the virtual input/object controller 210 may receive a control command for a virtual object from an operator. The virtual input/object controller 210 may change the state of the virtual object in the virtual environment by processing the input control command. The virtual object may include a virtual rigid body and a virtual fluid. The virtual input/object controller 210 may perform a graphic rendering command to the graphic renderer 230 according to the control command.

The graphic renderer 230 may render visual information. The graphic renderer 230 may render a depth texture for an object, and may render visual feedback information to be provided to the object.

For example, the graphic renderer 230 may render visual information based on an image received through a camera and output it to the display 500. The graphic renderer 230 may process, convert, and output visual information suitable for the display 500.

The graphic renderer 230 may render a depth texture for an object. The graphic renderer 230 may generate 3D depth information in the form of a texture through rendering of the depth texture. The graphic renderer 230 may output the generated texture to the focus point calculator 250. In addition, the graphic renderer 230 may transmit a virtual rigid particle generation command to the focus point calculator 250 together.

The graphic renderer 230 may configure a virtual camera for generating virtual rigid particles. The graphic renderer 230 may configure a virtual camera using rendering information corresponding to the location of the transmitter 300 (eg, rotation information and viewing angle information of the transmitter 300 ). The graphic renderer 230 may generate a depth texture of an object using a virtual camera.

In the example of FIG. 6, the trapezoidal area may represent a projection range rendered by the virtual camera of the graphic renderer 230. The graphic renderer 230 may generate a depth texture for the trapezoidal area and output it to the focus point calculator 250.

Hereinafter, operations of the focus point calculator 250, the particle simulator 270, and the transmitter 300 will be described with reference to FIGS. 7 to 9.

7 shows an example of the operation of the transmitter, the focus point calculator, and the particle simulator shown in FIG. 3, FIG. 8 shows an example of a rigid particle generated by the processor shown in FIG. 1, and FIG. 9 is a focus point on an object. It is a figure for explaining.

7 to 9, the focus point calculator 250 may generate virtual rigid particles based on the depth texture transmitted from the graphic renderer 230. The concentration point calculator 250 may output the generated virtual rigid particle to the particle simulator 270. For example, the concentration point calculator 250 may output virtual rigid particles to the particle simulator 270 in the form of an array of virtual rigid particles.

8 shows an example of a virtual rigid particle generated by the concentration point calculator 250. That is, virtual rigid particles can be generated on the surface of the hand when the object is a hand.

The focus point calculator 250 may restore each pixel to 3D virtual coordinates according to the setting of the virtual camera based on the received depth texture of the object. The focus point calculator 250 may convert the restored 3D virtual coordinates into a virtual rigid particle array according to an existing virtual rigid particle setting. Through this, the tactile feedback providing apparatus 10 may implement a rendering area in a virtual environment.

The focus point calculator 250 may generate a virtual rigid particle array in real time by receiving the depth texture of the object in real time. In addition, the concentration point calculator 250 may output the generated virtual rigid particle array to the particle simulator 270 in real time. The example of FIG. 8 may represent an example of an arrangement of virtual rigid particles generated by the focus point calculator 250.

The particle simulator 270 may analyze an interaction between the received virtual rigid particle and the virtual fluid particle in the simulation. The particle simulator 270 may calculate the interaction between the virtual rigid body and the virtual fluid, and calculate the location and physical properties of the virtual fluid particle.

The particle simulator 270 may calculate a force exerted by the virtual fluid particle on the virtual rigid particle through simulation. The particle simulator 270 may generate a force array acting on the virtual rigid particle based on the calculated force. The particle simulator 270 may output the generated force array to the concentration point calculator 250.

The focus point calculator 250 may calculate the focus point by receiving a force array from the particle simulator 270. The concentration point calculator 250 may derive the maximum force point using the force array generated by the particle simulator 270.

In general, the force of a fluid acting on a rigid body may be greater as the number of fluid particles adjacent to each rigid body particle increases, and the higher the force applied by each fluid particle is increased. Therefore, it can be seen that the maximum force encodes the flow information of the fluid.

The focus point calculator 250 may derive a maximum force point based on the received force arrangement and use it for calculating the focus point. The focus point calculator 250 may calculate a plurality of virtual focus points by filtering the force array and extracting rigid particle information.

The virtual focus point generated by the focus point calculator 250 may include information on a location of a 3D physical space of the virtual focus point, a force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point. . The focus point calculator 250 may calculate the virtual focus point using the position of the virtual rigid particle and the magnitude of the force acting on the virtual rigid particle.

The focus point calculator 250 may convert the virtual focus point into a physical focus point (or focus point) reflecting a physical location, and output this to the transmitter 300. The transmitter 300 may provide tactile feedback to the object based on the received focus point.

In the example of FIG. 9, the left side may indicate an area in which the particle simulator 270 performs simulation. The right side may indicate a focus point at which the transmitter provides tactile feedback. In this case, the transmitter 300 may have an arbitrary number of feedbacks.

When the object is a human hand, the focus point calculator 250 may generate a focus point only for a maximum force point because the cognitive spatial resolution of the hand (for example, it is possible to distinguish when 2 cm apart) is low.

FIG. 10 is a graph for explaining an operation of calculating a focus point by the focus point calculator of FIG. 3, FIG. 11 shows an example of a method of allocating a plurality of focus points, and FIG. 12 is a control of a modulation frequency for a focus point. Here is an example of the operation to be performed

10 to 12, the focus point calculator 250 may generate a focus point by receiving a force array from the particle simulator 270 and filtering the received force array. The example of FIG. 10 may represent an operation of allocating one focus point by filtering a force array in consideration of a person's cognitive resolution.

Since the force acting on an arbitrary particle is affected by the number of adjacent particles, the tactile feedback providing device 10 can grasp a rigid-fluid collision situation and a flow situation through haptic feedback using force information.

The focus point calculator 250 may generate one focus point per frame through a force array filtering module. The focus point calculator 250 may use a queue technique to allocate a plurality of focus points.

Elements in the queue may be defined as focus points having an arbitrary modulation period. The focus point calculator 250 may reduce the modulation period inside the queue whenever the focus point is allocated. The focus point calculator 250 may adaptively define a modulation frequency of the focus point using information in the queue.

Considering the case where the object is a human body, the receptor in the skin can detect vibration in the range of 0.4 Hz-500 Hz. The focus point calculator 250 may allocate a plurality of focus points at the same time by allocating a plurality of focus points based on a queue-based allocation method as illustrated in FIG. 12.

At this time, since a collision may occur when a focus point is repeatedly allocated to an adjacent position, the focus point calculator 250 may control the modulation frequency based on whether or not the focus point is allocated to the adjacent position again.

13A shows an example of a simulation result of the simulator shown in FIG. 3, and FIG. 13B shows an example of visualizing the distribution of force acting on a virtual rigid particle.

13A and 13B, the display 500 may provide visual information (or visual feedback) to an object. 13A and 13B may represent examples of visual information provided by the display 500.

13A may show a result of visualizing a situation in which a virtual rigid body-virtual fluid interacts in a virtual environment. 13B may show a result of visualizing a force acting on a rigid body by an interaction between a virtual rigid body and a virtual fluid.

13B, it can be visually confirmed that the force applied to the virtual rigid body is large in an area where there are many adjacent virtual fluid particles. The tactile feedback providing apparatus 10 may encode collision information between a virtual rigid body and a virtual fluid particle in real time as tactile feedback so that a user can experience the flow of the virtual fluid through simulation.

14 is a flowchart illustrating an operation of the apparatus for providing tactile feedback shown in FIG. 1.

Referring to FIG. 14, the sensor 100 may detect an object. The processor 200 may render the shape of the object based on the detection result of the sensor 100 (1410 ).

The processor 200 may render a shape as a depth texture for the object based on depth information from a position sensing the object to the surface of the object.

The processor 200 may generate a virtual rigid body particle based on the rendered shape (1430). The processor 200 may generate virtual rigid particles corresponding to a part of the surface of the object based on the rendered object.

The processor 200 may analyze an interaction between a virtual rigid particle and a virtual fluid particle to obtain a focus point at which a force of the virtual fluid acts on the object.

The processor 200 may generate a force array acting on the virtual rigid particle by performing particle simulation between the virtual rigid particle and the virtual fluid particle. Specifically, the processor 200 may calculate a force acting between the virtual rigid particle and the virtual fluid particle, and calculate the position of the virtual fluid particle based on the calculated force.

The processor 200 may generate a force arrangement based on the position of the virtual fluid particle and the calculated force.

The processor 200 may acquire the focus point based on the generated force arrangement. The processor 200 may derive a maximum force point based on the force arrangement. Specifically, the processor 200 may determine a point where the maximum force acts in the force array as the maximum force point, or determine a point in the force array where the local maximum force acts as the maximum force point.

The processor 200 may calculate the virtual focus point based on the maximum force point. Specifically, the processor 200 may calculate at least one of a location of a virtual focus point, a force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point based on the maximum force point.

The processor 200 may convert the calculated virtual focus point into a focus point.

The transmitter 300 may provide tactile feedback to the object based on the focus point. The transmitter 300 may provide tactile feedback by determining a characteristic of a wave based on a concentration point and outputting the wave whose characteristic is determined to an object.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

As described above, although the embodiments have been described by the limited drawings, a person of ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the following claims.

Claims (20)

Detecting an object;
Rendering the shape of the object based on the detection result;
Generating virtual rigid body particles based on the rendered shape; And
Analyzing the interaction between the virtual rigid particle and the virtual fluid particle to obtain a focus point at which the force of the virtual fluid acts on the object.
Including,
The step of obtaining the focus point,
Obtaining the concentration point based on the maximum force point derived using the force array acting on the virtual rigid particle
A method of providing tactile feedback comprising a.
The method of claim 1,
Providing tactile feedback to the object based on the focus point
A method for providing tactile feedback further comprising a.
The method of claim 2,
The providing step,
Determining a characteristic of the wave based on the concentration point; And
Providing the tactile feedback by outputting the wave to the object
A method of providing tactile feedback comprising a.
The method of claim 1,
The rendering step,
Rendering the shape as a depth texture for the object based on depth information from a position detecting the object to the surface of the object
A method of providing tactile feedback comprising a.
The method of claim 1,
The step of generating the virtual rigid particle,
Generating virtual rigid particles corresponding to a part of the surface of the object based on the rendered object
A method of providing tactile feedback comprising a.
The method of claim 1,
The step of obtaining the focus point,
Performing a particle simulation between the virtual rigid particle and the virtual fluid particle to generate the force array acting on the virtual rigid particle; And
Obtaining the focus point based on the force arrangement
A method of providing tactile feedback comprising a.
The method of claim 6,
The step of generating the force arrangement,
Calculating a force acting between the virtual rigid particle and the virtual fluid particle;
Calculating the position of the virtual fluid particle based on the force; And
Generating the force arrangement based on the force and the position of the virtual fluid particle.
A method of providing tactile feedback comprising a.
The method of claim 6,
Obtaining the focus point based on the force arrangement,
Deriving the maximum force point based on the force arrangement;
Calculating a virtual focus point based on the maximum force point; And
Converting the virtual focus point into the focus point
A method of providing tactile feedback comprising a.
The method of claim 8,
The step of deriving a maximum force point based on the force arrangement,
Determining a point at which a maximum force acts in the force arrangement as the maximum force point; or
Determining a point at which the local maximum force acts in the force arrangement as the maximum force point
A method of providing tactile feedback comprising a.
The method of claim 8,
The step of calculating the virtual focus point,
Calculating at least one of a position of the virtual focus point, a force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point based on the maximum force point
A method of providing tactile feedback comprising a.
A sensor that detects an object;
Rendering the shape of the object based on the detection result, generating a virtual rigid particle based on the rendered shape, analyzing the interaction between the virtual rigid particle and the virtual fluid particle, the force of the virtual fluid on the object A processor that acquires a working focus point
Including,
The processor,
A tactile feedback providing device for acquiring the concentration point based on a maximum force point derived using a force array acting on the virtual rigid particle.
The method of claim 11,
Transmitter providing tactile feedback to the object based on the focus point
A tactile feedback providing device further comprising a.
The method of claim 12,
The transmitter,
Determining the characteristics of the wave based on the concentration point, and providing the tactile feedback by outputting the wave to the object
Device for providing tactile feedback.
The method of claim 11,
The processor,
Rendering the shape as a depth texture for the object based on depth information from the position sensing the object to the surface of the object
Device for providing tactile feedback.
The method of claim 11,
The processor,
Generates virtual rigid particles corresponding to a part of the surface of the object based on the rendered object
Device for providing tactile feedback.
The method of claim 11,
The processor,
Performing particle simulation between the virtual rigid particle and the virtual fluid particle to generate the force array acting on the virtual rigid particle, and obtaining the focus point based on the force array
Device for providing tactile feedback.
The method of claim 16,
The processor,
Calculate a force acting between the virtual rigid particle and the virtual fluid particle, calculate the position of the virtual fluid particle based on the force, and generate the force array based on the position and the force of the virtual fluid particle doing
Device for providing tactile feedback.
The method of claim 16,
The processor,
Deriving the maximum force point based on the force arrangement, calculating a virtual concentration point based on the maximum force point, and converting the virtual concentration point into the concentration point
Device for providing tactile feedback.
The method of claim 18,
The processor,
Determining a point where the maximum force acts in the force array as the maximum force point, or determining a point where the local maximum force acts in the force array as the maximum force point
Device for providing tactile feedback.
The method of claim 18,
The processor,
Calculating at least one of the position of the virtual focus point, the magnitude of the force acting on the virtual focus point, and a modulation frequency corresponding to the virtual focus point based on the maximum force point
Device for providing tactile feedback.

Images