EP4720824A2 - Haptic device for use with ar/vr systems - Google Patents

Haptic device for use with ar/vr systems

Info

Publication number
EP4720824A2
EP4720824A2 EP24816296.8A EP24816296A EP4720824A2 EP 4720824 A2 EP4720824 A2 EP 4720824A2 EP 24816296 A EP24816296 A EP 24816296A EP 4720824 A2 EP4720824 A2 EP 4720824A2
Authority
EP
European Patent Office
Prior art keywords
haptic
array
electroosmotic
controller
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP24816296.8A
Other languages
German (de)
French (fr)
Inventor
Craig Daniel SHULTZ
Joseph Mullenbach
Vivian Shen
Tucker RAE-GRANT
Christopher Harrison
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Carnegie Mellon University
Original Assignee
Carnegie Mellon University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carnegie Mellon University filed Critical Carnegie Mellon University
Publication of EP4720824A2 publication Critical patent/EP4720824A2/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014Hand-worn input/output arrangements, e.g. data gloves

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A haptic device uses a group of electroosmotic pumps arranged in an array to provide tactile feedback to a user of a virtual/augmented reality system. The electroosmotic pumps are arranged in defined patterns, forming an array of haptic pixels. The pumps pressurize a working fluid to cause a displacement of a flexible membrane in contact with the working fluid. A controller can be used to address individual pumps of the array and to provide simulate effects such as texture, object compliance, detents and buttons, and to provide haptic animations.

Description

CMU 2023-229 TITLE HAPTIC DEVICE FOR USE WITH AR/VR SYSTEMS CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefit under 35 U.S.C. § 119 of U.S. Provisional Application Serial No. 63/469,148, filed on May 26, 2023, which is incorporated herein by reference. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH [0002] Not applicable. BACKGROUND OF THE INVENTION [0003] The present disclosure generally relates to haptic devices. More specifically, the disclosure relates to a haptic device that can be used with virtual reality and augmented reality systems. [0004] Virtual and augmented reality (VR/AR) headsets are entering the consumer mainstream, with tens of millions of headsets already sold. These devices continue to make impressive strides in audio-visual immersion, bringing compelling virtual experiences to life. However, when users reach out to physically interact with these virtual worlds, the sense of touch falls flat. The most advanced consumer-grade controllers are inherently limited by their use of vibrotactile haptic actuators, which can only produce clicks and buzzes--an exceedingly small pallet of expressivity with which to represent the rich tactile world. [0005] Many approaches to overcoming the limitations of vibrotactile actuators have been proposed and implemented, including kinesthetic, thermal, electrotactile, and skin stretch actuation. More recently, however, considerable attention has focused on shape-changing pin and soft actuator arrays. When instrumented on the hands, these types of arrays hold the promise of enabling fine-grained tactile feedback via high density cutaneous haptic stimulation. This is because fingertips are highly innervated, with around 500 mechanoreceptors per fingertip, and the "neural coding" of these receptors is known to preserve incredible spatial and temporal detail, such as millimeter scale shape information and temporal "animation" information as high as 20Hz. [0006] While prior systems offer impressive tactile capability, there are serious limitations to their commercial impact and scale. Therefore, it would be advantageous to develop a haptic CMU 2023-229 device that provides fine-grained haptic feedback while demonstrating a lower cost and reduced weight compared to existing systems. BRIEF SUMMARY [0007] According to embodiments of the present disclosure is a haptic device comprising an array of ‘haptic pixels’ created from electroosmotic pumps that are capable of providing sensory signals to a user’s fingertips. The array of electroosmotic pumps can be embedded in a glove, with each finger of the glove having a separate array comprised of many haptic pixels. These small embedded electroosmotic pumps generate high pressures and fast dynamic flows. The use of electroosmotic pumps enables several orders of magnitude reduction in size compared to existing solutions, in addition to providing reduced power consumption, weight, and cost. BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS [0008] Figs.1A-1C show the haptic device, according to one embodiment. [0009] Fig.2A shows components of an array of electroosmotic pumps. [0010] Fig. 2B is a detailed view of a component of the array, with separate electrodes for individual pumps visible. [0011] Fig.3 shows a controller and electrical components of the device. [0012] Fig. 4 is a graph showing performance characteristics, with voltage, current and pressure shown. [0013] Fig.5 is another graph showing performance characteristics. [0014] Fig.6 is yet another graph showing performance characteristics. [0015] Fig.7 shows the configuration of haptic pixels of an array correlated to a virtual object. [0016] Fig.8 shows the activation of haptic pixels to depict compliance of a virtual object. [0017] Fig.9 depicts impulses of the haptic array, used to simulate slider detents. DETAILED DESCRIPTION [0018] According to embodiments of the disclosure is a haptic device 100 comprising an array 101 of electroosmotic pumps 102. Fig. 1A shows the device 100 implemented as a glove wearable by a user, with a total of five arrays 101 on the device 100. Each array 101 contains a plurality of electroosmotic pumps 102, which are referred to as ‘haptic pixels’ when implemented as a group in an array 101. In the example embodiment shown in Fig.1A, the device 100 includes an array 101 disposed on the distal end of each fingertip of the glove. A CMU 2023-229 controller 140 is located near the wrist of the glove and is capable of controlling each individual electroosmotic pump 102 of the arrays 101. Fig.1B shows a close-up view of the arrays 101 disposed on each fingertip. Fig. 1C shows a view of a single array 101 with a bi-stable clip 130 in an open position, where individual haptic pixels 102 are visible on the surface of the array 101. In Fig.1C, several pumps 102 are activated to cause a displacement from the surface of the array 101, forming a ring shape. [0019] Electroosmotic pumps 102 are a type of electrokinetic pump, meaning they directly generate fluid pressure and flow from an applied electric field. The pumps 102 achieve this by acting on charge densities in a fluid contained within the pump 102, which are a result of the spontaneous surface chemistry interactions between the pumping membrane 111 and the working fluid 112. This charge is pulled through the pumping membrane 111 by an applied electric field coming from a voltage on an external set of electrodes 113. The charge viscously couples to the rest of the fluid 112, causing bulk flow to occur. One advantage of an electroosmotic pump 102 is its ability to produce fast, reversible, and pulse-free flows in an exceedingly compact and lightweight form factor, and with applied voltages 20x less than other types of electrokinetic pumps. [0020] Fig. 2A is a diagram showing a plurality of electroosmotic pumps 102 arranged in a fingertip array 101. When in use, this array 101 sits beneath the user’s highly sensitive fingertip. These fingerpad arrays 101 can be duplicated (one per finger) into a glove device 100. As shown in Fig.2A, the array 101 comprises a silicone membrane 115 forming a contact surface for the user’s finger, a pump membrane 111, and top and bottom printed circuit boards containing a plurality of electrodes 113. Further shown are other components such as spacers 116, pressure sensitive adhesives 117, a fluid reservoir 118, and a reservoir cover 119. Optionally, spacers 116 can be a separate material, or it can be integrated into the part of the pumping membrane 111 by filling in the porosity of holes of the membrane 111, effectively blocking flow between pixels 102 and providing a solid sealing surface for the adhesives 117 and remaining housing. In an integrated configuration, the pumping membrane 111 can be filled by methods such as heat sealing, liquid adhesive filling and curing, high pressure crushing, or other methods. The pressure sensitive adhesives 117 may also be replaced by other adhesive systems and methods, such as heat bonding, thermoplastic adhesives, heat staking, fusion bonding, ultrasonic welding, epoxy bonding, or other methods. Fig. 2B is a close-up view of the printed circuit board, with electrodes 113 for each individual electroosmotic pump 102 visible. As shown in Fig.2B, 32 individual pumps 102 are disposed on a single fingertip array 101, with each pump 102 having a circular shape, in this example embodiment. CMU 2023-229 Depending on the expected application, however, a different number of haptic pixels 102 can be provided on each array 101 and the shape can be a shape other than a circle, that is, the pixels 102 can be an arbitrary shape, only defined by a 2D, flat geometry. Certain shapes and packing arrangements, such as squares or hexagons, and hex packing arrangements, being particularly useful for high packing densities of pixels 102. [0021] In the example embodiment shown in Fig. 1A, each pad array 101 is 5mm thick comprising the compliant display output surface (or surface membrane) 115, a laminated pump membrane 111, and a fluid reservoir 118. Both the display surface 115 and the fluid reservoir 118 can be cast from elastomers, such as skin-safe silicone (Smooth-On Ecoflex 00-30). In one embodiment, these components 115/118 are cast using laser-etched Delrin molds and adhered to the other components of the array 101 using silicone adhesive (Smooth-On Sil-Poxy). The pump array 101 comprises 32 laser-cut glass fiber filter (GFF) pump membranes 111 set into holes in a laser-cut polyethylene terephthalate (PET) spacer 116 and sandwiched on both sides by multilayer printed circuit boards with their electrodes 113 facing the interior of the assembly. Fluid 112 displaced by the pump 102 will displace the contact surface 115 directly above the pumping membrane 111, forming a physical deviation in the contact surface 115 perceptible by the user. [0022] Each electrode 113, which matches the size and spacing of each pump 102, or haptic pixel, is 1.6mm in diameter and is spaced with a 2.5mm horizontal pitch and a 2.36mm pitch along rows at 58° to the horizontal. This configuration yields a haptic pixel density of approximately 20 pumps/cm2. Generally, a horizontal distance of less than 3mm, but more than 0.5mm between the centers of adjacent holes would provide an acceptable pixel density for the fingertip, though other densities are more acceptable for other parts of the body, such at 1cm to 2cm in the hand and fingers. In general, a variable pixel density can be used which coarsely follows the innervation density of mechanoreceptors in the skin. [0023] The printed circuit boards containing the electrodes 113 can be adhered to the spacer layer 116 using laser-cut pressure-sensitive adhesive sheets 117 (3M 467MP), and the complete pump assembly (i.e. membrane 111, spacer 116, and electrodes 113) is 0.88mm thick. An ultraviolet laser (LPKF U4) can be used to process the spacers 116 and a CO2 laser (ULS VLS 4.60) can be used to process the pressure sensitive adhesive 117 and Delrin mold. Each printed circuit board has a 32-position surface-mount fat flex cable connector 120 (Molex 530480- 3200) soldered to its non-electrode side. Polyimide tape (3M 1205) applied across the connector vias on the electrode-sides of the board provides insulation between the connectors. Before use, propylene carbonate as the working fluid 112 can be injected into the reservoir 118 CMU 2023-229 using a hypodermic needle and the remaining air is evacuated using the same method. In addition to propylene carbonate, other fluids such as ethylene carbonate, isopropyl alcohol, acetone, deionized water, additional cyclic carbonates, acetonitrile, other high purity organic solvents, formahide, glycerol, any high dielectric strength, low ionic conductivity liquid or combinations of these liquids can be used as the working fluid 112. [0024] The pad array 101 can be mounted to the finger using a custom hinged finger clip 130, as shown in Figs. 1B-1C. The clip 130 is used to keep the pad array 101 in reliable contact pressure and position on the finger pad of the user, as this can increase performance. In the embodiment shown in Figs.1B-1C, the clip 130 comprises a pair of low spring constant rubber bands, or springs 131, (ex. 3/16" orthodonic bands) holding the array 101 and an adjacent contact pad 132 in compression. The pivoting design of the clip 130 also accommodates fingers of varying thicknesses. Above the fingernail, a small piece of soft silicone is used to improve comfort and to help maintain position. As shown in Fig.1C, the clip 130 is designed to be bi-stable, such that it flips and stays open. This allows a user to remove a fingerpad array 101 when not in use without removing the glove, which allows them to, for example, type on a keyboard or use a touchscreen. In one embodiment, two sizes of finger clips can be used-- one for the four fingers and one for the thumb. In the example embodiment shown in Figs.1B- 1C, a complete clip 130 with pad array 101 weighs 6.2g. [0025] The drive electronics, or controller 140, is designed to be modular and can expand from a single finger (32 haptic pixels) to a whole hand (160 haptic pixels). The controller 140, in one example embodiment, can be built around a Teensy 4.0 microcontroller and high voltage (300V) 64-channel shift register (Microchip HV507) module boards. Fig.3 depicts a general layout of the controller 140 and associated electronics. As shown in Fig.3, the controller 140 may comprise a microcontroller board 141, a high voltage DC/DC converter 142, and a single high voltage serial-to-parallel drive module 143. The drive module 143 may contain the shift register. The bottom portion of Fig. 3 shows the controller 140 and associated electronics assembled with two fingerpad arrays 101 powered by one high voltage drive module 143. The associated electronics may also include a wireless communication module and a power source, such as a battery. [0026] The microcontroller 140 communicates with the high voltage shift registers of the drive module 143 via a logic level converter using a low voltage digital serial peripheral interface (SPI) protocol. A total of six HV507 shift registers can be daisy-chained at once (two drivers per module board), corresponding to six possible pad arrays 101 being driven by a single microcontroller 140 (1 shift register per pad array). Each haptic pixel requires two output lines, CMU 2023-229 one connected to each pump electrode 113. Pumps 102 are off with a configuration of 0V/0V or 300V/300V, and they are driven in opposite directions (inflating or deflating) with configurations of 0V/300V and 300V/0V. [0027] Fully loaded, the microcontroller 140 can adjust 384 high voltage output lines, or 192 haptic pixels. For use in a glove embodiment of the device 100, however, the controller 140 typically drives a single pad array 101 (64 outputs lines/32 haptic pixels), or five pad arrays 101 (320 output lines/160 haptic pixels). The high voltage shift registers can be supplied power through an off-the-shelf adjustable high voltage capacitor charging DC/DC converter 142 and a 5V-to-300V flyback boost converter for a wireless implementation. As a safety measure, a high-side switch, controlled via firmware in the controller 140, can be used to disconnect power to the device 100 when not in use. The use of high channel count, high voltage shift registers, the development of a modular platform, and the custom 5V-to-300V DC/DC converter 142 (with latching and safety monitoring circuitry) permit low-weight, low-cost implementation of a haptic device 100 compared to prior attempts. [0028] The firmware translates the bits coming from the application software into an output state for the pad arrays 101. Application software may include, but is not limited to, a virtual environment running in Unity on a host PC, or a custom Java application running a demonstration and debugging platform. The firmware also initializes and controls various aspects of the high-voltage system. Updates to the pad arrays 101 are sent from the application layer to the microcontroller 140 using USB serial connection, running at a baud rate of 250,000 bits/second. A single command is structured with a pad array address (which finger to update) and a payload of 64 bytes (the data to update). Upon receipt, the firmware immediately updates the new pad array status. [0029] Pad arrays 101 are daisy-chained on a single SPI port, which runs at a clock of 8MHz. Fully loaded, this means an entire hand, 320 outputs, can be updated in 40µs. This firmware structure means pad array 101 behavior can be controlled directly at the application level. A command structure can be added to send additional device 100 commands (such as enable/disable the PSU), and data addressing can be implemented to route the data bytes to the correct pad array 101. [0030] In the glove embodiment of the device 100 depicted in Fig. 1A, the haptic fingerpad arrays 101 are duplicated to instrument every finger of the user. Comfort and the ease of donning and doffing are maximized to minimize any unwanted tactile stimuli. As the associated electronics are potentially the main source of interference with the natural CMU 2023-229 movement of the hands, their bulk and resistance to motion is reduced. Further, the controller 140 and drive electronics are mounted to the wrist of the glove 100, keeping finger mass low. [0031] In total, the glove device 100, including base glove, drive electronics/controller 140, haptic arrays 140, cables, and an overglove, weighs 147g. Including an off-the-shelf Raspberry Pi Zero 2 W for wireless operation and a 4.44Wh battery raises the mass to 207g. Assuming continuous presentation of haptic animations (186mW per array) to all fingers, and a measured system overhead power consumption from the Raspberry Pi and Teensy of 1.3W, it is estimated that total system battery life is roughly 2 hours for the five-finger wireless glove device 100 seen in Fig.1A. [0032] The performance characteristics of the device 100 compares favorably to other haptic devices. For reference, the ‘gold standard’ currently for these types of haptic displays calls for 400 pixels at 1mm pitch, which are capable of 1µm to 2mm displacements from 0Hz to 300Hz. One performance characteristic often analyzed is the pressure exerted against the skin of the user. In order to displace the skin, the array 101 should be able to apply enough hydraulic pressure to overcome the distributed pressing force coming from the finger. This pressure can often range from 10-30kPa for a light touch needed for contact patch spreading of the fingertip (<1N total applied force). This characteristic can be measured by hydraulically coupling a MEMS absolute pressure sensor to the top of individual pixels (i.e. pumps 102) and recording the response due to an applied voltage. [0033] Results of the pressure evaluation are shown in Fig.4, which depicts system response to a triangle wave. Frequency was fixed at 1Hz, and data was collected in 10s increments. Fig. 4 shows the current and pressure responses. Currents are generally in the 0.1mA peak range, while peak pressure is nearly 50kPa, above the maximum specified in the above-referenced standard. The response of the device 100 is overall relatively linear, with both current and pressure retaining the triangle wave shape. Further, the pressure transitions are especially fast. [0034] Surface deformation is another performance characteristic that has an impact on overall performance, as skin displacement directly impacts haptic performance. No-load displacement can be measured to evaluate best case deformation response. Actual displacement in use can be lower due to loading from the user’s skin. Acceptable displacement for haptic perception varies depending on mechanoreceptor type, frequency, and contact location, but generally displacements between 0.1-1.0mm are acceptable for low frequency (<1Hz) perception, while deformation in the 1µm range can be felt at high frequency (>100Hz). [0035] Fig.5 shows the results of displacement evaluation, where a 0.1Hz square wave of +/- 250V was applied to single pixels and corresponding current and pixel displacement (peak CMU 2023-229 height) was recorded. As shown in Fig. 5, most pixels underwent rapid inflation, achieving 0.3-0.5mm displacement within the first 0.5s. For reference, this is approximately the height of a standard Braille dot (0.48mm). This initial rapid filling phase corresponds to the pixel approaching a half-sphere shape, as the radius of the pixels is roughly 0.8mm. After this, deformation rate slows as the stiffness of the pump 102 greatly increases. On average, pixels reached max displacement of just over 1mm after 3 seconds. This level of deformation is sufficient for static indentation tasks, as well as for short-time (<1s) tactile animations. [0036] While the transient response from square wave inflations can be seen in Fig. 5, evaluation of high-frequency performance can be useful for stimuli such as buzzes and textures, where a human response is most sensitive to 10-300Hz frequencies. Fig. shows displacement recordings for seven different four-second long sine waveforms: 5, 10, 20, 40, 80, 160, and 320Hz. Applied voltage was fixed at +/-250V. As can be seen, displacement decreases systematically with increasing frequency. Notably, although the range of displacements at high frequency is small, approaching <10µm peak-to-peak, they are still easily within the range of human tactile perception, as humans are capable of sensing displacements under 2µm peak-to- peak at frequencies over 100 Hz. [0037] Finally, power consumption can be an important factor when the devices are powered by batteries or other wireless technologies. Power consumption of one actuator array 101 was recorded at 100Hz with the onboard voltage and current monitors under various conditions. The baseline power draw is 61mW at all times. These power characteristics compare favorably to other haptic technologies and permit wireless operation. [0038] The haptic device 100 is capable of a wide pallet of effects. Feedback can vary in both time and space, as well as interesting combinations of the two. In the following paragraphs, the capabilities of the device 100 are described through a VR design lens, focusing on the tactile properties of virtual objects one might want in VR/AR scenes. [0039] Object Contact and Impression Geometry [0040] The most basic, yet fundamental haptic effect conveyed to users in VR/AR systems is a sense that they have come into physical contact with a virtual surface or object. The high resolution of the arrays 101 permits the device 100 to convey partial and complex contact. To implement this technique, a series of small sphere colliders are arranged in virtual 3D space, being locked to the tracked position of the hand. These positions are automatically updated by hand tracking functionality of the system, and the locations of these colliders can be configured, in Unity or another virtual spatial environment software, to match the real-world geometry of the fingerpad haptic array 101. Hand tracking, which may be provided in the virtual CMU 2023-229 environment, can be accomplished using optical or electromagnetic sensors. For example, an electromagnetic sensor could include . Colliders are software objects that emulate physical contact. This is instantiated in Unity by a custom Unity prefab which is attached to any finger augmented with haptics (i.e., all five fingers in the glove implementation of the device 100). However this contact collider can be computed by the virtual environment by looking at the outer bounds of a virtual sphere and checking all other nearby virtual objects (via Euclidean distance) to see if the boundaries geometrically overlap. When users reach out to interact with a virtual object, the colliders are triggered based on the VR contact geometry. [0041] In the main loop of the Unity-based software, the device 100 checks the collision status of all haptic pixels or pumps 102. On geometry such as the edge or corner of a box, only a subset of sphere colliders are in contact and therefore triggered, creating the haptic impression of an edge or point, as shown in Fig.7. The objects (top row) and the corresponding array 101 configuration (bottom row) depicted in Fig. 7 include, from left to right, smooth marble, a rough and irregular rock, a rigid bottle, and a bumpy basketball. The state of all haptic pixels 102 is streamed over USB (60 FPS) or wirelessly to the device hardware, ensuring that users can translate their fingers over virtual geometry to explore features in real time. [0042] This contact-based cutaneous feedback can also create the sensation of grip of a virtual object. Here, the device 100 can use the Oculus hand tracking SDK to trigger a grasp of the object paired with the same collision logic as before, checking each haptic pixel 102 for contact with the virtual object, and conveying overlaps to the fingertip haptic arrays 101. [0043] Static Contact Texture [0044] A slight variant of object contact can be implemented to enable a new expressive dimension of contact texture. Instead of activating all pixels 102 that have collided with a virtual object, the device 100 applies additional logic to activate only a subset of haptic pixels 102 according to a predefined pattern that is stored as metadata for virtual objects. Fig.7 offers an example of a user gripping four objects with different contact textures, ranging from smooth, to rough, to ridged, to bumpy. [0045] Spatial Textures [0046] Imbuing virtual objects with spatially-varying textures adds a high degree of expressivity, which can be used to unlock truly immersive tactile AR/VR experiences. Two methods can be run on the device 100 for implementing textures. The first method uses predefined mathematical functions to generate haptic patterns correlated to function parameters. For instance, to create a ridged texture like corrugated metal, the device 100 can use a sine function with the appropriate coefficient that varies in response to a user’s lateral CMU 2023-229 motions. Other arbitrary mathematical functions can also be used. The device 100 can similarly create irregular, high-frequency textures like sandpaper using random or Perlin noise functions. [0047] The second method uses absolute spatial mapping, which can be more versatile and easier to design. In this second method, the device 100 attaches not only visual textures to objects in the virtual environment, but also invisible haptic texture layers based on the visuals. The transformation function is straightforward--when in collision with the textured surface, every contacting haptic pixel 102 uses the object’s haptic texture like a lookup table. Some haptic texture pixels correlate to the contacting haptic pixel being on, while other haptic pixels are off. This happens at 60 FPS, meaning that as a user’s hand translates across an object’s surface, the textural haptic effect properly translates and scales with velocity. Note, this limits temporal texture information to 30Hz due to Nyquist, however this more than enough bandwidth for the textures. [0048] Haptic Animations [0049] The arrays 101 of the device 100 not only have high spatial resolution, but also offer high frequency response. The high frequency response can be utilized to play haptic "animations" on the finger arrays 101, which is particularly useful in creating object-bound and environmental effects. For example, haptic animations can include fan wind (directional swiping animations), water drops (haptically actuating a quickly expanding sphere), and electrical sparks (using random high-frequency pixel actuation). There are also certain classes of objects that generate haptic effects on a statically held finger, such as touching a finger to a running motor (synchronized 10Hz oscillation rendered on all contacting haptic pixels. [0050] Object Compliance [0051] The device 100 can imbue virtual objects with varying levels of compliance, adding to the canvas of effects that can be applied to create rich virtual worlds. For this, the device 100 varies contact area in response to compression. Put simply, the more a user compresses a virtual object, the more haptic pixels 102 that are activated, radiating outward from the point of contact, increasing pressure integrated on the fingertips. The actuation is spatially mapped to how far the user’s finger is pressed into the virtual object, so compliance properly scales with velocity, like in spatial textures. Fig. 8 provides an illustrative example of this progression. For softer compliance, such as the virtual spring in Fig.8, more haptic pixels 102 are gradually actuated as the user’s fingers descend into the material. The rate of change affects the perception of compliance. [0052] UI Widget Haptics CMU 2023-229 [0053] Finally, a specialized but high-value haptic effect is that of "clicks"--transient impulses and other event-driven effects that add realism and useful confirmatory feedback to user interface widgets. Two "click" examples include a brief full-array 200ms impulse when a button passes a depression threshold (e.g.5mm) and a scroll/slide bar can offer "detents" when passing specific values or elements. The “detents” effect is shown in Fig.9, where brief (e.g. 200ms) impulses are activated for each detent. Thus, when a user slides their finger a distance from the slider origin, the impulse will activate to simulate each detent. As another example, the device 100 can provide buttons with a buckling-spring effect. More specifically these buttons exhibit some compliance (see previous section), allowing the finger to depress the button approximately 1cm before the spring buckles and the button surface snaps down, turning off the haptic array and removing any finger pressure. [0054] When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps, or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components. [0055] The invention may also broadly consist in the parts, elements, steps, examples and/or features referred to or indicated in the specification individually or collectively in any and all combinations of two or more said parts, elements, steps, examples and/or features. In particular, one or more features in any of the embodiments described herein may be combined with one or more features from any other embodiment(s) described herein. [0056] Protection may be sought for any features disclosed in any one or more published documents referenced herein in combination with the present disclosure. Although certain example embodiments of the invention have been described, the scope of the appended claims is not intended to be limited solely to these embodiments. The claims are to be construed literally, purposively, and/or to encompass equivalents.

Claims

CMU 2023-229 CLAIMS What is claimed is: 1. A haptic device comprising: a array comprising: a plurality of electroosmotic pumps, a working fluid contained within the plurality of electroosmotic pumps, and a flexible membrane capping the plurality of electroosmotic pumps and contacting a portion of the working fluid, wherein the flexible membrane has an outer surface adapted to abut skin of a user; and a controller electrically connected to the plurality of electroosmotic pumps, wherein activation generates an increase in a pressure of the working fluid to cause distention of the flexible membrane associated with an activated electroosmotic pump of the plurality of electroosmotic pumps. 2. The device of claim 1, wherein the working fluid is contained only within the array. 3. The device of claim 1, wherein the controller is electrically connected to the plurality of electroosmotic pumps through a flexible wiring network. 4. The device of claim 1, wherein the array further comprises a reservoir in fluid communication with each electroosmotic pump of the plurality of electroosmotic pumps. 5. The device of claim 1, wherein the plurality of electroosmotic pumps comprises: a pumping membrane defining a grouping of holes arranged in a pattern; an individual electrode associated with each hole in the grouping of holes, wherein each individual electrode is in electrical communication with the controller. 6. The device of claim 5, wherein the grouping of electrodes are disposed on a printed circuit board. 7. The device of claim 5, wherein the grouping of holes has a distance between a center of adjacent holes of less than 3mm. CMU 2023-229 8. The device of claim 1, wherein a density of the grouping of holes is varied for use on different body parts of the user. 9. The device of claim 1, further comprising: a clip holding the array and a contact pad adapted to press against a user’s fingernail, wherein the array and the contact pad are held in compression against a user’s finger via a spring. 10. The device of claim 9, wherein the spring comprises a low spring constant rubber band connected at a first end to the array and a second end to the contact pad, wherein the clip further comprises a pivotable hinge adapted to stably occupy an open or a closed position. 11. The device of claim 1, wherein the controller comprises: a controller board, a high voltage DC/DC converter, and a high voltage drive module. 12. The device of claim 11, wherein the controller comprises: a communication link to a virtual environment, wherein the virtual environment includes object tracking and hand tracking. 13. The device of claim 12, wherein the hand tracking is provided by optical or electromechanic sensing. 14. The device of claim 12, wherein the communication link is wireless. 15. The device of claim 11, wherein the high voltage DC/DC converter includes a latching switch is adapted to disconnect power to the array upon command from the controller. 16. The device of claim 1, wherein each electroosmotic pump is capable of displacing the flexible membrane in a range of 1µm to 2mm 17. The device of claim 1, wherein each electroosmotic pump is capable of activating and deactivating in response to a signal from the controller at a frequency of 0Hz to 300Hz. CMU 2023-229 18. The device of claim 1, wherein each electroosmotic pump is capable of pressuring the working fluid to at least 10 kPa. 19. A method of controlling a haptic device having a plurality of haptic pixels arranged in an array, the method comprising: activating one or more of the plurality of haptic pixels in the array, wherein each activated haptic pixel corresponds to a contact point with a virtual object. 20. The method of claim 19, wherein the one or more activated haptic pixels corresponds to a pre-defined pattern associated with a static contact texture. 21. The method of claim 19, wherein the one or more activated haptic pixels is changed over a period of time to form a haptic pattern. 22. The method of claim 21, wherein the haptic pattern comprises a mathematical function that varies in response to a user’s lateral motion. 23. The method of claim 21, wherein the haptic pattern comprises a series of textures correlated to a lookup table associated with a texture. 24. The method of claim 19, wherein the one or more activated haptic pixels is changed at a frequency. 25. The method of claim 19, wherein the one or more activated haptic pixels increases or decreases over a time period to simulate a changing contact area with a virtual object. 26. The method of claim 19, wherein the one or more activated haptic pixels increases or decreases over a time period to simulate a changing level of grip on a virtual object. 22. The method of claim 19, wherein the one or more activated haptic pixels is cycled to an inactivated state in pulses ranging up to 200 ms.
EP24816296.8A 2023-05-26 2024-05-28 Haptic device for use with ar/vr systems Pending EP4720824A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363469148P 2023-05-26 2023-05-26
PCT/US2024/031294 WO2024249434A2 (en) 2023-05-26 2024-05-28 Haptic device for use with ar/vr systems

Publications (1)

Publication Number Publication Date
EP4720824A2 true EP4720824A2 (en) 2026-04-08

Family

ID=93658863

Family Applications (1)

Application Number Title Priority Date Filing Date
EP24816296.8A Pending EP4720824A2 (en) 2023-05-26 2024-05-28 Haptic device for use with ar/vr systems

Country Status (2)

Country Link
EP (1) EP4720824A2 (en)
WO (1) WO2024249434A2 (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1146230A (en) * 1997-03-20 1999-02-16 Northern Telecom Ltd Personal communication equipment and method for signaling call-processing state
US7009595B2 (en) * 2002-01-03 2006-03-07 United States Of America Extended refreshable tactile graphic array for scanned tactile display
EP3803324A4 (en) * 2018-06-01 2023-02-22 S2 Genomics, Inc. METHOD AND DEVICE FOR PROCESSING TISSUE SAMPLES
US20250028394A1 (en) * 2021-08-16 2025-01-23 Carnegie Mellon University System and method for an interaction surface with shape-changing tactile elements

Also Published As

Publication number Publication date
WO2024249434A3 (en) 2025-04-24
WO2024249434A2 (en) 2024-12-05
WO2024249434A9 (en) 2025-07-17

Similar Documents

Publication Publication Date Title
US8847741B2 (en) System and method for display of multiple data channels on a single haptic display
US8570296B2 (en) System and method for display of multiple data channels on a single haptic display
US9405369B2 (en) Simulation of tangible user interface interactions and gestures using array of haptic cells
US10048758B2 (en) Haptic feedback for interactions with foldable-bendable displays
US20110140870A1 (en) Haptic Feedback Device Using Standing Waves
US9639158B2 (en) Systems and methods for generating friction and vibrotactile effects
KR20150114899A (en) Wearable device with flexibly mounted haptic output device
JP2018109982A (en) Single sensor and haptic actuator
JP6594731B2 (en) Tactile output device
US20190265796A1 (en) Systems and Methods for Providing Electrostatic Haptic Effects via a Wearable or Handheld Device
KR20100020065A (en) Haptic module using electro-rheological fluid, control method thereof and haptic device therewith
KR20180110475A (en) Apparatus for transparent haptic interface for generating tactile texture
KR101524906B1 (en) Apparatus for generating tactile sensation, dielectricpolymer high-perpormance driver, actuator, interfacing apparatus, apparatus for providing tactile feedback using the same
KR102322078B1 (en) Haptic display device and method for driving the same
WO2024249434A2 (en) Haptic device for use with ar/vr systems
KR101466226B1 (en) Flexible complex haptic generating module
US20250028394A1 (en) System and method for an interaction surface with shape-changing tactile elements
KR20090122516A (en) Passive haptic drive using rheological fluid, haptic device having same and haptic providing method using same
US12073024B1 (en) Systems and methods of controlling activation of physical or virtual keyboard keys using an adjustable electrostatically-controlled actuator, and methods of manufacturing electrostatically-controlled actuators for use therewith
KR20160080751A (en) Haptic display device and method for driving the same
Brucker Technology for Tactile Tangibles
KR20100064493A (en) Haptic apparatus for providing feedback and method for providing haptic feedback using the same

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20251217

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR