KR20170069936A - Systems and methods for position-based haptic effects - Google Patents

Abstract

One exemplary system disclosed herein includes a sensor configured to detect a gesture and transmit an associated sensor signal. The gesture includes a first position at a distance from the surface and a second position in contact with the surface. The system also includes a processor in communication with the sensor, wherein the processor is configured to receive the sensor signal from the sensor and to determine one or more haptic effects based at least in part on the sensor signal. The one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture. The processor is also configured to generate one or more haptic signals and at least one haptic signals based at least in part on the one or more haptic effects. The system includes a haptic output device that receives one or more haptic signals and outputs one or more haptic effects.

Description

[0001] SYSTEM AND METHODS FOR POSITION-BASED HAPTIC EFFECTS [0002]

The present invention relates to the field of user interface devices. More particularly, the present invention relates to position based haptic effects.

The quality of the interfaces that people use to interact with computer-based systems is becoming increasingly important. To create more intuitive and improved user experiences, such systems may use visual, audio, and / or haptic feedback to reproduce aspects of the interactions in the physical world. This feedback is often generated by the user touching the touch screen display or other user interface controls. For example, a computer-based system may generate haptic feedback when a user contacts a virtual button on a touch screen display. However, contact-based haptic feedback may not accurately simulate interaction with real-world objects. Accordingly, there is a need for additional systems and methods for providing haptic feedback.

Embodiments of the present disclosure include computing devices configured to generate location based haptic effects. In one embodiment, the system of the present disclosure may comprise a sensor configured to detect a gesture and transmit a sensor signal associated with the gesture. The gesture can include at least two positions, wherein the first of the at least two positions comprises a distance from the surface and the second of the at least two positions comprises the contact with the surface. The system may also include a processor in communication with the sensor. The processor may be configured to receive the sensor signal from the sensor. The processor may also be configured to determine one or more haptic effects based at least in part on the sensor signal. The one or more haptic effects may be configured to provide substantially continuous haptic feedback throughout the gesture. The processor may be further configured to generate one or more haptic signals and to transmit the one or more haptic signals based at least in part on the one or more haptic effects. The system may further include a haptic output device in communication with the processor. The haptic output device may be configured to receive one or more haptic signals and output one or more haptic effects.

In another embodiment, a method of the present disclosure may comprise receiving a sensor signal associated with a gesture from a sensor, wherein the gesture includes at least two positions, wherein a first one of the at least two positions And the second of the at least two locations comprises the contact with the surface. The method may also include determining at least one haptic effect based at least in part on the sensor signal. The one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture. The method may further include generating one or more haptic signals based at least in part on the one or more haptic effects, and transmitting the one or more haptic signals to the haptic output device. The haptic output device may be configured to receive one or more haptic signals and output one or more haptic effects. Yet another embodiment includes a computer readable medium for implementing the method.

These exemplary embodiments are not intended to limit or limit the scope of the present invention, but rather to provide examples to assist in the understanding thereof. Exemplary embodiments are discussed in the Detailed Description for carrying out the invention, and further description is provided therein. Advantages provided by the various embodiments may also be understood by reference to the specification and / or by one or more embodiments of the claimed subject matter.

The details of what is possible are described in more detail in the remainder of the specification. The specification refers to the following drawings.
1A illustrates one embodiment of a system for providing location based haptic effects;
1B illustrates another embodiment of a system for providing position based haptic effects;
2 is a block diagram illustrating a system for providing location based haptic effects;
Figure 3 illustrates one embodiment of a system for providing location based haptic effects;
Figure 4 illustrates another embodiment of a system for providing location based haptic effects;
Figure 5 illustrates another embodiment of a system for providing location based haptic effects;
6 is a flowchart of steps of performing a method of providing location based haptic effects, in accordance with one embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The various and alternative exemplary embodiments and the accompanying drawings will now be described in detail. Each example is provided as an illustration, not a limitation. It will be apparent to those of ordinary skill in the art that modifications and variations can be made. For example, features illustrated or described as part of one embodiment may be used in other embodiments to produce other additional embodiments. It is therefore intended that the present disclosure cover all such modifications and variations as fall within the scope of the appended claims and their equivalents.

An Illustrative Example of Location-Based Haptic Effects

One exemplary embodiment of the present disclosure includes a mobile device (e.g., a smart phone, tablet, e-reader, etc.). The mobile device includes a touch screen display configured to output a graphical user interface (GUI) having one or more virtual buttons. The GUI may include, for example, a home screen that includes a plurality of icons associated with programs stored on the mobile device. A user may interact with a virtual button, for example, to execute a program or otherwise provide input to a mobile device.

In an exemplary embodiment, the mobile device is configured to provide substantially continuous haptic feedback to a user when the user approaches and, ultimately, contacts the touch screen display, e.g., to interact with a virtual button. For example, in an exemplary embodiment, the mobile device is configured to detect a distance between the user's finger and the touch screen display. As the user's fingertips are getting closer and closer to the virtual button output on the touch screen display, the mobile device is able to substantially remotely manipulate remote haptic effects (e.g., pneumatic, ultrasonic pressure waves, and / or laser beam stimulation) And is configured to output continuously. In some embodiments, the mobile device outputs remote haptic effects whose intensity gradually increases as the user approaches the touch screen display. This can simulate a spring force or other type of resistance, typically associated with an actual physical button. In addition, in an exemplary embodiment, the mobile device is configured to output a local haptic effect (e.g., a click feeling) in response to a user contacting the virtual button via a touch screen display. Local haptic effects can simulate the operation of physical buttons. In such an embodiment, providing a continuous haptic feedback when a user approaches and ultimately contacts a virtual button output on the touch screen display more accurately simulates the actual interaction with the physical button and / or otherwise The user interface can provide a three-dimensional feel to the two-dimensional user interface.

In another exemplary embodiment, the mobile device includes a GUI that includes a plurality of user interface levels that can interact with a user. When the user approaches and ultimately contacts the touch screen display, the mobile device is configured to activate different user interface levels. A user may interact with a particular user interface level by performing a gesture at a distance (from the mobile device) associated with the user interface level.

For example, in some embodiments, the mobile device may execute a medical simulation that includes a GUI having a plurality of user interface levels. Each user interface level may be configured to output images of different sides of a body part (e.g., a person's arm) and / or to provide information associated with them. For example, the mobile device may activate the first user interface level in response to detecting that the user's finger is a long distance from the mobile device. The first user interface level may output an image of the outside of the body part (e.g., skin, hair, etc.). The mobile device may activate the second user interface level in response to detecting that the user's finger is at a medium distance from the mobile device. The second user interface level may output images of tissues and / or ligaments associated with body parts where key features are labeled. The mobile device may activate the third user interface level in response to the user contacting the mobile device. The third user interface level may output an image of the bone associated with the body part. As such, when the user accesses and / or contacts the mobile device, the mobile device activates the "deeper" user interface levels, where the user can gradually access more and / or different information . This, in addition or alternatively, can provide a three-dimensional feel to a two-dimensional interface if not otherwise.

In some embodiments, when the user approaches and contacts the touch screen display, the mobile device is configured to provide substantially continuous haptic feedback associated with the user interface levels. For example, in the medical simulation example described above, when a user approaches and touches the touch screen display, the mobile device is configured to simulate a substantially continuous haptic device configured to simulate pushing the finger deeper through different layers of the actual body part. Feedback can be output. In this embodiment, the mobile device may output a haptic effect configured to simulate skin and / or hair, in response to detecting that the user's finger is at a distance associated with the first user interface level. The mobile device may output a haptic effect configured to simulate the tissue and / or ligament in response to detecting that the user's finger is at a distance associated with the second user interface level. The mobile device may output a haptic effect configured to simulate the bone in response to detecting that the user's finger has contacted the mobile device. For example, a combination of haptic effects, provided in a substantially continuous manner, can more realistically simulate real-world interactions with body parts.

The above description of the exemplary embodiments is provided as an example only. Various other embodiments of the invention are described herein and variations of these embodiments will be understood by those of ordinary skill in the art. Advantages provided by the various embodiments may also be understood by reference to the specification and / or by one or more embodiments of the claimed subject matter.

An exemplary system for position-based haptic effects

Figure 1a shows an embodiment of a system for providing location based haptic effects. In this embodiment, the system includes a computing device 100 configured to provide substantially continuous haptic feedback when a user approaches and is ultimately in contact with the computing device 100. Additionally or alternatively, the computing device 100 may be configured to provide substantially continuous haptic feedback when the computing device 100 approaches and ultimately contacts the user. As used herein, substantially continuous haptic feedback includes one or more haptic effects that are provided throughout the duration of the gesture performed by the user.

A gesture is any movement and / or positioning of a body and / or physical object that conveys meaning or user intent. It will be appreciated that simple gestures can be combined to form more complex gestures. For example, touching a finger with a surface may be referred to as a "finger on" gesture, while detaching a finger from a surface may be referred to as a separate "finger off" . If the time between the "finger on" gesture and the "finger off" gesture is relatively short, the combined gesture can be referred to as "tapping"; If the time between the "finger on" gesture and the "finger off" gesture is relatively long, the combined gesture can be referred to as "long tapping"; If the distance between the two-dimensional (x, y) positions of the "finger on" gesture and the "finger off" gesture is relatively large, the combined gesture can be referred to as "swiping"; If the distance between the two-dimensional (x, y) positions of the "finger on" gesture and the "finger off" gesture is relatively small, the combined gesture may be "smearing", "smudging" May be referred to as "flicking. &Quot; The gestures can, in addition or alternatively, be three-dimensional. For example, a gesture can include locating a body part and / or a physical object at a specific location in the real space. In some embodiments, if the distance between three-dimensional (x, y, z) positions during finger movement (e.g., in real space) is relatively large, the combined gesture may be referred to as "swiping" have. If the distance between the three-dimensional (x, y, z) positions during finger movement is relatively small, the combined gesture can be referred to as "smearing", "smoothing", or "flicking". Any number of simple or complex two-dimensional or three-dimensional gestures can be combined in any manner to form any number of different gestures. The gesture may also be any type of movement or positioning (e.g., of a body part or physical object) that is recognized by the computing device 100 and converted into electronic signals. These electronic signals can activate a haptic effect, such as substantially continuous haptic feedback, where the position sensor 104 captures the intent of the user to generate the haptic effect.

In some embodiments, the computing device 100 provides substantially continuous haptic feedback to guide the user to the location of a particular virtual object (e.g., an enabled virtual button) output on the touch screen display 102 . For example, in such an embodiment, the computing device 100 may output a remote haptic effect (e.g., via the haptic output device 106) in response to detecting that the user is located on a particular virtual object . When the user is not in contact with the computing device 100, the remote haptic effect may be perceivable by the user. The remote haptic effect can help the user to initially locate the virtual object. In this embodiment, the computing device 100 is further configured to continuously output remote haptic effects to the user when the user makes a gesture toward the touch screen display 102 while remaining positioned over the virtual object. This may notify the user that the user is accessing the virtual object. In this embodiment, the computing device 100 is also configured to output a local haptic effect in response to the user detecting touching the virtual button via the touch screen display 102. [ When the user is in contact with the computing device 100, the local haptic effect may be perceivable by the user. The local haptic effect may notify the user that the user has interacted with the virtual object. Continuous haptic feedback when the user hovers over, approaches, and ultimately contacts the virtual object can guide the user to the virtual object, for example, without the user having to look at the touch screen display 102 .

As a more specific example, in some embodiments, the computing device 100 may generate a gesture at a first distance from the computing device 100 (e.g., within the distance range 112 from the computing device 100) (E.g., via the haptic output device 106) in response to detecting the first haptic effect. In this embodiment, the computing device 100 is configured to detect a gesture at a second distance from the computing device 100 (e.g., within the distance range 114 but not in contact with the computing device 100) (E.g., via the haptic output device 108) in response to the first haptic effect. In addition, in such an embodiment, the computing device 100 is capable of communicating at a third distance from the computing device 100 (e.g., in contact with the computing device 100 and / or the touch screen display 102) In response to detecting the gesture, output a third haptic effect (e.g., via the haptic output device 110). The first haptic effect, the second haptic effect, and / or the third haptic effect may be the same or different from each other. For example, the first haptic effect may include air flushing, and the second haptic effect may include a static ESF effect (e.g., as described in more detail in connection with FIG. 2) May include vibration.

In some embodiments, the computing device 100 is configured to include at least two characteristics of haptic effects (e.g., waveform, type, amplitude, duration of the output, duration of the output, , Frequency, and / or time). For example, the computing device 100 may configure the characteristics of the first haptic effect, the second haptic effect, and the third haptic effect, such that the transition between haptic effects is smooth to the user. This may provide the user with a substantially consistent haptic experience, for example throughout the gesture. For example, it may provide a substantially consistent haptic experience to a user as the user approaches the computing device 100, moves in contact with it, and / or moves away from it. In some embodiments, the computing device 100 constructs the characteristics of the at least two haptic effects so that the at least two haptic effects are clearly distinguishable to the user. For example, computing device 100 may configure the characteristics of a first haptic effect, a second haptic effect, and a third haptic effect such that at least one transition between haptic effects is discernible to the user have. This may provide information to the user, such as the distance between the body part of the user and the computing device 100.

Embodiments may output haptic effects of any number and configuration to provide substantially continuous haptic feedback throughout the gesture. For example, in some embodiments, the computing device 100 may output a fourth haptic effect in response to detecting another gesture within a different distance range from the computing device 100, Outputs the fifth haptic effect and the sixth haptic effect in response to detecting another gesture within the distance range, and the like.

Although the embodiments discussed above relate to a user accessing and ultimately contacting a computing device 100, some embodiments may include at least one (e.g., one or more) And one position in which the user is in contact with the surface. ≪ RTI ID = 0.0 > [0034] < / RTI > For example, some embodiments may allow a user to access the computing device 100 with his / her fingers, contact the computing device 100 with his / her fingers, and subsequently, substantially during the gesture of lifting his / her fingers away from the computing device 100 It is possible to provide continuous haptic feedback. As such, substantially continuous haptic feedback can be initiated and ended by the user not contacting the computing device 100.

As another example, some embodiments provide substantially continuous haptic feedback throughout a gesture that begins after the user contacts the computing device 100 and ends after the user lifts the finger away from the computing device 100. [ For example, a user may interact with the surface of the computing device 100 (e.g., the touch screen display 102) with a body part. The computing device 100 may detect user interaction and output a local haptic effect. The user may then lift the body part off of the computing device 100 (e.g., when the user has finished interacting with the computing device 100). In some embodiments, computing device 100 may provide haptic feedback (e. G., A remote haptic effect < / RTI > The user can continue to output. For example, the computing device 100 may output such haptic feedback until the distance between the user ' s body part and the computing device 100 exceeds a threshold value. As such, substantially continuous haptic feedback is initiated while the user is in contact with the computing device 100 and may end while the user is not in contact with the computing device 100.

In some embodiments, the computing device 100 may use the same class of haptic effects (e.g., static ESF haptic effects) to output different sensations to the user, e.g., at different locations throughout the gesture. For example, the computing device 100 may be configured to respond to a first static ESF effect (e.g., configured to simulate sub-amplitude vibrations) in response to detecting a gesture within a distance range 112 from the computing device 100, A second static ESF effect (e.g., configured to simulate medium size vibration), and / or a gesture that includes contacting the computing device 100 in response to detecting a gesture within the distance range 114 from the computing device 100 And outputting a third static ESF effect (e.g., a higher magnitude vibration) in response to the detection.

1B illustrates another embodiment of a system for providing location based haptic effects. In this embodiment, the system includes a computing device 100 configured to output (e.g., via a touch screen display 102) a user interface that includes a plurality of user interface levels that can interact with a user. The user interface may include any number of user interface levels. In addition, the computing device 100 may associate any number and / or configuration of user locations with any number and / or configuration of user interface levels throughout the gesture. For example, in some embodiments, the mobile device may associate three different user locations with the same user interface level throughout the gesture. In some embodiments, the user interface levels may be simulated as overlaying each other, for example, to provide a simulated depth to the user interface. In some embodiments, the computing device 100 is configured to execute one or more functions associated with each user interface level.

In the embodiment shown in FIG. 1B, the computing device 100 is outputting a virtual map associated with the navigation application. When the user approaches (and / or contacts) the computing device 100 (e.g., as shown by the dashed line), the computing device 100 is configured to cycle through one or more user interface levels, To provide details of the different levels associated with the user. For example, in response to a user placing a finger within a distance range 112 from the computing device 100, the computing device 100 may be configured to provide a first user You can activate the interface level. When the user moves his or her finger from the computing device 100 into the second distance range 114 but not in contact therewith, the computing device 100 determines the details, such as, for example, a town or street name, Lt; RTI ID = 0.0 > user interface < / RTI > In response to the user contacting the computing device 100, the computing device 100 may activate a third user interface level configured to zoom in, e.g., at a particular location on the virtual map. As such, the computing device 100 may activate different user interface levels associated with the virtual map, in response to the user detecting that the gaming device is performing gestures within different distance ranges from the computing device 100.

In some embodiments, the computing device 100 is configured to determine a haptic effect to output based at least in part on a user interface level and / or gesture. These haptic effects can be configured, for example, to notify a user that a particular user interface level has been interacting with the user and / or that the user has switched between user interface levels. For example, the computing device 100 may, in response to activating the first user interface level, output a first haptic effect that is distinctly distinct from the second haptic effect associated with activating the second user interface level . The distinguishable haptic effects may inform the user that the user has switched between user interface levels.

In some embodiments, the computing device 100 is configured to output haptic effects configured to indicate the accuracy with which the computing device 100 detected the gesture. For example, when a gesture is performed at a greater distance from the computing device 100, the computing device 100 may detect the gesture (e.g., a portion of the gesture, such as a user's particular location) can do. For example, when a gesture is performed at a distance of one meter from the computing device 100, the computing device 100 may detect the location associated with the gesture within one centimeter of the actual location. When the gesture is at a distance of one centimeter from the computing device 100, the computing device 100 may detect the position associated with the gesture within one millimeter of the actual position. In some embodiments, the computing device 100 outputs a haptic effect that the user perceives as ambiguous in response to detecting the gesture with less precision. The computing device 100 outputs a haptic effect that the user perceives as distinct in response to detecting the gesture with higher precision. In some embodiments, the computing device 100 outputs a haptic effect to a larger surface area of the user's body in response to detecting the gesture with less precision. The computing device 100 may output the haptic effect to a smaller surface area of the user's body in response to detecting the gesture with greater precision.

2 is a block diagram illustrating a system for position based haptic effects, in accordance with one embodiment. The computing device 201 may be a computing device such as a laptop computer, a desktop computer, a game controller, a game pad, a remote control, a medical device, an automotive computing device (e.g., a stereo, HVAC, lighting, navigation, A computer for controlling automotive systems or devices), a wand, a stylus, a pen, and / or a portable game device. In some embodiments, computing device 201 may be a wearable device (e.g., a ring, a shoe, an armband, a sleeve, a jacket, a spectacle, a glove, a wrist watch, a wristband, a bracelet, a garment, a hat, a headband, And is configured to be worn by a user and / or coupled to a user ' s body.

In some embodiments, components of computing device 201 (e.g., processor 202, network interface device 210, haptic output device 218, sensors 230, position sensor 232, (216), etc.) may be integrated within a single housing. In other embodiments, components may be distributed (e.g., between multiple housings or locations) and in electrical communication with one another. The computing device 201 may or may not include all of the components shown in FIG. For example, in some embodiments, the computing device 201 may not include the sensor 230.

The computing device 201 includes a processor 202 for interfacing with other hardware via a bus 206. The memory 204, which may include any suitable tangible (and non-volatile) computer readable medium such as RAM, ROM, EEPROM, etc., may implement program components that constitute the operation of the computing device 201. In some embodiments, computing device 201 may further include one or more network interface devices 210, input / output (I / O) interface components 212, and additional storage 214.

The network interface device 210 may represent one or more of any components that facilitate network connectivity or may facilitate communication between electronic devices in other manners. Examples include a wired interface such as Ethernet, USB, IEEE 1394, and / or a radio interface for accessing IEEE 802.11, Bluetooth, near-field communication (NFC) (E.g., a transceiver / antenna for accessing CDMA, GSM, UMTS or other mobile communication networks).

I / O components 212 may be used to input or output data, one or more displays, touch sensitive surfaces 216, keyboards, mice, speakers, microphones, buttons, and / Lt; RTI ID = 0.0 > hardware, e. The storage 214 represents a non-volatile storage such as read-only memory, flash memory, ferroelectric RAM (F-RAM), magnetic, optical, or other storage media included in or coupled to the computing device 201 .

The computing device 201 may include a touch sensitive surface 216. In some embodiments, the touch sensitive surface 216 is soft or deformable. The touch sensitive surface 216 represents any surface that is configured to sense the tactile input of the user. The one or more touch sensors 208 detect a touch in the touch region (e.g., when an object such as a user's finger or stylus contacts the touch sensitive surface 216) and transmit signals associated with the touch to the processor 202 . Any suitable number, type or arrangement of touch sensors 208 may be used. For example, in some embodiments, resistive and / or capacitive sensors are embedded in the touch sensitive surface 216 and are used to determine other information, such as the position of the touch and the pressure, speed, and / or direction of the touch Can be used. In some embodiments, capacitive sensors may detect proximity of the user's finger to the touch sensor 208 (e.g., embedded in the touch sensitive surface 216). For example, the touch sensor 208 may include a capacitive sensor configured to detect a change in capacitance as the user's finger approaches the touch sensor 208. [ The touch sensor 208 can determine whether the user's finger is within a certain distance from the touch sensor 208 based on the change in capacitance.

The touch sensor 208 may additionally or alternatively include other types of sensors. For example, optical sensors along with a view of the touch sensitive surface 216 may be used to determine the touch location. As another example, the touch sensor 208 may include a light emitting diode (LED) finger detector mounted on the side of the display. In some embodiments, the touch sensor 208 may be configured to detect multiple aspects of user interaction. For example, the touch sensor 208 may detect the speed, pressure, and direction of user interaction and may include this information in the signal transmitted to the processor 202.

In some embodiments, the computing device 201 includes a touch sensitive display 216 that combines a touch sensitive surface 216 and a display of the device. The touch sensitive surface 216 may correspond to one or more material layers on the outside of the display or components of the display. In other embodiments, the touch sensitive surface 216 may not (or otherwise correspond to) the display, depending on the particular configuration of the computing device 201. [

In some embodiments, the computing device 201 includes one or more sensor (s) The sensor (s) 230 are configured to transmit the sensor signals to the processor 202. The sensor (s) 230 may include, for example, a range sensor, a depth sensor, a biosensor, a camera, and / or a temperature sensor. The computing device 201 may determine one or more haptic effects based, at least in part, on the sensor signals from the sensor 230. For example, in some embodiments, the computing device 201 may use data from the sensor (s) 230 to determine one or more characteristics of a user's body part to which the haptic effect will be applied. The computing device 201 may determine one or more characteristics of the haptic effect based on the body part, for example, to improve the quality of the perceived haptic effect and / or to prevent injury to the user.

For example, in some embodiments, the computing device 201 executes a game, such as a war game. The computing device 201 may be configured to output a haptic effect (e.g., a remote haptic effect) in response to a game event, such as a virtual explosion. In some embodiments, the computing device 201 may receive one or more images from the sensor 230 (e.g., a camera where the user's body is visible). The computing device 201 may analyze the images and determine that the destination location on the user's body for the haptic effect includes, for example, hair. In some embodiments, the user may perceive the haptic effect as being weaker than on the hairless body part on the body part including the hair. In such an embodiment, the computing device 201 may determine and output the haptic effect based on the detected hair. For example, the computing device 201 may determine and output a haptic effect that includes an increased amplitude. This can improve the quality of the haptic effect perceived by the user.

As another example, in some embodiments, the computing device 201 may include a body part to which a haptic effect (e.g., a projected / remote haptic effect) should be applied, such as a body part that is particularly sensitive . For example, the computing device 201 may analyze one or more images from the sensor 230 and, based on the images, determine that the computing device 201 will output the haptic effect to a particular body part. Computing device 201 may use a look-up table and / or algorithm, for example, to determine that a particular body part includes a sensitive body part. For example, the computing device 201 may use a look-up table to map a particular body part to a corresponding sensitivity level. The computing device 201 is responsive to a haptic device having one or more characteristics (e.g., type, amplitude, waveform, frequency, or other characteristics) configured to prevent and / or reduce the likelihood of injury to a body part The effect can be determined.

For example, the computing device 201 may be executing a game simulating virtual worms. In some embodiments, the computing device 201 may determine and output haptic effects configured to simulate worms crawling on the user, such as a user's ear. For example, a computing device may determine a remote haptic effect, including air purge (or ultrasound), which should be emitted towards the user's ear, e.g., to simulate a worm in the user's ear. In some embodiments, the computing device 201 may determine that the user's ears are a sensitive body part. In response, computing device 201 may change the nature of the haptic effect to reduce the likelihood of hurting the user. For example, the computing device 201 may reduce the amplitude or intensity of the airflow (or ultrasound), for example, to reduce the likelihood of impairing the hearing of the user.

As another example, the computing device 201 may be executing a military game in which a user may control a virtual character. In response to the virtual character being shot, the computing device 201 may be configured to output a haptic effect (e.g., a remote haptic effect) as part of the user's body associated with where the virtual character hit the bullet. For example, the computing device 201 may be configured to project a solid material (e.g., a plastic bullet) onto a portion of the user's body associated with where the virtual character hit the bullet. This can simulate shooting. In some embodiments, the computing device 201 may determine that the haptic effect should be applied to or projected onto a sensitive portion of the user's body, such as the user ' s eyes. In some embodiments, the computing device 201 may in response cause the haptic effect to be of a different type (such as ejection of air) from one type (e.g., a remote haptic effect involving the release of a rigid material, such as a plastic bullet) A remote haptic effect involving the release of gas). This can reduce the possibility of injury to the user's eyes.

In some embodiments, the computing device 201 may determine that there is a risk that the haptic effect will be applied to or projected onto a sensitive body part. For example, in the military game embodiment, the computing device 201 may determine a remote haptic effect that is configured to be projected towards the user's nose. For example, the computing device 201 may determine that, in response to the user's virtual character being shot at the nose, the plastic bullet should be projected toward the user's nose. However, the computing device 201 may also determine that the user is moving and that the projected material is increasing the likelihood of contact with sensitive body parts, such as the user's eyes. Computing device 201 may be adapted to receive a haptic effect from one type (e.g., a remote haptic effect, including the release of a rigid material, such as a plastic bullet) Remote haptic effect). This can reduce the likelihood of injury to sensitive body parts.

In some embodiments, the sensor 230 (e.g., a temperature sensor) can be configured to detect the temperature of the user (e.g., the skin temperature of the user). In some embodiments, the computing device 201 may modify the characteristics of the haptic effect based on temperature. For example, the computing device 201 may be configured to output a haptic effect that includes applying heat to at least a portion of a user's body (e.g., a user's abdomen) in response to a game event, such as a virtual explosion . In such an embodiment, the computing device 201 may determine, for example, that there is no haptic effect in response to the sensor signal from the sensor 230 indicating that the user's body temperature is above a threshold. For example, the computing device 201 may not output a haptic effect, in response to determining that the user's body is too hot. This can prevent and / or reduce the likelihood of injury to the user (e.g., heat stroke).

The computing device 201 communicates with the position sensor 232. In some embodiments, the position sensor 232 may be a camera, a 3D imaging system (e.g., a 3D imaging system commonly sold under the trademark Microsoft Kinect®), a depth sensor, an ultrasonic transducer, an accelerometer, a gyroscope, a radio frequency identification (GPS) device, a magnetometer, a wireless interface (e.g., an IEEE 802.11 or Bluetooth interface), an infrared sensor, a range sensor, and / or an LED Based tracking system. The position sensor 232 is configured to detect one or more positions of a user's body part and / or physical object and to transmit one or more associated sensor signals to the processor 202. [ The location may be a three-dimensional location in the real space or a location relative to the computing device 201. In some embodiments, the position sensor 232 is configured to detect a three-dimensional gesture in the real space, e.g., based on a user's body part and / or a plurality of detected three-dimensional positions of the physical object.

In some embodiments, the processor 202 is configured to determine a gesture based on one or more sensor signals from the position sensor 232. For example, a user may perform a gesture (e.g., a sweeping gesture in a real space). The computing device 201 may analyze a plurality of camera images associated with the gesture (e.g., photographed by the position sensor 232) and determine one or more characteristics of the gesture (e.g., type of gesture) based on the plurality of camera images Can be determined. For example, the processor 202 may determine that the user has performed a particular type of gesture based on the camera images. As another example, in some embodiments, the position sensor 232 may be configured to detect the strength of a wireless signal that is worn by a user and / or emitted by an electronic device (e.g., held by a user) Lt; / RTI > The position sensor 232 may transmit a sensor signal associated with the wireless signal strength to the processor 202. [ Based on a plurality of different radio signal strengths detected by the position sensor 232 over a period of time, the processor 202 may determine, for example, that the electronic device has moved in a particular pattern associated with a particular gesture have.

In some embodiments, the position sensor 232 is external to the computing device 201 (e.g., as shown in connection with FIG. 3) and is in wired or wireless communication with the computing device 201. For example, the position sensor 232 may include a camera associated with a wearable device (e.g., a pair of glasses or a tie) and may communicate with the computing device 201. As another example, the position sensor 232 may include a 3D imaging system and / or an LED-based tracking system located outside the computing device 201 (e.g., on a shelf in the user's home) and may communicate with the computing device 201 can do.

In some embodiments, the computing device 201 includes a haptic output device 218 in communication with the processor 202. In some embodiments, a single haptic output device 218 is configured to output both local haptic effects and remote haptic effects. The haptic output device 218 is configured to output a haptic effect (e.g., a remote haptic effect and / or a local haptic effect) in response to the haptic signal. The haptic effect may include, for example, a change in temperature, a feeling of stroking, an electrotactic effect, a change in temperature, and / or a surface variation (e.g., a variation in surface associated with computing device 201). In addition, some haptic effects may use multiple haptic output devices 218 of the same or different types sequentially and / or at once. Although a single haptic output device 218 is shown in FIG. 2, embodiments can use multiple haptic output devices 218 of the same or different types to generate a haptic effect.

In some embodiments, the haptic output device 218 is external to the computing device 201 (such as Ethernet, USB, a wired interface such as IEEE 1394, and / or a wireless interface such as an IEEE 802.11, Bluetooth, or radio interface) (E.g., via a wireless interface) to the computing device 201. For example, the haptic output device 218 may be configured to be associated (e.g., coupled to) a wearable device and receive haptic signals from the processor 202.

In some embodiments, the haptic output device 218 is configured to output a haptic effect that includes vibration. The haptic output device 218 may be, for example, a piezoelectric actuator, an electric motor, an electromagnetic actuator, a voice coil, a shape memory alloy, an electroactive polymer, a solenoid, an eccentric rotating mass motor (ERM) And / or a plurality of conductive electrodes configured to vibrate due to electrostatic attraction and repulsion between the electrodes.

In some embodiments, the haptic output device 218 is configured to output a haptic effect that modulates the perceived friction coefficient of the surface associated with the haptic output device 218. In one embodiment, the haptic output device 218 includes an ultrasonic actuator. The ultrasonic actuator may vibrate at an ultrasonic frequency, for example, 20 kHz, to increase or decrease the perceived coefficient of the associated surface. In some embodiments, the ultrasonic actuator may comprise a piezoelectric material.

In some embodiments, the haptic output device 218 uses electrostatic attraction to output an electrostatic haptic effect. In some embodiments, the electrostatic haptic effect may include a perceived change in the simulated texture, simulated vibration, stroking feeling, and / or friction coefficient on the surface associated with the computing device 201.

For example, in some embodiments, the haptic output device 218 may include a conductive layer and an insulating layer. In some embodiments, the conductive layer is formed of a material selected from the group consisting of copper, aluminum, gold, silver, graphene, carbon nanotubes, indium tin oxide (ITO), platinum, poly (3,4- ethylenedioxythiophene) polystyrenesulfonate : ≪ / RTI > PSS), or any combination thereof. In some embodiments, the insulating layer is formed of a material selected from the group consisting of glass, plastic, polymer, silica, parylene, (SiO _2), aluminum oxide (Al ₂ O _3), silicon nitride (Si ₃ N _4), or may include any combination thereof. in addition, the processor 202 is an electrical signal, for example, AC The high voltage amplifier may generate an AC signal, which may be applied to the conductive layer and the haptic output device 218. In some embodiments, (E.g., a user's finger, head, foot, Arms, shoulders, legs, or other body parts, or stylus.) Changing the attraction levels between the object and the conductive layer may change the haptic effect perceived by the user.

In some embodiments, the conductive layer comprises a plurality of electrodes. The plurality of electrodes may be arranged in a specific pattern or configuration. For example, the plurality of electrodes may comprise a plurality of electrode strips. The electrode strips may be arranged in a diagonal pattern, a horizontal pattern, a vertical pattern, or other pattern across one or more surfaces of the computing device 201. As another example, one or more of the electrodes may comprise a circular, triangular, elliptical, square, rectangular, or other shape. For example, the peripheral or peripheral portion of the electrodes may include a circular, triangular, oval, square, rectangular, or other shape. The plurality of electrodes may comprise any number of electrodes arranged in any configuration and including any number of shapes. In some embodiments, the processor 202 may operate the haptic output device 218 by applying an electrical signal to all or a subset of the plurality of electrodes. The electrical signal can cause capacitive coupling between the electrodes and objects in the vicinity of or in contact with the electrodes. The user can recognize the capacitive coupling as a haptic effect.

In some embodiments, the electrostatic haptic effect includes a "dynamic ESF effect ". The dynamic ESF effect may include a perceived electrostatic haptic effect as the user's body part (e.g., a finger) moves relative to the surface associated with the haptic output device 218. For example, the dynamic ESF effect may be perceivable by the user as the user slides the finger along the surface of the insulator layer of the haptic output device 218. [ As another example, the dynamic ESF effect may be recognizable by the user as the computing device 201 moves against the user ' s body (e.g., while the user remains stationary).

For example, the computing device 201 may output a graphical user interface (GUI) comprising one or more virtual user interface components (e.g., buttons, sliders, knobs, etc.) on the touch screen display. For example, the GUI may include a virtual button. The user interface component may include a virtual texture, such as a plastic texture. In some embodiments, the computing device 201 may output a haptic effect in response to a user sliding a finger across a location on the touch screen display associated with the user interface component. The haptic effect may include, for example, a dynamic ESF effect configured to simulate a plastic texture.

In some embodiments, the electrostatic haptic effect includes a static ESF effect. The static ESF effect may include a perceivable electrostatic haptic effect without requiring the user to move the body part against a surface associated with the haptic output device 218 (e.g., across it or perpendicular thereto). For example, the user can still perceive the haptic effect without substantially moving against the surface. In addition, the user may not need to make any contact with the surface to recognize the static ESF effect. For example, in the GUI environment, the computing device 201 may output a static ESF haptic effect in response to the user hovering the finger above the touch screen display and on the user interface component. In some embodiments, static ESF effects include repeatedly pulling and pushing a user's body part (e.g., a finger) using an electrostatic force to generate a vibrational impression, for example. The haptic effect may, for example, notify the user that the user is accessing and / or hovering over an enabled or disabled button.

In some embodiments, the haptic output device 218 includes a modification device configured to output a modified haptic effect. The deformation haptic effect may include raising or lowering portions of the surface associated with the computing device 201. For example, the deformation haptic effect may include elevating portions of the surface of the computing device 201 to generate a rugged texture. In some embodiments, the deformed haptic effect may be achieved by bending, corrugating, rolling, bending, squeezing, bending, changing its shape, or otherwise deforming the surface associated with computing device 201 Or may be modified in other ways. For example, a deformed haptic effect may apply a force to a surface associated with computing device 201 or computing device 201 to cause it to bend, wrinkle, roll, bend, squeeze, bend, Altered, or otherwise modified. For example, in response to a user accessing the computing device 201, the computing device 201 may output a haptic effect configured to cause the computing device 201 to bend towards the user. This modified haptic effect may be at least part of the substantially continuous haptic feedback provided to the user when the user approaches and eventually contacts the computing device 201.

In some embodiments, the haptic output device 218 includes a fluid or other material configured to output a deforming haptic effect (e.g., to flex or deform a surface associated with the computing device 201 or the computing device 201) do. For example, the fluid may include a smart gel. Smart gels include materials with mechanical or structural properties that vary in response to stimuli or stimuli (e.g., electric field, magnetic field, temperature, ultraviolet light, shake, or pH variation). For example, in response to stimulation, the smart gel may vary in stiffness, volume, transparency, and / or color. In some embodiments, the stiffness may include resistance to deformation of the surface associated with the computing device 201. In some embodiments, one or more wires may be embedded in or coupled to the smart gel. As current flows through the wires, heat is released, causing the smart gel to expand or contract. This may modify the surface associated with the computing device 201 or the computing device 201.

As another example, the fluid may comprise a rheological (e.g., magnetorheological or electrorheological) fluid. A rheological fluid includes metal particles (e.g., iron particles) that are suspended in a fluid (e.g., oil or water). In response to an electric field or magnetic field, the order of the molecules in the fluid can be rearranged to change the overall attenuation and / or viscosity of the fluid. This may modify the surface associated with the computing device 201 or the computing device 201.

In some embodiments, the haptic output device 218 includes a mechanical deforming device. For example, in some embodiments, the haptic output device 218 may include an actuator coupled to an arm that rotates the deformable component. The deformable component may, for example, comprise an oval, star, or wavy shape. The deformation component may be configured such that the surface associated with computing device 201 moves at some rotational angle while the others do not. The actuators include piezoelectric actuators, rotary / linear actuators, solenoids, electroactive polymer actuators, macrofiber composite (MFC) actuators, shape memory alloy (SMA) actuators, and / or other actuators can do. As the actuator rotates the deformable component, the deformable component can move the surface to deform the surface. In such an embodiment, the deforming component can be started at a flat surface. In response to receiving a signal from the processor 202, the actuator may rotate the deformable component. Rotating the deforming component may raise or lower one or more portions of the surface. The deformation component may remain in this rotated state until, in some embodiments, the processor 202 signals the actuator to rotate the deformation component back to its original position.

In addition, other techniques or methods may be used to modify the surface associated with computing device 201 and / or to generate haptic effects in other ways. For example, the haptic output device 218 may be configured to modify its surface or to modify its texture (e.g., based on contact from a surface reconfigurable haptic substrate (including, but not limited to, fibers and nanotubes) And a flexible surface layer configured to vary the thickness of the substrate. In some embodiments, the haptic output device 218 may include, for example, a motor coupled to the wires, air or a pocket, a resonating mechanical element, a micro-electromechanical systems (MEMS) element or pump, a thermal fluid pocket, Laminar flow modulation is used to generate deformed or haptic effects in different ways.

In some embodiments, the haptic output device 218 is configured to remotely project the haptic outputs to the user (e.g., to generate remote haptic effects). For example, the haptic output device 218 may include one or more vents configured to emit materials (e.g., solids, liquids, gases, or plasmas) toward a user (e.g., a user's body part). In one such embodiment, the haptic output device 218 includes a gas inlet configured to emit a stream or a flow of oxygen, nitrogen, or other gas having various characteristics upon receipt of a haptic signal. As another example, the haptic output device 218 may include one or more ultrasonic transducers or speakers configured to project pressure waves in the direction of the user. In one such embodiment, the processor 202 may cause the haptic output device 218 to emit a concentrated pressure wave toward the user, in response to the user making a gesture in real space. The concentrated pressure wave may vibrate a portion of the user's body (e.g., the user's finger). The user can perceive the vibration as a haptic effect.

In some embodiments, the haptic output device 218 may be part of the housing of the computing device 201. The haptic output device 218 may be housed within a flexible housing that covers a surface associated with the computing device 201 (e.g., the front or back of the computing device 201). In some embodiments, the haptic output device 218 itself may be malleable.

Looking at memory 204, modules 224, 226, and 228 are shown in some embodiments to show how a device can be configured to provide position-based haptic effects. In some embodiments, the gesture detection module 224 includes one or more algorithms or lookup tables that can be used by the processor 202, for example, to determine a gesture by a user.

In some embodiments, the gesture detection module 224 includes code for detecting contact with the computing device 201. For example, in this embodiment, the gesture detection module 224 may sample the touch sensor 208 to determine if the user has contacted the touch sensitive surface 216. In other such embodiments, the gesture detection module 224 may analyze a plurality of images from the position sensor 232 and determine whether the pixels corresponding to the user ' s body part are overlaid with pixels associated with the computing device 201 have. In such a case, the gesture detection module 224 may determine, for example, that the user is in contact with the computing device 201.

In some embodiments, the gesture detection module 224 includes code for analyzing the data received via the network interface device 210 to determine the location of the user with respect to the computing device 201. For example, in some embodiments, a user may be wearing a device, such as a smart ring or a smart watch, which includes a GPS sensor. In such an embodiment, the device may be configured to send signals associated with the GPS location of the device (e.g., the GPS location of the user's body part) to the computing device 201. [ In some embodiments, the gesture detection module 224 receives a GPS location from the device and provides a code for comparing the GPS location with the GPS location of the computing device 201 (e.g., as detected by the sensor 230) . This may allow the computing device 201 to determine the location of the device (e.g., user) for the computing device 201. Gesture detection module 224 may include code for determining a gesture based on location.

The haptic effect determination module 226 represents a program component that analyzes the data to determine the haptic effect to generate. The haptic effect determination module 226 may include code for selecting one or more haptic effects to output using one or more algorithms or lookup tables. In some embodiments, the haptic effect determination module 226 includes code for determining one or more haptic effects to output based on the detected gesture. For example, processor 202 may access a look-up table stored in haptic effect determination module 226 to map a particular gesture to a specific haptic effect, such as a pneumatic or electrostatic haptic effect.

In some embodiments, haptic effect determination module 226 determines a plurality of haptic effects to output using one or more haptic output devices 218, for example, to provide substantially continuous haptic feedback throughout the gesture . For example, the haptic effect determination module 226 may include a plurality of available haptic output devices 226 for use in combination with one another to provide substantially continuous haptic feedback when a user approaches and contacts the computing device 201, May include code for selecting two or more different haptic output devices from among the plurality of haptic output devices.

The haptic effect generating module 228 represents programming that causes the processor 202 to generate and transmit haptic signals to the haptic output device 218 to generate the selected haptic effect. For example, the haptic effect generation module 228 may access stored waveforms or commands to be transmitted to the haptic output device 218 to produce a desired haptic effect. In some embodiments, the haptic effect generating module 228 may comprise algorithms for determining a haptic signal. In addition, in some embodiments, the haptic effect generating module 228 may include algorithms for determining target coordinates (e.g., coordinates for a position on the body of the user to output a haptic effect) for the haptic effect.

Although modules 224, 226, 228 are shown as program components in memory 204 in Figure 2, in some embodiments modules 224, 226, 228 may comprise hardware. For example, modules 224, 226, and 228 may include analog-to-digital converters, processors, microcontrollers, comparators, amplifiers, transistors, and other analog or digital circuitry.

Figure 3 illustrates one embodiment of a system 300 that provides location based haptic effects. System 300 includes a wearable computing device 302, a graspable computing device 304, a position sensor 316, and / or a remote haptic output device 318 coupled to a network 314. The wearable computing device 302 and / or the graspable computing device may be configured substantially the same as the computing device 201 of FIG.

The computing devices 302 and 304, the position sensor 316, and / or the remote haptic output device 318 may communicate with one another directly and / or via the network 314. For example, the wearable computing device 302 may communicate directly with the graspable computing device 304 (e.g., using Bluetooth) wirelessly.

System 300 may include any number, combination, and / or configuration of wearable computing devices 302, graspable computing devices 304, position sensors (not shown) to provide substantially continuous haptic feedback throughout the gesture 316, and / or remote haptic output devices 318. For example, in some embodiments, the wearable computing device 302, the graspable computing device 304, the position sensor 316, and the remote haptic output device 318 may be configured such that, for example, When accessing, contacting, and / or lifting a finger of a user interface component of the fingable computing device 304, such as a joystick 308, a button 310, and / or a directional pad 312, And cooperate to provide substantially continuous haptic feedback to the user.

As a specific example, in some embodiments, a user may be wearing a virtual reality headset to play a video game in which the user's view is substantially blocked. In such an embodiment, the system 300 may be configured to allow a user to access a user with a phageable computing device 304, which may not be visible if the user does not otherwise, when the user accesses and ultimately picks up the phonable computing device 304 And output substantially continuous haptic feedback for guidance. For example, the position sensor 316 may detect a user accessing the graspable computing device 304 and transmit the associated sensor signals to the wearable computing device 302 and / or the remote haptic output device 318. The wearable computing device 302 and / or the remote haptic output device 318 may responsively output haptic effects as the user approaches the graspable computing device 304, e.g., in decreasing sizes . In response to the user finally contacting the graspable computing device 304, the graspable computing device 304 may output a haptic effect, such as vibration. In some embodiments, a combination of haptic effects from various components of the system 300 may provide a substantially continuous haptic feedback to the user, for example, to guide the user towards the graspable computing device 304 .

In some embodiments, the system 300 is configured to output one or more haptic effects configured to provide information to a user. For example, the user may capture the graspable computing device 304, e.g., to play a video game. The system 300 may detect that a user's body part is hovering above it without contacting a particular user interface component of the graspable computing device 304 (e.g., the trigger 306). In some embodiments, the system 300 may, in response, output a haptic effect (e.g., air blast) towards the user (e.g., toward the back of the user's finger above the trigger 306). The haptic effect can be configured, for example, to notify the user of the characteristics of the user interface component. For example, in this embodiment, the haptic effect may notify the user of the type, location, function, and / or status of the user interface component, such as whether the user interface component is enabled. This may allow a user to determine, for example, whether a particular user interface is enabled without physically touching the user interface component.

In addition, in some embodiments, the system 300 is configured to provide continuous haptic feedback when the user hoveres above, approaches, and ultimately contacts the user interface component. In some embodiments, the system 300 may additionally or alternatively provide continuous haptic feedback when the user moves his / her finger away from the user interface component (e.g., after interacting with the user interface component) . The haptic feedback output when the user moves away from the user interface component may differ from the haptic feedback output when the user approaches and / or contacts the user interface component. For example, the system 300 may output haptic feedback when the user moves away from a user interface component that is configured to inform the user that the user interface component is no longer enabled.

In some embodiments, the system 300 may additionally or alternatively output (e.g., via a display) a user interface having a plurality of user interface levels. For example, the system 300 may execute a video game that includes a user interface (e.g., the phishable computing device 304 itself may execute a video game). A video game may include virtual objects, such as virtual laser guns. In some embodiments, the system 300 may be configured to allow a user to interact with a first gesture (e.g., a user placing a body part and / or a physical object in a first position relative to the graspable computing device 304) And may interact with and / or activate the first user interface level of the user interface. For example, in response to a user hovering a finger above the trigger 306 of the graspable computing device 304, the system 300 may allow the user to interact with the first user interface level and / Activation can be detected. In some embodiments, the system 300 may perform a function associated with a first user interface level (e.g., loading a magazine into a virtual laser gun). Additionally or alternatively, the system 300 may output a haptic effect associated with the first user interface level, such as a haptic effect configured to simulate loading the magazine into the virtual laser gun.

In addition, in some embodiments, the system 300 may be configured to allow a user to interact with a second user interface level, and / or, in response to the user placing a body part in a second position relative to the graspable computing device 304, / RTI > and / or activating it. For example, in response to a user hovering a finger above the trigger 306 closer to the graspable computing device 304, the system 300 may allow the user to interact with the second user interface level and / Or activating it. In some embodiments, the system 300 may perform a function associated with a second user interface level (e.g., loading a virtual laser gun). The system 300 can additionally or alternatively output a haptic effect associated with a second user interface level, such as a haptic effect configured to simulate loading on a virtual laser gun.

In addition, in some embodiments, the system 300 may be configured to allow a user to interact with a third user interface level, and / or in response to a user placing a body part in a third position relative to the graspable computing device 304, / RTI > and / or activating it. For example, the system 300 may detect that the user interacts with and / or activates a third user interface level in response to the user pressing the trigger 306 of the graspable computing device 304 can do. In some embodiments, the system 300 may perform a function associated with a third user interface level (e.g., firing a virtual laser gun). The system 300 may additionally or alternatively output a haptic effect associated with a third user interface level, such as a haptic effect configured to simulate firing a virtual laser gun. As such, a user may be able to perform a plurality of functions by locating body parts at different locations above a particular user interface component. This can provide a more intuitive and simplified experience for the user.

Figure 4 illustrates another embodiment of a system 400 that provides location based haptic effects. System 400 includes a vehicle. The vehicle includes a computing device (e.g., internal to the vehicle). Although the vehicle of Fig. 4 includes an automobile, in other embodiments, the vehicle may include a truck, a boat, an airplane, a motorcycle, and the like.

A computing device communicates one or more user interface components (e.g., a joystick, button, trigger, switch, knob, touch sensitive surface, etc.). In the example shown in FIG. 4, a computing device communicates with a touch screen display 402 and multiple buttons 406. The system 400 also includes a position sensor 404 and a haptic output device. The haptic output device is configured to provide local and / or remote haptic effects to a user.

In some embodiments, the computing device is configured to provide haptic feedback (e.g., substantially continuous haptic feedback) to a user in response to a gesture in real space. For example, the computing device may output via the touchscreen display 402 a GUI configured to change one or more vehicle settings, such as, for example, audio volume, radio station, air conditioner or thermal level, and / or GPS navigation settings . In some embodiments, the computing device may be configured so that when a user approaches, interacts with, and / or moves away from a particular virtual object (e.g., a virtual button for adjusting audio volume) of the GUI, To provide the user with haptic feedback. In such an embodiment, the computing device may be configured so that the user no longer does what the user gestures above the virtual object (e.g., hovering the finger above the virtual object), and / In response to detecting that the user has moved far enough away from the virtual object (e.g., the user moved far enough away from the virtual object). In some embodiments, such continuous haptic feedback may allow the user to position the desired user interface components and / or securely interact with the computing device, for example, without visually focusing on the touch screen display Which can prevent accidents or injuries.

As an example, in some embodiments, the computing device is configured to detect that a user is gesturing (e.g., with a finger) above a virtual button that is output on the GUI to change, for example, a radio station. The computing device is configured to output a remote haptic effect (e. G., An ultrasonic pressure wave) configured to notify a user that, for example, the user is gesturing over a particular virtual object in response. In some embodiments, the computing device is configured to continue outputting remote haptic effects, in response to detecting that the user is accessing a virtual button. In such an embodiment, the computing device may modulate the intensity of the remote haptic effects, for example, so that the user perceives the remote haptic effects as having a substantially constant magnitude. Thus, for example, even though the user may be moving closer to the source of remote haptic effects, the user may perceive the remote haptic effects as having a substantially constant size. In some embodiments, the computing device is configured to detect that a user has contacted a virtual object (e.g., via touchscreen display 402 and / or location sensor 404). The computing device may, in response, stop outputting remote haptic effects and / or output local haptic effects, e.g., including pulsed vibrations. This may allow the user to confirm that the computing device has received the user input. In some embodiments, the computing device may continuously output a local haptic effect, for example, until the computing device detects that the user is moving away from the virtual object and / . The computing device may then output one or more remote haptic effects, for example, until the distance between the user and the virtual object exceeds a threshold value.

Although the discussion above refers to a virtual object (e.g., output on the touch screen display 402), in some embodiments the computing device may include buttons 406, triggers, switches, joysticks 408, Knobs, and the like, based on gestures performed on one or more physical user interface components.

In some embodiments, the computing device outputs a user interface including a plurality of user interface levels, e.g., via the touchscreen display 402. For example, in some embodiments, the computing device is configured to output a GUI associated with the radio in the vehicle. In some embodiments, the user may interact with the first user interface level of the user interface by hovering the finger above the virtual button, such as a radio station scan button, output on the touch screen display 402. In some embodiments, the computing device may perform the functions associated with the first user interface level. For example, the computing device may output (e.g., via the touchscreen display 402) a description of the function associated with the virtual button, such as "HOVER CLOSER TO SCAN, TAP TO TURN OFF ". The computing device may, for example, output a haptic effect configured to simulate the presence or operation of a physical button. In addition, in some embodiments, the user may interact with the second user interface level by gesturing closer to the virtual button. In some embodiments, the computing device is configured to scan a plurality of radio stations in response. In addition, in some embodiments, the user may interact with the third user interface level by contacting the touch screen display 402 (e.g., and / or a virtual button output thereon). In some embodiments, the computing device is configured to turn off the radio in response. As such, in some embodiments, the computing device is configured to perform a plurality of functions in response to a user interacting with various user interface levels.

Figure 5 illustrates another embodiment of a system for providing location based haptic effects. The system includes a computing device 502 (e.g., a smartphone or tablet) that outputs a virtual object 506 on a display (e.g., a touch screen display). The computing device 502 is outputting (e.g., via a display) a virtual object 506 that includes, for example, fire. Computing device 502 may detect a gesture, such as a user accessing and / or moving away from virtual object 506 (e.g., as shown by the dashed arrow), and in response, . The haptic effects are configured to provide a more realistic and / or immersive haptic experience for the user. For example, haptic effects can be configured to simulate, for example, the heat of fire. For example, in the example illustrated in FIG. 5, the computing device 502 may be configured to receive incremental intensities in response to detecting a user accessing the computing device 502 and / (E.g., vibrational or projected heat) that includes the decreasing intensities in response to motion. In addition, in some embodiments, the computing device 502 is configured to output a local haptic effect, such as a striking or shock impression, in response to detecting a user contacting the computing device 502. In some embodiments, a substantially continuous combination of remote and / or local haptic effects may more realistically simulate the characteristics of a virtual object, such as a row of increasing or decreasing fire as the user moves against fire.

As another example, in some embodiments, the computing device 502 is configured to output (e.g., via a display) a virtual object 506 that includes, for example, magnets and / or force fields. In such an embodiment, the computing device 502 may output haptic effects including increasing intensities, in response to the user detecting that the computing device 502 is approaching. For example, the haptic effect may comprise a flow of fluid or gas configured to resist movement of a user. This may allow the user to resist accessing the virtual object 506 (e.g., simulating magnetoresistance or force field). The computing device 502 may output haptic effects including decreasing intensities in response to detecting that the user is moving away from the computing device 502. For example, the haptic effect may include a flow of fluid or gas configured to help the user move away from the computing device 502. This makes it possible to more realistically simulate the effects of the magnetic field and / or the force field of the magnet.

In some embodiments, haptic effects are configured to provide information to a user. For example, the computing device 502 may execute an application. The application may perform a function that may cause damage to the computing device 502, for example, when the function is interrupted. In some embodiments, computing device 502 may detect that a user is gesturing towards computing device 502 (e.g., via location sensor 504) and may, in response, A remote haptic effect configured to resist movement of the haptic device. In some embodiments, the computing device 502 may increase the strength and / or amount of resistance when the user continues to access the computing device 502. This may inform the user that the user should not interact with the computing device 502. In some embodiments, the computing device 502 is configured to detect that the user is ultimately in contact with the computing device 502 and, responsive thereto, is configured to output a haptic effect, e.g., comprising a series of pulsed vibrations. This may inform the user that the user should not interact with the computing device 502. In addition, in some embodiments, computing device 502 may detect that a user is sliding a finger across the surface of computing device 502 (e.g., toward a virtual button that is output on the touch screen display) For example, a sensed increase in the coefficient of friction. This may make it physically more difficult for a user to interact with the computing device 502. The combination of haptic effects may more appropriately inform the user of the status, characteristics, and / or other characteristics of the application.

In some embodiments, the computing device 502 may additionally or alternatively, for example, interact with the virtual object 506 at different depths and / or interact with different surfaces of the virtual object 506 And outputs a user interface comprising a plurality of user interface levels configured to simulate operating. For example, the virtual object 506 may include a virtual fruit, such as a peach. The user may interact with a portion of the virtual fruit, such as an outer layer, by gesturing at a first location in the real space (e.g., by locating a body part). In some embodiments, the computing device 502 may detect the gesture and in response may output a haptic effect configured to, for example, simulate the fuzziness or skin of the fruit. In addition, the user can interact with other portions of the virtual fruit, such as an inner layer, by gesturing at a second location in the real space. The second location may be closer to the computing device 502 than the first location. In some embodiments, the computing device 502 may detect the gesture in a second position and in response may output a haptic effect configured to simulate internal characteristics of the fruit, such as, for example, a vignetted or soft texture. In this manner, the computing device 502 may simulate different surfaces and / or depths of the virtual object 506, in response to different gestures.

Exemplary methods of providing location based haptic effects

6 is a flow chart of steps for performing a method of providing location based haptic effects, in accordance with an embodiment. In some embodiments, the steps in FIG. 6 may be implemented in a processor, e. G., A general purpose computer, mobile device, or program code executed by a processor in a server. In some embodiments, these steps may be implemented by a set of processors. In some embodiments, one or more of the steps shown in Figure 6 may be omitted or performed in a different order. Similarly, in some embodiments, additional steps not shown in Fig. 6 may also be performed. The following steps are described with reference to the components described above in connection with the computing device 201 shown in FIG.

For simplicity, the following steps are described with reference to the drawing application. However, the following steps are not limited to such an embodiment, and any combination of steps may be used with other types of applications and / or devices.

The method 600 begins at step 602 where the processor 202 receives a sensor signal associated with a gesture that includes at least a contact with a surface (e.g., of the computing device 201) and a noncontact with the surface. For example, the processor 202 may execute a drawing application. A user may move a stylus (e.g., a pen or other object) toward the computing device 201, for example, to perform one or more drawing operations in the drawing application. The position sensor 232 (e.g., a camera) may capture one or more images associated with the movement of the stylus and transmit the associated sensor signal to the processor 202. In such an embodiment, the images may include foreground features, such as a portion of a stylus, and background features, such as a chair, table, wall, desk, and the like.

The method 600 continues at step 604 where the processor 202 determines a user interface level based at least in part on the gesture. In some embodiments, the processor 202 is configured to rely on programming, lookup tables, and / or algorithms included in the memory 204 to determine the user interface level. For example, the processor 202 may use a look-up table to map a location in the real space (associated with the gesture) to a particular user interface level of the drawing application. For example, the processor 202 may map a first location in the real space a long distance from the computing device 201 to a first user interface level associated with a perspective of the drawing output in the drawing application. The processor 202 may map a second location at a medium distance from the computing device 201 to a second user interface level associated with an enlargement and / or cross-section of the drawing output in the drawing application.

 In some embodiments, the processor 202 may associate a single gesture with a user interface level. For example, processor 202 may associate one specific gesture with a first user interface level and another gesture with a second user interface level. In another embodiment, the processor 202 may associate a plurality of gestures with a user interface level. For example, in some embodiments, the processor 202 may perform any gestures that occur within a range of distances from the computing device 201 (e.g., at 4 inches to 8 inches from the surface of the computing device 201) 0.0 > user interface < / RTI >

The method 600 continues at step 606 where the processor 202 determines a user interaction based on the gesture. For example, the processor 202 may determine that the gesture includes a swipe gesture, a hovering gesture, or other type of gesture based on, for example, the three-dimensional starting and / or ending position of the gesture. The processor 202 may correlate a particular type and / or location of the gesture with a particular user interaction associated with the drawing application, such as activating a line tool, a square shape tool, a paint brush size modification tool, or the like.

The method 600 continues at step 608 where the processor 202 determines a function based at least in part on user interface level and / or user interaction. In some embodiments, the function is to output audio data, video data, and / or information (e.g., near a gas station, restaurant, theater, police station, or hospital, or traffic situation or speed limit); Calling; Sending a text message, an SMS message, or an e-mail; Opening a web browser or accessing a web site; Opening an application; Closing the application; Performing background processing; Performing foreground processing; Saving the file; Opening a file; Performing game functions; Receiving data; Transmitting data; Changing application settings; Ordering products through a network; Printing a document; Adding entries to the list; Removing entries from the list; Recording sound; Playing media content; opening an e-book; Performing calculations; Performing a price comparison; Checking bank balances; And / or any other number and / or configuration of computer functions.

In some embodiments, the functionality includes outputting, removing, changing, updating, and / or deleting a virtual object in the user interface. For example, in the drawing application embodiment, the computing device 201 may be configured to determine whether the user is hovering a finger or stylus at a distance from the computing device 201, for example, The first function can be determined. The first function may include, for example, outputting a perspective view of the drawing. The computing device 201 may further determine a second function associated with the first user interface level, e.g., in response to detecting another gesture in the real space at a distance from the computing device 201. [ The second function may include, for example, drawing along a path defined by the motion of the gesture on a virtual canvas (e.g., with a virtual paint brush). In some embodiments, computing device 201 may determine a third function associated with a second user interface level, e.g., in response to a user detecting hovering a finger at an intermediate distance from computing device 201 . This function may include, for example, outputting an enlarged view of the drawing and / or a section view. In some embodiments, the computing device 201 may further determine a fourth function associated with the second user interface level, e.g., in response to detecting another gesture at a medium distance from the computing device 201. The fourth function may include, for example, erasing along a path defined by the motion of the gesture. In some embodiments, computing device 201 may determine a fifth function associated with a third user interface level, e.g., in response to detecting a user contacting computing device 201. This function may include, for example, storing a virtual image.

The method 600 continues at step 610 where the processor 202 performs the function. In some embodiments, the processor 202 may perform a function by executing one or more sub-functions. For example, if the function includes storing a virtual image, the processor 202 may include one or more servers (e.g., an image storage database) to obtain a secret key or password for securely storing the virtual image in a database Communication can be performed. Processor 202 may then store the image in a database.

The method 600 continues at step 612 where the processor 202 determines one or more haptic effects that are configured to provide substantially continuous haptic feedback throughout the gesture.

In some embodiments, the processor 202 determines one or more haptic effects based, at least in part, on the position in the real space associated with the gesture and / or gesture. For example, processor 202 may access a lookup table stored in memory 204 to map positions throughout the gesture to specific haptic effects. In such an embodiment, the look-up table may be pre-programmed in memory and / or programmable by the user.

In some embodiments, the processor 202 determines one or more haptic effects based, at least in part, on the distance between the user and the computing device 201. For example, in some embodiments, a user may provide a first haptic effect to a computing device 201 (e.g., a computer system), including a projected solid, liquid, gas, plasma, sound pressure wave, and / (E.g., from 1 mm to 10 cm) from a first distance range (e.g., from about 1 mm to about 10 cm). As such, in some embodiments, the processor 202, in response to detecting that the user is located within a first distance range (e.g., from 1 mm to 10 cm) from the computing device 201, The haptic effect is determined.

As another example, in some embodiments, a user may select a second haptic effect from computing device 201 within a second distance range (e.g., from 0 mm to 1 mm), including electrostatic-based capacitive coupling, It can be perceived as stronger than other types of haptic effects. As such, in some embodiments, the processor 202 determines a second haptic effect in response to detecting that the user is located within a second distance range from the computing device 201. As another example, in some embodiments, a user may select a third haptic effect from a computing device 201 within a third distance range (e.g., 0 mm and / or a computing device (e.g., 201), which is more robust than other types of haptic effects. As such, in some embodiments, the processor 202, in response to detecting that the user is located within a third distance range (e.g., in contact with a computing device) from the computing device 201, .

In some embodiments, the processor 202 determines a plurality of haptic effects when the user accesses, for example, the computing device 201 and moves across the range of distances from the computing device 201. The plurality of haptic effects may include any combination of remote and / or local haptic effects to provide the user with substantially continuous haptic feedback.

In some embodiments, the processor 202 determines a haptic effect that includes one or more characteristics (e.g., size, frequency, duration) configured to be recognized by the user as being substantially the same as the previously output haptic effect. For example, in some embodiments, the processor 202 may detect what the user is gesturing in the real world (e.g., moving a finger towards the computing device 201). In such an embodiment, the processor 202 may thereby reduce the size of the haptic effect and / or allow the user to interact with the computing device 201 in response to determining that the user is moving closer to the computing device 201, The size of the haptic effect can be increased in response to determining that the haptic effect is moving further away from the haptic effect. This may provide the user with a substantially consistent haptic experience, for example, regardless of how close or far the user is to computing device 201 (e.g., haptic output device 218).

In some embodiments, the processor 202 may include a haptic effect including one or more characteristics (e.g., size, frequency, duration) configured to be perceived by the user as being distinct and / or distinguishable from previously output haptic effects . For example, in some embodiments, the processor 202 may detect that the user is gesturing in the real space. In this embodiment, the processor 202, in response, can maintain the characteristics of the haptic effect constant. Since the user may be moving closer and / or further away to the source of the haptic effect during the gesture, the user may (e.g., have the processor 202 keep the characteristics of the haptic effect constant) It can be recognized that the intensity of the haptic effect is changed. This may provide information to the user, such as how close the user is to the computing device 201. [

In some embodiments, the processor 202 determines a haptic effect based at least in part on a user interface level, gestures, and / or user interaction. For example, the processor 202 may be configured to map a user interface level (e.g., associated with viewing in a drawing application or drawing on a virtual canvas) to specific haptic effects, such as air spitting or other remote haptic effects The lookup table stored in the memory 204 can be accessed. In some embodiments, the processor 202 is configured to determine a haptic effect based on the type, location, duration, and / or other characteristics of the gesture. In some embodiments, the processor 202 may determine the haptic effect based on the type, location, duration, and / or other characteristics of the user interaction. For example, in some embodiments, the processor 202 determines that the user interaction is configured to erase the image in the drawing application and, based on the user interaction, simulates, for example, erasing the board with a blackboard eraser To determine the associated haptic effect.

In some embodiments, the processor 202 determines a haptic effect based at least in part on a function (e.g., a characteristic associated with the function). For example, in some embodiments, the processor 202 may determine the haptic effect associated with the type of function. For example, in one embodiment, the processor 202 can determine a haptic effect, including a short burst of air if the function includes opening an application, such as a drawing application. In such an embodiment, the processor 202 may determine a haptic effect that includes three short bursts of air if the function involves closing the application. This can inform the user of the function. In some embodiments, the processor 202 determines a haptic effect that is configured to indicate that the function has been performed. For example, the processor 202 may determine a haptic effect that includes an electrostatic haptic effect, once the function has been executed that includes selecting a new color for the paint brush tool in the drawing application. In some embodiments, the processor 202 is configured to determine different haptic effects based on the results of performing the function. For example, the processor 202 may output a different haptic effect than when the function was not successfully executed, if the function was successfully executed.

The method 600 continues at step 614 where the processor 202 outputs one or more haptic effects. The processor 202 transmits one or more haptic signals associated with the one or more haptic effects to the haptic output device 218 and the haptic output device 218 outputs one or more haptic effects. In some embodiments, the one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture. For example, the processor 202 may transmit a plurality of haptic signals associated with a plurality of haptic effects (e.g., remote and / or local haptic effects). The plurality of haptic effects are configured to provide substantially continuous haptic feedback throughout gestures and / or other motions in the real space.

Advantages of location-based haptic effects

There are a number of advantages of position based haptic effects. For example, such systems can provide a more realistic or immersive user experience. For example, in some embodiments, such systems may provide substantially continuous haptic feedback to a user throughout a gesture, e.g., when a user approaches and / or contacts a computing device. This can more realistically simulate physical phenomena such as gravity, magnetic field, electric field, wind, resistance to force application, and / or heat. For example, in some embodiments, the computing device may execute a virtual terminal game in which the virtual reactor is core dissolved. The computing device may detect that the user is gesturing towards the computing device and in response may output haptic effects configured to, for example, simulate radioactive contamination. For example, a computing device may have substantially constant amplitude, in response to detecting a user approaching a computing device, substantially continuously outputting remote haptic effects that end in intense vibration in response to a user contacting the computing device can do. Continuous, intense haptic effects can more realistically simulate approaching a radioactive contaminated site.

As another example, in some embodiments, a user may interact with a virtual object (e.g., output on a display) by gesturing at a first location in a real space associated with a first user interface level. The computing device may, in response, output haptic feedback configured to simulate a surface associated with the virtual object (e.g., a pillow seat on a pillow). The user may further interact with the virtual object by gesturing at a second location associated with a second user interface level. The computing device may, in response, output haptic feedback configured to simulate different surfaces associated with the virtual object (e.g., the interior of the pillow, such as a feather or memory foam). In some embodiments, this may make the virtual object feel more realistic and / or three-dimensional. In addition, by associating different haptic effects with different user interface levels, the user will be able to recognize multiple haptic effects while interacting with a single virtual object. This may enable a greater range of haptic experiences.

In some embodiments, the location based haptic effects will allow the user to make a state determination (e.g., to determine the mode in which the device is located) without looking at the device. As such, the user will be able to keep focus on other tasks. For example, the computing device may project remote haptic effects to the user in a program or associated with operations available on the user interface. This allows the user to interact with the user interface without having to visually focus on the display. Similarly, the haptic effect may serve as an acknowledgment that the action is available, completed, or has a particular importance.

In some embodiments, location based haptic effects may enable more unique, interactive, and effective user interfaces. For example, in some embodiments, a user may perform a wide range of functions while interacting with different user interface levels associated with the user interface component. In addition, in some embodiments, a user may be required to position the user interface component by hovering the body part above the user interface component and to move the body part to another location in the physical space (e.g., toward the user interface component) To activate the user interface component. This will allow the user to interact with the user interface, for example, without physically touching the user interface component (e.g., tapping and / or gesturing the user interface component). Physical contact with a user interface component may be difficult in a moving vehicle, for example, where ambient motion, cognitive load, and visual distractions can make it difficult to accurately tap user interface components.

In another embodiment, the location-based haptic effects may enable a non-visual interface. For example, in some embodiments, a user may be able to navigate a non-visual user interface with the user's finger by gesturing in real space (e.g., at a location associated with the user interface level). When the user's finger interacts with the location of an invisible interface component (e.g., a virtual volume switch), the computing device may output haptic feedback. This will allow the user to identify the location of the invisible interface component. When identifying a location of an interface component, the user may interact with the interface component (e.g., by interacting with the interface component) by, for example, moving the user's finger to another location in the physical space You will be able to press). This may allow the computing device to perform functions associated with invisible interface components (e.g., increasing the volume level). The computing device may further provide haptic feedback to the user, for example, to confirm receipt of the user input.

General Considerations

The methods, systems and devices discussed above are examples. The various configurations, where appropriate, may omit, replace, or add various procedures or components. For example, in alternate configurations, the methods may be performed in a different order than described, and / or the various stages may be added, omitted, and / or combined. In addition, the features described in connection with the specific arrangements may be combined in various other arrangements. Different aspects and elements of the arrangements may be combined in a similar manner. Also, techniques are evolving, and thus many of the elements are examples, and do not limit the scope of the present disclosure or claims.

Specific details are set forth in the description to provide a thorough understanding of exemplary configurations (including implementations). However, configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures and techniques are shown without unnecessary detail in order to avoid obscuring the configurations. This description provides only exemplary configurations and does not limit the scope, applicability, or configuration of the claims. Rather, the foregoing description of the embodiments will provide those skilled in the art with a description that makes it possible to implement the techniques described. Various changes may be made in the function and arrangement of the elements without departing from the spirit or scope of the disclosure.

Also, configurations may be described as a process or a process represented as a block diagram. Although each of these operations may be described as a sequential process, many of the operations may be performed in parallel or concurrently. In addition, the order of operations can be rearranged. The process may have additional steps not included in the drawing. In addition, examples of methods may be implemented in hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware or microcode, program code or code segments that perform the necessary tasks may be stored on non-volatile computer readable media, such as a storage medium. Processors can perform the described tasks.

While several exemplary configurations have been described, various modifications, alternative constructions, and equivalents may be used without departing from the scope of the present disclosure. For example, the above elements may be components of a larger system, where other rules may prioritize or otherwise modify the application field of the present invention. Also, before, during, or after a number of factors are considered, a number of steps may be performed. Accordingly, the above description does not limit the scope of the claims.

The use of " comprising "or" configured to " herein is intended to be an open, inclusive representation that does not exclude devices configured or configured to perform additional tasks or steps. In addition, the use of "based on" means that a process, step, calculation, or other operation based on one or more of the stated conditions or values is, in fact, And is intended to be open-ended and inclusive. The titles, lists, and numbering included herein are for convenience of description and are not intended to be limiting.

Embodiments in accordance with aspects of the present invention may be implemented in digital electronic circuitry, in computer hardware, in firmware, in software, or in combinations thereof. In one embodiment, the computer may comprise a processor or processors. A processor includes or accesses a computer-readable medium, such as random access memory (RAM), coupled to a processor. The processor executes computer executable program instructions stored in memory, such as executing one or more computer programs including sensor sampling routines, selection routines, and other routines to perform the methods described above.

Such processors may include a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), and a state machine. Such processors may be programmable electronic devices, such as programmable logic controllers (PLCs), programmable logic controllers (PICs), programmable logic devices (PLDs), programmable read-only memories (PROMs), electrically programmable read only memories (EPROMs or EEPROMs) Devices. ≪ / RTI >

Such processors include, but are not limited to, a medium, such as a tangible computer readable medium, which when executed by a processor may store instructions that cause the processor to perform the steps described herein as being performed or supported by the processor And may communicate with it. Embodiments of computer readable media can include, but are not limited to, any electronic, optical, magnetic or other storage device capable of providing computer readable instructions to a processor, such as a processor within a web server. Other examples of media include, but are not limited to, a floppy disk, a CD-ROM, a magnetic disk, a memory chip, a ROM, a RAM, an ASIC, a configured processor, any optical media, any magnetic tape or other magnetic medium, Media, but are not limited thereto. In addition, various other devices, such as routers, private or public networks, or other transport devices, may include computer readable media. The described processors and processes may be in one or more structures and may be distributed over one or more structures. A processor may include code for performing one or more (or portions of methods) of the methods described herein.

While the present invention has been described in detail with respect to specific embodiments thereof, those skilled in the art will readily appreciate that many modifications are possible in the embodiments without materially departing from the scope of the invention, It is possible to manufacture it. Accordingly, it is intended that the present disclosure be shown by way of illustration and not of limitation, as would be readily apparent to those of ordinary skill in the art, and that the inclusion of such modifications, variations, and / You will know that you do not.

Claims (23)

As a system,
A sensor configured to detect a gesture and transmit a sensor signal associated with the gesture, the gesture comprising at least two positions, wherein a first one of the at least two positions includes a distance from a surface, The second location including contact with the surface; And
A processor in communication with the sensor,
Receiving the sensor signal from the sensor,
Wherein the one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture,
Generate one or more haptic signals based at least in part on the one or more haptic effects,
Configured to transmit the at least one haptic signals; And
A haptic output device configured to communicate with the processor and receive the at least one haptic signals to output the at least one haptic effects,
. 2. The system of claim 1, wherein the gesture includes a starting position that is remote from the surface and an ending position that is remote from the surface. 2. The system of claim 1, wherein the gesture is between the at least two positions, the gesture starting at the first position and ending at the second position. 4. The method of claim 3, wherein the one or more haptic effects include a remote haptic effect and a local haptic effect,
Cause the haptic output device to output the remote haptic effect in response to detecting a first interaction at the first location;
And to cause the haptic output device to output the local haptic effect in response to detecting a second interaction at the second location. 5. The haptic device of claim 4, wherein the haptic output device comprises a first haptic output device located on a wearable device configured to be worn by a user and a second haptic output device coupled to the surface;
The processor
Cause the first haptic output device to output the remote haptic effect in response to detecting the first interaction at the first location;
Wherein the second haptic output device is further configured to cause the second haptic output device to output the local haptic effect in response to detecting the second interaction at the second location. 5. The method of claim 4,
The gesture comprising movement of a body part;
Wherein the remote haptic effect comprises generating electrostatic coupling with the body part or releasing a solid, liquid, gas, plasma, acoustic wave, or laser beam toward the body part;
The local haptic effect includes vibrations, textures, perceived changes in friction coefficients, surface deformations, or variations in temperature;
Wherein the sensor comprises a camera, a 3D imaging device, a gyroscope, a global positioning system (GPS), a light emitting diode (LED), an infrared sensor, or an ultrasonic transducer. 2. The apparatus of claim 1, wherein the processor
Determine a user interface level from a plurality of available user interface levels based at least in part on the location of the at least two locations;
Determine a function associated with the user interface level;
Wherein the system is further configured to perform the function. 2. The apparatus of claim 1, wherein the processor
Determine a body part based on a second sensor signal from a second sensor;
Determine that said body part comprises a sensitive body part;
Determine that at least one haptic effect of the one or more haptic effects is configured to apply to the sensitive body part;
Wherein the system is further configured to determine a characteristic of the at least one haptic effect configured to reduce the likelihood of injury to the sensitive body part. As a method,
Receiving a sensor signal associated with a gesture from a sensor, the gesture comprising at least two positions, a first one of the at least two positions comprising a distance from a surface, and a second one of the at least two positions Contacting the surface;
Determining one or more haptic effects based at least in part on the sensor signal, wherein the one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture;
Generating one or more haptic signals based at least in part on the one or more haptic effects; And
And transmitting the one or more haptic signals to a haptic output device, the haptic output device configured to receive the one or more haptic signals and output the one or more haptic effects. 10. The method of claim 9, wherein the gesture includes a starting position that is remote from the surface and an ending position that is remote from the surface. 10. The method of claim 9, wherein the gesture is between the at least two locations, and wherein the gesture starts at the first location and ends at the second location. 12. The method of claim 11, wherein the one or more haptic effects include a remote haptic effect and a local haptic effect,
Outputting the remote haptic effect in response to detecting a first interaction at the first location; And
Further comprising outputting the local haptic effect in response to detecting a second interaction at the second location. 13. The haptic device of claim 12, wherein the haptic output device comprises a first haptic output device located at the wearable device and a second haptic output device coupled to the surface;
The method
Outputting the remote haptic effect via the first haptic output device in response to detecting the first interaction at the first location; And
And outputting the local haptic effect via the second haptic output device in response to detecting the second interaction at the second location. 13. The method of claim 12,
The gesture comprising movement of a body part;
Wherein the remote haptic effect comprises generating electrostatic coupling with the body part or releasing a solid, liquid, gas, plasma, acoustic wave, or laser beam toward the body part;
The local haptic effect includes vibrations, textures, perceived changes in friction coefficients, surface deformations, or variations in temperature;
Wherein the sensor comprises a camera, a 3D imaging device, a gyroscope, a global positioning system (GPS), or an ultrasonic transducer. 10. The method of claim 9,
Determining a user interface level from a plurality of available user interface levels, based at least in part on the location of the at least two locations;
Determining a function associated with the user interface level; And
And performing the function. 10. The method of claim 9,
Determining a body part based on a second sensor signal from a second sensor;
Determining that said body part comprises a sensitive body part;
Determining that at least one haptic effect of the one or more haptic effects is configured to apply to the sensitive body part; And
Further comprising determining a characteristic of the at least one haptic effect configured to reduce the likelihood of injury to the sensitive body part. 18. A non-transitory computer readable medium comprising program code, wherein the program code, when executed by a processor,
Wherein the gesture includes at least two positions, a first one of the at least two positions includes a distance from a surface, and a second one of the at least two positions Contacting the surface;
Determine one or more haptic effects based at least in part on the sensor signal, wherein the one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture;
Cause one or more haptic signals to be generated based at least in part on the one or more haptic effects;
Wherein the haptic output device is configured to receive the at least one haptic signals and output the at least one haptic effects to cause the at least one haptic signals to be transmitted to the haptic output device. 18. The non-transitory computer readable medium of claim 17, wherein the gesture comprises a starting position remote from the surface and an ending position remote from the surface. 18. The method of claim 17,
The gesture being between the at least two positions, the gesture starting at the first position and ending at the second position;
Wherein the one or more haptic effects include a remote haptic effect and a local haptic effect;
The non-transitory computer readable medium, when executed by the processor, causes the processor to:
Cause the haptic output device to output the remote haptic effect in response to detecting a first interaction at the first location;
And program code configured to cause the haptic output device to output the local haptic effect in response to detecting a second interaction at the second location. 20. The haptic device of claim 19, wherein the haptic output device comprises a first haptic output device located at the wearable device and a second haptic output device coupled to the surface;
The non-transitory computer readable medium, when executed by the processor, causes the processor to:
Cause the first haptic output device to output the remote haptic effect in response to detecting the first interaction at the first location;
And program code configured to cause the second haptic output device to output the local haptic effect in response to detecting the second interaction at the second location. 20. The method of claim 19,
The gesture comprising movement of a body part;
Wherein the remote haptic effect comprises generating electrostatic coupling with the body part or releasing a solid, liquid, gas, plasma, acoustic wave, or laser beam toward the body part;
The local haptic effect includes vibrations, textures, perceived changes in friction coefficients, surface deformations, or variations in temperature;
Wherein the sensor comprises a camera, a 3D imaging device, a gyroscope, a global positioning system (GPS), or an ultrasonic transducer. 18. The method of claim 17, further comprising: when executed by the processor,
Determine a user interface level from a plurality of available user interface levels based, at least in part, on the location of the at least two locations;
Determine a function associated with the user interface level;
Further comprising program code configured to cause the computer to perform the function. 18. The method of claim 17, further comprising: when executed by the processor,
Determine a body part based on a second sensor signal from a second sensor;
Determine that said body part comprises a sensitive body part;
Determine that at least one haptic effect of the one or more haptic effects is configured to apply to the sensitive body part;
Further comprising program code configured to determine a characteristic of the at least one haptic effect configured to reduce the likelihood of injury to the sensitive body part.

Images