KR20180058209A - Devices and methods for modifying haptic effects - Google Patents

Abstract

A device for modifying a haptic effect and a method thereof are provided. The device includes a computer system and a haptic enabled device. A haptic output command configured to produce a desired haptic effect can be determined. A fatigue level is determined according to haptic effect density of a performed haptic effect and an input received from a control device. According to the fatigue level, a modified haptic effect is determined to replace the desired haptic effect and a corresponding haptic output command is generated and outputted to a haptic output device. Accordingly, the present invention can modify the haptic effect according to the fatigue level of a user.

Description

[0001] DEVICES AND METHODS FOR MODIFYING HAPTIC EFFECTS [0002]

Cross reference of related application

This application claims the benefit of prior US patent application Ser. No. 62 / 425,795, filed November 23, 2016, which is hereby incorporated by reference in its entirety for all purposes.

Embodiments of the present invention are directed to a device and method for altering a haptic effect, in particular a haptic effect, in accordance with a user fatigue level.

Video games and virtual reality systems are becoming increasingly popular due to marketing and casual gamer involvement in casual gamers. In a typical implementation, the computer system displays a visual or graphical environment for the user on the display device. The user can interact with the displayed environment by entering commands or data from the controller or peripheral device. The computer updates the environment in response to the user's manipulation of the to-be-manipulated portion, such as a joystick knob, and provides visual feedback to the user using the display screen.

Conventional video game devices or controllers use visual and audible cues to provide feedback to the user. In some controllers or peripheral devices, kinesthetic feedback (e.g., active and resistive haptic feedback) and / or tactile feedback (e.g., vibration, texture and heat) are also provided to the user, collectively referred to as "haptic feedback" Haptic effect ". The haptic feedback can provide a queue that enforces and simplifies the user controller or peripheral device. For example, a vibrating or vibrotactile haptic effect may be useful for providing a cue to a user of the electronic device to alert the user to a particular event or to provide realistic feedback to create a greater sensory immersion within the simulated environment or virtual environment . Conventional haptic feedback systems for gaming and other devices typically include actuators for generating haptic feedback, attached to the housing of the controller / peripheral. More specifically, a motor or other actuator of a controller or peripheral device is housed within a controller and connected to a control computer system. The computer system receives the sensor signal from the controller or peripheral device and sends the appropriate haptic feedback control signal to the actuator. The actuator then provides haptic feedback to the user of the controller. Thus, the computer system can convey the physical sensation to the user with other visual and auditory feedback.

The gaming peripheral may include a trigger, button, joystick, joypad, etc., which are used to control events in the game. These triggers may include haptic characteristics that further enhance and provide a more immersive experience for the player. The gaming peripheral may have more than one trigger, and the trigger actuator may provide a vibrotactile and / or kinesthetic haptic effect.

The haptic effect on gaming peripheral triggers can increase mobility in a video game, but may also cause fatigue during long game play periods. Fatigue can occur in a number of ways, including physical fatigue due to cognitive fatigue, which is not strongly perceived due to excessive overstimulation, and long-term haptic feedback. Others have attempted to solve this by a pre-program that weakens the entire haptic effect or is attenuated from the effect. However, this does not provide the optimal haptic and immersion experience in the largest number of situations, since it is very difficult to predict how the user will proceed in a given video game.

In one embodiment, a device is provided for altering a change in haptic output to a haptic executable device. The device comprising at least one processor, the at least one processor determining a haptic output command comprising an instruction to activate a haptic output device to perform an original haptic effect, and outputting a haptic output command to the haptic output device, Determining a fatigue level indicative of an amount of user fatigue based on at least one of the input received from the control device, comparing the fatigue level to the fatigue threshold, and based on a comparison of the fatigue level to the fatigue threshold, To execute a computer instruction to determine a replacement haptic output command that includes an instruction to activate a haptic output device to perform a modified haptic effect.

In another embodiment, a computer implemented method for changing a haptic output is provided. The method may be performed by at least one processor executing computer instructions. The method comprising: determining a haptic output command comprising an instruction to activate a haptic output device to perform an original haptic effect by the at least one processor; determining a haptic output command by the haptic output device transmitted by the at least one processor to the haptic output device Determining a fatigue level indicative of an amount of user fatigue based on an input received from the command and control device; comparing the fatigue level to a fatigue threshold by the at least one processor; Determining an alternative haptic output command comprising an instruction to activate the haptic output device to perform a modified haptic effect, instead of the original haptic effect, based on a comparison of the fatigue level to the fatigue threshold.

The foregoing and other features and advantages of the present invention will become apparent from the following description of embodiments of the invention as illustrated in the accompanying drawings. The accompanying drawings, which are incorporated in and form a part of this specification, further serve to explain the principles of the invention and to enable a person skilled in the relevant art (s) to make and use the invention. The drawings do not scale.

1 is a block diagram of a system for providing a haptic output command to a haptic executable device in accordance with an embodiment of the present invention.
Figure 2 is a schematic diagram of the system of Figure 1, wherein the haptic executable device is a haptic joystick.
Figures 3 and 4 are perspective views of a haptic executable device in accordance with one embodiment of the present invention, wherein the haptic executable device is a portable gaming controller.
5 shows a block diagram of the gaming controller of Figs. 3 and 4. Fig.
6 is a perspective view of a system for providing a haptic output command to a haptic executable device in accordance with an embodiment of the present invention, wherein the haptic executable device is a gaming tablet controller that can be used with a tablet computer.
Figure 7 shows a block diagram of the system of Figure 6;
Figure 8 is a process diagram that illustrates the operation of a system for modifying a haptic effect consistent with an embodiment of the present invention.
9 is a process diagram that illustrates the operation of a system for modifying a haptic effect consistent with an embodiment of the present invention.
Figure 10 is a process diagram illustrating the operation of a system for modifying a haptic effect consistent with an embodiment of the present invention.
Figure 11 is a process diagram illustrating the operation of a system for modifying a haptic effect consistent with an embodiment of the present invention.

The following detailed description is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Further, there is no intention to be bound by any expression or implied theory provided in the technical field, the background of the invention, the solution of the problem, or the concrete contents for carrying out the succeeding invention. It will also be appreciated that while the following description is primarily directed to controllers for gaming devices and gaming devices, it is to be appreciated by those of ordinary skill in the art that such description applies equally to other systems, including virtual reality systems and peripherals for virtual reality systems .

Embodiments of the present invention are directed to a tracking system or automated relief system for preventing or reducing a user's fatigue or numbness during use of a device including a control device and / or a haptic actuator. The solution may include software and / or firmware components and may be implemented by computer instructions executed by at least one processor. The at least one processor may be located in a control device, a haptic device, and / or a central console to which such devices are connected. The proposed solution enables the automatic change of the haptic feedback based on the calculated fatigue level, which can be calculated according to the haptic density representing the user fatigue and other measurements. Haptic density means a combination of different values that define the haptic effect, including intensity or size, duration, number, frequency of repetition, and any other characteristic of the haptic effect. Other measures of user fatigue may include use of control devices including measurements of control device activation, such as trigger pull and button presses, joypad and joystick use, and measurement of the total time spent actively using the control device. Depending on the calculated fatigue level, the system can change the haptic effect to reduce intensity, number, duration, frequency, and / or provide haptic assistance to the user. Thus, the system can cause the haptic effect to be rendered dynamically and automatically in a controlled manner so that the user can optimally feel the haptic effect that reflects the gameplay.

Embodiments described herein relate to devices and systems that may include one or more of a computer system and a haptic executable device.

A computer system consistent with the present invention may be implemented in any computing system that includes a server (e.g., including one or more server blades, processors, etc.), a gaming console, a portable gaming device, a personal computer (e.g., a desktop computer, a laptop computer, And / or other device that can be programmed to provide a haptic output command. In some implementations, the computer system consistent with the present invention may comprise a cloud-based computer platform. (Also referred to herein interchangeably as processors, processor (s) or processors), one or more storage devices, haptic communications units or units, and / or other components. Lt; RTI ID = 0.0 > and / or < / RTI & Various program instructions may be described as performing operations when programming a processor (and hence a computer system) such that the various instructions in effect carry out the operations. The haptic communication unit may be any wired or wireless connected device capable of transmitting or communicating a haptic output command. For example, the haptic communication unit may be a wireless device, such as a haptic effect, And a Bluetooth antenna configured to communicate with the haptic executable peripheral device to communicate the haptic output command to enable the haptic communication device to perform the haptic output command. The haptic communication unit may further include a wired port for communicating the haptic output command In some implementations, the haptic communication Units may be a dedicated unit alone is configured as to convey the haptic output command. In some implementations, the haptic communication unit may function additionally to carry a number of other communication to external devices in a wired or wireless manner.

A haptic executable device includes a device having one or more haptic output devices for communicating a haptic effect to a user. In some implementations, the haptic executable device may be, for example, a device that includes one or more haptic output devices that receive haptic commands directly from the computer system for operation. In some implementations, the haptic executable device may further comprise one or more processors capable of processing or interpreting the received haptic output signal before delivering an actuating signal to the one or more haptic actuators. In some implementations, the haptic executable device may further include control elements, such as a user input element, e.g., a trigger, button, joystick, joypad, etc., to allow the user to interact with the computer system. The haptic executable device may be a haptic executable device, i.e. a device designed to function as an accessory or peripheral unit for a central device such as a computer system consistent with embodiments of the present invention. The haptic executable device may also further include, in some embodiments, all of the functionality of the computer system consistent with the present invention. Thus, the haptic executable device may function as a computer system and may include a haptic output device and a control element.

The haptic output command may be used to directly or indirectly cause activation and / or activation of the haptic output device. In some implementations, the haptic output command may include a haptic output signal transmitted over wire or wireless, which causes the haptic output device to generate a haptic effect. The haptic output signal may include an activation signal received by the haptic output device to cause a haptic effect. In addition, the haptic output signal may include signals transmitted between other system components along with information about the required haptic effect. For example, a computer system processor may output a haptic output signal containing information about a haptic effect that occurs to a processor associated with the haptic executable device. The haptic executable device may receive the haptic output signal, process it, and output another haptic output signal to the haptic output device to cause the haptic effect. Thus, the haptic output signal may comprise any signal used to generate the haptic effect. The haptic output command may further include a software command. That is, the software interaction may generate a haptic output command that includes information for causing the haptic output device to operate. The haptic output command in the form of a software command may cause the processor to generate a haptic output command in the form of a haptic output signal.

In some implementations, the haptic output command may constitute a haptic track, i. E. Information representing a series of haptic effects to be triggered. The haptic track may comprise a predetermined sequence of haptic output commands intended to be executed in sequence. The methods, systems, and devices described herein for generating modified haptic output commands may similarly be applied to generating modified haptic tracks.

According to some embodiments, the haptic executable device and / or the processor associated with the computer system may be configured to change the haptic output command for the haptic output device according to the determined fatigue level. Changing or changing the haptic effect according to the determined fatigue level may be performed using various combinations of the devices and systems described herein. For example, an exemplary computer system may communicate a haptic output command to a processor of the haptic executable device, which may then modify the haptic output command before outputting the modified haptic output command to the haptic output device. In another example, the computer system processor may internally change the software-based haptic output command before generating and sending the haptic output command to cause the haptic output device operation. Various other combinations of the devices and systems described herein may be used to provide haptic command changes in accordance with fatigue levels, and no particular example described herein is intended to be limiting.

 Figures 1-7 illustrate systems and devices consistent with embodiments of the present invention for modifying haptic commands in accordance with a determined fatigue level indicative of user fatigue.

FIG. 1 is a block diagram of a system 100 for providing haptic feedback to a haptic executable device 102 in accordance with an embodiment of the present invention, and FIG. 2 is a schematic diagram of the system of FIG. In the embodiment of Figures 1 and 2, the haptic executable device 102 includes a haptic joystick with a manipulandum 122. Those skilled in the art will appreciate that haptic joysticks are merely exemplary embodiments of haptic executable devices and that haptic executable devices having different configurations, shapes and sizes may be used. For example, as will be described in greater detail herein, a haptic executable device may be a portable for a gaming system as shown in FIGS. 3-5 having similar shapes and sizes to many "game pads & A gaming controller 302, a haptic executable device 602 that may be used with the tablet computer 604 shown in Figures 6 and 7, or a mobile phone, personal digital assistants (PDAs), tablets, But may be other controllers including, but not limited to, user input (UI) elements such as other controllers for a virtual reality system known to those of ordinary skill in the art.

Referring to the embodiment of Figures 1 and 2, the haptic executable device 102 may communicate with the computer system 104 in a wired or wireless manner. The computer system 104 may further include a visual display 106 and may be configured to create a virtual environment for a user on a display, e.g., a screen and / or a set of heads. As shown in FIG. 2, the computer system 104 may include at least one processor 108, memory 110, and visual display 106. The computer system 104 may execute software instructions stored in the memory 110 and executed by the processor 108. [ The processor 108 may include one or more of any type of general purpose processor and may be a processor specifically designed to provide a haptic effect signal. The processor 108 may be the same processor operating the entire computer system 104, and / or may be a separate processor. The processor 108 may execute computer instructions to determine in what order to transmit the haptic commands and haptic commands for dispatch to the haptic executable device 102. [ The memory 110 may include one or more of any type of storage device or non-transitory computer-readable medium, such as but not limited to random access memory (RAM) or read-only memory have. The memory 110 may be located within the host processor, or it may be any combination of internal and external memory.

The computer system 104 may be coupled to the visual display 106 via wired or wireless means. The visual display 106 may include any type of media that provides graphical information to a user, such as, but not limited to, one or more monitors, a television screen, a plasma LCD, a projector, or any other display device. In one embodiment, the computer system 104 may be a gaming device console, and the visual display 106 may be a monitor coupled to a gaming device console, as is known in the art. In other embodiments, computer system 104 and visual display 106 may be combined into a single device, as is known to those of ordinary skill in the art.

The computer system 104 may also include a haptic communication unit 105. The haptic communication unit 105 may comprise a wired or wireless communication unit. The haptic communication unit 105 may be configured to send or send a haptic command to the haptic executable device 102. [ In some implementations, the haptic communication unit 105 may be dedicated to providing haptic commands. In some implementations, the haptic communication unit 105 may be configured for a wide variety of communication tasks, including, but not limited to, providing haptic commands. Additional communication tasks may include, for example, control inputs and outputs, output to additional accessory devices, and so on.

As shown in Figures 1 and 2, the computer system 104 may communicate with the haptic executable device 102 via a wired connection 103. In other embodiments, the haptic executable device 102 may communicate with the computer system 104 using wired or wireless communication means known to those of ordinary skill in the art. These may include, but are not limited to, serial or Bluetooth connections. In addition, the computer system 104 may reside in the cloud and may not need to be wired or wirelessly connected in a local manner.

As shown in FIG. 2, the haptic executable device 102 may include a housing or base 120 and a to-be-operated portion or user input device 122 that can physically move within one or more degrees of freedom. The haptic executable device 102, including the to-be-operated portion and / or other input receiving structure, may also be referred to as a control device. And the to-be-operated portion 122 may extend from the housing 120. [ Although FIG. 2 illustrates a joystick as a to-be-operated portion of a haptic executable device, one of ordinary skill in the art is aware that the present number is not limited to the joystick-to-be-manipulated portion but may also be moved in whole or in part, It will be understood that it includes any possible devices. It will be appreciated by those of ordinary skill in the art that the joystick is only one exemplary embodiment of the to-be-operated portion of the controller and that the to-be-operated portion having a different configuration, such as a trigger, button, or other user- It can be used. In some implementations, the haptic executable device 102 may not include any input receive structure, and may be configured for haptic output only.

1, the haptic executable device 102 may include a processor 112, a memory 114, a to-be-operated sensor 123, and at least one haptic output device 118. The haptic executable device 102 may alternatively be configured not to include the processor 112 so that all input / output signals from the haptic executable device 102 are processed directly by the computer system 104 Treated and treated. The processor 112 may be coupled to the haptic output device 118 to provide a haptic output command to the haptic output device based on the haptic output command received from the computer system 104. Similar to processor 108, processor 112 may determine a haptic command to be sent to one or more haptic output devices 118 to cause a haptic effect. In addition, the haptic executable device 102 includes more than one haptic output device, and the processor 112 may determine that the haptic output device will receive the haptic output command. Local memory 114, which may also be any type of storage device or computer-readable medium, may reside in RAM (random access memory) or read-only memory (ROM), similar to memory 110 of computer system 104, But is not limited thereto. Local memory 114 may be located within the local processor, or it may be any combination of internal and external memory.

The to-be-operated portion 122 of the haptic executable device 102 may be physically moved within one or more degrees of freedom. For example, the user can move the operated portion 122 to the front, rear, left, or right side. When the user moves the to-be-operated portion 122, the to-be-operated portion sensor 123 can detect the movement and / or the position of the to-be-operated portion and transmits the sensor signal to the processor 112. [ The processor 112 may then communicate or transmit the sensor signal to the computer system 104. [ Based on the received sensor signal, the computer system 104 may perform actions within the video game and update the virtual environment. Movement of the to-be-operated portion 122 of the haptic executable device 102 represents input from a user, which includes a first-person shooter game, a third person character correlation, a vehicle-related game, or a video game relating to a computer simulation, Which allows a user to interact with a software application that operates on the computer system 104, Movement of the to-be-operated portion 122 may correspond to movement of a computer generated graphical object, such as a cursor or other image, or some other graphical object displayed by the computer system 104 via the visual display 106, , Or some other object that may be found in a game or computer simulation.

In addition to receiving the sensor signal from the to-be-operated portion sensor 123, the processor 112 may also receive a haptic output command from the computer system 104 associated with the haptic effect output from the haptic output device 118 . The processor 112 receives and processes the haptic output command to generate control, actuation, and / or drive signals for the haptic output device 118 based on high-level haptic output commands received from the computer system 104 Thereby providing a haptic output command. The computer system 104 may include a high level haptic output command for the processor 112 such as any type of haptic effect output (e.g., vibration, gesture, detent, Can be provided. The processor 112 may instruct the haptic output device 118 about a particular characteristic (e.g., size, frequency, duration, etc.) of the expected output haptic effect that is consistent with the haptic output command. The processor 112 may retrieve the type, size, frequency, duration, or other characteristic of the haptic effect that matches the haptic output command from the combined local memory 114. [ Depending on the game behavior and control signals received from the computer system 104, the processor 112 may be adapted to output one of a wide variety of haptic effects or sensations, including vibrations, detents, textures, And send a haptic output command to the controller 118.

The haptic output device 118 may be an inertial or kinesthetic actuator known to those of ordinary skill in the art of virtual reality systems. Possible actuators include "ERM" (eccentric rotating mass) actuators in which the eccentric mass is moved by the motor, "LRAs" (linear resonant actuators) in which the masses attached to the springs are driven back and forth, piezoelectric actuators, Electrostrictive friction (ESF), ultrasonic surface friction (USF), or alternatively, electrostrictive materials, such as, for example, electromagnetic motors, moving electromagnetic motors, vibratory haptic actuators, inertial actuators, shape memory alloys, But is not limited to, any combination of the actuators described above. Possible actuators further include kinesthetic and semi-kinesthetic force-feedback actuators. Such a force-feedback actuator may include an electromagnetic motor that is connected to the to-be-operated portion 122 directly or via a transmission linkage or gearing. In another embodiment, the actuator may include a solenoid for changing the stiffness / attenuation of the to-be-operated portion 122 and / or the housing 120, a solenoid for changing the size of the to-be-operated portion 122 and / Lt; RTI ID = 0.0 > haptic feedback < / RTI > In some implementations in which the haptic executable device 102 includes a trigger or a button, the haptic output device 118 may be a device configured to provide resistance or assistance for trigger pull and button push.

In some embodiments, the haptic executable device 102 may further include an additional sensor 126 as shown in FIG. The additional sensor 126 may be a sensor configured to collect additional data regarding the potential fatigue level of the object. Additional sensors 126 may include biometric sensors such as, for example, temperature sensors, humidity sensors, galvanic skin reaction sensors, cameras, and the like. The additional sensor 126 may further include a force sensor configured to measure an amount of gripping force exerted on the housing 120 by a user.

As described above, the haptic executable device 102 is only one exemplary embodiment of a haptic executable device, and haptic executable devices having different configurations, shapes, and sizes may be used. For example, Figures 3-5 illustrate another embodiment of a haptic executable device 302 that may be used in embodiments consistent with embodiments of the present invention. 3 and 4 show another perspective view of a haptic executable device 302 in which the haptic executable device is a portable gaming controller. FIG. 5 shows a computer system 104, a haptic communication unit 105 and a visual display 106, Lt; RTI ID = 0.0 > 302 < / RTI > The housing 324 of the haptic executable device 302 has a configuration that readily accommodates the two hands holding the device by a left-handed or right-handed user. If the general skill in the art can haptic execution device 302 is a video game console system into many one example of "game pad" to the controller of a similar shape and size ever performed only examples that are currently purchased for, Wii ^TM remote or Wii ^TM U controller, Sony® SixAxis ^TM controller or Sony® wand controller, Xbox ^TM controller or controllers and other shapes as well, such as a controller similar to the controller as well as real-world objects (e.g., a tennis racket, golf cluster, baseball bat, etc.), the shape It will be appreciated that controllers of other configurations, including, but not limited to, user input elements, shapes and sizes, may be used.

The haptic executable device 302 includes some user input elements or components to be manipulated, including a joystick 322, a button 330 and a trigger 332. As used herein, a user input element refers to an interface device, such as a trigger, button, or joystick, that is manipulated by a user to interact with the computer system 104. As is known in the art, and as shown in Figures 3 and 4, more than one user input element and additional user input element may be included on the haptic executable device 302. Thus, for example, the present technique of the trigger 332 does not limit the haptic executable device 302 to a single trigger. 5 also shows only one of each of the joystick 322, the button and the directional controller 330, and the trigger 332. However, one of ordinary skill in the art will appreciate that multiple joysticks, buttons, and triggers, as well as other user input elements, can be used as described above.

As shown in the block diagram of Figure 5, the haptic executable device 302 includes a target haptic output device or motor 318A, 318B, 318C to directly drive each of its user input elements 322, 330, 332, As well as one or more conventional haptic output devices (326, 328) coupled to the housing (324) at locations where the user's hand is generally located. The joystick 322 may include a target haptic output device or motor 318A coupled to it and the button 330 may include a target haptic output device or motor 318B coupled thereto, May include a target haptic output device or motor 318C coupled thereto. In addition to the plurality of target haptic output devices, the haptic executable device 302 may include a position sensor coupled to each of its user input elements. The joystick 322 may include a position sensor 323 coupled thereto and the button 330 may include a position sensor 331 coupled thereto and the trigger 332 may be coupled to a position sensor 333 ). The processor 312 may be individually coupled to the position sensors 323, 331 and 333 of the joystick 322, the button 330 and the trigger 332 as well as the target haptic output devices 318A, 318B and 318C. In response to the signals received from the position sensors 323, 331 and 333, the processor 312 individually controls the joystick 322, the button 330 and the trigger 332, Haptic output devices 318A, 318B, and 318C. Such a target effect is identifiable or distinguishable from a generic or rumble haptic effect generated by a general haptic output device 326, 328 along the entire body of the controller. For example, as multiple aspects such as video, audio, and haptics are simultaneously associated, holistic haptic effects can provide the user with a greater immersion experience for the game. Similar to the haptic executable device 102 and the computer system 104, the haptic executable device 302 may be configured to communicate with the computer system 104, for example, via the haptic communication unit 105. The processor 312 of the haptic executable device 302 may be coupled to each haptic output device to provide a haptic output command based on the high level haptic output command received from the computer system 104. [ The haptic output device of the haptic executable device 302 may be any type of haptic output device enumerated herein for the haptic output device 118 of the haptic executable device 102. Similar to the haptic executable device 102, the haptic executable device 302 may include an additional sensor 126 configured to collect additional data related to the potential fatigue level of the object, as shown in FIG. 5 have.

Figures 6 and 7 illustrate a haptic executable device 602 in accordance with another embodiment of the present invention wherein the haptic executable device 602 is a gaming tablet controller that can be used with the tablet computer 604, One of ordinary skill in the art will understand the integrated visual and audible output. The tablet computer 604 may be a commercially available device, such as but not limited to a device specifically designed for gaming activities available from, for example, Razer Inc., and Apple® iPad®, Kindle® Fire® and Samsung® Galaxy Tab® Lt; RTI ID = 0.0 > device. ≪ / RTI > The haptic executable device 602 includes a docking portion 640 configured to receive tabs 642 and 644 and a tablet computer 604 on which a controlled portion is disposed to control a game on the tablet computer 604 . The docking portion 640 is a tabbed portion of the tablet computer 604 so that the user's actions on the tabs 642 and 644, such as button presses, joystick movements, trigger presses, 604 to the haptic executable device 602.

Handles 642 and 644 include a conventional controlled member or user input element found on the controller. The to-be-operated portion will be described with respect to the handle 644. [ However, those of ordinary skill in the art will appreciate that the same or similar controlled member may be used on the handle 642. In particular, the handle 644 may include a joystick 622, a button 630, and a trigger 632. 6, more than one of each of these respective user input elements may be included on each handle 642, 644, as is known to those of ordinary skill in the art. The handles 642 and 644 may also be used to provide general or rumble haptic effects to the handles 642 and 644, as described above in connection with the conventional or rumble haptic output devices 326 and 328, And a general or rumble haptic output device (626, 628) attached thereto at a location located at the location of the haptic device.

Haptic executable device 602 may include a processor 612 that communicates with tablet computer 604 via docking portion 640, as shown in the block diagram of Fig. 7 also shows only one of each of the joystick 622, the button and the directional controller 630, and the trigger 632. However, one of ordinary skill in the art will appreciate that multiple joysticks, buttons, and triggers, as well as other user input elements, can be used as described above. The processor 612 may be individually coupled to the position sensors 623, 631, 633 of the joystick 622, the button 630 and the trigger 632 as well as the target haptic output devices 618A, 618B, 618C. In response to the signals received from the position sensors 623, 631 and 633, the processor 612 individually controls the joystick 622, the button 630 and the trigger 632, Haptic output devices 618A, 618B, and 618C. The processor 612 of the haptic executable device 602 may be coupled to each haptic output device to provide a haptic output command thereto based on a high level haptic output command from the tablet computer 604. [ The haptic output device of the haptic executable device 602 may be any type of haptic output device listed herein for the haptic output device 118 of the haptic executable device 102. Similar to the haptic executable device 102, the haptic executable device 602 may include an additional sensor 126 that is configured to collect additional data related to the potential fatigue level of the object, have.

Although Figures 6 and 7 illustrate the haptic executable device 602 as a gaming tablet controller that can be used with the tablet computer 604, one of ordinary skill in the art will recognize that the haptic executable device and the host or tablet computer are the same Device, or housing. ≪ RTI ID = 0.0 > Thus, a gaming tablet controller may not be an associated peripheral, but rather a gaming tablet in which the haptic output device is integrated into the gaming tablet itself as a single device.

Regardless of which haptic executable device configuration or embodiment is used, the processor and / or the computer system processor of the haptic executable device may be configured to modify the haptic output command in accordance with a determined fatigue level indicative of user fatigue. The system processor and / or device processor may generate a haptic output command that is configured to generate a modified haptic effect relative to that originally intended. For example, a software instruction executed on a computer system processor may include an original haptic output command for generating an intended haptic effect or effects. Prior to transmission of the original haptic output command for a suitable haptic output device, an alternate haptic output command configured to generate a modified haptic effect may be determined by the various aspects of the system. For example, the computer system processor may include additional software instructions for determining a level of fatigue and generating an alternate haptic output command indicative of a modified haptic output effect. That is, both the required haptic effect and the modified haptic effect can be determined by software commands executable within the application. In another example, a computer system processor may output a haptic output command indicative of a haptic effect originally intended for another processor associated with the connected haptic executable device. The processor of the haptic executable device can determine the fatigue level and perform the necessary comparisons to determine whether to change the induced haptic effects or effects. Any or all of the operations described below with respect to the determination of the haptic output command to cause the haptic effect and haptic effect change may be performed by a processor associated with the computer system and / or the haptic executable device. In addition, the haptic output command may be processed and modified at the level of the software instructions and / or at the level of the electrical signal.

Figures 8 and 9 are process diagrams illustrating the functionality of the system described herein in performing the haptic modification process. In an embodiment, the functions of the process diagrams of FIGS. 8 and 9 may be stored in the memory of the software and / or firmware and / or haptic executable device stored in the memory of the computer system and executed by the processor of the computer system, Lt; / RTI > may be implemented by software and / or firmware executed by a processor of a processor-enabled device. In other embodiments, the functionality may be implemented in hardware and / or in hardware and / or software by use of an application specific integrated circuit (ASIC), a programmable gate array (PGA), a field programmable gate array May be performed by any combination. In some embodiments, the functionality of the process diagrams of Figures 8 and 9 may be performed by a processor associated with both the computer system and the haptic executable device. Those of ordinary skill in the art will appreciate that the functions of FIGS. 8 and 9 may be implemented within a computer system 104, a haptic executable device 102, a haptic executable device 302, a device consistent with a haptic executable device 602, Or by a haptic executable device or computer system having other configurations known by the system and / or known in the art.

FIG. 8 is a process diagram illustrating a control loop 800 of a haptic change determination process in accordance with an embodiment of the present invention. In one embodiment shown in FIG. 8, the system may be operable to monitor and update the calculated fatigue level on a continuous ongoing basis. As shown in FIG. 8, the control loop 800 may be operable to monitor and increase or decrease the value of the determined fatigue level as required as the occurrence of the haptic effect occurs. When the fatigue level exceeds the threshold value, the haptic output command change flag can be set and subsequently the haptic output command can be changed according to the setting of the haptic output command change flag. The haptic output command change flag may be removed when the fatigue level is reduced below the threshold value. In another embodiment, the system may operate to monitor and update the fatigue level only when the haptic signal is received by the haptic device, as described below.

The control loop 800 of the haptic modification decision process may be a process loop executed by one or more processors associated with the system and device described herein. The control loop 800 may be executed by a processor associated with a computer system as described herein, and / or may be executed by a processor associated with a haptic executable device as described herein. In some implementations, different operations of control loop 800 may be performed in a communication state by different processors, including different processors located in different devices / systems.

As shown in FIG. 8, the haptic effect modification process may begin with the recognition of the haptic output command by the processor in the haptic output command recognition operation 801. The haptic output command may be configured to cause one or more haptic effects in the one or more haptic actuators. Recognition of the haptic output command may include recognition that transmission, reception and / or origination of the haptic output command has occurred.

The transmitted haptic output command includes a haptic output command sent from one processor to another, for example, between a processor of a computer system and a processor of a haptic executable device and / or between two processors located within the same system and device can do. The haptic output command may include a haptic output command sent directly from the processor to the haptic actuator.

The reception of the haptic output command may include a haptic command received by the first processor from the second processor. The first and second processors may be part of the same system or device, and / or may be located in different systems or devices.

The origination of the haptic output command may include determining, based on the software command being executed, whether the particular haptic output command is appropriate. Thus, for example, computer system executable software instructions for playing a video game may originate a haptic output command for generating a haptic effect based on play of the video game.

The haptic output command may also be part of a haptic output track, which in turn comprises several haptic output commands. This haptic output track may be recognized (e.g., received, transmitted, and transmitted) in the same manner as a haptic output command, as described herein.

The haptic output commands transmitted between the processors may be stored in a memory of a memory device such as a memory card or a memory card that contains a desired haptic effect including a type (e.g., rumble, vibration, flex, etc.), size, duration, frequency, Which is a haptic information signal. A haptic output command (e.g., a voltage signal transmitted to the haptic motor) that is configured to directly generate a haptic effect when received by the haptic output device may be referred to as a haptic activation signal. In some implementations, a processor that receives a haptic output command as a haptic information signal may interpret the received command and may output an alternate haptic output command as an actuation signal directly to the haptic actuator. In some implementations, the processor may generate a haptic output command as a haptic information signal and / or a haptic activation signal in response to an executed software command.

For example, a haptic executable device may include two haptic output devices, ERM and LRA. Each of these devices may have specific input requirements and specific output characteristics. The haptic executable device processor may receive the haptic information signal from the computer system and output a haptic actuation signal configured to achieve a desired haptic effect of the haptic information signal as performed by the specific haptic output device of the haptic executable device . In a second haptic executable device, the same haptic information signal may be processed with a different haptic activation signal due to differences in the characteristics of the haptic output device associated with the second haptic executable device.

In some implementations, the haptic output command in the form of a haptic activation signal may be transmitted from one processor to another before being routed to the appropriate haptic output device.

The recognition of the haptic output command performed in the haptic output command recognition operation 801 may determine whether one or more haptic output commands have been received, transmitted and / or originated within the most recent iteration of the control loop. Such a determination can be made, for example, on a continual iterative basis as part of the control loop.

After any haptic output command that occurs after the previous control loop iteration is recognized, the fatigue level may be updated at operation 802. [ The fatigue level, as described above, may be based on at least one of the haptic effect density and the measured user activity. The measured user activity may include a measurement of the control device usage. The use of the control device may be characterized by the amount of time the control device is in use and / or the user activity of the control associated with the control device. Individual values of control device use and haptic effect density can be combined to determine the fatigue level. In some embodiments, only the value of the haptic effect density and / or the value of the control device usage may be used. As discussed above, the calculated value of the haptic effect density can be determined from the haptic output command sent to the haptic output device, and the value of the control device usage can be determined based on the input received from the control device. Thus, the fatigue level indicative of the amount of user fatigue can be determined based on either or both of a haptic output command sent to the haptic output device and an input received from the control device.

The fatigue levels as well as the haptic effect density and / or control device usage can be calculated for the control system as a whole, for the individual actuators, for the individual user input elements and / or for the individual body parts to which the haptic effect is applied. The fatigue level can be calculated for the control system as a whole by considering the entirety of the control device usage within the system and / or the overall haptic effect played by the system. The fatigue levels can be individually calculated for the individual actuators, i. E. Individual fatigue level calculations, which include both haptic effect densities and / or control device use for different actuators associated with a particular user input element. For example, if the user input element has a force-feedback actuator and a vibrating haptic actuator, respectively, the fatigue level can be computed separately for each of those. The fatigue level can be calculated for a particular user input element, i.e. combining fatigue level values including control device usage and / or haptic effect density for all actuators associated with a particular user input element. Fatigue levels, including haptic effect density and / or control device use, may be calculated for individual body portions. For example, if a user input element is arranged to operate multiple user input elements, such as multiple buttons operated by thumb, for which a particular body portion is more than one, the fatigue level value of each button is combined with the fatigue level value of the remaining button So as to generate cumulative fatigue level values for the intended thumb to operate all of them.

The haptic effect density may be determined by activation of one or more haptic output devices associated with the monitored haptic executable device. The haptic effect density may also be determined according to the haptic output command sent by the processor to cause the haptic effect. In some implementations, the haptic effect density may be determined according to the haptic output command transmitted by the processor without knowing whether the haptic command was executed by the haptic output device and how it was executed. For example, a video game controller may have two actuators on either side of the controller, and two additional actuators on each trigger, which may provide both vibrotactile and kinesthetic haptic feedback. In this example, each actuator may have its own calculated haptic effect density, and their respective haptic effect density values may be combined as needed.

The haptic effect density can be calculated using various parameters. These factors may include the size of the effect, the duration of the effect, the frequency or duration of the effect, the user preference setting how much haptic feedback the user desires, and / or one or more previously performed haptic effects, May include other factors. Further, the haptic effect density calculation may be based on, for example, how much the trigger is pressed, the throw of the trigger or the percentage of distance in use, the strength of the actuator and the subsequent impact force on the user's finger, and / Etc. < / RTI > Further, the haptic effect density can be changed by combining one or more factors with other factors. For example, when the trigger is fully depressed, the perceived intensity of the haptic effect computed through the trigger may be reduced compared to when the trigger is depressed by half. Thus, the calculated haptic effect density of the trigger oscillation can be reduced for a fully pressed trigger, for example, compared to a half-triggered trigger. Any or all of the exemplary factors may be combined to calculate the haptic effect density.

The effect of each haptic effect on the haptic effect density calculation may be based on the haptic density accumulation factor and the haptic density decay factor. The haptic density accumulation factor indicates how strongly the haptic effect contributes to the haptic effect density value, while the haptic density attenuation factor indicates how quickly such contribution is removed from the haptic effect density value.

The haptic density accumulation factor of the haptic effect refers to the amount by which the effect increases the measurement of the haptic effect density, based on the factors described above. The haptic density accumulation factor can be regarded as a normalization factor that enables the same scale measurement because it can serve to provide a measure of the haptic effect density for haptic effects of different classes and characteristics. For example, a haptic effect with a large size and a long duration may have the same size but a higher haptic density accumulation factor than an effect with a shorter duration. In another example, the haptic density accumulation factor may be used to normalize the addition to the haptic effect density from other types of haptic effects. For example, although the vibrotactile effect and the kinesthetic effect are characterized by different parameters, each has a haptic density accumulation factor so that these different types of effects can be measured on the same scale. Since the haptic effect density is adjusted by the haptic density attenuation factor for each haptic effect previously played, it can be determined by summing the haptic density accumulation factors for each haptic effect previously played.

Each haptic effect may also include a haptic density attenuation factor. The haptic density attenuation factor may be a measure of how quickly the effect of the haptic effect on the haptic effect density is reduced. The haptic density attenuation factor may be determined by both the attenuation rate and type of the effect of the haptic effect on the haptic effect density over a specified decay period while the effect of the haptic effect is maintained. The haptic effect can increase the haptic effect density based on the haptic density accumulation factor of the haptic. The increase for the haptic effect density caused by the haptic effect can then be reduced over the specified damping period of the haptic effect, depending on the attenuation rate of the haptic density attenuation factor. In an embodiment of the invention, the haptic density attenuation factor may be linear and may cause a reduction in the haptic effect density at a constant rate over a specified attenuation period until the contribution of the attenuating haptic effect is zero. In another embodiment of the present invention, the haptic density attenuation factor can be exponential, causing a reduction in the haptic effect density at exponential increase or decrease rates over a specified attenuation period. In another embodiment of the present invention, the haptic density attenuation factor can be logarithmic, which can cause a reduction in haptic density at an algebraic increase or decrease rate over a specified decay period. In another embodiment of the present invention, the haptic density attenuation factor may also be binary, so that the total contribution of the haptic effect can be removed from the haptic effect density at a time after the specified attenuation period. The haptic density attenuation factor may also include any other suitable mathematical function that may be applied to reduce the amount contributed by the specific haptic effect to the haptic effect density. The haptic density attenuation factor, e.g., the rate and type of attenuation and the designated attenuation period, may be determined according to the nature of the haptic effect. For example, the vibrotactile effect may have different size and type of attenuation than the kinesthetic effect.

In some embodiments, the haptic density attenuation factor for the haptic effect and the designated attenuation period can be dynamically assigned according to user activity. Increased user activity can serve to extend the specified decay period, and / or reduce the rate at which the haptic density attenuation factor reduces the contribution of the haptic effect to the haptic effect density. Thus, under conditions of high user activity, such as fast trigger pull or button press, the contribution to the haptic effect density of any individual haptic effect can be kept longer and can be reduced at a slower rate.

As a specific example, the density range can be from 0 to 10, where 0 indicates no haptic density / fatigue and 10 indicates maximum haptic density / maximum fatigue. In the example, the maximum magnitude effect for a particular haptic output device that repeats at 10 ms intervals for 2 seconds can yield a haptic density accumulation factor of 3. An additional effect of operating at 100 ms at 35% size can yield a haptic density accumulation factor of 2. If the generated effects are close in time to each other, a haptic effect density of about 5 will be obtained. If the effect has a linear haptic density attenuation factor of 30 seconds, the contribution of each effect to the fatigue level will start damping at a constant rate immediately, which will result in 0 after 30 seconds for each contribution. In this example, the fatigue level will be approximately 2.5 after 15 seconds.

As discussed above, the fatigue level may include both the calculation of the haptic effect density and the measurement of the use of the control device. The measurement of the control device usage reflects how long the user has used the control device and / or how often the device is used. Thus, a high fatigue level can be determined for a user who has operated the controller for a longer time than the user who operated the controller for a shorter time. The measurement of the control device usage may include how often the control device was used, for example, how many trigger bumps, button presses, joystick / joypad movements, etc. were performed. In some implementations, the depth of trigger pull, button press, etc. may be included in the specification of control device usage. In some implementations, the amount of force required for each of these actions may be included in the measurement of the control device usage. The specification of the control device usage may be determined by the control device usage accumulation factor and the control device usage attenuation factor, similar to that described above for the haptic effect. That is, different control measurements can contribute a relatively higher amount to the measurement of control device usage, and each control movement can be attenuated from the measurement of control device usage over time. The control device use attenuation factor may vary in magnitude and type, including both of the same type as the haptic density attenuation factor.

At operation 802, the fatigue level may be updated based on the haptic effect and / or control device usage that has occurred since the previous iteration and update of the fatigue level. Each generated haptic effect can contribute to the value of the haptic effect density according to the haptic density accumulation factor. The use of each control device can contribute to the value of the control device usage depending on the accumulation factor of that control device.

Further, at operation 802, the haptic effect density and control device usage measurements may be reduced as the contribution from the haptic effect dampens based on the time since they were generated. In some implementations, each haptic effect and control device usage contributing to the fatigue level may be stored according to their accumulation factor, attenuation factor, and time of occurrence. In some embodiments, all characteristics of the generated haptic effect and / or control device usage can be stored. Based on the attenuation factor, time of occurrence and current time, the contribution of each haptic effect and / or control device usage to the fatigue level, e.g., based on its accumulation factor, can be reduced to determine the current fatigue level.

As discussed above, the system may include additional sensors to collect additional data for use in fatigue level calculations. For example, the additional sensor may include a biometric sensor. Biometric data from the biometric sensor may be included in the fatigue level calculation in some embodiments. For example, a change in temperature of a particular body portion may indicate increased or decreased blood flow as a result of the haptic effect. Other biometric data may be monitored to calculate fatigue levels, including, but not limited to, sweat level, pupil dilation, galvanic skin response, and the like. Each of these can measure user activity and / or stress that can contribute to user fatigue. In some embodiments, the system may include additional force sensors within the housing of the controller. Such a force sensor can measure the amount of force the user uses to grasp or retain the controller. For example, an increased gripping force from an activated muscle group, such as a grabbed fist or hand, can increase user fatigue.

The fatigue level can be compared to the fatigue threshold at operation 803. [ A comparison between the fatigue level and the fatigue threshold can be used to determine whether the haptic effect is to be altered or not. The fatigue threshold may include one or more threshold levels. A loose threshold level can be used to determine the reduction of the haptic effect. A strict threshold can be used to determine if one or more types of haptic effects will be removed. Based on the comparison between the fatigue level and the fatigue threshold, the system determines whether to change some or all of the haptic output commands, e.g., reduce or eliminate the planned haptic effect from the haptic effect playback, and / or change some or all of the haptic output commands Or not.

The fatigue threshold used in the comparison with the fatigue level to determine the haptic effect change can be predetermined, dynamically determined, and / or user defined. In some implementations, the predetermined fatigue threshold may be set according to a default setting that is determined by the average user fatigue experience. In some implementations, the user may be allowed to adjust the predetermined fatigue threshold, which allows the threshold to be increased or decreased according to user preference to set a user defined fatigue threshold. In some implementations, the user may be allowed to adjust the threshold for a particular haptic output device and / or a specific haptic effect. In some implementations, the fatigue threshold can be determined dynamically. The dynamic fatigue threshold may be determined, for example, according to the measured user activity. User activity, including specific user input elements, actuators, body parts, and / or increased user activity of the entire controller, can lower the fatigue threshold required for haptic effect changes. Conversely, reduced user activity can increase the fatigue threshold required for haptic effect modification. For example, some gaming situations require fast and repetitive action, such as fast trigger pull. This type of user activity can be used, for example, to dynamically adjust lower fatigue thresholds. Thus, although the haptic effect associated with fast trigger pulling can cause a fast accumulation of haptic effect density, fast trigger pull may also lower the dynamic threshold for haptic effect modification.

As discussed above, a particular level of fatigue can be calculated for the entire control system, for individual actuators, for individual user input elements, and / or for individual body portions to which the haptic effect is applied. The fatigue threshold may similarly be specified for the control system as a whole, for the individual actuators, for the individual user input elements, and / or for the individual body parts. Each of these aspects may have an associated fatigue profile that defines a fatigue threshold at which the haptic effect is to be altered for a particular aspect. For example, the fatigue threshold for comparing trigger fatigue levels is different from the fatigue threshold for comparing joystick fatigue levels. Similarly, the fatigue threshold for the thumb fatigue level differs from the fatigue threshold for the index finger fatigue level.

When the comparison between the fatigue level and the fatigue threshold determines that the fatigue level has exceeded the fatigue threshold, a decision to change the haptic effect may be made at operation 804. For example, if the comparison determines that the fatigue level has exceeded a soft threshold, the haptic effect change flag may be set. The haptic effect change flag signals a signal to a processor (e.g., a processor of a computer system or an associated haptic executable device) to change any or all of the haptic output commands to effect a reduction in the haptic density accumulation factor of the associated haptic effect Can play a role. The haptic effect change flag may be binary, indicating only whether the haptic effect is to be changed or not. In this case, the amount of modification of the haptic effect can be determined in advance and / or can be determined by the user. The haptic effect change flag may include a value indicating the amount by which the haptic effect is to be reduced. In some embodiments, the value of the haptic modification flag may be determined according to the amount by which the fatigue level exceeds the loose threshold value. In another example, if the comparison between the fatigue level and the fatigue threshold determines that the fatigue level has exceeded a stringent fatigue threshold, the haptic effect change flag may be set to indicate that the future haptic effect will be canceled.

The haptic effect modification may include a change to the haptic output command that alters the effect produced by the haptic output device. Such a modification may include adjustment to the planned haptic effect to reduce the haptic density accumulation factor and thereby reduce the future haptic effect density level. Thus, the haptic output command may be modified to reduce the intensity, magnitude, duration and / or frequency of the haptic effect. The haptic output command may be used to reduce the haptic density accumulation factor of the planned haptic effect by a certain amount, to reduce the haptic effect to a predetermined level, and / or to reduce the haptic effect by a predetermined percentage of the original intended value . In some implementations, the haptic output command may be changed to change the haptic density accumulation factor of the individual haptic effect. For example, the total haptic density accumulation factor of a series of vibrations may be reduced by decreasing the intensity and / or duration of each oscillation in series. In some implementations, the haptic output command may be modified to change the haptic density accumulation factor of the grouping of individual haptic effects. For example, the total haptic density accumulation factor of a series of vibrations can be reduced by eliminating each other vibration.

In some embodiments, the haptic effect modification may include switching from the original haptic output track to the modified haptic output track. The original haptic output track may be a standard haptic output track used to cause unaltered haptic effects. If a haptic change flag is set and a haptic effect change is desired, the alternative and modified haptic output track can be used to cause the modified haptic effect until the fatigue level drops back below the threshold. Both the original haptic output track and the modified haptic output track can be pre-recorded.

In some embodiments, the haptic effect modification may include a change to the haptic output command to reduce the control device usage accumulation factor. For example, firing of the trigger pull or increasing distance causes an increase in the accumulation factor of the control device usage. The modification of the haptic output command may include reducing the amount of trigger firing required to effect the desired effect in the game being played by the user. Modifications to the haptic output command may assist in pulling the user's trigger by using a force feedback haptic output device to reduce the amount of force required to pull the trigger and / or to provide a force to help the user complete the trigger pull . Similar changes may be made with the haptic output device in connection with other types of user input methods including buttons and joystick / joypad.

In some embodiments, the haptic effect modification may include alterations to alternate actuators, alternating user input elements, and / or haptic output commands to cause a haptic effect in alternating body portions. If the fatigue level associated with a particular actuator, user input element and / or body portion exceeds the fatigue threshold associated with such actuator, user element and / or body portion, the haptic effect may be implemented in different actuator, user element and / . For example, if it is determined that the user's right index finger has an associated fatigue level that exceeds the fatigue threshold, the haptic output command may be applied to the right index finger by activating an actuator positioned where the user's right thumb is intended to rest The intended haptic effect can be changed to be experienced by the user ' s right thumb. Thus, the user experiences a haptic effect at least somewhat similar to the originally intended haptic effect, without experiencing excessive fatigue at any one finger.

In some embodiments, the haptic effect modification may include replacing the originally intended haptic effect with auditory and / or other effects. For example, if the fatigue level associated with the entire controller exceeds the fatigue threshold, the haptic effect modification may involve combining an increased size auditory effect with a lowered size rumble effect.

When the comparison between the fatigue level and the fatigue threshold at operation 803 determines that the fatigue level did not exceed the fatigue threshold, a determination may be made at operation 805 that the haptic output command is not changed. When such a result occurs, any preset haptic modification flags can be eliminated and all future haptic effects can be generated as originally intended until a new haptic modification flag is set.

In some implementations, the system may determine a plurality of haptic modification flags according to a group of multiple haptic output devices. As discussed above, the control device associated with the system may include more than one haptic output device. During the control loop 800 of the haptic modification decision process of Figure 8, the fatigue level may be determined for a group of multiple haptic output devices, which groups individually for each haptic output device in the control device, And / or groups for all haptic output devices. Each determined fatigue level may be compared to a fatigue threshold suitable for a group of haptic output devices associated with it, and the system may set a separate haptic change flag for each group of haptic output devices. Thus, in effect, the system can determine that, for example, the haptic output command attached to the left trigger is subject to change while the haptic output command attached to the right trigger remains unchanged. In some implementations, a single process control loop 800 may be performed to determine haptic output changes for all haptic output device groups. In some implementations, multiple process control loops 800 may be implemented, each of which is associated with determining a haptic modification for one or more haptic output device groups.

The control loop 800 of the haptic modification decision process of FIG. 8 is completed in a loop iteration operation 806. FIG. The control loop then returns to the haptic output command recognition operation 801 to determine if a haptic output command has occurred since the previous haptic output command recognition operation 801. [

As a result, the control loop 800 of the haptic modification determination process of FIG. 8 may maintain a continuously updated determination of the fatigue level and may set the haptic modification flag based on its level of fatigue. The haptic change flag may be referenced by a processor associated with the system to determine whether to change or not to change the haptic output command as shown in the haptic command process 900 shown in FIG.

FIG. 9 illustrates a haptic command process 900 executable by one or more processors capable of modifying and outputting a haptic output command.

At operation 902, the haptic output command may be determined by the computer system and / or at least one processor associated with the connected haptic executable device. The determined haptic output command may be configured to cause one or more haptic effects by one or more haptic output devices associated with the haptic executable device. The haptic output command may thus include an instruction to activate the haptic output device to perform the original haptic effect. The determination of the haptic output command may include causing and / or receiving a haptic output command. Triggering a haptic command may include generating a haptic output command based on execution of a software command, as described above. Receiving the haptic output command may include receiving a haptic output command from one processor by another processor.

At operation 904, the haptic command change state may be determined by the at least one processor associated with the computer system and / or the connected haptic executable device. The haptic command change state may include information about whether to change the specific haptic command and how to change it. In the implementation of FIG. 9, the haptic command change state may include information about whether to change the haptic output command and how to change it. As described above, the haptic command modification may include a change to the intensity, size, frequency, and duration of one or more haptic effects embodied by the haptic output command. The haptic command modification may further include removal of one or more specified haptic effects. In some implementations, the determination of the haptic command change state may include identifying a haptic change flag corresponding to the determined haptic output command. For example, a haptic output command for providing a vibration haptic effect to a specific haptic output device may lead to an identification of all haptic change flags associated with the particular haptic output device. As described above, the haptic output device may be included in one or more haptic output device groups for change flag purposes. Further, as described above, the haptic modification flag may indicate how the haptic command is changed, for example, by a reduction in effect and / or a cancellation of the effect. In accordance with the haptic change flag, the haptic command change state can be determined.

At operation 906, an alternate haptic output command may be determined by the computer system and / or at least one processor associated with the connected haptic executable device. The alternate haptic command may be determined based on the haptic output command and the haptic command change state.

The alternate haptic output command may be a haptic output command that is configured to cause the same haptic output effect as indicated by the original haptic output command, if the haptic command change state indicates that a change is not required. In this case, the original haptic output command and the alternative haptic output command may be the same. In this scenario, the original haptic output command and the alternative haptic output command may be different. As discussed above, the haptic output commands may differ depending on their source and target. For example, a processor associated with a haptic executable device may receive a haptic output command from a processor associated with the computer system. The haptic executable device processor may determine that a haptic effect change is not required based on the haptic command change state but may output a modified haptic command modified for reception by the haptic output device associated with the haptic executable device. For example, a haptic executable device processor may receive a haptic information signal that requires a specific set of planned haptic effects. The haptic executable device processor may process the haptic information signal when received by the haptic output device and output a haptic activation signal configured to cause a specific set of projected haptic effects. Thus, in this situation, the first receiving haptic output command and the second transmitting haptic output command are different, but the alternate haptic output command is not modified for haptic effect modification.

The alternate haptic output command may be a haptic output command that is configured to cause a modified haptic effect to that indicated by the original haptic output command if the haptic command change state indicates that a change is needed. Any and all haptic effect changes described herein can be applied to alternate haptic output commands.

In some implementations, if the haptic command change state indicates that a haptic effect change is needed, the processor may determine, based on an override indicator of the original haptic output command, that the same haptic effect change as indicated by the original haptic output command Lt; RTI ID = 0.0 > haptic < / RTI > For example, in video games, certain haptic effects are essential to game play, so changing it may be detrimental to the user experience. The haptic output command indicating such a haptic effect may include an invalidation indicator. Upon identifying the invalidation indicator, the processor may determine an alternate haptic output command indicative of the original haptic effect despite the haptic command change state indicating that an effect change is required.

At operation 908, an alternate haptic output command may be output by the computer system and / or at least one processor associated with the connected haptic executable device. As discussed above, the alternate haptic output command may be a command to indicate a command to generate a haptic effect, and to generate a haptic effect that has not changed or changed, depending on the result of operation 906. [ The alternate haptic output command may be output by the processor to cause the haptic effect to be changed or unaltered directly or indirectly. The alternate haptic output command includes, for example, a haptic control signal sent directly to the haptic output device to actuate the haptic output device to trigger operation. In addition, the alternate haptic output command may include, for example, commands issued indirectly to the haptic output device via another processor.

In an alternative implementation consistent with FIG. 9, the fatigue level may be updated and a threshold comparison may occur when a haptic output command is received. In such an implementation of the system, the control loop 800 of the haptic modification determination process of FIG. 8 is not required. Instead, the haptic command change state at operation 904 may include updating the fatigue level and comparing the updated fatigue level to the fatigue threshold to determine whether to change the haptic output command. Fatigue level updates and threshold comparisons may occur when a haptic output command is received. In this embodiment, the fatigue level may be updated in the same manner as described for operation 802. [ All previous haptic effects that have occurred since the previous fatigue level update can be processed to update the fatigue level according to their respective haptic density accumulation factors. The use of the control device after the previous fatigue level update may be processed to update the fatigue level according to the respective control device usage accumulation factor. The fatigue level may be further updated in accordance with the haptic density attenuation factor, the control device use attenuation factor, and the time elapsed since the occurrence of each haptic effect and control device usage. After the update, the fatigue level may be compared to the fatigue threshold to determine the haptic command change state, similar to that described above for operation 803. The determined haptic command change state may include information about whether to change the original haptic output command and how to change it. As described above, the haptic command modification may include a change to the intensity, size, frequency, and duration of one or more haptic effects embodied by the haptic output command. The haptic command modification may further include removal of one or more specified haptic effects.

Figure 10 is a process diagram illustrating the operation of a system for modifying a haptic effect consistent with an embodiment of the present invention. The haptic system command process 1000 may be executed by one or more processors to change the haptic density in the system. The haptic system command process 1000 can operate continuously during use of the haptic playback system, continuously calculate the haptic density, and adjust the haptic output level as needed.

In operation 1002, the haptic playback system is started. The haptic playback system includes various haptic executable devices for use with the system. For example, the haptic playback system may include a haptic executable trigger, a button, and a joystick of one or more one- or two-hand controllers as shown in Figs. 2-4, as well as any rumble type vibration actuator included therein .

At operation 1004, referring to FIG. 10, the haptic effect density, which may also be referred to as the density of the haptic reproduction or the haptic reproduction density, can be calculated. The density of haptic playback can be calculated for the system as a whole, for each individual haptic executable device in the system, and for a particular user body portion. For example, in a system comprising a controller with left and right rumble devices as well as left and right haptic executable triggers, each rumble device and trigger may have its own haptic effect density, while a combination of all of the devices Haptic effect density. If the system includes a controller with a right bumper intended to be activated by the same body part, e.g. a right index finger, and a multipurpose control part intended to be used by a right trigger, then the system will have the haptic effect density experienced by the right index finger The haptic effect densities associated with both the trigger and the bumper can be combined. The haptic effect density may be calculated according to the size, duration, and frequency or duration of the haptic effect being performed. For a haptic executable force-feedback trigger, the haptic effect density can be calculated according to the percentage of trigger pressures, the percentage of firing or distance of trigger movements, the amount of impact force on the user's fingers, the position of the user's fingers on the trigger, have. Each executed haptic effect may contribute to the haptic reproduction density for a specified period of time after execution, e.g., during the decay period. During the decay period, the contribution of the specific haptic effect to the haptic reproduction density can be reduced based on the time since the execution. In some implementations, the contribution of the specific haptic effect to the haptic reproduction density can remain the same throughout the damping period. After the decay period expires, the contribution of each haptic effect to the reproduction density is eliminated.

At operation 1006, the system may determine whether the density of haptic playback in the system requires adjustment. The adjustment decision may be made based on a comparison between one of the predetermined density threshold or the user preference density threshold and the calculated haptic playback density of the overall system. If the haptic reproduction density exceeds the threshold value, a decision can be made to reduce the density of the haptic reproduction. If the haptic playback density does not exceed the threshold, a determination may be made to increase the density of the haptic playback to correspond to the base or original haptic output level.

At operation 1008, the system may adjust the haptic output level in accordance with the determination made at operation 1006. [ If the haptic playback density exceeds a threshold (of a predetermined or user preference), the haptic output level may be adjusted to reduce future haptic playback density. The haptic output level can be adjusted by changing the size, duration, and frequency or duration of each haptic effect. In the case of a force feedback trigger, the allowable throw distance can be reduced to a percentage of the maximum firing distance. For example, a force feedback actuator may limit the total distance that can be used for each trigger pull. At the conclusion of operation 1008, the system flow may return to operation 1004 to calculate the haptic reproduction density.

During execution of the haptic command process 1000, other system behaviors, such as game-play, may be contiguous apart from the haptic command process 1000. The haptic command process 1000 can be continuously cycled through operations 1004, 1006, and 1008 to calculate the current haptic playback density and adjust the haptic output level up or down accordingly.

Figure 11 is a process diagram illustrating the operation of a control loop for modifying a haptic effect consistent with an embodiment of the present invention. The haptic control loop 1100 may be executed by one or more processors to change the haptic effect density in the system.

At operation 1102, it is determined if a haptic effect is currently being executed. A haptic density accumulation factor, also referred to as the density value or rating of the effect, is added to the total haptic effect density or fatigue value in the system at operation 1103, and the control loop The process may return to operation 1102 to continue validation for haptic effect execution and proceed to operation 1105 to determine whether the haptic effect or previous haptic effect has occurred within their specified damping period or decay threshold, Will be.

If it is determined that the effect is not being played at operation 1102, then the density value or rating of the recently executed haptic effect at operation 1104 can be determined. This density value or rating is then evaluated based on their haptic density attenuation factor to determine if an effect occurred within the specified attenuation period or attenuation threshold at act 1105. [ The haptic density value or grade of the effect played within the designated damping period can continue to contribute to the total haptic effect density value at operation 1103. [ In some implementations, this haptic effect density class may be reduced with time after execution, i.e., with a decay rate for a specified decay period. The haptic effect density grade of the effect played outside the specified damping period may be removed from the haptic effect density value and the haptic effect density value may be reduced by an appropriate amount in operation 1106. [

At act 1107, the total haptic effect density value is compared to a predetermined fatigue, threshold, dynamic fatigue threshold, and / or user defined fatigue threshold to determine if the haptic effect density value remains within acceptable parameters . If the haptic effect density value exceeds the threshold value, additional, next, or future haptic effects may be altered, e.g., reduced, attenuated, or eliminated at operation 1109. If the haptic effect density value does not exceed the threshold value, the additional, next, or future haptic effects may remain unchanged at 1108. [ Following operations 1109 and 1108, control may return to the start of haptic control loop 1100.

Accordingly, a system, a device, and a method for performing a haptic effect change according to a fatigue level indicating a user fatigue are provided. While various embodiments in accordance with the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. It will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention. Accordingly, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the appended claims and their equivalents. It is also to be understood that each reference incorporated herein and each configuration of each embodiment described herein may be used in combination with the configuration of any other embodiment. That is, aspects of the method for rendering a haptic effect can be used in any combination with the other methods described herein, and the methods can be used separately. All patents and publications described herein are incorporated herein by reference in their entirety.

Claims (26)

A device for modifying a haptic effect of a haptic executable device,
Comprising at least one processor,
The at least one processor
Determining a haptic output command including an instruction to activate the haptic output device to perform the original haptic effect,
Determining a fatigue level indicative of an amount of user fatigue based on at least one of a haptic output command sent to the haptic output device and an input received from the control device,
Comparing the fatigue level to a predetermined fatigue threshold,
Determining an alternative haptic output command comprising an instruction to activate the haptic output device to perform a modified haptic effect, instead of the original haptic effect, based on a comparison of the fatigue level to the predetermined fatigue threshold
Configured to execute computer instructions
device. 2. The method of claim 1, wherein the fatigue level is determined based on a haptic effect density determined by a characteristic of a previous haptic effect output by the haptic output device, the characteristic comprising at least one of a frequency, / RTI > 3. The apparatus of claim 2, wherein the at least one processor
Further comprising instructions to increase the value of the haptic effect density according to the characteristics of the previous haptic effect. 3. The apparatus of claim 2, wherein the at least one processor
Further comprising decreasing the value of the haptic effect density according to the elapsed time since the occurrence of the previous haptic effect. 2. The method of claim 1, wherein the at least one processor further comprises instructions to increase a fatigue level in accordance with a measured user activity comprising at least one of an amount of time using the control device and activation of a control associated with the control device device. 6. The device of claim 5, wherein the at least one processor further comprises decreasing a fatigue level according to an amount of time since the most recent activation of control associated with the control device. 2. The method of claim 1, wherein the processor instructions for determining an alternate haptic output command include instructions for determining an alternate haptic output command comprising instructions for generating a modified haptic effect, A duration, and a frequency. 2. The device of claim 1, wherein the processor instructions that determine the alternate haptic output command comprise instructions to remove the original haptic effect. 2. The device of claim 1, wherein the processor instructions for determining a replacement haptic output command comprise instructions for providing haptic assistance to a user. 2. The device of claim 1, wherein the at least one processor further comprises instructions for receiving a user input to determine the predetermined fatigue threshold. 2. The device of claim 1, wherein the device comprises a haptic output device. 12. The device of claim 11, wherein the device comprises a control device. The apparatus of claim 1, further comprising a haptic communication unit,
The haptic communication unit
Connected to a haptic executable device comprising a haptic actuator,
Configured to output a haptic output command to the connected haptic executable device
device. A computer-implemented method for altering a haptic effect, the method being performed by at least one processor executing computer instructions, the method comprising:
Determining, by the at least one processor, a haptic output command including an instruction to activate a haptic output device to perform an original haptic effect;
Determining, by the at least one processor, a fatigue level indicative of an amount of user fatigue based on at least one of a haptic output command transmitted to the haptic output device and an input received from the control device,
Comparing the fatigue level to a predetermined fatigue threshold by the at least one processor;
An alternative haptic output including instructions for activating a haptic output device to perform a modified haptic effect, instead of the original haptic effect, based on a comparison of the fatigue level to the predetermined fatigue threshold by the at least one processor And determining a command. 15. The method of claim 14, wherein the level of fatigue is determined based on a haptic effect density determined by a characteristic of a previous haptic effect output by the haptic output device, the characteristic comprising at least one of a frequency, / RTI > 16. The method of claim 15,
Further comprising increasing the value of the haptic effect density according to the characteristics of the previous haptic effect. 16. The method of claim 15,
Further comprising reducing the value of the haptic effect density according to the elapsed time since the occurrence of the previous haptic effect. 15. The method of claim 14,
Further comprising increasing the fatigue level according to the measured activity of the user including at least one of the amount of time using the control device and activation of control associated with the control device. 19. The method of claim 18,
Further comprising reducing the fatigue level according to the amount of time that has elapsed since the most recent activation of control associated with the control device. 15. The method of claim 14, wherein determining an alternate haptic output command comprises determining an alternate haptic output command comprising an instruction to generate a modified haptic effect, And a decrease in at least one of the frequency. 15. The computer-implemented method of claim 14, wherein determining an alternate haptic output command comprises removing the original haptic effect. 15. The computer-implemented method of claim 14, wherein determining an alternate haptic output command comprises determining an alternate haptic output command that includes an instruction to provide a haptic assist to the user. 15. The computer-implemented method of claim 14, further comprising receiving a user input to determine a predetermined fatigue threshold. 15. The computer-implemented method of claim 14, wherein the at least one processor is included in a haptic executable device that includes a haptic output device. 25. The computer-implemented method of claim 24, wherein the haptic executable device further comprises a control device. 15. The computer system of claim 14, wherein the at least one processor is included in a computing device including a haptic output port,
The haptic output port
Connected to a haptic executable device comprising a haptic output device,
Configured to output a haptic output command to the connected haptic executable device
Computer-implemented method.

Images