Classifications

• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/20—Input arrangements for video game devices
  • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
  • A63F13/214—Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/20—Input arrangements for video game devices
  • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
  • A63F13/218—Input arrangements for video game devices characterised by their sensors, purposes or types using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/20—Input arrangements for video game devices
  • A63F13/22—Setup operations, e.g. calibration, key configuration or button assignment
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/20—Input arrangements for video game devices
  • A63F13/24—Constructional details thereof, e.g. game controllers with detachable joystick handles
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
  • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
  • A63F13/44—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment involving timing of operations, e.g. performing an action within a time slot
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
  • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
  • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
  • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
  • A63F2300/1043—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being characterized by constructional details

Definitions

• Handheld controllers are used in an array of architectures for providing input, for example, to a computing device.
• handheld controllers are utilized in the gaming industry to allow players to interact with a personal computing device executing a gaming application, a game console, a game server, and/or the like.
• handheld controllers may find use in virtual reality (VR) environments and may mimic natural interactions such as grasping, throwing, squeezing, etc. While current handheld controllers provide a range of functionality, further technical improvements may enhance user experience.
• VR virtual reality
• FIG. 1 depicts a controller with a hand retainer in an open position according to an example embodiment of the present disclosure.
• FIG. 2 depicts the controller of FIG. 1 in an open, palm-up, hand of a user according to an example embodiment of the present disclosure.
• FIG. 3 depicts the controller of FIG. 1 in a closed hand of the user according to an example embodiment of the present disclosure.
• FIG. 4 depicts the controller of FIG. 1 in closed, palm-down, hand of the user according to an example embodiment of the present disclosure.
• FIG. 5 depicts a pair of controllers, with hand retainers in an open position, according to an example embodiment of the present disclosure.
• FIG. 6 depicts a proximity sensor of the controller of FIG. 1, according to an example embodiment of the present disclosure.
• FIG. 7 depicts an example process for configuring the proximity sensor of the controller of FIG. 1 according to an example embodiment of the present disclosure.
• FIG. 8 depicts an example process for configuring the proximity sensor of the controller of FIG. 1 according to an example embodiment of the present disclosure.
• FIG. 9 depicts an example process for configuring the proximity sensor of the controller of FIG. 1 according to an example embodiment of the present disclosure.
• FIG. 10 depicts an example process for configuring the proximity sensor of the controller of FIG. 1 according to an example embodiment of the present disclosure.
• FIG. 11 depicts an example chart illustrating configuring the proximity sensor of the controller of FIG. 1 according to an example embodiment of the present disclosure.
• FIG. 12 depicts example components of the controller of FIG. 1 according to an example embodiment of the present disclosure.
• handheld controllers having touch-sensitive controls and/or methods for continuously calibrating the touch-sensitive controls.
• handheld controllers described herein may control a device (e.g., a television, audio system, personal computing device, game console, etc.) to engage in video game play and/or the like.
• the handheld controllers may include one or more controls such as joysticks, trackpads, trackballs, buttons, or other controls operable by a user of the handheld controller.
• the handheld controllers may include sensors configured to detect a presence, proximity, location, and/or gesture of the user.
• the handheld controllers may include a proximity sensor configured to detect a proximity of one or more objects, such as a finger of the user, or portions of the finger, relative to the handheld controller.
• the handheld controller may include an infrared sensor, a sensor that utilizes acoustic soundwaves, and/or any other type of sensor configured to detect touch input at the handheld controller and/or a proximity of one or more objects relative to the handheld controller.
• the proximity sensor may include an array of proximity sensors or capacitive pads arranged in rows, columns, a grid, and/or group(s).
• the capacitive pads may generate proximity data (e.g., capacitance values) that correspond to a proximity of the fmger(s) relative to the handheld controller.
• the handheld controller may continuously monitor the capacitance values detected by individual capacitive pads to associate the capacitance values with a touch or a relative hover of the finger above the handheld controller.
• the proximity data may indicate that one or more fingers contact (e.g., physically touch) the handheld controller and/or may indicate that one or more fingers hover above the handheld controller.
• the proximity data may also be used to determine a distance the finger hovers above, or relative to, the handheld controller.
• the proximity data may indicate how far or close the fingers are disposed above (or from) the handheld controller.
• the relative position of the finger i.e., between being fully extended and touching the handheld controller
• the capacitive pads may be associated with particular fingers of the user (e.g., middle, ring, pinky, etc.) or certain portions of the finger (e.g., fingertip).
• the handheld controller may map the capacitance values to certain fingers of the user (or portion of the finger).
• the handheld controller may include one or more processors that communicatively couple to the proximity sensor to receive and/or analyze the proximity data.
• the one or more processors may receive, from individual capacitive pads, the proximity data (e.g., a capacitance value).
• the one or more processors, or other logic e.g., software, hardware, firmware, etc.
• the handheld controller may utilize the proximity data to determine the presence of the finger and/or a location (or“position”) of the fmger(s).
• the handheld controller may receive the proximity data from individual capacitive pads and may associate, or group, the proximity data with particular fingers of the user. Associating the proximity data with certain fingers may allow a gaming console, remote system, head-mounted display, virtual reality environment, or other computing devices and/or handheld controller(s) to generate image data depicting a hand gesture of the user.
• the proximity data may indicate that the user grips the handheld controller with their middle finger and ring finger, but not with his or her pinky finger.
• the proximity data may also indicate a relative proximity of the pinky finger to the handheld controller (i.e., how far the pinky finger is disposed from the handheld controller).
• the handheld controller (or another communicatively coupled device) may generate an indication (or other information) representing the finger positions.
• a gaming console may utilize the indication generate image data corresponding to a hand gesture showing the user holding an object with their middle finger and ring finger, but not with the pinky finger.
• the handheld controller may continuously calibrate the proximity sensor to accommodate for varying grips (e.g., different hand size, different finger size, etc.) and/or varying environmental conditions (e.g., humidity, temperature, etc.). For instance, the handheld controller may continuously monitor the proximity data to associate capacitance values with finger positions of the user. As an example, the detected proximity data (e.g., the capacitance values) may vary depending on the grip of the user, such as how the user holds the handheld controller, what game the user plays, and/or physical features of the hand of the user (e.g., length of finger, width of finger, etc.). In some instances, continuously calibrating the proximity sensor may include determining a range of capacitance values detected by individual capacitive pads.
• the range of capacitance values may include a capacitance value associated with the finger touching the handheld controller and a capacitance value associated with the finger being fully extended from the handheld controller. In some instances, these capacitance values may correspond to a maximum capacitance value and a minimum capacitance value, respectively. Knowing the range of capacitance values for individual capacitive pads allows for subsequent capacitance values to be normalized, which in turn, may be used to determine finger position. Individual capacitive pads of the proximity sensor may include a maximum capacitance value and a minimum capacitance value, such that subsequent capacitance values received from individual capacitive pads may be normalized.
• the handheld controller may continuously determine a scale factor and/or a bias associated with the capacitive pad.
• the scale factor and/or bias may be used to normalize (or calibrate) the capacitance values received at individual capacitive pads and/or normalize capacitance values received across capacitive pads of the proximity sensor.
• the bias, scale factor, and/or range of capacitance values for individual capacitive pads may vary as the capacitive pads may respond differently and/or generate different capacitance values.
• the capacitance values detected at the capacitive pads may depend on a size of the capacitive pad and/or manufacturing conditions. Calculating the scale factor and/or knowing the bias for individual capacitive pads allows for the capacitance values to be normalized with capacitance values received from other capacitive pads of the proximity sensor. The normalization and/or calibration of the capacitive pads in relation to one another may accurately determine hand gestures performed by the user.
• the capacitance values detected by the capacitive pads of the proximity sensor may vary throughout a gameplay experience, such as if the hand of the user becomes moist (e.g., sweaty).
• the proximity data measured or detected by the capacitive pads may change, for instance, as the hands of the user become damp, as the humidity increases within an environment, and/or as the user adjusts his or her grip. Consequently, the range of the capacitance values detected by a given capacitive pad may change.
• the bias and scale factor may correspondingly change.
• the handheld controller may decay a limit associated with the maximum capacitance value and/or a limit associated with the minimum capacitance value.
• the handheld controller may continuously decay the maximum capacitance value and/or the minimum capacitance value towards the detected capacitance values. In doing so, the range of capacitance values may continuously vary such that the range of capacitance values correspond to the capacitance values detected at the proximity sensor. In some instances, if the handheld controller does not detect a capacitance value within a predetermined range of the maximum capacitance value for a threshold amount of time, number of frames, or number of counts, the maximum capacitance value may decay.
• the predetermined range of the maximum capacitance value may be a quarter of the range of the capacitance values detected by the handheld controller (i.e., between the maximum capacitance value and the minimum capacitance value), and if the handheld controller does not detect a capacitance value within the predetermined range of the maximum capacitance value, the maximum capacitance value may decay towards the detected capacitance values. In other words, if the capacitance value(s) is/are not within the predetermined range of the maximum capacitance value, the maximum capacitance value may decay towards the minimum capacitance value.
• the maximum capacitance value may decay.
• the handheld controller may observe a decrease in detected capacitance values.
• the maximum capacitance value may decay towards the minimum capacitance value to update or determine a new maximum capacitance value.
• the handheld controller may accurately determine finger positions. Decaying the maximum capacitance value may also be used to determine an updated range of capacitance values, an updated bias, and/or an updated scale factor for individual capacitive pads of the proximity sensor.
• the minimum capacitance value may similarly decay. That is, based at least in part on receiving capacitance values, the handheld controller may continuously decay the minimum capacitance value towards the detected capacitance values such that the range of capacitance values varies to correspond to the capacitance values detected at the proximity sensor. For instance, if the handheld controller does not detect a capacitance value within a predetermined range of the minimum capacitance value for a threshold amount of time, number of frames, or number of counts, the minimum capacitance value may decay.
• the predetermined range of the minimum capacitance value may be a quarter of the range of the capacitance values detected by the handheld controller, and if the handheld controller does not detect a capacitance value within the predetermined range of the minimum capacitance value, the minimum capacitance value may decay. In other instances, if an average of the capacitance values detected over a previous amount of time, number of frames, or number of counts, is not within the predetermined range of the minimum capacitance value, the minimum capacitance value may decay. As the minimum capacitance value is associated with an object (e.g., finger) being fully extended from the handheld controller, by decaying the minimum capacitance value, the handheld controller (or another communicatively coupled device) may accurately determine finger positions.
• an object e.g., finger
• Decaying the minimum capacitance value may also be used to determine an updated range of capacitance values, an updated bias, and/or an updated scale factor for individual capacitive pads of the proximity sensor.
• an amount the maximum capacitance value and/or an amount the minimum capacitance value decays may be based on one or more variable decay rates, a range of capacitance values detected over a predetermined past number of frames, and/or a range of capacitance values between the maximum capacitance value and the minimum capacitance value.
• an amount of the decay may be based on a percentage of the difference between the detected capacitance value and the range of capacitance values detected over the predetermined past number of frames or a percentage of the difference between the detected capacitance value and the range of capacitance values between the maximum capacitance value and the minimum capacitance value. In other instances, the amount of the decay may be based on a percentage of the difference between the detected capacitance value and the maximum capacitance value or the minimum capacitance value, respectively. In other instances, the decay, or the variable decay rate, may be optimized to accurately associate the proximity of the finger with the maximum capacitance value and the minimum capacitance value, respectively. In some instances, decaying the maximum capacitance value and the minimum capacitance value may be limited such that the minimum capacitance value and the maximum capacitance value are separated by a threshold amount.
• the rate at which the maximum capacitance value or the minimum capacitance value decays may depend on a volatility and/or a variability of the measured capacitance values. For instance, if a large variability exists between the capacitance values detected over the predetermined past number of frames, the maximum capacitance value or the minimum capacitance value may decay at a faster rate, as compared to if a small variability exists between the capacitance values detected over the predetermined past number of frames. For instance, if a first user has a maximum capacitance value and a minimum capacitance value associated with his or her grip, and a second user operates the handheld controller, a large variability may exist between the maximum capacitance value and/or the minimum capacitance value of the first user and the second user.
• the rate of decay may take this into account and decay the maximum capacitance value and/or the minimum capacitance value at a faster rate.
• the rate at which the maximum capacitance value or minimum capacitance value decays may depend on a number of capacitance values received by the handheld controller. For instance, if the user rapidly taps their fingers on the handheld controller, the handheld controller may decay or converge the maximum capacitance value (or limit) and/or the minimum capacitance value (or limit) to the detected capacitance values at a faster rate and if the user does not rapidly tap the handheld controller.
• decaying and continuously calibrating the capacitive pads of the proximity sensor may adjust to the grip of the user and accommodate for environmental factors to accurately determine finger position.
• failing to decay the maximum capacitance value, or a limit associated with the maximum capacitance value may result in not accurately associating a received capacitance value with the finger touching the handheld controller. That is, the handheld controller may not compare the capacitance value to an updated maximum capacitance value associated with the finger touching the handheld controller.
• failing to decay the minimum capacitance value, or a limit associated with the minimum capacitance value may result in not accurately associating a received capacitance value with the finger being fully extended from the handheld controller.
• the handheld controller may not compare the capacitance value to an updated minimum capacitance value associated with the finger being fully extended from the handheld controller.
• the handheld controller may simultaneously determine the range of capacitance values, the bias, the scale factor, the decay, and/or the decay rate for each capacitive pad of the proximity sensor for each frame displayed in a VR environment.
• the handheld controller may detect capacitance values that are greater than or less than the maximum capacitance value and/or the minimum capacitance value, respectively. In other words, through continuously updating the maximum capacitance value and/or the minimum capacitance value, the handheld controller may receive capacitance values that are greater than the determined maximum capacitance value and/or may receive capacitance values lower than the determined minimum capacitance value.
• the handheld controller may detect increased capacitance values and the capacitance values may“overshoot” or exceed the maximum capacitance value and/or the minimum capacitance value.
• the maximum capacitance value and/or the minimum capacitance value may extend and make up for this difference based at least in part on an average of the capacitance values detected over a past number of frames.
• the rate at which the maximum capacitance value and/or the minimum capacitance value updates may be based on a growth constant as well as the average capacitance values detected over the past number of frames. In doing so, the handheld controller (or another communicatively coupled device) may determine an updated maximum capacitance value and/or the minimum capacitance, an updated bias, an updated scale factor, and so forth.
• the handheld controller may also sense, detect, or measure, via a pressure sensor, an amount of force associated with touch input at the handheld controller. For instance, as a finger of a user presses against the handheld controller, a portion of the controller, such as a cover disposed above the pressure sensor, may deflect to contact the pressure sensor.
• the pressure sensor may couple to the one or more processors such that touch input may result in force data being provided to the one or more processors.
• the pressure sensor may comprise a force-sensing resistor (FSR) sensor, a piezoelectric sensor, a load cell, a strain gauge, a capacitive-type pressure sensor that measures capacitive force measurements, or any other type of pressure sensor.
• FSR force-sensing resistor
• the proximity data and the force data may be interpreted together and associated with a predefined command or hand gesture (e.g., squeezing).
• the handheld controller may dynamically calibrate the proximity sensor to adapt to changing grips of a user(s) and/or environmental conditions, for instance.
• the handheld controller may decay capacitance values associated with finger touching the handheld controller (e.g., the maximum capacitance value) and the finger being fully extended from the handheld controller (e.g., the minimum capacitance value).
• the handheld controller may determine the relative location and/or proximity of the finger to the handheld controller.
• the handheld controller (or the a communicatively coupled device) may continuously calibrate the proximity sensor.
• FIG. 1 illustrates a front view of an example controller 100 that may include one or more touch-sensitive controls.
• the controller 100 may be utilized by an electronic system such as a VR video gaming system, robot, weapon, or medical device.
• the controller 100 may include a controller body 110 having a handle 112 and a hand retainer 120.
• the controller body 110 may include a head disposed between the handle 112 and a distal end 111 of the controller 100, which may include one or more thumb- operated controls 114, 115, 116.
• a thumb-operated control may include a tilting button, and/or any other button, knob, wheel, joystick, or trackball conveniently manipulated by a thumb of a user during normal operation when the controller 100 is held in the hand of the user.
• the handle 112 may include a substantially cylindrical tubular housing. However, in this context, a substantially cylindrical shape need not have constant a diameter or a circular cross-section.
• the handle 112 may include a proximity sensor having a plurality of proximity sensors or capacitive pads spatially distributed at least partially or completely around an outer surface of the handle 112.
• the capacitive pads may be spatially distributed beneath the outer surface of the handle 112 and/or may be embedded under the outer surface of the handle 112.
• the capacitive pads may be responsive to a user touching, gripping, or grasping the handle 112 to identify the presence, position, proximity, and/or gesture of one or more fingers of the user.
• the capacitive pads may be responsive to one or more fingers hovering above the handle 112 and may generate proximity data (e.g., capacitance values) indicating the proximity of the fingers above the handle 112.
• the proximity data may indicate a presence, location, and/or proximity of the fmger(s) relative to the controller 100.
• the proximity data may be utilized by the controller 100 and/or other communicatively coupled device(s) to generate hand gestures.
• the outer surface of the handle 112 may comprise an electrically insulative material.
• the hand retainer 120 may couple to the controller 100 to bias a palm of the user against the outside surface of the handle 112. In FIG. 1, the hand retainer 120 is shown in the open position.
• the hand retainer 120 may optionally bias in the open position by a curved resilient member 122 to facilitate the insertion of a hand between the hand retainer 120 and the controller body 110.
• the curved resilient member 122 may include a flexible metal strip or plastic material, such as nylon, that elastically bends.
• a fabric material 124 may partially or completely cover the curved resilient member 122 to cushion or increase a comfort of the user. Alternatively, the cushion or fabric material 124 may adhere to only the side of the curved resilient member 122 facing the hand of the user.
• the hand retainer 120 may adjust in length, for example, by including a draw cord 126 that is cinched by a spring-biased chock 128. The draw cord 126 may optionally have an excess length for use as a lanyard. In some examples, the cushion or fabric material 124 may attach to the draw cord 126.
• the curved resilient member 122 may be preloaded by the tension of the cinched draw cord 126 and in such embodiments, the tension that the curved resilient member 122 imparts to the hand retainer 120 (to bias it in the open position) may cause the hand retainer 120 to automatically open when the draw cord 126 is un-cinched.
• alternative conventional ways to adjust the length of a hand retainer 120 such as a cleat, an elastic band (that temporarily stretches when the hand is inserted to apply elastic tension to press against the back of the hand), a hook & loop strap attachment that allows length adjustment, etc. may be used.
• the hand retainer 120 may be disposed between the handle 112 and a tracking member 130, and may contact the back of the hand of the user.
• the tracking member 130 may affix to the controller body 110 and may optionally include two noses 132, 134, where each nose may protrude from a corresponding one of two opposing distal ends of the tracking member 130.
• the tracking member 130 may include an arc having a substantially arcuate shape.
• the tracking member 130 may include tracking transducers disposed therein, for example, with at least one tracking transducer disposed in each protruding nose 132, 134.
• the controller body 110 may include additional tracking transducers, such as a tracking transducer disposed adjacent the distal end 111.
• the controller 100 may include a rechargeable battery disposed within the controller body 110.
• the hand retainer 120 may include an electrically-conductive charging wire electrically coupled to the rechargeable battery.
• the controller 100 may also include a radio frequency (RF) transmitter for communication with the rest of an electronic system (e.g., gaming console).
• the rechargeable battery may power the RF transmitter and the RF transmitted may respond to the thumb-operated controls 114, 115, 116, the proximity sensor (e.g., the capacitive pads) in the handle 112, and/or tracking sensors in the tracking member 130.
• RF radio frequency
• the controller body 110 may comprise a single piece of injection molded plastic or any other material rigid enough to transfer a force from a finger of the user to the proximity sensor and thin enough to allow for capacitive coupling between a finger of the user and the proximity sensor.
• the controller body 110 and the tracking member 130 may be fabricated separately and then later assembled together.
• FIG. 2 illustrates a front view of the controller 100, showing the controller 100 during operation with the left hand of the user inserted therein but not grasping the controller body 110.
• the hand retainer 120 is cinched over the hand of the user to physically bias the palm of the user against the outside surface of the handle 112.
• the hand retainer 120 when closed, may retain the controller 100 within the hand of the user even when the hand is not grasping the controller body 110.
• the hand retainer 120 may prevent the controller 100 from falling out of hand of the user.
• the hand retainer 120 may allow the user to“let go” of the controller 100 without the controller 100 actually separating from the hand, being thrown, and/or dropped. For example, if the release and restoration of the user grasping the handle 112 of the controller body 110 is sensed, the release or grasping may be incorporated into the game to display throwing or grasping objects (e.g., in VR environment). The hand retainer 120 may allow such a function to be accomplished repeatedly and safely. Further, the hand retainer 120 may also prevent fingers of the user from excessively translating relative to the proximity sensor to more reliably sense finger motion, proximity, and/or placement on the handle 112.
• FIGS. 3 and 4 illustrate the controller 100 during operation when the hand retainer 120 is cinched while the hand of the user grasps the controller body 110 to retain the controller 100 in the hand of the user.
• the thumb of the user may operate one or more of the thumb-operated controls 114, 115, 116.
• FIG. 5 illustrates that in certain embodiments, the controller 100 may be a left controller in a pair of controllers that includes a similar right controller 500.
• the controllers 100 and 500 may (together) track the motion and grip of both of the hands of the user, simultaneously, for example, to enhance a VR experience.
• FIG. 6 illustrates a proximity sensor 600 having a plurality of capacitive pads 602 configured to detect a proximity of one or more objects (e.g., finger) relative to the controller 100.
• the proximity sensor 600 may additionally or alternatively include different types of sensors other than the capacitive pads 602 to detect the proximity of the one or more objects relative to the controller 100, such as an infrared or acoustic sensor.
• the capacitive pads 602 may mount, attach, or otherwise couple to a flexible printed circuit assembly (FPCA) 604.
• the FPCA 604 may include a connector 606 for connecting to a printed circuit board (PCB) of the controller 100 that includes one or more processors.
• PCB printed circuit board
• the capacitive pads 602 may communicatively connect to the connector 606 via traces 608 disposed on the FPCA 604 to provide proximity data (e.g., capacitance values) to the one or more processors.
• the proximity data may indicate the proximity of the finger relative to the controller 100, such as the handle 112. Accordingly, the proximity sensor 600 may measure or receive the capacitance of individual capacitive pads 602, where the capacitance value may be associated with a proximity of the finger to the controller 100.
• the proximity sensor 600 may couple to an interior surface within the controller body 110, such as a structure mounted within the handle 112 of the controller body 110, or a structure mounted underneath the handle 112 of the controller body 110. In doing so, the proximity sensor 600 may be disposed beneath the outer surface of the handle 112 to detect a proximity of the fingers relative to the handle 112. When coupled to the controller 100, the proximity sensor 600 may angularly span around a circumference or a portion of the handle 112. For instance, the FPCA 604 may couple (e.g., adhesive) to the inner surface of the controller body 110. In some embodiments, the proximity sensor 600 may extend at least 100 degrees but not more than 170 degrees around the circumference of the handle 112. Additionally, or alternatively, the proximity sensor 600 may couple to the outer surface of the controller 110, such as an outer surface of the handle 112.
• the capacitive pads 602 may be spaced apart from one another to detect a proximity of different fingers, or different portions of the fmger(s) (e.g., fingertip).
• the capacitive pads 602 and may be arranged into rows, columns, a grid, sets, subsets, or groups 610.
• the proximity sensor 600 may include six groups 610 of capacitive pads 602, where the groups 610 may extend horizontally across the FPCA 604. However, in some embodiments, the proximity sensor 600 may include more than or less than six groups 610.
• the capacitive pads 602 are not necessarily of equal size and may not necessarily have substantially equal spacing therebetween.
• the capacitive pads 602 may be similarly shaped and/or may comprise a grid with substantially equally spacing therebetween.
• Individual groups 610 of the capacitive pads 602 may correspond to a particular finger of the user (e.g., index finger, middle finger, ring finger, pinky finger). Additionally, or alternatively, multiple groups 610 of the capacitive pads 602 or capacitive pads 602 from multiple groups 610 may correspond to a single finger of the user. For instance, two or more groups 610 may correspond to a finger of the user (e.g., middle finger).
• the controller 100 may utilize proximity data to generate hand gestures of the user. For instance, as the user grips the controller 100 with certain fingers and/or hovers certain fingers above the controller 100, a voltage is applied to the capacitive pads 602 that results in an electrostatic field. When a conductor, such as a finger, touches or nears the capacitive pads 602, a change in capacitance may occur.
• the capacitance may be sensed by connecting an RC oscillator circuit to the proximity sensor 600 (via the connector 606) and noting that a time constant (and therefore the period and frequency of oscillation) will vary with the capacitance.
• a time constant and therefore the period and frequency of oscillation
• the controller 100 may detect a change in capacitance.
• the capacitance values of the capacitive pads 602 are used to determine the location of the conductor as well as the proximity of the conductor relative to the capacitive pad 602. As fmger(s) act as a conductor, those capacitive pads 602 underlying the handle 112 may measure a capacitance value.
• the grip of the user may change or other factors may affect values of the proximity data. For instance, throughout a gameplay experience, the hand of the user may sweat and become damp. Additionally, or alternatively, a humidity of an environment in which a game is being played may change. These, and other factors, may affect an amount of capacitance detected by the capacitive pad(s) 602.
• the controller 100 may continuously calibrate the proximity sensor 600 based on the detected capacitance values. For example, at a first instance, a capacitive pad 602 may measure a first capacitance value when a finger is fully extended from the controller 100.
• the capacitive pad 602 may measure a second capacitance value when the finger of the user is fully extended from the controller 100.
• the controller 100 may calibrate the capacitive pad 602.
• calibrating the capacitive pad(s) 602 may include determining an updated range of capacitance values detected at the individual capacitive pad(s) 602 or decaying capacitance values associated with a touch at the controller 100 and/or the finger being fully extended from the controller 100. Therein, knowing the updated range, the detected capacitance values may be normalized.
• the capacitive pads 602 may remap to correspond to different fingers. For instance, depending on the grip of the user or the hand size of the user, a particular row or rows of the proximity sensor 600 may correspond to a particular finger of the user (e.g., middle finger), while in other instances, may correspond to a different finger of the user (e.g., ring finger).
• the capacitive pads 602 may remap and correspond to different fingers of the user(s). In other words, the capacitive pads 602 may be designated in some instances to correspond to different fingers of the user depending on the grip of a particular user with the handle 112.
• the controller 100 may include different controller configurations (i.e., a first controller configuration, a second controller configuration, and/or a third controller configuration) to associate different capacitive pads 602 with different fingers.
• This dynamic adapting of capacitive pads 602 to certain fingers may allow for accurate gestures to be generated in a VR environment.
• FIG. 6 illustrates the proximity sensor 600 including a certain number of capacitive pads 602 and/or a certain number of groups 610, in some instances, the proximity sensor 600 may include more or less capacitive pads 602 than illustrated in FIG. 6.
• the one or more processors of the controller 100 may include algorithms and/or machine-learning techniques embodying anatomically-possible motions of fingers, to better use the proximity data to detect the opening the hand of the user, finger pointing, or other motions of fingers relative to the controller 100 or relative to each other.
• the movement of the controller 100 and/or fingers of the user may help control a VR gaming system, defense system, medical system, industrial robot or machine, or another device.
• the proximity data may be utilized to render the release of an object based on the sensed release of the fingers of the user from the outer surface of the handle 112.
• a communicatively coupled computing device e.g., a host computing device, a game console, etc.
• the controller 100 is interacting with may detect the gesture(s) using the proximity data.
• FIGS. 7-10 illustrate various processes as a collection of blocks in a logical flow graph, which represent a sequence of operations that can be implemented in hardware, software, or a combination thereof.
• the blocks represent computer executable instructions that, when executed by one or more processors, perform the recited operations.
• computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.
• the order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and/or in parallel to implement the processes.
• FIG. 7 illustrates a process 700 for calibrating a proximity sensor 600.
• the process 700 may be performed by a controller 100.
• logic of the controller 100 may receive proximity data from the proximity sensor 600.
• an object e.g., finger, thumb, etc.
• the proximity data may indicate a capacitance detected or measured by the capacitive pads 602 of the proximity sensor 600.
• the capacitance value may be larger than compared to a finger hovering above the controller 100. In this sense, the detected capacitance value may indicate, or represent, a proximity of the finger relative to the controller 100.
• logic of the controller 100 may convert the capacitance value into a digitized value.
• the proximity data received at 702 may represent raw data that is not calibrated and/or normalized with proximity data received across capacitive pads 602 of the proximity sensor 600. That is, the proximity data received at 702 may represent raw data in the sense that for a particular capacitive pad 602, the capacitive pad 602 may detect a capacitance value over a range of capacitance values.
• logic of the controller 100 may normalize the proximity data. For instance, individual capacitive pads 602 may generate unique capacitance values depending on manufacturing conditions of the capacitive pad 602, a size of the capacitive pad 602, and so forth. Normalizing the proximity data may assist in standardizing the proximity data received at the capacitive pads 602. In some instances, normalizing the proximity data may include utilizing trained models to determine maximum and minimum capacitance values measured at the capacitive pads 602. For instance, a range of capacitance values detected at a capacitive pad 602 may be arbitrary when compared to a range of capacitance values detected at a different capacitive pad 602.
• the trained models may include a representative sample of capacitance values, or a range of capacitance values, historically detected across the capacitive pads 602 of the proximity sensor 600.
• capacitance values received from like capacitive pads 602 of different controllers may be used to normalize the capacitance values.
• calibrating the proximity sensor 600 may include calculating a bias, a range of capacitance values, and/or a scale factor for capacitance values detected at the capacitive pads 602.
• calibrating the proximity sensor 600 may involve, for individual capacitive pads 602, determining the range of capacitance values detected, a bias, and/or a scale factor. Calibrating the proximity sensor may therefore allow for subsequent capacitance values received to be normalized.
• the process 700 may involve more detailed operations to calibrate the proximity sensor 600.
• calibrating the proximity sensor 600 may include sub-blocks 708 and 710.
• calibrating the proximity sensor 600 may include a discrete gesture recognition which may represent (or correspond to) a discrete gesture performed by the user. For instance, if all, a portion, or a threshold percentage of the capacitive pads 602 sense a sudden drop in capacitance values, the controller 100 may associate this drop with the user releasing their hand from the controller 100 or releasing a particular finger from the controller 100. That is, as certain capacitive pads 602 may be associated with certain fingers of the user, the controller 100 may detect a decrease in capacitance and associate this decrease in capacitance values with the user releasing their hand from the controller 100.
• the capacitance values received as the user suddenly releases their finger from the controller 100 may correspond to a minimum capacitance value within a range of capacitance values detected for a particular capacitive pad 602 (e.g., the capacitance value represents when the finger is fully extended from the controller 100). This minimum capacitance value may therefore, at future instances, be used to determine when the finger is fully extended from the controller 100.
• the capacitance value received prior to this sudden decrease in capacitance values may correspond to a maximum capacitance value of the range of capacitance values detected for the particular capacitive pad 602 (e.g., the capacitance value represents when the finger touches the controller 100). This maximum capacitance value may therefore, at future instances, be used to determine when the finger is touching the controller 100.
• logic of the controller 100 may determine a range of capacitance values detected at the capacitive pad 602. This process may occur for each capacitive pad 602 of the proximity sensor 600. Accordingly, as the controller 100 receives subsequent proximity data, the controller 100 may determine a relative position of the finger of the user through comparing the detected capacitance value with the range of capacitance values for a particular capacitive pad 602.
• calibrating the proximity sensor 600 may also include a continuous minimum capacitance value and/or maximum capacitance value detection. For instance, as the controller 100 may continuously receive proximity data from the proximity sensor 600, the controller 100 may monitor the proximity data (e.g., the capacitance values) to re-calibrate (e.g., decay) the minimum capacitance value and/or the maximum capacitance value. As will be discussed herein, through iteratively receiving proximity data from the proximity sensor 600 (e.g., as a user interacts with the controller 100), the proximity data may indicate capacitance values measured by the capacitive pads 602.
• the proximity data may indicate capacitance values measured by the capacitive pads 602.
• the capacitance values may be utilized to update and/or modify the range of capacitance values detected by individual capacitive pads 602.
• the capacitance values detected by the capacitive pads 602 may change based on skin properties of the user and/or environmental conditions in which the controller 100 resides. Accordingly, at 710, for respective capacitive pads 602 of the proximity sensor 600, the controller 100 may analyze the proximity data to potentially update the minimum capacitance value, the maximum capacitance value, and/or the range of capacitance values for the capacitive pads 602.
• the controller 100 upon receiving capacitance values from the capacitive pads 602, may compare the capacitance values with the maximum capacitance value and/or the minimum capacitance value to determine whether the capacitance value is greater than or less than the maximum capacitance value and/or the minimum capacitance value, respectively. If so, the controller 100 may update the maximum capacitance value and/or the minimum capacitance value, respectively. Moreover, the controller 100 may utilize the capacitance values to decay the maximum capacitance value and/or the minimum capacitance value towards the received capacitance values.
• the process 700 may loop to 702 where the process 700 may receive additional proximity data from the proximity sensor 600. Through iteratively receiving proximity data, the process 700 may continuously determine and/or update the range of capacitance values detected by the capacitive pads 602. Additional details of continuously calibrating the proximity sensor 600 utilizing the proximity data is discussed herein with regard to FIG. 8.
• logic of the controller 100 may generate calibrated proximity data based at least in part on calibrating the proximity sensor 600 and/or the individual capacitive pads 602. For instance, utilizing range of capacitance values detected at the capacitive pads 602, the bias, and/or the scale factor, the controller 100 may calculate a calibrated (or normalized) capacitance value.
• the calibrated proximity data may normalize the proximity data received at the particular capacitive pad 602 and/or across capacitive pads 602 of the proximity sensor 600.
• logic of the controller 100 may sum the proximity data for a group of capacitive pads 602.
• the capacitive pads 602 may be arranged into groups 610 that are associated with respective fingers of the user.
• the groups 610 may include more than one capacitive pad 602
• the capacitance values for the capacitive pads 602 of a group of capacitive pads 602 may be summed to indicate a finger position of the user, where the summed capacitance value may indicate a proximity of the finger relative to the controller 100.
• the summed capacitance value for a particular group 610 may include a finger value on a scale of [0,1], where the finger value may indicate a proximity of the finger relative to the controller 100.
• the process 700 may determine the finger positions of the user relative to the controller 100.
• the capacitance values may be summed using a set of weights applied to the individual capacitive pads 602 within a group of capacitive pads 602. For instance, if four capacitive pads 602 are associated with the middle finger of a user, an equal weight may be assigned the four capacitive pads 602. The capacitance values received from these capacitive pads 602 may have a weight of one-quarter that is used when determining a finger position. However, in some instances, certain capacitance values from capacitive pads 602 may be disregarded in the weighted sum if the capacitance value for a capacitive pad 602 is unreliable (e.g., contains a large amount of noise). The weight of the capacitive pad 602 may be ignored such that the remaining capacitive pads have a weight of one-third. Therein, the capacitance values are summed and divided by the sum of the weights of the capacitive pads 602 being used.
• logic of the controller 100 may transmit the proximity data and/or the finger value. For instance, after determining the calibrated proximity data for a particular finger or group of capacitive pads 602, the controller 100 may transmit the proximity data and/or an indication of the finger value (position) to one or more computing devices. In some instances, the one or more computing devices may utilize the proximity data and/or the finger values to generate image data corresponding to a gesture of the hand of the user on the controller 100.
• the process 700 may be performed for each capacitive pad 602 and each time (e.g., count, frame) the proximity sensor 600 generates proximity data.
• the proximity sensor 600 may generate the proximity data for each frame displayed in the VR environment.
• the proximity data may represent the respective capacitance values detected at the capacitive pads 602 and the process 700 may be performed for the capacitance values measured at individual capacitive pads 602.
• the process 700 may be performed for a first capacitance value detected at a first capacitive pad 602 and a second capacitance value detected at a second capacitive pad 602.
• the process 700 for the first capacitive pad 602 and the second capacitive pad 602 may be performed in parallel, and for any number of capacitive pads 602. Additionally, while some or all of the process 700 is described being performed by the controller 100, in some instances, the one or more communicatively coupled devices may perform all of or a portion of the blocks of the process 700.
• the computing device(s) may include increased processing power for calibrating the proximity sensor 600. In such instances, the computing device(s) may transmit an instruction to calibrate the proximity sensor 600.
• FIG. 8 illustrates an example process 800 for analyzing proximity data received from a proximity sensor 600 to calibrate the proximity sensor 600.
• the process 800 may be performed by a controller 100, however, one or more communicatively coupled devices may perform all or a portion of the process 800.
• logic of the controller 100 may receive a capacitance value (e.g., the proximity data) from the proximity sensor 600.
• the controller 100 may receive, from individual capacitive pads 602 of the proximity sensor 600, and at each frame displayed in a VR environment, the capacitance value.
• a capacitive pad 602 of the proximity sensor 600 may generate the capacitance value. As indicated above, as the distance between the object and the capacitive pad 602 decreases, the measured capacitance value may increase.
• logic of the controller 100 may normalize the capacitance value using a range of capacitance values for the capacitive pad 602, a scale factor associated with the capacitive pad 602, and a bias associated with the capacitive pad 602.
• the controller 100 may decay a limit, or level, associated with a maximum capacitance value to gradually assign or adjust the maximum capacitance value. Decaying the maximum capacitance value may adjust to varying environment conditions and/or characteristics of the fmger(s) of the user. For instance, over the course of a gameplay experience, the hands of a user may become damp (e.g., sweaty), the user may adjust his or her grip on the controller 100, and/or conditions of an environment may change (e.g., humidity, temperature, etc.). In some instances, these factors may affect the capacitance values detected by the proximity sensor 600 and the range of the capacitance values may be updated and/or the capacitance value associated with a finger touching the controller 100 may be updated.
• damp e.g., sweaty
• conditions of an environment e.g., humidity, temperature, etc.
• a capacitance value associated with the finger touching the controller 100 may change.
• the maximum capacitance value may decay.
• a first capacitance value may represent or correspond to a finger touching the controller 100 and at a second instance in time, a second capacitance value may represent or correspond to the finger touching the controller 100.
• the second capacitance value may be less than the first capacitance value.
• the capacitance value associated with the finger touching the controller 100 may update.
• the controller 100 may dynamically adjust the capacitance value associated with the finger grasping the controller 100.
• decaying the maximum capacitance value may be based at least in part on receiving a threshold number of capacitance values that are not within a predetermined range of the maximum capacitance value. Additionally, or alternatively, decaying the maximum capacitance value may occur if a threshold amount of time, number of frames, or number of counts pass without receiving capacitance values within the predetermined range of the maximum capacitance value and/or the capacitance values changing (e.g., variability) by a threshold amount. In other instances, if an average of the capacitance values detected over a previous amount of time, number of frame, or number of counts, is not within the predetermined range of the maximum capacitance value, the maximum capacitance value may decay. Additionally, although decaying the maximum capacitance value has been described with respect to the capacitance value not being within the predetermined range of the maximum capacitance value, in some instances, the predetermined range may be with respect to the minimum capacitance value.
• the maximum capacitance value may continuously decay such that the range of capacitance values continuously updates.
• the amount of the decay, or an amount the maximum capacitance value decays may be based on the range of capacitance values detected over a previous number of frames (e.g., 20 frames, 50 frames, 100 frames, and so forth), a range of capacitance values between the maximum capacitance value and the minimum capacitance value, a difference between the maximum capacitance value and the average capacitance value detected over a previous number of counts or frames (e.g., 20 frames, 50 frames, 100 frames, and so forth), and/or a percentage of the difference between the maximum capacitance value and the average capacitance value.
• the decay may be determined by calculating (1) the range of capacitance values detected over the previous number of frames, (2) the range of capacitance values between the maximum capacitance value the minimum capacitance value, and (3) a decay variable.
• the decay variable for the maximum capacitance value may be a percentage of the variability of the capacitance values between the maximum capacitance value the minimum capacitance value, as detected over a previous number of frames.
• the decay variable for the maximum capacitance value may be ten percent of the range of capacitance values between the maximum capacitance value the minimum capacitance value for each frame. In other words, for every frame, the decay variable may be calculated by multiplying the range of capacitance values between the maximum capacitance value the minimum capacitance value by ten percent. This decay variable may then be used to determine the updated maximum capacitance value.
• the maximum capacitance value may decay approximately, or substantially, about thirteen percent of the range between the maximum capacitance value and the minimum capacitance value.
• the amount the maximum capacitance value decays may be limited such that the maximum capacitance value does not decay more than fifty percent of the range of capacitance values between the maximum capacitance value the minimum capacitance value. In this sense, the maximum capacitance value may be limited to decaying to more than half of the range of capacitance values between the maximum capacitance value the minimum capacitance value.
• the amount the maximum capacitance value decays may be proportional to the range of capacitance values between the maximum capacitance value the minimum capacitance value.
• the maximum capacitance value may be limited to decaying to within a certain threshold of the minimum capacitance value. For instance, to reduce an impact of noise within the capacitance values, the maximum capacitance value may not decay to within a threshold amount of the minimum capacitance value. In other words, the maximum capacitance value and the minimum capacitance value may be separated by a threshold range to reduce the impact of noise.
• a rate at which the maximum capacitance value decays may depend on a variability of the measured capacitance values. For instance, if a large variability exists between the capacitance values detected over a past number of frames, the maximum capacitance value may decay at a faster rate, as compared to if a small variability exists between the capacitance values detected over the past number of frames. Additionally, or alternatively, the rate at which the maximum capacitance value decays may depend on a number of capacitance values received. For instance, if the user is rapidly tapping his or her fingers on the controller 100 and the controller 100 receives a large number of capacitance values, the controller 100 may decay the maximum capacitance value at a faster rate.
• Decaying the maximum capacitance value may be used to determine an updated maximum capacitance value for the capacitive pads 602. Additionally, the updated maximum capacitance value may also be used to determine an updated range of capacitance values, an updated bias, and/or an updated scale factor for individual capacitive pads 602 of the proximity sensor 600.
• logic of the controller 100 may decay a limit, or level, associated with the minimum capacitance value to gradually assign or adjust the minimum capacitance value.
• the minimum capacitance value may continuously decay such that the range of capacitance values continuously updates. That is, as discussed above with regard to decaying the maximum capacitance value, over the course of a gameplay experience, certain conditions may affect the capacitance values detected at the proximity sensor 600. For instance, the capacitance value associated with the finger being fully extended from the controller 100 may change.
• a first capacitance value may represent or correspond to a finger being fully extended from the controller 100 and at a second instance in time, a second capacitance value may represent or correspond to the finger being fully extended from the controller 100.
• the second capacitance value may be less than the first capacitance value.
• the capacitance value associated with the finger being fully extended from the controller 100 may update. That is, although the second capacitance value may represent the user fully extending their finger from the controller 100, the second capacitance value may not be associated with the minimum capacitance value representing the finger being fully extended. Through decaying the minimum capacitance value, the controller 100 may dynamically adjust the capacitance value associated with the finger being fully extended.
• Decaying the minimum capacitance value may be based at least in part on receiving a threshold amount of capacitance values not within predetermined range of the minimum capacitance value. Additionally, or alternatively, decaying the minimum capacitance value may be based at least in part on a threshold amount of time, number of frames, or number of counts passing since receiving capacitance values within the predetermined range of the minimum capacitance value, and/or the capacitance values over a predetermined past number of capacitance values changing by a threshold amount.
• the amount of the decay may be based on the range of capacitance values detected over a previous number of frames (e.g., 20 frames, 50 frames, 100 frames, and so forth), a range of capacitance values between the minimum capacitance value and the maximum capacitance value, a difference between the minimum capacitance value and the average capacitance value detected over a predetermined number of past frames, and/or a percentage of the difference between the minimum capacitance value and the average capacitance value.
• the minimum capacitance value may decay.
• the decay may be determined by calculating (1) the range of capacitance values detected over the previous number of counts or frames, (2) the range of capacitance values between the minimum capacitance value the maximum capacitance value, and (3) a decay variable.
• the decay variable for the minimum capacitance value may be a percentage of the variability of the capacitance values between the minimum capacitance value the maximum capacitance value as detected over a previous number of frames.
• the decay variable for the minimum capacitance value may be twenty percent of the range of capacitance values between the minimum capacitance value and the maximum capacitance value for each frame. In other words, for every frame, the decay variable may be calculated by multiplying the range of capacitance values between the minimum capacitance value the maximum capacitance value by twenty percent.
• the minimum capacitance value may be permitted to decay twice as fast as the maximum capacitance value. That is, because the minimum capacitance value, or capacitance values within a lower-range of the capacitance values may be susceptible to noise, the minimum capacitance value may decay twice as fast as the maximum capacitance value.
• the amount the minimum capacitance value decays may be limited such that the minimum capacitance value does not decay more than fifty percent of the range of capacitance values between the maximum capacitance value the minimum capacitance value. In this sense, the minimum capacitance value may be limited to decaying more than half of the range of capacitance values between the maximum capacitance value the minimum capacitance value.
• the amount the minimum capacitance value decays may be proportional to the range of capacitance values between the maximum capacitance value the minimum capacitance value. Moreover, the minimum capacitance value may be limited to decaying to within a certain threshold of the maximum capacitance value to reduce noise.
• Decaying the minimum capacitance value may be used to determine an updated minimum capacitance value for the capacitive pads 602. Additionally, in combination with decaying the maximum capacitance value, the updated minimum capacitance value may also be used to determine an updated range of capacitance values, an updated bias, and/or an updated scale factor for individual capacitive pads 602 of the proximity sensor 600.
• a rate at which the minimum capacitance value decays may depend on a variability of the measured capacitance values. For instance, if a large variability exists between the capacitance values detected over a past number of frames, the minimum capacitance value may decay at a faster rate, as compared to if a small variability exists between the capacitance values detected over the past number of frames.
• the controller 100 may determine an average capacitance value detected by the capacitive pads 602 over a previous number of counts or frames. For instance, the controller 100 may determine an average capacitance value detected by individual capacitive pads 602 over the previous twenty samples. However, as noted above, decaying the maximum capacitance value and/or the minimum capacitance value may be based on the average capacitance value determined the previous number of frames. Hence, the average capacitance value may be known.
• logic of the controller 100 may determine whether the average capacitance value determined at 810 is greater than the updated maximum capacitance value determined at 806. That is, based at least in part on determining the average capacitance value, the controller 100 may compare the average capacitance value with the maximum capacitance value to determine whether the average capacitance value is greater than the maximum capacitance value detected at the capacitive pad 602. In other words, each capacitive pad 602 may be associated with a maximum capacitance value and the controller 100 may determine whether the average capacitance value is greater than the maximum capacitance value.
• the process 800 may follow the“YES” route to 814 where logic of the controller 100 may determine a new maximum capacitance value.
• logic of the controller 100 may determine that the maximum capacitance value is too low and that the average capacitance value has exceeded the maximum capacitance value.
• logic of the controller 100 may determine a new maximum capacitance value for the capacitive pad 602 based at least in part on a percentage, or weight, of the average capacitance values received over a predetermined number of frames. For instance, the new maximum capacitance value, instead of decaying, may extend towards the average capacitance value determined from a previous number of past samples.
• the amount the maximum capacitance value“grows” may be determined by multiplying the average capacitance values from the previous number of frames by a variable, such as 0.2. However, the variable may be optimized such that the maximum capacitance value“grows” towards the average capacitance value determined over the previous number of frames to accurately determine the hand gesture of the user.
• the amount of growth of the maximum capacitance value may also be based at least in part on an amount of the average capacitance value exceeds the maximum capacitance value.
• the process 800 may follow the“NO” route to 816, where logic of the controller 100 may determine whether the average capacitance value is lower than the minimum capacitance value determined at 808. That is, based at least in part on determining the average capacitance value, the controller 100 may compare the average capacitance value with the minimum capacitance value to determine whether the average capacitance value is less than the minimum capacitance value detected at the capacitive pad 602. In other words, each capacitive pad 602 may be associated with a minimum capacitance value and the logic of the controller 100 may determine whether the average capacitance value is less than the minimum capacitance value.
• the process 800 may follow the“YES” route to 818 where logic of the controller 100 may determine a new minimum capacitance value.
• the logic of the controller 100 may determine that the minimum capacitance value is too high and that the average capacitance value has exceeded the minimum capacitance value.
• the controller 100 may determine a new minimum capacitance value for the capacitive pad 602 based at least in part on a percentage, or weight, of the average capacitance values over a previous number of frames.
• the new minimum capacitance value instead of decaying, may extend towards the average capacitance value of the previous number of frames.
• the amount the minimum capacitance value“grows” may be determined by multiplying the average capacitance value of a predetermined number of previous capacitance values by a variable, such as 0.4.
• the variable may be optimized such that the minimum capacitance value“grows” towards the average capacitance value determined over the previous number of frames to accurately determine the hand gesture of the user.
• the amount of growth of the minimum capacitance value may also be based at least in part on an amount the average capacitance value exceeds the minimum capacitance value.
• the process 800 may follow the“NO” route to 820 where the controller 100 may calibrate the proximity sensor 600.
• Calibrating the proximity sensor 600 may also follow from determining the new maximum capacitance value (e.g., 814) and/or determining the new minimum capacitance value (e.g., 818).
• calibrating the proximity sensor 600 may involve calculating the bias and scale factor for the capacitive pads 602.
• calibrating the proximity sensor 600 may involve, for each capacitive pad 602, determining the range of capacitance values detected, a bias, and/or a scale factor. Accordingly, to calibrate the proximity sensor 600, the controller 100 may utilize the new maximum capacitance value and/or the minimum capacitance value.
• the process 800 may loop to 802 where the controller 100 may receive subsequent capacitance values. That is, through iteratively receiving capacitance values, the process 800 may continuously decay the maximum capacitance value, the minimum capacitance value, determine and/or update the range of capacitance values detected by the capacitive pads 602, and/or to normalize the capacitance values.
• FIG. 9 illustrates a process 900 for receiving capacitance values associated with a hand gesture at a controller 100, calibrating a proximity sensor 600, and transmitting the capacitance values and/or an indication of a proximity of a finger relative to the controller 100.
• the process 900 may receive first capacitance values.
• one or more processors of the controller 100 may receive the first capacitance values from the proximity sensor 600.
• the first capacitance values may represent the capacitance values detected by individual capacitive pads 602 of the proximity sensor 600, where the first capacitance values represent a proximity of individual fingers of the user relative to the controller 100.
• the first capacitance values may correspond to a first hand gesture 904 and the first capacitance values may indicate a proximity of the fmger(s) of the user relative to the capacitive pads 602.
• the process 900 may normalize the first capacitance values.
• the one or more processors of the controller 100 may normalize the first capacitance values received from the individual capacitive pads 602 utilizing operations of the process(es) 700 and/or 800 discussed and explained hereinabove.
• the process 900 may calibrate the proximity sensor 600 based at least in part on the first capacitance values.
• the one or more processors of the controller 100 may calibrate the proximity sensor 600 through determining a range of capacitance values detected at the capacitive pads 602, a bias associated with the capacitive pads 602, and/or a scale factor associated with the capacitive pads 602, as discussed above with regard to the process(es) 700 and/or 800 discussed and explained hereinabove.
• the process 900 may transmit the first capacitance values and/or an indication of the finger position(s).
• the one or more processors of the controller 100 may utilize one or more interfaces of the controller 100 to transmit the first capacitance values and/or the indication of the finger position(s) to a computing device.
• the computing device may utilize the first capacitance values and/or the indication of the finger position(s) to generate image data depicting the first hand gesture 904.
• the indication of the finger position(s) may indicate a numerical value on a scale of [0,1] corresponding to the relative position of the fmger(s) to the controller 100.
• a value of one on the [0,1] scale may indicate that a finger touches the controller 100 whereas a value of zero on the [0,1] scale may indicate that the finger is fully extended from the controller 100.
• the controller 100 may transmit an indication including a finger value on a scale of [0,1] that correspond to a position of the finger relative to the controller 100.
• the finger values may therein be used to generate image data corresponding to the first hand gesture 904.
• the process 900 may receive second capacitance values.
• the one or more processors of the controller 100 may receive the second capacitance values from the proximity sensor 600.
• the second capacitance values may represent the capacitance values detected by individual capacitive pads 602 of the proximity sensor 600, where the second capacitance values represent a proximity of individual fingers of the user relative to the controller 100.
• the second capacitance values may correspond to a second hand gesture 914 and the second capacitance values may indicate a proximity of the fmger(s) relative to the capacitive pads 602.
• the process 900 may normalize the second capacitance values and at 918 the process 900 may calibrate the proximity sensor 600 (e.g., determining a range, a bias, and/or a scale factor associated with the capacitive pads 602).
• the process 900 may transmit the second capacitance values and/or an indication of the finger position(s).
• the one or more processors of the controller 100 may utilize one or more interfaces of the controller 100 to transmit the second capacitance values and/or the indication of the finger position(s) to a computing device.
• the computing device may utilize the second capacitance values and/or the indication of the finger position(s) to generate image data depicting the second hand gesture 914.
• the indication of the finger position(s) may indicate a numerical value on a scale of [0,1] the relative position of the fmger(s) relative to the controller 100.
• FIG. 10 illustrates a process 1000 for calibrating a proximity sensor 600 of a controller 100, reducing noise within proximity data, and utilizing the proximity data to generate image data representative of hand gestures performed by the user.
• the process 1000 may receive proximity data from the proximity sensor 600, where the proximity data represents or indicates raw capacitance values detected by capacitive pads 602 of the proximity sensor 600. As illustrated in FIG. 10, in some instances, the process 1000 may receive capacitance values from individual capacitive pads 602, as illustrated by ai, ai, and a N . In some instances, the process 1000 may receive the raw capacitance values from the proximity sensor 600 for each frame displayed in a VR environment. [0104] At 1004, the process 1000 may perform a factory normalization to normalize the raw capacitance values. For instance, the capacitive pads 602 may have different biases, scale factors, and offsets depending on manufacturing conditions, a size of the capacitive pad 602, and so forth. In some instances, the factory normalization may involve a first order calibration to remove the bias within the capacitance value and normalizing the capacitance value.
• the process 1000 may perform a grip calibration. As shown, the grip calibration may involve sub-blocks 1008, 1010, and 1012, which are discussed in detail in turn.
• the process 1000 may perform a statistical analysis to observe a range of capacitance values, the maximum capacitance value received, the minimum capacitance value received, the average capacitance value, and/or the median capacitance value for respective capacitive pads 602.
• the process 1000 may perform a discrete gesture detection.
• the process 1000 may analyze the proximity data (i.e., the capacitance values), after being normalized according to the factory normalization, to detect a discrete gesture at the controller 100. For instance, if the proximity data indicates that capacitance values of the capacitive pads 602, or a portion thereof, suddenly drop, the process 1000 may associate this drop of capacitance values with the user releasing his or her hand from the controller 100 or releasing a particular finger from the controller 100.
• the capacitance values received as the user suddenly releases his or her finger from the controller 100 may correspond to a minimum capacitance value for a range of capacitance values detected by a particular capacitive pad 602 (e.g., when the finger is not touching the controller 100).
• the capacitance value received prior to the sudden drop may correspond to a maximum capacitance value of the range of capacitance values detected by a particular capacitive pad 602 (e.g., where the capacitance value represents when the finger is touching the controller 100).
• the process 1000 may determine a bias and a scale factor for capacitance values of the capacitive pads 602 to normalize the capacitance values received at respective capacitive pads 602 of the proximity sensor 600.
• the process 1000 may perform a continuous calibration update and decay. As the process 1000 may continuously receive proximity data from the proximity sensor 600, the process 1000 may continuously monitor the proximity data to re-calibrate or reset the minimum capacitance value and/or the maximum capacitance value for the range of capacitance values for a given capacitive pad 602. In other words, through continuously receiving the proximity data from the individual capacitive pads 602, the process 1000 may determine whether the capacitance values are lower than or higher than the previously determined minimum capacitance value of the range and/or the maximum capacitance value of the range, respectively.
• the process 1000 may determine, or set, a new minimum capacitance value or a new maximum capacitance value, thereby adjusting the range of capacitance values detected by a capacitive pad 602. Additionally, in some instances, the continuous calibration may reduce a dependence on the process 1000 determining the discrete gesture detection at 1010.
• the process 1000 may assign a weight or percentage to a newly detected minimum capacitance value or a newly detected maximum capacitance value to update the minimum capacitance value or maximum capacitance value, respectively. For instance, if the process 1000 detects a capacitance value below a previously detected minimum capacitance value for a certain amount of time, or a certain number of frames, the process 1000 may weigh the capacitance value to update the minimum capacitance value. Additionally, the minimum capacitance value and/or the maximum capacitance value may decay over time depending on how the user grips the controller 100, environmental conditions (e.g., humidity), or other properties (e.g., skin wetness).
• environmental conditions e.g., humidity
• other properties e.g., skin wetness
• the amount the minimum capacitance value and the maximum capacitance value may decay may be limited, such that the minimum capacitance value and the maximum capacitance value are separated by a threshold amount to reduce sensor noise.
• the decay may depend on time and/or a rate of change in the capacitance values detected by the proximity sensor 600. For instance, if the user taps their fingers on the controller 100 such that the controller 100 detects a threshold amount of capacitance values within a certain period of time, or frames, or the controller 100 switches users, thereby potentially causing a large change (or variability) in the capacitance values received, the rate of decay may increase to reduce an amount of time required to update the minimum capacitance value and/or the maximum capacitance value.
• the capacitance values sensed from each capacitive pad 602 may be normalized on a scale of [0,1]
• the scale of [0,1] may represent, for a particular grip of a user and for the individual capacitive pads 602, the high and low levels for the capacitance values sensed from the proximity sensor 600.
• the process 1000 may perform a weighted sum of the capacitance values.
• the process 1010 may assign a weight to capacitance values from the capacitive pads 602 depending on the controller configuration. That is, the capacitance values are normalized between [0,1] and weights are assigned to individual capacitance values received the individual capacitive pads 602. For instance, if a certain controller configuration includes five capacitive pads 602 assigned to a particular finger, the capacitance values may include an equal weight (e.g., one-fifth). In other words, when the capacitive pads 602 detect a maximum capacitance value, the output of the weighted sum may equal one.
• determining the weighted sum may involve sub-blocks 1016, 1018, 1020, and 1022.
• Sub-block 1016 may include the controller configuration models for the controller 100. Noted above, the controller configurations designate a mapping of capacitive pads 602 of the proximity sensor 600 to associate certain capacitive pads 602 with certain fingers of the user.
• the process 1000 may perform a dynamic controller configuration selection, where the process 1000 inputs the capacitance values into the controller configuration model(s) to determine a best match, or highest matched, controller configuration according to the grip of the user. Selecting the controller configuration may involve determining a ratio of (1) the variance of capacitance values for a group of capacitive pads 602 assigned to a particular finger and (2) the variance of capacitance values between groups of capacitive pads 602. A ratio is calculated for each controller configuration to determine a most likely match for the grip of the user.
• the process 1000 may filter noise contained within the capacitance values. For instance, when a finger is not touching the controller 100, such as when a finger is fully extended, those capacitive pads 602 associated with the finger not touching the controller 100 (e.g., finger is fully extended) may be susceptible to noise. Here, detecting a small amount of capacitance may cause a large amount of noise within the received capacitance value. In instances where the capacitance values from the proximity sensor 600 for individual capacitive pads 602 fall below a certain threshold, or if the capacitance values are within a certain limit of the low-level capacitance value for the capacitive pads 602, the process 1000 may suppress a detected capacitance.
• the process 1000 may assign a low weight to the capacitance values.
• the process 1000 may reject certain capacitance values from capacitive pads 602 of the proximity sensor 600 and/or fingers associated with respective capacitive pads 602. For instance, at 1020 the process 1000 may identify capacitive pads 602 that have a small range between the minimum capacitance value or maximum capacitance value.
• capacitance values received by the capacitive pads 602 may introduce noise and disregarding certain capacitive pads 602, or groups of capacitive pads 602, may increase a reliability that the proximity data corresponds to the hand gesture of the user. That is, if the range of capacitance values detected by a capacitive pad 602 is small, the capacitive pad 602 may be susceptible to large amounts of noise.
• the capacitive pad and finger rejection may include identifying capacitive pads 602 whose minimum capacitance value or maximum capacitance value are within a threshold range of one another. For instance, if the minimum capacitance value and the maximum capacitance value are separated by a small range, the capacitive pad 602 may be unable to accurately sense and detect the finger positions of the user in sufficient detail.
• the capacitive pad 602 may detect capacitance values within a threshold range, the measured capacitance values may not accurately correspond to the finger position.
• certain fingers may be associated with a number of capacitive pads 602 that decreases the reliability of the detected capacitance values.
• capacitance values received by the capacitive pads 602 may introduce noise and disregarding certain capacitive pads 602, or groups of capacitive pads 602, may increase a reliability that the capacitance values correspond to the hand gesture of the user. Additionally, or alternatively, certain fingers may be associated with a number of capacitive pads 602 having a low reliability. Rejecting certain fingers, or a group of capacitive pads 602, introduces a contingency behavior for controller configurations for small hand. In these scenarios, a respective finger may associate with adjacent fingers (e.g., pinky finger associates with ring finger).
• the process 1000 may perform a final normalization. For example, in some instances, a capacitive pad 602 assigned to a particular finger may not detect a capacitance value or the capacitance values may be unreliable. Here, the user may not touch certain capacitive pads 602 of the proximity sensor 600 because of the size of the hand or in instances where the user readjusted his or her grip. Additionally, in some instances where the minimum capacitance value and the maximum capacitance value are narrow, or separated by a small range, the capacitance values may be unreliable and noise may significantly impact finger motion.
• the final normalization 1024 may determine a reliability of the capacitance values, and if a reliability is low, the weight of a capacitance value from a capacitive pad 602 is removed from the weighted sum. Therein, the capacitance values are summed and divided by the sum of the weights of the capacitive pads 602 being used.
• the process 1000 may filter and curve fit the proximity data to represent a hand gesture of the user.
• the filtering and curve fitting may involve linearizing the final normalization of the proximity data on the [0,1] scale to achieve a linear relationship between the proximity data and a position of a finger (e.g., curled, extended, half-way extended, etc.).
• the final normalization values determined at 1024 may follow an exponential curve, such that as the hand of the user comes into proximity of the controller 100, or grips the controller 100, the final normalized values exponentially increase.
• the summed capacitance value may be exponentially related to the proximity with the finger disposed on/around the controller 100.
• Linearizing the values on a [0,1] scale such that the capacitance values are correlated with a finger position may reduce sensitivity and an impact that noise may have when a finger is extended from the controller 100, as well as when a finger touches or is in close proximity to the controller 100.
• the filtering and curve fitting may involve various sub-blocks to achieve final values that are utilized to generate hand gestures.
• the process 1000 may apply filtering before or after the curve fit.
• the sub-blocks may involve filtering capacitance values within a low-level range of capacitance values when the capacitive pads 602 are susceptible to noise.
• the capacitive pads 602 are less susceptible to noise.
• the process 1000 may apply adaptive filtering at 1028 to adjust the amount of filtering performed on the capacitance values.
• the adaptive filtering may adaptively filter to more aggressively filter the capacitance values within the low-range of capacitance values compared to the capacitance values are within the high range of capacitance values.
• the adaptive filtering may involve sub-blocks 1030, 1032, and 1034.
• the adaptive filtering at 1026 may utilize the results of sub-blocks 1030, 1032, and 1034 to determine how much noise exists in the normalized value to determine an amount of filtering to be applied to the normalized capacitance values.
• Determining an amount of noise that exists in the capacitance values may involve determining which capacitive pads 602 are being used to generate the capacitance values, as well as the minimum capacitance value and the maximum capacitance value for the respective capacitive pads 602. For instance, the capacitive pads 602 may have a baseline noise and if the range between the minimum capacitance value and the maximum capacitance value for a capacitive pad 602 is low, the baseline noise of the capacitive pad 602 may equate to a large amount of finger movement (i.e., baseline noise is a large fraction of the range of capacitance values the capacitive pad 602 is able to sense).
• the signal to noise ratio may be high.
• the process 1000 may filter capacitance values more heavily than when the range of capacitance values is larger.
• the filtering and curve fit 1026 may be repeated for each capacitive pad 602 as each capacitive pad 602 may include respective high and low-level capacitance values. Additionally, the amount of filtering applied at 1026 may depend on which capacitive pads 602 and/or which capacitive pads 602 have been rejected (e.g., pad and finger rejection 1022).
• the sum noise prediction at 1030 may filter the capacitance values based on which capacitive pads 602 are being used, the weights assigned the capacitive pads 602, as well as the respective baseline noise for the capacitive pads 602.
• the process 1000 may include a default capacitive pad noise at 1028, which may represent an estimated baseline noise for individual capacitive pads 602.
• the sum noise prediction step at 1030 may therefore determine, for those capacitive pads 602 being used, their respective baseline noise values.
• the sum noise prediction step may also determine the expected or predicted noise for the capacitive pads 602.
• the capacitive pads 602 being used sense capacitance values over a large range (i.e., between the minimum capacitance value and the maximum capacitance value), the capacitance values may include a low amount of noise and less filtering may be applied.
• the range of capacitance values for a capacitive pad 602 is narrow (i.e., between the minimum capacitance value and the maximum capacitance value)
• the capacitance values may include a large amount of noise and the process 1000 may apply a greater amount of filtering.
• the dNorm/dt at 1032 may take into consideration a change in capacitance value over time.
• the potential noise introduced within the capacitance values may be disregarded or weighted accordingly. That is, instead of filtering the capacitance values and introducing latency, if the capacitance values change over a threshold amount over a threshold amount of time, less filtering may be applied to the capacitance values. In this sense, less filtering may be applied when greater finger movement is detected and more filtering may be applied when less finger movement is detected.
• the dCurl/dNorm at 1034 may filter the normalized capacitance value based on an amount of capacitance detected. For instance, in the high-range of capacitance values, where the finger grips the controller, less filtering may be applied because the noise may have less of an impact on finger position. However, at the low range of capacitance values, where the finger is displaced from the controller, or in close proximity, more filtering may be applied because small changes in capacitance values may have a significant impact on the position of the finger. Here, a small change in capacitance values may result in a large change of the finger gesture.
• the lo-pass filter may represent a tunable lo-pass mean filter that adjusts the amount of filtering on detected capacitance values.
• the amount of filtering may be on a scale of [0,1] and may be based on the result of the amount of filtering determined at the adaptive filter 1028. That is, the lo-pass filter may filter the capacitance values as determined from the adaptive filtering.
• the process 1000 may curve fit the capacitance values on a [0,1] scale to associate the capacitance values with a position of the finger, or a hand animation.
• the output of the curve fit may include a number for each finger, where the number indicates the finger position of each finger of the hand.
• the process 1000 may apply a backlash filter after the curve to filter changes in capacitance values that fall below a threshold. For instance, if the capacitance values do not change by a threshold amount on the [0,1] scale, the capacitance values may be filtered. Such filtering may reduce perceived finger twitching and motion by the user.
• the joint model may correspond to a hand animation (e.g., a hand skeleton). For instance, the joint model may generate a hand animation corresponding to the number assigned to individual fingers of the hand from the curve fit at 1040.
• FIG. 11 illustrates an example chart 1100 for updating a minimum capacitance value associated with a proximity sensor 600.
• the example chart 1100 is shown illustrating capacitance values 1102 received from a capacitive pad 602 of the proximity sensor 600.
• the controller 100 may receive the capacitance values 1102 at every frame displayed in a VR environment.
• the capacitive pad 602 may measure the capacitance values 1102 associated with an object (e.g., finger) in proximity to the capacitive pad 602.
• the controller 100 may determine a maximum capacitance value 1104, a minimum capacitance value 1106, and/or a range 1108 between the maximum capacitance value 1104 and the minimum capacitance value 1106.
• the maximum capacitance value 1104 may correspond to a capacitance value received when the object (e.g., finger) is touching the controller 100.
• the controller 100 may associate this capacitance value with the maximum capacitance value 1104 and a touch at the controller 100. This association may be utilized by one or more computing devices to generate image data of the user holding an object (e.g., weapon, ball, etc.).
• the minimum capacitance value 1106 may correspond to a capacitance value received when the object (e.g., finger) is not touching the controller 100 and is fully extended from the controller 100.
• the controller 100 may associate this capacitance value with the minimum capacitance value 1106 and the user fully extending his or her finger from the controller 100. This association may be utilized by one or more computing devices to generate image data of the user fully extending his or her finger (e.g., dropping a ball).
• the controller 100 may determine an average capacitance value 1110.
• the average capacitance value 1110 may be determined from capacitance values from a predetermined number of past samples. For instance, the average capacitance value 1110 may be determined using the past twenty capacitance values received.
• the controller 100 may determine a bias, and/or a scale factor for the capacitive pad 602 to determine a relative finger position of the user between being fully extended and touching the controller 100. Throughout a gameplay experience, however, one or more factors may impact detected the capacitance values 1102.
• a temperature of an environment in which the controller 100 resides may decrease and/or a wetness of the user hand/fmger may increase. In some instances, these factors may result in lower capacitance values 1102.
• the maximum capacitance value 1104 (or a level/limit associated therewith) may decay. In other words, the maximum capacitance value 1104 may decay to a new capacitance value to accurately detect and/or associate the capacitance values 1102(A) with a finger position of the user (e.g., using the bias, scale factor, etc.)
• FIG. 11 illustrates that the maximum capacitance value 1104 may decay.
• the decay may occur as a result of the controller 100 not receiving capacitance values 1102 within a certain threshold of the maximum capacitance value 1104.
• the maximum capacitance value 1104 may decay.
• the maximum capacitance value 1104 may decay if the average capacitance value 1110 over a past number of N samples 1112 is not within the certain threshold of the maximum capacitance value 1104.
• the maximum capacitance 1106 value may continuously decay to converge towards the capacitance values 1102 and/or the average capacitance value 1110.
• an amount of the decay may be based at least in part on a variable decay rate, the range 1108, and/or a range 1114 determined over the past number of N samples 1112.
• the variable decay rate may be substantially ten percent of the range 1108.
• the maximum capacitance value 1104 may gradually decay over a period of time to converge towards the detected capacitance values 1102 and/or the average capacitance value 1110. Additionally, an amount the maximum capacitance value 1104 decays may be limited such that the maximum capacitance value 1104 and the minimum capacitance value 1106 are separated by a threshold amount.
• a rate at which the maximum capacitance value 1104 decays may be based at least in part on a number of frames, the controller 100 has not received capacitance values 1102 within the threshold of the maximum capacitance value 1104.
• the rate at which the maximum capacitance value 1104 decays may also depend on a variability of the measured capacitance values 1102 over the past number of N samples 1112. For instance, if a large variability exists between the capacitance values 1102 detected over the past number of N samples 1112, the maximum capacitance value 1104 may decay at a faster rate, as compared to if a small a variability exists between the capacitance values 1102 detected over the past number of N samples 1112.
• the maximum capacitance value 1104 may decay to a new value and a new bias, scale factor, and/or range 1116 may be determined.
• the controller 100 may detect capacitance values 1102 that are greater than the maximum capacitance value. That is, through continuously updating the maximum capacitance value 1104, the controller 100 may receive capacitance values 1102 that are greater than the determined maximum capacitance value. These capacitance values 1102 may“overshoot” the maximum capacitance value 1104. In such instances, if the capacitance values 1102 overshoot the maximum capacitance value 1104, or if the average capacitance value 1110 overshoots the maximum capacitance value 1104, the maximum capacitance value 1104 may“grow” or extend towards the capacitance values 1102 or the average capacitance value 1110.
• the maximum capacitance value 1104 may make up for this difference.
• the amount and/or the rate at which the maximum capacitance value 1104 updates may be based on a growth variable (e.g., 0.2) as well as the average capacitance value 1110 detected over a past number of frames.
• the handheld controller may update the maximum capacitance value 1104 and determine an updated bias, an updated scale factor, and so forth.
• the maximum capacitance value 1104 may continue to extend until the maximum capacitance value 1104 catches up with the capacitance values 1102 and/or the average capacitance value 1110.
• the minimum capacitance value 1106 may similarly decay based on the capacitance values 1102 and/or the average capacitance value 1110. For instance, the minimum capacitance value 1106 may decay as a result of the controller 100 not receiving capacitance values 1102 with a threshold of the minimum capacitance value 1106 over for a threshold number of frames. In doing so, the minimum capacitance value 1106 may decay to a new capacitance value to accurately detect and/or associate the capacitance values 1102 with a finger position of the user (e.g., using the bias, scale factor, etc.).
• An amount or rate at which the minimum capacitance value 1106 decays may be based at least in part on a period of time the controller 100 has not received capacitance values 1102 within a threshold of the minimum capacitance value 1106.
• the amount the minimum capacitance value 1106 decays may be limited such that the minimum capacitance value 1106 and the maximum capacitance value 1104 are separated by a threshold amount or range.
• the minimum capacitance value 1106 may decay at a rate twice as the maximum capacitance value 1104.
• the maximum capacitance value 1104 and the minimum capacitance value 1106 may decay at the same time based on the controller 100 receiving the capacitance values 1102.
• an updated range, an updated bias, and/or an updated scale factor may be determined for use in calibrating the capacitive pad 602 and/or normalizing capacitance values 1102 received from the capacitive pad 602.
• FIG. 12 illustrates example components of a controller 1200, such as the controller 100.
• the controller 100 may include one or more input/output (I/O) devices 1202, such as the controls described above (e.g., joysticks, trackpads, triggers, etc.), a proximity sensor 600, and/or potentially any other type of input or output devices.
• I/O devices 1202 may include one or more microphones to receive audio input, such as user voice input.
• one or more cameras or other types of sensors e.g., inertial measurement unit (IMU) may function as input devices to receive gestural input, such as motion of the controller 100.
• IMU inertial measurement unit
• additional input devices may be provided in the form of a keyboard, keypad, mouse, touch screen, joystick, control buttons and the like.
• the input device(s) may further include control mechanisms, such as basic volume control button(s) for increasing/decreasing volume, as well as power and reset buttons.
• the output devices may include a display, a light element (e.g., LED), a vibrator to create haptic sensations, a speaker(s) (e.g., headphones), and/or the like.
• output by the EO devices may be based on input received by one or more of the input devices.
• touch sensed via the proximity sensor 600 of the controller 100 may result in the output of a haptic response by a vibrator located adjacent (e.g., underneath) the proximity sensor 600.
• the output may vary based at least in part on a characteristic of a proximity of one or more objects relative to the proximity sensor 600, such as the capacitive pads 602 disposed on/within the handle 112 of the controller 100.
• a touch input (or proximity of an object) at a first location on the handle 112 may result in a first haptic output
• a touch input (or proximity of an object) at a second location on the handle 112 may result in a second haptic output
• a particular gesture on the handle 112 may result in a particular haptic output (or other type of output).
• a tap and hold gesture (detected by the proximity sensor 600) on the handle 112 may result in a first type of haptic output
• a tap and release gesture on the handle 112 may result in a second type of haptic output
• a hard tap of the handle 112 may result in a third type of haptic output.
• the controller 100 may additionally or alternatively other types of EO devices.
• the controller 100 may include one or more communication interfaces 1204 to facilitate a wireless connection to a network and/or to one or more remote systems (e.g., a host computing device executing an application, a game console, other controllers, etc.).
• the communication interfaces 1204 may implement one or more of various wireless technologies, such as Wi-Fi, Bluetooth, radio frequency (RF), and so on.
• the controller 100 may include physical ports to facilitate a wired connection to a network, a connected peripheral device, or a plug-in network device that communicates with other wireless networks.
• the controller 100 further includes one or more processors 1206 and computer-readable media 1208.
• the processors(s) 1206 may include a central processing unit (CPU), a graphics processing unit (GPU), both CPU and GPU, a microprocessor, a digital signal processor, or other known processing units or components. Additionally, or alternatively, the functionally described herein may be performed, at least in part, by one or more hardware logic components.
• illustrative types of hardware logic components include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc.
• FPGAs field-programmable gate arrays
• ASICs application-specific integrated circuits
• ASSPs application-specific standard products
• SOCs system-on-a-chip systems
• CPLDs complex programmable logic devices
• each of the processor(s) 1206 may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.
• the computer-readable media 1208 may include volatile and nonvolatile memory, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data.
• memory includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium which can be used to store the desired information and which can be accessed by a computing device.
• the computer-readable media 1208 may be implemented as computer-readable storage media (“CRSM”), which may be any available physical media accessible by the processor(s) 1206 to execute instructions stored on the computer-readable media 1208.
• CRSM may include random access memory (“RAM”) and Flash memory.
• RAM random access memory
• CRSM may include, but is not limited to, read-only memory (“ROM”), electrically erasable programmable read-only memory (“EEPROM”), or any other tangible medium which can be used to store the desired information and which can be accessed by the processor(s) 1206.
• modules such as instruction, datastores, and so forth may be stored within the computer-readable media 1208 and configured to execute on the processor(s) 1206.
• a few example functional modules are shown as stored in the computer-readable media 1208 and executed on the processor(s) 1206, although the same functionality may alternatively be implemented in hardware, firmware, or as a system on a chip (SOC).
• An operating system module 1210 may be configured to manage hardware within and coupled to the controller 100 for the benefit of other modules.
• the computer-readable media 1208 may store a network-communications module 1212 that enables the controller 100 to communicate, via one or more of the communication interfaces 1204, with one or more other devices, such as a personal computing device executing an application (e.g., a game application), a game console, a remote server, other controlled s), computing devices, or the like.
• the computer-readable media 1208 may further include a game-session database 1214 to store data associated with a game (or other application) executing on the controller 100 or on a computing device coupled to the controller 100.
• the computer-readable media 1208 may also include a device record database 1216 that stores data associated with devices coupled to the controller 100, such as the personal computing device, game console, remote server, or the like.
• the computer-readable media 1208 may further store game-control instructions 1218 that configure the controller 100 to function as a gaming controller, and universal-control instructions 1220 that configure the controller 100 to function as a controller of other, non gaming devices.
• the computer-readable media 1208 may additionally store controller configuration(s) 1222.
• the controller configuration(s) 1222 may represent or include data associated with the assignment of the capacitive pads 602 of the proximity sensor 600 to associate certain capacitive pads 602 with respective fingers of a user operating the controller 100.
• the computer-readable media 1208 may additionally store data associated with a range of capacitance values 1224.
• the range of capacitance values 1224 may indicate a range of capacitance values detected or capable of being detected at individual capacitive pads 602 of the proximity sensor 600.
• the controller 100 may calculate and/or determine the range of capacitance values received at the capacitive pads 602.
• the computer-readable media 1208 may also store a minimum capacitance value 1226 and/or a maximum capacitance value 1228.
• the computer-readable media 1208 may also store a scale factor and bias 1230.
• the scale factor and bias 1230 may be associated with individual capacitive pads 602 and may be determined utilizing the range of capacitance values 1224. Accordingly, utilizing the range of capacitance values 1226 and/or the scale factor and bias 1230, the controller 1200 may normalize and/or calibrate proximity data received from the proximity sensor 600. CONCLUSION
• a handheld controller comprising:
• a proximity sensor configured to detect capacitance values based on an object in proximity to the proximity sensor
• one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
• the proximity sensor comprises capacitive pads
• individual capacitive pads of the capacitive pads are configured to generate a capacitance value associated with a proximity of the object relative to the handheld controller.
• the maximum capacitance value to the updated maximum capacitance value or the minimum capacitance value to the updated minimum capacitance value.
• determining a range of proximity values detected by a proximity sensor the range proximity of values including a maximum value and a minimum value
• the first rate is based at least in part on a variability of the proximity values received during a predetermined number of previous frames
• the second rate is based at least in part on the variability of the proximity values received during the predetermined number of previous frames
• the decaying the limit associated with the maximum value is based at least in part on the first rate
• the decaying the limit associated with the minimum value is based at least in part on the second rate.
• the average value is greater than the limit associated with the maximum capacitance value
• the average value is less than the limit associated with the minimum capacitance value.
• the average value is determined over a predetermined number of previous frames
• the updated limit associated with the maximum value is based at least in part on the average value determined over the predetermined number of previous frames
• the updated limit associated with the minimum value is based at least in part on the average value determined over the predetermined number of previous frames.
• the proximity sensor comprises capacitive pads; and individual capacitive pads are configured to generate the proximity values, wherein proximity the values are associated with an object being in proximity to a controller.
• a controller comprising:
• one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
• the updated range of proximity values including at least one of a second maximum proximity value or a second minimum proximity value
• the first maximum proximity value is associated with a first proximity of an object relative to the sensor and the first minimum proximity value is associated with a second proximity of the object relative to the sensor;
• the second maximum proximity value is associated with the first proximity of the object relative to the sensor and the second minimum proximity value is associated with the second proximity of the object relative to the sensor.
• the one or more non-transitory computer- readable media store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform an act further comprising transmitting an indication of the proximity to a computing device.
• the average proximity value is greater than the second maximum proximity value
• the average proximity value is less than the second minimum proximity value; and determining at least one of a third maximum proximity value or a third minimum proximity value.
• decaying the first maximum proximity value is based at least in part on determining that one or more of the proximity values are outside a first threshold range of the first maximum proximity value for a threshold period of time;
• decaying the first minimum proximity value is based at least in part on determining that one or more of the proximity values are outside a second threshold range of the first minimum proximity value for the threshold period of time.
• the decaying the first maximum proximity value is based at least in part on the range of proximity values and a range of proximity values from a subset of the proximity values; and the decaying the first minimum proximity value is based at least in part on the range of proximity values and the range of proximity values from the subset of the proximity values.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Human Computer Interaction (AREA)
• General Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• User Interface Of Digital Computer (AREA)
• Electronic Switches (AREA)
• Position Input By Displaying (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=72236994

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (14)

Citations (8)

Family Cites Families (49)

• 2019
  • 2019-02-28 US US16/289,259 patent/US10905946B2/en active Active
• 2020
  • 2020-01-28 EP EP20762861.1A patent/EP3914365A4/en active Pending
  • 2020-01-28 CN CN202080016276.2A patent/CN113490532A/zh active Pending
  • 2020-01-28 JP JP2021549743A patent/JP7628498B2/ja active Active
  • 2020-01-28 KR KR1020217027461A patent/KR20210130159A/ko not_active Ceased
  • 2020-01-28 WO PCT/US2020/015435 patent/WO2020176185A1/en not_active Ceased
• 2021
  • 2021-02-01 US US17/164,475 patent/US11806609B2/en active Active

Patent Citations (8)

Also Published As

Similar Documents

Legal Events