TW202345950A - Accessible controller and controller hub - Google Patents

Abstract

Methods, systems, and storage media for providing accessibility to users with limited mobility in a virtual environment are disclosed. Exemplary implementations may: couple a controller to a controller hub; detect controller inputs to the controller along each of the six degrees of freedom; couple accessories to the controller hub; detect hub inputs from the accessories of the controller hub; and cause interactions in the virtual environment based on the controller inputs and the hub inputs.

Description

Accessibility Controllers and Controller Hubs

The present disclosure relates generally to accessibility controllers and controller hubs, and more specifically to providing accessibility to users with limited mobility in virtual environments.

Conventionally, an artificial reality (for example, virtual reality, augmented reality, etc.) system includes a computing device, a head-mounted device, and one or more controllers as input devices for allowing users to move within the artificial environment. . Current controllers use joysticks and handheld devices that allow the user to move in all directions and orientations to provide six degrees of freedom (6DoF) as input to the system. Some users have physical limitations that prevent or limit manipulation of such input devices, making existing controllers inappropriate for such users. Existing controllers often do not provide accessibility for users with physical limitations.

The present disclosure provides systems and methods for accessibility controllers and controller hubs. Allows the user to provide input to a VR system corresponding to six degrees of freedom. The input is typically provided by one or more standard controllers moving around in space, where the position and orientation of the standard controllers correspond to those obtained by using a The VR system anticipates the position and movement of the input controls or multiple external controllers, each of which provides one or more degrees of motion input.

One aspect of the present disclosure relates to a method for providing accessibility to users with limited mobility in a virtual environment. The method may include coupling the controller to a controller hub. The controller is configured for use by users with limited mobility. The controller may include an actuator with six degrees of freedom. Methods may include detecting controller input to the controller along each of the six degrees of freedom. The method may include coupling the accessory to the controller hub. Methods may include detecting hub input from accessories in the controller hub. Methods may include causing interactions in the virtual environment based on controller input and hub input.

Another aspect of the present disclosure relates to a system configured for providing accessibility to users with limited mobility in a virtual environment. The system may include one or more hardware processors configured by machine-readable instructions. The processor can be configured to couple the controller to the controller hub. The controller is configured for use by users with limited mobility. The controller may include an actuator with six degrees of freedom. Actuators may include joysticks, knobs, and sensors that sense motion input along each of the six degrees of freedom. The processor can be configured to detect controller input to the controller along each of the six degrees of freedom. The processor can be configured to couple accessories to the controller hub. The processor can be configured to detect hub input from accessories in the controller hub. The processor can be configured to cause interaction in the virtual environment based on controller input and hub input.

Yet another aspect of the present disclosure relates to a non-transitory computer-readable storage medium embodying instructions executable by one or more processors for providing services to a computer having restricted mobility in a virtual environment. User-provided accessibility methods. The method may include coupling the controller to a controller hub. The controller is configured for use by users with limited mobility. The controller may include an actuator with six degrees of freedom. Actuators may include joysticks, knobs, and sensors that sense motion input along each of the six degrees of freedom. Controllers may include sensors for up/down, back/forward, up/down, roll, yaw and/or pitch. Methods may include detecting controller input to the controller along each of the six degrees of freedom. The method may include coupling the accessory to the controller hub. Accessories can include buttons, triggers, sensors and additional controller functionality. Methods may include detecting hub input from accessories in the controller hub. Methods may include causing interactions in the virtual environment based on controller input and hub input.

Yet another aspect of the present disclosure relates to a system configured for providing accessibility to users with limited mobility in a virtual environment. The system may include means for coupling the controller to the controller hub. The controller is configured for use by users with limited mobility. The controller may include an actuator with six degrees of freedom. The system may include means for detecting controller input to the controller along each of the six degrees of freedom. The system may include means for coupling accessories to the controller hub. The system may include means for detecting hub input from accessories in the controller hub. The system may include components for causing interactions in the virtual environment based on controller input and hub input.

In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to one of ordinary skill in the art that specific examples of the present disclosure may be practiced without some of these specific details. In other instances, well-known structures and techniques have not been shown in detail so as not to obscure the present disclosure.

Existing controllers do not provide input devices that allow users with limited mobility to interact with artificial reality systems. In addition, existing controllers generally do not accept third-party input devices that provide alternative mechanisms to artificial reality systems. These limitations do not provide an accessible controller for users with limited mobility.

The present disclosure provides systems and methods for accessibility controllers and controller hubs. A user with limited movement is allowed to interact with the VR system to provide input to the VR system in six degrees of freedom without having to move her or her arms and hands in six degrees of freedom. Additionally, users are allowed to attach third-party input devices to the VR system that provide alternative mechanisms.

By providing an accessible controller device and a controller hub that accepts input from a user from a variety of different input devices that are converted into the artificial reality system generated by the existing controller Data Input,The implementation described in this article addresses the,foregoing shortcomings and other shortcomings. Acceptance of several input devices allows the user to select an input device appropriate to the user's mobility constraints while simultaneously supplying input commands to the artificial reality system consistent with other controller devices.

Specific examples of the disclosed technology may include or be implemented in conjunction with artificial reality systems. Artificial reality, extended reality or additional reality (collectively, "XR") is a form of reality that has been adapted in some way before being presented to the user. It may include, for example, virtual reality (VR) ), augmented reality (AR), mixed reality (MR), compound reality or some combination and/or derivative thereof. Artificial reality content may include fully generated content, or generated content combined with captured content (eg, real-world photos). Artificial reality content may include video, audio, haptic feedback, or some combination thereof, any of which may be presented in a single channel or in multiple channels (such as stereoscopic video that produces a three-dimensional effect on the viewer). In addition, in some specific examples, artificial reality may be related to, for example, applications, products, products used to generate content in artificial reality, and/or used in artificial reality (e.g., performing activities in artificial reality). accessories, services, or some combination thereof. Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs) connected to host computer systems, stand-alone HMDs, mobile devices or computing systems, "cave" environments, or other projection systems , or any other hardware platform capable of delivering artificial reality content to one or more viewers.

As used herein, "virtual reality" or "VR" refers to an immersive experience in which a user's visual input is controlled by a computing system. "Augmented reality" or "AR" refers to a system in which a user views real-world images after they have been passed through a computing system. For example, a tablet with a camera on the back can capture images of the real world and then display the images on a screen on the side of the tablet opposite the camera. The tablet can process and adjust or "augment" the images as they pass through the system, such as by adding virtual objects. "Mixed reality" or "MR" refers to a system in which the light entering the user's eyes is partly generated by the computing system and partly constitutes light reflected from objects in the real world. For example, an MR headset may be shaped as a pair of glasses with a transparent display that allows light from the real world to pass through while emitting light from a projector in the MR headset, thereby allowing the MR headset to The device presents virtual objects that are intermingled with real objects that the user can view. As used herein, "artificial reality", "additional reality" or "XR" refers to any of VR, AR, MR or any combination or hybrid thereof.

Several implementations are discussed in more detail below with reference to the accompanying drawings. 1 is a block diagram illustrating an overview of a device upon which some implementations of the disclosed technology may operate. Devices may include hardware components of computing system 100 that can create, manage, and provide interaction patterns for artificial reality collaborative work environments. In various implementations, computing system 100 may include a single computing device 103 or multiple computing devices (eg, computing device 101 , computing device 102 , and computing device 103 ) that communicate via wired or wireless channels to distribute processing and share input data. In some implementations, computing system 100 may include a stand-alone headset capable of providing a user with a computer-generated or augmented experience without the need for external processing or sensors. In other implementations, computing system 100 may include multiple computing devices, such as headsets, and core processing components (such as consoles, mobile devices, or server systems), where some processing operations are performed on the headset devices and other processing operations are performed on the headset devices. Operations are offloaded to core processing components. An example headset is described below with respect to Figures 2A and 2B. In some implementations, location and environmental data may be collected solely by sensors incorporated into the headset device, while in other implementations, one or more of the non-headset computing devices may include a trackable environment. or sensor components for location data.

Computing system 100 may include one or more processors 110 (eg, central processing units (CPU), graphical processing units (GPU), holographic processing units (HPU), etc.) . Processor 110 may be a single processing unit or multiple processing units in a device or distributed across multiple devices (eg, distributed across two or more of computing devices 101 - 103 ).

Computing system 100 may include one or more input devices 120 that provide input to processor 110 and inform its actions. Actions may be mediated by a hardware controller that interprets signals received from the input devices and communicates the information to processor 110 using a communications protocol. Each input device 120 may include, for example, a mouse, keyboard, touch screen, trackpad, wearable input device (such as tactile gloves, bracelets, rings, earrings, necklaces, watches, etc.), cameras (or other light-based input devices). input device (such as an infrared sensor), microphone or other user input device.

Processor 110 may be coupled to other hardware devices, such as by using internal or external buses, such as PCI buses, SCSI buses, or wireless connections. Processor 110 may communicate with a hardware controller for a device, such as for display 130 . The display 130 can be used to display text and graphics. In some implementations, display 130 includes the input device as part of the display, such as when the input device is a touch screen or is equipped with an eye direction monitoring system. In some implementations, the display is separate from the input device. Examples of display devices are LCD display screens, LED display screens, projection, holographic or augmented reality displays (such as heads-up displays or head-mounted devices), etc. Other I/O devices 140 may also be coupled to the processor, such as network chips or cards, video chips or cards, audio chips or cards, USB, FireWire or other external devices, cameras, printers, speakers, CD-ROMs Drives, DVD drives, disk drives, etc.

Computing system 100 may include communication devices capable of communicating wirelessly or wiredly with other local computing devices or network nodes. The communication device may communicate with another device or server over the network using, for example, TCP/IP protocols. Computing system 100 may utilize communication devices to distribute operations across multiple network devices.

Processor 110 may access memory 150 , which may be included on one of the computing devices of computing system 100 or may be distributed across one of a plurality of computing devices of computing system 100 or other external devices. Memory includes one or more hardware devices for volatile or non-volatile storage, and may include both read-only memory and writable memory. For example, memory may include random access memory (RAM), various cache memories, CPU registers, read-only memory (ROM), and writable non-volatile memory. One or more of memories (such as flash memory, hard drives, floppy disks, CDs, DVDs, magnetic storage devices, tape drives, etc.). Memory is not a propagation signal separate from the underlying hardware; memory is therefore non-transitory. Memory 150 may include program memory 160 that stores programs and software, such as operating system 162, XR operating system 164, and other applications 166. Memory 150 may also include data memory 170 , which may include information provided to program memory 160 or any component of computing system 100 .

Some implementations are operable with a variety of other computing system environments or configurations. Examples of computing systems, environments, and/or configurations that may be suitable for use with the technology include, but are not limited to, XR headsets, personal computers, server computers, handheld or laptop devices, cellular phones, wearable Electronic devices, game consoles, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, microcomputers, mainframe computers, including the above systems or A distributed computing environment of any of the devices or the like.

FIG. 2A is a line diagram of a virtual reality head-mounted display (HMD) 200 according to one or more embodiments. The HMD 200 includes a front rigid body 205 and a belt 210 . The front rigid body 205 includes one or more electronic display elements of the electronic display 245 , an inertial motion unit (IMU) 215 , one or more position sensors 220 , a positioner 225 and one or more computing units 230 . The position sensor 220, IMU 215, and computing unit 230 may be internal to the HMD 200 and may not be visible to the user. In various implementations, the IMU 215, the position sensor 220, and the positioner 225 can track the HMD 200 in the real world and in the virtual environment in three degrees of freedom (3DoF) or six degrees of freedom (6DoF). Movement and location. For example, the locator 225 may emit an infrared beam that creates light spots on real objects around the HMD 200 . As another example, IMU 215 may include, for example, one or more accelerometers, gyroscopes, magnetometers, other non-camera based position, force or orientation sensors, or combinations thereof. One or more cameras (not shown in the figure) integrated with the HMD 200 can detect light points. The computing unit 230 in the HMD 200 can use the detected light points to extrapolate the position and movement of the HMD 200, and identify the shapes and positions of real objects around the HMD 200.

An electronic display 245 may be integrated with the front rigid body 205 and may provide image light to the user as specified by the computing unit 230 . In various embodiments, electronic display 245 may be a single electronic display or multiple electronic displays (eg, one for each user's eye). Examples of the electronic display 245 include: liquid crystal display (LCD), organic light-emitting diode (OLED) display, active-matrix organic light-emitting diode display (active-matrix organic light-emitting diode) ; AMOLED), a display including one or more quantum dot light-emitting diodes (QOLED) sub-pixels, a projector unit (e.g., micro-LED, laser, etc.), some other display or other a certain combination.

In some implementations, HMD 200 may be coupled to a core processing component, such as a personal computer (PC) (not shown) and/or one or more external sensors (not shown). External sensors may monitor HMD 200 (eg, via light emitted from HMD 200), and in combination with the output from IMU 215 and position sensor 220, the PC may use HMD 200 to determine the position and movement of HMD 200.

FIG. 2B is a line diagram of a mixed reality HMD system 250 including a mixed reality HMD 252 and a core processing component 254. Mixed reality HMD 252 and core processing component 254 may communicate via a wireless connection (eg, a 60 GHz link) as indicated by link 256 . In other implementations, the mixed reality system 250 includes only the headset, without external computing devices, or other wired or wireless connections between the mixed reality HMD 252 and the core processing component 254 . The mixed reality HMD 252 includes a transparent display 258 and a frame 260 . The frame 260 can accommodate various electronic components (not shown in the figure), such as light projectors (eg, lasers, LEDs, etc.), cameras, eye tracking sensors, MEMS components, network connection components, etc.

A projector may be coupled to the transparent display 258, such as via optical elements, to display media to the user. These optical components may include one or more waveguide assemblies, reflectors, lenses, mirrors, collimators, gratings, etc., for directing light from the projector to the user's eyes. Image data may be transmitted from core processing component 254 to HMD 252 via link 256 . The controller in the HMD 252 can convert image data into light pulses from the projector, and these light pulses can be transmitted to the user's eyes as output light through optical elements. The output light may be mixed with the light passing through the display 258, allowing the output light to render the virtual object as if it existed in the real world.

Similar to the HMD 200, the HMD system 250 may also include a motion and position tracking unit, a camera, a light source, etc., which allows the HMD system 250 to track itself, track the user's parts (such as hands, feet, head, or other parts), for example, in 3DoF or 6DoF. body parts), map virtual objects to appear stationary when the HMD 252 moves, and enable virtual objects to react to gestures and other real-world objects.

The accessibility controller includes an accessibility controller hub with a plurality of input ports that support the connection of external controller devices to the VR system to provide motion input in six degrees of freedom for users, including but not Restricted to persons with: weakness and limitations (temperature and electric heat) of muscle control (involuntary movement, lack of coordination, paralysis), sensory limitations, joint disorders (such as arthritis), pain that impedes movement, missing limbs, and situational limitations . The use of such external controllers allows the selection of one or more controller devices to overcome the user's specific limitations. The Accessibility Controller Hub receives input data generated by an external controller for conversion into input data generated by a standard controller device. Applications running on VR systems that support receiving input from the controller hub will allow users to attach all external devices supported by the controller hub without requiring application developers to integrate numerous external controls within each application. device.

3A and 3B illustrate isometric views of handheld controller 300 of example computing system 100 in accordance with one or more implementations. In some embodiments, controller 300 includes a first user input key 320A, and may include one or more additional user input keys 320B, 320C. A user input key is a button, knob, switch, thumbstick, directional pad, or anything that a user presses or manipulates in some other way to perform a specific action in a virtual reality system (e.g., during a game) Other such parts. In some embodiments, first user input key 320A may be selected from the group consisting of: a rocker, a button, a trigger, and a directional pad. In some specific examples, the first user input key 320A may be a button selected from the group consisting of: an A or X button, a B or Y button, a start button, a back button, a forward button, and a home button. The A or B button may correspond to a selection action between at least two options presented to the user in the gaming system. The X or Y buttons may correspond to negative or positive decisions resulting from the user specifying how the avatar will progress through the game. X can correspond to the action of "no" or "end", and Y can correspond to "yes" or "go/continue". The start button may be a button activated by the user to begin a virtual reality (eg, game) experience, and the push-back and forward buttons may indicate the direction in which the user requests the image entity to move. The home button may be a button activated to return the gaming experience to the main menu or to start the game or activity from the beginning.

In some embodiments, handheld controller 300 further includes user input surface 310 including first user input keys 320A. The user input surface 310 includes a plurality of user input keys 320A, 320B, and 320C. Alternatively, the user input surface includes a single user input key. In the examples of FIGS. 3A and 3B , the user input keys include a joystick 320A and buttons 320B and 320C. Thus, user input surface 310 is a surface on controller 300 in which a user initiates one or more actions corresponding to the user's request to perform in artificial reality system 100 (eg, computing system 100 ). Input keys (for example, by pressing a button or pushing a knob) deliver input.

Each of user input keys 320A, 320B, and 320C is configured to communicate with computing system 100 to convert a user's operation of the corresponding user input key into an image entity displayed on the HMD of virtual reality system 100 corresponding action.

In some embodiments, user input surface 310 includes a plurality of user input keys, including first user input key 320A, and respective user input keys 320B of the plurality of user input keys. , 320C is selected from the group consisting of: joysticks, buttons, triggers and directional pads.

In some embodiments, the home button is positioned further away from another user input key. This configuration will allow the most commonly used user input keys (eg, directional pads to indicate the direction of movement of the image entity, such as up, down, left, and right) to be placed closer to the fingers and thumb. This configuration provides the advantage that the user should require less range to press the more frequently used user input keys, thus mitigating the potential for ergonomic pain associated with overreaching and overstretching the fingers.

In some embodiments, at least a portion of the user input surface 310 is a touch-sensitive surface that is divided into a plurality of segments. Each section corresponds to a respective user input key of a plurality of user input keys. In this configuration, at least one touch sensor is positioned on the bottom surface of user input surface 310 to detect touches on corresponding segments. When the user's touch on the key is detected by the sensor, the action associated with the touch and the corresponding user input key is converted into an action of the individual image in the virtual reality system.

In some embodiments, clamp 330 is coupled to user input surface 310 . The clamp 330 is a protruding structure of the controller 300, and the user grasps the protruding structure in one hand to hold the controller 300. This configuration allows the user to grasp the controller 300 between the palm and fingers (eg, three or fewer fingers) while releasing the thumb and, in some embodiments, another finger (eg, the middle finger) to Used to operate user input keys 320A, 320B and 320C. In some embodiments, the middle finger is released to operate trigger 390 that is at least partially mounted on clamp 330, as will be described below.

In some embodiments, clamp 330 is a separate portion of controller 300 that is removably coupled to user input surface 310 and/or cage 340 . Clamp 330 and the user input surface may be coupled by methods appropriate to the materials of their construction. For example, the clamp and user input surface 310 may be formed from hard plastic and may be coupled to each other by ultrasonic welding. Alternatively, clamp 330 and user input surface 310 may be coupled to each other by a fastening mechanism such as screws or bolts, or may be threadably engaged with each other.

In some embodiments, clamp 330 is integrally formed with user input surface 310 and/or cage 340 as one component (eg, which may be formed by molding).

In some embodiments, the clamp 330 is tilted at a predetermined angle relative to the user input surface 310 (eg, a plane passing through the user input surface or a portion thereof) to facilitate at least one of retaining the clamp in the thumb and operating the thumb with the thumb. Provide a comfortable (i.e., optimal) ergonomic balance for the user between user input keys.

In some embodiments, cage 340 is coupled to user input surface 310 . Cage 340 (which may also be referred to as a tracking cage) has an exterior surface 145 on which a plurality of illumination sources 350 are positioned.

In the example of FIGS. 3A and 3B , the user input surface 310 faces outward relative to the cage 340 . Alternatively, user input surface 310 may face inward relative to cage 340. For example, in some embodiments, user input surface 310 forms an interior front surface of cage 340 or is in contact with an interior surface of cage 340 .

In some embodiments, clamp 330 and/or cage 340 may be formed from an overmolded rubber material (eg, to provide a surface that provides sufficient friction with the user's palm and thus improves the clamp). In some embodiments, clamp 330 and/or cage 340 may be formed from hard plastic, including but not limited to high density polyethylene, which provides increased structural rigidity. Additionally, any other suitable material may be used.

In some embodiments, cage 340 may be removably coupled to at least one of user input surface 310 and clamp 330 . Cage 340 may be slidably coupled to user input surface 310 via protrusions slidably across the width of each end portion of cage 340 and positioned on the outer circumference of user input surface 310 The corresponding groove meshes. Cage 340 may be coupled to clamp 330 via a fastening mechanism such as bolts, screws, or the like. The removable configuration of cage 340 to clamp 330 or user input surface 310 provides the advantage of detaching the aforementioned components for calibration as needed. Removable coupling of components also allows for separate and potentially cheaper manufacturing processes of the components. Additionally, removable coupling of cage 340 to at least one of user input surface 310 and clamp 330 allows cage 340 to detach after controller 300 is dropped, thereby reducing the need to replace the entire unit after damage. Needed, but instead focused on fixing/replacing individually damaged components.

In some embodiments, as illustrated in FIGS. 3A and 3B , controller 300 may further include structural webs 395 coupling cage 340 to user input surface 310 . The larger structural web 395 further provides structural rigidity to the coupling between the cage 340 and the user input surface 310 to mitigate damage and separation of these components after a user drops the controller 300.

In some embodiments, trigger 390 is at least partially mounted on structural web 395 . That is, trigger 390 may be mounted between structural web 395 and clamp 330 . This configuration results in the advantage that when the clamp 330 is held in the neutral position, the trigger is positioned adjacent to the position of the user's finger (eg, the middle finger). In some embodiments, the trigger can be pushed and pulled by the middle finger, thereby providing increased control when manipulating the trigger to achieve the desired action. Trigger 390 is an example of a user input key.

Some implementations may include cameras, such as any type or brand of commercially available camera products, such as web cameras, IR capable cameras, and/or USB cameras, etc. In some embodiments, the camera may be electrically connected to a power source, which may or may not be the same power source that provides power to the HMD. The camera and HMD can be wireless; therefore, the power source can be one or more batteries.

In some embodiments, illumination source 350 is positioned to allow the camera to detect at least six degrees of freedom of controller 300 . The six degrees of freedom are the position of the controller 300 in the x, y, and z coordinates of space and the orientation of the controller 300 including the yaw, pitch, and roll of the controller 300. The six degrees of freedom detected by the camera can be used to determine the movement of the user of controller 300 and model these movements in virtual reality.

In some specific examples, the illumination source 350 is a light-emitting diode (LED). In some specific examples, the illumination source 350 is an infrared (IR) LED. The camera can be configured to detect IR light emitted by the IR LED on the controller 300 and record the signal sensed from the emitted light to determine the position and orientation of the controller 300 .

In some embodiments, illumination sources 350 may be positioned on the outward facing surface 345 of the controller 300 in any suitable pattern, order, or array. In some embodiments, outward surface 345 is the exterior surface of cage 340 coupled to clamp 330 . Illumination source 350 may be fixedly or removably positioned on cage 340 and thus coupled to cage 340 by any suitable method. For example, the illumination source 350 may be mounted on or embedded within the exterior surface 345 of the cage 340 . Alternatively, the illumination source 350 may be on a sleeve surrounding the cage 340 and effectively forming the outer surface 345 of the cage 340 . Although the illumination source 350 is described as being positioned on the exterior surface 345 of the cage 340 , it may additionally or alternatively be coupled to any other surface on the cage 340 and/or the remainder of the controller 300 .

Illumination source 350 may be electrically connected to a power source, which may or may not be the same power source that provides power to controller 300 . Controller 300 may be wireless; therefore, the power source may be one or more batteries. The illumination source 350 may be housed in a diffuse case that includes a current limiting resistor to keep the current from the power supply to the illumination source 350 below the illumination source 350 maximum current level in order to ensure the maximum life of the illumination source 350. Illumination source 350 can be activated when the appropriate voltage is applied. With the illumination source 350 positioned on the controller 300 in an area detectable by the camera, the movement of the light generated by the illumination source 350 detected by the camera is used as an indication of the position and movement of the controller 300 . In this way, the movement of the controller is tracked by the camera, allowing corresponding VR hand movements to be displayed on the HMD. For example, when the user punches while playing a boxing game, the movement of the illumination source 350 can be detected corresponding to the punching method and used to individually model the image displayed on the HMD in the computing system 100 Movement of the user. In this way, the present invention provides the advantage of enhancing the user's virtual reality experience by adding additional elements of hand movement to the image entities.

Figure 4A illustrates an example accessibility controller in accordance with one or more implementations. This specific example of an accessibility controller includes a rotating knob 401 that accepts movement in six degrees of freedom, which movement allows the user to mimic the 3D movement of the user's head when interacting with the VR system. In some implementations, the accessibility controller may be configured to control the virtual position and/or orientation of an HMD (eg, HMD 200 and/or HMD 252). In some implementations, an accessibility controller may be configured to control the virtual position and/or orientation of one or more other controllers (eg, standard and/or accessibility controllers). The rotating knob can demonstrate inputs representing each of the six degrees of freedom including forward lateral motion 411 , orthogonal lateral motion perpendicular to the forward lateral motion 412 , vertical press or downward motion 413 , forward rotation movement 414, orthogonal rotation movement 415 and horizontal rotation movement 416.

Figure 4B illustrates example motion control operations of the accessibility controller of Figure 4A, according to one or more implementations. The knob 401 can have forward lateral movement 411 independent of other movements, orthogonal lateral movement 412 perpendicular to the forward lateral movement, vertical pressing or downward movement 413, forward rotational movement 414, orthogonal rotational movement 415, and Any of the horizontal rotational motions 416 move.

The forward lateral movement 411 is caused by the vertical axis of the knob 401 moving in a lateral movement along the first axis. In this particular movement, the vertical axis remains in its current position while the entire knob 401 moves along the forward and rearward axes. In the preferred embodiment, forward lateral movement 411 corresponds to the X-axis as described with reference to FIG. 8 .

Orthogonal lateral motion 412 perpendicular to the forward lateral motion is caused by the vertical axis of knob 401 moving in a lateral motion along the second axis. The second axis is perpendicular to the first axis. In this particular movement, the vertical axis remains in its current position while the entire knob 401 moves along the forward and rearward axes. In the preferred embodiment, forward lateral movement 411 corresponds to the Z-axis.

The vertical press or downward movement 413 is caused by the knob 401 moving in upward and downward movements along the third axis. The third axis is perpendicular to both the first axis and the second axis. During this particular movement, the first and second axes remain in their current orientations while the entire knob 401 moves along the upward and downward orientation axes. In the preferred embodiment, forward lateral movement 411 corresponds to the Y-axis.

Forward rotational motion 414 is a motion in which the position of the knob 401 remains stationary on the first two axes, and the position of the knob is tilted forward and backward to produce a rotational motion about the X-axis. This rotational movement corresponds to a rolling rotation about the X-axis.

Orthogonal rotational motion 415 is a motion in which the position of the knob 401 remains stationary on the first two axes, and the position of the knob is tilted forward and backward to produce a rotational motion about the Z-axis. This rotational movement corresponds to a yaw rotation about the X-axis.

Horizontal rotational motion 416 is a motion in which the position of the knob 401 remains stationary on the first two axes, and the position of the knob is tilted forward and backward to produce a rotational motion about the X-axis. This rotational movement corresponds to a pitch rotation about the Z-axis.

Figure 5 illustrates another example accessibility controller in accordance with one or more implementations. Another embodiment of an accessibility controller 500 includes a rocker 501 coupled to a device body 502 that is attached to a controller hub. The joystick 501 moves in two dimensions, which can be converted by the controller hub into two of six degrees of freedom for transmission to the VR system. A plurality of pressable buttons 503 may be positioned around the device body 502 to provide additional user input signals to the VR system. The rocker 501 and the device body 502 can be held in the hand of a single user, while allowing the user to control the rocker 501 and press the button 503 with their thumb or thumb and fingers. A user with limited mobility can hold the accessibility controller 501 to provide two degrees of motion while keeping his or her hands still.

Accessibility controller 500 may be attached to the controller hub using cables or wireless connections, such as IR transceivers, RF transceivers, Bluetooth™ connections, and any other connections and communication channels. The accessibility controller hub can be configured to assign the joystick 501 to any two of the six degrees of freedom as part of its configuration. Similarly, a plurality of pushable buttons 503 can be mapped to provide an input signal recognized by the controller hub as a simple on/off switch. The user can use similar controllers held in the opposite hand to provide two additional degrees of motion input to the VR system.

Figure 6 illustrates yet another example accessibility controller and hub in accordance with one or more implementations. A collection of controller and hub 601 and external controller devices 603-604 are shown in this specific example of accessibility controller and hub 600. Controller and hub 601 includes a plurality of input actuator devices 602 that allow these input devices to be used to provide motion input to the VR system. The plurality of input actuator devices 602 may include joysticks, knobs, and sensors that sense motion input along each of the six degrees of motion, with controls including for up/down, back/forward, up/down. Down, roll, yaw and/or pitch sensors. The set of external controller devices 603-604 may also include joysticks, knobs, and sensors that sense motion input along each of the six degrees of motion, with controls including for up/down, back/forward, Up/down, roll, yaw and/or pitch sensors, each of which may be communicatively coupled to the controller and hub 601. In alternative embodiments, controller hub 601 may not include input actuator devices integrated into the hub body, with all input actuator devices corresponding to external controller devices 603-604.

Figure 7 illustrates yet another example of an accessibility controller and hub in accordance with one or more implementations. Another exemplary embodiment of a controller hub 700 is shown with a plurality of input actuator devices 703a-703c, a plurality of pressable buttons 704a-d, and a four-way switch 705 integrated into the controller body 701. The plurality of input actuator devices 703a-703c may include one or more joysticks, knobs, and sensors that sense motion input along each of the six degrees of motion, with controls including for up/down, reverse /Sensors for forward, up/down, roll, yaw and/or pitch. A plurality of pressable buttons 704a-d can be mapped to act as simple on/off switches to provide input signals recognized by the controller hub. The four-way switch 705 allows the user to send motion input corresponding to two orthogonal axes in a three-dimensional (3D) space.

The controller hub 700 is also shown having a plurality of external controller ports 702a-n utilizing electrical connectors that mate with electrical connectors on the external controller device. When an external controller device is communicatively coupled to one of external controller ports 702a-n, controller hub 700 is configured to accept input generated by a particular external controller. Controller hub 700 can automatically detect the type of external controller coupled to each of external controller ports 702a-n to automatically configure operation of the external controller. Controller hub 700 can be configured to map any of the external controllers and their inputs to various inputs of the VR system.

Figure 8 illustrates six degrees of motion control operation for an accessibility controller in accordance with one or more implementations. The position and orientation of any object in 3D space can be defined using the six-value coordinate system 800. In a VR system, a typical user interacts with the system to provide input using one or more input controllers in their hands. The position of each hand is called a position in 3D space, which can be defined in a six-axis coordinate system, where each axis defines one of the six degrees of freedom.

The six-value coordinate system 800 uses three axes including the X-axis 801, the Y-axis 802, and the Z-axis 803 to define points in 3D space. All objects located at this point in 3D space can also be defined using three angular rotation values about three position axes 801 to 703. These angular rotation values define roll 811 , pitch 812 and yaw 813 rotation angles about corresponding axes 801 to 803 . Roll value 811 defines an angular measurement of rotation about the X-axis 801 . Pitch value 812 defines an angular measurement of rotation about Y-axis 802 . Yaw value 813 defines an angular measurement of rotation about X-axis 803 .

Users typically interact with VR systems by changing the position and orientation of one or more controllers in their hands by changing the position and orientation of the controllers. Position and orientation and/or changes in position and orientation can be interpreted as input by the VR system to anticipate the user's actions in the VR space. These six degrees of freedom combined with one or more input buttons allow the user to provide all the input needed to interact with the VR system.

The disclosed system solves a problem associated with traditional accessibility controllers and controller hub technology associated with computer technology, namely that the controller does not provide an input device that allows a user with limited mobility to interact with the VR system, and accept Third-party input devices that provide alternative mechanisms to VR systems. The disclosed system solves this technical problem by providing a solution also originating in computer technology, namely by providing accessibility to users with limited mobility in virtual environments. The disclosed technology further provides improvements in the functionality of the computer itself due to improved processing and efficiency in accessibility controllers and controller hubs.

Figure 9 illustrates a system 900 configured for accessibility controllers and controller hubs in accordance with certain aspects of the present disclosure. In some implementations, system 900 may include one or more computing platforms 902. Computing platform 902 may be configured to communicate with one or more remote platforms 904 according to a client/server architecture, a peer-to-peer architecture, and/or other architectures. Remote platform 904 may be configured to communicate with other remote platforms via computing platform 902 and/or according to a client/server architecture, a peer-to-peer architecture, and/or other architectures. Users can access system 900 via remote platform 904.

Computing platform 902 may be configured by machine-readable instructions 906 . Machine-readable instructions 906 may include one or more instruction modules. Instruction modules may include computer program modules. The command module may include a controller coupling module 908, a controller input detection module 910, a controller hub coupling module 912, a hub input detection module 914, an interaction causing module 916, and/or other command modules. one or more of the group.

Controller coupling module 908 may be configured to couple a controller to the controller hub. By way of non-limiting example, the controller may include sensors for up/down, back/forward, up/down, roll, yaw and/or pitch. The controller hub may include directional pad/directional pad functionality. The controller hub may include a plurality of controller ports for electrically coupling one or more external controllers to the controller hub. By way of non-limiting example, each of the plurality of controller ports may include a USB-A connection, a micro-USB connection, a mini-USB connection, a USB-C connection, and a Thunderbolt connection. In some implementations, the controller can be configured for use via a head mounted display without a controller hub, where the head mounted display is configured to convert the six degree of freedom controls into a head mounted display or control Six degrees of freedom motion of one or both of the devices.

By way of non-limiting example, external controls may include joysticks, knobs, directional pads/directional pad functionality, thumb wheels, track pads, and depressible actuator buttons. Directional pads/directional pads feature functionality that provides 2 degrees of motion input. By way of non-limiting example, controllers, controller hubs, and external controllers can provide haptic feedback. Haptic feedback provides motion-based feedback to the user by providing detectable motion with the controller, and the detectable motion indicates feedback information is provided to the user by the VR system 100 .

By way of non-limiting example, the controller, controller hub, and external controller may be programmed as a left controller and a right controller. The left and right controllers may have the same number and location of input devices, including joysticks, knobs, directional pad/directional pad functionality, thumbwheels, track pads, and pushable actuators button. However, when in use, the input device on the left controller can be programmed to provide input data to the VR system that is different from the input data provided by the right controller. The controller is configured for use by a user with limited mobility, typically a user with limited ability to move the actuator by using the VR system to determine input data values. The controller may include an actuator with six degrees of motion.

The actuator may include a joystick. The actuator may include sensors that sense motion input along each of the six degrees of motion, or changes in the position or orientation of the actuator. The actuator may include a four-way switch providing two degrees of motion input.

Controller input detection module 910 may be configured to detect controller input to the controller along each of the six degrees of motion.

Controller hub coupling module 912 may be configured to couple accessories to the controller hub. Joysticks, knobs, directional pads/directional pad functionality, thumb wheels, track pads, and pushable actuator buttons from third-party manufacturers for coupling using the Controller Hub Coupling Module 912 Examples of accessory types to controller hubs. By way of non-limiting example, accessories may include buttons, triggers, sensors, and additional controller functionality. Accessories and external controllers may be optional to provide motion input in each of the six degrees of freedom. Accessories and external controls can include light-cap touch buttons.

The hub input detection module 914 may be configured to detect hub input from accessories of the controller hub. These hub inputs correspond to input values received by the hub input detection module 914 that are converted into inputs to the VR system provided by the standard controller.

The interaction causing module 916 can be configured to cause interactions in the virtual environment based on controller input and hub input. The interaction elicitation module 916 converts input values received from external devices into inputs to the VR system provided by standard controllers. An input device that provides a motion input is assigned only to its corresponding axis in the six-degree coordinate system. An input device that provides two or more input degrees of motion converts data received from the input device into axis input data corresponding to the amount of input motion contained in the input motion of the input device that is assigned to it The input device is represented for each axis in the six-degree coordinate system. Using such conversion operations, input motion corresponding to input motion data generated by a standard controller can be provided to the VR system, and can be provided by an external controller.

In some implementations, the actuator may include a knob. In some implementations, the actuator is pushable. In some implementations, the knob can provide six degrees of freedom input. In some implementations, by way of non-limiting example, each of the six degrees of freedom inputs may include forward lateral motion, orthogonal lateral motion perpendicular to the forward lateral motion, vertical press or downward motion, forward lateral motion, Forward rotational motion, orthogonal rotational motion and horizontal rotational motion. In some implementations, the joystick can provide two degrees of motion input.

In some implementations, computing platform 902, remote platform 904, and/or external resources 918 may be operatively linked via one or more electronic communication links. For example, such electronic communication links may be established, at least in part, via a network such as the Internet and/or other networks. It should be understood that this is not intended to be limiting, and the scope of the present disclosure includes implementations in which computing platform 902, remote platform 904, and/or external resources 918 may be operably linked via some other communication medium.

A given remote platform 904 may include one or more processors configured to execute computer program modules. Computer programming modules may be configured to enable experts or users associated with a given remote platform 904 to interface with the system 900 and/or external resources 918 and/or to provide information attributed herein to the remote platform 904 other functionality. By way of non-limiting example, a given remote platform 904 and/or a given computing platform 902 may include a server, desktop computer, laptop computer, handheld computer, tablet computing platform, mini-notebook computer ( NetBook), smartphones, game consoles, and/or other computing platforms.

External resources 918 may include information sources external to system 900, external entities participating in system 900, and/or other resources. In some implementations, some or all of the functionality attributed herein to external resources 918 may be provided by resources included in system 900 .

Computing platform 902 may include electronic storage 920, one or more processors 922, and/or other components. Computing platform 902 may include communication pipes or ports to enable the exchange of information with networks and/or other computing platforms. The illustration of computing platform 902 in Figure 9 is not intended to be limiting. Computing platform 902 may include a plurality of hardware, software, and/or firmware components that operate together to provide the functionality attributed herein to computing platform 902. For example, computing platform 902 may be implemented in the cloud together as a computing platform for which computing platform 902 operates.

Electronic storage 920 may include non-transitory storage media that stores information electronically. Electronic storage media for electronic storage 920 may include system storage provided integrally with computing platform 902 (i.e., substantially non-removable), and/or via, for example, a port (e.g., USB port, FireWire port, etc.) or drive ( For example, a disk drive, etc.) is removably connected to one or both of the removable storage of computing platform 902 . The electronic storage 920 may include one or more of the following: optically readable storage media (eg, optical discs, etc.), magnetically readable storage media (eg, magnetic tapes, magnetic hard drives, floppy disk drives, etc.), based on Charge storage media (such as EEPROM, RAM, etc.), solid-state storage media (such as flash drives, etc.) and/or other electronically readable storage media. Electronic storage 920 may include one or more virtual storage resources (eg, cloud storage, virtual private network, and/or other virtual storage resources). Electronic storage 920 may store software algorithms, information determined by processor 922, information received from computing platform 902, information received from remote platform 904, and/or enable computing platform 902 as described herein. Other information that works.

Processor 922 may be configured to provide information processing capabilities in computing platform 902 . Thus, processor 922 may include a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for processing information electronically. one or more. Although processor 922 is shown as a single entity in Figure 9, this is for illustrative purposes only. In some implementations, processor 922 may include a plurality of processing units. These processing units may be physically located within the same device, or processor 922 may represent the processing functionality of multiple devices operating in coordination. Processor 922 may be configured to execute modules 908, 910, 912, 914, and/or 916 and/or other modules. Processor 922 may be configured to execute modules 908, 910, 912, 914, and/or 916 and/or other modules by: software; hardware; firmware; software, hardware, and/or firmware Some combination of; and/or other mechanisms for configuring processing capabilities on processor 922. As used herein, the term "module" may refer to any component or collection of components that performs the functionality attributed to a module. This may include one or more physical processors, processor-readable instructions, circuitry, hardware, storage media, or any other component during execution of the processor-readable instructions.

It should be understood that although modules 908, 910, 912, 914, and/or 916 are illustrated in FIG. 9 as being implemented within a single processing unit, in implementations in which processor 922 includes multiple processing units, modules 908, 910, One or more of 912, 914, and/or 916 may be implemented remotely from other modules. The functionality provided by the various modules 908, 910, 912, 914, and/or 916 described below are described for purposes of illustration and are not intended to be limiting. This is because the modules 908, 910, 912 Any of , 914 and/or 916 may provide more or less functionality than described. For example, one or more of modules 908, 910, 912, 914, and/or 916 may be eliminated, and some or all of their functionality may be provided by modules 908, 910, 912, 914, and/or 916. provided by others. As another example, processor 922 may be configured to execute one or more additional modules, which may perform one of the modules 908, 910, 912, 914, and/or 916 below. some or all of its functionality.

In certain embodiments, one or more objects of a computing system (eg, content or other types of objects) may be associated with one or more privacy settings. One or more objects may be stored on or otherwise associated with any suitable computing system or application, such as a social network connection system, a client system, a third party system, a social network connection application, messaging application, photo sharing application or any other suitable computing system or application. Although the examples discussed herein are in the context of online social networks, these privacy settings may be applied to any other suitable computing system. An object's privacy settings (or "access settings") may be stored in any suitable manner, such as associated with the object, in an index on the authorization server, in another suitable manner, or in any suitable combination thereof. An object's privacy settings may specify how the object (or certain information associated with the object) may be accessed, stored, or otherwise used on the online social network (e.g., viewed, shared, modified, copied, executed, presented, or recognized) . When an object's privacy settings allow a specific user or other entity to access that object, the object is described as "visible" to that user or other entity. By way of example, and not by way of limitation, users of an online social network may specify privacy settings for a user profile page that identifies the set of users who have access to work experience information on the user profile page, This prevents other users from accessing that information.

In certain embodiments, privacy settings for an object may specify a "blacklist" of users or other entities that should not be allowed to access certain information associated with the object. In certain embodiments, the blacklist may include third party entities. The blacklist can specify one or more users or entities for which the object is not visible. As an example, and not as a limitation, a user may specify a set of users that may not have access to the photo album associated with the user, thus excluding those users from accessing the photo album (while also potentially allowing those users who are not in the specified user set) Some users have access to photo albums). In certain embodiments, privacy settings may be associated with specific social graph elements. Privacy settings for social graph elements such as nodes or edges can specify how the social graph element, information associated with the social graph element, or objects associated with the social graph element can be accessed using the online social network. By way of example, and not by way of limitation, a particular concept node corresponding to a particular photo may have privacy settings that specify that the photo may be accessed only by users tagged in the photo and friends of users tagged in the photo. In certain embodiments, privacy settings may allow users to opt in or out of having their content, information or actions stored/recorded by social network connected systems or shared with other systems (e.g., third-party systems). Although this disclosure describes the use of particular privacy settings in a specific manner, this disclosure covers the use of any appropriate privacy setting in any appropriate manner.

In certain embodiments, privacy settings may be based on one or more nodes or edges of the social graph. Privacy settings may be specified for one or more edges or edge types of the social graph, or with respect to one or more nodes or node types of the social graph. Privacy settings applied to a specific edge connecting two nodes control whether the relationship between the two entities corresponding to the node is visible to other users of the online social network. Similarly, privacy settings applied to a specific node can control whether users or concepts corresponding to the node are visible to other users of the online social network. By way of example and not limitation, the first user may share the object to the social network connection system. The object may be associated with a concept node connected by an edge to a user node of the first user. The first user may specify a privacy setting that applies to a specific edge connected to a concept node of an object, or may specify a privacy setting that applies to all edges connected to a concept node. As another example, and not by way of limitation, the first user may share an object collection of a particular object type (eg, an image collection). The first user may specify privacy settings for all objects of that particular object type associated with the first user as having a particular privacy setting (e.g., specifying that all images posted by the first user are only visible to the first user Visible to friends and/or users tagged in the image).

In certain embodiments, the social network connection system may present a "Privacy Wizard" to the first user (e.g., within a web page, module, one or more dialog boxes, or any other suitable interface) to assist the first user. or specify one or more privacy settings. The Privacy Wizard may display instructions, appropriate privacy-related information, current privacy settings, one or more input fields for accepting one or more inputs from the first user to change or confirm a specified privacy setting, or any of its Suitable combination. In certain embodiments, the social network connection system may provide "dashboard" functionality to the first user that may display to the first user the first user's current privacy settings. The dashboard functionality may be displayed to the first user at any appropriate time (eg, following input from the first user summoning the dashboard functionality, following the occurrence of a specific event or triggering action). Dashboard functionality may allow the first user to modify one or more of the first user's current privacy settings at any time in any suitable manner (eg, redirecting the first user to the privacy wizard).

Privacy settings associated with an object may specify any appropriate level of detail for allowing or denying access. By way of example, and not by way of limitation, access or denial of access may be to specific users (e.g., just me, my roommates, my boss), users within a specific degree of separation (e.g., friends, friends of friends) , user groups (e.g., Game Club, My Family), user networks (e.g., employees of a specific employer, students or alumni of a specific university), all users ("Public"), non-users (" "Private"), users of third-party systems, specific applications (e.g., third-party applications, external websites), other appropriate entities, or any appropriate combination thereof. Although this disclosure describes a specific granularity at which access is granted or denied, this disclosure encompasses any suitable granularity at which access is granted or denied.

In certain embodiments, one or more servers may be authorization/privacy servers for enforcing privacy settings. In response to a request from a user (or other entity) for a particular object stored in the data storage, the social network connection system may send a request to the data storage for the object. If the authorization server determines that the user is authorized to access the object based on the privacy settings associated with the object, the request may identify the user associated with the request and the object may be sent only to that user (or client system). If the requesting user is not authorized to access the object, the authorization server may prevent the requested object from being retrieved from the data storage or may prevent the requested object from being sent to the user. In the context of a search query, only if the querying user is authorized to access the object, such as if the privacy settings for the object allow it to be presented to, discovered by, or otherwise accessible to the querying user. If visible, the object can be provided as a search result. In certain embodiments, the object may represent content visible to the user via the user's news feed. As an example and not a limitation, one or more objects may be visible to the user's Trending page. In certain embodiments, objects may correspond to specific users. The object may be content associated with a specific user, or may be a specific user's account or information stored on a social network connection system or other computing system. As an example and not as a limitation, the first user may view one or more of the online social networks through the "People You May Know" function of the online social network or by viewing the first user's friend list. Two users. By way of example, and not by way of limitation, a first user may specify that they do not wish to see items associated with a particular second user in their news feed or friends list. Objects can be excluded from search results if their privacy settings do not allow them to be presented to, discovered by, or visible to users. Although this disclosure describes enforcing privacy settings in a specific manner, this disclosure covers enforcing privacy settings in any suitable manner.

In certain embodiments, different objects of the same type associated with a user may have different privacy settings. Different types of objects associated with users can also have different types of privacy settings. As an example and not a limitation, the first user may designate the first user's status updates as public, but any images shared by the first user are visible only to the first user's friends on the online social network. As another example and not limitation, users may specify different privacy settings for different types of entities, such as individual users, friends of friends, followers, user groups, or corporate entities. As another example, and not by way of limitation, the first user may designate a group of users that can view the video posted by the first user, while keeping the video visible to the first user's employer. In certain embodiments, different privacy settings may be provided for different user groups or user demographics. As an example and not as a limitation, the first user may specify that other users who attended the same university as the first user may view the first user's image, but other users who are family members of the first user may not View their identical images.

In certain embodiments, the social network connection system may provide one or more default privacy settings for each object of a particular object type. The privacy settings of an object that is set as default can be changed by the user associated with that object. By way of example, and not by way of limitation, all images posted by the first user can have a default privacy setting visible only to the first user's friends, and for a particular image, the first user can change the privacy setting of that image to Visible to friends and friends of friends.

In certain embodiments, privacy settings may allow the first user to specify (e.g., by opting out, by not opting in) whether the social network connected system may receive, collect, record, or store and use it for any purpose. specific objects or information associated with them. In certain embodiments, privacy settings may allow a first user to specify whether a specific application or program can access, store, or use specific objects or information associated with the user. Privacy settings may allow first users to opt in or out of having objects or information accessed, stored, or used by a particular application or program. The social network connection system may access this information in order to provide specific functions or services to the first user, but the social network connection system does not need to access this information for any other purpose. Before accessing, storing, or using such objects or information, the social network connection system may prompt the user to specify which applications or programs (if any) may access, store, or use the items before allowing any such action. Privacy settings for objects or information. As an example and not as a limitation, the first user may transmit messages to the second user via an application related to an online social network (e.g., a messaging application) and may specify that such messages should not be transmitted by the social network Connect to system-stored privacy settings.

In certain embodiments, a user may specify whether specific types of objects or information associated with the first user may be accessed, stored, or used by the social network connection system. By way of example and not limitation, the first user may specify that images sent by the first user via the social network connection system may not be stored by the social network connection system. As another example, and not by way of limitation, a first user may specify that messages sent from the first user to a particular second user may not be stored by the social network connection system. As yet another example and not limitation, the first user may specify that all objects sent via a particular application may be saved by the social network connection system.

In certain embodiments, privacy settings may allow the first user to specify whether certain objects or information associated with the first user are accessible from certain client systems or third-party systems. Privacy settings may allow the first user to opt in or out of having objects or information available from a specific device (e.g., a phone book on the user's smartphone), from a specific application (e.g., a messaging application), or from a specific system (e.g., email server) access. The social network connection system may provide default privacy settings for each device, system, or application, and/or may prompt the first user to specify specific privacy settings for each context. By way of example, and not by way of limitation, the first user may utilize the location services feature of the social network connection system to provide recommendations to the user for restaurants or other nearby establishments. The first user's default privacy setting may specify that the social network connection system may use location information provided from the first user's client device to provide location-based services, but the social network connection system may not store the first user. location information or provide it to any third-party system. The first user may then update the privacy settings to allow location information to be used by third-party image sharing applications for geotagging photos.

In certain embodiments, privacy settings may allow users to specify one or more geographic locations from which objects may be accessed. Access or denial of access to an object may depend on the geographic location of the user attempting to access the object. As an example and not a limitation, users can share an object and specify that only users in the same city can access or view the object. As another example and not limitation, a first user can share an object and specify that the object is only visible to a second user when the first user is in a specific location. If the first user leaves a specific location, the object is no longer visible to the second user. As another example, and not by way of limitation, a first user may specify that an object is only visible to a second user within a threshold distance from the first user. If the first user subsequently changes location, the original second user who accessed the object may lose access, while a new group of second users may gain access if they appear within a threshold distance of the first user. Pick.

In certain instances, changes to privacy settings may be retroactively effective, thereby affecting the visibility of objects and content that were shared prior to the change. By way of example and not limitation, the first user may share the first image and specify that the first image will be public to all other users. At a later time, the first user may specify that any images shared by the first user should be visible only to the first user group. The social network connection system may determine that this privacy setting also applies to the first image, and make the first image visible only to the first user group. In certain instances, changes to privacy settings may only take effect in the future. Continuing with the above example, if the first user changes privacy settings and then shares the second image, the second image may be visible only to the first user group, but the first image may remain visible to all users. In certain embodiments, in response to a user action to change privacy settings, the social network connection system may further prompt the user to indicate whether the user wants to retroactively apply the changes to the privacy settings. In certain embodiments, a user's change to privacy settings may be a one-time change specific to an object. In certain embodiments, changes to privacy by a user may be global changes for all objects associated with the user.

In certain embodiments, the social network connection system may determine that the first user may want to change one or more privacy settings in response to a trigger action associated with the first user. The triggering action can be any suitable action on an online social network. As an example and not as a limitation, the triggering action may be a change in the relationship between the first user and the second user of the online social network (e.g., "removing" the user as a friend, changing the relationship between the users) relationship status). In certain embodiments, after determining that the triggering action has occurred, the social network connection system may prompt the first user to change privacy settings regarding the visibility of objects associated with the first user. The prompt may redirect the first user to a workflow for editing privacy settings regarding one or more entities associated with the triggered action. Privacy settings associated with a first user may change only in response to explicit input from the first user, and may not be changed without the first user's approval. By way of example, and not by way of limitation, the workflow may include providing the first user with the current privacy settings for the second user or group of users (e.g., not adding the first user or second user from a specific object). tags, change the visibility of specific objects with respect to a second user or group of users), and receive instructions from the first user to change privacy settings based on any of the methods described herein, or to maintain existing privacy settings.

In certain embodiments, a user may be required to provide verification of privacy settings before the user is allowed to perform certain actions on an online social network, or to provide verification before changing certain privacy settings. When a specific action is performed or a specific privacy setting is changed, a prompt may be presented to the user to remind the user of his or her current privacy settings and require the user to verify the privacy settings for the specific action. Additionally, the user may be required to provide confirmation, double-opt, authentication, or other suitable type of verification before proceeding with a particular action, and the action may not be completed until such verification is provided. By way of example, and not by way of limitation, a user's default privacy setting may indicate that the personal relationship status is visible to all users (i.e., "public"). However, if the user changes his or her relationship status, the social network connection system may determine that this action may be sensitive and may prompt the user to confirm that his or her relationship status should remain public before proceeding. As another example, and not by way of limitation, a user's privacy settings may specify that the user's posts are visible only to the user's friends. However, if a user changes the privacy settings of his or her posts to public, the social network connection system may prompt the user with a reminder that the user's current post privacy settings are visible only to friends, and this change A warning that will make all of the user's past posts visible to the public. The user may then be required to provide secondary verification, enter authentication credentials, or provide other types of verification before proceeding with privacy settings changes. In certain instances, users may be required to provide periodic verification of privacy settings. Prompts or reminders can be sent to the user periodically based on elapsed time or number of user actions. By way of example, and not by way of limitation, the social network connection system may send a reminder to the user to confirm his or her privacy settings every six months or after every ten photo posts. In certain embodiments, privacy settings may also allow users to control access to objects or information on a per-request basis. By way of example, and not by way of limitation, the social network connection system may notify the user whenever a third-party system attempts to access information associated with the user, and require the user to provide verification that access should be allowed before proceeding.

The techniques described herein may be implemented as a method performed by a physical computing device; implemented as one or more non-transitory computer-readable storage media storing instructions that, when executed by the computing device, cause the method to be performed; Or implemented as a physical computing device specially configured with a combination of hardware and software that enables performing the method.

Figure 10 illustrates an example flow diagram (eg, process 1000) of an accessibility controller and controller hub in accordance with certain aspects of the present disclosure. For purposes of explanation, example process 1000 is described herein with reference to FIGS. 1-9. Further for purposes of explanation, the steps of example process 1000 are described herein as occurring sequentially or linearly. However, multiple instances of the example process 1000 may occur in parallel. For the purpose of explaining the present technology, process 1000 will be discussed with reference to FIGS. 1-9.

At step 1002, process 1000 may include coupling a controller to a controller hub. The controller is configured for use by users with limited mobility. The controller may include an actuator with six degrees of freedom. At step 1004, process 1000 may include detecting controller input to the controller along each of the six degrees of freedom. At step 1006, process 1000 may include coupling the accessory to the controller hub. At step 1008, process 1000 may include detecting hub input from an accessory of the controller hub. At step 1010, process 1000 may include causing interaction in the virtual environment based on controller input and hub input.

For example, as described above with respect to FIG. 9 , at step 1002 , process 1000 may include coupling a controller to a controller hub via controller coupling module 908 . The controller is configured for use by users with limited mobility. The controller may include an actuator with six degrees of freedom. At step 1004, process 1000 may include detecting controller input to the controller along each of the six degrees of freedom via controller input detection module 910. At step 1006 , process 1000 may include coupling the accessory to the controller hub via controller hub coupling module 912 . At step 1008 , process 1000 may include detecting hub input from an accessory of the controller hub via hub input detection module 914 . At step 1010, process 1000 may include causing interaction in the virtual environment based on the controller input and the hub input via the interaction causing module 916.

Depending on the aspect, the controller is configured for use via a head mounted display without a controller hub, and the head mounted display is configured to convert six degrees of freedom control into either the head mounted display or the controller. Six degrees of freedom motion for one or both.

Depending on the aspect, the actuator includes a joystick.

Depending on the aspect, the actuator includes a knob.

Depending on the configuration, the actuator can push.

According to aspects, the actuator includes a sensor that senses motion input along each of the six degrees of freedom.

Depending on the aspect, the controller includes sensors for up/down, back/forward, up/down, roll, yaw and/or pitch.

Depending on the form, accessories include buttons, triggers, sensors and additional controller functionality.

Depending on the mode, the controller hub contains directional pad/directional pad functionality.

According to aspects, a controller hub includes a plurality of controller ports for electrically coupling one or more external controllers to the controller hub.

Depending on the mode, each of the plurality of controller ports includes a USB-A connection, a micro-USB connection, a mini-0 USB connection, a USB-C connection, and a thunderbolt connection.

11 is a block diagram illustrating an exemplary computer system 1100 in which the techniques of this disclosure may be implemented. In some aspects, computer system 1100 may be implemented using hardware, or a combination of software and hardware, on a dedicated server, integrated into another entity, or distributed across multiple entities.

Computer system 1100 (eg, servers and/or clients) includes a bus 1108 or other communications mechanism for communicating information, and a processor 1102 coupled to bus 1108 for processing information. By way of example, computer system 1100 may be implemented with one or more processors 1102 . The processor 1102 can be a general-purpose microprocessor, a microcontroller, a digital signal processor (Digital Signal Processor; DSP), an application specific integrated circuit (Application Specific Integrated Circuit; ASIC), or a field programmable gate array (Field Programmable Gate). Array; FPGA), programmable logic device (Programmable Logic Device; PLD), controller, state machine, gate control logic, discrete hardware component, or any other suitable entity that can perform calculations or other manipulations of information.

In addition to hardware, computer system 1100 may include code that creates an execution environment for the computer programs in question, for example, constituting processor firmware, protocol stacks, database management systems, operating systems, or stored in included memory 1104 Program code that is a combination of one or more of the above, including memory such as random access memory (Random Access Memory; RAM), flash memory, and read-only memory (Read-Only Memory; ROM) ), Programmable Read-Only Memory (PROM), Erasable PROM (EPROM), scratchpad, hard disk, removable disk, CD-ROM, DVD or any Other suitable storage devices are coupled to bus 1108 for storing information and instructions to be executed by processor 1102. The processor 1102 and memory 1104 may be supplemented by or incorporated into special purpose logic circuitry.

Instructions may be stored in memory 1104 and one or more modules of computer program instructions encoded on a computer-readable medium according to any method known to one of ordinary skill in the art. Implemented in computer program products for the computer system 1100 to execute or control the operation of the computer system 1100, these instructions include but are not limited to computer languages such as the following: data-oriented languages (for example, SQL, dBase), system languages (for example, C , Objective-C, C++, Assembly), architectural languages (e.g., Java, .NET), and application languages (e.g., PHP, Ruby, Perl, Python). Instructions can also be implemented in computer languages, such as array languages, feature-oriented languages, assembly languages, production languages, command line interface languages, compiled languages, parallel languages, curly bracket languages, data flow languages, data structured languages, declarative languages, esoteric languages Languages, extended languages, fourth-generation languages, functional languages, interactive model languages, interpreted languages, iterative languages, serial-based languages, small languages, logic-based languages, machine languages, macro languages, metaprograms Design languages, multiparadigm languages, numerical analysis, non-English languages, object-oriented classification languages, object-oriented prototype-based languages, off-site rule languages, procedural languages, reflective languages, rule-based languages, script processing Languages, stack-based languages, synchronization languages, syntax processing languages, visual languages, wirth languages and xml-based languages. Memory 1104 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 1102.

Computer programs as discussed herein do not necessarily correspond to files in a file system. A program may be stored as part of a file that holds other programs or data (for example, one or more scripts in a markup language file), in a single file dedicated to the program in question, or in multiple In a coordinated file (for example, a file that stores one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network. The programs and logic flows described in this specification may be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and producing output.

Computer system 1100 further includes a data storage device 1106, such as a magnetic disk or optical disk, coupled to bus 1108 for storing information and instructions. Computer system 1100 may be coupled to various devices via input/output modules 1110 . Input/output module 1110 can be any input/output module. Exemplary input/output module 1110 includes a data port, such as a USB port. Input/output module 1110 is configured to connect to communication module 1112 . Exemplary communication modules 1112 include network connection interface cards, such as Ethernet cards and modems. In some aspects, input/output module 1110 is configured to connect to a plurality of devices, such as input device 1114 and/or output device 1116 . Exemplary input devices 1114 include a keyboard and a pointing device, such as a mouse or trackball, through which a user can provide input to computer system 1100 . Other types of input devices 1114 may also be used to provide interaction with the user, such as tactile input devices, visual input devices, audio input devices, or brain-computer interface devices. For example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and any form of input may be received from the user, including acoustic input, voice input, tactile input, or Brainwave input. Exemplary output devices 1116 include display devices, such as liquid crystal display (LCD) monitors, for displaying information to a user.

According to one aspect of the present disclosure, the game system described above may be implemented using the computer system 1100 in response to the processor 1102 executing one or more sequences of one or more instructions contained in the memory 1104 . Such instructions may be read into memory 1104 from another machine-readable medium, such as data storage device 1106 . Execution of sequences of instructions contained in main memory 1104 causes processor 1102 to perform the process steps described herein. One or more processors in a multi-processing configuration may also be used to execute sequences of instructions contained in memory 1104 . In alternative aspects, hardwired circuitry may be used in place of or in combination with software instructions to implement aspects of the present disclosure. Therefore, the present disclosure is not limited to any specific combination of hardware circuitry and software.

Various aspects of the subject matter described in this specification may be implemented in computing systems that include back-end components, such as data servers, or include middleware components, such as application servers, or include front-end components, such as having A client computer through which a user may interact with implementations of the subject matter described in this specification through a graphical user interface or web browser, or any of one or more such back-end components, middleware components, or front-end components combination. The components of the system may be interconnected by any form or medium of digital data communication (eg, communications network). A communication network may include, for example, any one or more of a LAN, a WAN, the Internet, and the like. In addition, the communication network may include, but is not limited to, any one or more of the following network topologies, including bus network, star network, ring network, mesh network, star bus network , tree or hierarchical network or the like. The communication module may be, for example, a modem or an Ethernet card.

Computer system 1100 may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship between client and server is created by computer programs that run on separate computers and have a master-slave relationship with each other. Computer system 1100 may be, for example, but not limited to, a desktop computer, a laptop computer, or a tablet computer. Computer system 1100 may also be embedded in another device, such as but not limited to a mobile phone, PDA, mobile audio player, Global Positioning System (GPS) receiver, video game console and/or television top box .

The term "machine-readable storage medium" or "computer-readable medium" as used herein refers to any medium or media that participates in providing instructions to processor 1102 for execution. Such media can take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as data storage device 1106 . Volatile media includes dynamic memory, such as memory 1104. Transmission media include coaxial cable, copper wire, and fiber optics, including the wires including bus 1108 . Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punched cards, paper tape, tapes with hole patterns any other physical media, RAM, PROM, EPROM, FLASH EPROM, any other memory chip or cartridge, or any other media that can be read by a computer. The machine-readable storage medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects a machine-readable propagation signal, or a combination of one or more thereof.

As the user computing system 1100 reads the game data and serves the game, the information may be read from the game data and stored in a memory device such as memory 1104 . Additionally, data from the memory 1104 server or data store 1106 accessed via the network bus 1108 may be read and loaded into the memory 1104 . Although the data is described as being found in memory 1104, it should be understood that the data need not be stored in memory 1104 and may be stored in other memories accessible to processor 1102 or distributed among a number of media, such as data storage Device 1106.

As used herein, the phrase "at least one of" preceding a list of items by the terms "and" or "or" separating any of those items modifies the list as a whole, rather than the items in the list Each member (that is, each project). The phrase "at least one of" does not require the selection of at least one of the items; in fact, the phrase is allowed to include at least one of any of the items and/or at least one of any combination of the items and/or the meaning of at least one of each of these items. By way of example, the phrase "at least one of A, B and C" or "at least one of A, B or C" means respectively only A, only B or only C; any combination of A, B and C and/or at least one of each of A, B and C.

To the extent that the terms "include," "have," or the like are used in this specification or the claimed scope, such terms are intended to be interpreted similarly to the term "includes" when "includes" is used as a transition word in the claimed scope. The translation is inclusive. The term "exemplary" is used herein to mean "serving as an example, example, or illustration." Any specific example described herein as "illustrative" is not necessarily to be construed as being better or superior to other specific examples.

Unless specifically stated otherwise, references to an element in the singular are not intended to mean "one and only one" but rather "one or more." All structural and functional equivalents of the elements in the various configurations described throughout this disclosure that are known or hereafter to be known to those skilled in the art are expressly incorporated herein by reference and are intended to be covered by this disclosure. Technology covered. Furthermore, nothing disclosed herein is intended to be exclusive to the public, whether or not such disclosure is explicitly recited in the description above.

Although this specification contains numerous features, these features should not be construed as limiting the scope of what may be claimed but rather as descriptions of particular implementations of the subject matter. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be deleted from that combination, and the claimed combination Can target sub-combinations or changes in sub-combinations.

The subject matter of this specification has been described with respect to certain aspects, but other aspects may be practiced and are within the scope of the following claims. For example, although operations are depicted in a specific order in the drawings, this should not be understood to require that such operations be performed in the specific order shown, or in sequential order, or that all illustrated operations be performed to achieve desirable results. result. The actions recited in the claimed scope may be performed in a different order and still achieve the desired results. As an example, the processes depicted in the figures do not necessarily require the specific order shown, or sequential order, to achieve desirable results. In certain environments, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the aspects described above should not be construed as requiring such separation in all aspects, and it is understood that the program components and systems described may generally be integrated together in a single software product Or packaged into multiple software products. Other changes are within the scope of the following patent applications.

100:Computing system 101:Computing device 102:Computing device 103:Computing device 110: Processor 120:Input device 130:Display 140:I/O device 150:Memory 160:Program memory 162:Operating system 164:XR working system 166:Other applications 170:Data memory 200:HMD 205: Front rigid body 210:bring 215:Inertial motion unit 220: Position sensor 225:Locator 230:Computing unit 245: Electronic display 250: Mixed reality HMD system 252: Mixed Reality HMD 254: Core processing component 256:Link 258:Transparent display 260:Frame 300:Handheld controller 310:User input surface 320A: First user input key 320B: User input key 320C: User input key 330: Fixture 340:Cage 345:Extroverted surface 350: Illumination source 390:Trigger 395: Structural web 401: Knob 411: Forward and lateral movement 412: Orthogonal lateral motion 413:Downward movement 414: Forward rotational movement 415: Orthogonal rotational motion 416: Horizontal rotation movement 500:Accessibility Controller 501: rocker 502:Device body 503:Button 600: Accessibility controllers and hubs 601:Controller and hub 602: Multiple brake devices 603:External controller device 604:External controller device 700:Controller Hub 701:Controller body 702a-n: External controller port 703a: Input brake device 703c: Input brake device 704a-d: Buttons can be pressed 705: Four-way switch 800: Six-value coordinate system 801:X axis 802: Y axis 803: Y axis 811:Scroll 812:Pitch 813:Yaw 900:System 902:Computing platform 904:Remote platform 906: Machine-readable instructions 908:Controller coupling module 910:Controller input detection module 912:Controller hub coupling module 914: Hub input detection module 916: Interaction causes module 918:External resources 920: Electronic storage 922: Processor 1000:Process 1002: Steps 1004: Steps 1006: Steps 1008: Steps 1010: Steps 1100:Computer system 1102: Processor 1104:Memory 1106:Data storage device 1108:Bus 1110:Input/output module 1112: Communication module 1114:Input device 1116:Output device

To easily identify discussion of any particular element or act, the most significant digit or digits of a reference number refer to the figure number in which that element first introduced.

[FIG. 1] is a block diagram illustrating an overview of a device by which some implementations of the disclosed technology may be operated.

[FIG. 2A] is a line diagram of a virtual reality head-mounted display (HMD) according to one or more implementations.

[FIG. 2B] is a line diagram of a mixed reality HMD system including a mixed reality HMD and core processing components in accordance with one or more implementations.

[FIG. 3A] and [FIG. 3B] illustrate isometric views of a handheld controller of an exemplary artificial reality system in accordance with one or more implementations.

[Figure 4A] illustrates an example accessibility controller in accordance with one or more implementations.

[FIG. 4B] illustrates example motion control operations of the accessibility controller of FIG. 4A, in accordance with one or more implementations.

[Figure 5] illustrates another example accessibility controller in accordance with one or more implementations.

[Figure 6] illustrates yet another example accessibility controller and hub in accordance with one or more implementations.

[Figure 7] illustrates yet another example accessibility controller and hub in accordance with one or more implementations.

[FIG. 8] illustrates six degrees of motion control operation for an accessibility controller in accordance with one or more implementations.

[Figure 9] illustrates a system configured for accessibility controllers and controller hubs in accordance with one or more implementations.

[FIG. 10] An example flow diagram illustrating an accessibility controller and controller hub in accordance with certain aspects of the present disclosure.

[FIG. 11] is a block diagram illustrating an example computer system (e.g., representing both a client and a server) in which the techniques of this disclosure may be implemented.

In one or more implementations, not all components depicted in each figure may be required, and one or more implementations may include additional components not shown in the figures. Changes in configuration and type of components may be made without departing from the scope of this disclosure. Additional components, different components, or fewer components may be utilized within the scope of this disclosure.

200:HMD

205: Front rigid body

210:bring

215:Inertial motion unit

220: Position sensor

225:Locator

230:Computing unit

245: Electronic display

Claims (20)

A computer-implemented method for providing accessibility to users with limited mobility in a virtual environment, comprising: coupling a controller to a controller hub, the controller configured for use by a user with limited mobility, the controller including an actuator having six degrees of freedom; detecting controller input to the controller along each of the six degrees of freedom; coupling accessories to the controller hub; Detect the hub input of the accessories from the controller hub; and Interactions in the virtual environment are caused based on the controller inputs and the hub inputs. The method of claim 1, wherein the controller is configured for use without the controller hub via a head mounted display, the head mounted display configured to convert six degrees of freedom controls to the Six degrees of freedom motion for either or both the head-mounted display or the controller. The method of claim 1, wherein the actuator includes one or both of a joystick or a knob. The method of claim 1, wherein the actuator is pushable. The method of claim 1, wherein the actuator includes a sensor that senses motion input along each of the six degrees of freedom. The method of claim 1, wherein the controller includes sensors for up/down, backward/forward, up/down, roll, yaw and/or pitch. The method of claim 1, wherein the accessories include buttons, triggers, sensors and additional controller functionality. The method of claim 1, wherein the controller hub includes a directional pad/a directional pad functionality. The method of claim 1, wherein the controller hub includes a plurality of controller ports, and the plurality of controller ports are used to electrically couple one or more external controllers to the controller hub. The method of claim 8, wherein each of the plurality of controller ports includes a USB-A connection, a micro USB connection, a mini USB connection, a USB-C connection, and a Thunderbolt connection. A system configured for providing accessibility to users with limited mobility in a virtual environment, the system comprising: One or more hardware processors configured by machine-readable instructions to: A controller is coupled to a controller hub, the controller configured for use by a user with limited mobility, the controller including an actuator having six degrees of freedom, the actuator including a control A lever, a knob, and sensors that sense motion input along each of the six degrees of freedom; detecting controller input to the controller along each of the six degrees of freedom; coupling accessories to the controller hub; Detect the hub input of the accessories from the controller hub; and Interactions in the virtual environment are caused based on the controller inputs and the hub inputs. The system of claim 11, wherein the actuator is pushable. The system of claim 11, wherein the controller includes sensors for up/down, backward/forward, up/down, roll, yaw and/or pitch. The system of claim 11, wherein the accessories include buttons, triggers, sensors and additional controller functionality. The system of claim 11, wherein the controller hub includes a directional pad/directional pad functionality. The system of claim 11, wherein the controller hub includes a plurality of controller ports for electrically coupling one or more external controllers to the controller hub, and wherein the plurality of control ports Each of the ports includes a USB-A connection, a micro USB connection, a mini USB connection, a USB-C connection, and a Thunderbolt connection. A non-transitory computer-readable storage medium having instructions embodied thereon that are executable by one or more processors for providing accessibility to users with limited mobility in a virtual environment method, which contains: A controller is coupled to a controller hub, the controller configured for use by a user with limited mobility, the controller including an actuator having six degrees of freedom, the actuator including a control A lever, a knob, and sensors that sense motion input along each of the six degrees of freedom, the controller including controls for up/down, back/forward, up/down, roll, yaw, and /or pitch sensor; detecting controller input to the controller along each of the six degrees of freedom; Coupling accessories to the controller hub, the accessories including buttons, triggers, sensors, and additional controller functionality; Detect the hub input of the accessories from the controller hub; and Interactions in the virtual environment are caused based on the controller inputs and the hub inputs. The non-transitory computer readable storage medium of claim 17, wherein the controller hub includes a plurality of controllers for electrically coupling one or more external controllers to the controller hub, and Each of the plurality of controller ports includes a USB-A connection, a micro USB connection, a mini USB connection, a USB-C connection and a Thunderbolt connection. The non-transitory computer-readable storage medium of claim 17, wherein the actuator is pushable. The non-transitory computer readable storage medium of claim 17, wherein the controller hub includes a directional pad/a directional pad functionality.