KR20200090786A - Dynamically balanced multi-purpose handheld controller - Google Patents

Abstract

The controller can control the asset or target in physical and/or virtual three-dimensional space with one hand by limiting cross-coupling (unintentional operation) while generating control inputs with four or more degrees of freedom. The controller includes a first control member configured to be gripped by a user's hand, a second control member disposed at or near the upper end of the first control member movable at least one degree of freedom independently of the movement of the first control member, and And a third control member positioned on the first control member for displacement by one or more fingers of one hand of the user and engaged with the second control member to move opposite to the movement of the second control member.

Description

Handheld controller with multiple degrees of freedom of dynamic balance

The present invention relates generally to a control system, and more specifically to a command for up to six independent degrees of freedom while substantially limiting cross-coupling by using a one-handed controller. It relates to a controller that provides a user with the ability to transmit signals.

Conventionally, a number of individual controllers are used to allow a user to control a control target with at least three degrees of freedom. Moreover, any conventional control system that controls a control target with 6 degrees of freedom requires multiple individual controllers. For example, a set of independent controllers or input devices (e.g. joysticks, control columns, cyclic sticks, foot pedals and/or other independent controllers known to one or more skilled artisans) may control control targets (e.g. aircraft, submersibles, spacecrafts). , A control target in a virtual environment, and/or various other control targets known to one or more skilled artisans) to receive various different rotation parameters (eg pitch, yaw and roll) from the user. Similarly, an independent set of controllers may be provided to control other search parameters, such as translation in three-dimensional (3D) space, velocity, acceleration and/or various other command parameters (eg, x, y and z axis movement). Can.

U.S. Patent Application Nos. 13/797,184 and 15/071,624, filed on March 12, 2013 and March 16, 2016, respectively, all of which are incorporated herein by reference in their entirety using a single controller. Various embodiments of a control system that allow simultaneous control target control with 6 degrees of freedom (6-DoF) are described. In one embodiment, the integrated hand controller is configured to receive a first control member and a translational input (eg displacement along the X, Y and Z axes) for receiving rotational input (eg pitch, yaw and roll) from the user. It may include a second control member extending from the one control member. The first control member and the second control member on the integrated hand controller can be positioned by the user with one hand to control the control target at up to 6-DoF.

The previously known drones, virtual reality, augmented reality, computer and game input devices are not intuitive and require considerable initial and skilled training and are operated with both hands. They are also generally not mobile.

Various aspects of the single processing controller described below allow the user to control an asset or target in a physical and/or virtual three-dimensional space by generating control inputs while limiting cross-coupling (unintentional operation) during or during rest. In order to provide individual and/or in combination with one of these aspects, several more possible improvements to the user (e.g. computer augmented or virtual reality gamers, pilots, hikers, skiers, security/SAR personnel, fighters and others). Using controllers with these features, the controllers can be used to control computer-aided design (CAD), drone flight, various types of computer games, virtual and augmented reality, and other physical and physical control requirements that require precise movement through space. Allows translation to be separated from posture adjustment.

According to one aspect of the present disclosure, the hand controller includes first, second and third control members. The first control member is movable with at least two degrees of freedom and generates a first set of independent control inputs in response. The movement or displacement of the first member is sensed for each degree of freedom using one or more sensors and control inputs can be generated, and each sensor can detect and measure displacement with the freedom of one or more displacements if desired. . The first control member is configured such that the user holds it in one hand of the user by placing a hand on his palm and wrapping at least several fingers at least partially around the body of the first member. The second control member is disposed at or near the top of the first member where the thumb or index finger of the hand can rest when the first member is gripped. In one embodiment, at least one degree of freedom, and in other embodiments, is movable with 2 or 3 degrees of freedom independent of the movement of the first control member. In response to independent movement, movement of the second control member results in a second set of control inputs, one for each degree of freedom that can be displaced. The second set of control inputs is independent of the first set of control inputs.

The extended operation of the controller with the second member with fingers for independent control inputs can cause fatigue, especially when pulling or pressing the second member with the thumb. The third control member is positioned on the first member to displace by one or more fingers of one hand of the user and is coupled with the second member, such that in one of the degrees of freedom of movement of the second control member, the user's thumb controls the second In order to displace the member, it moves in the opposite direction of the movement of the second control member with the degree of freedom of movement being pulled. The third control member is mounted to the first member in a position where one or more fingers on the user's hand that are not used to displace the second control member can press the third member and cause displacement. The third member is coupled to the second member to displace the second member when the third member is displaced therein by the user pressing or pulling the third member with one or more fingers. If desired, pressing the second control member down also pushes the third control member out from the controller to allow the user's thumb or index finger to be dynamically balanced by the user's other fingers.

In a separate aspect of the present disclosure, a hand controller having at least a first and second control member (and optionally a third control member) configured to grip with one hand of a user acts as a reference for a rotation axis, particularly yaw. It can be combined with a wrist or forearm brace. The yaw is difficult to measure with an inertial measurement unit (IMU) within a handheld controller. For example, the IMU of the hand controller can detect and measure sufficient precision and sensitivity pitch and roll (rotate around the X and Y axes) of the first member, but rotate around the Z axis corresponding to the yaw of the first control member The IMU output for can be noisy. In one embodiment, the sensor measures the displacement of the linkage between the user's wrist or forearm and the first control member generated by the user bending the user's wrist to rotate the controller to indicate yaw.

As presented by some representative embodiments described below, a one-handed controller is mounted on the wrist and registers displacement from a defined neutral position relative to the wrist, so as to fly, game or augmented reality with exactly 6 degrees of freedom (6-DoF) movement. Enable motion control. Passive mechanical, vibrational haptic or active mechanical feedback can inform the user of the displacement from zero in each of these 6-DoFs. This one-handed control allows movement through the air like a fighter pilot with intuitive (unintentional perception) input.

According to another aspect of the present disclosure, the forearm brace coupled with the controller can be used in combination with the index finger loop to open or close the grip of the object in the virtual world.

Another aspect of the different exemplary embodiments of the hand controller described below is a two-handed controller that provides a consistent, known reference frame that is stable by a non-dominant hand during movement, such as walking, skiing, running, or driving. Includes. One optional embodiment of the hand controller can be plugged into the surface of the base, allowing the non-flying hand to stabilize the base as it flies.

Moving the reference point (POR) through the physical or virtual space through the hand controller solves the problem of requiring insight into displacement with all degrees of freedom controlled such that the position of the "zero input" is independently and simultaneously known for each degree of freedom. Occurs. For a drone, for example, you must always know the zero input position for the x, y and z axes and yaw. Other flight systems such as virtual and augmented reality, computer games and surgical robots can simultaneously require up to six independent degrees of freedom (movement, pitch, yaw and roll along the x, y and z axes). In addition, a drone flight and, in particular in the case of virtual and augmented reality systems, the ability to move while maintaining precise control of the reference point is desirable.

In one of the exemplary embodiments, the first control member in the form of a joystick mounted on the base notifies the user of a zero command by tactile touch, yaw and roll inputs with the centering mechanism to which the first control member is connected to the base. That can generate the power. The second control member on the top of the joystick is controlled with up to three degrees of freedom by tactile feedback of the zero command, at a position that can be displaced by a thumb or other finger along one or more X, Y and Z axes relative to the first control member Signal. For example, a magnetic or mechanical detent defines a center or “zero” input for each of the multiple degrees of freedom of one or more controllers and allows the user to feel a slight increase in force as the control member starts from the center or “zero input” position. Can be used to When recentering the center of the movement range of the controller member along one of the movement degrees of freedom, a slight force change is felt because the "zero input" is restored. This detent force can be felt simultaneously and independently for each degree of freedom commanded from the user's hand.

Additional aspects, advantages, features and embodiments are described below in conjunction with the accompanying drawings. All patents, patent applications, articles, other publications, documents and objects referenced herein are incorporated herein in their entirety for all purposes. As far as there is a discrepancy or conflict in the definition or use of terms between the publications, documents or things contained therein and this application, those of this application shall prevail.

In order to enhance the understanding of the principles of the invention as claimed below, reference will now be made to the embodiments or examples shown in the accompanying drawings. By describing certain embodiments and examples, it will be understood that it is not intended to limit the scope of the invention beyond the literal terms set forth in the claims. Modifications and further modifications to the described embodiments and examples are possible while using the claimed subject matter, and are therefore contemplated as being within the scope of the claimed invention.
1 is a schematic diagram of an embodiment of a control system.
2 is a flowchart illustrating an embodiment of a method for controlling a control target.
3A is a side view showing an embodiment of a user using the controller shown in FIGS. 2A-2G with one hand.
3B is a cross-sectional view of the embodiment shown in FIG. 3A.
3C is a front view of the embodiment shown in FIG. 3A.
FIG. 4A is a side view showing an embodiment of a physical or virtual vehicle control target that performs movement according to the method of FIG. 2.
4B is a plan view illustrating an embodiment of the physical or virtual vehicle control target of FIG. 4A performing movement according to the method of FIG. 4A.
4C is a front view showing an embodiment of the physical or virtual vehicle control target of FIG. 4A performing movement according to the method of FIG. 2;
4D is a perspective view showing an embodiment of a tool control target for performing movement according to the method of FIG. 2.
5 is a flowchart illustrating an embodiment of a method for controlling a control target.
6 is a flowchart illustrating an embodiment of a method for configuring a controller.
7 is a side view of a first exemplary embodiment of a one-handed controller.
8A is a perspective view of a second exemplary embodiment of a partially assembled one-hand controller, with a pivoting platform for a second control member in a first position.
8B is a perspective view of a second exemplary embodiment of a partially assembled one-hand controller, with a pivoting platform for a second control member in a second position.
8C is a perspective view of a second exemplary embodiment of a one-handed controller in an assembled state different from that shown in FIGS. 8A and 8B, with half of the housing forming the first control member removed.
9 shows a perspective view of a third exemplary embodiment of a controller with a secondary control member in the form of a thumb loop.
10 shows a perspective view of a fourth exemplary embodiment of a controller with a gantry-type secondary control member.
11 shows a perspective view of a fifth exemplary embodiment of a controller having a trackball type secondary control member.
12 is a perspective view of a mobile two-hand control system with a controller mounted on the base.
13 is a perspective view of a controller mounted on a base having an input button.
14 is a perspective view of a one-handed controller mounted on a wired base.
15 is a perspective view of one representative controller of another representative example and embodiment mounted on a bracket connected to a user's forearm.
16 is a perspective view of another representative example and embodiment of a one-handed controller connected to a forearm attachment worn by a user.
17 is a perspective view of a representative example of a handle controller coupled with a cuff mounted on a user's forearm.
18 is a side view of a representative example of a handle controller coupled with a cuff mounted on the user's forearm shown in FIG. 17.
19A is a top view of a representative example of a control system with a dual gimbal link between a forearm attachment and a hand controller.
19B is a side view of the control system of FIG. 19A.
19C is a perspective view of the control system of FIG. 19A.
19D is a perspective view of a second representative example of a control system with a dual gimbal link between a forearm attachment and a hand controller.
20A is a side view of another representative example of a control system having a dual gimbal link between a forearm attachment and a hand controller.
20B is another side view of the control system of FIG. 20A.
21A-21F illustrate a controller according to one embodiment.
22A-22F illustrate a controller according to one embodiment.
23 is a side view of a hand controller.
24A and 24B schematically show two versions of another embodiment of a hand controller.
25A and 25B show two positions of another embodiment of the hand controller.
26 is a schematic diagram of another embodiment of a controller.
27 is a schematic diagram of a connector for detachably connecting the hand controller to the base.
28 schematically shows the gimbal.
29 is a cross-sectional view of FIG. 28.

In the following drawings and descriptions, the drawings are not necessarily to scale.

Certain features of the invention may be exaggerated in scale or schematic form. For the sake of clarity or conciseness, the details or presence of conventional or previously described elements may not be shown.

The controller of the present disclosure can be implemented in several forms while still providing at least one of the advantages mentioned below. Many of the specific examples described below provide several advantages. It should be understood that the present disclosure is to be considered as illustrative of the principles of the invention and is not intended to limit the invention to what is shown and described herein, and specific examples are described in detail and shown in the drawings. It should be fully appreciated that different teachings of the examples described below can be used individually or in any suitable combination to produce the desired results. The various features mentioned above, as well as other features and features described in more detail below, will be apparent to those skilled in the art by reading the following description of exemplary embodiments of the invention and referring to the accompanying drawings in the specification.

This disclosure describes several embodiments of a control system that allows a user to control a control target or reference point (POR) at up to 6 degrees of freedom (6-DoF) using a single controller. In one embodiment, the integrated hand controller can receive a first set of 1, 2 or 3 additional inputs from the user and a first control member for receiving a first set of 1, 2 or 3 additional inputs from the user. The second control member may extend from the first control member. User input is generated by the user displacing each control member with a maximum of 3 degrees of freedom. The controller maps user input to a preselected output used to control the target control system. The first control member and the second control member on the integrated hand controller can be repositioned by the user with one hand to control the control target with up to 6 degrees of freedom.

More specifically, in some embodiments of the control system described below, a user can control a control target at 6-DoF using a single controller. In one embodiment, the integrated hand controller extends from the first control member and the first control member to receive rotational inputs (eg pitch, yaw and roll) and translates (eg, moves along the x, y and z axes) It may include a second control member for receiving. Alternatively, the user can program this control system input into a different coordinate system as desired or necessary for the operation being performed. As described in further detail below, the first control member and the second control member on the integrated hand controller can be repositioned by the user with one hand to control the control target in 6-DoF.

The embodiments described below are examples of improved one-handed controllers with one or more additional features compared to conventional hand controllers. These additional features and improvements include: improved Z axis spring force and self centering/zeroing capability for the second member controlled by the user's thumb when gripping the first member of the controller; A larger gantry at the top of the first member to move the second member along the X and Y axes; A replaceable or resizable thumb loop for the second control member; Forearm or wrist stabilizers for ambulatory use (potentiometers, Hall effect sensors or optical encoders for translation along the X, Y and Z axes, such as those used in drone applications and integrated with virtual/augmented reality); Mouse-based implementation for improved CAD object manipulation; And combinations of any two or more of the foregoing features.

Hand controllers with one or more of these features and their variations include flight simulation, computer aided design (CAD), drone flight, fixed and rotating wing flight, computer games, virtual and augmented reality exploration, aerial refueling, and surgical robotics. , Ground and maritime robot controls and some of them can be used in many others described below.

Referring first to FIG. 1, a control system 100 for controlling a control target in 6-DoF is shown. The control system 100 includes a controller 102 coupled to a signal conversion system 104 further coupled to a control target 106. In one embodiment, the control target 106 includes an end effector (e.g., the end of a robot tong, a robot arm end effector with a snare), a camera field of view (e.g., including a camera center field of view and zoom), a vehicle speed vector, etc. can do. Although controller 102 and signal conversion system 104 are shown separately, those skilled in the art will recognize that some or all of controller 102 and signal conversion system 104 may be combined without departing from the scope of the present disclosure. will be.

The controller 102 includes a first control member 102a and a second control member 102b located on the first control member 102a. In some embodiments, the controller 102 may also include a third control member (not shown) located on the first control member 102a. In this description, controller 102 is intended to represent all controllers described herein unless otherwise indicated. The controller processor 102c is coupled to each of the first control member 102a and the second control member 102b. In one embodiment, controller processor 102c may be a central processing unit, a programmable logic controller, and/or various other processors known to those skilled in the art. The controller processor 102c is also coupled to the rotation module 102d, the conversion module 102e, and the transmitter 102f, respectively. Although not shown or described in further detail, the first control member 102a, the second control member 102b, the controller processor 102c, the rotation module 102d, the conversion module 102e while maintaining the scope of the present disclosure And other connections and bonds between the transmitter 102f. In addition, components of the controller may be combined or replaced with other components known to those skilled in the art while maintaining the scope of the present disclosure.

The signal conversion system 104 of the control system 100 is a transceiver 104a that can be coupled to the transmitter 102f of the controller 102 via a wired connection, a wireless connection, and/or various other connections known to one or more skilled artisans. It includes. The conversion processor 104b is coupled to a transceiver 104a, a control module 104c, and configuration parameters 104d that may be included on memory, storage devices and/or other computer readable media known to one or more skilled artisans. In one embodiment, conversion processor 104b may be a central processing unit, a programmable logic controller, and/or various other processors known to those skilled in the art. Although not shown or described in more detail, other connections and couplings may exist between the transceiver 104a, the conversion processor 104b, the control module 104c, and the configuration parameters 104d while maintaining the scope of the present disclosure. In addition, components of the signal conversion system 104 may be combined or replaced with other components known to those skilled in the art while maintaining the scope of the present disclosure. The control module 104c can be coupled to the control target 106 via wired connection, wireless connection and/or various other connections known to those skilled in the art.

In one embodiment, the controller 102 is configured to receive input from the user through the first control member 102a and/or the second control member 102b and transmit a signal based on the input. For example, the controller 102 is a “joystick” to navigate in a virtual environment (eg, video games, real simulators, remote control virtual/real control systems and/or various other virtual environments known to one or more skilled artisans). Can be provided. In another example, the controller 102 can be provided as a control stick for controlling a vehicle (eg, aircraft, submersible, spacecraft, and/or various other vehicles known to those skilled in the art). In another example, the controller 102 can be provided as a control stick for controlling a robot or other non-vehicle device (eg, surgical device, assembly device, and/or various other non-vehicle devices known to those skilled in the art).

In the embodiments discussed in further detail below, the controller 102 includes a control stick as the first control member 102a configured to be relocated by the user. Repositioning of the first control member 102a of the control stick allows the user to provide a rotational input using the first control member 102a including pitch input, yaw input and roll input, and the controller processor 102c Causes a rotational movement output signal including a pitch movement output signal, a yaw movement output signal and a roll movement output signal to be output. In particular, when the first control member 102a of the control stick is tilted back and forth, it is possible to provide a pitch input that generates a pitch movement output signal, and the first control member 102a of the control stick is laterally around the longitudinal axis. When rotated, a yaw input for generating a yaw movement output signal can be provided, and when the first control member 102a of the control stick is tilted sideways, a roll input for generating a roll movement output signal can be provided. As will be described later, the movement output signal resulting from the repositioning of the first control member 102a is reconstructed from that described above such that movement similar to that described above of the first control member 102a results in different input and movement output signals. [Example: Tilting the first control member 102a of the control stick to the side can be configured to provide a yaw input that generates a yaw movement output signal while the first control member 102a of the control stick is lengthened. Rotating around the axial axis can be configured to provide a roll movement output signal, a roll input that generates a roll]

The rotation input using the first control member 102a of the control stick can be detected and/or measured using the rotation module 102d. For example, the rotation module 102d may include a displacement detector for detecting displacement of the first control member 102a of the control stick from the starting position as one or more of the above-described pitch input, yaw input and roll input. The displacement detector is a light detector for detecting light rays, rotational and/or linear potentiometers, inductively coupled coils (hole effect sensors), physical actuators, gyroscopes, switches, transducers and/or various other displacement detectors known to one or more skilled artisans. It may include. In some embodiments, the rotation module 102d may include an accelerometer to detect displacement of the first control member 102a of the control stick from the starting position of the space. For example, the accelerometer can each measure the appropriate acceleration of the first control member 102a of the control stick relative to the inertial reference frame.

In another embodiment, the input using the first control member 102a of the control stick has three movement ranges of the first control member 102a of the control stick (eg, forward and backward movement, and lateral movement, longitudinal axis) Circumferential rotation) can be detected and/or measured using breakout switches, transducers and/or direct switches for each. For example, the breakout switch can be used to detect when the first control member 102a of the control stick is initially moved from the null position (eg 2°) for each rotation range, and the transducer can be used for each It is possible to provide a signal proportional to the displacement of the first control member 102a of the control stick for the range of movement, and the direct switch is not moved when it is further moved from the null position (e.g. 12 °) for each range of movement. Can be detected. The breakout switch and direct switch also allow the acceleration of the first control member 102a of the control stick to be detected. In one embodiment, a redundant detector and/or switch may be provided to the controller 102 to ensure that the control system 100 is fault-tolerant.

In the embodiments discussed in further detail below, the second control member 102b extends from the upper distal portion of the first control member 102a of the control stick and is independent of the first control member 102a of the control stick and It is configured to be repositioned by the user with respect to the first control member of the control stick. Repositioning of the second control member 102b discussed below allows the user to provide translational input using a second control member 102b that includes x-axis input, y-axis input, and z-axis input, and controls The processor 102c causes the output of a translational movement output signal including an x-axis movement output signal, a y-axis movement output signal, and a z-axis movement output signal. For example, if the second control member 102b is tilted back and forth, it is possible to provide an x-axis input that generates an x-axis movement output signal, and when the second control member 102b is tilted sideways, the y-axis movement output sinhol The generated y-axis input can be provided, and when the second control member 102b is moved up and down, a z-axis input for generating a z-axis movement output signal can be provided. As will be described later, signals arising from the relocation of the second control member 102b can be reconstructed from those described above such that movement of the second control member 102b similar to that described above results in different input and movement output signals. [Eg, moving the second control member 102b in the front-rear direction can be configured to provide a z-axis input that generates a z-axis movement output signal, while moving the second control member 102b up and down It can be configured to provide x-axis input]. In one embodiment, the second control member 102b is configured to be repositioned only by the thumb of the user while the user grips the first control member 102a of the control stick with a hand that includes a thumb.

The translational input using the second control member 102b can be detected and/or measured using the translation module 102e. For example, the translation module 102e may include a translation detector for detecting displacement of the second control member 102b from the starting position as one or more of the x-axis input, y-axis input, and z-axis input described above. The translation detector may include a physical actuator, translation accelerometer, and/or various other translation detectors known to one or more skilled artisans (e.g., many of the detectors and switches described above to detect and/or measure rotational inputs are used to translate translation inputs). Can be repurposed for sensing and/or measuring.)

It should be understood that the first control member 102a is not limited to rotational input and the second control member 102b is not limited to translational input. For example, the first control member 102a can correspond to a translational input, while the second control member 102b can correspond to a rotational input. In some embodiments, the input associated with each rotational or translational movement may be based on user preferences.

In one embodiment, controller processor 102c of controller 102 is configured to generate a control signal to be transmitted by transmitter 102f. As described above, the controller processor 102c is based on one or more rotational inputs detected and/or measured by the rotation module 102d and/or one or more translational inputs detected and/or measured by the translation module 102e. It can be configured to generate a control signal. The control signal generated by the controller processor 102c defines movement output signals for one or more of 6-DoF (ie, pitch, yaw, roll, movement along the x axis, movement along the y axis, movement along the z axis). It may include parameters. In some embodiments, individual control signal types (eg yaw output signal, pitch output signal, roll output signal, x-axis movement output signal, y-axis movement output signal and z-axis movement output signal) each generate individual control signals. Individual predefined movements of the first control member 102a for providing pitch input, first control member 102a for providing yaw input, first control member 102a for providing yaw input, and first control member 102a for providing roll input ), moving the second control member 102b to provide the x-axis input, moving the second control member 102b to provide the y-axis input, and moving the second control member 102b to provide the z-axis input ]. In addition to 6-DoF control, individual functions such as ON/OFF, trim and other multi-function commands can be sent to the control target. Conversely, data or feedback can be received on controller 102 (eg, an indicator such as an LED can be illuminated green to indicate that controller 102 is on).

In one embodiment, the transmitter 102f of the controller 102 is configured to transmit control signals over a wired or wireless connection. For example, the control signal can be one or more of a radio frequency (“RF”) signal, an infrared (“IR”) signal, a visible light signal and/or various other control signals known to one or more skilled artisans. In some embodiments, the transmitter 102f may be a BLUETOOTH ^® transmitter configured to transmit a control signal as an RF signal according to the BLUETOOTH ^® protocol (BLUETOOTH ^® is a commercial trade association based in Kirkland, Washington, USA, which remains private) It is a registered trademark of the Bluetooth Special Interest Group).

In one embodiment, the transceiver 104a of the signal conversion system 104 receives the control signal transmitted by the transmitter 102f of the controller 102 via the wired or wireless connection described above, and signals the received control signal. It is configured to provide to the conversion processor 104b of the conversion system 104.

In one embodiment, conversion processor 104b is configured to process the control signal received from controller 102. For example, when the conversion processor 104b is executed by the conversion processor 104b, the conversion processor 104b converts the control signal into a movement command and uses the control module 104c of the signal conversion system 104 to transfer the movement command. It can thus be coupled to a computer readable medium including instructions to provide a control program configured to control the control target 106. In one embodiment, the conversion processor 104b controls the control signal in a virtual three dimensional (“3D”) environment (eg, a virtual representation of a surgical patient, video game, simulator and/or various other virtual 3D environments known to one or more skilled artisans). ). Accordingly, the control target 106 may exist in the virtual space, and the user may be provided with a viewpoint or a virtual representation of the virtual environment from the viewpoint inside the control target (ie, the control system 100 controls the virtual environment to the user) And a display providing a view from the target. In other examples, the control target 106 may be a physical device such as a robot, end effector, surgical tool, lifting system, and/or unmanned or remotely controlled vehicle (eg, “drone”); Manned, unmanned or remotely controlled vehicles and land ships; Manned, unmanned or remote controlled aircraft; Manned, unmanned or remotely controlled vessels; Manned, unmanned or remotely controlled submersibles; It can be a variety of steerable mechanical devices including, without limitation, vehicles such as manned, unmanned or remotely controlled spacecraft, rockets, satellites, and the like.

In one embodiment, the control module 104c of the signal conversion system 104 is configured to control the movement of the control target 106 based on the movement command provided from the control program in the signal conversion system 104. In some embodiments, if the control target 106 is in a virtual environment, the control module 104c may include an application programming interface (API) for moving a virtual representation or viewpoint within the virtual environment. The API can also provide feedback from the virtual environment, such as collision feedback, to the control module 104c. In some embodiments, feedback from the control target 106 avoids collisions with the control module 104c, for example, collisions with designated areas (eg, objects in a real or virtual environment, critical areas such as real or virtual patients). In order to adjust the movement of the control target automatically. In other embodiments, when the control target 106 is a physical device, the control module 104c can include one or more controllers to control movement of the physical device. For example, the signal conversion system 104 may be mounted on a vehicle, and the control module 104c may include various physical controllers for controlling various propulsion and/or steering mechanisms of the vehicle.

In one embodiment, signal conversion system 104 includes configuration parameters 104d for use by conversion processor 104b when generating a move command using a signal from controller 102. Operating parameters may include gain (i.e., sensitivity), start rate (i.e., delay), deadband (i.e., neutral), limit (i.e., maximum angular displacement), and/or various known to one or more skilled artisans. Other operating parameters include, but are not limited to. In one embodiment, the gains of the first control member 102a and the second control member 102b can be independently defined by the user. In this example, the second control member 102b is for example a first control member 102a of the control stick to compensate for the second control member 102b having a smaller range of movement than the first control member 102a of the control stick The sensitivity can be increased compared to. Similarly, the starting speed for the first control member 102a and the second control member 102b is before the repositioning of the first control member 102a and the second control member is converted to the actual movement of the control target 106. It can be defined independently to determine the amount of time (ie delay) that must pass. The limits and deadbands of the first control member 102a and the second control member 102b can be defined independently by calibrating the respective neutral and maximum positions.

In one embodiment, the operating parameters are also such that the signal transmitted from the controller 102 is converted into a movement command transmitted to the control target in response to different movements of the first control member 102a and the second control member 102b. Can be defined. As described above, the specific movement of the first control member 102a can generate a pitch, yaw and roll rotational movement output signal, while the specific movement of the second control member 102b is the x-axis, y-axis and z-axis A translational movement output signal can be generated. In one embodiment, the operating parameters define which movement command is sent to the control target 106 in response to movement from the first control member 102a and the second control member 102b and hence the movement output signal. Can be.

One-handed controllers such as those shown in FIGS. 7-20B can provide up to six degrees of freedom control. For applications in many types of physical and virtual 3D environments, all six degrees of freedom may be required, such as spacecraft or multiple types of aircraft movement or certain computer games and virtual and augmented reality environments. In many of these cases, the best way to manage them is to map the x-axis, y-axis and z-axis translational output signals generated by the displacement of the second control member to x-axis, y-axis and z-axis movement, and 1 It is to provide a rotation control output signal for controlling pitch, roll and yaw in target application using a pitch, roll and yaw rotation output signal generated by displacement of the control member.

However, in many other applications, such as drone flight, if only four command axes are required, the user input is divided in different ways depending on whether the hand controller is stabilized by a non-dominant hand or mounted on the fixed base of the controller combined with the forearm brace. Can be. For example, when using the forearm brace to support the hand controller and provide a reference frame, the second member is used to control the drone's y-axis movement, but the first control member is used to control the x-axis movement and yaw. It may be desirable. Since the individual input "devices" of the controller are easily programmable, the user can select the desired combination of inputs and axes.

In some embodiments, configuration parameters 104d may be received from an external computing device (not shown) operated by the user. For example, the external computing device can be pre-configured with software for interfacing with the controller 102 and/or signal conversion system 104. In other embodiments, configuration parameters 104d may be entered directly by the user using a display screen included in controller 102 or signal conversion system 104. For example, the first control member 102a and/or the second control member 102b can be used to navigate a configuration menu for defining configuration parameters 104d.

Referring now to FIGS. 2 and 3A-3C, a method 400 for controlling a control target is shown using one as a one-handed controller. The controller shown in FIGS. 3A-3C can be displaced to generate a first set of control outputs and is positioned on the first control member and the first control member held by the user's hand to grip the first control member Represents a one-handed controller having a second control member that is manipulated by the thumb of a hand to generate a second set of control outputs. Any one-handed controller described herein can be used with the methods described in connection with these figures, unless specifically stated otherwise. As in the case of other methods described herein, various embodiments may not include all of the steps described below, may include additional steps, and the steps may be ordered differently. Therefore, the arrangement of the specific steps shown in FIG. 2 should not be interpreted as limiting the range controlling the movement of the control target.

Method 400 begins at block 402, where input is received from a user. As described above, the user can grip the first control member by hand while using the thumb on the second control member. 3A to 3C, the user grips the first control member 204 with the hand 402a while extending the thumb 402b through the thumb channel defined by the second control member 208. can do. Also, the user can place the finger 402c on the control button 206. A person skilled in the art has shown a specific embodiment having a second control member 208 positioned for thumb operation and a control button 206 for finger operation, but repositioning of the second control member 208 (e.g., a finger other than a thumb) (For operation by), repositioning of control button 206 (e.g., for operation by fingers other than the fingers shown in FIGS. 3A-3C), other embodiments including additional control buttons and various other features It is also within the scope of the present disclosure.

In one embodiment, the input from the user at block 402 of the method 400 includes one or more rotational inputs (ie yaw inputs, pitch inputs and roll inputs) and one or more provided by the user using, for example, a controller. Translational input (ie, movement along the x-axis, y-axis, and/or z-axis). The user can reposition the first control member to provide a rotational input and reposition the second control member to provide a translational input. The controller is "integrated" in that it can be operated with one hand of the user. In other words, the controller allows the user to simultaneously provide the rotational input and the translational input with one hand without cross-coupled input (ie, the output from the hand controller is "pure").

As described above, the rotational and translational inputs are various devices such as light, rotational and/or linear potentiometers, inductively coupled coils, physical actuators, gyroscopes, accelerometers and photo detectors for detecting various other devices known to one or more skilled artisans. It can be detected using. Specific examples of the movement of the first control member and the second control member and their results on the control target 106 are described below, however, as described above, any movement of the first control member and the second control member is required by the user. It may be reprogrammed (including reprogramming a reference frame by exchanging a coordinate system based on a user's needs) or repurposed accordingly, so the discussion below is only an example of one embodiment of the present disclosure.

Reference is now mainly made to FIGS. 3A-3C but with continued reference to the method 400 of FIG. 2 and the control system 100 of FIG. 1, the controller 200 is presented in more detail. In one embodiment, the controller 200 may be the controller 102 described above with reference to FIG. 1. The controller 200 includes a base 202 that includes a first control member mounting portion 202a extending from the base 202 and forming a first control member mounting cavity 202b. The base 202 can be mounted to the support, for example, using an opening 202c positioned in a spaced-around orientation around the circumference of the base 202 and configured to receive a fastening member such as a screw. Alternatively, dovetail fittings with guide mounting and removal or other mechanical, magnetic or other adhesive fastening mechanisms known in the art can be used. The first control member 204, which may be the first control member 102a described above with reference to FIG. 1, is a base engaging member 204a located in the first control member mounting cavity 202b as shown in FIG. 3B. It is coupled to the base 200 through. In the illustrated embodiment, the coupling between the base coupling member 204a and the first control member mounting portion 202a is shown and described as a ball-joint coupling, however, those skilled in the art between the base 202 and the first control member 204 It will be appreciated that various other combinations of are within the scope of this disclosure. In one embodiment, an elastic member 205 such as a spring, for example, is first controlled in the first control member mounting cavity 202b to provide elastic movement up or down along the longitudinal axis of the first control member 204. It may be located between the member 204 and the base 202. In addition, an elastic member may be provided from the elastic member 205 to the opposite side of the base coupling member 204a to limit the upward movement of the first control member 204. In some embodiments, the entrance to the first control member mounting cavity 202b may be smaller than the base engagement member 204a such that the first control member 204 is secured to the base 202.

The first control member 204 includes an elongated first section 204b extending from the base engaging member 204a. The first control member 204 also includes a gripping portion 204c that is coupled to the first section 204b of the first control member 204 opposite the first section 204b from the base engaging member 204a. . The gripping portion 204c of the first control member 204 includes a top surface 204d located opposite the gripping portion 204c from the first section 204b of the first control member 204. In the illustrated embodiment, the top surface 204d of the gripping portion 204c is also the top surface of the first control member 204. The gripping portion 204c defines a second control member mounting cavity 204e extending from the top surface 204d into the gripping portion 204c. The control button 206 is located on the first control member 204 at the junction of the first section 204b and the gripping portion 204c. Although a single control button 206 is shown, those skilled in the art will recognize that multiple control buttons can be provided at different locations on the first control member 204 without departing from the scope of the present disclosure.

The second control member 208, which may be the second control member 102b described above with reference to FIG. 1, is a first control member engaging member located in the second control member mounting cavity 204e as shown in FIG. 3B. It is coupled to the first control member 204 via 208a. In the illustrated embodiment, the engagement between the first control member engagement member 208a and the first control member 204 is shown and described as a ball-joint engagement, however, those skilled in the art can control the control member 204 and the second control member ( It will be appreciated that various other combinations between 208) are within the scope of this disclosure. In one embodiment, an elastic member 209 such as a spring, for example, is provided with a second control member mounting cavity (2) to provide the elastic member up or down in a direction generally perpendicular to the top surface 204d of the gripping portion 204c. 204e) may be located between the second control member 208 and the first control member 204. In some embodiments, the entrance of the second control member mounting cavity 204e is such that the first control member engaging member 208a is such that the second control member 208 is secured to the first control member 204 and extends from the first control member. ).

The second control member 208 includes a support portion 208b extending from the first control member engaging member 208a. The second control member 208 also includes an actuating portion 208c coupled to the support portion 208b of the first control member 204 that is opposite the support portion 208b of the first control member engaging member 208a. do. In the illustrated embodiment, the working portion 208c of the second control member 208 forms a thumb channel that extends through the working portion 208c of the second control member 208. Although a specific operating portion 208c is shown, those skilled in the art will recognize that the operating portion 208c can have a different structure and include various other features while remaining within the scope of the present disclosure.

3B shows cabling 210 extending from the second control member 208 through the controller 200 and extending to the base 202 through the first control member 204 (connected to the control button 206). City. Although not explicitly shown, those skilled in the art will recognize that some or all of the features of the controller 102 described above with reference to FIG. 1 may be included in the controller 200. For example, features of the rotation module 102d and the translation module 102e, such as detectors, switches, accelerometers and/or other components for detecting movement of the first control member 204 and the second control member 208, are: It may be positioned adjacent to the base engagement member 204a and the first control member engagement member 208a to detect and measure the movement of the first control member 204 and the second control member 208 described above. Also, the controller processor 102c and the transmitter 102f can be located in the base 202, for example. In one embodiment, a cord that includes a connector can be coupled to the cabling 210 and act to connect the controller 200 to a control system (eg, control system 100). In other embodiments, the transmitter 102f may allow wireless communication between the controller 200 and the control system as described above.

As shown in FIGS. 3A-3C, a user follows the first control member 204 along line A using his hand 402a to provide a pitch input to the controller 200 [eg, a controller ( In combination with the base 202 for the controller 200 by tilting the gripping portion 204c of the first control member 204 along line A for the bottom portion of the first control member 204 for 200] It can be moved back and forth. As shown in FIGS. 3A-3C, the user first controls in arc B on his or her hand 402a (eg, in space for the controller 200) to provide a yaw input to the controller 200. By rotating the gripping portion 204c of the member 204, the first control member 204 can be rotated back and forth in combination with the base 202 to the controller 200. As shown in FIGS. 3A-3C, the user follows the first control member 204 along line C using his hand 402a to provide roll input to the controller 200 (eg, a controller ( In combination with the base 202 for the controller 200 by tilting the gripping portion 204c of the first control member 204 along line B for the bottom portion of the first control member 204 for 300] Can be moved sideways. Additionally, additional input may be provided using other features of the controller 200. For example, the elastic member 205 uses the first control member 204 to compress the elastic member 205 to provide a first input, and uses the first control member 204 to extend the elastic member 205 If desired, a neutral position of the first control member 204 can be provided to provide a second input.

As shown in FIGS. 3A-3C, a user follows line E using thumb 402b to provide x-axis input to controller 200 (eg, in engagement with first control member 204) ] The second control member 208 can be moved back and forth. As shown in FIGS. 3A-3C, the user follows line D using thumb 402b to provide y-axis input to controller 200 (eg, in engagement with first control member 204) ] The second control member 208 can be moved back and forth. As shown in FIGS. 3A-3C, the user follows line F using thumb 402b to provide z-axis input to controller 200 (eg, from elastic member 205 in some embodiments) In combination with the first control member 204, including the resistance of the] can move the second control member 208 up and down. In one embodiment, when the elastic member 209 compresses the elastic member 209 using the second control member 208, the first z-axis input for moving the z-axis of the control target 106 in the first direction When the elastic member 209 is extended using the second control member 208, the second z-axis input for moving the z-axis of the control target 106 in the second direction opposite to the first direction is provided. To provide, a neutral position of the second control member 208 can be provided.

Method 400 proceeds to block 404 where a control signal is generated and then transmitted based on the user input received at block 402. As described above, the controller processor 102c and the rotation module 102d may generate a rotation movement output signal in response to detecting and/or measuring the rotation input described above, and the control processor 102c and the conversion module 102e ) May generate a translational movement output signal in response to detecting and/or measuring the translational input described above. In addition, the control signals may include absolute deflection or displacement of the control member, speed of deflection or displacement of the control member, duration of deflection or displacement of the control member, deviation of the control member from the central deadband, and/or various other control signals known in the art. Indications may include. For example, the control signal may be generated on the basis of the rotational and / or translational inputs or input of the BLUETOOTH ^® protocol. Once generated, the control signal can be transmitted as an RF signal by an RF transmitter according to the BLUETOOTH ^® protocol. Those skilled in the art will understand that RF signals can be generated and transmitted according to various other RF protocols such as ZIGBEE protocol, wireless USB protocol, and the like. In another example, the control signal can be transmitted as an IR signal, a visible light signal, or some other signal suitable for transmitting control information. (ZIGBEE ^® is a registered trademark of the ZigBee Alliance, a company association headquartered in San Ramon, California, USA).

The method 400 then proceeds to block 406 where the transceiver receives the signal generated and transmitted by the controller. In one embodiment, transceiver 102 of signal conversion system 104 receives control signals generated and transmitted by controllers 102 and 200. In embodiments where the control signal is an RF signal, the transceiver 104a includes an RF sensor configured to receive signals according to appropriate protocols (eg, BLUETOOTH ^® , ZIGBEE ^® , wireless USB, etc.).

In other embodiments, the control signal can be transmitted over a wired connection. In this case, the transmitter 102f of the controller 102 and the transceiver 104a of the signal conversion system 104 can be physically connected by cables such as universal serial bus (USB) cables, serial cables, parallel cables, dedicated cables, etc. have.

The method 400 then proceeds to block 408 where the control program provided by the conversion processor 104b of the signal conversion system 104 commands movement based on the control signal received at block 406. In one embodiment, the control program may convert the control signal to a movement command and the movement command may include a rotation movement command and/or a translation movement command based on the rotation movement output signal and/or the translation movement output signal of the control signal. have. Other individual features such as ON/OFF, camera zoom and shared capture can also be relayed. For example, the move command may specify parameters for defining the movement of the control target 106 in one or more DoFs. By using the example described above, when the user moves the first control member 204 back and forth along line A using his hand 402a (shown in FIGS. 3A-3C), the resulting control signal is controlled. It can be used to generate a movement command including a pitch movement instruction to modify the pitch of the control target 106 by the program. When the user rotates the first control member 204 back and forth around the longitudinal axis around the arc B using his hand 402a (shown in FIGS. 3A-3C), the resulting control signal is transmitted to the control program. Can be used to generate a movement command including a yaw movement instruction for modifying the yaw of the control target 106. When the user moves the first control member 204 sideways along line C using his hand 402a (shown in FIGS. 3A-3C), the resulting control signal is controlled by the control program ( 106) can be used to generate a move command, including a roll move command to modify the roll.

Further, when the user moves the second control member 208 back and forth along the line E using the thumb 402b (shown in Figs. 3A to 3C), the resulting control signal is along the x axis by the control program. It can be used to generate a movement command including an x-axis movement instruction to modify the position of the control target 106. When the user moves the second control member 208 back and forth along the line E using the thumb 402b (shown in Figs. 3A to 3C), the resulting control signal is controlled by the control program along the y axis. It can be used to generate a movement command including a y-axis movement instruction to modify the position of 106. When the user moves the second control member 208 sideways along the line D using the thumb 402b (shown in Figs. 3A to 3C), the resulting control signal is controlled along the z axis by the control program. It can be used to generate a movement command including a z-axis movement instruction to modify the position of the target 106.

The method 400 then proceeds to block 410 where the movement of the control target 106 is performed based on the movement command. In one embodiment, the user's point of view or virtual representation may be moved in a virtual environment based on a move command at block 410 of method 400. In other embodiments, the end effector, propulsion mechanism and/or vehicle's steering mechanism may be activated based on the move command at block 410 of method 400.

4A, 4B and 4C show a control target 410a, which may be the control target 106 described above with reference to FIG. 1, for example. As described above, the control target 410a is a physical vehicle that the user is located in the vehicle, a remotely operated vehicle in which the user remotely operates the vehicle from the vehicle, a virtual operated by the user by providing the user with a viewpoint in the virtual vehicle Vehicles, and/or various other control targets known to one or more skilled artisans. Using the above example (FIGS. 3A-3C), when the user moves the first control member 204 back and forth along line A using his hand 402a (shown in FIGS. 3A-3C), The movement command due to the generated control signal will cause the control target 410a to modify the pitch for arc AA, as shown in FIG. 4B. When the user rotates the first control member 204 back and forth around the longitudinal axis for arc B using his hand 402a (shown in FIGS. 3A to 3C), a movement command due to the generated control signal Will cause the control target 410a to correct the yaw for the arc BB shown in FIG. 4B. When the user moves the first control member 204 sideways along the line C using his hand 402a (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is a control target ( 410a) will cause the roll for arc CC shown in FIG. 4C to be modified.

Further, when the user moves the second control member 208 back and forth along the line E using his thumb 402b (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is controlled. It will cause the target 410a to move along the line EE (ie, x-axis) shown in FIGS. 4B and 4C. When the user moves the second control member 208 sideways along the line D using his thumb 402b (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is a control target. It will cause 410a to move along the line DD (i.e., y-axis) shown in FIGS. 4A and 4B. When the user moves the second control member 208 back and forth along the line F using his thumb 402b (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is the control target ( 410a) will move along the line FF (ie, z-axis) shown in FIGS. 4A and 4C. In some embodiments, control button 206 and/or other control buttons on controller 102 or 200 can be used, for example, to operate other systems (eg, weapon systems) within control target 410a.

FIG. 4D shows a control target 410b which may be, for example, the control target 106 described above with reference to FIG. 1. As described above, the control target 410b may include a physical device or other tool that performs movement according to a signal transmitted from the controller 102 or 200. Using the above example (FIGS. 3A-3C), when the user moves the first control member 204 back and forth along line A using his hand 402a (shown in FIGS. 3A-3C), The movement command due to the generated control signal will cause the control target 410b to rotate the tool member or end effector 410c around the joint 410d along the arc AAA shown in FIG. 4D. When the user rotates the first control member 204 back and forth around the longitudinal axis for arc B using his hand 402a (shown in FIGS. 3A to 3C), a movement command due to the generated control signal The silver control target 410b will cause the tool member or end effector 410c to rotate about the joint 410e along the arc BBB shown in FIG. 4D. When the user moves the first control member 204 sideways along the line C using his hand 402a (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is a control target ( 410b) will rotate the tool member or end effector 410c about the joint 410f along the arc CCC shown in FIG. 4D.

Further, when the user moves the second control member 208 back and forth along the line E using his thumb 402b (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is a tool. The member or end effector 410c will cause it to move along the line EEE (ie, x axis) shown in FIG. 4D. When the user moves the second control member 208 back and forth along the line E using his thumb 402b (shown in FIGS. 3A to 3C), the movement command due to the generated control signal is the control target ( 410b) will move along the line EEE shown in FIG. 4D (ie, the y axis through joint 410f). When the user moves the second control member 208 sideways along the line D using his thumb 402b (shown in Figs. 3A to 3C), the movement command due to the generated control signal is the absence of the tool. Or, the end effector 410c may be moved along the line DDD (ie, z axis) shown in FIG. 4D. In some embodiments, control buttons 206 and/or other control buttons on controller 102 or 200 may be used to perform operations using, for example, tool member 210c. In addition, those skilled in the art will recognize that the tool member or end effector 410c shown in FIG. 4D can be replaced or supplemented with various tool members (eg, surgical tools, etc.) without departing from the scope of the present invention. As described above, the control target 410a may include a camera on or adjacent the tool member or end effector 410c to provide a field of view to allow navigation to the target.

Referring now to FIG. 5, a method 500 for controlling a control target is shown. As in the case of other methods described herein, various embodiments may not include all of the steps described below, may include additional steps, and the steps may be ordered differently. Therefore, the arrangement of the specific steps shown in FIG. 5 should not be interpreted as limiting the range controlling the movement of the control target.

Method 500 can begin at block 502 where a rotation input is received from a user. The user can provide a rotational input by repositioning the first control member 204 of the controller 200 (FIGS. 3A-3C) similar to the one described above. In some embodiments, the rotation input can be detected manually by a physical device such as an actuator. In other embodiments, the rotation input can be detected electrically by a sensor, such as an accelerometer.

Method 500 can proceed concurrently with block 504 where a translational input is received from a user. The user can provide translational input by repositioning the second control member 208 of the controller 200 similarly to that described above. The rotation input and translation input can be provided simultaneously by the user using one hand of the user. In some embodiments, the translational input can be detected manually by a physical device such as an actuator.

In one embodiment, rotation and translation inputs may be provided by a user viewing the current location of the control target 106 (FIG. 1) on the display screen. For example, the user can view the current location of the surgical device presented in the virtual representation of the patient on the display screen. In this example, rotation input and translation input can be provided using the current view on the display screen as a reference frame.

The method 500 then proceeds to block 506 where a control signal is generated based on the transmitted rotational input and translational input. When the rotation input is manually detected, a control signal can be generated based on the rotation input and the translation input detected by a plurality of actuators, the actuators being the first control member 204 and the second control member 208 The mechanical force applied to is converted into an electrical signal to be interpreted as a rotational input and a translational input, respectively (FIGS. 3A to 3C). When the rotational input is detected electronically, a control signal can be generated based on the rotational input detected by the accelerometer and the translational input detected by the actuator.

In one embodiment, the control signal may be generated based on the input rotation and the translation type according to the BLUETOOTH ^® protocol. Once generated, the control signal can be transmitted as an RF signal by an RF transmitter according to the BLUETOOTH ^® protocol. Those skilled in the art will understand that RF signals can be generated and transmitted according to various other RF protocols such as ZIGBEE ^® protocol, wireless USB protocol, and the like. In another example, the control signal can be transmitted as an IR signal, visible light signal, or other signal suitable for transmitting control information.

Still referring to FIG. 5 but referring to FIG. 1, method 500 proceeds to block 508 and transceiver 104a of signal conversion system 104 receives a control signal. When the control signal is an RF signal, the transceiver 104a includes an RF sensor configured to receive the signal according to an appropriate protocol (eg, BLUETOOTH ^® , ZIGBEE ^® , wireless USB, etc.). In other embodiments, the control signal can be transmitted over a wired connection. In this case, the transmitter 102f and the transceiver 104a are physically connected by cables such as universal serial bus (USB) cables, serial cables, parallel cables, dedicated cables, and the like.

The method 500 then proceeds to block 510 where the conversion processor 104b commands the movement at 6 DoF based on the received control signal. Specifically, the control signal can be converted into a movement command based on the rotation and/or translation input of the control signal. The movement command may specify parameters for defining a user's viewpoint or movement of a virtual expression in one or more DoFs in a virtual 3D environment. For example, if the second control member is repositioned upward by the user, the resulting control signal can be used to generate a movement command to move the viewpoint of the surgical device upward along the z axis within the 3D representation of the patient body. In another example, when the first control member is tilted to the left and the second control member is repositioned down, the resulting control signal moves the surgical device to the left while moving the surgical device down along the z axis in the 3D representation of the patient's body. Can be used to generate a move command to roll. Any combination of rotational and translational input can be provided to generate a movement command with various parameter combinations in one or more DoFs.

The method 500 then proceeds to block 512 where proportional movement is performed in the virtual and/or real environment based on the movement command. For example, in the virtual representation of the patient, the viewpoint of the surgical device may be repositioned according to a movement command, and the viewpoint corresponds to a camera or sensor attached to the surgical device. In this example, the surgical device can also be repositioned on the patient's body according to the movement of the surgical device in a virtual representation of the patient's body. The integrated controller allows the surgeon to navigate the surgical device at 6-DoF in the patient's body with one hand.

Referring now to FIG. 6 in conjunction with FIG. 1, a method 600 of configuring a controller is shown. As in the case of the other methods described herein, various embodiments may not include all of the steps described below, but may include additional steps, and the steps may be ordered differently. Therefore, the arrangement of the specific steps shown in FIG. 6 should not be interpreted as limiting the range controlling the movement of the control target.

The method 600 begins at block 602 where the controller 102 is connected to an external computing device. The controller 102 can be connected via a physical connection (eg, USB cable) or any number of wireless protocols (eg, BLUETOOTH ^® protocol). The external computing device may be pre-configured with software for interfacing with the controller 102.

The method 600 then proceeds to block 604 where configuration data is received by the controller 102 from an external computing device. The configuration data can specify configuration parameters such as gain (ie sensitivity), start speed (ie delay), deadband (ie neutral) and/or limit (ie maximum angular displacement). The configuration data can also assign movement commands to the control target to movement of the first control member and the second control member. Configuration parameters may be specified by the user using software configured to interface with the controller 102.

The method 600 then proceeds to block 606 where the operating parameters of the controller 102 are adjusted based on the configuration data. The operating parameters can be stored in memory and can then be used by the controller 102 to remote control the control target as described above with respect to FIGS. 2 and 5. In some embodiments, the method 600 sets a “trim” and establishes a translational speed (eg cm/sec) or direction change (eg deg/sec), or the [display or controller 102 itself Low] “Auto-sequence” is initiated by auto-pilot movement.

In other embodiments, the controller 102 can include an input device that allows the user to configure the operating parameters of the controller 102 directly. For example, the controller 102 can include a display screen with a configuration menu navigable using the first control member 204 and/or the second control member 208 (FIGS. 3A-3C ).

Computer readable program products stored on tangible storage media can be used to facilitate any of the above-described embodiments, such as the control program described above. For example, embodiments of the present invention may include a computer readable medium such as an optical disc, such as a compact disc (CD), digital versatile disc (DVD), diskette, tape, file, flash memory card or any other computer readable storage device. Can be stored in. In this example, the execution of the computer readable program product may cause the processor to perform the methods described above with respect to FIGS. 2, 5 and 6.

In the following example of a one-handed controller, various aspects allow the controller to separate individual translations from attitude adjustments in the control requirements of computer aided design (CAD), drone flight, various types of computer games, virtual and augmented reality, and other virtual and physical tasks. And provides tactile feedback when moving away from the "null command" or zero input position while requiring precise movement through space.

For example, the extended operation of a controller that uses a thumb for independent control inputs can cause "hitchhiker's thumb" fatigue problems. By adding a third control member, such as an interlocking paddle for the third, fourth and fifth fingers (or some subset thereof) of the user's hand to grip or rotate while gripping the first control member, 2 The controller can be held up (in the + z direction) or pressed up to provide relief. Further, the third control member and the second control member may be interlocked to push the second control member down to push the paddle or the third control member. Therefore, the thumb and accessory fingers are in a dynamic balance, so they can be quickly mastered. Alternatively, the index finger can be interlocked to provide a counterweight to the thumb. Since the user generally has a fine motor control function of the index finger, the index finger can be used to provide a desired relief while finely controlling the displacement along the Z axis.

In another embodiment, a one-handed controller can be used as part of a control system with a wrist or forearm brace that serves as a reference for yaw difficult to measure with a rotating axis, particularly an inertial measurement unit (IMU). For example, the IMU in the body of the first control member of the hand controller can work well for pitch and roll but the yaw can be noisy. This can be improved by software modification, but some exemplary embodiments described herein are connected to the wrist, allowing the potentiometer or optical encoder to accurately measure all three axes of rotation. In some variations of the forearm brace implementation, the index finger loop used to open and close the grip on an object in the virtual world can be used.

The hand controller examples presented in connection with FIGS. 7-20B and variations thereof can be used in applications as presented above in the preceding section such as flight simulation, CAD, drone flight, and the like. Optional additional features that can be used alone or in some cases in combination with one or more of the other features include: adjustable z spring force and self centering/zeroing capability; A relatively large x-y gantry at the top of the joystick for the second control member; A replaceable or resizable thumb loop for the second control member; Stabilization of the forearm or wrist for ambulatory use (potentiometer for X/Y/Z conversion, such as drone application and integration with virtual/augmented reality, Hall effect sensor or optical encoder); And a mouse-based implementation for improved CAD object manipulation.

Referring now to FIGS. 7-11, the controllers 700, 900, 1000 and 1100 show different representative embodiments of a one-handed controller with three control members, one of which provides Z-axis secondary control.

The controllers shown and described in FIGS. 12-20B as well as the exemplary controllers 700, 900, 1000, 1000, translational input representing movement along the X, Y, and Z axes are preferably received from the user's thumb. The thumb maps more closely to the brain than the other parts of the hand. This controller utilizes dexterity to provide input along the X, Y and Z axes. The thumb movement is based on the first control member in the form of a joystick in these examples. The translation can be separated from the posture control of the target control object. Squeezing the third control member positioned on the first control member may cause the user's thumb to be lifted by applying one or more of the second, third, fourth, or fifth fingers in the user's hand to apply upward force or upward movement. Support. The force and movement of the third control member is transmitted or applied to the second control member through the inner engagement and thus to the thumb.

These embodiments use an inertial measurement unit to measure the displacement of the first control member. As an alternative, however, these controllers are when the controller is mounted to pivot to the base, in which case sensors for sensing roll, pitch and yaw can be positioned within the base or when coupled to the user's wrist to provide a reference frame. , In this case one or more sensors for pitch, roll and yaw can be configured to use an external sensor that can be incorporated into the coupling. Examples of these arrangements are shown in subsequent figures.

In the following description, the first control member may be generally referred to as a “joystick” or a “control stick”, which is structurally similar to at least a part of a joystick of a previously known type where the first control member is gripped and This is because, in some cases, other types of joysticks can be gripped by the human hand and act like other types of joysticks because they are intended to be displaced (translated and/or rotated) or moved to indicate pitch, roll, yaw or motion. . However, this should not imply other structures that can be found in conventional joysticks, and is intended to mean only elongated structural elements that can be gripped.

Referring now to the embodiments of FIGS. 7 and 8A, 8B, and 8C, the controller 700 can be gripped by at least two of the thumb of the hand and at least two of the third, fourth, and fifth fingers 703 ) And a first control member that can be referred to as a joystick having a pistol-like grip body 702 formed by an upper portion 705 located above the gripped position. Within the first control member, only the dotted line is schematic because the internal structure with one or more integrated inertial measurement units (IMU) 704 (body 702) for sensing pitch, roll and yaw control of the first control member is not visible. It is indicated as]. This embodiment includes an optional quick connection 718 for connection to a base or other structure. This particular embodiment also includes an optional button, such as a trigger 706 (located to be actuated by the index finger) and a posture hold button 708. It can be operated by the finger of the hand holding the controller or the other hand of the user.

The second control member is mounted on top of the first control member in a position that can be manipulated by the thumb of the person holding the body 702 of the first control member. The second control member allows the user to move up or down to generate input indicating that the user is moving along the z axis as well as displaced from front to back and left to right in order to generate input indicating moving along the y and x axes. And a gantry device 710 for displacement. In this particular example, the gantry device 710 is mounted on a platform 712 that moves the gantry device up and down. Different methods of moving the platform (or gantry 710) up and down can be used, but this particular example places the gantry 710 at one end of the hinge platform 712. This allows the gantry device to move down relative to the first control member. When the gantry is pushed down, the platform 712 moves downward to indicate the input for controlling the Z axis, while pulling the thumb loop (not shown) moves in the opposite direction along the Z axis.

Part of the Z-axis input device on this controller also includes a third control member 714 in this example. In this example, the third control member takes the form of a paddle 716, where the second, third, fourth and/or fifth finger of the user's hand grips the first control member around the body 702. The paddle 716 can be selectively squeezed by the user when gripping the controller. The paddle 716 and the platform 712 can be spring loaded to allow the paddle and platform to be in the zero position such that the z-axis input indicates movement in any direction from the zero position. The third control member acts as secondary Z-axis control. The third control member is interlocked or coupled with the second control member. By including a third control member such as a finger paddle 716, the second control member is “flattened” to relieve the user's hitchhiker thumb fatigue and provide fine motor control of the user input along the Z axis, while providing X and Y axes. It enables simultaneous movement of the gantry along the axis.

8A and 8B show the controller 700 with multiple elements removed to more clearly represent the cooperative movement of the paddle 716 and the platform 712. In FIG. 8A, the platform is in the fully depressed position, and in FIG. 8B, the platform 712 is in a fully extended position corresponding to the overall movement of the second control member along the z axis. In FIG. 8A the paddle 716 is in a fully extended position relative to the body 702, and in FIG. 8B the paddle 716 is fully pressed against the body 702.

8C, a perspective view of the controller 700 with half of the body removed, along with most other internal components, is shown to reveal an example of a mechanical linkage. In this example, paddle 716 pivots about pivot axis 720. The lever 722 connected to the paddle 716 but opposite to the pivot axis 720 is pivotally connected to the linkage 724. The other end of the linkage 724 is connected to a lever arm 726 to which the platform 712 is connected. The platform 712 pivots about a pin forming the axis 728. Although not shown in the figure, the spring can be placed in the area indicated by reference numeral 730 to deflect the paddle 716 and thus deflect the entire linkage to a zero or neutral position. Additional springs may also be used to provide balance and deflect the linkage to position the paddle and gantry in the Z-axis zero position.

9, 10 and 11, the controllers 900, 1000 and 1100 share the same external components that make up the first and third control members. Each has a body 902 shaped like a joystick or pistol grip that is intended to grip and hold the first control member and, generally speaking, the user's hand. Each includes a paddle 904 (eg, turning from the top) that can be actuated by one or more fingers of the user holding the first control member, such as controller 700. Each also has a programmable button 905 where the second finger loop can be replaced.

Similarly, each has a second control member over the body. Each second control member includes a platform 906 that moves up and down (by a hinge or other mechanism) to provide a Z axis input. However, each is different in the properties of the second control member. The controller 900 is a thumb loop mounted on a gantry 906 that can be displaced back and forth and left and right to provide x and y axis inputs, and can also displace the gantry in both directions along the z axis by raising and lowering the thumb ( 908). This thumb loop can be made to different sizes, preferably using inserts (not shown) that can accommodate different sizes. The thumb loop shown in any controller in this disclosure can be resized using an insert or other adjustable mechanism if desired. The controller 1000 of FIG. 10 uses a control member 1002 similar to that shown in FIG. 7. The controller 1100 of FIG. 11 uses a trackball 1102 mounted on the platform 906 for x and y axis input. Press the trackball down to enter the z axis. Paddle 904 is used to provide input in different directions along the z axis.

In each controller 900, 1000, 1100 as well as the hand controller shown in the remaining figures, the second and third control members are mechanical linkages disposed within the body of the first control member, such as the linkages shown in FIG. 8C. It is combined by. However, the linkages of FIG. 8C are generally intended to represent such linkages because different arrangements and number of links may be used depending on the particular geometry of the various parts and elements. Other types of couplings or transmissions can be used to transfer displacement and force between the primary and secondary z-axis control elements in any controller shown and described in FIGS. 7-20B. These can be electrical and magnetic transmissions that transmit positions, selectively apply forces, and combine any two or more of these types, as well as other types of mechanical transmissions (eg, cables). However, the mechanical linkage is advantageous because providing a direct coupling between the two control members is relatively simple, reliable, and immediately transmits force and position to provide a comfortable dynamic balance.

Moreover, all controllers shown in FIGS. 7 to 11 as well as all controllers shown in FIGS. 12 to 20b have a re-centering mechanism for each degree of freedom to provide the user with a sense of “zero” or null commands. It is preferred to have. When the control member is displaced along one of the degrees of freedom, it preferably generates tactile feedback such as force, shake or other haptic signals of the control members to return the control members to a position for zero input (zero position). The mechanism may simply consist of a spring that reacts with the spring force or may be an active system that senses displacement and/or force and generates responsive movement, force, other types of vibrational haptic feedback, or combinations thereof.

Although not shown in FIGS. 7-11, each controller 700, 900, 1000 and 1100 as well as other controllers shown in the remaining figures include at least the elements shown in FIG. 1. For example, sensors for detecting displacement of the first, second and third control members (eg inertial measurement unit, potentiometer, optical encoder, etc.); A processor for processing signals from sensors; And a transmitter for transmitting an input signal from a controller, which may be radio frequency, optical or wired (electrical or optical). Such sensors may take the form of inertial measurement units, potentiometers, optical encoders, and the like.

In any embodiment of the controller described with respect to FIGS. 1-20B, user feedback may be provided from the controller by one or more of a number of mechanisms. For example, haptic vibrations can provide subtle vibration feedback. Force feedback can provide feedback with some or all degrees of freedom. Ambient heat and air can provide radiant heat and blown air. Virtual reality multi-sensing integration can generate precise control within the virtual world. The integrated audio can provide sound feedback from a control target, such as a drone or other target device. The controller can also provide surface heat and cold to provide feedback through the heat and cooling sensations. The user interface (UI/UX) may optionally include an integrated touch screen and visual indicators such as lighting, flash color, and the like.

Turning now to FIGS. 12, 13 and 14, three modified base structures 1200, 1300 and 1400 to which any of the controllers 700, 900, 1000 and 1100 can be connected are illustrated. Those shown in any other figures may also be applied. In the figure, the controller 900 is used as an example, but other controllers can be configured to be used with any base. The base can provide one or more of the following functions: function as a reference frame for measuring displacement of the first control member of the controller; Signal conditioning circuitry for interface sensors to measure displacement, processors for executing software programmed processes such as those described above and elsewhere, batteries or other sources for power, interfaces for other hardware, and for wireless communication Ability to accommodate transmitters and receivers.

12 shows a mobile two-hand controller system. The two-handed controller provides a consistent and known reference frame (stabilized by a non-dominant hand) on the go, such as walking, skiing, running, driving. For certain types of applications, such as inspection, security, and drone missions for cinematography, the hand controller can be mounted on a platform that can be fixed or otherwise stabilized by the user's hand. The platform can include secondary control and a display unit if desired. In one example, all 6-DoF inputs can be reacted through the platform. With this arrangement, this example of the control system facilitates movement through the air, such as a fighter pilot with intuitive (unintentional perception) input.

A hand controller, such as hand controller 900, is plugged (or alternatively permanently mounted) into the top surface of the base. In order for the user's other hand to grip while using the hand controller 900, for example, a handle or grip 1204 in the form of a pistol grip is provided on the opposite side of the base. (Other shapes and types of handles may also be contemplated by those skilled in the art) This allows other users' hands, such as non-dominant hands, to hold or stabilize the base. The base may optionally include additional user interface elements 1206 and 1208 such as keys, buttons, dials, touchpads, trackpads, trackball balls, and the like. The display 1210 is mounted on the base or integrated into the base in a location visible to the user. One or more video or graphic images of a controlled application such as a live video or game from a drone can be displayed in real time on a display. Alternatively, the base can include a mount on which a smartphone or similar device can be placed or mounted. Alternative or optional features include a combination of any one or more of the following features. The base can be reconfigured for both hands with quick detachment of the joystick and two mounting points. It can be asymmetric (shown) or symmetrical with sufficient space for secondary control. This may include a smartphone attachment with a tilt function on the top surface. This includes a secondary joystick that enables pan and tilt control of the drone camera, and a capacitive or pressure deadman switch that can prevent or stop movement of the target when the user grips the joystick and is not engaged. can do. It may also include a large display mount and surface area for secondary control. In an alternative embodiment, the grip or handle can be located on the more midline of the controller, thus reducing some off-axis moments. In other embodiments, the base may be stabilized by mounting the base on the user's body instead of holding the base. Examples of mounting points of the base to the user's body include chest mounts, belts, and clothing articles.

13 is an example of a base that can be moved to provide another input, in this case a mouse with additional input buttons 1304 and 1306. In this example, a secondary connection point 1308 for a hand controller is provided to accommodate both left-handed and right-handed users. One example is to navigate a 3D image on a computer screen, and the existing mouse function is used to move the cursor in the field of view and manipulate drop-down menus, while the controller 900 is a 3-way with multiple degrees of freedom. Used to reorient and/or move D objects.

14 shows an example of a wired fixed base, one-handed controller 1400.

Although not required, each figure shows an exemplary embodiment in which the controller can be quickly connected to the base at its bottom. In each example of the base, the controller 900 is connected to small levers 1202, 1302 and 1402 of joystick type. This lever can be used to provide pitch, roll and yaw inputs where the sensor is located within the base, but this is not necessary. Instead (or additionally) can be used to center the first control member in the zero position and provide feedback to the user. Signals can be communicated from sensors within the controller using an RF or wired connection between the controller and the base.

15 shows an example of an embodiment of a hand controller 1500 such as a controller 900 that includes an index finger loop 1502 in addition to a thumb loop 1503 that functions as a second control member. This detection loop can be used to control the opening or closing of a physical or virtual end effector, such as a handle on an object in the virtual world. The design can be fitted with a very low profile ergonomically on the palm and optimized for virtual/augmented reality or drone flight. For example, a detection loop for opening and closing the end effector may be added, which may be beneficial for virtual/augmented reality applications.

In addition, FIG. 15 schematically shows an attachment 1504 for placement on the user's forearm 1506. The coupling 1508 between the attachment 1504 and the hand controller 1500 supports the hand controller and pivots the controller 1500 against the forearm attachment 1504 even when removed from the base station using a potentiometer or optical encoder It is possible to accurately measure the angular displacement of the pitch, roll and yaw of the controller 1500 when connected to the point 1510. On the other hand, indexing of the wrist or forearm is possible. In one embodiment, the hand controller does not use an IMU to detect one or more pitches, rolls or yaws using other types of sensors instead. Alternatively, the system can use two or more IMUs and software filtering of the data to measure relative displacement and command flight control.

Moving the reference point through physical or virtual space through the hand controller requires constant insight into displacement with all the degrees of freedom being controlled. In other words, it is important to know the yaw for the drone and where the "zero input" always moves along the x, y and z directions. Other flight systems, such as virtual and augmented reality, computer games and surgical robots, can simultaneously require six independent degrees of freedom (X, Y, Z, pitch, yaw, roll). Also, in the case of drone flight and virtual reality and especially augmented reality, the ability to move while maintaining accurate control of the reference point (POR) is desirable.

In some embodiments, index finger loop 1502 may be configured to constrain the index finger to prevent the index finger from moving. Index restraint provides stability of the thumb loop 1503 against X, Y and Z translational movements and can further facilitate fine independent control.

15-20B show some representative embodiments of a two-part control system: a consistent and known reference frame (user's) while the user or the user's arm is moving or accelerating, such as walking, skiing, running or driving. Handheld controller providing a fixed wrist) and a forearm attachment in the form of a suitable brace or configured to mount on the user's forearm or wrist.

In the example shown in these figures, the forearm attachment can take one of a number of forms. For example, it may include a brace, a wrist wrap (which wraps around the forearm or wrist and can be secured using, for example, Velcro), a slab-bracelet or other item that follows at least a portion of the forearm. However, it may also include a relatively rigid support structure. Forearm attachments may be referred to as braces, cuffs, or “gauntlets” because they are structurally and/or functionally similar in some respects to these items. However, the use of these terms should not imply more structure than is indicated or required in the statement function.

The hand controller and forearm attachment are connected by a mechanical linkage, post or support. In one embodiment, it is a manual linkage; In other embodiments it is not. One type of manual mechanical linkage used in the examples described below is a two-axis gimbal pivot with a centering spring and potentiometer to measure displacement. Alternatively, it is possible to use a cable embedded in a partial glove, a double piston mechanism (compression spring), a pneumatic cylinder or a manual stiffener/batten. In an example, the linkage unit gives the user a force to allow the user to sense zero input on at least one, at least two or all three rotation axes on the joystick.

A small inertial measurement unit (IMU) can also be placed in the forearm attachment and the primary control member of the controller, allowing for the detection of pure differential (relative) movement between the forearm and the controller, for example. For example, the noise signal can be managed by oversampling and subsequent decimation with digital adaptive filtering to mechanically measure the relative movement of the arm and hand in a noisy environment (while hiking, running or otherwise moving). . However, in the embodiments described below that can measure one or more pitches, rolls, or yaws with other mechanisms, the IMU may only need one or two of the rotational displacements of the primary control member.

In alternative embodiments, passive or active mechanical feedback can be used to inform the user of displacement in a given axis of rotation. Feedback can also include vibration haptic and force feedback.

For drone flight, one embodiment includes two gimbal degrees of freedom at the wrist and two gimbal degrees of freedom at the thumb: wrist pitch (X or before/after) and wrist yaw (left/right pivot); Thumb/Z paddle (up/down translation) and thumb Y (left/right translation).

Alternatively, displacement in the roll of the forearm can be measured by physically sensing the movement of the radius bone relative to the ulna by a gauntlet extending at least half over the forearm. This may be desirable for augmented reality applications, but drone flight does not require full six degrees of freedom control, including measuring forearm rolls. The yaw and Y translation inputs described above may be replaced in user preferences according to flight testing and personal preferences.

The thumb loop/"Z paddle" is maintained while using the "gantry" on the top of the joystick to measure the intended displacement to the side. Other methods of measuring forearm rolls include EMG detection of forearm muscle potential, conformal forearm wrap with pressure sensor to select differential contour of forearm with rotation function, differential IMU or IMU to indicate rotation and camera system (wrist to elbow). There is a combination. The latter solution requires vibrating haptic or force feedback to inform the user of the roll's zero position.

One or more of the following features can be incorporated into any of the embodiments described herein: reconfigurable with both hands; Symmetrical shape with buttons available on both sides; Quick wear and tear off of the wrist wrap (don and doff) or disconnection of the joystick; Attach a smartphone with a tilt function to the wrist wrap; A secondary joystick at the base of the joystick to allow (pan)/tilt of the drone camera; A secondary joystick that can retract and extend from the base of the joystick, such as a ballpoint pen; A capacitive or pressure activated dead man switch that makes the controller safe; A modular joystick that can be removed and placed on a table top base, or stand-alone or on a base of other types of specific functions such as those described above.

In various embodiments described herein, such as gimbals, mechanisms that allow pivoting of the control member to indicate displacement, such as a potentiometer and Hall effect sensor to measure angular displacement and torsion springs for selectively centering the control member It may include a sensor. Preferably, for example, the coupling or linkage connecting the joystick to the gimbal can be made adjustable or adaptable to accommodate different size joysticks for different sized users.

The universal smartphone holder may also include a holder attached to a forearm attachment or a bracket mounted on a brace.

The hand controller of the following figure includes a one-hand control device of six degrees of freedom, with a first control member (or device such as a joystick) in the form of a joystick and a second control member (loop, gantry, trackball, touch pad) for the user's thumb Or other input device) is configured to be used by one or more fingers of the same hand and the Z-axis movement is enlarged by another third control member moving in the opposite direction in connection with the second control member.

For example, additional features useful for drone flight or virtual/augmented reality applications may include forearm braces that can detect pitch, roll and yaw, for example using a mobile potentiometer, Hall effect or optical encoder. Like the smart device (smartphone, tablet) holder, the pan/tilt control can also be integrated into the controller. The base structure to which the hand controller is attached may also include a second handle for the user's other hand or non-dominant hand to allow sensing of the mobile potentiometer or optical encoder.

Alternative solutions for yaw precision include induction magnetic wrist wristbands, differential IMUs, IMU's software filtering to reduce yaw related noise, reaction wheel (high precision gyro) and inertia by potentiometer or optical encoder (high precision yaw gyro) One or more of the balance urine may be included. Software filtering of IMU data can include dynamic re-zeroing.

The control signal of the controller can be further augmented by an additional input. For example, a “connection sensor” mounted on the head or body can be used. It can use a grid-type infrared input or other optically based deformation such as RF directivity or omni-directional tracking. The connection sensor can be an interactive virtual reality application or a head mount, such as a wrist mount. You can use "dot" tracking for more general body position input. The type of dot tracking can be, for example, magnetic or photogrammetric.

Referring now to FIG. 16, controller 1600 is substantially similar to the other hand controller described in the previous paragraph. In this example, it is connected to a forearm attachment 1602 that includes a video display 1604 and additional user input 1606 in the form of buttons and other types of user input. The connection 1608 between the controller 1600 and the forearm attachment 1604 maintains the relative position of the controller 1600 together with the attachment 1604, and uses an internal sensor or an external sensor mounted at the end of the connection 1608 It is a relatively rigid linkage that provides a pivot point to measure pitch, yaw and roll.

Reference is now made to FIGS. 17 and 18 showing an alternative embodiment of the cuff 1700 acting as a forearm attachment. In this example, hand controller 1702 is schematically illustrated. This represents any hand controller described herein. Any hand controller described herein can be configured for use in this example. In this example, the controller is connected to the pitch sensor 1706 attached to the cuff 1700 with a mechanical linkage or strut 1708 that is positioned below the controller and can be adjusted as indicated by the length adjuster 1710. The end of the mechanical linkage 1708 is attached to the forearm attachment using a spherical bearing 1712 to allow different angles. Like the length adjuster 1710, this will be tightened if the user satisfactorily adjusts the position of the controller.

This example takes into account that the IMU is not used in the controller, at least for pitch and yaw measurements. Rather, the yaw, roll and pitch sensors are integrated into the base 1703 of a mechanical connection or support between the controller and the floor or forearm attachment of the hand controller 1702. Such a sensor may take the form of a gimbal with one or more detectors, such as a potentiometer or Hall effect sensor in one example, and a spring (eg torsion spring) providing feedback from zero position. In this example, yaw sensor 1714 is integrated at the bottom of controller 1702, but may also be integrated at the base of link or strut 1708 where pitch sensor 1706 is placed. The invisible roll sensor may be disposed at the base of the linkage 1708 on which the pitch sensor is disposed, or at the bottom or base portion of the controller 1702.

Referring now to FIGS. 19A, 19B, 19C, and 19D, a control system 1900 with a specific example of a dual gimbal link 1902 between the forearm attachment 1904 and the hand controller 1906 (FIG. 19D only). An embodiment of is shown. The double gimbal links 1902 attach gimbals 1908 and 1910 disposed at 90 degrees to each other to measure pitch and yaw, respectively. The hand controller is connected to a hand controller mount 1912 that serves as a lever arm and is connected to a yaw gimbal 1910. Forearm attachments, which include sleeves or braces 1914, to which straps can be attached to attach to the arm, are supported by lever arms 1916 connected to one side of the pitch gimbal 1908. The hand controller mount 1912 shown in FIG. 19C is a variation of that shown in FIGS. 19A and 19B in that it is adjustable. Phone holder 1918 is mounted or attached to arm attachment 1904 for viewing by a user. In this example, you can adjust the phone holder to hold phones of different types and sizes.

Turning now to FIGS. 20A and 20B, another example of a control system similar to FIGS. 19A-19D is shown. In this example, the control system uses a pitch gimbal 2002 and a yaw gimbal 2004 that respectively measure the pitch and yaw associated with the bracket 2006 in a manner similar to that shown in FIGS. 19A-19D. The pitch gimbal 2002 is mounted to the forearm attachment in the form of a brace 2008 disposed near the pivoting position of the wrist joint when gripping and rotating the controller 2010. The brace is held by the strap 2012. The brace acts as a ballast as in the above-described embodiment. The controller 2010 is mounted on a length adjustable lever arm 2014. In this example, the controller 2010 has a body 2016, like other hand controllers of the above-described embodiment, the body being grippable by a user used to input rotational displacement (two are measured by the gimbal) The first control member, the second control member on the top of the body 2016 in the form of a thumb loop 2018 for X, Y, Z input. In front of the bottom of the body is a joystick 2022 that can be used, for example, for camera pan and tilt input, or tool manipulation.

Referring now to FIGS. 21A-21F, an exemplary embodiment of a two-hand controller system 2100 that is operable to be operated by a user's hand at up to 6 DoF is presented. The controller system 2100 is operable to move and hold by the user's hand, such as the user's non-dominant hand, that does not grip the first control member 2106. However, the controller system 2100 can be positioned on a static surface, secured to the user's body, or mounted on the user's body by a harness, belt, or other such method. The controller system 2100 includes a base structure 2102 and a one-handed controller 2104. The controller system 2100 includes a base structure 2102 and a one-handed controller 2104. The controller system 2100 functions and operates in the same manner as the controllers described above, such as at least the controllers 700, 900, 1000, 1100 and controllers described below. The controller 2104 includes a second control member 2108 in addition to the first control member 2106. The controller 2104 may further include a third control member (not shown) similar to other third control members described herein. The first control member 2106 is attached or coupled to the base to allow the user to grip the first control member and push it to allow rotational displacement relative to the base with up to three independent rotational degrees of freedom. The second control member 2108 can be displaced along the Z axis alone or in combination with the third control member.

The controller system 2100 includes a mount 2110 that can be placed or mounted to communicate with a controller or a target that a similar device controls, or to run an application to interact with the controller system, such as changing parameters. Includes additionally. The phone may be wired to the base, but for example, the phone may be connected to the base wirelessly. The mounting portion 2110 is composed of a bracket having a first end connected to the base 2102 and a second end for mounting a smartphone. Mounting portion 2110 may have a top portion that extends over the top portion of hand controller 2104. The hand controller 2104 is angled toward the front of the base structure 2102 and the mounting portion 2110 is angled toward the rear of the base structure 2102. In another embodiment, the mounting portion 2110 extends laterally past the back of the base structure 2102. In one embodiment, the mounting portion is adjustable to allow positioning of the smartphone.

Referring now to FIGS. 22A-22F, a controller system 2200, such as a controller system 2100 having a one-handed controller capable of entering with 4-6 degrees of freedom while allowing the other hand of the user to maintain the base 2202. An exemplary embodiment of is shown. The controller system 2200 can thus be used in a mobile environment and can be held by the user's other hand rather than the hand holding the first control member. The base 2202 of the controller system 2200 is shaped like a tablet. However, unlike the other control systems described herein, when one end of the hand controller is coupled to the lower end for rotational displacement around the pivot point, the first control member of this embodiment has up to three extending through the pivot. It is coupled to the base by a pivot 2202, such as a ball joint, gimbal, or other device near the mid-point to allow rotational displacement with up to 3 degrees of freedom by turning or rotating around an orthogonal axis.

The controller 2200 functions similarly to the previously disclosed controller and others described herein. The controller 2204 includes a first control member 2206 that can be rotatably displaced to a maximum of 3 degrees of freedom (or less than 3 degrees in another embodiment if desired) and a first to 3 degree of freedom depending on the embodiment. 2 includes a control member 2208. Although not shown, the controller 2204 may further include a third control member similar to other third control members described above and below. The controller system 2200 further includes a mount 2210 disposed on the top surface of the base structure 2202 on which a smartphone or similar device can be placed or mounted.

The hand controller 2204 is shown in a stowed position with the hand controller 2204 oriented in a position parallel to the base structure 2202. For operation, hand controller 2204 is rotated about pivot 2212 to an operating position (not shown). In one embodiment, the user can set a desired null position once rotated to a desired null operating position or to a preset and stored position. The sensor for detecting the rotational displacement of the first control unit can detect the movement of the loading position, but other sensors or switches may be used.

Referring now to FIG. 23, a one-handed controller as described above and below can be designed with a third control member having an arrangement and size that can be controlled by different sized hands. The controller 2300 includes a first control member 2302, a second control member 2304, and a third control member 2306, each of which can operate or function similarly to the other controllers described above. The first hand 2310 is smaller than the second hand 2312. The first height 2314 represents a non-limiting approximate height range for the index fingers of different sized hands. The second height 2316 represents a non-limiting approximate height range for fingers 3, 4 and 5 of different sizes. In an alternative embodiment, the control number 2306 can be placed on the gripping portion of the first controller at a higher position so that it can be pressed by the index fingers of the user of different hand sizes.

Referring now to FIGS. 24A and 24B, a schematic diagram of a four degree of freedom hand controller suitable for flight, such as a drone aircraft, is shown. Two versions of 2400A and 2400B are shown. Although not shown as being connected to the base, it is shown and connected to the base as described above or used with a forearm brace. Each version is similar. Each has a first control member 2402 that is connected to the base 2404 and is intended to be gripped by the user's hand. Each has a second control member 2406 mounted on the first control member for displacement by the user's thumb or index finger, but in the figure, the second control member is shown in the form of a thumb loop. In other embodiments, the thumb loop can be replaced with other types of control elements. The difference between them is the position on the first control member of the third control member, referred to as 2408A in Fig. 24A and 2408B in Fig. 24B. The third control member 2408A is located at the bottom to be operated by the user's third, fourth and/or fifth fingers. The third control member 2408B is positioned higher so that it is pressed or displaced by the index finger of the user gripping the first control member. Unlike other examples of hand controllers described herein, the second control member 2404 of each example 2400A and 2400B generally moves with only one degree of freedom along an axis oriented along the central axis of the first control member. The third control member 2406 is coupled to the second control member by the linkage 2410 to enable the user to dynamically balance the second and third control members. When a force is applied to one of the control members, a force is applied to the other control member. The sensor is used to detect the direction of displacement of the second control member and the third control member. In this example, a change in the magnetic field generated by one or more magnets or other elements (not shown) on the linkage 2410 or one magnet or other element (or both) on the second and third control members is detected. To do this, a circuit board 2412 is provided in the first control member to which one or more Hall effect sensors 2414 are mounted.

25A and 25B show this dynamic balance for hand controller 2500. The base is omitted, but will be combined with the base or forearm base as described above for sensor rotational displacement. 24A and 24B, like the several other controllers described above, the controller has three control members: a first control member 2502, a second control member 2504, and a third control member 2506 ). The user's hand 2508 grips the first control member in an area of the first member that is specially formed or suitable for gripping. The user's thumb 2510 is used to displace the second control member 2504 along the Z axis. In this example, a thumb loop is used to allow the user's thumb to pull the second control member up. However, it is not necessary to use a thumb loop. The third control member is mounted at the bottom on the gripping portion and is sufficiently large that any one or more of the user's third, fourth or fifth fingers 2514 pressed the third control member inward toward the first control member. Alternatively, the third control member can be mounted high enough so that the user's index finger 2512 can press it. In Fig. 25A, the second control member is extended upward, and the third control member is pressed. The user can cause this displacement by pressing the third control member, pulling the second control member up, or a combination of the two. In Fig. 25B, the second control member is pushed down toward the first control member, causing the third control member to push outward from the first control member. The ability to push the third control member back by squeezing with one or more fingers allows displacement to be more easily controlled by the user than by the thumb.

In some embodiments of each of the controller systems 2100, 2200, 2400 and hand controllers 2500 and 2600, as well as some of the other controllers described herein, the first control member of the hand controller has up to three degrees of freedom (or other implementations). In the example, if desired, the degree of freedom (less than 3 degrees of freedom) can be displaced rotationally. Similarly, the second control member of the hand controller is pivoted to indicate translational movement (movement up and down along the Z axis as well as left and right and back and forth along the X and Y axes for the first control member), and/or displacement It can be configured to be displaced by one degree of freedom, two degrees of freedom, or up to three degrees of freedom using a rotational motion for Unless otherwise indicated, each control system can be configured to allow different degrees of freedom of displacement for each of the first and second control members in alternative embodiments. When used, the third control member can be used in one embodiment to dynamically balance the displacement of the second control member along the Z axis, which is generally aligned with the central axis of the first control member. However, in an alternative embodiment, the displacement of the third control member can be used as other control inputs and not linked to the second control member.

26 is a schematic diagram of a hand controller 2600 such as controller 2500 shown in FIGS. 25A and 25B. It includes first, second and third control members 2602, 2604, 2608 that operate as described above in connection with other hand controllers. However, as with the controller 2500, the first control member is an extension 2610 in which there is a display displaying information transmitted from the target, such as an aerial drone (in this example, it can be attached as a separate member, but is integrally formed) It includes. Examples of information that can be displayed include movement direction, altitude, and other location or direction information.

Referring now to FIG. 27, in various examples of the controller system given above, each hand controller is connected to a base, frame, brace or other element, with the first control member having up to 3 degrees of freedom around up to 3 axes It is reacted to cause displacement, which provides a reference frame for measuring the displacement. In most of these exemplary embodiments, a handle controller, such as a representative controller 2700 having a first control member 2702, a second control member 2704 and a third control member 2706, uses a connector to base or It can be constructed or manufactured to be selectively removable to other devices. In this representative example, the bottom of the hand controller is plugged into connector 2708. The connector may include a contact 2710 for electrically connecting to transmit signals and power to the hand controller. The connector is connected to a post 2712 pivoted using, for example, a rocker, ball, gimbal or other mechanism, such that rotation or angular displacement of the post of at least one degree of freedom around up to three mutually orthogonal axes with a common origin at the pivot point. Detects. A button, detent or other retaining mechanism, represented by a button 2714 that actuates the detent to engage the base of the hand controller, can be used to connect and release the hand controller. This particular example is intended to be connected to the post of a ball joint or gimbal to allow user displacement of the first control member.

28 and 29 schematically show an example of a gimbal 2800 that can be used with a sensor to simultaneously allow displacement and displacement measurement of the control member, particularly the first control member, into two degrees of freedom. The gimbal can be mounted to the base with a post 2802 for engaging with the hand controller or a hand controller with the post connected to the base. The gimbal can also be configured for use with a sensor for measuring displacement of the second control member.

In this particular exemplary embodiment, the gimbal 2800 provides two detents 2804 in the form of a ball deflected inwardly toward the ball 2806 by, for example, a spring 2805. Note that only a pair of detents are shown. The other pair will be oriented perpendicular to the visible pair. A single detent can be used for each direction of rotation, but the pair provides balance. The ball 2806 is mounted within the socket 2808 (although it can be used to secure the ball with a rotational freedom of 1), the ball can rotate freely within the socket with 2 degrees of freedom. The base 2809 represents a structure for mounting the gimbal, and a hand controller can respond to it. The cap 2810 extends over the spherical outer surface of the socket so that the post can pivot the cap. The extension or key 2812 fits into the complementary opening formed in the ball 2806, and the angular displacement of the post 2802 also rotates the ball. When the ball is rotated to the null position in both directions of rotation, all detents engage grooves 2814. The two pairs of detents that engage and disengage provide tactile feedback to the user in a null position on two axes of rotation (eg pitch and roll). For sensor rotation, one or more magnets 2816 (when in the zero position) are placed on the bottom ball 2806. This allows the PCB 2818 with at least one Hall effect sensor 2820 to be positioned closely to detect and measure the angular displacement of the ball in two rotational degrees of freedom, thereby generating a signal indicative of the displacement. One advantage of this arrangement is that the springs and joysticks are high, allowing the gimbal bottom to be retained to place the Hall effect sensor. Other types of sensors may be replaced by Hall effect sensors and magnets in other embodiments. This gimbal mount can be used in other hand applications as well as the hand controller described herein.

In the embodiment of the hand controller described above, when the hand controller is mounted to the base, the first control member is connected to a ball joint or gimbal, for example for rotational displacements and up to 3 degrees of freedom around up to 3 axes. The base in the illustrated embodiment also includes wired and/or wireless interfaces for communicating signal condition circuits, processes, memory (to store data and program instructions) and power, as well as control signals generated by the controller system. can do. 1 is a non-limiting example of such a component.

Accordingly, a system and method has been described that includes a controller that allows a user to provide rotation and translation commands with six independent degrees of freedom using one hand. Systems and methods can be used in a wide variety of control scenarios. Although multiple control scenarios are described below, this example is not intended to be limiting, and many control scenarios use one hand to rotate and translate movements even if fewer than all control outputs are required for all six degrees of freedom. You will recognize the benefits of being able to provide.

In one embodiment, the control systems and methods described above can be used in a wide variety of medical applications. Although a number of medical applications are described below, these examples are not intended to be limiting, and those skilled in the art will appreciate the benefits that many other medical applications can provide for rotation and translational movement using one hand. In addition, in this embodiment, in addition to the rotational and translational movement provided using the first and second control members described above, the control buttons may include, for example, end effector capture, biopsy, suture, radiographing, photography, and/or one or more. It can be configured for tasks such as various other medical tasks known to those skilled in the art.

For example, the above-described control system and method may provide a control system for performing laparoscopic surgery and/or a method for performing laparoscopic surgery. Conventional laparoscopic surgery is performed using a control system that requires both hands of the surgeon to operate the control system. The use of the control systems and/or methods described above provides several advantages when performing laparoscopic surgery, including fine dexterity manipulation of one or more surgical instruments, potentially without a straight and rigid path to the end effector.

In another example, the control systems and methods described above can provide a control system for performing minimally invasive or natural orifice surgery and/or a method for performing minimally invasive or natural orifice surgery. Conventional minimally invasive or natural orifice surgery is performed using a control system that requires both hands of the surgeon to operate the control system. The use of the control systems and/or methods described above provides several advantages when performing minimally invasive or natural orifice surgery, including fine dexterity manipulation of one or more surgical instruments without potentially a straight and rigid path to the end effector. .

In another example, the control systems and methods described above can provide a control system for performing prenatal intrauterine surgery and/or a method for performing prenatal surgery. Conventional prenatal surgery is performed using a control system that requires both hands of the surgeon to operate the control system in a very strict range. The use of the control systems and/or methods described above provides several advantages when performing prenatal surgery, including fine dexterity manipulation of one or more surgical instruments, potentially without a straight and rigid path to the end effector.

For any of the above surgical examples, the control systems and methods described above can provide highly stable control systems for performing microscopic surgery and/or methods for performing microscopic surgery. Using the control system and/or method described above provides several advantages when performing microscopic surgery, including highly accurate camera and end effector pointing.

In another example, the control systems and methods described above can provide a control system for performing interventional radiation and/or a method for performing interventional radiation. Conventional interventional radiation is performed using a control system that requires both hands of the surgeon to operate the control system. The use of the control systems and/or methods described above provides several advantages when performing interventional radiation, including highly accurate navigation through interventional radiation. In another example, the control systems and methods described above can provide a control system for performing interventional cardiology and/or a method for performing interventional cardiology. Conventional interventional cardiology is performed using a control system that requires both hands of the intervention expert to operate the control system. The use of the control systems and/or methods described above provides several advantages when performing interventional cardiology, including highly accurate navigation through a vascular tree with one hand.

In another example, the control system and method described above can provide a control system including Hansen/Da Vinci robot control and/or a method for performing Hansen/Da Vinci robot control. Conventional Hansen/Da Vinci robot control is performed using a control system that requires both hands of the surgeon to operate the control system. The use of the control system and/or method described above provides several advantages when performing Hansen/Da Vinci robot control, including fluid, continuous translation, and reorientation without shaking the end effector for longer operation.

In another example, the control systems and methods described above can provide a control system for performing 3D or 4D image guidance and/or a method for performing 3D or 4D image guidance. Conventional 3D or 4D image guidance is performed using a control system that requires both hands of the surgeon to operate the control system. Using the control system and/or method described above provides several advantages when performing 3D or 4D image guidance, including fluid, continuous translation and reorientation without shaking the end effector for longer operation.

In another example, the control systems and methods described above can provide a control system and/or a method for performing endoscopic inspection to perform endoscopic examination. Conventional endoscopy is performed using a control system that requires both hands to operate the control system. The use of the control systems and/or methods described above provides several advantages when performing endoscopy including fluid, continuous translation and reorientation without shaking the end effector for longer operation. This also applies to colonoscopy, cystoscopy, bronchoscopy, and other flexible examination ranges.

In one embodiment, the control systems and methods described above can be used in a wide variety of defensive or military applications. Although a number of defensive or military applications are described below, these examples are not intended to be limiting, and those skilled in the art will recognize the advantage that many other defensive or military applications can provide rotational and translational movement using one hand.

For example, the control systems and methods described above can provide a control system for an unmanned aerial system and/or a method for controlling an unmanned aerial system. Conventional unmanned aerial systems are controlled using a control system that requires both hands of the operator to operate the control system. Using the control system and/or method described above provides several advantages when controlling an unmanned aerial system, including accurate, intuitive, one-handed, non-combined motion in the airspace.

In another example, the control systems and methods described above can provide a control system for an unmanned underwater system and/or a method for controlling an unmanned underwater system. Conventional unmanned underwater systems are controlled using a control system that requires both hands of the operator to operate the control system. The use of the control system and/or method described above provides several advantages when controlling an unmanned submersible system, including accurate, intuitive, one-handed, non-combined motion in a submersible airspace.

In another example, the control systems and methods described above can provide a control system for a weapon aiming system and/or a method for controlling a weapon aiming system. Conventional weapon aiming systems are controlled using a control system that requires both hands of the operator to operate the control system. Using the control system and/or method described above provides several advantages when controlling the weapon aiming system, including aiming accurately and intuitively with one hand.

In another example, the control systems and methods described above can provide a control system for a counter-improvised-explosive-device (IED) system and/or a method for controlling an immediate explosion device system. Conventional instant explosive device systems are controlled using a control system that requires both hands of the operator to operate the control system. Using the control system and/or method described above provides several advantages when controlling an explosive device system immediately, including accurate and intuitive one-handed pointing or targeting.

In another example, the control systems and methods described above can provide control systems for heavy mechanized vehicles and/or methods for controlling heavy mechanized vehicles. Conventional heavy heavy-duty mechanized vehicles are controlled using a control system that requires both hands of the operator to operate the control system. Using the control system and/or method described above provides several advantages when controlling heavy mechanized vehicles, including aiming with accurate and intuitive one hand.

In another example, the control system and method described above can provide a control system for a pilot aircraft (eg, a rotary wing aircraft) and/or a method for controlling a pilot aircraft. Conventional pilot aircraft are controlled using a control system that requires both hands of the operator to operate the control system. The use of the control system and/or method described above provides several advantages when controlling a pilot aircraft, including accurate, intuitive, one-handed, non-combined motion within a pilot aircraft's airspace.

In another example, the control systems and methods described above can provide a control system for spacecraft rendezvous and docking and/or a method for controlling spacecraft rendezvous and docking. Conventional spaceship rendezvous and docking are controlled using a control system that requires both hands of the operator to operate the control system. The use of the control systems and/or methods described above provides several advantages when controlling spacecraft and docking, including single-handed, non-combined, combined motion within the airspace for spaceship rendezvous and docking.

In another example, the control systems and methods described above can provide a control system for air-to-air refueling (eg, boom control) and/or a method for controlling air-to-air refueling. Conventional air-to-air refueling is controlled using a control system that requires both hands of the operator to operate the control system. The use of the control system and/or method described above provides several advantages when controlling air-to-air refueling, including accurate, intuitive, one-handed, non-combined motion in the airspace for refueling.

In another example, the control systems and methods described above can provide a control system for exploration in a virtual environment (eg, operational and simulated war) and/or a method for controlling exploration in a virtual environment. Conventional searching in a virtual environment is controlled using a control system that requires both hands of the operator to operate the control system. The use of the control systems and/or methods described above provides several advantages when controlling navigation in a virtual environment including accurate, intuitive, one-handed, non-comparative combined motion within the virtual environment.

In one embodiment, the control systems and methods described above can be used in a wide variety of industrial applications. Although many industrial applications are described below, these examples are not intended to be limiting, and those skilled in the art will appreciate the benefits that many other industrial applications can provide for rotation and translational movement using one hand.

For example, the control systems and methods described above can provide control systems for oil exploration systems (eg, drills, 3D visualization tools, etc.) and/or methods for controlling oil exploration systems. Conventional oil exploration systems are controlled using a control system that requires both hands of the operator to operate the control system. Using the control system and/or method described above provides several advantages when controlling the oil exploration system, including accurate, intuitive, one-handed, non-combined motion within the formation.

In another example, the control system and method described above can provide a control system for an overhead crane and/or a method for controlling an overhead crane. Conventional overhead cranes are controlled using a control system that requires both hands of the operator to operate the control system. The use of the control systems and/or methods described above provides advantages in controlling overhead cranes, where uniaxial movement is often limited by increasing process speed and increasing accuracy.

In another example, the control system and method described above can provide a control system for a cherry picker or other mobile industrial lift and/or a method for controlling a cherry picker or other mobile industrial lift. Conventional cherry pickers or other mobile industrial lifts are often controlled using a control system that requires both hands of the operator to operate the control system and often allow translation in only one direction at a time (ie, x, y and/or z motion). do. Using the control system and/or method described above provides several advantages when controlling a cherry picker or other mobile industrial lift, including simultaneous multi-axis motion via a one-handed controller.

In another example, the control systems and methods described above can provide a control system for a fire fighting system (eg, water cannon, ladder truck, etc.) and/or a method for controlling the fire fighting system. Conventional fire fighting systems are often controlled using control systems that require both hands of the operator to operate the control system, and generally do not tolerate multi-axis redirection and translation. Using the control system and/or method described above provides several advantages when controlling a fire fighting system, including simultaneous multi-axis motion through a one-handed controller.

In another example, the control systems and methods described above can provide a control system for handling nuclear material (eg, glove boxes, fuel rods in a core, etc.) and/or a method for controlling nuclear material handling. Conventional nuclear material handling systems are controlled using a control system that requires both hands of the operator to operate the control system. The use of the control systems and/or methods described above provides several advantages when controlling nuclear material handling, including highly accurate, fluid, one-handed, multi-axis operation using sensitive materials.

In another example, the control systems and methods described above can provide control systems for steel fabrication and other high temperature processes and/or methods for controlling steel fabrication and other high temperature processes. Conventional steel fabrication and other high temperature processes are controlled using a control system that requires both hands of the operator to operate the control system. Using the control systems and/or methods described above provides several advantages when controlling steel fabrication and other high temperature processes, including highly accurate, fluid, one-handed, multi-axis operation with sensitive materials.

In another example, the control systems and methods described above can provide a control system for handling explosives (eg, in a mining application) and/or a method for controlling explosive handling. Conventional explosive handling is controlled using a control system that requires both hands of the operator to operate the control system. Using the control systems and/or methods described above provides several advantages when controlling explosive handling, including highly accurate, fluid, one-handed, multi-axis operation using sensitive materials.

In another example, the control systems and methods described above can provide a control system for a waste management system and/or a method for controlling a waste management system. Conventional waste management systems are controlled using a control system that requires both hands of the operator to operate the control system. Using the control systems and/or methods described above provides several advantages when controlling waste management systems, including highly accurate, fluid, one-handed, multi-axis operation using sensitive materials.

In one embodiment, the control systems and methods described above can be used in a wide variety of consumer applications. Although a number of consumer applications are described below, these examples are not intended to be limiting, and those skilled in the art will appreciate the benefits that many other consumer applications can provide for rotation and translational movement using one hand.

For example, the control systems and methods described above include consumer electronic devices such as Nintendo Wii ^® (Nintendo, America Inc. of Redmond, Washington), Nintendo DS ^® , Microsoft Xbox ^® (Microsoft Corporation, Redmond, Washington, USA), it is possible to provide a control system for controlling the Sony PlayStation ^® (Sony computer Entertainment Japan, Inc., Tokyo,) and one skilled in the art that other video console and / or known to the consumer electronic device. Conventional consumer electronic devices use a control system that requires both hands of the operator to operate the control system (eg, hand controller and keyboard, both hands on one controller, Wii ^® "nunchuck" z-phage I/O device, etc.) Is controlled. Using the control systems and/or methods described above provides several advantages when controlling consumer electronic devices, including the ability to accurately navigate through virtual spaces with fluidity, accuracy and speed via an intuitive hand controller. to provide.

In another example, the control systems and methods described above can provide a control system for 3D computer search and/or a method for controlling 3D computer search. Conventional 3D computer navigation is controlled using a control system where the operator needs both hands to operate the control system or does not allow fluid multi-axis movement through space. Using the control system and/or method described above provides several advantages when controlling 3D computer navigation, including highly accurate, fluid, one-handed multi-axis operation.

In another example, the control system and method described above can provide a control system for a radio control vehicle and/or a method for controlling a radio control vehicle. Conventional radio controlled vehicles are controlled using a control system that requires both hands of the operator to operate the control system. The use of the control system and/or method described above provides several advantages when controlling a radio controlled vehicle, including one-handed, non-combined vehicle combined motion, in an accurate, intuitive and one-handed airspace within the radio-controlled vehicle.

In another example, the control system and method described above may provide a control system for manipulating 3D CAD (Computer Aided Design) images and/or a method for controlling 3D CAD image manipulation. Conventional 3D CAD image manipulation is controlled using a control system that requires both hands of the operator to operate the control system or does not allow fluid multi-axis movement through the 3D space. The use of the control system and/or method described above provides several advantages when controlling 3D CAD image manipulation, including single-handed, non-combined motion combined within 3D airspace.

In another example, the control systems and methods described above can provide a control system for general aviation and/or a method for controlling general aviation. Conventional general aviation is controlled using a control system that requires both hands and feet of the operator to operate the control system. The use of the control systems and/or methods described above provides several advantages when controlling general aviation, including single-handed, non-combined combined motion within the airspace for general aviation.

It is understood that the above modifications can be made without departing from the scope of the present invention. Although certain embodiments have been shown and described, modifications may be made by those skilled in the art without departing from the spirit or teachings of the invention. The embodiments as described are merely illustrative and not restrictive. Many variations and modifications are possible and are within the scope of the present invention. Also, one or more elements of exemplary embodiments may be omitted, combined with one or more elements of other exemplary embodiments, or replaced entirely or in part. Accordingly, the scope of protection is not limited to the described embodiments, but is limited only by the following claims, which will include all equivalents of the subject matter of the claims.

Claims (49)

A controller for generating control inputs for at least 4 degrees of freedom,
A first control member shaped to be gripped by a user's hand, the first control member configured to be displaced by the user with at least two degrees of freedom relative to a predetermined reference frame;
A first sensor for measuring displacement of the first control member with each of at least two degrees of freedom;
A second control member mounted on the first control member for displacement with respect to the first control member with two or more degrees of freedom, the second control by a thumb or index finger of a user hand while gripping the first control member The second control member positioned on the first control member at a position allowing displacement with at least one of two or more degrees of freedom of the member;
A second sensor for measuring the displacement of the second control member at each of two or more degrees of freedom for the first control member; And
A third control mounted on the first control member for displacement by any one or more fingers of the user hand that is not used for displacement of the second control member while the user's hand grips the first control member As a member, the third control member includes the third control member, the third control member being engaged with the second control member to displace the second control member when depressed. According to claim 1,
The third control member includes a paddle formed on a grip. The method of claim 1 or 2,
The second control member comprises a gantry. The method of claim 1 or 2,
The second control member includes a ball mounted to rotate on the controller for rotation by a user's thumb or index finger, and the second sensor rotates the ball around two axes extending through the center of the ball A controller that further measures. The method according to any one of claims 1 to 4,
Each of the first and second sensors for measuring displacement includes one or more detectors for measuring angular or linear displacement. The method according to any one of claims 1 to 5,
The first control member is coupled to a gimbal, and the first sensor for measuring displacement comprises one or more Hall effect detectors arranged to measure the angular displacement of the gimbal around at least two axes. The method according to any one of claims 1 to 6,
Each of the first and second sensors for measuring displacement is each comprised of one or more detectors, essentially inertial measurement units, accelerometers, optical encoders, gyroscopes, photo detectors, rotary potentiometers , A linear potentiometer, inductively coupled coils, physical actuators, gyroscopes, switches, load cells, electromechanical sensors and transducers, each selected from the group consisting of transducers. The method according to any one of claims 1 to 7,
And the first sensor for measuring the displacement of the first control member measures the displacement of the first control member with respect to each of the three rotation axes. The method according to any one of claims 1 to 8,
The second control member is translated into each of the at least two degrees of freedom, and the second sensor for measuring the displacement of the second control member changes the displacement of the second control member along each of the at least two translation axes. Measuring, controller. The method of claim 9,
And the second sensor measures translation of the second control member along three translation axes relative to the first control member. The method according to any one of claims 1 to 10,
The second control member is disposed on top of the first control member for manipulation by the thumb. The method according to any one of claims 1 to 11,
The controller further comprises a detection loop mounted on the first control member to provide stability. The method according to any one of claims 1 to 12,
The first control member is coupled to the base for angular displacement relative to the base. The method of claim 13,
And further comprising a connector for removably coupling the first control member to the base, the connection comprising electronic connectors for transmitting electrical signals between the first control member and the base. The method of claim 13,
Further comprising a gimbal mounted in the base to connect to the first control member, the first sensor for measuring the displacement of the first control member is angular rotation of the gimbal by the first control member Measuring, controller. The method according to any one of claims 1 to 15,
The first control member is in the form of a joystick, a controller. The method according to any one of claims 1 to 16,
The first control member has an elongate shape with a portion configured to be gripped by a user's hand, the third control member being positioned on a grip to be pressed by one or more fingers of the user. The method according to any one of claims 1 to 17,
The first sensor measures the displacement of the first control member and generates a signal indicative of the displacement of each degree of freedom of the first control member independently of each of the at least two degrees of freedom of the first control member;
And the second sensor measures displacement of the second control member and generates a signal indicative of each displacement of the second control member at each of at least two degrees of freedom of the second control member. The method according to any one of claims 1 to 18,
The second control member comprises a thumb loop. The method according to any one of claims 1 to 19,
The controller controls a target having at least 4 movement degrees of freedom, receives from the first and second sensor signals indicating the displacement amount of the first and second sensors and corresponds to each of at least four movement degrees of freedom of the target And further comprising a processor for generating control inputs for transmission to the target. A controller displaceable with at least 4 degrees of freedom to generate control inputs for controlling a target having at least 4 degrees of freedom,
A first control member shaped to be gripped by a user's hand and displaceable with respect to a reference frame known as at least two of the four degrees of freedom;
A second control member mounted to the first control member at a displacement position relative to the first control member with one or more of the four degrees of freedom by a finger of the user's hand while gripping the first control member, wherein The second control member, wherein a finger is the thumb or index finger of the user and the displacement of the second control member to one of the one or more degrees of freedom requires bending and extending of the finger;
While the user's hand grips the first control member, it is mounted to the first control member for displacement by any one or more fingers of the user's hand that is not used for displacement of the second control member. A third control member, the third control member being coupled with the second control member to displace the second control member with one of the one or more degrees of freedom;
A sensor for measuring a displacement amount of the first and second control members with each of the at least 4 degrees of freedom independently of other degrees of freedom among the at least 4 degrees of freedom; And
At least one processor for receiving indications of the measured displacement amounts of the first and second control members at each of the at least 4 degrees of freedom from the sensors and at least generating control inputs to the target, each control input A controller comprising the at least one processor, corresponding to one of the at least four degrees of freedom. The method of claim 21,
And the second control member is displaceable by the user's finger with two or more of the at least four degrees of freedom. The method of claim 21 or 22,
The two or more degrees of freedom of the first control member include a rotational displacement of the first control member around at least two axes of rotation, each of the one or more degrees of freedom of the second control member being fixed relative to the first control member And translational movement of the second control member relative to the reference frame. The method according to any one of claims 21 to 23,
The first control member includes a joystick, and the second control member is mounted to an upper end of the joystick. The method according to any one of claims 21 to 24,
And further comprising a visual display extending from an upper end of the first control member. (none) (none) (none) (none) As a controller,
It is movable with three degrees of freedom and in response provides a first set of three independent control inputs, having an elongated shape for gripping with one hand of the user. A first control member; And
A second control member extending from the gripping portion of the first control member, movable in at least two independent degrees of freedom, in response to providing a second set of at least two independent control inputs, the two One of the second set of independent control inputs corresponds to the translational movement of the second control member along a first translation axis, the second set of control inputs being independent of the first set of control inputs, 2 control member; And
And an index finger loop for fixing the position of the index finger of the user with respect to the first and second control members. As a controller,
A first control having an elongate shape for gripping within the user's hand, but with a sensor for moving in three independent rotational degrees of freedom and generating a set of three independent rotational control inputs in response. absence;
Mounted on the first control member for movement by the thumb or index finger of the user hand when gripping the first control member with at least two independent translational degrees of freedom and in response to at least two independent translational controls A second control member having sensors for generating a second set of inputs, wherein the second set of control inputs are independent of the first set of control inputs; And
A sensor for mounting on the gripping portion of the first control member and generating an independent translational control input in response to being moved by one or more fingers of the user with independent translational freedom when placed in the gripping portion Wherein the control input comprises the third control member, independent of the control inputs for the first and second sets. The method of claim 31,
The first control member includes a joystick, and the third control member includes a paddle on the joystick. As a control system,
A controller shaped to be gripped by a user's hand, the controller being movable by the user at three rotational degrees of freedom, and in response to generating three rotational control inputs corresponding to each of the three rotational degrees of freedom; And
A frame configured to be removably mounted to a user's arm, the control system comprising the frame coupled with the controller to measure displacement of the controller by the user with the three degrees of freedom of rotation. The method of claim 33,
Further comprising an adjustable linkage for coupling the controller to the frame, the adjustable linkage comprises a single pivot point maintained at a fixed position relative to the frame to which the controller is coupled for rotational displacement. Having, control system. The method of claim 33 or 34,
The frame is coupled to the controller by a linkage having two or more rotational inputs, wherein the rotation axes extend through the user's wrist to sense movement of the controller by the user relative to the forearm. The method according to any one of claims 33 to 35,
The frame is configured to be removably mounted on the user's forearm, the control system. As a controller,
A first control member configured to be gripped by a user's hand;
A first providing a first set of independent signals measuring the displacement of the first control member around each of the at least two of the three axes of rotation and in response to indicating the measured displacement for each of the at least two axes of rotation 1 sensor;
Mounted on the first control member at a displacement position by the thumb or index finger of the user's hand while gripping the first control member along at least one translation axis of the three translation axes fixed relative to the first control member A second control member; And
And a second sensor that measures displacement of the second control member along at least one axis independently of the movement of the first control member and generates an independent control signal indicative of the measured displacement for each of the at least one axis. and,
The first control member is coupled to the base for rotational displacement relative to the base through a releasable connector. The method of claim 37,
Wherein the connector comprises connections for transmitting electrical signals. The method of claim 37 or 38,
A controller further comprising a gimbal mounted in the base to connect to the first control member, wherein the first sensor measures angular rotation of the gimbal. The method according to any one of claims 37 to 39,
The connector comprises electrical connections for communicating electrical signals between the first connector and the base. The method according to any one of claims 37 to 40,
The base is configured to be held by the user's hand without gripping the first control member. The method according to any one of claims 37 to 40,
The base can be mounted on a person, a controller. The method according to any one of claims 37 to 40,
The base comprises a computer mouse, a controller. The method according to any one of claims 37 to 43,
The base further comprises a mounting for a smartphone, a controller. As a controller,
A first control member configured to be gripped by a user's hand;
Measuring a rotational displacement of the first control member around each of at least two of the three rotational axes and in response to providing a first set of independent signals representing the measured displacement for each of the at least two rotational axes A first sensor;
Mounted on the first control member at a displacement position by the thumb or index finger of the user's hand while gripping the first control member along at least one translation axis of the three translation axes fixed relative to the first control member A second control member;
A second sensor measuring displacement of the second control member along at least one axis independently of the movement of the first control member and generating an independent control signal indicative of the measured displacement with respect to each of the at least one axis; And
And a re-centering mechanism coupled with one of the first and second control members to generate tactile feedback to a user in a null position in response to the displacement of the control member. The method of claim 45,
And the tactile feedback comprises one or more of force, shake or other haptic signals. The method of claim 45 or 46,
The recentering mechanism controls the control member to a null position. The method according to any one of claims 44 to 47,
The re-centering mechanism comprises means for sensing displacement and/or force and generating reaction movement, force, and other types of vibrational haptic feedback. As a controller:
A control member configured to be gripped by a user's hand;
Measuring a rotational displacement of a first control member around each of at least two of the three rotational axes and in response to providing a first set of independent signals indicative of the measured displacement for each of the at least two rotational axes sensor;
A gimbal to which the control member is coupled, comprising a ball mounted on a socket, the gimbal including a detent and a groove formed in the ball.

Images