CN117021072A - Redundant mechanical arm interaction method and system based on virtual reality technology - Google Patents
Redundant mechanical arm interaction method and system based on virtual reality technology Download PDFInfo
- Publication number
- CN117021072A CN117021072A CN202310867849.8A CN202310867849A CN117021072A CN 117021072 A CN117021072 A CN 117021072A CN 202310867849 A CN202310867849 A CN 202310867849A CN 117021072 A CN117021072 A CN 117021072A
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- handle
- redundant
- virtual reality
- movement amount
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000003993 interaction Effects 0.000 title claims abstract description 45
- 238000005516 engineering process Methods 0.000 title claims abstract description 28
- 238000004088 simulation Methods 0.000 claims abstract description 15
- 230000008569 process Effects 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 239000011521 glass Substances 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 2
- 230000008859 change Effects 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 210000000707 wrist Anatomy 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
- B25J13/02—Hand grip control means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1628—Programme controls characterised by the control loop
- B25J9/163—Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1689—Teleoperation
Abstract
The invention relates to a redundant mechanical arm interaction method and system based on a virtual reality technology, belonging to the technical field of man-machine interaction; the method comprises the steps of establishing a kinematic model of the redundant mechanical arm; designing a mechanical arm tail end position control mode of the handle; designing a mechanical arm configuration control mode of the handle; designing a mechanical arm joint control mode of a handle; in a simulation environment, connecting a VR head-mounted display with a handle controller; programming in a simulation environment, detecting the movement amount of the handle controller, and converting the movement amount into the movement amount for driving the joint angle of the mechanical arm through the three control modes. The invention solves the problems of poor presence feeling, low operation efficiency of operators and the like in the prior art, improves the presence feeling of operators in the human-computer interaction process, and improves the human-computer interaction efficiency.
Description
Technical Field
The invention belongs to the technical field of man-machine interaction, and particularly relates to a redundant mechanical arm interaction method and system based on a virtual reality technology.
Background
Man-machine interaction is an important link for realizing information communication between a person and a computer as well as between mechanical arms. From the language command interaction stage represented by typewriter and keyboard to mouse and GUI interaction and then to the multimedia multi-channel man-machine interaction mode represented by voice, gesture, sight and the like, the requirements of the robot are met from the gradual development of the human adaptation machine to the gradual development of the robot, and the improvement of the man-machine interaction mode also enables the operation of the mechanical arm by the human to be more efficient and convenient. At present, the interaction modes with the mechanical arm, such as a force feedback hand controller, a data glove, an exoskeleton and the like, fully exert the decision making capability and the operation capability of a person by introducing the person into an operation loop, solve the problems of redundant mechanical arm configuration selection and the like, and reduce hardware cost. Wherein, force feedback hand controller, such as: delta series, omega series, etc., are the most widely used interactive devices at present; the data glove can control the pose of the remote mechanical arm and feed back the contact force to an operator.
However, in the mode, operators use the visual picture of the two-dimensional screen as a reference to control the mechanical arm, and the mechanical arm lacks of feeling in the field and immersing. In recent years, with the advent of Virtual Reality (VR) technology, a helmet or glasses may display a three-dimensional scene, so that a wearer has an immersive sensation. The VR technology is used for man-machine interaction, so that man-machine interaction efficiency can be improved, and the presence of operators is improved.
Disclosure of Invention
The technical problems to be solved are as follows:
in order to avoid the defects of the prior art, the invention provides the redundant mechanical arm interaction method and the system based on the virtual reality technology, and an operator can change the precession amount of the handle controller into the variation amount of the tail end position, the joint position, the tail end gesture and the arm angle of the redundant mechanical arm by wearing the VR helmet/glasses and the handle controller, so that the operator can control the movement of the redundant mechanical arm more accurately and intuitively. The method comprehensively considers the requirements of various tasks and lays a good foundation for the actual operation of the later mechanical arm. The invention solves the problems of poor presence feeling, low operation efficiency of operators and the like in the prior art, improves the presence feeling of operators in the human-computer interaction process, and improves the human-computer interaction efficiency.
The technical scheme of the invention is as follows: a redundant mechanical arm interaction method based on a virtual reality technology comprises the following specific steps:
establishing a kinematic model of the redundant mechanical arm;
designing a mechanical arm tail end position control mode of the handle;
designing a mechanical arm configuration control mode of the handle;
designing a mechanical arm joint control mode of a handle;
in a simulation environment, connecting a VR head-mounted display with a handle controller;
programming in a simulation environment, detecting the movement amount of the handle controller, and converting the movement amount into the movement amount for driving the joint angle of the mechanical arm through the three control modes.
The invention further adopts the technical scheme that: the kinematic model is established according to an arm angle parameterization method, and an inverse kinematic model of the mechanical arm is obtained:
in the method, in the process of the invention, 0 T 7 representing a pose matrix of the robot arm tip relative to the base,represents the arm angle, Θ= [ θ ] 1 θ 2 θ 3 θ 4 θ 5 θ 6 θ 7 ] T Represents 7 joint angles of the mechanical arm, each arm angle +>8 groups of inverse solutions of the mechanical arm can be obtained; r is R 7 Representing the gesture rotation matrix of the tail end of the mechanical arm relative to the base, P 7 ＝[x y z] T Representing the position vector of the end of the arm relative to the base.
The invention further adopts the technical scheme that: the mechanical arm tail end position control mode of the handle is as follows:
wherein Deltax is the movement amount of the end position of the redundant mechanical arm, lambda 1 As precession coefficient, deltax R For the movement of the right handle, deltax R The range of (2) is limited to prevent the steering error caused by the shake of the handle.
The invention further adopts the technical scheme that: the mechanical arm configuration control mode of the handle is as follows:
wherein,is a redundant mechanical arm angle +>Is lambda of the movement of 2 As precession coefficient, deltax R Is the movement amount of the right handle.
The invention further adopts the technical scheme that: the mechanical arm joint control mode of the handle is as follows:
wherein, delta theta i For the variation of the ith joint angle of the redundant mechanical arm, lambda 3 As precession coefficient, deltax L Is the movement amount of the left handle.
The invention further adopts the technical scheme that: the precession coefficient lambda 1 ＝0.01，λ 2 ＝0.1，λ 3 ＝0.05。
The invention further adopts the technical scheme that: the simulation environment is a Unreal Engine or a Unity 3D simulation Engine; the VR head-mounted display and the handle controller are HTC VIVE or PICO virtual reality glasses and helmets.
An electronic device comprising at least one processor, and a memory communicatively coupled to the at least one processor; the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the redundant robot interaction method based on virtual reality technology.
A computer readable storage medium storing computer instructions for enabling a processor to implement the virtual reality technology based redundant robotic arm interaction method when executed.
A redundant mechanical arm interaction system based on a virtual reality technology comprises a mechanical arm tail end position control module of a handle, a control module and a control module, wherein the mechanical arm tail end position control module is used for acquiring the movement amount of the redundant mechanical arm tail end position; the mechanical arm configuration control module of the handle is used for acquiring the movement amount of the redundant mechanical arm angle; the mechanical arm joint control module of the handle is used for acquiring the movement amount of the ith joint angle of the redundant mechanical arm; a VR head mounted display and handle controller for providing a reference scene for an operator and obtaining an operator intent; the redundant robot arm is used for executing actions.
Advantageous effects
The invention has the beneficial effects that: the invention provides a redundant mechanical arm interaction method based on a virtual reality technology, which comprises the steps of establishing a kinematic model of a redundant mechanical arm, establishing an inverse kinematic model of the mechanical arm according to an arm angle parameterization method, designing a mechanical arm tail end position control mode, a mechanical arm configuration control mode and a mechanical arm joint control mode of a handle, determining precession coefficients of three control modes through multiple experiments, and finally writing a program script in a simulation environment to realize the three control modes. In the prior art, most of the interaction methods of the redundant mechanical arms are performed through a keyboard or a hand controller, an operator needs to observe a two-dimensional display to track the actions of the remote mechanical arm, the feeling of reality is poor, and the operation efficiency of the operator can be affected. The method and the system provided by the invention can operate the remote mechanical arm through VR technology, and improve the feeling of presence and the operation efficiency of an operator by designing different control modes. The effectiveness of the interaction method is verified through simulation, and the result shows that the interaction method can improve the presence feeling and the operation efficiency of operators and has great application value in the interaction of the later mechanical arms.
Drawings
FIG. 1 is a KUKA iiwa 7 degree of freedom ball wrist type redundant manipulator physical object and a connecting rod model;
FIG. 2 is a schematic view of an operator manipulating a position of a distal end of a robotic arm using a handle;
FIG. 3 is a schematic diagram of the change of the end position in the end position control mode of the mechanical arm;
FIG. 4 is a schematic illustration of an operator manipulating a robotic arm configuration using a handle;
FIG. 5 is a schematic view of arm angle change in a robotic arm configuration handling mode;
FIG. 6 is a schematic illustration of an operator manipulating a robotic arm joint using a handle;
FIG. 7 is a schematic illustration of a change in joint position in a robotic arm joint manipulation mode;
FIG. 8 is a schematic diagram of an operator using an HTC VIVE PRO helmet and handle;
FIG. 9 is a schematic view of a ball-wrist redundant manipulator arm angle;
fig. 10 is a schematic diagram of an experimental procedure of an operator operating the mechanical arm.
Detailed Description
The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.
In order to improve the presence of operators in the human-computer interaction process and improve the human-computer interaction efficiency, the invention provides a redundant mechanical arm interaction method based on a virtual reality technology. The method comprehensively considers various task demands, lays a good foundation for the actual operation of the later mechanical arm, and is concretely implemented as follows.
A redundant mechanical arm interaction method based on a virtual reality technology comprises the following steps:
step 1, a kinematic model of a redundant mechanical arm is established, taking the mechanical arm in fig. 1 as an example, DH parameters of the mechanical arm are shown in table 1, an inverse kinematic model of the mechanical arm is established according to an arm angle parameterization method in fig. 9, and a group of inverse solutions with the minimum 2 norms of the inverse solution vector difference value at the last moment are taken as inverse solution results.
T in 7 Representing a pose matrix of the robot arm tip relative to the base,represents the arm angle, θ= [ θ ] 1 θ 2 θ 3 θ 4 θ 5 θ 6 θ 7 ]Represents 7 joint angles of the mechanical arm, theta last Q is the inverse solution vector of the previous moment i Is the i-th set of inverse solution vectors in the 8 sets of inverse solutions.
TABLE 1 Kuka iiwa seven degree of freedom mechanical arm D-H parameters
Step 2: and designing a mechanical arm tail end position control mode of the handle. The movement amount Deltax of the right handle R Multiplied by precession coefficient lambda 1 =0.01 as the movement amount Δx of the redundant robot arm end position. Precession coefficient lambda 1 Obtained by multiple simulation experiments, when lambda 1 When=0.01, the arm end position tracking is smoother. X is x 0 +Δx is the new position vector P 7 ' gesture rotation matrix R 7 'unchanged,' inverse kinematics solution is performed. Based on Unreal Engine 4 software development simulation interface, performing inverse kinematics calculation of each frame by using a self-defined frame number of a tick, and finally driving the mechanical arm joint to move through the calculated mechanical arm joint angular movement.
P 7 ′＝x 0 +Δx (7)
Step 3: the mechanical arm configuration control mode of the handle is designed. The movement amount Deltax of the right handle R Multiplied by precession coefficient lambda 2 As redundant mechanical arm angleIs>λ 2 Is adjustable, where lambda 2 =0.1. And taking the changed arm angle as an input of inverse kinematics, resolving, and finally driving the mechanical arm to move according to the calculated movement amount of the joint angle of the mechanical arm.
Since this mode is performed as a right handle as in step 2, the right handle trigger key of VIVE is used as the mode switch key. Note that the amount of right handle movement used herein is scalar, referring to Δx R ＝[Δx R Δy R Δz R ]Δx in (a) R A component.
Step 4: and designing a mechanical arm joint control mode of the handle. Move the left handle by an amount of Deltax L Multiplied by precession coefficient lambda 3 Delta theta as variation of ith joint angle of redundant mechanical arm i The mechanical arm is directly driven to move. Lambda (lambda) 3 Is adjustable, where lambda 3 =0.05. Since the Kuka iwa robotic arm used had 7 joints in total, the left handle trigger key of the VIVE was used as the articulation switch key. Note that the amount of left handle movement as used herein is scalar, meaningΔx in (a) L A component.
Step 5: taking Unreal Engine 4 as a simulation environment, starting a SteamVR plug-in the Unreal Engine 4 Engine, and connecting an HTC VIVE PRO head-mounted display; the Steam Controller Plugin plug-in is enabled in the Unreal Engine 4 Engine, connecting the HTC VIVE handle.
Step 6: and (3) writing a blueprint and a script in the Unreal Engine 4, detecting the movement amount of the handle, and converting the movement amount into the movement amount of the driving mechanical arm through the formulas in the steps 2, 3 and 4. Then, compare VR technique control arm and keyboard interaction control arm, carry out the experiment that removes arm end to appointed position respectively to record experimental time respectively. The keyboard interaction is to increase and decrease the position quantity Deltax of the tail end of the mechanical arm through keys of the keyboard, and to solve the angular movement quantity of the joint of the mechanical arm through a formula (7) to drive the joint of the mechanical arm to move. Experimental procedure as in fig. 10, the operator uses VR technology and keyboard to operate the robot arm tip, moves the robot arm tip home position to the cyan target area, and compares the operation efficiency of the two modes with the timing.
TABLE 2 Kuka iiwa seven degree-of-freedom mechanical arm D-H parameters
Finally, the application potential of the redundant mechanical arm interaction method based on the virtual reality technology is verified through testing. An operator controls the tail end position of the mechanical arm through the method provided by the invention to be shown in fig. 2, the change condition of the tail end position of the mechanical arm in the tail end position control mode of the mechanical arm is shown in fig. 3, the configuration of the mechanical arm is shown in fig. 4, the change condition of the arm type angle of the mechanical arm in the configuration control mode of the mechanical arm is shown in fig. 5, the joint position of the mechanical arm is shown in fig. 6, the change condition of each joint in the joint mode of the mechanical arm is shown in fig. 7, the operator wears the HTC VIVE PRO helmet and the handle control mechanical arm is shown in fig. 8, the ball wrist type redundant mechanical arm type angle is shown in fig. 9, the operator wears the HTC VIVE PRO helmet and the handle control mechanical arm, or the experiment process of the mechanical arm is shown in fig. 10 through a keyboard, the operator can refer to the three-dimensional view in VR, the time used in comparison experiment is shown in table 2, the operation process is higher than the keyboard interactive control mechanical arm, and the in-field feeling and immersion feeling are greatly improved.
The redundant mechanical arm interaction system based on the virtual reality technology comprises a mechanical arm tail end position control module of a handle, wherein the mechanical arm tail end position control module is used for acquiring the movement amount of the redundant mechanical arm tail end position; the mechanical arm configuration control module of the handle is used for acquiring the movement amount of the redundant mechanical arm angle; the mechanical arm joint control module of the handle is used for acquiring the movement amount of the ith joint angle of the redundant mechanical arm; a VR head mounted display and handle controller for providing a reference scene for an operator and obtaining an operator intent; the redundant robot arm is used for executing actions.
Although embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives, and variations may be made in the above embodiments by those skilled in the art without departing from the spirit and principles of the invention.
Claims (10)
1. A redundant mechanical arm interaction method based on a virtual reality technology is characterized by comprising the following specific steps:
establishing a kinematic model of the redundant mechanical arm;
designing a mechanical arm tail end position control mode of the handle;
designing a mechanical arm configuration control mode of the handle;
designing a mechanical arm joint control mode of a handle;
in a simulation environment, connecting a VR head-mounted display with a handle controller;
programming in a simulation environment, detecting the movement amount of the handle controller, and converting the movement amount into the movement amount for driving the joint angle of the mechanical arm through the three control modes.
2. The virtual reality technology-based redundant manipulator interaction method according to claim 1, wherein the method comprises the following steps: the kinematic model is established according to an arm angle parameterization method, and an inverse kinematic model of the mechanical arm is obtained:
in the method, in the process of the invention, 0 T 7 representing a pose matrix of the robot arm tip relative to the base,represents the arm angle, Θ= [ θ ] 1 θ 2 θ 3 θ 4 θ 5 θ 6 θ 7 ] T Represents 7 joint angles of the mechanical arm, each arm angle +>8 groups of inverse solutions of the mechanical arm can be obtained; r is R 7 Representing the gesture rotation matrix of the tail end of the mechanical arm relative to the base, P 7 ＝[x y z] T Representing the position vector of the end of the arm relative to the base.
3. The virtual reality technology-based redundant manipulator interaction method according to claim 2, wherein the method comprises the following steps: the mechanical arm tail end position control mode of the handle is as follows:
wherein Deltax is the movement amount of the end position of the redundant mechanical arm, lambda 1 As precession coefficient, deltax R For the movement of the right handle, deltax R The range of (2) is limited to prevent the steering error caused by the shake of the handle.
4. The method for interacting with a redundant manipulator based on virtual reality technology according to claim 3, wherein: the mechanical arm configuration control mode of the handle is as follows:
wherein,is a redundant mechanical arm angle +>Is lambda of the movement of 2 As precession coefficient, deltax R Is the movement amount of the right handle.
5. The virtual reality technology-based redundant manipulator interaction method according to claim 4, wherein the method comprises the following steps: the mechanical arm joint control mode of the handle is as follows:
wherein, delta theta i For the variation of the ith joint angle of the redundant mechanical arm, lambda 3 As precession coefficient, deltax L Is the movement amount of the left handle.
6. The virtual reality technology-based redundant manipulator interaction method according to claim 5, wherein the method comprises the following steps: the precession coefficient lambda 1 ＝0.01，λ 2 ＝0.1，λ 3 ＝0.05。
7. The virtual reality technology-based redundant manipulator interaction method according to claim 1, wherein the method comprises the following steps: the simulation environment is a Unreal Engine or a Unity 3D simulation Engine; the VR head-mounted display and the handle controller are HTC VIVE or PICO virtual reality glasses and helmets.
8. An electronic device, characterized in that: comprising at least one processor, and a memory communicatively coupled to the at least one processor; the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the redundant robot interaction method based on virtual reality technology of any one of claims 1-7.
9. A computer readable storage medium, characterized by: the computer readable storage medium stores computer instructions for enabling the processor to implement the virtual reality technology-based redundant manipulator interaction method of any one of claims 1-7 when executed.
10. Redundant mechanical arm interaction system based on virtual reality technology, its characterized in that: the manipulator comprises a manipulator tail end position control module of a handle, a manipulator control module and a manipulator control module, wherein the manipulator tail end position control module is used for acquiring the movement amount of the tail end position of the redundant manipulator; the mechanical arm configuration control module of the handle is used for acquiring the movement amount of the redundant mechanical arm angle; the mechanical arm joint control module of the handle is used for acquiring the movement amount of the ith joint angle of the redundant mechanical arm; a VR head mounted display and handle controller for providing a reference scene for an operator and obtaining an operator intent; the redundant robot arm is used for executing actions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310867849.8A CN117021072A (en) | 2023-07-16 | 2023-07-16 | Redundant mechanical arm interaction method and system based on virtual reality technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310867849.8A CN117021072A (en) | 2023-07-16 | 2023-07-16 | Redundant mechanical arm interaction method and system based on virtual reality technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117021072A true CN117021072A (en) | 2023-11-10 |
Family
ID=88627070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310867849.8A Pending CN117021072A (en) | 2023-07-16 | 2023-07-16 | Redundant mechanical arm interaction method and system based on virtual reality technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117021072A (en) |
-
2023
- 2023-07-16 CN CN202310867849.8A patent/CN117021072A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| De Giorgio et al. | Human-machine collaboration in virtual reality for adaptive production engineering | |
| Sun et al. | A new mixed-reality-based teleoperation system for telepresence and maneuverability enhancement | |
| CN107932504B (en) | PyQt-based mechanical arm operation control system | |
| CN108161882B (en) | Robot teaching reproduction method and device based on augmented reality | |
| Gutierrez et al. | IMA-VR: A multimodal virtual training system for skills transfer in Industrial Maintenance and Assembly tasks | |
| US9186792B2 (en) | Teaching system, teaching method and robot system | |
| Li et al. | Survey on mapping human hand motion to robotic hands for teleoperation | |
| US20150273689A1 (en) | Robot control device, robot, robotic system, teaching method, and program | |
| CN109483534B (en) | Object grabbing method, device and system | |
| Pan et al. | Augmented reality-based robot teleoperation system using RGB-D imaging and attitude teaching device | |
| CN111459274B (en) | 5G + AR-based remote operation method for unstructured environment | |
| CN114932555B (en) | Mechanical arm collaborative operation system and mechanical arm control method | |
| CN112207835B (en) | Method for realizing double-arm cooperative work task based on teaching learning | |
| CN110695988A (en) | Method and system for cooperative motion of double mechanical arms | |
| CN110815189A (en) | Robot rapid teaching system and method based on mixed reality | |
| CN114516060A (en) | Apparatus and method for controlling a robotic device | |
| CN115686193A (en) | Virtual model three-dimensional gesture control method and system in augmented reality environment | |
| Xiong et al. | Predictive display and interaction of telerobots based on augmented reality | |
| CN117021072A (en) | Redundant mechanical arm interaction method and system based on virtual reality technology | |
| Sun et al. | Shared mixed reality-bilateral telerobotic system | |
| Mosbach et al. | Accelerating interactive human-like manipulation learning with gpu-based simulation and high-quality demonstrations | |
| Humberston et al. | Hands on: interactive animation of precision manipulation and contact | |
| Chou et al. | Multijoint robot hand design for puppet operations | |
| Xu et al. | A Novel Robot Teaching System Based on Mixed Reality | |
| Zhao et al. | Intuitive robot teaching by hand guided demonstration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |