CN116476100A - Remote operation system of multi-branch space robot - Google Patents
Remote operation system of multi-branch space robot Download PDFInfo
- Publication number
- CN116476100A CN116476100A CN202310724944.2A CN202310724944A CN116476100A CN 116476100 A CN116476100 A CN 116476100A CN 202310724944 A CN202310724944 A CN 202310724944A CN 116476100 A CN116476100 A CN 116476100A
- Authority
- CN
- China
- Prior art keywords
- space robot
- equipment
- control
- processing unit
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 claims abstract description 66
- 238000012545 processing Methods 0.000 claims abstract description 57
- 230000002452 interceptive effect Effects 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 20
- 238000004088 simulation Methods 0.000 claims abstract description 13
- 238000007654 immersion Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 34
- 238000000034 method Methods 0.000 abstract description 13
- 238000007405 data analysis Methods 0.000 abstract description 4
- 238000005457 optimization Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 8
- 238000013461 design Methods 0.000 description 7
- 238000007726 management method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000009434 installation Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000013439 planning Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003183 myoelectrical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000005486 microgravity Effects 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
- B25J13/006—Controls for manipulators by means of a wireless system for controlling one or several manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64G—COSMONAUTICS; VEHICLES OR EQUIPMENT THEREFOR
- B64G4/00—Tools specially adapted for use in space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64G—COSMONAUTICS; VEHICLES OR EQUIPMENT THEREFOR
- B64G4/00—Tools specially adapted for use in space
- B64G2004/005—Robotic manipulator systems for use in space
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Remote Sensing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Manipulator (AREA)
Abstract
The application relates to the technical field of space robots, in particular to a teleoperation system of a multi-branch space robot, which comprises a control unit, a high-performance processing unit, interaction equipment and display equipment, wherein: the interactive device comprises a control interactive device and an immersion interactive device; the control unit is respectively connected with the space station equipment, the space robot, the control interaction equipment and the high-performance processing unit; the high-performance processing unit is respectively connected with the display equipment and the immersed interaction equipment; the display device is used for displaying video information, simulation information and monitoring, tracking and training interfaces of the space robot. According to the method and the device, the interaction capability of the astronaut and the multi-branch space robot is greatly improved through the cooperation of the control unit and the high-performance processing unit, the on-orbit fine operation capability of the multi-branch space robot is met, and the full life cycle monitoring, tracking, data analysis and task optimization of the multi-branch space robot can be realized.
Description
Technical Field
The application relates to the technical field of space robots, in particular to a teleoperation system of a multi-branch space robot.
Background
2023 is that the space station is built comprehensively in China, the space station formally enters an on-orbit operation stage, space tasks such as the task of taking out the cabin of an astronaut, maintenance/replacement of the external equipment in the cabin, various test care and the like are increased increasingly, and the task of taking out the cabin of the astronaut faces the difficulties of less experience, more times, large workload, wide operation range, poor flexibility of the operation outside the cabin and the like.
The cabin-out operation is carried out in a space irradiation environment, a complex illumination environment and a microgravity environment, three major categories of equipment replacement, equipment installation, equipment operation and the like are mainly covered, the multi-branch space robot has strong fine operation capability and expandability because of long-time operation, and each branch arm can cooperatively carry out operation tasks, has redundant fault tolerance capability and is ideal supporting equipment for replacing the cabin-out operation of astronauts.
In order to meet the requirements of high-efficiency, convenient, safe and reliable out-of-cabin operation of the multi-branch space robot, a set of safe, reliable, natural, friendly, convenient and intelligent man-machine interaction system needs to be constructed between an astronaut and the multi-branch space robot, the current on-orbit operation system of the space station mechanical arm is designed for the operation of the core cabin mechanical arm and the space cabin mechanical arm, and the operation control of the multi-arm collaborative fine operation is difficult to meet only one mechanical arm action at the same time.
Disclosure of Invention
The application provides a teleoperation system of a multi-branch space robot, which is characterized in that the teleoperation requirements of equipment replacement, equipment installation and equipment operation are met outside a cabin of the multi-branch space robot through an integrated space station operation stage, and the on-orbit teleoperation of the multi-branch space robot is realized by using modularized and light-small integrated design.
In order to achieve the above object, the present application provides a multi-branch space robot teleoperation system, including a manipulation unit, a high performance processing unit, an interaction device, and a display device, wherein: the interactive device comprises control interactive equipment and immersion interactive equipment, and is used for operating an interactive interface between an astronaut and the space robot and providing various interactive means for the astronaut; the control unit is respectively connected with the space station equipment, the space robot, the control interaction equipment and the high-performance processing unit and is used for monitoring, data interaction and basic remote control of the multi-branch space robot; the high-performance processing unit is respectively connected with the display equipment and the immersed interaction equipment and is used for managing on-orbit data, autonomously learning, interaction training and full-flow tracking and monitoring of tasks; the display device is used for displaying video information, simulation information and monitoring, tracking and training interfaces of the space robot.
Furthermore, the control unit adopts an aerospace-level processing platform, and comprises three working modes, namely a ground teleoperation mode, an on-orbit teleoperation mode and an on-orbit training mode.
Furthermore, the high-performance processing unit adopts a computing processing platform for realizing the digital twin function of the multi-branch space robot.
Furthermore, a function expansion interface is reserved on the high-performance processing unit.
Further, the control interaction device comprises a six-degree-of-freedom hand controller, an operation panel and a special audio unit, wherein: the six-degree-of-freedom hand controller is a serial heterogeneous hand controller, is connected with the control unit and can provide six-degree-of-freedom force feedback; the operation panel consists of a display screen and a light guide panel, is connected with the control unit and provides a graphical operation interface and emergency treatment keys for the space robot; the special audio unit is connected with the control unit and provides voice instruction control and voice broadcasting reminding for astronauts in a voice recognition mode.
Further, the immersive interactive device includes a head mounted display, an exoskeleton hand control, and a data glove, wherein: the helmet-mounted display is connected with the high-performance processing unit and internally provided with a left micro display screen and a right micro display screen; the exoskeleton hand controller is connected with the high-performance processing unit and is designed according to the structure and the movement mode of the human arm; the data glove is connected with the high-performance processing unit and is designed according to the structure and the movement mode of the human hand.
Further, a camera is arranged at the tail end of the space robot arm, and the camera displays the collected space robot artificial environment image video information through a helmet-mounted display.
The teleoperation system of the multi-branch space robot has the following beneficial effects:
1. according to the method and the device, the interaction capability of the astronaut and the multi-branch space robot is greatly improved through the cooperation of the control unit and the high-performance processing unit, the on-orbit fine operation capability of the multi-branch space robot is met, the replacement, the installation and the operation of the equipment outside the cabin are realized, and the monitoring, the tracking, the data analysis and the task optimization of the full life cycle of the multi-branch space robot can be realized.
2. The remote operation system is flexible in framework and strong in expandability, can cover the remote operation function of the follow-up space robot by reserving an expansion interface and a software upgrading mode, supports the expansion application of novel man-machine interaction equipment, and can be suitable for the wide requirements of on-orbit operation of the multi-branch space robot.
3. The man-machine interaction operation can realize reliable interaction of the multi-branch space robot in a traditional typical operation mode, and can also perform on-orbit training simulation and interaction operation through immersive interaction.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, are included to provide a further understanding of the application and to provide a further understanding of the application with regard to the other features, objects and advantages of the application. The drawings of the illustrative embodiments of the present application and their descriptions are for the purpose of illustrating the present application and are not to be construed as unduly limiting the present application. In the drawings:
fig. 1 is a schematic diagram of the composition of a teleoperation system of a multi-branch space robot provided according to an embodiment of the present application;
FIG. 2 is a functional topology diagram of a multi-branch space robotic teleoperation system provided in accordance with an embodiment of the present application;
fig. 3 is a functional schematic of a manipulation unit of a teleoperation system of a multi-branch space robot according to an embodiment of the present application;
fig. 4 is a functional schematic diagram of a high performance processing unit of a multi-branch space robot teleoperation system provided according to an embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the present application described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the present application, the terms "upper", "lower", "left", "right", "front", "rear", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate an azimuth or a positional relationship based on that shown in the drawings. These terms are used primarily to better describe the present application and its embodiments and are not intended to limit the indicated device, element or component to a particular orientation or to be constructed and operated in a particular orientation.
Also, some of the terms described above may be used to indicate other meanings in addition to orientation or positional relationships, for example, the term "upper" may also be used to indicate some sort of attachment or connection in some cases. The specific meaning of these terms in this application will be understood by those of ordinary skill in the art as appropriate.
In addition, the term "plurality" shall mean two as well as more than two.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
As shown in fig. 1, the present application provides a teleoperation system for a multi-branch space robot, which includes a control unit, a high-performance processing unit, an interaction device, and a display device, wherein: the interactive device comprises control interactive equipment and immersion interactive equipment, and is used for operating an interactive interface between an astronaut and the space robot and providing various interactive means for the astronaut; the control unit is respectively connected with the space station equipment, the space robot, the control interaction equipment and the high-performance processing unit and is used for monitoring, data interaction and basic remote control of the multi-branch space robot; the high-performance processing unit is respectively connected with the display equipment and the immersed interaction equipment and is used for managing on-orbit data, autonomously learning, interaction training and full-flow tracking and monitoring of tasks; the display device is used for displaying video information, simulation information and monitoring, tracking and training interfaces of the space robot.
Specifically, the teleoperation system of the multi-branch space robot provided by the embodiment of the application performs equipment replacement, equipment installation and teleoperation on the outside of the cabin of the multi-branch space robot by integrating the operation stage of the space station, considers the service cycle of the teleoperation of the space robot in the whole operation stage of the space station, adopts the thought of supporting the basic function, the auxiliary function and the expansion function to realize the maximum envelope of tasks, adopts modularized and light-small integrated design, realizes the on-orbit teleoperation of the multi-branch space robot, reserves an expansion interface, and can be compatible with the teleoperation expansion of other space robots and the space application verification of advanced man-machine interaction technology. The control unit is used as high-reliability equipment and is responsible for long-time monitoring, data interaction and basic remote control of the multi-branch space robot, can respectively receive data transmitted by the space station, the control interaction equipment and the high-performance processing unit, recognizes and processes the data into corresponding control instructions, and sends the corresponding control instructions to the space robot so as to enable the space robot to operate according to the control instructions; the high-performance processing unit is used as an intelligent processing unit of a teleoperation system and is responsible for the management of on-orbit data, autonomous learning, interactive training and the whole-flow tracking monitoring of tasks; the control unit is connected with the high-performance processing unit and provides state data required by digital twin of the robot and collected astronaut operation interaction data for the high-performance processing unit; the whole interactive equipment is used as an operation interactive interface between the astronaut and the robot, provides various interaction means such as vision, touch sense, force sense, sound sense and the like for the astronaut, converts the interaction means into control instructions through the control unit and the high-performance processing unit and sends the control instructions to the robot, and realizes operation control of the multi-branch space robot; the control unit receives operation data of the control interaction device, converts the operation data into instruction data which can be identified and processed by the multi-branch space robot and sends the instruction data to the space robot working outside the cabin through a system bus, and the control interaction device can receive data uploaded by the space station device through the control unit according to long-term working requirements, so that the function upgrading of a product is completed; the immersive interaction device performs data interaction with the high-performance processing unit in a wireless communication mode, and provides an immersive interaction environment for on-orbit simulation, training and learning of the multi-branch space robot.
More specifically, as shown in fig. 2, the teleoperation system of the multi-branch space robot provided in the embodiment of the present application is divided into a basic function, an auxiliary function and an extended function. The basic functions are mainly functions with definite inheritance and tasks, and comprise functions of operation control, autonomous path planning, information monitoring and management, state display, alarm prompt and the like in the man-machine interaction process when the robot works outside the cabin around the multi-branch space; the auxiliary functions mainly comprise support functions of on-orbit task simulation, training, autonomous health management, data storage and analysis, on-orbit test and the like for improving natural interaction of intelligent, convenient and the like of a teleoperation system; the extended functions mainly comprise support functions for a novel human-computer interaction device (an electroencephalogram system and a myoelectric system) to be connected with a platform system and future system function extended support functions.
Furthermore, the control unit adopts an aerospace-level processing platform, and comprises three working modes, namely a ground teleoperation mode, an on-orbit teleoperation mode and an on-orbit training mode. The control unit is realized by adopting an aerospace-level high-reliability processing platform, adopts an anti-irradiation AI (advanced technology) acceleration multi-core artificial intelligent processing chip, and externally expands Norflash, nandflash, DDR and other storage units and high-speed interfaces as peripheral interface management equipment of a system and management and resolving equipment of important data of the system, and has three working modes of ground teleoperation, on-orbit teleoperation and on-orbit training; in a ground teleoperation mode, the astronaut only takes part in tasks as a monitor and can only take part in emergency processing tasks; in the on-orbit teleoperation mode, an astronaut can control the multi-branch space robot in real time through equipment directly connected with the control unit, and can also receive an instruction sequence generated by the high-performance processing unit and forward the instruction sequence to the multi-branch space robot to realize remote control; the control unit does not send instructions out in the track training mode, which forwards the control instruction data to the high performance processing unit.
More specifically, as shown in fig. 3, the control unit is used as a basic state monitoring, initializing, remote control command sending and peripheral interface management device for the multi-branch space robot, and reliability and safety are greatly improved by adopting an aerospace-level high-reliability processing platform. Through cooperation with the control interaction equipment, simple interaction, control, monitoring and the like of the space robot can be realized. The control unit provides a ground remote control operation interface, can receive remote control instruction data of ground uplink, and simultaneously, can downlink control states and operation data of the multi-branch space robot to the ground, so that the ground can monitor and remotely control the robot; a man-machine interface is provided for the remote control multi-branch space robot, so that the working state, the movement mode, the control mode and the like of the space robot are manually controlled, the planning of the movement path of the space robot can be completed according to the instruction data for controlling the interaction equipment, the movement path of the space robot is converted into the driving data of each joint, and the automatic task planning and the calculation of the driving data of each joint of the space robot are realized; meanwhile, the space robot emergency treatment plan can be provided, the risk of on-orbit operation is reduced, and the operation can be quickly recovered according to requirements after the risk is identified.
Furthermore, the high-performance processing unit adopts a computing processing platform for realizing the digital twin function of the multi-branch space robot. As shown in fig. 4, the high-performance processing unit is used as an intelligent bearing platform of a teleoperation system, and is mainly used for realizing the digital twin function of the multi-branch space robot and is responsible for on-orbit data management, autonomous learning, on-orbit interactive training of off-board operation, overall process tracking and monitoring of tasks and the like. The control unit is mainly used for transmitting the control instruction generated by the conversion of the control unit to the outdoor robot so as to realize the operation control of the outdoor robot. The high-performance processing unit generally adopts a commercial mature processing platform, is based on a domestic high-performance computing platform, and provides more natural and comfortable interaction experience for astronauts by strengthening, adaptively modifying and space environment adaptability to the processing platform, utilizing graphic rendering, accelerating and higher computing power of the processing unit, enabling the processing unit to be used in space teleoperation and providing the interaction experience for the astronauts through the cooperation of immersed interaction equipment. The digital twin is a digital model of an existing or to-be-existing physical entity object, the state of the physical entity object is perceived, diagnosed and predicted in real time through actual measurement, simulation and data analysis, the behavior of the physical entity object is regulated and controlled through optimization and instructions, the digital twin evolves itself through mutual learning among related digital models, and meanwhile, decisions of stakeholders in the life cycle of the physical entity object are improved.
More specifically, the digital twin body is not only a mirror image of the multi-branch space robot, but also can remotely operate the multi-branch space robot, receive feedback information, and further optimize remote operation behavior according to the feedback information. In the whole life cycle, the digital twin body also needs to realize real-time monitoring and state prediction of the outdoor robot, and realizes effective support of on-orbit maintenance of the outdoor robot. Digital twins need to be integrated into four processes, namely, digitization (digital modeling), interaction (real-time interaction with the multi-branch space robot), foreknowledge (predicting the state of the multi-branch space robot through known information and data), foreknowledge (optimizing remote control operations through machine autonomous learning and training). In addition, the high-performance processing unit can also store, manage and analyze mass data, and provide hardware support such as disk arrays for storing data types such as digital twins, machine autonomous learning, real-time state monitoring and simulation control models.
Furthermore, a functional expansion interface is reserved on the high-performance processing unit, so that teleoperation function coverage of a subsequent space robot can be realized, and expansion application of novel man-machine interaction equipment (an electroencephalogram system and a myoelectric system) is supported, so that the method can be suitable for wide requirements of on-orbit operation of the multi-branch space robot.
Specifically, the embodiment of the application also gives consideration to the interactive operation tasks of other various space robots in the operation and maintenance stage of the space station, adopts an open design in the system design, sets a function expansion interface, leaves sufficient interfaces for the access of novel interactive equipment, adopts a standardized design in the interface design, meets the requirements of verifying novel interactive technology and terminal equipment in the future, and covers the front edge technology verification of a teleoperation system comprehensively.
Further, the control interaction device comprises a six-degree-of-freedom hand controller, an operation panel and a special audio unit, wherein: the six-degree-of-freedom hand controller is a serial heterogeneous hand controller, is connected with the control unit and can provide six-degree-of-freedom force feedback; the operation panel consists of a display screen and a light guide panel, is connected with the control unit and provides a graphical operation interface and emergency treatment keys for the space robot; the special audio unit is connected with the control unit and provides voice instruction control and voice broadcasting reminding for astronauts in a voice recognition mode.
Specifically, the six-degree-of-freedom hand controller is a serial heterogeneous hand controller, preferably 2 hand controllers are arranged, six-degree-of-freedom force feedback can be provided, and meanwhile, partial keys can be arranged on the handle of the hand controller so as to control the tail ends (such as clamping, rotating and the like) of the branched mechanical arms. The manual control function devices such as the six-degree-of-freedom manual controller and the operation panel are main interfaces for transmitting human-robot information in a teleoperation system, and control of a multi-branch space robot by a astronaut is realized by providing various manual control instruction transmitting functions and manual controller control functions, so that the system not only has the functions of supporting the astronaut to directly control double arms of the multi-branch space robot to work cooperatively through direct mapping of the manual controller, but also can realize the function of transmitting manual control instructions through designing a friendly instruction transmitting function interface, can realize high-reliability instruction transmitting processing capacity through emergency keys, and fully covers the manual operation functions which the teleoperation system should have; the manual control operation device realizes a first input way of control information of astronauts through human body touch and force sense channels, and functionally forms a first information interaction human-in loop of a teleoperation system together with a visual information output channel realized by the teleoperation display device.
More specifically, the special audio unit is used as voice function equipment, and mainly utilizes a human body sound sense channel, and through voice interaction of multiple modes such as voice synthesis, voice recognition and the like, the receiving of information and the sending of voice control instructions by astronauts are realized, so that the aim of man-machine interaction with the multi-branch space robot is fulfilled; the voice command can realize mode switching, command sending, operation page switching, parameter state inquiry and the like in a normal state, and the voice synthesis is used for state broadcasting, position posture reminding, astronaut inquiry information broadcasting, part voice command replying and the like in the operation process; the voice interaction forms a complete information interaction person-in-loop of man-machine interaction from the information flow, and redundancy is carried out on the first information interaction person-in-loop from the system function, so that the reliability and convenience of a teleoperation system are improved.
Further, the immersive interactive device includes a head mounted display, an exoskeleton hand control, and a data glove, wherein: the helmet-mounted display is connected with the high-performance processing unit and internally provided with a left micro display screen and a right micro display screen; the exoskeleton hand controller is connected with the high-performance processing unit and is designed according to the structure and the movement mode of the human arm; the data glove is connected with the high-performance processing unit and is designed according to the structure and the movement mode of the human hand. The immersive interaction device mainly utilizes an immersive interaction mode, realizes interaction with the space robot system through novel interaction modes such as virtual reality, enhanced display, wearing equipment and the like, is mainly applied to on-orbit training, simulation, task simulation and the like of robots in consideration of reliability and safety, and does not send operation instructions to the outdoor multi-branch space robots in the interaction process; a complete information interaction loop for on-orbit simulation, learning and training is provided for astronauts, which is not only a supplement of manual control and voice interaction, but also a redundant design on system functions, and is more focused on asynchronous operation of a space robot. The exoskeleton hand controller and the data glove are designed according to the structures of the arms and the hands of a human body, and can respectively apply feedback force on the big arm, the small arm, the wrist and the fingers of an astronaut, so that an immersive interaction environment is provided for on-orbit simulation, training and learning of the multi-branch space robot.
Further, a camera is arranged at the tail end of the space robot arm, and the camera displays the collected space robot artificial environment image video information through a helmet-mounted display. The tail end of the space robot arm is provided with a left camera and a right camera, the helmet type display is internally provided with a left micro display screen and a right micro display screen, the image and video information of the robot working environment collected by the left camera and the right camera at the tail end of the robot arm are respectively synchronized, three-dimensional stereoscopic vision feedback information is provided for a astronaut to perceive the robot working environment, meanwhile, the helmet type display can be used for positioning eyes (balls) of the astronaut, so that eye vision tracking is realized, and tracking results are used for controlling the directions of the left camera and the right camera at the tail end of the robot arm to keep consistent with the eye vision direction, so that observation of a working area according to the intention of the astronaut is realized.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (7)
1. The teleoperation system of the multi-branch space robot is characterized by comprising a control unit, a high-performance processing unit, interaction equipment and display equipment, wherein:
the interactive equipment comprises control interactive equipment and immersion interactive equipment, and is used for operating an interactive interface between an astronaut and a space robot and providing various interactive means for the astronaut;
the control unit is respectively connected with the space station equipment, the space robot, the control interaction equipment and the high-performance processing unit and is used for monitoring, data interaction and basic remote control of the multi-branch space robot;
the high-performance processing unit is respectively connected with the display equipment and the immersive interaction equipment and is used for managing on-orbit data, autonomously learning, interactively training and tracking and monitoring the whole flow of tasks;
the display device is used for displaying video information, simulation information and monitoring, tracking and training interfaces of the space robot.
2. The multi-branch space robot teleoperation system according to claim 1, wherein the control unit adopts an aerospace level processing platform, and comprises three working modes, namely a ground teleoperation mode, an on-orbit teleoperation mode and an on-orbit training mode.
3. The teleoperation system of a multi-branched space robot according to claim 1, wherein the high-performance processing unit employs a computing processing platform for implementing digital twin functions of the multi-branched space robot.
4. A multi-branch space robot teleoperation system according to claim 3, characterized in that a functional expansion interface is also reserved on the high-performance processing unit.
5. The multi-branch space robot teleoperation system according to claim 1, wherein the control interaction device comprises a six-degree-of-freedom hand control, an operation panel, and a dedicated audio unit, wherein:
the six-degree-of-freedom hand controller is a serial heterogeneous hand controller, is connected with the control unit and can provide six-degree-of-freedom force feedback;
the operation panel consists of a display screen and a light guide panel, is connected with the control unit and provides a graphical operation interface and emergency treatment keys for the space robot;
the special audio unit is connected with the control unit, and provides voice instruction control and voice broadcasting reminding for astronauts in a voice recognition mode.
6. The multi-branch space robotic teleoperational system of claim 1, wherein the immersive interactive device comprises a head-mounted display, an exoskeleton hand control, and a data glove, wherein:
the helmet-mounted display is connected with the high-performance processing unit and internally provided with a left micro display screen and a right micro display screen;
the exoskeleton hand controller is connected with the high-performance processing unit and is designed according to a human arm structure and a movement mode;
the data glove is connected with the high-performance processing unit and is designed according to the structure and the movement mode of a human hand.
7. The teleoperation system of a multi-branched space robot of claim 6, wherein a camera is provided at a distal end of the space robot arm, the camera displaying the captured space robot artificial environment image video information through the head mounted display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310724944.2A CN116476100A (en) | 2023-06-19 | 2023-06-19 | Remote operation system of multi-branch space robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310724944.2A CN116476100A (en) | 2023-06-19 | 2023-06-19 | Remote operation system of multi-branch space robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116476100A true CN116476100A (en) | 2023-07-25 |
Family
ID=87225403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310724944.2A Pending CN116476100A (en) | 2023-06-19 | 2023-06-19 | Remote operation system of multi-branch space robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116476100A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140254896A1 (en) * | 2011-07-18 | 2014-09-11 | Tiger T G Zhou | Unmanned drone, robot system for delivering mail, goods, humanoid security, crisis negotiation, mobile payments, smart humanoid mailbox and wearable personal exoskeleton heavy load flying machine |
| CN106485973A (en) * | 2016-10-21 | 2017-03-08 | 上海申电教育培训有限公司 | Electric power skills training platform based on virtual reality technology |
| CN109571513A (en) * | 2018-12-15 | 2019-04-05 | 华南理工大学 | A kind of mobile crawl service robot system of immersion |
| CN110039547A (en) * | 2019-05-27 | 2019-07-23 | 清华大学深圳研究生院 | A kind of human-computer interaction terminal and method of flexible mechanical arm remote operating |
| CN110815258A (en) * | 2019-10-30 | 2020-02-21 | 华南理工大学 | Robot teleoperation system and method based on electromagnetic force feedback and augmented reality |
| CN111113456A (en) * | 2019-12-24 | 2020-05-08 | 兰州空间技术物理研究所 | Space manipulator on-orbit operation platform and system |
| CN111438673A (en) * | 2020-03-24 | 2020-07-24 | 西安交通大学 | High-altitude operation teleoperation method and system based on stereoscopic vision and gesture control |
| CN112150635A (en) * | 2020-09-02 | 2020-12-29 | 北京捷安申谋军工科技有限公司 | AR individual soldier and robot mixed-editing combat system and method based on digital twin |
| CN112148000A (en) * | 2020-08-28 | 2020-12-29 | 上海宇航系统工程研究所 | In-cabin simulation platform for simulating operation scene of space maintenance robot and implementation method |
| CN112329246A (en) * | 2020-11-10 | 2021-02-05 | 上海精密计量测试研究所 | Virtual verification method and system for maintainability design of solar cell array of space station |
| CN112435521A (en) * | 2020-10-27 | 2021-03-02 | 北京控制工程研究所 | Space teleoperation mechanical arm on-orbit training system and method |
| CN113888723A (en) * | 2021-09-30 | 2022-01-04 | 创业慧康科技股份有限公司 | Ultrahigh-definition diagnosis-level medical data MR panoramic display system and method |
| CN115303515A (en) * | 2022-08-10 | 2022-11-08 | 北京空间飞行器总体设计部 | Space double-arm on-orbit operation-oriented astronaut in-cabin operation and display control system |
| CN115423132A (en) * | 2022-10-03 | 2022-12-02 | 长沙理工大学 | Engineering machinery predictive maintenance method based on digital twinning |
| CN116168586A (en) * | 2022-12-28 | 2023-05-26 | 成都西南信息控制研究院有限公司 | Immersive ground carrying equipment simulation training system |
| CN116197899A (en) * | 2023-01-10 | 2023-06-02 | 北京航空航天大学 | Active robot teleoperation system based on VR |
-
2023
- 2023-06-19 CN CN202310724944.2A patent/CN116476100A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140254896A1 (en) * | 2011-07-18 | 2014-09-11 | Tiger T G Zhou | Unmanned drone, robot system for delivering mail, goods, humanoid security, crisis negotiation, mobile payments, smart humanoid mailbox and wearable personal exoskeleton heavy load flying machine |
| CN106485973A (en) * | 2016-10-21 | 2017-03-08 | 上海申电教育培训有限公司 | Electric power skills training platform based on virtual reality technology |
| CN109571513A (en) * | 2018-12-15 | 2019-04-05 | 华南理工大学 | A kind of mobile crawl service robot system of immersion |
| CN110039547A (en) * | 2019-05-27 | 2019-07-23 | 清华大学深圳研究生院 | A kind of human-computer interaction terminal and method of flexible mechanical arm remote operating |
| CN110815258A (en) * | 2019-10-30 | 2020-02-21 | 华南理工大学 | Robot teleoperation system and method based on electromagnetic force feedback and augmented reality |
| CN111113456A (en) * | 2019-12-24 | 2020-05-08 | 兰州空间技术物理研究所 | Space manipulator on-orbit operation platform and system |
| CN111438673A (en) * | 2020-03-24 | 2020-07-24 | 西安交通大学 | High-altitude operation teleoperation method and system based on stereoscopic vision and gesture control |
| CN112148000A (en) * | 2020-08-28 | 2020-12-29 | 上海宇航系统工程研究所 | In-cabin simulation platform for simulating operation scene of space maintenance robot and implementation method |
| CN112150635A (en) * | 2020-09-02 | 2020-12-29 | 北京捷安申谋军工科技有限公司 | AR individual soldier and robot mixed-editing combat system and method based on digital twin |
| CN112435521A (en) * | 2020-10-27 | 2021-03-02 | 北京控制工程研究所 | Space teleoperation mechanical arm on-orbit training system and method |
| CN112329246A (en) * | 2020-11-10 | 2021-02-05 | 上海精密计量测试研究所 | Virtual verification method and system for maintainability design of solar cell array of space station |
| CN113888723A (en) * | 2021-09-30 | 2022-01-04 | 创业慧康科技股份有限公司 | Ultrahigh-definition diagnosis-level medical data MR panoramic display system and method |
| CN115303515A (en) * | 2022-08-10 | 2022-11-08 | 北京空间飞行器总体设计部 | Space double-arm on-orbit operation-oriented astronaut in-cabin operation and display control system |
| CN115423132A (en) * | 2022-10-03 | 2022-12-02 | 长沙理工大学 | Engineering machinery predictive maintenance method based on digital twinning |
| CN116168586A (en) * | 2022-12-28 | 2023-05-26 | 成都西南信息控制研究院有限公司 | Immersive ground carrying equipment simulation training system |
| CN116197899A (en) * | 2023-01-10 | 2023-06-02 | 北京航空航天大学 | Active robot teleoperation system based on VR |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107221223B (en) | Virtual reality cockpit system with force/tactile feedback | |
| CN103631221B (en) | A kind of distant operate services robot system | |
| CN111113456A (en) | Space manipulator on-orbit operation platform and system | |
| CN110834330B (en) | Flexible mechanical arm teleoperation man-machine interaction terminal and method | |
| CN107193371A (en) | A kind of real time human-machine interaction system and method based on virtual reality | |
| CN109262609A (en) | Mechanical arm tele-control system and method based on virtual reality technology | |
| Szczurek et al. | Multimodal multi-user mixed reality human–robot interface for remote operations in hazardous environments | |
| CN105922262A (en) | Robot and remote control equipment and remote control method thereof | |
| CN110039547A (en) | A kind of human-computer interaction terminal and method of flexible mechanical arm remote operating | |
| CN115303515A (en) | Space double-arm on-orbit operation-oriented astronaut in-cabin operation and display control system | |
| CN110524531A (en) | A kind of robot control system and its workflow based on Internet of Things cloud service | |
| CN203019374U (en) | Novel human-simulated intelligent robot | |
| Krupke et al. | Prototyping of immersive HRI scenarios | |
| CN113595239A (en) | Cloud-side-end-collaborative intelligent substation management and control system | |
| CN111590567A (en) | Space manipulator teleoperation planning method based on Omega handle | |
| CN113421470A (en) | Teleoperation simulation training system and teleoperation simulation training method for space manipulator | |
| CN116197899A (en) | Active robot teleoperation system based on VR | |
| CN116476100A (en) | Remote operation system of multi-branch space robot | |
| CN112148000B (en) | In-cabin simulation platform for simulating operation scene of space maintenance robot and implementation method | |
| CN116442286B (en) | Robot work object positioning system, method, device, robot and medium | |
| CN106119840A (en) | A kind of laser melting coating integrated control system | |
| JPS6130385A (en) | Robot | |
| Rossmann et al. | The virtual testbed: Latest virtual reality technologies for space robotic applications | |
| CN209132616U (en) | A kind of robot control system based on pattern-recognition | |
| CN109834713B (en) | Control center of humanoid brain robot and control center of humanoid brain walking robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |