CN109933097A - A kind of robot for space remote control system based on three-dimension gesture - Google Patents
A kind of robot for space remote control system based on three-dimension gesture Download PDFInfo
- Publication number
- CN109933097A CN109933097A CN201910277148.2A CN201910277148A CN109933097A CN 109933097 A CN109933097 A CN 109933097A CN 201910277148 A CN201910277148 A CN 201910277148A CN 109933097 A CN109933097 A CN 109933097A
- Authority
- CN
- China
- Prior art keywords
- robot
- space
- gesture
- module
- subsystem
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J3/00—Manipulators of master-slave type, i.e. both controlling unit and controlled unit perform corresponding spatial movements
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B17/00—Systems involving the use of models or simulators of said systems
- G05B17/02—Systems involving the use of models or simulators of said systems electric
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D3/00—Control of position or direction
- G05D3/12—Control of position or direction using feedback
Abstract
A kind of robot for space remote control system based on three-dimension gesture, including gesture interaction subsystem, predictive simulation subsystem, information management subsystem；Gesture interaction subsystem acquires equipment acquisition manpower posture information by gesture and is pre-processed；Predictive simulation subsystem is connect with gesture interaction subsystem, the legitimacy for the distant behaviour instruction that gesture interaction subsystem is sent is checked, and monitor the current status data of robot for space, it is predicted by state of the dynamics simulation to robot for space next step, and three-dimensional real-time display is carried out by predicted state and current state of the Three-dimensional Display module to robot for space；Information management subsystem is connect with predictive simulation subsystem, is communicated by earth station and robot for space, is realized that distant behaviour instructs to upload and is transferred with monitoring state data.With the application of the invention, can accurately control robot for space in real time completes a series of tasks such as maintainable technology on-orbits, in-orbit module replacing, in-orbit fuel adding.
Description
The application be application No. is 201611023564.2, it is entitled that " a kind of robot for space based on three-dimension gesture is distant
The divisional application of the application for a patent for invention of operating system ".
Technical field
It is distant more particularly, to a kind of robot for space based on three-dimension gesture the present invention relates to the distant behaviour's technology of robot for space
Operating system.
Background technique
With the continuous development of mankind's space operation, it is anticipated that future will have a large amount of Space-time idea, space adds
Work, Space configuration, space maintenance and repair need of work carry out.Due to the risk and astronaut's life support of space environment
The limitation of high cost, robot plays an increasingly important role in aerospace field.However, by mechanism, control
The limitation of the support technologies such as system, sensing and artificial intelligence, developing can work from master mode entirely under unknown or complex environment
Intelligent robot, be the target being difficult to realize in a short time.Therefore, the prior art, the existing locally autonomous energy of development are made full use of
Teleoperation robot (Telerobotics) system that the intelligence of people includes is again a kind of selection that comparison is real by power.
Remote operating (Teleoperation) can be understood as remote-controlled operation on literal.In Teleoperation Systems
In, operator is intermittently communicated with the robot at distant end as supervisor, and the robot system of Cong Yaoduan obtains needs
Primary data information (pdi), while the instruction such as goal task is sent to distant robot, distant robot is according to the instruction received, in addition certainly
The human perception and intelligence of body execute task.It can be found that teleoperation robot is the robot part that a kind of someone participates in
Self-control system is related to the interaction of people and robot and the interaction of machine human and environment, it has given full play to people and machine
The respective advantage of people and the perception and capacity for having expanded people, therefore before suffering from aerospace field and being widely applied
Scape.In solar-system operation, it is only necessary to which ground handling operator and/or the astronaut in cabin carry out distant behaviour to robot for space
Make, so that it may complete the tasks such as maintenance or the spacecraft fuel adding of space station, the even moon or areographic exploration.It keeps away
It is dangerous to astronaut's bring to have exempted from cabin out, has significantly reduced the cost of space mission, and expanded the detectivity of the mankind.
In common human-computer interaction technology, the mechanical equipment of the contacts such as rocking bar, the controller for imitating robot shape is
It is often used as the tool of interaction between operator and robot.However the disadvantage of this quasi-controller be need operator into
The quite non-intuitive arm action of row controls robot, this, which just needs operator to have certain operating experience, can just have
Effect accurately controls robot.Another man-machine interaction mode is real using carrying out to the position of manpower and pose
When the system that tracks.The equipment for belonging to this type has electromagnetic tracking device, and inertial sensor, data glove etc., these are belonged to
The sensor of contact, disadvantage is it is also obvious that be exactly that can hinder the normal hand motion of operator.Relative to friendship mentioned above
Mutual technology, the interaction technique based on machine vision have apparent advantage.Due to being contactless, whole operation mode
It is non-intrusion type, the interference of operator can be greatly reduced.In this kind of interaction technique, allow operator can be with intuitive and natural
Command mode go to be controlled, effectively avoid contact with interference brought by formula equipment.
Summary of the invention
The present invention is based on the thinkings of the interaction technique of machine vision, and it is distant to provide a kind of robot for space based on three-dimension gesture
The control of robot for space is better achieved in operating system.
To achieve the above object, the invention adopts the following technical scheme:
A kind of robot for space remote control system based on three-dimension gesture, including gesture interaction subsystem, predictive simulation
System, information management subsystem；The gesture interaction subsystem acquires equipment acquisition manpower posture information by gesture and carries out
Pretreatment；The predictive simulation subsystem is connect with the gesture interaction subsystem, is sent to the gesture interaction subsystem
The legitimacy of distant behaviour's instruction is checked, and monitors the current status data of robot for space, by dynamics simulation to space
The state of robot next step predicted, and by Three-dimensional Display module to the predicted state and current state of robot for space
Carry out three-dimensional real-time display；The information management subsystem is connect with the predictive simulation subsystem, passes through earth station and space
Robot is communicated, and is realized that distant behaviour instructs to upload and is transferred with monitoring state data.
Further:
The predictive simulation subsystem includes dynamics simulation module, and the dynamics simulation module is led in simulation process
External force/torque suffered by the pedestal to robot for space, mechanical arm tail end and joint moment input are crossed, recursion is from robot for space
Pedestal is to the location/velocity relationship of mechanical arm tail end, and calculating speed, acceleration are to obtain position and the speed of robot for space
Degree, obtains the state of robot for space next step according to calculated result.
The predictive simulation subsystem includes collision detection module and safe early warning module, and the collision detection module is to sky
Between robot next step state carry out collision detection, the safe early warning module according to collision detection result carry out safety it is pre-
It is alert.
The collision detection module is based on bounding volume hierarchy (BVH) principle, using layering K-Dops Box technology, in space machine
Device examines the distance between each section, robot for space and object inside the mechanical arm of robot for space during manually making
It surveys, judges whether robot for space collides in simulation stage, be sent to space machine to avoid by unsafe control data
Device people.
The safe early warning module is based on Artificial Potential Field technology, adds one layer of peripheral package region for robot for space,
It is operation if the package region of robot for space reaches target object surface when being worked by gesture control robot for space
Person makes early warning, it is preferable that is based on prewarning area dynamic adjustment algorithm, makes early warning range with gesture data movement speed
And the variation dynamic of acceleration adjusts.
The predictive simulation subsystem includes virtual clamp module, and the virtual clamp module provides the virtual of pipeline pattern
Fixture, wherein the virtual clamp module generates advancing tube according to the estimated travel path of the mechanical arm tail end of robot for space
Road, when practical travel path deviates, the traveling pipeline is based on dynamic adjustment algorithm and recalculates generation, and is based on touching
Detection algorithm is hit, the travel path of the mechanical arm tail end of robot for space is constrained, when path deviation planned orbit is more than
When threshold value, warning is provided, so that operator adjusts travel path in time.
The gesture interaction subsystem includes gesture acquisition module, data-optimized module, network communication module, the gesture
Acquisition module obtains manpower pose data by capture, tracking and identification, and the data-optimized module utilizes average value processing method,
Manpower pose data are filtered using glide filter mode, obtain stable gesture data, the network communication module will
Gesture data is sent to the predictive simulation subsystem.
The gesture interaction subsystem provides two kinds of operation modes of real-time mode and incremental mode, the data-optimized module
Control the switching of two kinds of operation modes；When under real-time mode, the hand position and attitude data of operator is mapped directly to
Under robot for space ending coordinates system；When under incremental mode, operator can do out position and posture in the desired direction
Variation, changing value are mapped under robot for space ending coordinates system with certain scaling ratio, it is preferable that are also provided described in revocation
The reset gesture of changing value；The gesture interaction subsystem judges that current mode is in real-time mode or incremental mode,
To export robot end's pose or joint control instruction.
The gesture acquisition module acquires sensor as gesture using Leap motion sensor；Preferably, gesture is adopted
Collecting platform has double-layer structure, and Leap motion sensor is arranged on upper layer, and the operating space of operator is on sensor
Sky, operating space are in back taper.
The robot for space remote control system has gesture stability, position control and the pose control for robot for space
Make three kinds of control models.
Beneficial effects of the present invention:
In view of teleoperation robot in aerospace field huge potentiality and human-computer interaction skill based on machine vision
The advantages of art, the present invention propose a kind of novel human-computer interaction technology scheme applied to robot for space remote control system, obtain
It takes manpower posture abundant, monitor the current status data of robot for space, and predict the predicted state of robot for space, it can benefit
With three-dimensional reconstruction, virtual reality technology, shows and reappear by Three-dimensional Display module realization three-dimensional scenic, thus more natural
More intuitively control robot for space.The present invention can be applied to, and accurately control space in real time by three-dimension gesture on ground
Robot completes a series of tasks such as maintainable technology on-orbits, in-orbit module replacing, in-orbit fuel adding.
The embodiment of the present invention including but not limited to advantage in detail below:
1, manpower posture abundant can be obtained by three-dimensional reconstruction, reappeared by virtual reality technology in scene,
Gesture is used to control robot motion, so that more natural more intuitively control robot.
2, three kinds of gesture stability, position control and Pose Control control models are adapted to different remote operating tasks.
3, the telepresenc experience for increasing operator is shown using three-dimensional scenic.
4, dynamics simulation module is added in predictive simulation, the motion state of robot can be previously obtained.
5, collision detection and safe early warning module are added in predictive simulation, the safety of remote operating task can be improved.
6, the information exchange with robot for space is realized by information management subsystem.
Detailed description of the invention
Fig. 1 is the robot for space remote control system structural schematic diagram based on three-dimension gesture of an embodiment of the present invention.
Fig. 2 is the gesture interaction subsystem structure figure of an embodiment of the present invention.
Fig. 3 is the gesture acquisition device of an embodiment of the present invention.
Fig. 4 is the predictive simulation subsystem structure figure of an embodiment of the present invention.
Fig. 5 is the information management subsystem module map of an embodiment of the present invention.
Fig. 6 is the gesture remote control system software composition schematic diagram of an embodiment of the present invention.
Fig. 7 is the remote operating task course of work schematic diagram of an embodiment of the present invention.
Specific embodiment
It elaborates below to embodiments of the present invention.It is emphasized that following the description is only exemplary,
The range and its application being not intended to be limiting of the invention.
Refering to fig. 1 to Fig. 7, in one embodiment, a kind of robot for space remote control system based on three-dimension gesture is wrapped
Include gesture interaction subsystem, predictive simulation subsystem, information management subsystem；The gesture interaction subsystem is acquired by gesture
Equipment acquisition manpower pose (position, posture) information is simultaneously pre-processed；The predictive simulation subsystem and the gesture interaction
Subsystem connection checks the legitimacy for the distant behaviour instruction that the gesture interaction subsystem is sent, and monitors space machine
The current status data of people is predicted by state of the dynamics simulation to robot for space next step, and is shown by three-dimensional
Show that module carries out three-dimensional real-time display to the predicted state and current state of robot for space；The information management subsystem and institute
The connection of predictive simulation subsystem is stated, is communicated by earth station and robot for space, realizes that distant behaviour instructs and uploads and monitoring shape
The decentralization of state data.
In a preferred embodiment, the robot for space remote control system has the posture control for robot for space
Three kinds of system, position control and Pose Control control models.
As shown in figure 4, in a preferred embodiment, the predictive simulation subsystem includes dynamics simulation module, described
Dynamics simulation module passes through external force/torque and pass suffered by pedestal to robot for space, mechanical arm tail end in simulation process
Save torque input, location/velocity relationship of the recursion from the pedestal of robot for space to mechanical arm tail end, calculating speed, acceleration
To obtain the position and speed of robot for space, the state of robot for space next step is obtained according to calculated result.
As shown in figure 4, in a preferred embodiment, the predictive simulation subsystem includes that collision detection module and safety are pre-
Alert module, the collision detection module carry out collision detection, the safe early warning module to the next step state of robot for space
Safe early warning is carried out according to collision detection result.
In a more preferred embodiment, the collision detection module is based on bounding volume hierarchy (BVH) principle, utilizes layering K-Dops
Box technology, in the robot for space course of work to each section inside the mechanical arm of robot for space, robot for space with
The distance between object (such as target celestial body) is detected, and judges whether robot for space collides in simulation stage, with
It avoids unsafe control data being sent to robot for space.
In a more preferred embodiment, the safe early warning module is based on Artificial Potential Field technology, adds for robot for space
One layer of peripheral package region, when being worked by gesture control robot for space, if the package region of robot for space reaches
Target object surface (such as target satellite body surface face) then makes early warning for operator, it is preferable that based on prewarning area dynamic
Adjustment algorithm adjusts early warning range with the variation dynamic of gesture data movement speed and acceleration.
As shown in figure 4, in a preferred embodiment, the predictive simulation subsystem includes virtual clamp module, the void
Quasi- fixture module provides the virtual clamp of pipeline pattern, wherein mechanical arm end of the virtual clamp module according to robot for space
The estimated travel path in end generates traveling pipeline, and when practical travel path deviates, the traveling pipeline is based on dynamic and adjusts
Whole algorithm recalculates generation, and is based on collision detection algorithm, carries out to the travel path of the mechanical arm tail end of robot for space
Constraint provides warning, so that operator adjusts travel path in time when path deviation planned orbit is more than threshold value.
As shown in Fig. 2, in a preferred embodiment, the gesture interaction subsystem includes that gesture acquisition module, data are excellent
Change module, network communication module, the gesture acquisition module obtains manpower pose data by capture, tracking and identification, described
Data-optimized module utilizes average value processing method, is filtered, is stablized to manpower pose data using glide filter mode
Gesture data, gesture data sends the predictive simulation subsystem by the network communication module.
In a preferred embodiment, the gesture interaction subsystem provides two kinds of operation moulds of real-time mode and incremental mode
Formula, the data-optimized module control the switching of two kinds of operation modes；When under real-time mode, the hand position of operator and
Attitude data maps directly under robot for space ending coordinates system；When under incremental mode, operator can be by institute demander
To the variation for doing out position and posture, changing value is mapped under robot for space ending coordinates system with certain scaling ratio, excellent
Selection of land also provides the reset gesture for cancelling the changing value；The gesture interaction subsystem judges that current mode is in real
When mode or incremental mode, to export robot end's pose or joint control instruction.
In a preferred embodiment, the gesture acquisition module is acquired as gesture using Leap motion sensor and is passed
Sensor.As shown in figure 3, in a particularly preferred embodiment, gesture acquisition platform has double-layer structure, by a Leap
The setting of motion sensor is entreated in the upper layer, and the operating space of operator is in sensor overhead, and operating space is tapered.
Composition, principle and the function of robot for space remote control system of the present invention are further described below by way of specific example
Energy.
A kind of robot for space remote control system based on three-dimension gesture, this system include gesture interaction subsystem, prediction
Simulation subsystem, information management subsystem.The gesture interaction subsystem acquires equipment by gesture and acquires manpower posture information
And it is pre-processed；The predictive simulation subsystem is connect with gesture interaction subsystem, carries out safety inspection to instruction, and to machine
Device people's operative scenario carries out true reappearance；The information management subsystem is connect with predictive simulation subsystem, is completed instruction and is uploaded
With decentralization monitoring information data management, communicated by earth station and robot for space.Its function having has:
1, have three kinds of gesture stability, position control and Pose Control control models, and can switch over；
2, have real time data control and incremental data control both of which, and can switch over；
3, has three-dimensional scenic display function；
4, have the predicted state of real-time display robot and the function of current state；
5, have and carry out the function of instruction secure inspection by collision detection, safe early warning, the instruction means such as transfinite；
6, have from ground station reception, parsing, the function of saving robot real-time status data；
7 have the function of sending gesture control data in real time by earth station to robot；
8, real-time storage and backup are carried out to all test datas in remote operating test, and can played back.
Gesture interaction subsystem by gesture input device acquire operator's hand position posture information, to data carry out it is excellent
Change and sends control data.
Operation scenario is presented in predictive simulation subsystem based on three-dimensional scenic, including three dimensional display capabilities, dynamics are imitated
True function, collision checking function, safety precaution function.
Information management subsystem, which has, to be received, parses, shelf space teleoperation of robot monitoring data, and reception, preservation are pre-
The teleoperation of robot order for surveying simulation subsystem, the function of robot for space is sent to by earth station.
The network communication of ICP/IP protocol is used between each subsystem and with external communication.
Gesture interaction subsystem
Gesture interaction subsystem structure is as shown in Fig. 2, mainly include gesture acquisition, data-optimized, three moulds of network communication
Block.Gesture acquisition equipment relative position is completed first to be demarcated, on this basis, captured using gesture acquisition strategies module,
Tracking, identification obtain manpower pose data.Unstable gesture data is generated in order to reduce the reasons such as hand tremor, according to
It is worth the thought of processing, is filtered using glide filter, obtains stable manpower data.Judge at current robot operating mode
In real-time mode perhaps incremental mode output robot end's pose or joint control instruction.
A gesture acquisition module
Gesture collecting part includes gesture acquisition platform part and gesture acquisition strategies part.
The sensor of gesture acquisition uses Leap motion sensor.Gesture acquisition platform is designed using double-layer structure, whole
Body design is as shown in Figure 3.Leap motion sensor is mounted in the upper layer 1 of acquisition device according to layout shown in Fig. 3
Centre.Tablet computer and a main frame, tablet computer and 1 Leap motion place in gesture acquisition platform lower layer
Sensor is connected by USB cable, is in addition connected with main frame by cable.
Gesture acquisition strategies part is mainly gesture identification strategy and control model selection.In this acquisition device, design
Two kinds of gestures open gesture for five fingers and gesture of clenching fist.When gesture is five finger gesture, gesture acquisition platform acquires hand
Gesture data；When gesture is to clench fist gesture, stop gesture data acquisition at this time.Control model selection refers to gesture stability, position
Which kind of control mould the selection for setting three kinds of control models of control and Pose Control is currently at by the selection of gesture processing software interface
Formula.
The data-optimized module of b
When the input of gesture operation is in the data area of safety, influenced by mankind's non-human act, manpower is being grasped
It can not accomplish mechanical stability when making, unstable gesture data can be generated due to shake etc..In order to avoid it is this by
The negative effect of people's limbs characteristic bring bring is solved according to the thought of average value processing using the processing mode of glide filter
This problem.
Gesture interaction subsystem provides two kinds of operation modes, is real-time mode and incremental mode respectively.In real-time mode
When lower, the hand position and attitude data of operator will be mapped directly under robot end's coordinate system, embody telepresenc；It is in
When under incremental mode, operator can do the variation of out position and posture in the desired direction, and changing value is with certain scaling ratio
It is mapped under robot end's coordinate system, while reset gesture being provided, so that operation more meets human body custom.
C network communication module
Output data Jing Guo data processing is sent to predictive simulation subsystem based on socket by network communication module, can
It is sent in a manner of selecting through UDP multicast.
Predictive simulation subsystem
Predictive simulation subsystem is divided into six modules, is Three-dimensional Display module, collision detection module, safe early warning respectively
Module, virtual clamp module, instruction checking module and dynamics simulation module, as shown in Figure 4.Predictive simulation subsystem receives hand
When the gesture control data of gesture interactive subsystem input, by instruction checking module, the legitimacy of instruction is checked.Pass through dynamics
Module carry out dynamics calculation, predict robot next step motion state, by collision detection module to next step state into
Row collision detection safe early warning.Instruction by information management subsystem is sent to after above-mentioned inspection and in Three-dimensional Display module into
Row display.
Three-dimensional Display module
The software interface that Three-dimensional Display module is provided by using OpenSceneGraph renders for robot simulation interface
Various stereoscopic effects out are also beneficial to operator and make more to current robot working condition so that simulated effect is more life-like
Accurately judgement.
Collision detection module
Collision detection module is based on bounding volume hierarchy (BVH) theory, using layering K-Dops Box technology, works in robot
The distance between mechanical arm therein each section, robot and celestial body are detected in the process, judge machine in simulation stage
Whether device people collides, and avoids unsafe control data being sent to robot, to guarantee robot work safety.
Safe early warning module
Safe early warning module is based on Artificial Potential Field technology, one layer of peripheral package region is added for robot, when passing through hand
Gesture controls machine man-hour, is that operator makes prompt if robot is close to target celestial body, plays warning function.Early warning function
It can be based on prewarning area dynamic adjustment algorithm, early warning range can be adjusted with the variation dynamic of gesture data movement speed and acceleration
It is whole, to guarantee the validity of early warning.
Virtual clamp module
Virtual clamp module uses the fixture of pipeline pattern.Firstly, what virtual clamp module expected according to mechanical arm tail end
Travel path generate traveling pipeline, when practical travel path deviates, traveling pipeline by dynamic adjustment algorithm again based on
It calculates and generates.The traveling pipeline of virtual clamp module is based on collision detection algorithm, constrains the travel path of mechanical arm tail end,
When path deviation planned orbit is excessive, warning is provided, operator is allowed to adjust travel path in time.
Instruction checking module
Instruction checking module first checks for whether data format meets protocol requirement.Machine is mapped to gesture data later
The data of people end check, first check for mechanical arm tail end angular speed and linear velocity, and then inspection machine people kinematics is anti-
Each joint angular region and joint angular speed after solution.When total data meets normal data range, just data are transmitted to next
Module.
Dynamics simulation module
Dynamics simulation module in simulation process by external force/torque suffered by robot base, mechanical arm tail end and
Joint moment input, location/velocity relationship of the recursion from pedestal to end, the speed, acceleration of computing system is to obtain machine
The robotary of next step can be obtained according to calculated result for the position and speed of device people.
Information management subsystem
The subsystem includes two modules: communication management module and test playback module, as shown in Figure 5.
Communication module
Communication module is realized and the robot for space earth station communication of predictive simulation subsystem and outside.System was according to both
Fixed interface and transmission data, and with file storage data.The output format of control instruction has robot end's pose
With two kinds of output modes of joint angle angle.
Test playback
Playback of the playback module realization to robot for space remote operating task is tested, the experimental data based on storage is sent
Predictive simulation subsystem is given, the Three-dimensional Display playback of remote operating task is realized in predictive simulation subsystem.
Example
System software
A gesture interaction subsystem software
Gesture interaction subsystem software includes gesture acquisition software and gesture processing software.One formula of gesture acquisition software is a
It operates on tablet computer, is connected with Leap Motion, run on 7 operating system of Windows and leap SDK software drives
Under rotating ring border.One formula of gesture processing software is a, runs on 7 operating system of Windows and OpenScenegraph software branch
Under support ring border.
B predictive simulation subsystem software
One formula portion of predictive simulation subsystem software is run among individual computer, based on OpenSceneGraph into
Row emulation, including the moulds such as three dimensional display capabilities, Rapid Collision Detection Algorithm library, collision detection, safe early warning, dynamics simulation
Block.Software is based on the programming of OpenGL and MFC software library and realizes, runs on 7 operating system of Windows.
C information management subsystem software
Information management subsystem software, which is mainly realized, receives the function such as display with the information exchange of earth station, realization monitoring data
Energy.Software is based on .net 3.5 and MFC technology writes realization, runs on 7 system of Windows.
System hardware
Hardware is as shown in table 1.
1 hardware of table composition
Workflow
Manpower posture information is acquired by the gesture acquisition equipment of gesture interaction subsystem and is changed into robot control command,
It is sent to predictive simulation subsystem by certain time interval (such as 100ms), after the safety detection by predictive simulation subsystem
It is sent to information management subsystem, information management subsystem passes through earth station again and is sent to robot for space.Operator is according to pre-
It surveys simulation subsystem robot for space work three-dimensional scenic to be operated, basic procedure is as shown in Figure 7.
The above content is combine it is specific/further detailed description of the invention for preferred embodiment, cannot recognize
Fixed specific implementation of the invention is only limited to these instructions.For those of ordinary skill in the art to which the present invention belongs,
Without departing from the inventive concept of the premise, some replacements or modifications can also be made to the embodiment that these have been described,
And these substitutions or variant all shall be regarded as belonging to protection scope of the present invention.
Claims (8)
1. a kind of robot for space remote control system based on three-dimension gesture, which is characterized in that including gesture interaction subsystem, in advance
Survey simulation subsystem, information management subsystem；The gesture interaction subsystem acquires equipment by gesture and acquires manpower pose letter
It ceases and is pre-processed；The predictive simulation subsystem is connect with the gesture interaction subsystem, to the gesture interaction subsystem
The legitimacy that the distant behaviour sent instructs of uniting checks, and monitors the current status data of robot for space, imitative by dynamics
Very the state of robot for space next step is predicted, and is based on three-dimensional reconstruction, by Three-dimensional Display module to space machine
The predicted state and current state of people carries out three-dimensional real-time display；The information management subsystem and the predictive simulation subsystem
Connection, is communicated by earth station and robot for space, is realized that distant behaviour instructs to upload and is transferred with monitoring state data；The hand
Gesture interactive subsystem includes gesture acquisition module, data-optimized module, network communication module, and the gesture acquisition module is by catching
It catches, track and identification obtains manpower pose data, the data-optimized module utilizes average value processing method, using glide filter side
Formula is filtered manpower pose data, obtains stable gesture data, and the network communication module sends gesture data to
The predictive simulation subsystem；The gesture interaction subsystem provides two kinds of operation modes of real-time mode and incremental mode, described
Data-optimized module controls the switching of two kinds of operation modes；When under real-time mode, the hand position and posture number of operator
According to mapping directly under robot for space ending coordinates system；When under incremental mode, operator does out position in the desired direction
With the variation of posture, changing value is mapped under robot for space ending coordinates system with certain scaling ratio.
2. robot for space remote control system as described in claim 1, which is characterized in that the predictive simulation subsystem includes
Dynamics simulation module, the dynamics simulation module pass through the pedestal to robot for space, mechanical arm end in simulation process
Suffered external force/torque and joint moment input are held, location/velocity of the recursion from the pedestal of robot for space to mechanical arm tail end is closed
System, calculating speed, acceleration obtain under robot for space to obtain the position and speed of robot for space according to calculated result
The state of one step.
3. robot for space remote control system as described in claim 1, which is characterized in that the predictive simulation subsystem includes
Collision detection module and safe early warning module, the collision detection module carry out collision inspection to the next step state of robot for space
It surveys, the safe early warning module carries out safe early warning according to collision detection result.
4. robot for space remote control system as claimed in claim 3, which is characterized in that the collision detection module is based on layer
Secondary bounding box principle, using layering K-Dops Box technology, to the machine of robot for space in the robot for space course of work
The distance between each section, robot for space and object are detected inside tool arm, judge that robot for space is in simulation stage
It is no to collide, robot for space is sent to avoid by unsafe control data.
5. robot for space remote control system as claimed in claim 3, which is characterized in that the safe early warning module is based on people
Work potential field technology adds one layer of peripheral package region for robot for space, when being worked by gesture control robot for space,
It is that operator makes early warning if the package region of robot for space reaches target object surface, it is preferable that be based on early warning
Regional dynamics adjustment algorithm adjusts early warning range with the variation dynamic of gesture data movement speed and acceleration.
6. robot for space remote control system as described in claim 1, which is characterized in that the predictive simulation subsystem includes
Virtual clamp module, the virtual clamp module provide the virtual clamp of pipeline pattern, wherein the virtual clamp module according to
The estimated travel path of the mechanical arm tail end of robot for space generates traveling pipeline, when practical travel path deviates, institute
It states traveling pipeline and generation is recalculated based on dynamic adjustment algorithm, and be based on collision detection algorithm, to the machinery of robot for space
The travel path of arm end is constrained, and when path deviation planned orbit is more than threshold value, warning is provided, so that operator is timely
Adjust travel path.
7. robot for space remote control system as described in claim 1, which is characterized in that the gesture acquisition module uses
Leap motion sensor acquires sensor as gesture, it is preferable that gesture acquisition platform has double-layer structure, Leap
Motion sensor is arranged on upper layer, and the operating space of operator is in sensor overhead, and operating space is in back taper.
8. robot for space remote control system as described in any one of claim 1 to 7, which is characterized in that the space machine
People's remote control system has three kinds of gesture stability for robot for space, position control and Pose Control control models.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910277148.2A CN109933097A (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611023564.2A CN106444861B (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
| CN201910277148.2A CN109933097A (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611023564.2A Division CN106444861B (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109933097A true CN109933097A (en) | 2019-06-25 |
Family
ID=58220485
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910277148.2A Pending CN109933097A (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
| CN201910277753.XA Pending CN109976390A (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
| CN201611023564.2A Active CN106444861B (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
Family Applications After (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910277753.XA Pending CN109976390A (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
| CN201611023564.2A Active CN106444861B (en) | 2016-11-21 | 2016-11-21 | A kind of robot for space remote control system based on three-dimension gesture |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN109933097A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287616A (en) * | 2019-06-28 | 2019-09-27 | 中国科学院空间应用工程与技术中心 | A kind of immersion space microgravity fluid telescience experiment parallel system and method |
| CN111438673A (en) * | 2020-03-24 | 2020-07-24 | 西安交通大学 | High-altitude operation teleoperation method and system based on stereoscopic vision and gesture control |
| CN111580519A (en) * | 2020-05-12 | 2020-08-25 | 北京控制工程研究所 | Quasi-real-time teleoperation system for lunar surface detection under earth-moon time delay |
| CN113189916A (en) * | 2021-05-24 | 2021-07-30 | 深圳创维－Rgb电子有限公司 | Control method and system of controller, storage medium and control device |
| CN113296604A (en) * | 2021-05-24 | 2021-08-24 | 北京航空航天大学 | True 3D gesture interaction method based on convolutional neural network |
| CN114194358A (en) * | 2021-11-23 | 2022-03-18 | 泰斗微电子科技有限公司 | Ship navigation accident early warning method, device and terminal |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6724831B2 (en) * | 2017-03-16 | 2020-07-15 | 株式会社安川電機 | Control system, controller and control method |
| JP6619395B2 (en) | 2017-07-12 | 2019-12-11 | ファナック株式会社 | Robot system and robot teaching method |
| CN107553485B (en) * | 2017-07-13 | 2020-05-12 | 西北工业大学 | Method for generating dynamic virtual clamp in human-computer interaction process |
| CN107414835A (en) * | 2017-08-31 | 2017-12-01 | 智造未来（北京）机器人系统技术有限公司 | Mechanical arm control method and manned machine first |
| CN108333924B (en) * | 2018-01-18 | 2020-05-12 | 西北工业大学 | Virtual fixture optimization generation method in operation interaction process |
| CN109395375A (en) * | 2018-09-18 | 2019-03-01 | 华南理工大学 | A kind of 3d gaming method of interface interacted based on augmented reality and movement |
| CN110355750B (en) * | 2018-10-29 | 2022-05-10 | 西北工业大学 | Interaction control method for hand-eye coordination of teleoperation |
| CN109815557B (en) * | 2018-12-31 | 2023-06-27 | 深圳市越疆科技有限公司 | Robot model display method and device and intelligent terminal |
| CN110262280B (en) * | 2019-02-26 | 2020-07-14 | 北京控制工程研究所 | Intelligent data analysis and decision support system for spacecraft rendezvous and docking flight control |
| CN109955254B (en) * | 2019-04-30 | 2020-10-09 | 齐鲁工业大学 | Mobile robot control system and teleoperation control method for robot end pose |
| CN110411506B (en) * | 2019-07-08 | 2021-09-14 | 中国航天员科研训练中心 | Fine operation capability test system facing remote operation |
| CN110480657A (en) * | 2019-08-13 | 2019-11-22 | 北京控制工程研究所 | A kind of labyrinth environment space robot world remote control system |
| CN111113456B (en) * | 2019-12-24 | 2023-06-27 | 兰州空间技术物理研究所 | Space manipulator on-orbit operation platform and system |
| CN111025995A (en) * | 2019-12-26 | 2020-04-17 | 北京空间技术研制试验中心 | Space manipulator teleoperation communication system based on space-based relay communication |
| CN112148000B (en) * | 2020-08-28 | 2022-10-21 | 上海宇航系统工程研究所 | In-cabin simulation platform for simulating operation scene of space maintenance robot and implementation method |
| CN114434437A (en) * | 2020-10-30 | 2022-05-06 | 西门子（中国）有限公司 | Remote control method and device for robot |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030034994A1 (en) * | 1993-10-01 | 2003-02-20 | Massie Thomas H. | Force reflecting haptic interface |
| CN103722554A (en) * | 2014-01-15 | 2014-04-16 | 西北工业大学 | Space teleoperation method |
| CN104057450A (en) * | 2014-06-20 | 2014-09-24 | 哈尔滨工业大学深圳研究生院 | Teleoperation method of high-dimensional motion arm aiming at service robot |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2363396A1 (en) * | 2001-11-21 | 2003-05-21 | Handshake Interactive Technologies Inc | Hard real time control center |
| EP1486300B1 (en) * | 2002-03-15 | 2011-08-17 | Sony Corporation | Robot behavior control system, behavior control method, and robot device |
| JP2009537231A (en) * | 2006-05-19 | 2009-10-29 | マコ サージカル コーポレーション | Method and apparatus for controlling a haptic device |
| US7885732B2 (en) * | 2006-10-25 | 2011-02-08 | The Boeing Company | Systems and methods for haptics-enabled teleoperation of vehicles and other devices |
| CN102880063B (en) * | 2012-09-13 | 2016-01-20 | 中国人民解放军63921部队 | Synchro control remote control system and method |
| CN103302668B (en) * | 2013-05-22 | 2016-03-16 | 东南大学 | Based on control system and the method thereof of the Space teleoperation robot of Kinect |
| CN104589356B (en) * | 2014-11-27 | 2016-08-24 | 北京工业大学 | The Dextrous Hand remote operating control method caught based on Kinect human hand movement |
| CN105795555A (en) * | 2014-12-28 | 2016-07-27 | 天津心康科技发展有限公司 | A piece of novel multifunctional detection monitoring underwear |
| CN105796071A (en) * | 2014-12-28 | 2016-07-27 | 天津心康科技发展有限公司 | A novel multifunctional wearable remote vital sign monitoring wrist watch |
| CN106055107B (en) * | 2016-06-07 | 2019-04-05 | 中国人民解放军国防科学技术大学 | Space teleoperation technology ground validation system based on people in circuit |
-
2016
- 2016-11-21 CN CN201910277148.2A patent/CN109933097A/en active Pending
- 2016-11-21 CN CN201910277753.XA patent/CN109976390A/en active Pending
- 2016-11-21 CN CN201611023564.2A patent/CN106444861B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030034994A1 (en) * | 1993-10-01 | 2003-02-20 | Massie Thomas H. | Force reflecting haptic interface |
| CN103722554A (en) * | 2014-01-15 | 2014-04-16 | 西北工业大学 | Space teleoperation method |
| CN104057450A (en) * | 2014-06-20 | 2014-09-24 | 哈尔滨工业大学深圳研究生院 | Teleoperation method of high-dimensional motion arm aiming at service robot |
Non-Patent Citations (3)
| Title |
|---|
| 朱张斌: "机器人遥操作中安全技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 李泽玲: "基于手势的机器人自然人机交互系统", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 杜广龙等: "具有预警功能机器人遥操作动态虚拟夹具的设计", 《华南理工大学学报（自然科学版）》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287616A (en) * | 2019-06-28 | 2019-09-27 | 中国科学院空间应用工程与技术中心 | A kind of immersion space microgravity fluid telescience experiment parallel system and method |
| CN110287616B (en) * | 2019-06-28 | 2023-11-17 | 中国科学院空间应用工程与技术中心 | Immersion space microgravity fluid remote science experiment parallel system and method |
| CN111438673A (en) * | 2020-03-24 | 2020-07-24 | 西安交通大学 | High-altitude operation teleoperation method and system based on stereoscopic vision and gesture control |
| CN111580519A (en) * | 2020-05-12 | 2020-08-25 | 北京控制工程研究所 | Quasi-real-time teleoperation system for lunar surface detection under earth-moon time delay |
| CN111580519B (en) * | 2020-05-12 | 2023-06-30 | 北京控制工程研究所 | Quasi-real-time teleoperation system for lunar surface detection under earth-moon time delay |
| CN113189916A (en) * | 2021-05-24 | 2021-07-30 | 深圳创维－Rgb电子有限公司 | Control method and system of controller, storage medium and control device |
| CN113296604A (en) * | 2021-05-24 | 2021-08-24 | 北京航空航天大学 | True 3D gesture interaction method based on convolutional neural network |
| CN113296604B (en) * | 2021-05-24 | 2022-07-08 | 北京航空航天大学 | True 3D gesture interaction method based on convolutional neural network |
| CN114194358A (en) * | 2021-11-23 | 2022-03-18 | 泰斗微电子科技有限公司 | Ship navigation accident early warning method, device and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106444861A (en) | 2017-02-22 |
| CN106444861B (en) | 2019-05-10 |
| CN109976390A (en) | 2019-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106444861B (en) | A kind of robot for space remote control system based on three-dimension gesture | |
| CN107221223B (en) | Virtual reality cockpit system with force/tactile feedback | |
| WO2021196529A1 (en) | Air-ground cooperative intelligent inspection robot and inspection method | |
| CN104057450B (en) | A kind of higher-dimension motion arm teleoperation method for service robot | |
| CN107097228B (en) | Autonomous traveling robot system | |
| CN107589752A (en) | Unmanned plane cooperates with formation realization method and system with ground robot | |
| CN107193371A (en) | A kind of real time human-machine interaction system and method based on virtual reality | |
| CN103389699A (en) | Robot monitoring and automatic mobile system operation method based on distributed intelligent monitoring controlling nodes | |
| CN110216674A (en) | A kind of redundant degree of freedom mechanical arm visual servo obstacle avoidance system | |
| CN103149846A (en) | Flying robot control system semi-physical simulation platform | |
| CN103722554A (en) | Space teleoperation method | |
| CN103531051A (en) | Method and simulator for virtual reality training of crane operation | |
| CN109268010A (en) | A kind of long-range inspection interference method of virtual reality Mechanized | |
| CN108908298B (en) | Master-slave type spraying robot teaching system fusing virtual reality technology | |
| CN110480657A (en) | A kind of labyrinth environment space robot world remote control system | |
| CN211890820U (en) | Air-ground cooperative intelligent inspection robot | |
| CN107577159A (en) | Augmented reality analogue system | |
| CN105589459A (en) | Unmanned vehicle semi-autonomous remote control method | |
| Hou et al. | Dynamic kinesthetic boundary for haptic teleoperation of aerial robotic vehicles | |
| Kakiuchi et al. | Development of humanoid robot system for disaster response through team nedo-jsk's approach to darpa robotics challenge finals | |
| CN102608997A (en) | Trajectory tracking control method based on artificial field | |
| CN110656975B (en) | Tunnel rescue system and method based on virtual reality and ACP parallel intelligence | |
| CN109213306A (en) | A kind of robot remote control platform and its design method | |
| CN109917670A (en) | It is positioned while a kind of intelligent robot cluster and builds drawing method | |
| CN112148000B (en) | In-cabin simulation platform for simulating operation scene of space maintenance robot and implementation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190625 |