CN102120325B - Novel remote operation far-end robot control platform and method - Google Patents
Novel remote operation far-end robot control platform and method Download PDFInfo
- Publication number
- CN102120325B CN102120325B CN 201010032439 CN201010032439A CN102120325B CN 102120325 B CN102120325 B CN 102120325B CN 201010032439 CN201010032439 CN 201010032439 CN 201010032439 A CN201010032439 A CN 201010032439A CN 102120325 B CN102120325 B CN 102120325B
- Authority
- CN
- China
- Prior art keywords
- machine people
- model
- remote machine
- virtual feedback
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses a novel remote operation far-end robot control platform and a method. The conventional master-slave type man-machine interaction control system is usually recognized an as effective remote operation method, but the operation quality of the conventional master-slave type man-machine interaction control system is determined by operators, so long time is needed to train the operators, and the operators, who work in large labor intensity, go fatigue easily and cannot complete high-precision operation. In the invention, the control platform comprises a client (1) and a server (2), wherein the client transmits data to the server (3) through a network and comprises man-machine interaction equipment (4) which is connected with a video display(5), a data comparator (6), and a virtual operation model (7) and a virtual feedback model (8), which are connected together, at the same time; and the server comprises a far-end robot control unit (9) which is connected with a far-end robot (10) that is connected with a data acquisition unit (11) and video monitoring equipment (12). The control platform and the method are used for remote control and operation of the robot.
Description
Technical field:
The invention belongs to REMOTE MACHINE people control technology field, be specifically related to a kind of novel distant operation REMOTE MACHINE people console.
Background technology:
At present, human-computer interaction technology remains one of remote end robot control systematic research focus.Traditional master-slave mode people and intersection control routine are considered to a kind of effective distant method of operating usually; Be actually people, machine cooperative system, accomplish the operation under the complex environment by people's intelligence and experience, its operational quality is determined by operating personnel; Not only to spend long-time training and operation personnel; And the operator when working labour intensity big, fatiguability is difficult to accomplish high precision manipulation.
The development of virtual reality system has raised out the AR technology, and it generates virtual environment through computer, can carry out well alternately with environment by virtual reality peripheral hardware operator, and be that the operator has a kind of sensation on the spot in person.Lot of domestic and international scholar is used for distant operation with virtual reality technology, make traditional distant control more efficient.The AR technology is fused to the virtual information that computer generates in the real world; Realization replenishes real world; Make virtual information become the part of real world on every side from sense organ; But the AR technology just realizes the additional of real world rather than substitutes actual environment fully, certain limitation is still arranged.
The application of video integration technology has been played certain impetus to the development of control system of remote end robot; It merges video information and the pseudo operation model figure that actuating station feeds back in same display window; The operator is more directly observed easily; It has improved modeling error and motion add up error and motion add up error, has improved security, the accuracy of system to a certain extent, has reduced the activity duration.But still there is certain deficiency in it: 1) the video integration technology is that the fusion demonstration that relies on dummy model and video is controlled, and the transmission existence of video information contains much information and propagation delay time is big, be difficult for carrying out delay compensation; 2) because the coordinate that video merges back camera malleable not, therefore the error condition from the observation at a certain angle might not reflect real error condition, as if only adopting the video integration technology well not fulfil assignment.
Summary of the invention:
The purpose of this invention is to provide that a kind of to have collection video integration technology, dummy model Display Technique, Long-distance Control Internet be one Motoman industrial machine arm is controlled, propose the novel distant operation REMOTE MACHINE people that two dummy models stacks show and control platform.
The purpose of foregoing invention realizes through following technical scheme:
Novel distant operation REMOTE MACHINE people controls platform; Its composition comprises: client-side and server; Described client-side arrives server through network data transmission; Described client-side comprises the human-computer interaction device, and described human-computer interaction device connects video demonstration, data comparison and pseudo operation model that is superimposed and virtual feedback model simultaneously; Described server comprises REMOTE MACHINE people control module, and described REMOTE MACHINE people control module connects the REMOTE MACHINE people, and described REMOTE MACHINE people connects data acquisition unit and video surveillance devices.
Described novel distant operation REMOTE MACHINE people controls platform, and described pseudo operation model is connected video and shows with virtual feedback model, and described virtual feedback model connects data relatively.
Described novel distant operation REMOTE MACHINE people controls the method for operating of platform, shows two dummy models---pseudo operation model and virtual feedback model in same display window stack;
Described virtual feedback model is synchronous with true REMOTE MACHINE people's motion; The operator is through pseudo operation modeling REMOTE MACHINE people's motion; And movable information sent to the distal ends robot, and the motion of virtual feedback model is to be controlled by the motion feedback information of distal ends robot;
Described feedback information is real-time joint motions sensor information; Data are little; Short time delay, and can remedy the motion add up error that time delay causes through the model angle compensation, make the motion of virtual feedback model and true REMOTE MACHINE people's moving situation be consistent;
Described pair of dummy model shows and the fusion of far-end video feed information, can be more prone to detect with control intuitively through the true REMOTE MACHINE people's of video monitor window real time monitoring job status;
It is that the control system is judged through data that final position between described pseudo operation model and the virtual feedback model overlaps, and overlaps to judge it is accurate, does not overlap if motion is accomplished, and system then adds motion compensation according to data;
Described control program can be required by operation, and the track of real-time rendering two dummy model end effectors according to the display window track of being drawn, can be judged the validity of true REMOTE MACHINE people motion;
Described virtual feedback model can embody multiple real information more easily, intuitively reflects through change color and carries out short stressing conditions;
Described two dummy models can amplify with local in arbitrarily angled rotation adjustment, can each distance from different perspectives go to observe two position relations between the model.
Described novel teleoperation robot control method, two dummy model stacks show control; Control pseudo operation model moves at known working space, observes its operation conditions through the models show window; If satisfy job requirements, send instructions to the distant robot of far-end, make its movement locus carry out actual motion according to the pseudo operation model; Virtual feedback model shows real machine people's moving situation in real time on display window; Can keep watch on real machine people's operation situation through the video monitor window.The operator can remote control REMOTE MACHINE people accomplish the operation of high accuracy and relative complex, thereby realizes human in the harmful safe and reliable operation that reaches under the dynamic environment of danger.
This technical scheme has following beneficial effect:
1. of the present inventionly have the feedback transmission short time delay, can add the feedback delay compensation, can be from arbitrarily angled reflection REMOTE MACHINE people's moving situation, can show the information of more doing more physical exercises, combine with video integration technology ingenious, characteristics that the while can also observe virtual and real REMOTE MACHINE people's moving situation intuitively; Compare with traditional remote control system; The operation stand-by period shortens greatly; Improved the accuracy and the security of distant operation, operating efficiency is increased substantially.
2. the present invention is through two dummy models stack Display Techniques, and the operator can accomplish the operation of high accuracy and relative complex by remote control distal ends robot, thereby realizes human in the harmful safe and reliable operation that reaches under the dynamic environment of danger.
Description of drawings:
Fig. 1 is that novel distant operation REMOTE MACHINE people of the present invention controls the platform framework block diagram.
Fig. 2 is a control program operational flowchart of the present invention.
The specific embodiment of the present invention:
Embodiment 1:
Novel distant operation REMOTE MACHINE people controls platform; Its composition comprises: client-side 1 and server 2; Described client-side 1 arrives server 2 through network data transmission 3; Described client-side 1 comprises human-computer interaction device 4, and described human-computer interaction device 4 connects simultaneously that video shows 5, data comparison 6 and 13 together pseudo operation model 7 and the virtual feedback models 8 that superpose; Described server 2 comprises REMOTE MACHINE people control module 9, and described REMOTE MACHINE people control module 9 connects REMOTE MACHINE people 10, and described REMOTE MACHINE people 10 connects data acquisition unit 11 and video surveillance devices 12.
Described novel distant operation REMOTE MACHINE people controls platform, and described pseudo operation model is connected video and shows with virtual feedback model, and described virtual feedback model connects data relatively.
Embodiment 2:
Embodiment 1 described novel distant operation REMOTE MACHINE people controls the method for operating of platform, and described operating process shows two dummy models---pseudo operation model and virtual feedback model in same display window stack;
Described virtual feedback model is synchronous with true REMOTE MACHINE people's motion; The operator is through pseudo operation modeling REMOTE MACHINE people's motion; And movable information sent to the distal ends robot, and the motion of virtual feedback model is to be controlled by the motion feedback information of distal ends robot;
Described feedback information is real-time joint motions sensor information; Data are little; Short time delay, and can remedy the motion add up error that time delay causes through the model angle compensation, make the motion of virtual feedback model and true REMOTE MACHINE people's moving situation be consistent;
Described pair of dummy model shows and the fusion of far-end video feed information, can be more prone to detect with control intuitively through the true REMOTE MACHINE people's of video monitor window real time monitoring job status;
It is that the control system is judged through data that final position between described pseudo operation model and the virtual feedback model overlaps, and overlaps to judge it is accurate, does not overlap if motion is accomplished, and system then adds motion compensation according to data;
Described control program can be required by operation, and the track of real-time rendering two dummy model end effectors according to the display window track of being drawn, can be judged the validity of true REMOTE MACHINE people motion;
Described virtual feedback model can embody multiple real information more easily, intuitively reflects through change color and carries out short stressing conditions;
Described two dummy models can amplify with local in arbitrarily angled rotation adjustment, can each distance from different perspectives go to observe two position relations between the model.
Embodiment 3:
(1) open system servo power supply, planning REMOTE MACHINE people operation track, client-side control pseudo operation model, whether its running orbit of movement observation meets job requirements, comprises whether whether level and smooth, the final pose of avoiding barrier location is accurate.If will see the moving situation of looking into the pseudo operation model in detail, select track drafting to carry out detailed track and draw, can obtain the track of pseudo operation model clearly at display window.
(2) if the pseudo operation model satisfies corresponding job requirements; Client-side sends operating instruction and gives the true REMOTE MACHINE people of far-end; True REMOTE MACHINE people moves and beams back data simultaneously in real time to virtual feedback model, guarantee virtual feedback model and true REMOTE MACHINE people synchronously; If job requirements is not satisfied in the motion of pseudo operation model, but the joint angles of manual adjustment pseudo operation model or in the displacement of cartesian space coordinate system rationalizes movement locus.
(3) select then track drafting, but accurate Drawing goes out the movement locus of virtual feedback model, its track also is true REMOTE MACHINE people movement locus simultaneously.
(4) with operation display window concurrent working, also have video monitoring window, can open the video monitoring equipment according to job requirements, use the video integration technology working space is monitored in real time, prevent unexpected the generation, guarantee job safety.
(5) need by operation, adjustment observation angle slide block, the local slide block that amplifies, from different perspectives, far and near two dummy models are examined.
(6) fulfil assignment after, select work origin, pseudo operation model and virtual feedback model are successively got back to the operation initial point, and inregister.
Embodiment 4:
Claims (1)
1.
A kind of distant operation REMOTE MACHINE people controls the method for operating of platform, it is characterized in that: described operating process:
(1) open system servo power supply, planning REMOTE MACHINE people operation track, client-side control pseudo operation model; Whether its running orbit of movement observation meets job requirements; Comprise whether whether level and smooth, the final pose of avoiding barrier location is accurate,, select track drafting to carry out detailed track and draw if will see the moving situation of looking into the pseudo operation model in detail; Can obtain the track of pseudo operation model clearly at display window
(2) if the pseudo operation model satisfies corresponding job requirements; Client-side sends operating instruction and gives the true REMOTE MACHINE people of far-end; True REMOTE MACHINE people moves and beams back data simultaneously in real time to virtual feedback model, guarantee virtual feedback model and true REMOTE MACHINE people synchronously; If job requirements is not satisfied in the motion of pseudo operation model, but the joint angles of manual adjustment pseudo operation model or in the displacement of cartesian space coordinate system rationalizes movement locus,
(3) select track drafting, but accurate Drawing goes out the movement locus of virtual feedback model, its track also is true REMOTE MACHINE people movement locus simultaneously,
(4) with operation display window concurrent working, also have video monitoring window, can open the video monitoring equipment according to job requirements, use the video integration technology working space is monitored in real time, prevent unexpected the generation, guarantee job safety,
(5) need by operation, adjustment observation angle slide block, the local slide block that amplifies, from different perspectives, far and near two dummy models are examined,
(6) fulfil assignment after, select work origin, pseudo operation model and virtual feedback model are successively got back to the operation initial point, and inregister;
described novel distant operation REMOTE MACHINE people controls platform; Its composition comprises: client-side and server; Described client-side arrives server through network data transmission; Described client-side comprises the human-computer interaction device, described human-computer interaction device connects simultaneously that video shows, data comparison and two dummy models of being superimposed superpose Display Techniques pseudo operation model and virtual feedback model; Described server comprises REMOTE MACHINE people control module, and described REMOTE MACHINE people control module connects the REMOTE MACHINE people, and described REMOTE MACHINE people connects data acquisition unit and video surveillance devices; Described pseudo operation model is connected video and shows with virtual feedback model, described virtual feedback model connects data relatively, and described virtual feedback model embodies multiple real information more easily, intuitively reflects through change color and carries out short stressing conditions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010032439 CN102120325B (en) | 2010-01-08 | 2010-01-08 | Novel remote operation far-end robot control platform and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010032439 CN102120325B (en) | 2010-01-08 | 2010-01-08 | Novel remote operation far-end robot control platform and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102120325A CN102120325A (en) | 2011-07-13 |
| CN102120325B true CN102120325B (en) | 2012-12-12 |
Family
ID=44249040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010032439 Expired - Fee Related CN102120325B (en) | 2010-01-08 | 2010-01-08 | Novel remote operation far-end robot control platform and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102120325B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3572914A3 (en) * | 2018-05-24 | 2020-02-19 | TMRW Foundation IP & Holding S.A.R.L. | Two-way real-time 3d interactive operations of real-time 3d virtual objects within a real-time 3d virtual world representing the real world |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102825603B (en) * | 2012-09-10 | 2015-01-21 | 江苏科技大学 | Network teleoperation robot system and time delay overcoming method |
| CN103273495B (en) * | 2013-06-09 | 2016-01-20 | 中国科学院自动化研究所 | A kind of micro assemby real time demonstration based on OpenGL and collision detecting system |
| CN103722554B (en) * | 2014-01-15 | 2015-12-09 | 西北工业大学 | A kind of Space teleoperation method |
| CN104157114A (en) * | 2014-04-28 | 2014-11-19 | 北京理工大学 | Wireless transmission-based intelligent deformation magic scale |
| CN105137860B (en) * | 2015-07-31 | 2018-07-20 | 重庆世纪精信实业（集团）有限公司 | Industry mechanical arm remotely monitor based on internet and control system |
| CN106078747B (en) * | 2016-08-11 | 2018-11-06 | 贵州翰凯斯智能技术有限公司 | A kind of delay industrial operation control system based on virtual reality |
| CN106227166A (en) * | 2016-08-11 | 2016-12-14 | 贵州翰凯斯智能技术有限公司 | A kind of industrial real-time operation control system based on virtual reality |
| CN106157377B (en) * | 2016-08-22 | 2019-07-09 | 南京航空航天大学 | A kind of matching process of the real-time three-dimensional visualization virtual monitor towards intelligence manufacture |
| CN106354043A (en) * | 2016-11-05 | 2017-01-25 | 杭州畅动智能科技有限公司 | Control system of device |
| JP6944098B2 (en) | 2018-05-24 | 2021-10-06 | ザ カラニー ホールディング エスエーアールエル | Systems and methods for developing and testing digital real-world applications through the virtual world and deploying them in the real world |
| CN109984514B (en) * | 2019-03-22 | 2022-08-09 | 杨松 | Electric bed remote controller and method for remotely controlling electric bed |
| CN110225100B (en) * | 2019-05-22 | 2021-01-05 | 清华大学 | Virtual-real mapping system for intelligent assembly production line |
| US11115468B2 (en) | 2019-05-23 | 2021-09-07 | The Calany Holding S. À R.L. | Live management of real world via a persistent virtual world system |
| CN112100798A (en) | 2019-06-18 | 2020-12-18 | 明日基金知识产权控股有限公司 | System and method for deploying virtual copies of real-world elements into persistent virtual world systems |
| US11196964B2 (en) | 2019-06-18 | 2021-12-07 | The Calany Holding S. À R.L. | Merged reality live event management system and method |
| CN110181517B (en) * | 2019-06-21 | 2022-05-10 | 西北工业大学 | Double teleoperation training method based on virtual clamp |
| CN110413110A (en) * | 2019-07-05 | 2019-11-05 | 深圳市工匠社科技有限公司 | The control method and Related product of virtual role |
| CN113146612A (en) * | 2021-01-05 | 2021-07-23 | 上海大学 | Virtual-real combination and man-machine interaction underwater remote control robot manipulator operation system and method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1785608A (en) * | 2005-11-10 | 2006-06-14 | 上海大学 | Control platform of multifinger mechanical skillful closed ring real time action |
| CN1843712A (en) * | 2006-05-12 | 2006-10-11 | 上海大学 | Flexible and remote-controlled operation platform based on virtual robot |
-
2010
- 2010-01-08 CN CN 201010032439 patent/CN102120325B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1785608A (en) * | 2005-11-10 | 2006-06-14 | 上海大学 | Control platform of multifinger mechanical skillful closed ring real time action |
| CN1843712A (en) * | 2006-05-12 | 2006-10-11 | 上海大学 | Flexible and remote-controlled operation platform based on virtual robot |
Non-Patent Citations (1)
| Title |
|---|
| 李东洁等.基于Internet的遥机器人控制系统.《哈尔滨理工大学学报》.2009,第14卷(第4期),第41-45页. * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3572914A3 (en) * | 2018-05-24 | 2020-02-19 | TMRW Foundation IP & Holding S.A.R.L. | Two-way real-time 3d interactive operations of real-time 3d virtual objects within a real-time 3d virtual world representing the real world |
| JP7082416B2 (en) | 2018-05-24 | 2022-06-08 | ザ カラニー ホールディング エスエーアールエル | Real-time 3D that expresses the real world Two-way real-time 3D interactive operation of real-time 3D virtual objects in a virtual world |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102120325A (en) | 2011-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102120325B (en) | Novel remote operation far-end robot control platform and method | |
| CN111633644A (en) | Industrial robot digital twin system combined with intelligent vision and operation method thereof | |
| CN109434870A (en) | A kind of virtual reality operation system for robot livewire work | |
| CN103302668B (en) | Based on control system and the method thereof of the Space teleoperation robot of Kinect | |
| CN106527177A (en) | Multi-functional and one-stop type remote control design, the simulation system and method thereof | |
| CN105786483B (en) | Welding robot off-line programing system and its off-line programing method based on tablet computer | |
| CN109262609A (en) | Mechanical arm tele-control system and method based on virtual reality technology | |
| CN110394780A (en) | The simulator of robot | |
| CN103273497B (en) | Man-machine interactive manipulator control system and control method thereof | |
| CN108908298B (en) | Master-slave type spraying robot teaching system fusing virtual reality technology | |
| CN101587329A (en) | Robot predicting method and system | |
| CN109268010A (en) | A kind of long-range inspection interference method of virtual reality Mechanized | |
| CN107257946B (en) | System for virtual debugging | |
| CN110216674A (en) | A kind of redundant degree of freedom mechanical arm visual servo obstacle avoidance system | |
| CN109434826A (en) | A kind of hot line robot control system | |
| CN110977981A (en) | Robot virtual reality synchronization system and synchronization method | |
| CN110682291A (en) | Robot teleoperation system based on VR and teleoperation method thereof | |
| CN105108762A (en) | Remote control hand controller based on force/touch guiding | |
| CN205184791U (en) | Distant manipulation hand controller based on power / sense of touch guide | |
| CN110039561A (en) | Hot line robot remote operating staff training system and method based on cloud | |
| Mostefa et al. | Design of mobile robot teleoperation system based on virtual reality | |
| Le et al. | Intuitive virtual reality based control of a real-world mobile manipulator | |
| Khrueangsakun et al. | Design and development of cyber physical system for real-time web-based visualization and control of robot arm | |
| CN115303515A (en) | Space double-arm on-orbit operation-oriented astronaut in-cabin operation and display control system | |
| CN202079595U (en) | Novel control platform for tele-operation of remote robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20121212 Termination date: 20140108 |