CN111970502A - Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot - Google Patents
Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot Download PDFInfo
- Publication number
- CN111970502A CN111970502A CN202010882933.3A CN202010882933A CN111970502A CN 111970502 A CN111970502 A CN 111970502A CN 202010882933 A CN202010882933 A CN 202010882933A CN 111970502 A CN111970502 A CN 111970502A
- Authority
- CN
- China
- Prior art keywords
- binocular
- follow
- robot
- teleoperation
- support frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/296—Synchronisation thereof; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Abstract
The invention relates to the technical field of teleoperation of robots, in particular to a humanoid binocular follow-up virtual reality system suitable for teleoperation of robots, which comprises the following components: the two-dimensional platform is arranged on the robot body and provided with a follow-up mechanism; a binocular camera disposed on the two-dimensional platform; the VR helmet is worn on the head of an operator, and a gyroscope is arranged in the VR helmet and used for detecting head movement information; the control module is connected with the gyroscope and the follow-up mechanism, and the control module is used for controlling the follow-up mechanism to drive the binocular camera to move according to the head motion information detected by the gyroscope.
Description
Technical Field
The invention relates to the technical field of teleoperation of robots, in particular to a humanoid binocular follow-up virtual reality system suitable for teleoperation of robots.
Background
At present, the environment perception of the robot mainly adopts a visible light camera or a laser radar with a fixed position relative to a machine body, the field range of the robot is limited to a certain extent, the robot is difficult to aim at a target object when vibrating or shaking interference occurs, and particularly, the robot is remotely operated by using a master-slave control handle. The teleoperation system of robot usually adopts the camera to gather image information, shows image information through display or virtual reality helmet to make operating personnel watch the operation environment of robot in real time, reach the purpose of visual feedback. However, most of cameras in the conventional method are monocular cameras, the visual angle is relatively fixed, the stereoscopic impression of the operator is difficult to be brought to the scene, the visual angle of the camera cannot be adjusted in real time according to the observation requirement of the operator, and the visual field range is changed.
Disclosure of Invention
The invention aims to provide a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot.
In order to achieve the purpose, the invention provides the following technical scheme:
a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot, comprising:
the two-dimensional platform is arranged on the robot body and provided with a follow-up mechanism;
a binocular camera disposed on the two-dimensional platform;
the VR helmet is provided with a gyroscope, and the gyroscope is used for detecting head movement information;
the control module is connected with the gyroscope and the follow-up mechanism, and the control module is used for controlling the follow-up mechanism to drive the binocular camera to move according to the head motion information detected by the gyroscope.
Preferably, the humanoid binocular follow-up virtual reality system further comprises a communication module, the communication module is connected with the binocular camera and the VR helmet, and the communication module is used for transmitting image information shot by the binocular camera to the VR helmet.
Preferably, the two-dimensional platform further comprises an upper support frame and a lower support frame, the upper support frame is arranged on the lower support frame, and the binocular camera is arranged on the upper support frame.
Preferably, the follower mechanism includes a pitch motor for driving the upper support frame and a rotation motor for driving the lower support frame.
Preferably, the control module generates a pitch rotation instruction and a horizontal rotation instruction after receiving the head movement information detected by the gyroscope, and the pitch motor and the rotating motor respectively drive the upper support frame and the lower support frame to rotate according to the pitch rotation instruction and the horizontal rotation instruction.
Preferably, a first encoder is arranged in the pitching motor, a second encoder is arranged in the rotating motor, and the first encoder and the second encoder are used for detecting the pitching rotation angle and the horizontal rotation angle and feeding back the detected pitching rotation angle and the detected horizontal rotation angle to the control module.
Preferably, the VR helmet further comprises a detection lens, the detection lens is used for detecting the interpupillary distance of an operator, the binocular camera is provided with two camera lenses, and the control module is further used for adjusting the distance between the two camera lenses according to the interpupillary distance of the operator detected by the detection lens.
Preferably, the binocular camera further comprises an electric slide rail connected with the control module, and the electric slide rail is used for driving the two camera lenses to approach or be far away.
Compared with the prior art, the invention has the beneficial effects that:
(1) the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot, provided by the embodiment of the invention, places the binocular camera on the two-dimensional pan-tilt, and controls the binocular camera to synchronously pitch and rotate along with the head movement of an operator through the pitch motor and the rotating motor of the follow-up mechanism, so that the visual angle of the binocular camera is changed, the field range is changed, a binocular stereoscopic vision similar to human eyes can be obtained, and the distance sense, the stereoscopic sense and the immersion sense of an object in a three-dimensional space can be better presented;
(2) the binocular camera of the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot provided by the embodiment of the invention is provided with the two camera lenses, and the distance between the two camera lenses can be adjusted through the electric slide rail, so that the distance is matched with the interpupillary distance of two eyes of an operator, the VR helmet can image clearly, the visual habit of a human body is met, the dizzy feeling of the operator is reduced, and the binocular camera is more suitable for the operator to watch.
Drawings
Fig. 1 is one of structural block diagrams of a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention;
fig. 2 is a second structural block diagram of a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a two-dimensional platform of a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a binocular camera of the humanoid binocular follower virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention.
In the figure: 1. a two-dimensional platform; 11. an upper support frame; 12. a lower support frame; 13. a pitch motor; 14. a rotating electric machine; 15. a first encoder; 16. a second encoder; 2. a binocular camera; 21. a camera lens; 22. an electric slide rail; 3. a VR helmet; 31. a gyroscope; 32. detecting a lens; 4. a communication module; 5. and a control module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is one of structural block diagrams of a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention. As shown in fig. 1, an embodiment of the present invention provides a humanoid binocular following virtual reality system suitable for teleoperation of a robot, which may include:
the robot comprises a two-dimensional platform 1, wherein the two-dimensional platform 1 can be arranged on a robot body;
a binocular camera 2, the binocular camera 2 may be disposed on the two-dimensional platform 1;
the VR helmet 3 is worn on the head of an operator;
the communication module 4 is connected with the binocular camera 2 and the VR helmet 3, and the communication module 4 can be used for transmitting image information shot by the binocular camera 2 to the VR helmet 3;
when the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot is used, an operator teleoperates the robot body, image information shot by the binocular camera 2 on the two-dimensional platform 1 of the robot body is transmitted to the VR helmet 3 worn by the operator through the communication module 4, and therefore the operator can view the operation environment of the robot body in real time and the purpose of visual feedback is achieved.
In the embodiment of the present invention, examples of the communication module 4 may include, but are not limited to, bluetooth, WiFi, or a serial bus, etc., which are known to those skilled in the art.
Fig. 2 is a second structural block diagram of a human-like binocular follow-up virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention. As shown in fig. 2, in an embodiment of the present invention, the two-dimensional platform 1 of the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot is provided with a follow-up mechanism, a gyroscope 31 is arranged in the VR helmet 3, the gyroscope 31 is used for detecting head movement information, the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot further comprises a control module 5, the control module 5 is connected with the gyroscope 31 and the follow-up mechanism, and the control module 5 is used for controlling the follow-up mechanism to drive the binocular camera 2 to move according to the head movement information detected by the gyroscope 31. Through the technical scheme, the binocular camera 2 can be controlled to move synchronously along with the head movement of an operator through the following mechanism of the two-dimensional platform 1, so that the visual angle of the binocular camera is changed, the field range is changed, binocular stereoscopic vision similar to human eyes can be obtained, and the distance sense, the stereoscopic sense and the immersion sense of an object in a three-dimensional space can be better presented.
Fig. 3 is a schematic structural diagram of a two-dimensional platform of a humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention. Specifically, in one embodiment of the present invention, as shown in fig. 3, the two-dimensional platform 1 of the humanoid binocular follower virtual reality system suitable for teleoperation of the robot may further include an upper support frame 11 and a lower support frame 12, the upper support frame 11 may be disposed on the lower support frame 12, and the binocular camera 2 may be disposed on the upper support frame 11.
Further, as shown in fig. 3, the following mechanism of the two-dimensional platform 1 includes a pitch motor 13 that can be used to drive the upper support frame 11 and a rotation motor 14 that can be used to drive the lower support frame 12.
As shown in fig. 2, the control module 5 of the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot generates a pitch rotation instruction and a horizontal rotation instruction after receiving head movement information detected by the gyroscope 31 of the VR helmet 3, and the pitch motor 13 and the rotating motor 14 respectively drive the upper support frame 11 and the lower support frame 12 to rotate according to the pitch rotation instruction and the horizontal rotation instruction. Through the technical scheme, when the head of an operator moves, the control module 5 receives head motion information detected by the gyroscope 31 and then resolves the posture of the VR helmet 3 to obtain the pitch angle and the rotation angle of the head of the operator, so that a pitch rotation instruction and a horizontal rotation instruction are generated, the pitch motor 13 and the rotation motor 14 of the two-dimensional holder 1 are synchronously or respectively controlled to rotate to change the visual angle of the binocular camera 2, and the telepresence of teleoperation is improved.
In embodiments of the present invention, examples of control module 5 may include, but are not limited to, a general purpose processor, a special purpose processor, a conventional processor, a Digital Signal Processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of Integrated Circuit (IC), a state machine, a System On Chip (SOC), and the like.
As shown in fig. 2, in an embodiment of the present invention, a first encoder 15 may be disposed in the pitching motor 13, and a second encoder 16 may be disposed in the rotating motor 14, wherein the first encoder 15 and the second encoder 16 are used for detecting the pitching rotation angle and the horizontal rotation angle and feeding them back to the control module 5. Through the detection and feedback of the pitching rotation angle and the horizontal rotation angle by the first encoder 15 and the second encoder 16, the control module 5 can more accurately control the visual angle of the binocular camera 2, so that the visual angle of the binocular camera is consistent with that of an operator, and the field information can be accurately and timely acquired by the operator.
Fig. 4 is a schematic structural diagram of a binocular camera of the humanoid binocular follower virtual reality system suitable for teleoperation of a robot according to an embodiment of the present invention. As shown in fig. 4, in an embodiment of the present invention, the VR headset 3 of the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot may further include a detection lens 32, the detection lens 32 may be configured to detect a pupillary distance of an operator, the binocular camera 2 is provided with two camera lenses 21, and the control module 5 may be further configured to adjust an interval between the two camera lenses 21 according to the pupillary distance of the operator detected by the detection lens 32. Through above-mentioned technical scheme, can adjust two camera lens 21's of binocular camera 2 interval to match with the interpupillary distance of operating personnel two eyes, make VR helmet 3 imaging clear, accord with human vision custom simultaneously, reduce the dizzy sense of operating personnel, more be fit for operating personnel and watch.
Specifically, as shown in fig. 2 and 4, the binocular camera 2 of the humanoid binocular follow-up virtual reality system suitable for teleoperation of the robot may further include an electric slide rail 22 connected to the control module 5, and the electric slide rail 22 may be configured to drive the two camera lenses 21 to approach or move away from each other. Through the technical scheme, the control module 5 controls the two camera lenses 21 of the electric slide rail 22 to approach or leave according to the pupil distance of the operator detected by the detection lens 32, so that the distance between the two camera lenses 21 is adjusted, and the pupil distance of the two eyes of the operator is matched.
The using method comprises the following steps:
when the humanoid binocular follow-up virtual reality system suitable for remote operation of a robot is used, after an operator wears a VR helmet 3, the pupil distance of the operator is detected by a detection lens 32 in the VR helmet 3, and a control module 5 controls two camera lenses 21 of an electric slide rail 22 to approach or depart from the pupil distance of the operator, so that the distance between the two camera lenses 21 is adjusted, and the pupil distance is matched with the pupil distance of two eyes of the operator; an operator remotely operates the robot body, and image information shot by the binocular camera 2 on the two-dimensional platform 1 of the robot body is transmitted to the VR helmet 3 worn by the operator through the communication module 4, so that the operator can view the operation environment of the robot body in real time to achieve the purpose of visual feedback; when the head of an operator moves, the control module 5 receives head motion information detected by the gyroscope 31, and then the head motion information is resolved by the posture of the VR helmet 3 to obtain the pitch angle and the rotation angle of the head of the operator, so as to generate a pitch rotation instruction and a horizontal rotation instruction, and synchronously or respectively control the pitch motor 13 and the rotation motor 14 of the two-dimensional pan-tilt 1 to rotate to change the visual angle of the binocular camera 2, so as to change the visual field range, thereby obtaining a binocular stereoscopic vision similar to human eyes, and the distance, the stereoscopic impression and the immersion impression of objects in the three-dimensional space can be better presented.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. A humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot is characterized by comprising:
the robot comprises a two-dimensional platform (1), wherein the two-dimensional platform (1) is arranged on a robot body, and the two-dimensional platform (1) is provided with a follow-up mechanism;
the binocular camera (2), the said binocular camera (2) is set up on the said two-dimensional terrace (1);
a VR headset (3) in which a gyroscope (31) is arranged, the gyroscope (31) being used to detect head movement information;
the control module (5), the control module (5) with gyroscope (31) with the follow-up mechanism is connected, control module (5) are used for controlling according to the head motion information that gyroscope (31) detected follow-up mechanism drive binocular camera (2) remove.
2. The binocular human-simulated follow-up virtual reality system suitable for teleoperation of a robot of claim 1, further comprising a communication module (4), wherein the communication module (4) is connected with the binocular camera (2) and the VR helmet (3), and the communication module (4) is used for transmitting image information shot by the binocular camera (2) to the VR helmet (3).
3. The humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot of claim 1, wherein the two-dimensional platform (1) further comprises an upper support frame (11) and a lower support frame (12), the upper support frame (11) is disposed on the lower support frame (12), and the binocular camera (2) is disposed on the upper support frame (11).
4. The humanoid binocular follow-up virtual reality system suitable for teleoperation of a robot of claim 3, wherein the follow-up mechanism comprises a pitch motor (13) for driving the upper support frame (11) and a rotation motor (14) for driving the lower support frame (12).
5. The system of claim 4, wherein the control module (5) generates a pitch instruction and a horizontal instruction after receiving the head motion information detected by the gyroscope (31), and the pitch motor (13) and the rotating motor (14) respectively drive the upper support frame (11) and the lower support frame (12) to rotate according to the pitch instruction and the horizontal instruction.
6. The binocular human-simulated follow-up virtual reality system suitable for teleoperation of the robot is characterized in that a first encoder (15) is arranged in the pitching motor (13), a second encoder (16) is arranged in the rotating motor (14), and the first encoder (15) and the second encoder (16) are used for detecting a pitching rotation angle and a horizontal rotation angle and feeding back the pitching rotation angle and the horizontal rotation angle to the control module.
7. The system of any one of claims 1 to 6, wherein the VR helmet (3) further comprises a detection lens (32), the detection lens (32) is used for detecting the interpupillary distance of an operator, the binocular camera (2) is provided with two camera lenses (21), and the control module (5) is further used for adjusting the distance between the two camera lenses (21) according to the interpupillary distance of the operator detected by the detection lens (32).
8. The binocular robot binocular follow-up virtual reality system suitable for teleoperation of the robot according to any one of claim 7, wherein the binocular camera (2) further comprises an electric slide rail (22) connected with the control module (5), and the electric slide rail (22) is used for driving the two camera lenses (21) to approach or move away.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010882933.3A CN111970502A (en) | 2020-08-28 | 2020-08-28 | Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010882933.3A CN111970502A (en) | 2020-08-28 | 2020-08-28 | Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111970502A true CN111970502A (en) | 2020-11-20 |
Family
ID=73399775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010882933.3A Pending CN111970502A (en) | 2020-08-28 | 2020-08-28 | Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111970502A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461077A (en) * | 2022-03-09 | 2022-05-10 | 南昌威爱信息科技有限公司 | Industrial product design VR show platform |
-
2020
- 2020-08-28 CN CN202010882933.3A patent/CN111970502A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461077A (en) * | 2022-03-09 | 2022-05-10 | 南昌威爱信息科技有限公司 | Industrial product design VR show platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6339239B2 (en) | Head-mounted display device and video display system | |
| WO2016017245A1 (en) | Information processing device, information processing method, and image display system | |
| US10353207B2 (en) | Head-mounted display device and video display system | |
| JP6582921B2 (en) | Robot monitor system | |
| CN104781873A (en) | Image display device and image display method, mobile body device, image display system, and computer program | |
| JP2006502675A (en) | Bio-type automatic vision and gaze control system based on biological eye movement system | |
| JP7003633B2 (en) | Transparent display device, display control method, and computer program | |
| WO2017213070A1 (en) | Information processing device and method, and recording medium | |
| WO2017169841A1 (en) | Display device and display control method | |
| CN111970502A (en) | Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot | |
| US11833698B2 (en) | Vision system for a robot | |
| JP2016151566A (en) | Motion tracker device | |
| CN105828021A (en) | Specialized robot image acquisition control method and system based on augmented reality technology | |
| CN212628184U (en) | Humanoid binocular follow-up virtual reality system suitable for teleoperation of robot | |
| JP2017100204A (en) | Robot operation system | |
| JP2005297168A (en) | Remote control robot | |
| US20180268218A1 (en) | Information display system | |
| WO2021117606A1 (en) | Image processing device, system, image processing method and image processing program | |
| CN115076561A (en) | Tele-immersion type binocular holder follow-up system and method applied to engineering machinery | |
| JP6038701B2 (en) | Remote visual recognition device and remote visual operation system | |
| KR101459213B1 (en) | Stereo camera rig control device and control method thereof | |
| Watanabe et al. | The development of a 3D position measurement system for indoor aerial robots | |
| CN217020366U (en) | Working robot | |
| CN215871664U (en) | Video glasses | |
| Schneider et al. | Vision system for wearable and robotic uses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |