CN107911687B - Robot teleoperation auxiliary system based on binocular stereo vision - Google Patents
Robot teleoperation auxiliary system based on binocular stereo vision Download PDFInfo
- Publication number
- CN107911687B CN107911687B CN201711302665.8A CN201711302665A CN107911687B CN 107911687 B CN107911687 B CN 107911687B CN 201711302665 A CN201711302665 A CN 201711302665A CN 107911687 B CN107911687 B CN 107911687B
- Authority
- CN
- China
- Prior art keywords
- image data
- camera
- measurement
- processor
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a robot teleoperation auxiliary method based on binocular stereo vision, which comprises the following steps: an image data output step: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera; an image data processing step: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment; an image data display step: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch. The invention has high measurement precision, large field of view and easy target search.
Description
Technical Field
The invention relates to binocular stereo vision measurement and display technology, in particular to a robot teleoperation auxiliary method and system based on binocular stereo vision.
Background
The teleoperation robot can assist an operator to complete various tasks, and particularly needs to replace manual operation under the severe environment which is not suitable for field operation of the operator. An important part of teleoperation is the perception of the environment and the situation of the target. The vision sensor is the key for the robot to sense the environment, and a general two-dimensional image sensor cannot sense the depth information of the environment, needs to be used in combination with other sensors such as laser ranging and the like, and is complex in structure. Two-dimensional image information fed back by a general robot vision sensor cannot completely describe a three-dimensional environment, and a general teleoperation interaction medium cannot express real three-dimensional information, so that an operator cannot experience the experience of being personally on the scene. The existing binocular stereo vision system has single function, and if the field of view is required to be large, the precision is insufficient; however, the measurement accuracy is required to be high, the field of view is small, and the target is difficult to search.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a robot teleoperation auxiliary method and system based on binocular stereo vision.
In order to achieve the purpose, the invention adopts the following technical scheme:
the robot teleoperation auxiliary method based on binocular stereo vision comprises the following steps:
an image data output step: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera;
an image data processing step: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display step: and the stereo display equipment displays the image data output by the stereo display processor.
The image data output step includes:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
The image data output step includes:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
The stereoscopic display equipment comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual requirements.
The display helmet synthesizes the images of the left monitoring camera and the right monitoring camera into an image with a size of 1920 multiplied by 1080, and the image is sent to an LED liquid crystal display screen in the display helmet through an HDMI interface to be displayed.
The image data processing step comprises:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
In order to achieve the purpose, the invention also adopts the following technical scheme:
provided is a robot teleoperation auxiliary system based on binocular stereo vision, including:
an image data output unit: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera;
an image data processing unit: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display unit: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch.
The image data output unit:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
The image data output unit:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
The stereoscopic display equipment comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual requirements for an operator to watch.
The display helmet synthesizes the images of the left monitoring camera and the right monitoring camera into an image with a size of 1920 multiplied by 1080, and the image is sent to an LED liquid crystal display screen in the display helmet through an HDMI interface to be displayed.
In the image data processing unit:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
The invention has the beneficial effects that: the invention discloses a robot teleoperation auxiliary method and device based on binocular stereo vision, which are used for carrying out high-precision adjustment and calibration on a measuring camera and a monitoring camera according to a binocular camera erection scheme designed by combining measurement and monitoring functions, correcting and displaying binocular images by adopting a stereo helmet, manually marking interested targets and carrying out pose measurement and calculation. The invention can carry out real-time large-view-field immersion type monitoring, the operator has strong personally on-the-scene feeling, and meanwhile, the gesture measurement and calculation are carried out on the interested target, the actual measurement precision is superior to 0.95mm and 0.58 degrees, and the invention can effectively assist the operator to carry out teleoperation on the robot.
Drawings
Fig. 1 is a flow chart of the robot teleoperation assisting method based on binocular stereo vision according to the present invention.
Fig. 2 is a schematic view showing the coverage of the field of view in the horizontal direction at different horizontal angles.
Fig. 3 shows an optical design of a measuring camera.
Fig. 4 is a schematic view of the optical design of the monitoring camera.
Fig. 5 is a block diagram of the robot teleoperation auxiliary system based on binocular stereo vision according to the present invention.
Fig. 6 is a schematic block circuit diagram of the image measurement processor.
Fig. 7 is a block diagram showing an algorithm flow of the image measurement processor.
FIG. 8a shows a target depth map generated by the left measurement camera.
FIG. 8b shows the target depth map generated by the right measurement camera.
Fig. 9 is a diagram showing the coordinates of the object pose.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a robot teleoperation assisting method based on binocular stereo vision, including the following steps:
and step 3 is an image data display step: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch.
In the robot teleoperation auxiliary method based on binocular stereo vision, the robot teleoperation auxiliary method is practically applied to be composed of four cameras which are divided into two groups, the middle two cameras are monitoring cameras, the outer two cameras are measuring cameras, and a cube mirror for adjusting is installed at the top of each camera. In addition, the coordinate system of the imaging surface of the camera and the coordinate system of a prism of the camera are adjusted to be parallel through the auto-collimation theodolite, a reference prism is placed between the two monitoring cameras, the optical axes of the monitoring cameras are ensured to be parallel by adjusting the angle between the prism of the camera and the reference prism, and the included angle between the optical axis of the camera and the horizontal plane is measured to be 79 degrees. The internal and external parameter calibration of the four cameras uses a checkerboard calibration board, and the internal and external parameter calibration of the monitoring camera and the measuring camera is completed by using a Zhang Zhengyou plane calibration method.
In one embodiment, in the image data output step:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
As shown in fig. 2, in order to ensure the measurement accuracy and the working range of the measuring camera, when the focal length of the optical system of the measuring camera is designed to be 12mm, the field angle is 42 ° × 27 °, the base line B is fixed to be 183mm due to the volume limitation of the system, and the measurement distance is 350mm to 800mm, and the analytical relationship between the working range d1 d2 of the measuring camera and the arrangement angle α and the base line B of the camera is established, so that the arrangement of the binocular camera is determined.
As shown in fig. 3, when the horizontal included angle is 79 °, 254.4mm corresponds to a horizontal field angle of 40 ° when the horizontal included angle is measured from the optical axis of the camera to the horizontal, and 465.1mm corresponds to a horizontal field angle of 32 ° when the horizontal included angle is measured from the optical axis of the camera to the horizontal when the horizontal included angle is 350mm, and 465.1mm corresponds to a horizontal field angle of 32 ° when the horizontal included angle is 800 mm. Therefore, the optical axes of the two measuring cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
As shown in fig. 4, when the horizontal included angle is different, the coverage of the horizontal viewing field is 254.4mm at a distance of 350mm, which corresponds to a horizontal angle of view of 40 °, and is 465.1mm at a distance of 800mm, which corresponds to a horizontal angle of view of 32 °, when the optical axes of the left and right surveillance cameras and the horizontal included angle are 79 °. Therefore, the optical axes of the two cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
In one embodiment, the image data output step comprises:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor. The electronic part of the monitoring camera is designed the same as the electronic part of the measuring camera, and is composed of an on-the-eye VITA area array CMOS image sensor and a peripheral circuit, and digital image data is output according to a driving time sequence provided by an image measuring processor and transmitted to the image measuring processor, wherein a schematic circuit block diagram of the image measuring processor is shown in FIG. 6.
In one embodiment, the stereoscopic display device comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual needs for an operator to watch.
In one embodiment, the display helmet combines the left and right surveillance camera images into a 1920 × 1080 size image that is sent to the LED lcd display in the display helmet via an HDMI interface for display.
In one embodiment, the image data processing step:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
It should be noted that an edge is one of the most basic features of an image, which can greatly reduce the information to be processed but retain the shape information of the object in the image. The invention extracts the image edge as the main characteristic to complete the posture measurement of the three-dimensional object. As shown in fig. 7, the image measurement processor mainly performs edge extraction, edge matching, three-dimensional reconstruction, spatial plane fitting, and pose measurement of the left and right camera images.
It should be noted that, as shown in fig. 8a and 8b, two sets of target depth maps are generated by a binocular measuring camera, an algorithm automatically extracts a saliency plane of a target, performs three-dimensional reconstruction and spatial plane fitting, and calculates a pose of the target. The target pose coordinates are shown in fig. 9, and the target significant plane equation, the position coordinates and the normal angle side of the target position coordinates shown in fig. 9 refer to the following table;
| objective significant plane equation | Position coordinates | Normal angle | |
| Position 1 | -0.00192144x+0.00111288y+0.0212844z＝1 | (3.39，-5.572，47.43) | (84.84，87.01，5.95) |
| |
0.000629517x+0.000639321y+0.0208609z＝1 | (1.60，-5.926，48.16) | (88.27，88.24，3.00) |
In order to achieve the purpose, the invention also adopts the following technical scheme:
as shown in fig. 5, there is provided a robot teleoperation assistance system based on binocular stereo vision, including:
an image data output unit: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera;
an image data processing unit: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display unit: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch.
In the robot teleoperation auxiliary method based on binocular stereo vision, the robot teleoperation auxiliary method is practically applied to be composed of four cameras which are divided into two groups, the middle two cameras are monitoring cameras, the outer two cameras are measuring cameras, and a cube mirror for adjusting is installed at the top of each camera. In addition, the coordinate system of the imaging surface of the camera and the coordinate system of a prism of the camera are adjusted to be parallel through the auto-collimation theodolite, a reference prism is placed between the two monitoring cameras, the optical axes of the monitoring cameras are ensured to be parallel by adjusting the angle between the prism of the camera and the reference prism, and the included angle between the optical axis of the camera and the horizontal plane is measured to be 79 degrees. The internal and external parameter calibration of the four cameras uses a checkerboard calibration board, and the internal and external parameter calibration of the monitoring camera and the measuring camera is completed by using a Zhang Zhengyou plane calibration method.
In one embodiment, in the image data output unit:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
As shown in fig. 2, in order to ensure the measurement accuracy and the working range of the measuring camera, when the focal length of the optical system of the measuring camera is designed to be 12mm, the field angle is 42 ° × 27 °, the base line B is fixed to be 183mm due to the volume limitation of the system, and the measurement distance is 350mm to 800mm, and the analytical relationship between the working range d1 d2 of the measuring camera and the arrangement angle α and the base line B of the camera is established, so that the arrangement of the binocular camera is determined.
As shown in fig. 3, when the horizontal angle is different, the coverage of the horizontal viewing field is 254.4mm corresponding to the horizontal angle of view 40 ° at a distance of 350mm, and 465.1mm corresponding to the horizontal angle of view 32 ° at a distance of 800mm, when the camera optical axis forms a 79 ° angle with the horizontal. Therefore, the optical axes of the two measuring cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
As shown in fig. 4, when the horizontal included angle is different, the coverage of the horizontal viewing field is 254.4mm at a distance of 350mm, which corresponds to a horizontal angle of view of 40 °, and is 465.1mm at a distance of 800mm, which corresponds to a horizontal angle of view of 32 °, when the optical axes of the left and right surveillance cameras and the horizontal included angle are 79 °. Therefore, the optical axes of the two cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
In one embodiment, in the image data output unit:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor. The electronic part of the monitoring camera is designed the same as the electronic part of the measuring camera, and consists of a VITA area array CMOS image sensor and a peripheral circuit in the Anson, and digital image data is output according to a driving time sequence provided by the image measuring processor and transmitted to the image measuring processor.
In one embodiment, the stereoscopic display device comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual needs for an operator to watch.
In one embodiment, the display helmet combines the left and right surveillance camera images into a 1920 × 1080 size image that is sent to the LED lcd display in the display helmet via an HDMI interface for display.
In one embodiment, the image data processing unit operates as follows:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
It should be noted that an edge is one of the most basic features of an image, which can greatly reduce the information to be processed but retain the shape information of the object in the image. The invention extracts the image edge as the main characteristic to complete the posture measurement of the three-dimensional object. As shown in fig. 7, the image measurement processor mainly performs edge extraction, edge matching, three-dimensional reconstruction, spatial plane fitting, and pose measurement of the left and right camera images.
It should be noted that, as shown in fig. 8a and 8b, two sets of target depth maps are generated by a binocular measuring camera, an algorithm automatically extracts a saliency plane of a target, performs three-dimensional reconstruction and spatial plane fitting, and calculates a pose of the target. The target pose coordinates are shown in fig. 9, and the target significant plane equation, the position coordinates and the normal angle side of the target position coordinates shown in fig. 9 refer to the following table;
| objective significant plane equation | Position coordinates | Normal angle | |
| Position 1 | -0.00192144x+0.00111288y+0.0212844z＝1 | (3.39，-5.572，47.43) | (84.84，87.01，5.95) |
| |
0.000629517x+0.000639321y+0.0208609z＝1 | (1.60，-5.926，48.16) | (88.27，88.24，3.00) |
The above-described embodiments of the present invention should not be construed as limiting the scope of the present invention. Any other corresponding changes and modifications made according to the technical idea of the present invention should be included in the protection scope of the claims of the present invention.
Claims (8)
1. A robot teleoperation auxiliary method based on binocular stereo vision is characterized by comprising the following steps:
an image data output step: outputting the digital image data to an image measurement processor through the left and right measurement cameras, and simultaneously outputting the digital image data to a stereoscopic display processor through the left and right monitoring cameras;
the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera, and the imaging plane coordinate systems of the four cameras and the prism coordinate systems of the four cameras are adjusted to be parallel through the auto-collimation theodolite; a reference prism is arranged between the left monitoring camera and the right monitoring camera, the optical axes of the two monitoring cameras are ensured to be parallel by adjusting the angle between the camera prism and the reference prism, and the included angle between the optical axes of the two measuring cameras and the horizontal plane is 79 degrees;
an image data processing step: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system;
the stereoscopic display processor receives the image data of the monitoring camera to perform polar line correction and transmits the image data of the monitoring camera to the stereoscopic display equipment;
an image data display step: and the stereo display equipment displays the image data output by the stereo display processor.
2. The robot teleoperation assisting method based on binocular stereo vision according to claim 1, wherein in the image data outputting step:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
3. The binocular stereo-vision based robot teleoperation assisting method of claim 1, wherein the stereoscopic display device comprises a stereoscopic display helmet and a stereoscopic display television for an operator to watch.
4. The binocular stereo-vision-based robot teleoperation assisting method of claim 3, wherein the display helmet synthesizes the left and right surveillance camera images into a 1920 x 1080 size image, and the image is sent to an LED liquid crystal display in the display helmet through an HDMI interface for display.
5. The binocular stereo vision based robot teleoperation assisting method of claim 1, wherein in the image data processing step:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous dynamic random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
6. A robot teleoperation auxiliary system based on binocular stereo vision is characterized by comprising:
an image data output unit:
outputting the digital image data to an image measurement processor through the left and right measurement cameras, and simultaneously outputting the digital image data to a stereoscopic display processor through the left and right monitoring cameras;
the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera, and the imaging plane coordinate systems of the four cameras and the prism coordinate systems of the four cameras are adjusted to be parallel through the auto-collimation theodolite; a reference prism is arranged between the left monitoring camera and the right monitoring camera, the optical axes of the two monitoring cameras are ensured to be parallel by adjusting the angle between the camera prism and the reference prism, and the included angle between the optical axes of the two measuring cameras and the horizontal plane is 79 degrees;
an image data processing unit: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display unit: and the stereo display equipment displays the image data output by the stereo display processor.
7. The binocular stereo-vision-based robot teleoperation assistance system of claim 6, wherein in the image data output unit:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
8. The binocular stereo-vision based robot teleoperation assistance system of claim 6, wherein in the image data processing unit:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous dynamic random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711302665.8A CN107911687B (en) | 2017-12-11 | 2017-12-11 | Robot teleoperation auxiliary system based on binocular stereo vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711302665.8A CN107911687B (en) | 2017-12-11 | 2017-12-11 | Robot teleoperation auxiliary system based on binocular stereo vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107911687A CN107911687A (en) | 2018-04-13 |
| CN107911687B true CN107911687B (en) | 2020-04-10 |
Family
ID=61853999
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711302665.8A Active CN107911687B (en) | 2017-12-11 | 2017-12-11 | Robot teleoperation auxiliary system based on binocular stereo vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107911687B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115963917B (en) * | 2022-12-22 | 2024-04-16 | 北京百度网讯科技有限公司 | Visual data processing apparatus and visual data processing method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794349A (en) * | 2010-02-09 | 2010-08-04 | 北京邮电大学 | Experimental system and method for augmented reality of teleoperation of robot |
| CN102902271A (en) * | 2012-10-23 | 2013-01-30 | 上海大学 | Binocular vision-based robot target identifying and gripping system and method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201403545A (en) * | 2012-07-13 | 2014-01-16 | Vivotek Inc | A synthetic virtual perspective image processing systems and method |
| CN103231389A (en) * | 2013-04-13 | 2013-08-07 | 李享 | Object identification method based on robot binocular three-dimensional vision |
| CN103413313B (en) * | 2013-08-19 | 2016-08-10 | 国家电网公司 | The binocular vision navigation system of electrically-based robot and method |
| CN106003036A (en) * | 2016-06-16 | 2016-10-12 | 哈尔滨工程大学 | Object grabbing and placing system based on binocular vision guidance |
-
2017
- 2017-12-11 CN CN201711302665.8A patent/CN107911687B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794349A (en) * | 2010-02-09 | 2010-08-04 | 北京邮电大学 | Experimental system and method for augmented reality of teleoperation of robot |
| CN102902271A (en) * | 2012-10-23 | 2013-01-30 | 上海大学 | Binocular vision-based robot target identifying and gripping system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107911687A (en) | 2018-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11223820B2 (en) | Augmented reality displays with active alignment and corresponding methods | |
| US10869024B2 (en) | Augmented reality displays with active alignment and corresponding methods | |
| US20160295194A1 (en) | Stereoscopic vision system generatng stereoscopic images with a monoscopic endoscope and an external adapter lens and method using the same to generate stereoscopic images | |
| CN107316319B (en) | Rigid body tracking method, device and system | |
| US9307228B2 (en) | Depth of field maintaining apparatus, 3D display system and display method | |
| CN110617814A (en) | Monocular vision and inertial sensor integrated remote distance measuring system and method | |
| TW201100760A (en) | Distance measuring apparatus having dual stereo cameras | |
| CN109238235B (en) | Method for realizing rigid body pose parameter continuity measurement by monocular sequence image | |
| CN106920276B (en) | A kind of three-dimensional rebuilding method and system | |
| CN108093244B (en) | Remote follow-up stereoscopic vision system | |
| CN102620713A (en) | Method for measuring distance and positioning by utilizing dual camera | |
| CN105389543A (en) | Mobile robot obstacle avoidance device based on all-dimensional binocular vision depth information fusion | |
| CN111854636B (en) | Multi-camera array three-dimensional detection system and method | |
| CN109493378B (en) | Verticality detection method based on combination of monocular vision and binocular vision | |
| CN107911687B (en) | Robot teleoperation auxiliary system based on binocular stereo vision | |
| CN101854485A (en) | Method and device for automatically adjusting shake of hand-held three-dimensional shooting equipment | |
| CN107483915B (en) | Three-dimensional image control method and device | |
| CN113438465A (en) | Display adjusting method, device, equipment and medium | |
| CN210986289U (en) | Four-eye fisheye camera and binocular fisheye camera | |
| CN113034615B (en) | Equipment calibration method and related device for multi-source data fusion | |
| JP2006197036A (en) | Device and method for stereoscopic image display | |
| TW202203645A (en) | Method, processing device, and display system for information display | |
| CN111935472A (en) | Real-time three-dimensional panoramic video monitoring system and video monitoring method thereof | |
| CN109214984B (en) | Image acquisition method and device, autonomous positioning navigation system and computing equipment | |
| CN108844486B (en) | Fixed-image-distance binocular bionic three-dimensional measuring instrument |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |