CN108171754A - A kind of robot navigation device and method based on binocular vision - Google Patents
A kind of robot navigation device and method based on binocular vision Download PDFInfo
- Publication number
- CN108171754A CN108171754A CN201611114557.3A CN201611114557A CN108171754A CN 108171754 A CN108171754 A CN 108171754A CN 201611114557 A CN201611114557 A CN 201611114557A CN 108171754 A CN108171754 A CN 108171754A
- Authority
- CN
- China
- Prior art keywords
- card
- control
- robot
- personal computer
- industrial personal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Manipulator (AREA)
Abstract
The present invention proposes a kind of robot navigation device based on binocular vision, including：Display, robot control system, industrial personal computer, control box, vision guided navigation device；Wherein described industrial personal computer includes mainboard, control card, GPU video cards, image pick-up card；The vision guided navigation device includes binocular camera, LED light source；The control box includes power module and control module；The robot control system is connected with industrial personal computer, and the mainboard in the industrial personal computer is connected respectively at the display, control card, GPU video cards, image pick-up card；The control box and control card, image pick-up card, binocular camera, LED light source are connected.The present invention program had not only combined the characteristics of 2D image scenes quickly identify, but also can combine the accuracy that 3D depth informations ensure segmentation.
Description
Technical field
The present invention relates to robot navigation's technical fields, and in particular to a kind of robot navigation device based on binocular vision
And method.
Background technology
Intelligent robot navigation technology refers to effectively obtain the letter of environment and itself pose by self-contained sensor
Breath, while the detection of barrier and target in environment is completed, and contexture by self moves to the road of target location from initial position
Diameter operates so as to fulfill to target object.Itself has the modules such as complete perception, analysis, decision and execution, can be with
Independently it is engaged in production activity in the environment as the mankind.
Existing Algorithms of Robots Navigation System is mostly using sonar avoidance, by active emission detection signal, then received field
The echo of each object reflection, scene information is obtained by measuring transmitting and return signal in scape.Sonar avoidance only has part
Space avoidance feature (can only carry out certain height and the detection of obstacles at visual angle, detection range and detecting distance have larger limitation
Property), without the global visual field feature of detection.
Invention content
Present invention aims at solve the existing Algorithms of Robots Navigation System based on sonar only to have local space avoidance ability
The problem of (certain height and the detection of obstacles at visual angle can only be carried out), the visual field global without detection.It provides a kind of based on double
The Algorithms of Robots Navigation System visually felt, the system use the binocular camera demarcated, and detect position and the depth of field of barrier, then
According to current robot position and posture, the navigation of robot is realized.
In order to solve the above technical problems, the present invention adopts the following technical scheme that：
A kind of robot navigation device based on binocular vision, specifically includes：Display, robot control system, industry control
Machine, control box, vision guided navigation device；Wherein described industrial personal computer includes mainboard, control card, GPU video cards, image pick-up card；It is described
Vision guided navigation device includes binocular camera, LED light source；The control box includes power module and control module；The robot
Control system is connected with industrial personal computer, and the mainboard in the industrial personal computer is adopted respectively at the display, control card, GPU video cards, image
Truck is connected；The control box and control card, image pick-up card, binocular camera, LED light source are connected.
The binocular camera provides advanceable peace for barrier and target object in auxiliary robot identification scene
Complete trails, the optical axis keeping parallelism of two video cameras are fixed on same horizontal plane.
The LED light source is used to provide illumination for vision system in dark environment.
The control box is used to provide electric energy for binocular camera and light source, and receive the signal from control card, is responsible for
Control the acquisition of camera signal and the working condition of transmission and control LED light source.
The industrial personal computer is used to be responsible for the processing of information and controls the transmission of signal.
The display is checked for the image collected and handling result to be shown to user.
The present invention also provides a kind of robot navigation method based on binocular vision, key step includes：
Then S1, Binocular Stereo Vision System stereo calibration obtain the image information in circumstances not known；
S2 using algorithm for stereo matching, reconstructs the 3D disparity maps of environment；
S3 carries out the detection of barrier region using disparity map；
S4 carries out the detection of target object using the image of 2D and the disparity map of 3D；
S5 according to target and the location information of barrier, selects the secure path of robot P Passable.
The present invention has following advantageous effect compared with prior art：
The present invention program realizes the detection of barrier region using 3D disparity maps, in conjunction with regarding for the image using 2D and 3D
Difference figure carries out the detection of target object, finally according to target and the location information of barrier, selects the peace of robot P Passable
Complete trails.The present invention program had not only combined the characteristics of 2D image scenes quickly identify, but also can combine 3D depth informations and ensure to divide
The accuracy cut.
Description of the drawings
Fig. 1 is the structure chart of the robot navigation device based on binocular vision of the embodiment of the present invention.
Fig. 2 is the matched schematic diagram of parallel optical axis binocular vision of the embodiment of the present invention.
Fig. 3 is that the barrier region of the embodiment of the present invention represents schematic diagram.
Specific embodiment
Referring to Fig. 1, a kind of robot navigation device based on binocular vision of the embodiment of the present invention specifically includes：Display
Device 10, robot control system 20, industrial personal computer 30, control box 40, vision guided navigation device 50；Wherein described industrial personal computer 30 includes master
Plate 301, control card 302, GPU video cards 303, image pick-up card 304；The vision guided navigation device 50 includes binocular camera 501,
LED light source 502；The control box 40 includes power module 401 and control module 402；The robot control system 20 and work
Control machine 30 is connected, and the mainboard 301 in the industrial personal computer 30 is respectively at the display 10, control card 302, GPU video cards 303, figure
As capture card 304 is connected；The control box 40 and control card 302, image pick-up card 304, binocular camera 501, LED light source 502
It is connected.
Wherein, the optical axis keeping parallelism of two video cameras of the binocular camera 501 is fixed on same horizontal plane,
12X12X12mm can be used in appearance and size, and image resolution ratio is 352X288 pixels, and acquisition towel chastity rate is 25fps, and parallax range is protected
It holds as 40mm.
After the vision guided navigation device 50 obtains scene image, it is transmitted to image pick-up card 304 by control box 40 and caches
Afterwards, it send to mainboard 301 and is handled, robot control system 20 is then sent to by RS485, so as to which robot be controlled to transport
It is dynamic.
The embodiment of the present invention also provides a kind of robot navigation method based on binocular vision, and key step includes：
Then S1, Binocular Stereo Vision System stereo calibration obtain the image information in circumstances not known.According to prior calibration
Stereo visual system inside and outside parameter, correct left and right two images coordinate so that left images only in horizontal direction on
Containing parallax value, and it is consistent in vertical direction.Detailed process is：
The present embodiment employs Zhang Zhengyou chessboard calibration methods, and in calibration, camera model uses pin-hole model, definition
It is as follows：
Sm=A [R t] M, (1)
I.e.
In formula, (XW, YW, ZW) it is certain point coordinates under world coordinate system；(u, v) is (XW, YW, ZW) spot projection is in the plane of delineation
Coordinate；S is coordinate of the object in camera coordinate system；A is camera intrinsic parameter matrix；[R t] be outer parameter matrix, R
For spin matrix, t is translation matrix；M is certain point homogeneous coordinates under world coordinate system；fx, fyRespectively camera is on x, y-axis
Focal length；Cx, CyRespectively camera focus and imaging plane central point deviant.
Intrinsic Matrix is the relationship described between camera coordinate system and image coordinate system, can be calculated using intrinsic parameter
Go out certain point of image coordinate system corresponding to the point of camera coordinate system, calculate as follows：
In formula, (XC, YC) it is certain point in RGB image in the coordinate of camera coordinate system, ZCTo correspond to the depth image
In depth value.
S2 using algorithm for stereo matching, reconstructs the 3D disparity maps of environment.This system uses parallel optical axis binocular vision
Feel, its principle is as shown in Figure 2：The two identical cameras in left and right are accurately located on same plane, and chief ray is strictly parallel, phase
Position is fixed, represents the principal point (c of two cameras in left and right respectivelylx,cly) and (crx,cry) there is phase in the two images of left and right
Same pixel coordinate.Baseline length T is the distance between two image centers.Left and right camera imaging plane coordinate system is respectively
OlXlYlAnd OrXrYr, binocular camera coordinate system is Oxyz.Any point P (x, y, z) is in two imaging plane coordinates in coordinate system Oxyz
Point P is corresponded in system respectivelyl(xl,yl) and point Pr(xr,yr).The y-coordinate value y of point P in two imageslrFor be it is the same, i.e.,
yl=yr=ylr.Formula (5) can be obtained according to triangle geometrical principle and pinhole imaging system principle：
Assuming that d=xl-xr, d is referred to as parallax (being the deviation of same spatial point position in two images), substitutes into
Formula (5) can calculate three-dimensional coordinates of the point P under camera coordinates system, such as formula (6)：
S3 carries out the detection of barrier region using the image of 2D and the disparity map of 3D.With reference to Fig. 3, Fig. 3 is barrier area
Domain representation schematic diagram, if the collection for having obtained the point of three-dimensional coordinate is combined into S.For any point P (x, y, z) ∈ S, (x, y, z) is point P
3 d space coordinate under left camera coordinate system, if there are P'(x', y', z') ∈ S, and 2 points of distance D (P, P') meet
Equation 7 below：
Wherein DmaxFor the distance threshold of setting, then point P and P' are assigned into identity set, final set-partition into it is multiple mutually
Disjoint subclass S1,S2,...Sk, i.e. S=S1∪S2∪S…∪Sk, k is set number.Wherein for arbitrary 1≤i ＜
J≤k has
Each set SiRepresent the barrier of the edge point set, then three dimensions of some special object in three dimensions
Preceding near, rear far, left left, right right, high height are expressed as：Near=min (z), far=max (z), left
=min (x), right=max (x), height=min (- y), (x, y, z) are respectively set SiMiddle coordinate.
S4 according to target and the location information of barrier, selects the secure path of robot P Passable.Robot is obtained
The scene image obtained is divided into the map of multiple grids compositions, and the path planning in grating map is needed to consider robot size
Size, spinning movement and stopping action.The size of robot can be with to determine whether can have collision with periphery object
Danger, and then to judge whether have enough spaces when performing spinning movement and stopping action during robot motion, equally keep away
Exempt from the danger of collision, it is contemplated that the path planning algorithm flow of these factors is as follows：
S41：Traversal can pass through grid in neighborhood inside the grating map of octree structure；
S42：Obtained grid positions are either with or without the free space for meeting robot body size in detection S41, if there is
S43 is then gone to, it's not true goes to S44；
S43：Whether there is enough robots to perform the sky needed for rotary motion at the grid positions obtained in detection S41
Between if not then abandon current grid and return to S41, otherwise go to S44；
S44：At the grid positions obtained in detection S41 whether there is enough robots to perform the sky needed for stopping action
Between, it abandons current grid if it's not true and returns to S41, otherwise go to S45；
S45：Robot is moved at the grid positions, and S46 is gone to if being target point at the grid, is otherwise turned
To S41；
S46：The comprehensive grid positions generation passed by before reaches the path of target grid from initial position.
Claims (2)
1. a kind of robot navigation device based on binocular vision, which is characterized in that including：Display, robot control system,
Industrial personal computer, control box, vision guided navigation device；Wherein described industrial personal computer includes mainboard, control card, GPU video cards, image pick-up card；
The vision guided navigation device includes binocular camera, LED light source；The control box includes power module and control module；The machine
Device people's control system is connected with industrial personal computer, and the mainboard in the industrial personal computer is respectively at the display, control card, GPU video cards, figure
As capture card is connected；The control box and control card, image pick-up card, binocular camera, LED light source are connected；
The binocular camera provides advanceable safe road for barrier and target object in auxiliary robot identification scene
Diameter, the optical axis keeping parallelism of two video cameras are fixed on same horizontal plane；
The LED light source is used to provide illumination for vision system in dark environment；
The control box is used to provide electric energy for binocular camera and light source, and receive the signal from control card, is responsible for control
The acquisition and transmission of camera signal and the working condition for controlling LED light source；
The industrial personal computer is used to be responsible for the processing of information and controls the transmission of signal；
The display is checked for the image collected and handling result to be shown to user.
2. a kind of robot navigation method based on binocular vision, which is characterized in that key step includes：
Then S1, Binocular Stereo Vision System stereo calibration obtain the image information in circumstances not known；
S2 using algorithm for stereo matching, reconstructs the 3D disparity maps of environment；
S3 carries out the detection of barrier region using disparity map；
S4 carries out the detection of target object using the image of 2D and the disparity map of 3D；
S5 according to target and the location information of barrier, selects the secure path of robot P Passable.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611114557.3A CN108171754A (en) | 2016-12-07 | 2016-12-07 | A kind of robot navigation device and method based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611114557.3A CN108171754A (en) | 2016-12-07 | 2016-12-07 | A kind of robot navigation device and method based on binocular vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108171754A true CN108171754A (en) | 2018-06-15 |
Family
ID=62526406
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611114557.3A Pending CN108171754A (en) | 2016-12-07 | 2016-12-07 | A kind of robot navigation device and method based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108171754A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109084778A (en) * | 2018-09-19 | 2018-12-25 | 大连维德智能视觉技术创新中心有限公司 | A kind of navigation system and air navigation aid based on binocular vision and pathfinding edge technology |
| CN110587602A (en) * | 2019-08-26 | 2019-12-20 | 青岛森科特智能仪器有限公司 | Fish tank cleaning robot motion control device and control method based on three-dimensional vision |
| CN112000103A (en) * | 2020-08-27 | 2020-11-27 | 西安达升科技股份有限公司 | AGV robot positioning, mapping and navigation method and system |
| CN113016331A (en) * | 2021-02-26 | 2021-06-25 | 江苏大学 | Wide-narrow row ratoon rice harvesting regulation and control system and method based on binocular vision |
| CN110587602B (en) * | 2019-08-26 | 2024-05-14 | 青岛森科特智能仪器有限公司 | Fish tank cleaning robot motion control device and control method based on three-dimensional vision |
-
2016
- 2016-12-07 CN CN201611114557.3A patent/CN108171754A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109084778A (en) * | 2018-09-19 | 2018-12-25 | 大连维德智能视觉技术创新中心有限公司 | A kind of navigation system and air navigation aid based on binocular vision and pathfinding edge technology |
| CN109084778B (en) * | 2018-09-19 | 2022-11-25 | 大连维德智能视觉技术创新中心有限公司 | Navigation system and navigation method based on binocular vision and road edge finding technology |
| CN110587602A (en) * | 2019-08-26 | 2019-12-20 | 青岛森科特智能仪器有限公司 | Fish tank cleaning robot motion control device and control method based on three-dimensional vision |
| CN110587602B (en) * | 2019-08-26 | 2024-05-14 | 青岛森科特智能仪器有限公司 | Fish tank cleaning robot motion control device and control method based on three-dimensional vision |
| CN112000103A (en) * | 2020-08-27 | 2020-11-27 | 西安达升科技股份有限公司 | AGV robot positioning, mapping and navigation method and system |
| CN113016331A (en) * | 2021-02-26 | 2021-06-25 | 江苏大学 | Wide-narrow row ratoon rice harvesting regulation and control system and method based on binocular vision |
| CN113016331B (en) * | 2021-02-26 | 2022-04-26 | 江苏大学 | Wide-narrow row ratoon rice harvesting regulation and control system and method based on binocular vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106681353B (en) | The unmanned plane barrier-avoiding method and system merged based on binocular vision with light stream | |
| CN110458897B (en) | Multi-camera automatic calibration method and system and monitoring method and system | |
| CN108205315A (en) | A kind of robot automatic navigation method based on binocular vision | |
| CN105550670B (en) | A kind of target object dynamically track and measurement and positioning method | |
| CN108594245A (en) | A kind of object movement monitoring system and method | |
| CN108444390A (en) | A kind of pilotless automobile obstacle recognition method and device | |
| WO2015024407A1 (en) | Power robot based binocular vision navigation system and method based on | |
| CN108234984A (en) | Binocular depth camera system and depth image generation method | |
| CN108205314A (en) | Based on the matched robot navigation device of stereoscopic vision and system | |
| CN106033614B (en) | A kind of mobile camera motion object detection method under strong parallax | |
| CN102650886A (en) | Vision system based on active panoramic vision sensor for robot | |
| CN105389543A (en) | Mobile robot obstacle avoidance device based on all-dimensional binocular vision depth information fusion | |
| CN112204614B (en) | Motion segmentation in video from non-stationary cameras | |
| CN108279677B (en) | Rail robot detection method based on binocular vision sensor | |
| CN108171754A (en) | A kind of robot navigation device and method based on binocular vision | |
| CN102508548A (en) | Operation method and system for electronic information equipment | |
| CN113379848A (en) | Target positioning method based on binocular PTZ camera | |
| Häselich et al. | Calibration of multiple cameras to a 3D laser range finder | |
| CN208350997U (en) | A kind of object movement monitoring system | |
| JP2015179066A (en) | Parallax value derivation device, apparatus control system, moving body, robot, parallax value derivation method and program | |
| CN111860270B (en) | Obstacle detection method and device based on fisheye camera | |
| CN103260008A (en) | Projection converting method from image position to actual position | |
| Gehrig et al. | 6D vision goes fisheye for intersection assistance | |
| CN108090930A (en) | Barrier vision detection system and method based on binocular solid camera | |
| CN115526897A (en) | Flame positioning method and system for fire-fighting robot of extra-high voltage converter station |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180615 |