CN116721149A - Weed positioning method based on binocular vision - Google Patents

Weed positioning method based on binocular vision Download PDF

Info

Publication number
CN116721149A
CN116721149A CN202310724496.6A CN202310724496A CN116721149A CN 116721149 A CN116721149 A CN 116721149A CN 202310724496 A CN202310724496 A CN 202310724496A CN 116721149 A CN116721149 A CN 116721149A
Authority
CN
China
Prior art keywords
camera
binocular
weed
calibration
binocular vision
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310724496.6A
Other languages
Chinese (zh)
Inventor
顾煜洋
吴赛林
王金凤
张月月
冯浩宸
李跃华
胡彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nantong University
Original Assignee
Nantong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nantong University filed Critical Nantong University
Priority to CN202310724496.6A priority Critical patent/CN116721149A/en
Publication of CN116721149A publication Critical patent/CN116721149A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85Stereo camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30204Marker
    • G06T2207/30208Marker matrix

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention provides a weed positioning method based on binocular vision, and belongs to the technical field of computer vision. Solves the technical problem of accurately and efficiently positioning weeds in the agricultural field. The technical proposal is as follows: the method comprises the following steps: s1, calibrating a binocular camera to obtain detailed parameters of the camera; s2, three-dimensional correction of a binocular camera; s3, three-dimensional matching; and S4, obtaining the three-dimensional coordinates of the central point of the weed. The beneficial effects of the invention are as follows: the invention utilizes binocular vision technology to construct weed positioning algorithm, realizes precise positioning of weeds, and provides a precise weeding scheme for a laser weeding robot, wherein the precise weeding scheme can be practically applied.

Description

Weed positioning method based on binocular vision
Technical Field
The invention relates to the technical field of computer vision, in particular to a weed positioning method based on binocular vision.
Background
With the development of agriculture, the growth and propagation of weeds have a serious influence on the yield and quality of crops. The conventional weed management methods comprise the use of artificial weeding and chemical weeding, however, the methods all require higher labor cost and pollute the crop production environment. Therefore, how to develop an efficient and accurate weed positioning and management method is a current research hotspot.
In order to solve the problems, paper (research on monocular vision method of three-dimensional positioning of target) (computer software and computer application, 2022) proposes a target positioning method based on monocular vision, which makes up for the cost of manpower and material resources in the traditional management method, and scene capturing is performed by 3D detection of a single independent object and a single visual angle to position the weed position, thereby providing an emerging technology for the intelligent weeding field. However, along with the continuous development of artificial intelligence technology, the monocular vision has the problems of inaccurate positioning, low precision and the like in the aspects of depth estimation, environment perception, visual angle and the like.
Therefore, in order to improve and complement the problems of monocular vision in aspects of depth estimation, visual angle, environmental perception and the like, a weed positioning method based on binocular vision needs to be developed, a weed positioning algorithm is constructed by utilizing binocular vision, so that the accuracy and the processing speed of weed positioning are improved, the accurate positioning of weeds is realized, and an accurate weeding scheme capable of being practically applied is provided for a laser weeding robot.
Disclosure of Invention
The invention aims to provide a binocular vision-based weed positioning method, which solves the technical problem that weeds are difficult to accurately and efficiently position in the agricultural field, provides technical support for accurate weeding of crops, and improves the production efficiency and crop quality of farmlands.
In order to achieve the aim of the invention, the invention adopts the technical scheme that: a weed positioning method based on binocular vision comprises the following steps:
s1, calibrating a binocular camera to obtain detailed parameters of the camera;
(1) Manufacturing calibration plate
According to the invention, a 9X 7-specification chessboard is selected as a calibration plate for camera calibration, and the size of each chessboard is 28mm multiplied by 28mm.
(2) Collecting calibration images
The binocular camera is fixed, the positions and angles of the calibration plate and the binocular camera are continuously adjusted, and images obtained by the left camera and the right camera are acquired and stored.
(3) Single calibration of left and right cameras
And (5) importing images acquired by the left camera and the right camera into a camera calibration program in MATLAB to calibrate camera parameters.
S2, three-dimensional correction of a binocular camera;
because of a series of uncontrollable factors such as camera manufacturing errors and errors existing when a binocular system is artificially built, a certain included angle exists between imaging planes of a left camera and a right camera, so that line pixels of a left image and a right image shot by the binocular camera are not aligned, and errors are brought to subsequent weed positioning.
The problem that images shot by a camera are not parallel and coplanar can be well solved through stereo correction. The invention performs three-dimensional correction based on a Bouguet algorithm, and the specific process is as follows:
decomposing the rotation matrix R obtained by camera calibration in the step S1 into R l And r r The optical center axes of the binocular cameras can be made parallel to each other by the two matrices, i.e., the left and right imaging planes are made coplanar.
On the basis, in order to make the base line of the camera and the imaging plane of the camera in parallel, a transformation matrix R is also required to be constructed by using a translation matrix T obtained after camera calibration rect The pole of the two cameras can be transformed to infinity by using the transformation matrix, the pole is the intersection point of the base line of the cameras and the imaging plane, and the pole is equivalent to realizing line alignment when in the state of infinity.
Since the imaging plane is ultimately parallel to the baseline, the imaging plane is then
Wherein T= [ -T, T,0],e 1 And e 2 Orthogonal to e through the direction of the main optical axis (0, 1) 1 By cross multiplication, can be calculated
e 3 As long as it is with e 1 、e 2 Orthogonalization is enough, so there is
e 3 =e 2 ×e 1
Rotating transformation matrixAfter the solution is completed, the imaging of the left and right cameras can be realized through the matrix
The planes are transformed onto the same plane as follows:
in the above formula, R is a rotation matrix, T is a translation matrix, R rect For transforming matrix e 1 、e 2 、e 3 Is a vector.
S3, three-dimensional matching;
the stereo matching refers to that in a binocular stereo vision system, the relationship between corresponding points is found out by comparing images shot by a left camera and a right camera, namely, the center point of the weed in a left image corresponds to which pixel point in a right image, so that the position of the center point of the weed in a three-dimensional space is determined. The three-dimensional information of the same object is obtained under different visual angles, and applications such as depth perception and space reconstruction are realized.
S4, obtaining three-dimensional coordinates of a weed center point;
the basic principle of the binocular vision principle is that the left and right cameras are used for shooting the same object at the same time, and the position of the object in the three-dimensional space is calculated by comparing the parallaxes (namely the offset of pixels in the left and right images) between corresponding points in the left and right images. According to the binocular vision principle, the parallax value of the pixel points obtained in the step S3 is utilized, and the three-dimensional coordinates of the weed center point are calculated by combining the internal and external parameters of the camera.
As a further optimization scheme of the binocular vision-based weed positioning method, in the step S3, the method is based on an SGBM semi-global stereo matching algorithm, firstly, a disparity value is selected for each pixel point to form an initial disparity map, then a global energy function is set, the function is related to the disparity map, and finally, the minimization problem of the energy function is solved, so that the optimal disparity value of each pixel point is obtained.
Compared with the prior art, the invention has the beneficial effects that:
(1) Optimal disparity map based on SGBM semi-global stereo matching algorithm
In binocular vision, depth information can be obtained by matching images of left and right viewing angles, so that three-dimensional reconstruction of a scene is realized. The parallax image is an image representing depth information, and reflects the offset of each pixel point in a left view and a right view; the SGBM algorithm is a stereo matching algorithm based on global optimization, and has higher matching precision and robustness compared with the traditional algorithm based on a local window; the invention obtains the minimum value of the global energy function based on SGBM semi-global stereo matching algorithm, thereby obtaining the best right matching point of the left imaging point and obtaining the best parallax image.
(2) Binocular camera stereo correction based on Bouguet algorithm
When binocular stereo matching is performed, stereo correction is required because the images of the two viewing angles do not completely coincide, so that the two images are parallel and coplanar. When matching is performed, only one-dimensional searching is needed on the same row of pixels, so that the computational complexity is reduced. Meanwhile, the stereo correction can also improve the matching precision and stability. The invention adopts the Bouguet algorithm to carry out the stereo correction of the binocular camera. The Bouguet algorithm is a binocular correction algorithm based on a Zhang Zhengyou calibration method, and the images of the left camera and the right camera are calibrated to obtain the internal parameters and the external parameters of the cameras, so that a conversion matrix between the left camera and the right camera is calculated, and the correction of the images is realized. Compared with other algorithms, the Bouguet algorithm has the advantages of simplicity in calculation, high accuracy and the like. The invention performs three-dimensional correction on the binocular camera based on the Bouguet algorithm, ensures that the images are parallel and coplanar, only one-dimensional search is needed on the same row of pixels when the images are subjected to three-dimensional matching, and reduces the computational complexity.
(3) The invention utilizes binocular vision technology to construct weed positioning algorithm, realizes precise positioning of weeds, and provides a precise weeding scheme for a laser weeding robot, wherein the precise weeding scheme can be practically applied.
(1) Positioning accuracy is improved: the binocular vision technology can observe the target by using two visual angles, more accurate three-dimensional positioning can be realized by calculating parallax, and positioning accuracy can be improved relative to the monocular vision technology.
(2) Enhancing the anti-interference capability: in a weed positioning scene, due to the factors of complex environment, large illumination change and the like, the image quality is easy to be reduced, and the positioning effect of a monocular vision system is influenced. And the binocular vision system can observe the target by utilizing two visual angles, so that the anti-interference capability of the system is enhanced.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.
Fig. 1 is a schematic diagram of a calibration plate of a binocular vision-based weed positioning method.
Fig. 2 is a schematic diagram of a binocular camera in the binocular vision-based weed positioning method provided by the invention.
Fig. 3 is a weed locating result in the weed locating method based on binocular vision provided by the invention.
Fig. 4 is a flow chart of weed positioning in the binocular vision-based weed positioning method provided by the invention.
FIG. 5 is a schematic representation of weed placement in an embodiment of the invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. Of course, the specific embodiments described herein are for purposes of illustration only and are not intended to limit the invention.
Examples
Referring to fig. 1 to 5, the present embodiment provides a binocular vision-based weed positioning method, comprising the following specific steps:
s1, calibrating a binocular camera to obtain detailed parameters of the camera;
(1) Manufacturing calibration plate
The calibration plate for camera calibration refers to a chessboard image with specific size, and the invention selects a chessboard with the specification of 9 multiplied by 7, and the size of each chessboard is 28mm multiplied by 28mm.
(2) Collecting calibration images
Fixing the binocular camera, continuously adjusting the positions and angles of the calibration plate and the binocular camera, collecting images obtained by the corresponding left and right cameras, and storing the images;
(3) Single calibration of left and right cameras
And (5) importing images acquired by the left camera and the right camera into a camera calibration program in MATLAB to calibrate camera parameters.
S2, three-dimensional correction of a binocular camera;
because of a series of uncontrollable factors such as camera manufacturing errors and errors existing when a binocular system is artificially built, a certain included angle exists between imaging planes of a left camera and a right camera, so that line pixels of a left image and a right image shot by the binocular camera are not aligned, and errors are brought to subsequent weed positioning.
The problem that images shot by a camera are not parallel and coplanar can be well solved through stereo correction. The invention performs three-dimensional correction based on a Bouguet algorithm, and the specific process is as follows:
decomposing the rotation matrix R obtained by camera calibration in the step S1 into R l And r r The optical center axes of the binocular cameras can be made parallel to each other by the two matrices, i.e., the left and right imaging planes are made coplanar.
On the basis, in order to make the base line of the camera and the imaging plane of the camera in parallel, a transformation matrix R is also required to be constructed by using a translation matrix T obtained after camera calibration rect By using the transformation matrix, the poles of the two cameras can be transformed to infinity. This pole is the intersection of the camera baseline and the imaging plane, and when the pole is at infinity, it is equivalent to achieving line alignment.
Since the imaging plane is ultimately parallel to the baseline, the imaging plane is then
Wherein T= [ -T, T,0],e 1 And e 2 Orthogonal to e through the direction of the main optical axis (0, 1) 1 By cross multiplication, can be calculated
e 3 As long as it is with e 1 、e 2 Orthogonalization is enough, so there is
e 3 =e 2 ×e 1
Rotating transformation matrixAfter the solution is completed, the imaging planes of the left camera and the right camera can be transformed to the same plane through the matrix, and the following formula is adopted:
in the above formula, R is a rotation matrix, T is a translation matrix, R rect For transforming matrix e 1 、e 2 、e 3 Is a vector.
S3, three-dimensional matching, namely solving the optimal parallax value of each pixel point by adopting an SGBM semi-global three-dimensional matching algorithm;
and S4, calculating the three-dimensional coordinates of the weed center point by utilizing the parallax value of the pixel point obtained in the step S3 and combining the internal and external parameters of the camera.
Taking fig. 5 as an example, step S1 is performed to obtain detailed parameters of the binocular camera, including
Translation matrix [ -123.0086 0.5029-2.3001 ]
Then, based on a Bouguet algorithm, performing binocular camera stereo correction; then, calculating the minimum value of the global energy function by utilizing an SGBM semi-global stereo matching algorithm, so as to obtain the optimal right matching point of each left imaging point; and finally, calculating the three-dimensional coordinates of the weed center point through the parameters of the camera and the parallax value of the weed center point.
The method combines the stereo correction of the binocular camera and the SGBM stereo matching algorithm with the calculation of parameters and parallax values of the camera, thereby being capable of providing relatively accurate and reliable three-dimensional coordinates of the weed center point. This is of great value in situations where weed analysis, targeting or other related applications are required.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims (7)

1. The weed positioning method based on binocular vision is characterized by comprising the following steps of:
s1, calibrating a binocular camera to obtain detailed parameters of the camera;
s2, three-dimensional correction of a binocular camera;
s3, three-dimensional matching;
and S4, obtaining the three-dimensional coordinates of the central point of the weed.
2. The binocular vision-based weed positioning method according to claim 1, wherein in the step S1, the binocular camera is calibrated, and the detailed parameters of the camera are obtained as follows:
(1) Manufacturing calibration plate
The calibration plate used in camera calibration refers to a plate or a panel with a geometric structure and characteristic points, and comprises a checkerboard calibration plate and a circle calibration plate;
(2) Collecting calibration images
Fixing the binocular camera, continuously adjusting the positions and angles of the calibration plate and the binocular camera, collecting images obtained by the corresponding left and right cameras, and storing the images;
(3) Single calibration of left and right cameras
And (5) importing images acquired by the left camera and the right camera into a camera calibration program in MATLAB to calibrate camera parameters.
3. The binocular vision-based weed positioning method according to claim 1, wherein in the step S2, a certain included angle exists between imaging planes of the left and right cameras due to a series of uncontrollable factors of camera manufacturing errors and errors existing when a binocular system is artificially built, so that line pixels of left and right images shot by the binocular cameras are not aligned, and errors are brought to subsequent weed positioning; the problem that images shot by a camera are not parallel and coplanar is solved through stereo correction, and the stereo correction is carried out based on a Bouguet algorithm.
4. The binocular vision-based weed positioning method of claim 3, wherein in the step S2, the specific process of performing the stereo correction based on the Bouguet algorithm is as follows:
decomposing the rotation matrix R obtained by camera calibration in the step S1 into R l And r r The optical central axes of the binocular cameras are parallel to each other through the two matrixes, so that the left imaging plane and the right imaging plane are coplanar;
on the basis, in order to enable the base line of the camera and the imaging plane of the camera to be in a parallel state, a translation matrix T obtained after camera calibration is utilized to construct a transformation matrix R rect The pole of the two cameras is transformed to infinity by using the transformation matrix, the pole is the intersection point of a base line of the cameras and an imaging plane, and when the pole is in the infinity state, the line alignment is equivalent to being realized;
since the imaging plane is ultimately parallel to the baseline, the imaging plane is then
Wherein T= [ -T, T,0],e 1 And e 2 Orthogonal to e through the direction of the main optical axis (0, 1) 1 Performing cross multiplication and calculating to obtain
e 3 As long as it is with e 1 、e 2 Orthogonalization is enough, so there is
e 3 =e 2 ×e 1
Rotating transformation matrixAfter the solution is completed, the imaging planes of the left camera and the right camera can be transformed to the same plane through the matrix, and the following formula is adopted:
in the above-mentioned formula(s),r is a rotation matrix, T is a translation matrix, R rect For transforming matrix e 1 、e 2 、e 3 Is a vector.
5. The binocular vision-based weed positioning method according to claim 1, wherein in the step S3, the stereo matching is implemented by comparing images photographed by the left and right cameras in a binocular stereo vision system, finding out the relationship between corresponding points, and obtaining three-dimensional information of the same object under different viewing angles, thereby realizing depth perception and spatial reconstruction.
6. The binocular vision-based weed positioning method according to claim 1, wherein in step S3, based on SGBM semi-global stereo matching algorithm, firstly, a disparity value is selected for each pixel point to form an initial disparity map, then a global energy function is set, the function is related to the disparity map, and finally, the minimization problem of the energy function is solved, so that the optimal disparity value of each pixel point is obtained.
7. The binocular vision-based weed positioning method according to claim 1, wherein the specific process of obtaining the three-dimensional coordinates of the weed center point in the step S4 is as follows:
the basic principle of the binocular vision principle is that the left and right cameras are used for shooting the same object at the same time, and the position of the object in a three-dimensional space is calculated by comparing the parallax between corresponding points in the left and right images; and (3) according to the binocular vision principle, calculating the three-dimensional coordinates of the weed center point by utilizing the parallax value of the pixel point obtained in the step (S3) and combining the internal and external parameters of the camera.
CN202310724496.6A 2023-06-16 2023-06-16 Weed positioning method based on binocular vision Pending CN116721149A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310724496.6A CN116721149A (en) 2023-06-16 2023-06-16 Weed positioning method based on binocular vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310724496.6A CN116721149A (en) 2023-06-16 2023-06-16 Weed positioning method based on binocular vision

Publications (1)

Publication Number Publication Date
CN116721149A true CN116721149A (en) 2023-09-08

Family

ID=87871284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310724496.6A Pending CN116721149A (en) 2023-06-16 2023-06-16 Weed positioning method based on binocular vision

Country Status (1)

Country Link
CN (1) CN116721149A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117882697A (en) * 2024-03-14 2024-04-16 东海实验室 Accurate and rapid intelligent laser weeding device and method

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117882697A (en) * 2024-03-14 2024-04-16 东海实验室 Accurate and rapid intelligent laser weeding device and method

Similar Documents

Publication Publication Date Title
CN107977997B (en) Camera self-calibration method combined with laser radar three-dimensional point cloud data
CN110728715B (en) Intelligent inspection robot camera angle self-adaptive adjustment method
CN107560592B (en) Precise distance measurement method for photoelectric tracker linkage target
CN111862180B (en) Camera set pose acquisition method and device, storage medium and electronic equipment
CN108489398B (en) Method for measuring three-dimensional coordinates by laser and monocular vision under wide-angle scene
CN104376552A (en) Virtual-real registering algorithm of 3D model and two-dimensional image
EP2847741A1 (en) Camera scene fitting of real world scenes for camera pose determination
CN111768449B (en) Object grabbing method combining binocular vision with deep learning
CN110874854B (en) Camera binocular photogrammetry method based on small baseline condition
CN110889873A (en) Target positioning method and device, electronic equipment and storage medium
WO2021195939A1 (en) Calibrating method for external parameters of binocular photographing device, movable platform and system
CN105654547A (en) Three-dimensional reconstruction method
CN104794713A (en) Greenhouse crop digital-imaging method based on ARM and binocular vision
CN116721149A (en) Weed positioning method based on binocular vision
CN112288826A (en) Calibration method and device of binocular camera and terminal
CN115880344A (en) Binocular stereo matching data set parallax truth value acquisition method
CN112132900A (en) Visual repositioning method and system
CN111429571B (en) Rapid stereo matching method based on spatio-temporal image information joint correlation
CN116883480A (en) Corn plant height detection method based on binocular image and ground-based radar fusion point cloud
CN114998532B (en) Three-dimensional image visual transmission optimization method based on digital image reconstruction
CN113240749B (en) Remote binocular calibration and ranging method for recovery of unmanned aerial vehicle facing offshore ship platform
CN112700504B (en) Parallax measurement method of multi-view telecentric camera
CN113706391B (en) Real-time splicing method, system, equipment and storage medium for aerial images of unmanned aerial vehicle
CN112396687B (en) Binocular stereoscopic vision three-dimensional reconstruction system and method based on infrared micro-polarizer array
CN111829489B (en) Visual positioning method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination