CN112614045A - Method and system for eliminating visual perception perspective effect of agricultural machinery front operation environment - Google Patents
Method and system for eliminating visual perception perspective effect of agricultural machinery front operation environment Download PDFInfo
- Publication number
- CN112614045A CN112614045A CN202011490057.6A CN202011490057A CN112614045A CN 112614045 A CN112614045 A CN 112614045A CN 202011490057 A CN202011490057 A CN 202011490057A CN 112614045 A CN112614045 A CN 112614045A
- Authority
- CN
- China
- Prior art keywords
- perspective
- camera
- inverse
- roi
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000694 effects Effects 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000016776 visual perception Effects 0.000 title claims abstract description 25
- 230000009466 transformation Effects 0.000 claims abstract description 106
- 239000011159 matrix material Substances 0.000 claims abstract description 78
- 238000006243 chemical reaction Methods 0.000 claims abstract description 42
- 238000009434 installation Methods 0.000 claims abstract description 30
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000003384 imaging method Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method and a system for eliminating visual perception perspective effect of an agricultural machinery front operation environment, wherein the method comprises the following steps: calibrating a camera to obtain camera internal parameters and camera installation parameters; constructing a single-point perspective coordinate conversion formula according to camera internal parameters and camera installation parameters; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a corresponding coordinate in the reverse perspective; in the agricultural machinery operation process, an ROI vertex is selected from a perspective picture acquired by a camera, and an inverse perspective transformation matrix is determined according to a single-point perspective coordinate conversion formula and the ROI vertex; and according to the inverse perspective transformation matrix, performing inverse perspective transformation on each single point in the region range determined by the ROI vertex to obtain an inverse perspective view with the perspective effect eliminated. Therefore, the end-to-end direct conversion from the space ROI to the image ROI can be realized, and the image operation amount is obviously reduced.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method and a system for eliminating visual perception perspective effect of an agricultural machinery front operation environment.
Background
The front environment perception of the operating agricultural machine is the basis for realizing unmanned operation of the agricultural machine, and the vision technology is a common technology for the environment perception of the agricultural machine. Different from the common industrial application situation that the imaging axis of the camera is perpendicular to the sensed object, in order to enable the forward looking visual distance to be long so as to collect the condition of a far-end field, the camera is installed at a certain height and aims at the front field at a certain overlooking angle in order to sense the operation environment in front of the agricultural machine, and a perspective image of intersection of near-far and infinite row lines is obtained. Specifically, the parallel-like crop row lines converge in the image to the far side, and the rectangular field appears as a quadrangle of a small size and a large size in the perspective view. The more distant portions of the perspective image are not included in the agricultural work area. Finally, the visual perception is used for submitting processing results which are consistent with the field, such as crop row lines, and guiding the operation of agricultural machinery. The spacing between the rows and the lines of the crops, the sizes of the crops, the row and line directions and the like in the same field have certain consistency. In order to guide subsequent operation, simplify the algorithm and improve the recognition rate and the recognition precision, the visual perception of the field in front of the agricultural machinery needs to be subjected to the previous processing for eliminating the perspective effect. The method comprises the following steps of eliminating the perspective effect, wherein one operation is to cut a region of interest (ROI) with large and small distances caused by perspective; another operation is an inverse perspective transformation of points within the perspective ROI.
Through the search of the prior art, the application numbers are as follows: CN201810776433.4, entitled "crop row identification method and apparatus", discloses converting a crop row perspective view, which is obtained by shooting with a camera at a tilt angle with a road surface, into a crop row aerial view through an inverse perspective transformation matrix; acquiring crop row edge points of a vertical projection image of a preset area in a crop row perspective view, and acquiring skeleton line intersection point sets of all crop rows of a crop row aerial view; dividing the skeleton line intersection point sets of all the crop rows according to the coordinates of the crop row edge points to obtain the corresponding skeleton line intersection point set of each crop row; and respectively carrying out straight line fitting on the intersection point set of the skeleton lines of each corresponding crop row to obtain the corresponding crop row. However, this method has a large amount of data calculation and low processing efficiency.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method and a system for eliminating the visual perception perspective effect of an agricultural machinery front operation environment.
In a first aspect, the present invention provides a method for eliminating a visual perception perspective effect of an agricultural machinery front operation environment, including:
step 1: calibrating a camera to obtain camera internal parameters and camera installation parameters;
step 2: constructing a single-point perspective coordinate conversion formula according to the camera internal parameters and the installation parameters of the camera; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a coordinate corresponding to the reverse perspective;
and step 3: in the agricultural machinery operation process, an ROI vertex is selected from a perspective picture collected by a camera, and an inverse perspective transformation matrix is determined according to the single-point perspective coordinate conversion formula and the ROI vertex;
and 4, step 4: and according to the inverse perspective transformation matrix, performing inverse perspective transformation on each single point in the region range determined by the ROI vertex to obtain an inverse perspective view with the perspective effect eliminated.
Optionally, the camera internal reference in step 1 includes: the camera transverse field angle, the camera longitudinal field angle, the camera transverse resolution and the camera longitudinal resolution; the installation parameters of the camera include: camera installation pitch angle, camera installation height.
Optionally, the equation for converting the single-point perspective coordinate in step 2 is as follows:
wherein X represents an X-axis coordinate in a world coordinate system, Y represents a Y-axis coordinate in the world coordinate system, u represents an X-axis coordinate of a certain point in a perspective picture, v represents a Y-axis coordinate of a certain point in the perspective picture, h represents a camera mounting height, M represents a camera longitudinal resolution, θ represents a camera mounting pitch angle, 2 α represents a camera longitudinal field angle, 2 β represents a camera lateral field angle, N represents a camera lateral resolution, u 'represents an X-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, v' represents a Y-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, and C represents a pixel equivalent of reconstructed imaging.
Optionally, the step 3 includes:
in the agricultural machinery operation process, four ROI vertexes (u) of a trapezoidal area are selected in a perspective view sheet collected by a camera according to the forward distance and the transverse distance required by operation perceptioni,vi) Wherein u isiDenotes the abscissa, v, of the vertex of the ith ROIiAn ordinate representing the vertex of the ith ROI, i ═ 1,2,3, 4;
order to
Wherein u isi' represents the abscissa, v, of the i-th ROI vertex calculated based on a single-point perspective coordinate conversion formulai' represents the vertical coordinate of the ith ROI vertex calculated based on a single-point perspective coordinate conversion formula; a denotes an inverse perspective transformation matrix, a11Elements representing the 1 st row and 1 st column of the inverse perspective transformation matrix, a12Elements representing the 1 st row and 2 nd column of the inverse perspective transformation matrix, a13Elements representing the 1 st row and 3 rd column of the inverse perspective transformation matrix, a21Representing the 2 nd row and 1 st column of the inverse perspective transformation matrixElement of (a)22Elements representing the 2 nd row and 2 nd column of the inverse perspective transformation matrix, a23Elements representing the 2 nd row and 3 rd column of the inverse perspective transformation matrix, a31Elements representing the 3 rd row and 1 st column of the inverse perspective transformation matrix, a32Representing the elements of row 3 and column 2 of the inverse perspective transformation matrix.
Optionally, the step 4 includes:
selecting P single points in the region range determined by the ROI vertex, and performing inverse perspective transformation on the P single points through the inverse perspective transformation matrix to obtain corresponding P target points; wherein, P is a natural number more than 1;
performing interpolation operation based on the target point to obtain Q pixel points, wherein Q is a natural number larger than P;
and generating a reverse perspective view after eliminating the perspective effect based on the Q pixel points.
In a second aspect, the present invention provides a system for eliminating the visual perception perspective effect of the front working environment of an agricultural machine, including:
the camera calibration module is used for calibrating the camera to obtain camera internal parameters and camera installation parameters;
the conversion module is used for constructing a single-point perspective coordinate conversion formula according to the camera internal parameters and the installation parameters of the camera; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a coordinate corresponding to the reverse perspective;
the determining module is used for selecting an ROI vertex in the perspective picture acquired by the camera and determining an inverse perspective transformation matrix according to the single-point perspective coordinate conversion formula and the ROI vertex;
and the inverse transformation module is used for carrying out inverse perspective transformation on each single point in the region range determined by the ROI vertex according to the inverse perspective transformation matrix to obtain an inverse perspective view with the perspective effect eliminated.
Optionally, the camera internal reference comprises: the camera transverse field angle, the camera longitudinal field angle, the camera transverse resolution and the camera longitudinal resolution; the installation parameters of the camera include: camera installation pitch angle, camera installation height.
Optionally, the single-point perspective coordinate conversion formula is as follows:
wherein X represents an X-axis coordinate in a world coordinate system, Y represents a Y-axis coordinate in the world coordinate system, u represents an X-axis coordinate of a certain point in a perspective picture, v represents a Y-axis coordinate of a certain point in the perspective picture, h represents a camera mounting height, M represents a camera longitudinal resolution, θ represents a camera mounting pitch angle, 2 α represents a camera longitudinal field angle, 2 β represents a camera lateral field angle, N represents a camera lateral resolution, u 'represents an X-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, v' represents a Y-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, and C represents a pixel equivalent of reconstructed imaging.
Optionally, the determining module is specifically configured to:
in the agricultural machinery operation process, four ROI vertexes (u) of a trapezoidal area are selected in a perspective view sheet collected by a camera according to the forward distance and the transverse distance required by operation perceptioni,vi) Wherein u isiDenotes the abscissa, v, of the vertex of the ith ROIiAn ordinate representing the vertex of the ith ROI, i ═ 1,2,3, 4;
order to
Wherein u isi' expression is calculated based on a single-point perspective coordinate conversion formulaThe abscissa of the ith ROI vertex of (1), vi' represents the vertical coordinate of the ith ROI vertex calculated based on a single-point perspective coordinate conversion formula; a denotes an inverse perspective transformation matrix, a11Elements representing the 1 st row and 1 st column of the inverse perspective transformation matrix, a12Elements representing the 1 st row and 2 nd column of the inverse perspective transformation matrix, a13Elements representing the 1 st row and 3 rd column of the inverse perspective transformation matrix, a21Elements representing the 2 nd row and 1 st column of the inverse perspective transformation matrix, a22Elements representing the 2 nd row and 2 nd column of the inverse perspective transformation matrix, a23Elements representing the 2 nd row and 3 rd column of the inverse perspective transformation matrix, a31Elements representing the 3 rd row and 1 st column of the inverse perspective transformation matrix, a32Representing the elements of row 3 and column 2 of the inverse perspective transformation matrix.
Optionally, the inverse transform module is specifically configured to:
selecting P single points in the region range determined by the ROI vertex, and performing inverse perspective transformation on the P single points through the inverse perspective transformation matrix to obtain corresponding P target points; wherein, P is a natural number more than 1;
performing interpolation operation based on the target point to obtain Q pixel points, wherein Q is a natural number larger than P;
and generating a reverse perspective view after eliminating the perspective effect based on the Q pixel points.
Compared with the prior art, the invention has the following beneficial effects:
according to the method and the system for eliminating the visual perception perspective effect of the agricultural machinery front operation environment, the reverse perspective transformation matrix is determined through the ROI vertex selected from the perspective picture, the accuracy of the reverse perspective transformation matrix is effectively guaranteed, and the end-to-end direct conversion from the space ROI to the image ROI is realized by performing the reverse perspective transformation of each single point in the region formed by the ROI vertex input into the original perspective picture, so that the operation amount of image conversion is remarkably reduced, and the image processing efficiency is improved.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic diagram illustrating a method for eliminating a visual perception perspective effect of an agricultural machinery front working environment according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a selection result of a ROI vertex in a perspective picture according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an inverse perspective view after eliminating perspective effect in the embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
Fig. 1 is a schematic diagram illustrating a method for eliminating a visual perception perspective effect of an agricultural machinery front working environment according to an embodiment of the present invention, as shown in fig. 1, the present embodiment may include the following steps:
parameter calibration: and calibrating the camera to obtain the camera internal parameters and the installation parameters of the camera.
In this embodiment, the camera reference includes: the camera transverse field angle, the camera longitudinal field angle, the camera transverse resolution and the camera longitudinal resolution; the mounting parameters of the camera include: camera installation pitch angle, camera installation height.
The single-point perspective coordinate conversion formula parameter is obtained: constructing a single-point perspective coordinate conversion formula according to camera internal parameters and camera installation parameters; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a corresponding coordinate in the reverse perspective view.
In this embodiment, the single-point perspective coordinate transformation formula is as follows:
wherein X represents an X-axis coordinate in a world coordinate system, Y represents a Y-axis coordinate in the world coordinate system, u represents an X-axis coordinate of a certain point in a perspective picture, v represents a Y-axis coordinate of a certain point in the perspective picture, h represents a camera mounting height in meters, M represents a camera longitudinal resolution in pixels, θ represents a camera mounting pitch angle in degrees, 2 α represents a camera longitudinal field angle in degrees, 2 β represents a camera transverse field angle in degrees, N represents a camera transverse resolution in pixels, u 'represents an X-axis coordinate corresponding to a certain point in the perspective picture in an inverse perspective view, v' represents a Y-axis coordinate corresponding to a certain point in the perspective picture in the inverse view, C represents a pixel equivalent of reconstructed image in meters/pixel.
Matrix calibration: when the agricultural machinery works, an ROI vertex is selected from a perspective picture collected by a camera, and an inverse perspective transformation matrix is determined according to a single-point perspective coordinate conversion formula and the ROI vertex.
In this embodiment, in the agricultural machinery operation process, four ROI vertices (u) of the trapezoidal region are selected from the perspective view acquired by the camera according to the forward distance and the lateral distance required for operation perceptioni,vi) Wherein u isiDenotes the abscissa, v, of the vertex of the ith ROIiAn ordinate representing the vertex of the ith ROI, i ═ 1,2,3, 4;
order to
Wherein u isi' represents the abscissa, v, of the i-th ROI vertex calculated based on a single-point perspective coordinate conversion formulai' representation is based on single point transmissionCalculating the vertical coordinate of the ith ROI vertex by a visual coordinate conversion formula; a denotes an inverse perspective transformation matrix, a11Elements representing the 1 st row and 1 st column of the inverse perspective transformation matrix, a12Elements representing the 1 st row and 2 nd column of the inverse perspective transformation matrix, a13Elements representing the 1 st row and 3 rd column of the inverse perspective transformation matrix, a21Elements representing the 2 nd row and 1 st column of the inverse perspective transformation matrix, a22Elements representing the 2 nd row and 2 nd column of the inverse perspective transformation matrix, a23Elements representing the 2 nd row and 3 rd column of the inverse perspective transformation matrix, a31Elements representing the 3 rd row and 1 st column of the inverse perspective transformation matrix, a32Representing the elements of row 3 and column 2 of the inverse perspective transformation matrix.
Fig. 2 is a schematic diagram illustrating a selection result of ROI vertices in a perspective picture according to an embodiment of the present invention, as shown in fig. 2, 4 ROI vertices corresponding to a near side and a far side can be manually selected from an original camera perspective view according to the requirements of forward and lateral distances of job awareness, and the four ROI vertices are represented as 4 vertices of a quadrilateral with a large size and a small size in the perspective view.
An online change step: and according to the inverse perspective transformation matrix, performing inverse perspective transformation on each single point in the region range determined by the ROI vertex to obtain an inverse perspective view with the perspective effect eliminated.
In the embodiment, P single points are selected in the region range determined by the ROI vertex, and are subjected to inverse perspective transformation through an inverse perspective transformation matrix to obtain corresponding P target points; wherein, P is a natural number more than 1; performing interpolation operation based on the target point to obtain Q pixel points, wherein Q is a natural number larger than P; and generating a reverse perspective view after eliminating the perspective effect based on the Q pixel points.
Fig. 3 is a schematic diagram of an inverse perspective view after the perspective effect is eliminated in the embodiment of the present invention, and as shown in fig. 3, an inverse perspective transformation matrix is obtained by solving 4 ROI vertices, so that the accuracy of the inverse perspective transformation matrix is effectively ensured. And in the linear transformation, performing single-point inverse perspective transformation and interpolation calculation on a quadrilateral region formed by 4 ROI vertexes in the input original perspective. Compared with the operation that the ROI is selected by a mask after the integral image is directly subjected to inverse perspective transformation or the integral image is subjected to inverse perspective transformation, the method and the device remarkably reduce the operation amount and realize the end-to-end direct conversion from the space ROI to the image ROI.
The embodiment of the invention also provides a system for eliminating the visual perception perspective effect of the front operation environment of the agricultural machinery, which comprises:
the camera calibration module is used for calibrating the camera to obtain camera internal parameters and camera installation parameters;
the conversion module is used for constructing a single-point perspective coordinate conversion formula according to camera internal parameters and installation parameters of the camera; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a corresponding coordinate in the reverse perspective;
the determining module is used for selecting an ROI vertex in the perspective picture acquired by the camera and determining an inverse perspective transformation matrix according to a single-point perspective coordinate conversion formula and the ROI vertex;
and the inverse transformation module is used for carrying out inverse perspective transformation on each single point in the region range determined by the ROI vertex according to the inverse perspective transformation matrix to obtain an inverse perspective view with the perspective effect eliminated.
Optionally, the camera internal reference comprises: the camera transverse field angle, the camera longitudinal field angle, the camera transverse resolution and the camera longitudinal resolution; the mounting parameters of the camera include: camera installation pitch angle, camera installation height.
Optionally, the single-point perspective coordinate transformation formula is as follows:
wherein X represents an X-axis coordinate in a world coordinate system, Y represents a Y-axis coordinate in the world coordinate system, u represents an X-axis coordinate of a certain point in a perspective picture, v represents a Y-axis coordinate of a certain point in the perspective picture, h represents a camera mounting height, M represents a camera longitudinal resolution, θ represents a camera mounting pitch angle, 2 α represents a camera longitudinal field angle, 2 β represents a camera lateral field angle, N represents a camera lateral resolution, u 'represents an X-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, v' represents a Y-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, and C represents a pixel equivalent of reconstructed imaging.
Optionally, the determining module is specifically configured to:
in the agricultural machinery operation process, four ROI vertexes (u) of a trapezoidal area are selected in a perspective view sheet collected by a camera according to the forward distance and the transverse distance required by operation perceptioni,vi) Wherein u isiDenotes the abscissa, v, of the vertex of the ith ROIiAn ordinate representing the vertex of the ith ROI, i ═ 1,2,3, 4;
order to
Wherein u isi' represents the abscissa, v, of the i-th ROI vertex calculated based on a single-point perspective coordinate conversion formulai' represents the vertical coordinate of the ith ROI vertex calculated based on a single-point perspective coordinate conversion formula; a denotes an inverse perspective transformation matrix, a11Elements representing the 1 st row and 1 st column of the inverse perspective transformation matrix, a12Elements representing the 1 st row and 2 nd column of the inverse perspective transformation matrix, a13Elements representing the 1 st row and 3 rd column of the inverse perspective transformation matrix, a21Elements representing the 2 nd row and 1 st column of the inverse perspective transformation matrix, a22Elements representing the 2 nd row and 2 nd column of the inverse perspective transformation matrix, a23Elements representing the 2 nd row and 3 rd column of the inverse perspective transformation matrix, a31Elements representing the 3 rd row and 1 st column of the inverse perspective transformation matrix, a32Representing the elements of row 3 and column 2 of the inverse perspective transformation matrix.
Optionally, the inverse transform module is specifically configured to:
selecting P single points in the region range determined by the ROI vertex, and performing inverse perspective transformation on the P single points through an inverse perspective transformation matrix to obtain corresponding P target points; wherein, P is a natural number more than 1;
performing interpolation operation based on the target point to obtain Q pixel points, wherein Q is a natural number larger than P;
and generating a reverse perspective view after eliminating the perspective effect based on the Q pixel points.
It should be noted that, the steps in the method for eliminating the visual perception perspective effect of the agricultural machinery front operation environment provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system for eliminating the visual perception perspective effect of the agricultural machinery front operation environment, and those skilled in the art can refer to the technical scheme of the system to implement the steps of the method, that is, the embodiments in the system can be understood as preferred examples of the implementation method, and are not described herein again.
Those skilled in the art will appreciate that, in addition to implementing the system and its various devices provided by the present invention in purely computer readable program code means, the method steps can be fully programmed to implement the same functions by implementing the system and its various devices in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices thereof provided by the present invention can be regarded as a hardware component, and the devices included in the system and various devices thereof for realizing various functions can also be regarded as structures in the hardware component; means for performing the functions may also be regarded as structures within both software modules and hardware components for performing the methods.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (10)
1. A method for eliminating visual perception perspective effect of an agricultural machinery front operation environment is characterized by comprising the following steps:
step 1: calibrating a camera to obtain camera internal parameters and camera installation parameters;
step 2: constructing a single-point perspective coordinate conversion formula according to the camera internal parameters and the installation parameters of the camera; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a coordinate corresponding to the reverse perspective;
and step 3: in the agricultural machinery operation process, an ROI vertex is selected from a perspective picture collected by a camera, and an inverse perspective transformation matrix is determined according to the single-point perspective coordinate conversion formula and the ROI vertex;
and 4, step 4: and according to the inverse perspective transformation matrix, performing inverse perspective transformation on each single point in the region range determined by the ROI vertex to obtain an inverse perspective view with the perspective effect eliminated.
2. The method for eliminating the visual perception perspective effect of the agricultural machinery front working environment according to claim 1, wherein the camera reference in the step 1 comprises: the camera transverse field angle, the camera longitudinal field angle, the camera transverse resolution and the camera longitudinal resolution; the installation parameters of the camera include: camera installation pitch angle, camera installation height.
3. The method for eliminating the visual perception perspective effect of the agricultural machinery front working environment according to claim 1, wherein the single-point perspective coordinate transformation formula in the step 2 is as follows:
wherein X represents an X-axis coordinate in a world coordinate system, Y represents a Y-axis coordinate in the world coordinate system, u represents an X-axis coordinate of a certain point in a perspective picture, v represents a Y-axis coordinate of a certain point in the perspective picture, h represents a camera mounting height, M represents a camera longitudinal resolution, θ represents a camera mounting pitch angle, 2 α represents a camera longitudinal field angle, 2 β represents a camera lateral field angle, N represents a camera lateral resolution, u 'represents an X-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, v' represents a Y-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, and C represents a pixel equivalent of reconstructed imaging.
4. The method for eliminating the visual perception perspective effect of the agricultural machinery front working environment according to the claim 1, wherein the step 3 comprises:
in the agricultural machinery operation process, four ROI vertexes (u) of a trapezoidal area are selected in a perspective view sheet collected by a camera according to the forward distance and the transverse distance required by operation perceptioni,vi) Wherein u isiDenotes the abscissa, v, of the vertex of the ith ROIiAn ordinate representing the vertex of the ith ROI, i ═ 1,2,3, 4;
order to
Wherein u isi' represents the abscissa, v, of the i-th ROI vertex calculated based on a single-point perspective coordinate conversion formulai' represents the vertical coordinate of the ith ROI vertex calculated based on a single-point perspective coordinate conversion formula; a denotes an inverse perspective transformation matrix, a11Elements representing the 1 st row and 1 st column of the inverse perspective transformation matrix, a12Elements representing the 1 st row and 2 nd column of the inverse perspective transformation matrix, a13Representing inverse perspective transformation momentsElements of array 1 row 3 column, a21Elements representing the 2 nd row and 1 st column of the inverse perspective transformation matrix, a22Elements representing the 2 nd row and 2 nd column of the inverse perspective transformation matrix, a23Elements representing the 2 nd row and 3 rd column of the inverse perspective transformation matrix, a31Elements representing the 3 rd row and 1 st column of the inverse perspective transformation matrix, a32Representing the elements of row 3 and column 2 of the inverse perspective transformation matrix.
5. The method for eliminating the visual perception perspective effect of the agricultural machinery front working environment according to any one of the claims 1 to 4, wherein the step 4 comprises:
selecting P single points in the region range determined by the ROI vertex, and performing inverse perspective transformation on the P single points through the inverse perspective transformation matrix to obtain corresponding P target points; wherein, P is a natural number more than 1;
performing interpolation operation based on the target point to obtain Q pixel points, wherein Q is a natural number larger than P;
and generating a reverse perspective view after eliminating the perspective effect based on the Q pixel points.
6. A system for eliminating visual perception perspective effect of an agricultural machinery front operation environment is characterized by comprising:
the camera calibration module is used for calibrating the camera to obtain camera internal parameters and camera installation parameters;
the conversion module is used for constructing a single-point perspective coordinate conversion formula according to the camera internal parameters and the installation parameters of the camera; the single-point perspective coordinate conversion formula is used for converting the coordinate of any point in the perspective picture into a coordinate corresponding to a world coordinate system and a coordinate corresponding to the reverse perspective;
the determining module is used for selecting an ROI vertex in the perspective picture acquired by the camera and determining an inverse perspective transformation matrix according to the single-point perspective coordinate conversion formula and the ROI vertex;
and the inverse transformation module is used for carrying out inverse perspective transformation on each single point in the region range determined by the ROI vertex according to the inverse perspective transformation matrix to obtain an inverse perspective view with the perspective effect eliminated.
7. The system for eliminating the visual perception perspective effect of the agricultural forward working environment according to claim 6, wherein the camera internal reference comprises: the camera transverse field angle, the camera longitudinal field angle, the camera transverse resolution and the camera longitudinal resolution; the installation parameters of the camera include: camera installation pitch angle, camera installation height.
8. The system for eliminating the visual perception perspective effect of the agricultural machinery front working environment according to claim 6, wherein the single point perspective coordinate transformation formula is as follows:
wherein X represents an X-axis coordinate in a world coordinate system, Y represents a Y-axis coordinate in the world coordinate system, u represents an X-axis coordinate of a certain point in a perspective picture, v represents a Y-axis coordinate of a certain point in the perspective picture, h represents a camera mounting height, M represents a camera longitudinal resolution, θ represents a camera mounting pitch angle, 2 α represents a camera longitudinal field angle, 2 β represents a camera lateral field angle, N represents a camera lateral resolution, u 'represents an X-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, v' represents a Y-axis coordinate corresponding to a certain point in an inverse perspective view in the perspective picture, and C represents a pixel equivalent of reconstructed imaging.
9. The system for eliminating the visual perception perspective effect of the agricultural forward working environment according to claim 1, wherein the determining module is specifically configured to:
during the operation of the agricultural machinery, according to the workSelecting four ROI vertexes (u) of trapezoidal region in perspective picture collected by camerai,vi) Wherein u isiDenotes the abscissa, v, of the vertex of the ith ROIiAn ordinate representing the vertex of the ith ROI, i ═ 1,2,3, 4;
order to
Wherein u isi' represents the abscissa, v, of the i-th ROI vertex calculated based on a single-point perspective coordinate conversion formulai' represents the vertical coordinate of the ith ROI vertex calculated based on a single-point perspective coordinate conversion formula; a denotes an inverse perspective transformation matrix, a11Elements representing the 1 st row and 1 st column of the inverse perspective transformation matrix, a12Elements representing the 1 st row and 2 nd column of the inverse perspective transformation matrix, a13Elements representing the 1 st row and 3 rd column of the inverse perspective transformation matrix, a21Elements representing the 2 nd row and 1 st column of the inverse perspective transformation matrix, a22Elements representing the 2 nd row and 2 nd column of the inverse perspective transformation matrix, a23Elements representing the 2 nd row and 3 rd column of the inverse perspective transformation matrix, a31Elements representing the 3 rd row and 1 st column of the inverse perspective transformation matrix, a32Representing the elements of row 3 and column 2 of the inverse perspective transformation matrix.
10. The system for eliminating the visual perception perspective effect of the agricultural forward working environment according to any one of claims 6 to 9, wherein the inverse transformation module is specifically configured to:
selecting P single points in the region range determined by the ROI vertex, and performing inverse perspective transformation on the P single points through the inverse perspective transformation matrix to obtain corresponding P target points; wherein, P is a natural number more than 1;
performing interpolation operation based on the target point to obtain Q pixel points, wherein Q is a natural number larger than P;
and generating a reverse perspective view after eliminating the perspective effect based on the Q pixel points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011490057.6A CN112614045B (en) | 2020-12-16 | 2020-12-16 | Method and system for eliminating visual perception perspective effect of agricultural machinery front operation environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011490057.6A CN112614045B (en) | 2020-12-16 | 2020-12-16 | Method and system for eliminating visual perception perspective effect of agricultural machinery front operation environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112614045A true CN112614045A (en) | 2021-04-06 |
| CN112614045B CN112614045B (en) | 2022-05-31 |
Family
ID=75239879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011490057.6A Active CN112614045B (en) | 2020-12-16 | 2020-12-16 | Method and system for eliminating visual perception perspective effect of agricultural machinery front operation environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112614045B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504680A (en) * | 2014-11-23 | 2015-04-08 | 北京联合大学 | Inverse perspective calibration method used for intelligent vehicle |
| CN109101924A (en) * | 2018-08-14 | 2018-12-28 | 武汉大学 | A kind of pavement marking recognition methods based on machine learning |
| CN109190452A (en) * | 2018-07-09 | 2019-01-11 | 北京农业智能装备技术研究中心 | Crop row recognition methods and device |
| CN109215071A (en) * | 2018-10-08 | 2019-01-15 | 东南大学 | The intelligent harvester for both rice and wheat swath measurement method of view-based access control model |
| CN110322513A (en) * | 2018-03-30 | 2019-10-11 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method, device and electronic equipment |
| CN111738223A (en) * | 2020-07-28 | 2020-10-02 | 上海眼控科技股份有限公司 | Frame number image generation method and device, computer equipment and storage medium |
-
2020
- 2020-12-16 CN CN202011490057.6A patent/CN112614045B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504680A (en) * | 2014-11-23 | 2015-04-08 | 北京联合大学 | Inverse perspective calibration method used for intelligent vehicle |
| CN110322513A (en) * | 2018-03-30 | 2019-10-11 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method, device and electronic equipment |
| CN109190452A (en) * | 2018-07-09 | 2019-01-11 | 北京农业智能装备技术研究中心 | Crop row recognition methods and device |
| CN109101924A (en) * | 2018-08-14 | 2018-12-28 | 武汉大学 | A kind of pavement marking recognition methods based on machine learning |
| CN109215071A (en) * | 2018-10-08 | 2019-01-15 | 东南大学 | The intelligent harvester for both rice and wheat swath measurement method of view-based access control model |
| CN111738223A (en) * | 2020-07-28 | 2020-10-02 | 上海眼控科技股份有限公司 | Frame number image generation method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112614045B (en) | 2022-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111353969B (en) | Method and device for determining road drivable area and computer equipment | |
| US10430951B2 (en) | Method and device for straight line detection and image processing | |
| CN109064516B (en) | Camera self-calibration method based on absolute quadratic curve image | |
| DE112018001050T5 (en) | SYSTEM AND METHOD FOR VIRTUALLY ENHANCED VISUAL SIMULTANEOUS LOCALIZATION AND CARTOGRAPHY | |
| CN111028350B (en) | Method for constructing grid map by using binocular stereo camera | |
| CN114399554A (en) | Calibration method and system of multi-camera system | |
| DE102013209415A1 (en) | Dynamic Note Overlay with Image Pruning | |
| CN106780388A (en) | A kind of line-scan digital camera optical distortion antidote | |
| DE102020118635A1 (en) | 3D data generation device and robot control system | |
| CN111508027A (en) | Method and device for calibrating external parameters of camera | |
| CN111709995B (en) | Position calibration method between laser radar and camera | |
| EP4071713B1 (en) | Parameter calibration method and apapratus | |
| CN107862656B (en) | Regularization realization method and system for 3D image point cloud data | |
| DE102016203710A1 (en) | Distance and direction estimation of a target point from a vehicle using a monocular video camera | |
| CN114067001A (en) | Vehicle-mounted camera angle calibration method, terminal and storage medium | |
| CN111445513B (en) | Plant canopy volume acquisition method and device based on depth image, computer equipment and storage medium | |
| CN114445415B (en) | Method for dividing drivable region and related device | |
| CN106780308B (en) | Image perspective transformation method | |
| KR102023087B1 (en) | Method for camera calibration | |
| CN112446926B (en) | Relative position calibration method and device for laser radar and multi-eye fish-eye camera | |
| CN112614045B (en) | Method and system for eliminating visual perception perspective effect of agricultural machinery front operation environment | |
| US7561306B2 (en) | One-dimensional lens shading correction | |
| DE202010017899U1 (en) | Arrangement for recording geometric and photometric object data in space | |
| CN116664622A (en) | Visual movement control method and device | |
| JP4747293B2 (en) | Image processing apparatus, image processing method, and program used therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |