CN117671543B - Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment - Google Patents
Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment Download PDFInfo
- Publication number
- CN117671543B CN117671543B CN202410129986.6A CN202410129986A CN117671543B CN 117671543 B CN117671543 B CN 117671543B CN 202410129986 A CN202410129986 A CN 202410129986A CN 117671543 B CN117671543 B CN 117671543B
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- unmanned aerial
- pixel
- camera
- focal length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 18
- 238000013507 mapping Methods 0.000 claims description 15
- 238000003384 imaging method Methods 0.000 claims description 8
- 230000003321 amplification Effects 0.000 claims description 6
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 3
- 238000004590 computer program Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000035800 maturation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention relates to the technical field of unmanned aerial vehicle image coordinate calibration, and provides an unmanned aerial vehicle image coordinate calibration method, an unmanned aerial vehicle image coordinate calibration system and electronic equipment, wherein the unmanned aerial vehicle image coordinate calibration method comprises the steps of controlling a camera plane of an unmanned aerial vehicle to be parallel to the ground, tracking a preset tracking area by utilizing a tracking algorithm, and obtaining a first pixel coordinate of the area center of the preset tracking area; controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and the second pixel coordinate of the center of the area, and calculating the first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle. The actual height of the unmanned aerial vehicle can be accurately calculated without laser ranging, and the reliability, accuracy and stability of converting the pixel coordinates into the geographic coordinates are greatly improved.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicle image coordinate calibration, in particular to an unmanned aerial vehicle image coordinate calibration method and system and electronic equipment.
Background
Along with the maturation of unmanned aerial vehicle related art, its product is gushed into each trade in a large number, and among the shooting scheme of current unmanned aerial vehicle, pixel coordinate and GPS geographic coordinate mapping technique are mostly fixed camera, and unmanned aerial vehicle camera faced the problem: the unmanned aerial vehicle height generally only can obtain the relative height of the relative take-off ground, and the accuracy of the relative height measured by the air pressure sensor generally does not meet the requirement, so that when pixel coordinates are converted to geographic coordinates, errors are large, the unmanned aerial vehicle flight distance is generally long, the general topography difference is large, if the unmanned aerial vehicle is calibrated on site every time the unmanned aerial vehicle flies to a place, the efficiency is low in reality, and the operability is poor.
Aiming at the problems that the unmanned plane is generally only capable of obtaining the relative height of the relative take-off ground, and is low in efficiency and poor in operability in reality if the unmanned plane is calibrated on site every time the unmanned plane flies to one place, no effective solution is proposed at present.
Thus, the prior art is still to be further developed.
Disclosure of Invention
The invention aims to overcome the technical defects and provide an unmanned aerial vehicle image coordinate calibration method, an unmanned aerial vehicle image coordinate calibration system and electronic equipment, so as to solve the problems in the prior art.
To achieve the above technical object, according to a first aspect of the present invention, there is provided a method for calibrating image coordinates of an unmanned aerial vehicle, including:
S100, controlling a camera plane of the unmanned aerial vehicle to be parallel to the ground, and tracking a preset tracking area by using a tracking algorithm to obtain a first pixel coordinate of the area center of the preset tracking area;
S200, controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and a second pixel coordinate of the center of the area, and calculating a first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate;
S300, acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle.
Specifically, the acquiring the first pixel coordinate of the area center of the preset tracking area includes:
acquiring a current frame image shot by a camera of the unmanned aerial vehicle, taking a first projection point of a camera optical center of the unmanned aerial vehicle on an imaging plane as an area center of a preset tracking area, taking the first projection point as a geometric center, establishing the preset tracking area according to a preset image size, acquiring a pixel coordinate of the first projection point before flying, taking the pixel coordinate as a first pixel coordinate of the area center of the preset tracking area, acquiring a pixel coordinate of the first projection point after flying, and taking the pixel coordinate as a second pixel coordinate of the area center of the preset tracking area.
Specifically, the controlling the unmanned aerial vehicle to fly along the horizontal direction and obtaining the flight distance of the unmanned aerial vehicle includes:
And acquiring GPS positioning information before and after the unmanned aerial vehicle flies, and calculating the flying distance of the unmanned aerial vehicle according to the GPS positioning information before and after the unmanned aerial vehicle flies.
Specifically, the calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance includes:
calculating a first ratio of the flight distance to the first pixel offset;
the actual height of the unmanned aerial vehicle is the product of the first ratio and the focal length of the lens.
Specifically, the acquiring the focal length of the camera of the unmanned aerial vehicle includes:
When the unmanned aerial vehicle is static, the camera plane of the control unmanned aerial vehicle is parallel with the auxiliary calibration plane and is spaced by a preset distance, a mark point is placed in the auxiliary calibration plane, the mark point can be shot by a camera of the unmanned aerial vehicle, a second projection point of the unmanned aerial vehicle, which is positioned on the auxiliary calibration plane, is obtained, a first actual distance from the second projection point to the mark point is obtained in the auxiliary calibration plane, a second pixel offset from the second projection point to the mark point is obtained in the imaging plane, the current amplification factor of the camera of the unmanned aerial vehicle is obtained, and the actual focal length of the camera of the unmanned aerial vehicle at the current amplification factor is calculated according to the first actual distance, the second pixel offset and the preset distance.
Specifically, the acquiring the focal length of the camera of the unmanned aerial vehicle further includes:
Changing the magnification of the camera of the unmanned aerial vehicle, calculating the actual focal length of the camera of the unmanned aerial vehicle under different magnification, forming a mapping curve of the actual focal length and the magnification, acquiring the magnification of the camera currently used by the unmanned aerial vehicle in the flight process of the unmanned aerial vehicle, acquiring the current actual focal length of the camera of the unmanned aerial vehicle according to the mapping curve, and calculating the actual height of the unmanned aerial vehicle according to the current actual focal length of the unmanned aerial vehicle.
Specifically, the auxiliary calibration plane includes at least one of the following:
horizontal plane, vertical plane.
Specifically, the calibrating the geographic coordinates corresponding to each pixel point of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle includes:
and calculating the vertex coordinates of four vertexes of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle and the GPS positioning coordinates of the unmanned aerial vehicle, and according to the vertex coordinates of the four vertexes of the image shot by the unmanned aerial vehicle, corresponding geographic coordinates of all pixels of the image shot by the unmanned aerial vehicle.
According to a second aspect of the present invention, there is provided a unmanned aerial vehicle image coordinate calibration system comprising:
the acquisition module is used for acquiring a first pixel coordinate of the region center of a preset tracking region; the camera focal length of the unmanned aerial vehicle is acquired;
The control module is used for controlling the camera plane of the unmanned aerial vehicle to be parallel to the ground, tracking the preset tracking area by utilizing a tracking algorithm, and acquiring a first pixel coordinate of the area center of the preset tracking area; the method comprises the steps of controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and a second pixel coordinate of the center of the area, and calculating a first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and the method is used for calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating the geographic coordinates corresponding to each pixel point of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle.
According to a third aspect of the present invention, there is provided an electronic device comprising: a memory; and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions implement the unmanned aerial vehicle image coordinate calibration method according to the above when executed by the processor.
The beneficial effects are that:
1. The invention provides an unmanned aerial vehicle image coordinate calibration method, a system and electronic equipment, wherein the method comprises the steps of controlling the plane of a camera of an unmanned aerial vehicle to be parallel to the ground, tracking a preset tracking area by utilizing a tracking algorithm, and obtaining a first pixel coordinate of the area center of the preset tracking area; controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and the second pixel coordinate of the center of the area, and calculating the first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle. The actual height of the unmanned aerial vehicle can be accurately calculated without laser ranging, the weight and the volume of the unmanned aerial vehicle are reduced to a great extent, the influence of environmental interference on laser ranging is completely avoided, and the reliability, the accuracy and the stability of converting pixel coordinates into geographic coordinates are improved to a great extent.
2. The invention aims at generating larger deviation parameters such as focal length deviation generated in the camera zooming process for accurate measurement of an algorithm in the actual hardware of the unmanned aerial vehicle camera, carries out advanced calibration, generates mapping curves of actual focal length and magnification factor, such as carrying out actual calibration at 5 times zooming, 10 times zooming, 20 times zooming and other places, acquires the actual focal length deviation parameters, forms the mapping curve of the actual focal length and the magnification factor, can map the relation between the actual magnification factor and the focal length, and is more accurate when the actual sampling is more, thereby obtaining an accurate focal length value.
Drawings
Fig. 1 is a flowchart of a method for calibrating image coordinates of a drone according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a system component of an image coordinate calibration system of a unmanned aerial vehicle according to an embodiment of the present invention;
FIG. 3 is a geometric diagram of a view field angle calculated according to camera internal parameters of an unmanned aerial vehicle pan-tilt in an embodiment of the present invention;
fig. 4 is a schematic geometric diagram of a horizontal distance between a unmanned aerial vehicle and a center point of a photographed image in an embodiment of the present invention;
FIG. 5 is a schematic geometrical diagram of calculating a vertical distance between a unmanned aerial vehicle and a ground projection range in an embodiment of the invention;
FIG. 6 is a geometric diagram of calculating the relative coordinates of the four vertices of the ground projection range and the center point of the trapezoid in an embodiment of the present invention;
FIG. 7 is a geometric schematic of ground projection ranges after unmanned aerial vehicle yaw and cradle head yaw rotations;
fig. 8 is a geometric schematic diagram of calculating an actual focal length of a camera of the unmanned aerial vehicle at a current magnification in an embodiment of the present invention.
Detailed Description
In order to make the technical solution of the present application better understood by those skilled in the art, the technical solution of the present application will be clearly and completely described in the following with reference to the accompanying drawings, and based on the embodiments of the present application, other similar embodiments obtained by those skilled in the art without making any inventive effort should be included in the scope of protection of the present application. In addition, directional words such as "upper", "lower", "left", "right", and the like, as used in the following embodiments are merely directions with reference to the drawings, and thus, the directional words used are intended to illustrate, not to limit, the application.
The invention will be further described with reference to the drawings and preferred embodiments.
Referring to fig. 1 and fig. 2 to 8, the present invention provides a method for calibrating image coordinates of an unmanned aerial vehicle, which includes:
S100, controlling a camera plane of the unmanned aerial vehicle to be parallel to the ground, and tracking a preset tracking area by using a tracking algorithm to obtain a first pixel coordinate of the area center of the preset tracking area;
Specifically, the acquiring the first pixel coordinate of the area center of the preset tracking area includes:
acquiring a current frame image shot by a camera of the unmanned aerial vehicle, taking a first projection point of a camera optical center of the unmanned aerial vehicle on an imaging plane as an area center of a preset tracking area, taking the first projection point as a geometric center, establishing the preset tracking area according to a preset image size, acquiring a pixel coordinate of the first projection point before flying, taking the pixel coordinate as a first pixel coordinate of the area center of the preset tracking area, acquiring a pixel coordinate of the first projection point after flying, and taking the pixel coordinate as a second pixel coordinate of the area center of the preset tracking area.
It should be noted that, the preset image size may be specifically set according to actual needs, the present invention does not specifically require the preset image size, and preferably, the preset image size is set to 20 pixels in the length direction and the width direction, and the preset image size is set to 20 pixels in the length direction and the width direction, which are obtained by a plurality of experiments by the technicians of the present invention, so that tracking of the preset tracking area can be better realized.
It should be noted that, in the prior art that tracking the preset tracking area by using the tracking algorithm is mature, for example, a mean shift algorithm and a target tracking algorithm based on Kalman filtering, the present invention is not described in detail herein.
And S200, controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and the second pixel coordinate of the center of the area, and calculating the first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate.
Specifically, the controlling the unmanned aerial vehicle to fly along the horizontal direction and obtaining the flight distance of the unmanned aerial vehicle includes:
And acquiring GPS positioning information before and after the unmanned aerial vehicle flies, and calculating the flying distance of the unmanned aerial vehicle according to the GPS positioning information before and after the unmanned aerial vehicle flies.
S300, acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle.
Here, the method for obtaining the actual height of the unmanned aerial vehicle further includes:
and directly measuring by using a laser ranging device and sending the measured result to a control module.
Specifically, the calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance includes:
calculating a first ratio of the flight distance to the first pixel offset;
the actual height of the unmanned aerial vehicle is the product of the first ratio and the focal length of the lens.
Here, the calibrating the geographic coordinates corresponding to each pixel point of the image shot by the unmanned aerial vehicle according to the actual altitude of the unmanned aerial vehicle includes:
The method comprises the steps of obtaining camera internal reference information of a tripod head of the unmanned aerial vehicle and the actual height h of the unmanned aerial vehicle obtained through calculation by the method, wherein the camera internal reference information comprises a tripod head pitch angle, a tripod head yaw angle yaw, an unmanned aerial vehicle yaw angle head, a longitude lat, a latitude lon, a video resolution width, a video resolution height, a focal length focal_len and the like. Based on these internal reference information, the field of view of the unmanned aerial vehicle can be calculated by the following steps. Specifically, the method comprises the following steps:
step S3011, calculating a view field angle according to the camera internal parameters of the unmanned aerial vehicle holder
Referring to fig. 3, the field angles include a vertical field angle Vfov and a horizontal field angle Hfov, and in combination with the image sensor chip size and focal length of the unmanned aerial vehicle pan-tilt camera receiving the light source, the vertical field angle Vfov and the horizontal field angle Hfov may be calculated as follows:
vfov = 2 arctan (chip height/2/focal_len)
Hfov = 2 arctan (chip width/2/focal_len)
Step S3012, calculating the relative coordinates of the four vertices of the video shot by the unmanned aerial vehicle in the ground projection range. The method specifically comprises the following steps:
(1) Calculating nearest side horizontal distance between unmanned aerial vehicle and shooting area
Referring to fig. 4, the first angle angle1= (pi/2+pitch-Vfov/2), i.e. the pan-tilt angle minus the camera reference angle, the pan-tilt is vertically downward by-90 degrees.
The closest horizontal distance x_near=h from the unmanned aerial vehicle to the shooting area tan (angle 1).
(2) Calculating the furthest horizontal distance between the unmanned plane and the ground projection range
Second angle 2= (pi/2+pitch+ Vfov/2), i.e. pan-tilt angle plus camera internal reference angle.
The horizontal distance x_far=h×tan (angle 2) between the furthest side of the unmanned plane and the ground projection range.
(3) Horizontal distance between unmanned aerial vehicle and center point of photographed image
x0 = h*tan(pi/2+Pitch)
(4) Calculate unmanned aerial vehicle and ground projection scope perpendicular distance, combine fig. 5, wherein:
Nearest vertical distance y_near=h/cos (angle 1) tan (Hfov/2)
Furthest perpendicular distance y_far=h/cos (angle 2) tan (Hfov/2)
Distance from the center point of the projection range: y0=h/cos (pi/2+pitch) tan (Hfov/2)
(5) Calculating the relative coordinates of four vertexes of the ground projection range and the central point of the trapezoid
In connection with fig. 6, the coordinates of the four vertices relative to the drone are: (x_near, y_near), (x_near, -y_near), (x_far, y_far), (x_far, -y_far).
The coordinates of the center point are as follows: (x 0, y 0).
Then, after the unmanned aerial vehicle yaw angle and the cradle head yaw angle are obtained (combined with fig. 7), the relative coordinates of four vertexes are as follows:
(x_near * cos(head + yaw) - y_near * sin(head + yaw), x_near * sin(head + yaw) + y_near * cos(head + yaw))、
(x_near * cos(head + yaw) + y_near * sin(head + yaw), x_near * sin(head + yaw) - y_near * cos(head + yaw))、
(x_far* cos(head + yaw) - y_far* sin(head + yaw), x_far* sin(head + yaw) + y_far* cos(head + yaw))、
(x_far* cos(head + yaw) + y_far* sin(head + yaw), x_far* sin(head + yaw) - y_far* cos(head + yaw))
The yaw coordinates of the center point are as follows: (x 0x cos (head+yaw) +y0 sin (head+yaw), x0 sin (head+yaw) -y0 x cos (head+yaw))
Step S3013, calculating geographic coordinates of four vertices of the ground projection range, specifically including:
(1) Converting the GPS coordinates of the drone to planar coordinates (X, Y), wherein:
X = N + k0 * E2 * sin(2 * lat) / 2 + (k5 - k4 + k3 * cos(2 * lat) - k2 * cos(4 * lat) + k1 * cos(6 * lat)) * sin(lon - L0)
Y = M + k0 * E2 * sin(lat) * cos(lat) * (1 + E2 * cos(lat)2) / 2 + (k6 - k3 * cos(2 * lat) + k2 * cos(4 * lat) - k1 * cos(6 * lat)) * sin(2 * (lon - L0)) / 2. Wherein:
N and M are constants, E is the eccentricity of the ellipsoid, k 0-k 6 are coefficients, and L0 is the central meridian.
(2) The calculation of the longitude and latitude coordinates of the four vertexes of the ground projection range and the longitude and latitude coordinates of the center point specifically comprises the following steps:
The plane coordinates of the vertexes are calculated according to the relative coordinates of the plane coordinates of the unmanned plane and the vertexes, and are as follows:
neu_X = X + x
neu_Y = Y + y
In the substep (4) of the step S3012, X and Y are relative coordinates of four vertices of the unmanned aerial vehicle video ground projection range, and the calculated vertex plane coordinates neu_x and neu_y are converted into longitude and latitude coordinates lon and lat.
Step S3014, calculating a homography matrix of pixel coordinates and geographic coordinates for mutual conversion, which specifically includes:
=/>
wherein x1, y1 is longitude and latitude coordinates, and x2, y2 are pixel coordinates.
Form of matrix expansion:
Written in the form of a matrix ah=0:
= 0
After the homography matrix is obtained, the intercommunication conversion between the pixel coordinates and the geographic coordinates can be carried out, the pixel coordinates can be freely drawn in the video frame of the web end, and the pixel coordinates are converted into the geographic coordinates through the homography matrix; the surface composed of the geographic coordinates is also drawn at the webgis end, and the surface can be converted into pixel coordinates through a homography matrix and sent to a specified theme for calling.
Specifically, the acquiring the focal length of the camera of the unmanned aerial vehicle includes:
When the unmanned aerial vehicle is static, the camera plane of the control unmanned aerial vehicle is parallel with the auxiliary calibration plane and is spaced by a preset distance, a mark point is placed in the auxiliary calibration plane, the mark point can be shot by a camera of the unmanned aerial vehicle, a second projection point of the unmanned aerial vehicle, which is positioned on the auxiliary calibration plane, is obtained, a first actual distance from the second projection point to the mark point is obtained in the auxiliary calibration plane, a second pixel offset from the second projection point to the mark point is obtained in the imaging plane, the current amplification factor of the camera of the unmanned aerial vehicle is obtained, and the actual focal length of the camera of the unmanned aerial vehicle at the current amplification factor is calculated according to the first actual distance, the second pixel offset and the preset distance.
Here, it is noted that
Specifically, the acquiring the focal length of the camera of the unmanned aerial vehicle further includes:
Changing the magnification of the camera of the unmanned aerial vehicle, calculating the actual focal length of the camera of the unmanned aerial vehicle under different magnification, forming a mapping curve of the actual focal length and the magnification, acquiring the magnification of the camera currently used by the unmanned aerial vehicle in the flight process of the unmanned aerial vehicle, acquiring the current actual focal length of the camera of the unmanned aerial vehicle according to the mapping curve, and calculating the actual height of the unmanned aerial vehicle according to the current actual focal length of the unmanned aerial vehicle.
Specifically, the auxiliary calibration plane includes at least one of the following:
horizontal plane, vertical plane.
The following illustrates the method of the present invention for calculating the actual focal length of the camera of the unmanned aerial vehicle at the current magnification:
Referring to fig. 8, oc is a focal length of a camera, an auxiliary calibration plane is set as a vertical plane MN, when the unmanned aerial vehicle is stationary, the camera plane of the unmanned aerial vehicle is controlled to be parallel to the auxiliary calibration plane and to be spaced by a preset distance OE, it can be understood that a measuring method of the distance OE includes manual measurement, the camera plane of the unmanned aerial vehicle is controlled to be parallel to the auxiliary calibration plane and to be spaced by the preset distance OE, an O point is a camera optical center of the unmanned aerial vehicle, AB is an imaging plane of the camera, a mark point D is placed in the auxiliary calibration plane, the mark point D can be shot by the camera of the unmanned aerial vehicle, a second projection point E of the camera optical center of the unmanned aerial vehicle in the auxiliary calibration plane is acquired, a first actual distance DE from the second projection point E to the mark point D is acquired in the auxiliary calibration plane, the measuring method of the DE includes manual measurement, the setting mode of the mark point D is manual measurement, the specific physical structure of the mark point D is not limited, preferably, the mark point D used in the invention is an imaging plane of the camera, the mark point D is a mark point D is placed in the auxiliary calibration plane, the actual distance is set in the plane, the actual distance GC is calculated to be similar to the actual distance of the unmanned aerial vehicle, and the actual distance GC is calculated to the actual distance of the actual distance GC is equal to the actual distance of the camera.
Specifically, the calibrating the geographic coordinates corresponding to each pixel point of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle includes:
and calculating the vertex coordinates of four vertexes of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle and the GPS positioning coordinates of the unmanned aerial vehicle, and according to the vertex coordinates of the four vertexes of the image shot by the unmanned aerial vehicle, corresponding geographic coordinates of all pixels of the image shot by the unmanned aerial vehicle.
It can be appreciated that the invention provides an unmanned aerial vehicle image coordinate calibration method, an unmanned aerial vehicle image coordinate calibration system and electronic equipment, wherein the unmanned aerial vehicle image coordinate calibration method comprises the steps of controlling the plane of a camera of an unmanned aerial vehicle to be parallel to the ground, tracking a preset tracking area by utilizing a tracking algorithm, and obtaining a first pixel coordinate of the area center of the preset tracking area; controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and the second pixel coordinate of the center of the area, and calculating the first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle. The actual height of the unmanned aerial vehicle can be accurately calculated without laser ranging, the weight and the volume of the unmanned aerial vehicle are reduced to a great extent, the influence of environmental interference on laser ranging is completely avoided, and the reliability, the accuracy and the stability of converting pixel coordinates into geographic coordinates are improved to a great extent.
It should be noted that, in the invention, aiming at the fact that the algorithm is accurately measured in the actual hardware of the unmanned aerial vehicle camera to generate larger deviation parameters such as focal length deviation generated in the zooming process of the camera, the invention carries out advanced calibration to generate mapping curves of actual focal length and magnification factors, such as actual calibration at 5 times of zooming, 10 times of zooming, 20 times of zooming and other places, and obtains the actual focal length deviation parameters to form the mapping curves of actual focal length and magnification factors, and the relationship between the actual magnification factors and focal length can be mapped by the curves, so that the more actual samples are, the more accurate the actual samples are, and the accurate focal length value is obtained.
Referring to fig. 2, another embodiment of the present invention provides an unmanned aerial vehicle image coordinate calibration system, including:
An acquiring module 100, configured to acquire a first pixel coordinate of a region center of a preset tracking region; the camera focal length of the unmanned aerial vehicle is acquired;
The control module 200 is used for controlling the camera plane of the unmanned aerial vehicle to be parallel to the ground, tracking the preset tracking area by utilizing a tracking algorithm, and obtaining a first pixel coordinate of the area center of the preset tracking area; the method comprises the steps of controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and a second pixel coordinate of the center of the area, and calculating a first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and the method is used for calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating the geographic coordinates corresponding to each pixel point of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle.
It can be appreciated that the invention provides an unmanned aerial vehicle image coordinate calibration method, an unmanned aerial vehicle image coordinate calibration system and electronic equipment, wherein the unmanned aerial vehicle image coordinate calibration method comprises the steps of controlling the plane of a camera of an unmanned aerial vehicle to be parallel to the ground, tracking a preset tracking area by utilizing a tracking algorithm, and obtaining a first pixel coordinate of the area center of the preset tracking area; controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and the second pixel coordinate of the center of the area, and calculating the first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle. The actual height of the unmanned aerial vehicle can be accurately calculated without laser ranging, the weight and the volume of the unmanned aerial vehicle are reduced to a great extent, the influence of environmental interference on laser ranging is completely avoided, and the reliability, the accuracy and the stability of converting pixel coordinates into geographic coordinates are improved to a great extent.
It should be noted that, in the invention, aiming at the fact that the algorithm is accurately measured in the actual hardware of the unmanned aerial vehicle camera to generate larger deviation parameters such as focal length deviation generated in the zooming process of the camera, the invention carries out advanced calibration to generate mapping curves of actual focal length and magnification factors, such as actual calibration at 5 times of zooming, 10 times of zooming, 20 times of zooming and other places, and obtains the actual focal length deviation parameters to form the mapping curves of actual focal length and magnification factors, and the relationship between the actual magnification factors and focal length can be mapped by the curves, so that the more actual samples are, the more accurate the actual samples are, and the accurate focal length value is obtained.
In a preferred embodiment, the present application also provides an electronic device, including:
A memory; and a processor, wherein the memory stores computer readable instructions that when executed by the processor implement the unmanned aerial vehicle image coordinate calibration method. The computer device may be broadly a server, a terminal, or any other electronic device having the necessary computing and/or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc. connected by a system bus. The processor of the computer device may be used to provide the necessary computing, processing and/or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may have an operating system, computer programs, etc. stored therein or thereon. The internal memory may provide an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface and communication interface of the computer device may be used to connect and communicate with external devices via a network. Which when executed by a processor performs the steps of the method of the invention.
The present invention may be implemented as a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes steps of a method of an embodiment of the present invention to be performed. In one embodiment, the computer program is distributed over a plurality of computer devices or processors coupled by a network such that the computer program is stored, accessed, and executed by one or more computer devices or processors in a distributed fashion. A single method step/operation, or two or more method steps/operations, may be performed by a single computer device or processor, or by two or more computer devices or processors. One or more method steps/operations may be performed by one or more computer devices or processors, and one or more other method steps/operations may be performed by one or more other computer devices or processors. One or more computer devices or processors may perform a single method step/operation or two or more method steps/operations.
Those of ordinary skill in the art will appreciate that the method steps of the present invention may be implemented by a computer program, which may be stored on a non-transitory computer readable storage medium, to instruct related hardware such as a computer device or a processor, which when executed causes the steps of the present invention to be performed. Any reference herein to memory, storage, database, or other medium may include non-volatile and/or volatile memory, as the case may be. Examples of nonvolatile memory include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage, hard disk, solid state disk, and the like. Examples of volatile memory include Random Access Memory (RAM), external cache memory, and the like.
It can be appreciated that the invention provides an unmanned aerial vehicle image coordinate calibration method, an unmanned aerial vehicle image coordinate calibration system and electronic equipment, wherein the unmanned aerial vehicle image coordinate calibration method comprises the steps of controlling the plane of a camera of an unmanned aerial vehicle to be parallel to the ground, tracking a preset tracking area by utilizing a tracking algorithm, and obtaining a first pixel coordinate of the area center of the preset tracking area; controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and the second pixel coordinate of the center of the area, and calculating the first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle. The actual height of the unmanned aerial vehicle can be accurately calculated without laser ranging, the weight and the volume of the unmanned aerial vehicle are reduced to a great extent, the influence of environmental interference on laser ranging is completely avoided, and the reliability, the accuracy and the stability of converting pixel coordinates into geographic coordinates are improved to a great extent.
It should be noted that, in the invention, aiming at the fact that the algorithm is accurately measured in the actual hardware of the unmanned aerial vehicle camera to generate larger deviation parameters such as focal length deviation generated in the zooming process of the camera, the invention carries out advanced calibration to generate mapping curves of actual focal length and magnification factors, such as actual calibration at 5 times of zooming, 10 times of zooming, 20 times of zooming and other places, and obtains the actual focal length deviation parameters to form the mapping curves of actual focal length and magnification factors, and the relationship between the actual magnification factors and focal length can be mapped by the curves, so that the more actual samples are, the more accurate the actual samples are, and the accurate focal length value is obtained.
The technical features described above may be arbitrarily combined. Although not all possible combinations of features are described, any combination of features should be considered to be covered by the description provided that such combinations are not inconsistent.
The above-described embodiments of the present invention do not limit the scope of the present invention. Any other corresponding changes and modifications made in accordance with the technical idea of the present invention shall be included in the scope of the claims of the present invention.
Claims (7)
1. A method for calibrating image coordinates of an unmanned aerial vehicle, the method comprising:
S100, controlling a camera plane of the unmanned aerial vehicle to be parallel to the ground, and tracking a preset tracking area by using a tracking algorithm to obtain a first pixel coordinate of the area center of the preset tracking area;
S200, controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and a second pixel coordinate of the center of the area, and calculating a first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate;
s300, acquiring a camera focal length of the unmanned aerial vehicle, calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating geographic coordinates corresponding to each pixel point of an image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle;
The acquiring the first pixel coordinate of the area center of the preset tracking area includes:
Acquiring a current frame image shot by a camera of the unmanned aerial vehicle, taking a first projection point of a camera optical center of the unmanned aerial vehicle on an imaging plane as an area center of a preset tracking area, taking the first projection point as a geometric center, establishing the preset tracking area according to a preset image size, acquiring a pixel coordinate of the first projection point before flying, taking the pixel coordinate as a first pixel coordinate of the area center of the preset tracking area, acquiring a pixel coordinate of the first projection point after flying, and taking the pixel coordinate as a second pixel coordinate of the area center of the preset tracking area;
the controlling the unmanned aerial vehicle to fly along the horizontal direction and obtaining the flying distance of the unmanned aerial vehicle comprises the following steps:
acquiring GPS positioning information before and after the unmanned aerial vehicle flies, and calculating the flying distance of the unmanned aerial vehicle according to the GPS positioning information before and after the unmanned aerial vehicle flies;
according to the first pixel offset, the lens focal length and the flight distance, the actual height of the unmanned aerial vehicle is calculated, and the method comprises the following steps:
calculating a first ratio of the flight distance to the first pixel offset;
the actual height of the unmanned aerial vehicle is the product of the first ratio and the focal length of the lens.
2. The unmanned aerial vehicle image coordinate calibration method of claim 1, wherein the obtaining a focal length of a camera of the unmanned aerial vehicle comprises:
When the unmanned aerial vehicle is static, the camera plane of the control unmanned aerial vehicle is parallel with the auxiliary calibration plane and is spaced by a preset distance, a mark point is placed in the auxiliary calibration plane, the mark point can be shot by a camera of the unmanned aerial vehicle, a second projection point of the unmanned aerial vehicle, which is positioned on the auxiliary calibration plane, is obtained, a first actual distance from the second projection point to the mark point is obtained in the auxiliary calibration plane, a second pixel offset from the second projection point to the mark point is obtained in the imaging plane, the current amplification factor of the camera of the unmanned aerial vehicle is obtained, and the actual focal length of the camera of the unmanned aerial vehicle at the current amplification factor is calculated according to the first actual distance, the second pixel offset and the preset distance.
3. The unmanned aerial vehicle image coordinate calibration method of claim 2, wherein the acquiring the focal length of the camera of the unmanned aerial vehicle further comprises:
Changing the magnification of the camera of the unmanned aerial vehicle, calculating the actual focal length of the camera of the unmanned aerial vehicle under different magnification, forming a mapping curve of the actual focal length and the magnification, acquiring the magnification of the camera currently used by the unmanned aerial vehicle in the flight process of the unmanned aerial vehicle, acquiring the current actual focal length of the camera of the unmanned aerial vehicle according to the mapping curve, and calculating the actual height of the unmanned aerial vehicle according to the current actual focal length of the unmanned aerial vehicle.
4. The unmanned aerial vehicle image coordinate calibration method of claim 2, wherein the auxiliary calibration plane comprises at least one of:
horizontal plane, vertical plane.
5. The method for calibrating image coordinates of an unmanned aerial vehicle according to claim 1, wherein the calibrating the geographic coordinates corresponding to each pixel of the image captured by the unmanned aerial vehicle according to the actual altitude of the unmanned aerial vehicle comprises:
and calculating the vertex coordinates of four vertexes of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle and the GPS positioning coordinates of the unmanned aerial vehicle, and according to the vertex coordinates of the four vertexes of the image shot by the unmanned aerial vehicle, corresponding geographic coordinates of all pixels of the image shot by the unmanned aerial vehicle.
6. A unmanned aerial vehicle image coordinate calibration system, wherein the unmanned aerial vehicle image coordinate calibration method according to any one of claims 1 to 5 is employed, the system comprising:
the acquisition module is used for acquiring a first pixel coordinate of the region center of a preset tracking region; the camera focal length of the unmanned aerial vehicle is acquired;
The control module is used for controlling the camera plane of the unmanned aerial vehicle to be parallel to the ground, tracking the preset tracking area by utilizing a tracking algorithm, and acquiring a first pixel coordinate of the area center of the preset tracking area; the method comprises the steps of controlling the unmanned aerial vehicle to fly along the horizontal direction, acquiring the flying distance of the unmanned aerial vehicle and a second pixel coordinate of the center of the area, and calculating a first pixel offset of the center of the area by using the first pixel coordinate and the second pixel coordinate; and the method is used for calculating the actual height of the unmanned aerial vehicle according to the first pixel offset, the lens focal length and the flight distance, and calibrating the geographic coordinates corresponding to each pixel point of the image shot by the unmanned aerial vehicle according to the actual height of the unmanned aerial vehicle.
7. An electronic device, comprising:
A memory; and a processor having stored thereon computer readable instructions which when executed by the processor implement the unmanned aerial vehicle image coordinate calibration method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410129986.6A CN117671543B (en) | 2024-01-31 | 2024-01-31 | Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410129986.6A CN117671543B (en) | 2024-01-31 | 2024-01-31 | Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117671543A CN117671543A (en) | 2024-03-08 |
| CN117671543B true CN117671543B (en) | 2024-04-19 |
Family
ID=90064409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410129986.6A Active CN117671543B (en) | 2024-01-31 | 2024-01-31 | Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117671543B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107749072A (en) * | 2017-09-21 | 2018-03-02 | 黄仁杰 | A kind of unmanned plane image calibrating method suitable for domatic measurement |
| KR101874498B1 (en) * | 2017-02-23 | 2018-07-05 | 창원대학교 산학협력단 | System and Method for Aerial Photogrammetry of Ground Control Point for Space Information Acquisition based on Unmanned Aerial Vehicle System |
| CN110310243A (en) * | 2019-06-28 | 2019-10-08 | 广东工业大学 | A kind of method for correcting image that unmanned plane is photogrammetric, system and storage medium |
| CN113793270A (en) * | 2021-08-05 | 2021-12-14 | 杭州电子科技大学 | Aerial image geometric correction method based on unmanned aerial vehicle attitude information |
| KR20220080907A (en) * | 2020-12-08 | 2022-06-15 | 주식회사 엘지유플러스 | Flight method for unmanned aerial vehicle tracking objects based on pixel coordinates |
-
2024
- 2024-01-31 CN CN202410129986.6A patent/CN117671543B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101874498B1 (en) * | 2017-02-23 | 2018-07-05 | 창원대학교 산학협력단 | System and Method for Aerial Photogrammetry of Ground Control Point for Space Information Acquisition based on Unmanned Aerial Vehicle System |
| CN107749072A (en) * | 2017-09-21 | 2018-03-02 | 黄仁杰 | A kind of unmanned plane image calibrating method suitable for domatic measurement |
| CN110310243A (en) * | 2019-06-28 | 2019-10-08 | 广东工业大学 | A kind of method for correcting image that unmanned plane is photogrammetric, system and storage medium |
| KR20220080907A (en) * | 2020-12-08 | 2022-06-15 | 주식회사 엘지유플러스 | Flight method for unmanned aerial vehicle tracking objects based on pixel coordinates |
| CN113793270A (en) * | 2021-08-05 | 2021-12-14 | 杭州电子科技大学 | Aerial image geometric correction method based on unmanned aerial vehicle attitude information |
Non-Patent Citations (2)
| Title |
|---|
| Capability of Low Cost Digital Camera For Production of Orthophoto and Volumn Determination;Khairul Nizam Tahar et al;《2011 IEEE 7th International Colloquium on Signal Processing and its Applications》;20110502;第67-71页 * |
| 基于双目视觉的无人飞行器目标跟踪与定位;张梁 等;计算机工程与应用;20150104;50(24);第27-31, 54页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117671543A (en) | 2024-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109523471B (en) | Method, system and device for converting ground coordinates and wide-angle camera picture coordinates | |
| JP7037302B2 (en) | Survey data processing device, survey data processing method and survey data processing program | |
| US10467726B2 (en) | Post capture imagery processing and deployment systems | |
| CN111461994A (en) | Method for obtaining coordinate transformation matrix and positioning target in monitoring picture | |
| US9185289B2 (en) | Generating a composite field of view using a plurality of oblique panoramic images of a geographic area | |
| CN113345028B (en) | Method and equipment for determining target coordinate transformation information | |
| CN112470092A (en) | Surveying and mapping system, surveying and mapping method, device, equipment and medium | |
| CN110736447B (en) | Vertical-direction horizontal position calibration method for integrated image acquisition equipment | |
| CN109099889A (en) | Close range photogrammetric system and method | |
| CN115439531A (en) | Method and equipment for acquiring target space position information of target object | |
| CN115876197A (en) | Mooring lifting photoelectric imaging target positioning method | |
| CN111527375B (en) | Planning method and device for surveying and mapping sampling point, control terminal and storage medium | |
| CN113415433B (en) | Pod attitude correction method and device based on three-dimensional scene model and unmanned aerial vehicle | |
| CN112750075A (en) | Low-altitude remote sensing image splicing method and device | |
| US7768631B1 (en) | Method and system for providing a known reference point for an airborne imaging platform | |
| Zhou et al. | Automatic orthorectification and mosaicking of oblique images from a zoom lens aerial camera | |
| CN114565677A (en) | Positioning deviation rectifying method, monitoring equipment and computer readable storage medium | |
| CN117671543B (en) | Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment | |
| CN111103899A (en) | Holder positioning method and device | |
| KR102028323B1 (en) | Synthetic Aperture Radar Image Enhancement Apparatus and System | |
| US11415990B2 (en) | Optical object tracking on focal plane with dynamic focal length | |
| CN111868656A (en) | Operation control system, operation control method, device, equipment and medium | |
| Guntel et al. | Accuracy analysis of control point distribution for different terrain types on photogrammetric block | |
| WO2021134715A1 (en) | Control method and device, unmanned aerial vehicle and storage medium | |
| CN116839595B (en) | Method for creating unmanned aerial vehicle route |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |