CN114862959A - Method and apparatus for controlling camera - Google Patents
Method and apparatus for controlling camera Download PDFInfo
- Publication number
- CN114862959A CN114862959A CN202210302903.XA CN202210302903A CN114862959A CN 114862959 A CN114862959 A CN 114862959A CN 202210302903 A CN202210302903 A CN 202210302903A CN 114862959 A CN114862959 A CN 114862959A
- Authority
- CN
- China
- Prior art keywords
- camera
- coordinate system
- tilt
- pan
- pose
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
- G06T3/604—Rotation of a whole image or part thereof using a CORDIC [COordinate Rotation Digital Compute] device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
Abstract
A method and apparatus for controlling a camera, the method comprising: acquiring multiple groups of calibration data of the camera, wherein each group of calibration data comprises: PTZ data and satellite map data output by the camera; solving the camera pose estimation model according to the multiple groups of calibration data to acquire six-degree-of-freedom pose parameters of the camera; acquiring PTZ control parameters of a camera according to coordinates and pose parameters of a target to be observed in a world coordinate system; PTZ control parameters of the camera are output. The scheme does not depend on an external sensor, but depends on a satellite map to calibrate and link the camera, has low cost, models the camera in the pose with six degrees of freedom, can complete the calibration process by acquiring a small amount of calibration data, and improves the calibration efficiency and the control precision of the camera.
Description
Technical Field
The present application relates to the field of information technology, and in particular, to a method and an apparatus for controlling a camera.
Background
The observation camera can intuitively acquire real-time and historical site condition information, and is widely applied to various public traffic scenes as an important means of modern supervision, such as areas of expressways, urban roads, maritime ports and the like. The traditional supervision means mainly comprises the step of manually controlling the rotation angle and the magnification of the ball control machine to carry out manual observation, the working efficiency is low, and an observation target is easily lost. Therefore, at present, a camera intelligent linkage method is mostly adopted to automatically control the dome camera, and the detailed picture of the target scene can be acquired more quickly and efficiently.
In the existing intelligent observation scene, the mapping relation between the coordinate system of a sensor (such as a gunlock and a radar) fixed outside and the angle of a ball machine to be controlled is calibrated, and the mapping relation is converted into a ball machine control instruction according to the input coordinate under the external sensor. For example, in the gun and ball linkage scheme, a target to be observed may be selected from a fixed gun, and the target to be observed may be steered and tracked using a ball machine. The gun and ball linkage scheme requires the combined deployment of a gun camera and a ball machine, extra hardware cost is needed for partial old camera scenes, and meanwhile, the rotation angle of the ball machine is limited in the visual range of the gun camera, and the 360-degree rotation capability cannot be fully utilized. In addition, the gun-ball linkage scheme mostly adopts a grid method for calibration, a large number of preset positions need to be calibrated, and the rotation angle of the ball outlet machine is estimated through interpolation, so that the problems of complicated calibration, inaccurate control angle, low reliability and the like exist.
Therefore, there is a need for a method for controlling a camera to reduce the complexity of the camera calibration and camera linkage process.
Disclosure of Invention
The application provides a method and a device for controlling a camera, which are used for improving the calibration efficiency and the control precision of the camera.
In a first aspect, a method for controlling a camera is provided, comprising: acquiring multiple groups of calibration data of the camera, wherein each group of calibration data comprises: PTZ data and satellite map data output by the camera, wherein the PTZ data comprises a pan value and a tilt value of the camera corresponding to the characteristic point, and the satellite map data comprises coordinates of the characteristic point in a world coordinate system; solving a camera pose estimation model according to the multiple groups of calibration data to acquire pose parameters of six degrees of freedom of the camera; acquiring PTZ control parameters of the camera according to the coordinates of the target to be observed in the world coordinate system and the pose parameters; outputting PTZ control parameters of the camera.
In a second aspect, the present application provides an apparatus for controlling a camera, comprising: the communication module is used for acquiring multiple groups of calibration data of the camera, and each group of calibration data comprises: PTZ data and satellite map data output by the camera, wherein the PTZ data comprises a pan value and a tilt value of the camera corresponding to the characteristic point, and the satellite map data comprises coordinates of the characteristic point in a world coordinate system; the processing module is used for solving the camera pose estimation model according to the multiple groups of calibration data so as to acquire pose parameters of six degrees of freedom of the camera; the processing module is further used for acquiring PTZ control parameters of the camera according to the coordinates of the target to be observed in the world coordinate system and the pose parameters; the communication module is further to output PTZ control parameters of the camera.
In a third aspect, there is provided a computer device comprising a processor for invoking a computer program from a memory, the processor being adapted to perform the method of the first aspect when the computer program is executed.
In a fourth aspect, there is provided a computer readable storage medium for storing a computer program comprising code for performing the method of the first aspect described above.
In a fifth aspect, there is provided a computer program product comprising a computer program comprising code for performing the method of the first aspect described above.
In the embodiment of the application, a scheme for controlling the camera is provided, the scheme does not depend on an external sensor as the input of camera calibration, but depends on a satellite map to calibrate and link the camera, the cost is low, the transverse and longitudinal full-coverage angle control of the camera can be guaranteed, and the variable magnification control of the maximum magnification range is realized according to the distance between the target to be observed and the camera. The method and the device fully consider the installation state of the camera, carry out modeling of the pose with six degrees of freedom, and can finish the calibration process by acquiring a small amount of calibration data, thereby improving the calibration efficiency and the control precision of the camera.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an application scenario of yet another embodiment of the present application;
FIG. 3 is a schematic diagram of the internal structure of the computing device 100 according to an embodiment of the present application;
FIG. 4 is a scene schematic diagram of a camera pose of an embodiment of the present application;
FIG. 5 is a scene schematic diagram of a camera pose of an embodiment of the present application;
FIG. 6 is a flow chart of a method for controlling a camera according to an embodiment of the present application;
FIG. 7 is a schematic block diagram of an apparatus 700 according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an apparatus 800 according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
First, terms referred to in the present application will be explained.
Pose with six degrees of freedom: the object has six degrees of freedom in a three-dimensional coordinate system, including: the three orthogonal coordinate axes of x, y and z have freedom of movement in the directions and freedom of rotation around the three axes.
Calibrating a camera: and constructing a mapping model from a world coordinate system to a pixel coordinate system, and acquiring parameters required in the model.
PTZ: the camera pan/tilt/zoom abbreviation is used in security monitoring application and respectively represents the left and right movement, the up and down movement and the lens zooming coefficient of a camera pan-tilt.
The holder: the device is a supporting device for mounting and fixing a mobile phone, a camera and a video camera. The holder can rotate up and down or left and right to be suitable for application scenes.
World coordinate system: refers to the absolute coordinate System of the System, and the World coordinate System may be a geographic coordinate System, such as World Geodetic System (WGS 84), or any fixedly defined World coordinate System.
Camera coordinate system: the camera coordinate system is a relative coordinate system in which the optical center of the camera is used as an origin, the X axis points to the left and right directions of the camera, the Y axis points to the up and down directions of the camera, and the Z axis points to the direction observed by the camera, and changes with the movement of the camera.
Pixel coordinate system: the image pixel coordinate system is a plane rectangular coordinate system which is fixed on the image and takes pixels as units, the origin of the rectangular coordinate system is positioned at the upper left corner of the image, the X axis and the Y axis are parallel to the X axis and the Y axis of the camera coordinate system, and the rectangular coordinate system is a relative coordinate system depending on the camera coordinate system.
Body coordinate system of camera: the optical center of the camera is used as an origin, the X axis points to the observation direction of the camera, the Y axis is the left and right direction of the camera, and the Z axis is the up and down direction of the camera.
Fig. 1 is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in fig. 1, in a public transportation video system, an observation camera, an exchange system, and a supervision center are generally included. The observation cameras are widely arranged at various traffic intersections, ports and other positions to acquire images or videos of vehicles or ships in public traffic scenes, and the acquired images or videos are transmitted to the supervision center through the switch system. A display device, a storage device, and a computing device are typically included in a regulatory center. The storage device may be configured to store the acquired image or video information, the display device may be configured to display the observation image, and the computing device may parse and process the acquired image or video. In some examples, the computing device may include a server.
Alternatively, as shown in fig. 1, the observation camera may generally include both a ball machine type and a gun type, and the gun type camera is mounted in a fixed position to face only a certain observation position, so that the observation direction is limited. The ball camera is a spherical camera, and integrates a camera system, a zoom lens and an electronic pan-tilt, the ball camera can set a preset position for cruising according to the field condition and the user requirement, and set the dwell time and zoom-in and zoom-out of a lens at an important point position for key observation.
It should be understood that the description of the application scenario in fig. 1 is only by way of example and not limitation, and in practice, appropriate modifications and additions may be made to the above scenario and still apply to the solution of the embodiments of the present application.
In order to solve the above technical problems in the prior art, embodiments of the present application provide a scheme for controlling a camera, which does not rely on an external sensor as an input of camera calibration, but relies on a satellite map for camera calibration and linkage. According to the scheme, firstly, a camera pose estimation model with six degrees of freedom is determined through acquired calibration data, a mapping relation between a satellite map and a control angle of a camera cloud deck is established and calibrated, and then the rotation angle of the camera cloud deck and the zoom ratio of a camera lens are calculated based on the position of an object to be observed on the satellite map, so that the camera linkage is controlled in real time. According to the scheme, the camera pose modeling with six degrees of freedom can be completed only by acquiring less calibration data without depending on an external sensor, and the calibration efficiency and the control precision are improved.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a schematic diagram of an application scenario of a further embodiment of the present application. As shown in FIG. 2, the computing device 100 may acquire calibration data via the camera 200 while acquiring satellite map data via a satellite positioning system. The computing device 100 may determine a six-degree-of-freedom camera pose estimation model for establishing and calibrating a mapping relationship between a satellite map and a control angle of the camera according to the calibration data and the satellite map data, and then calculate a rotation angle of a camera pan-tilt and a zoom ratio of a camera lens based on a position of an object to be observed on the satellite map, thereby controlling camera linkage in real time.
The embodiment of the present application does not limit the type of the satellite Positioning System providing the satellite map, for example, the satellite map may be from a beidou Positioning System, an Automatic Identification System (AIS) Global Positioning System (GPS), and the like.
It should be noted that the camera 200 in the embodiment of the present application may include a ball machine, a half-ball machine, or any other type of camera with a tripod head.
Fig. 3 is a schematic structural diagram of the inside of the computing device 100 according to an embodiment of the present application. As shown in fig. 3, the computing device 100 includes: the calibration model calculation system comprises a calibration data acquisition module 110, a calibration model solving module 120, an angle calculation module 130 and a multiplying power calculation module 140. The input information to the computing device 100 includes satellite map data and calibration data collected by the camera 100. The output information of the computing device 100 includes PTZ control parameters of the camera. The PTZ control parameters described above may include angle parameters (e.g., pan and tilt) for controlling a camera pan-tilt and magnification parameters (zoom) of a camera lens.
The scheme for controlling the camera mainly comprises two parts, namely camera calibration and camera linkage. Wherein the input to the camera calibration section includes the PTZ values of the cameras and the satellite map data. The core module of camera calibration comprises a calibration data acquisition module 110 and a calibration model solving module 120, and the output of the camera calibration part is pose parameters.
The input of the camera linkage part is coordinate information and pose parameters of an object to be observed on a satellite map, and the core module comprises an angle calculation module 130 and a multiplying power calculation module 140. The satellite map coordinates are selected as input, so that stable and convenient acquisition can be realized. As an example, the acquisition mode of the satellite map coordinates includes vehicle GPS information, ship AIS data, or selecting a certain point pixel coordinate in a screenshot of the satellite map and converting the certain point pixel coordinate into longitude and latitude. The final output of the computing device 100 is the PTZ control parameter of the camera, and the angle and magnification control operation of the camera can be completed through a preset camera control protocol.
The functions of the modules in the computing device 100 will be described in detail below in conjunction with fig. 3.
A1. Calibration data acquisition module 110
The calibration data is used for acquiring calibration data required by camera calibration. Input information is provided for the subsequent calibration model solution module 120. Each camera can acquire multiple groups of calibration data, and each group of calibration data comprises the PTZ value of the camera corresponding to the characteristic point and the coordinate information of the characteristic point on the satellite map. Wherein the camera PTZ value comprises a pan value and a tilt value of the camera. As an example, each camera may capture at least 3 sets and more than 3 sets of data.
As an example, the zoom factor (zoom) value of the camera may be set to 1, and the angle of the camera 100 may be adjusted until a feature point in the display image where a specific location can be found in the satellite map appears. For example, the feature point may be a marker, a building edge, or the like, and the feature point is above the ground level. After the feature point is found, in order to calibrate the data more accurately, the zoom value of the camera may be enlarged (for example, enlarged by 10 times or more), the angle of the camera pan/tilt head may be finely adjusted until the feature point is located at the center of the displayed image, and the pan and tilt values of the camera pan/tilt head at that time and the coordinates (x, y) of the feature point on the satellite map are captured and recorded.
Where the coordinates (x, y) of the satellite map may refer to coordinates in a world coordinate system, in some examples, the coordinates (x, y) of the satellite map may be latitude and longitude coordinates.
A2. Calibration model solving module 120
The calibration model solving module 120 is configured to establish a six-degree-of-freedom camera pose estimation model, and perform optimal solution according to multiple sets of calibration data acquired by the calibration data acquisition module 110 to obtain pose parameters.
Fig. 4 and 5 are scene schematic diagrams of a camera pose according to an embodiment of the present application. Fig. 4 is a schematic diagram of the camera pan/tilt head not rotating, and fig. 5 is a schematic diagram of the camera pan/tilt head rotating. Wherein the camera rotation angle is denoted (pan, tilt).
Referring to fig. 4 and 5, the calibration model solving module 120 is to obtain pose parameters transformed from the body coordinate system of the camera to the world coordinate system, and includes: body coordinate system origin O of camera b Relative to the origin O of the world coordinate system w Translation vector (x) 0 ，y 0 ，z 0 ) And the angular vector (roll) of the three-axis rotation of the body coordinate system of the camera with respect to the world coordinate system 0 ，pitch 0 ，yaw 0 ). Wherein, the roll 0 Is referred to as winding X w Angle of rotation of the shaft, pitch 0 Is referred to as winding Y w Angle of rotation of the shaft, yaw 0 Is referred to as winding Z w The angle of rotation of the shaft.
The calibration model solving module 120 has two functions, the first part is to establish a camera pose estimation model, and the second part is to perform optimization solution on the model, which will be described in the following.
1) Establishing a camera pose estimation model
Before establishing a camera pose estimation model, different modeling schemes can be selected according to whether the camera is inclined or not during installation.
a) The camera does not have tilt condition when being installed
Continuing to refer to FIG. 4, FIG. 4 shows a schematic view of a camera without tilt during installation, and assuming that the camera does not tilt during installation, i.e., the camera does not tilt in both the front-back direction and the up-down direction, a roll 0 And pitch 0 To be 0, only x needs to be obtained when solving the model 0 ，y 0 ，z 0 ，yaw 0 The camera pose estimation model can be established according to the equation (1) and the equation (2).
x＝x 0 +z 0 *tan(tilt)*cos(yaw 0 +pan)； (1)
y＝y 0 +z 0 *tan(tilt)*sin(yaw 0 +pan)； (2)
Wherein (x, y) represents the coordinates of the feature point in the world coordinate system, and (tilt, pan) represents the pan value and tilt value of the corresponding camera when the feature point is located at the center of the camera image.
b) The camera has tilt condition when being installed
Roll if there is a tilt condition when the camera is installed 0 And pitch 0 Not zero, the current body coordinate system may be rotated to a body coordinate system without tilt using a coordinate system transformation method, and specifically, the rotation angle with respect to the body coordinate system without tilt may be obtained according to equation (3):
q(roll',tilt',pan')＝q(roll 0 ,pitch 0 ,yaw 0 )*q(0,tilt,pan)； (3)
wherein (roll ', tilt ', pan ') represents a rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the feature point is located at the center of the camera image, (roll 0 ,pitch 0 ,yaw 0 ) And (2) an angle vector representing the three-axis rotation of the body coordinate system of the inclined camera relative to the world coordinate system, wherein (0, tilt, pan) represents the pan value and tilt value of the corresponding camera when the characteristic point is positioned at the center of the camera image, and also represents the rotation angle of the body coordinate system of the camera relative to the inclined camera coordinate system when the characteristic point is positioned at the center of the camera image.
q (m, n, l) represents a quaternion (w, x, y, z) constructed by the three axis rotation angles, which can be represented by equation (4).
After the rotator coordinate system, tilt 'pan' obtained by formula (3) may be substituted into formulas (5) and (6) to build a camera pose estimation model.
x＝x 0 +z 0 *tan(tilt')*cos(pan')； (5)
y＝y 0 +z 0 *tan(tilt')*sin(pan')； (6)
2) Model optimization solution
After the camera pose estimation model is established, the camera pose estimation model can be utilized to solve to obtain positionsPosture parameter (x) 0 ，y 0 ，z 0 ) And (roll) 0 ，pitch 0 ，yaw 0 ). The model solving method is not limited in the embodiment of the present application. For example, the camera pose estimation model can be solved by a nonlinear equation set in a mode of interval iteration or fixed point method.
In some examples, a function to be optimized for a least square method may be constructed using a camera pose estimation model and solved by a gauss-newton iteration method.
For example, for the case where the camera is not tilted at the time of installation, i.e., the model established according to equations (1) and (2), the following equations (7) and (8) may be employed to solve the model.
For another example, for the case where the camera is tilted at the time of installation, i.e., the model established according to equations (5) and (6), the following equations (9) and (10) may be used to solve the model.
e 3 ＝∑(x-x 0 -z 0 *tan(tilt')*cos(pan')) 2 +(y-y 0 -z 0 *tan(tilt')*sin(pan')) 2 ； (9)
For another example, for a scene in which only the pan value of the camera is changed, i.e., a scene in which the camera moves only in the left-right direction, only x needs to be calculated 0 、y 0 And yaw 0 The three degree-of-freedom parameters, whether the camera is tilted or not at the time of installation, can be used to solve the model by the following equation (11).
Wherein, (x, y) represents the coordinates of the feature points in the world coordinate system, (tilt, pan) represents the pan value and tilt value of the camera corresponding to the feature points; (tilt ', pan') represents the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the feature point is located at the center of the camera image.
Wherein the function e 1 、e 3 The method is suitable for scenes with high camera installation positions and high tilt degrees during data acquisition. As an example, it may be applicable to scenes where the tilt degree is greater than 1 degree. Function e 2 、e 4 The method is suitable for scenes with low camera installation positions or serious inclination or small tilt values during data acquisition, and for example, the method can be suitable for scenes with more tilt values smaller than 1 degree. Function e 5 The method is suitable for a scene that the installation position of the dome camera is very low and all tilt values are about 0 degree during data acquisition, or the tilt values are fixed, and only a plane scene of a pan value needs to be controlled or the tilt values are used as e 1 、e 3 、e 3 、e 4 And an initial value acquisition method of the function.
A3. Angle calculation module 130
Used for acquiring the coordinate (x) of an object to be observed in a world coordinate system w ,y w ) And outputs pan and tilt angle parameters for controlling the camera pan/tilt by using the camera pose estimation model and pose parameters obtained by the calibration model solving module 120.
As an example, the pan value and the tilt value of controlling the camera pan head may be acquired according to the following formulas (12) to (14).
q(roll w ',tilt w ,pan w )＝q -1 (roll 0 ,pitch 0 ,yaw 0 )*q(0,tilt w ',pan w ')； (14)
Wherein (x) w ,y w ) Representing the coordinates of the target to be observed in a world coordinate system; (tilt) w ,pan w ) The pan value and the tilt value of the camera corresponding to the target to be observed are represented, namely the pan value and the tilt value of the camera when the target to be observed is positioned in the center of the camera image; (roll) w ',tilt w ',pan w ') indicates the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the object to be observed is located at the center of the camera image.
A4. Multiplying power calculating module 140
The camera position (x) is used for acquiring the coordinates (x, y) of the target to be observed and solving according to the coordinates (x, y) and the calibration 0 ，y 0 ) The zoom coefficient zoom of the control camera 200 is calculated.
As an example, the zoom coefficient zoom of the control camera 200 may be acquired according to the following formula (15).
Wherein (x) w ,y w ) The coordinate of the target to be observed in the world coordinate system is represented by s, the scale of the satellite map is represented by s, the coordinate system used by the satellite map is converted into metric scale, and k represents an empirical coefficient of magnification, and the empirical coefficient can be adjusted according to the size of the target to be observed. For example, one may default to a magnification of one time per 100 meters distance.
Finally, the computing device 100 may map the PTZ control parameters (pan) of the camera according to a preset camera control protocol w 、tilt w Zoom) to control the camera 200 to perform functions such as steering and zooming, thereby realizing camera linkage of the target to be observed.
In the embodiment of the application, a scheme for controlling the camera is provided, the scheme does not depend on an external sensor as the input of camera calibration, but depends on a satellite map to calibrate and link the camera, the cost is low, the transverse and longitudinal full-coverage angle control of the camera can be ensured, and the variable magnification control of the maximum magnification range is realized according to the distance between the target to be observed and the camera. The method fully considers the installation state of the camera, carries out modeling of six-degree-of-freedom pose, can finish the calibration process by acquiring a small amount of calibration data, for example, only 3 sets of calibration data are needed at least, and improves the calibration efficiency and the control precision of the camera.
In addition, various optimization functions applied to the calibration process are designed in the embodiment of the application, the camera erection scenes with different heights and postures can be flexibly adapted, and the calibration efficiency and the application flexibility are improved.
Fig. 6 is a flowchart illustrating a method for controlling a camera according to an embodiment of the present application. The method may be performed by the computing device of fig. 1-3. As shown in fig. 6, the method includes the following.
S601, acquiring multiple groups of calibration data, wherein each group of calibration data comprises: PTZ data and satellite map data that the camera output, PTZ data include pan value and tilt value of the camera that the characteristic point corresponds to, satellite map data include the coordinate of characteristic point in the world coordinate system.
As an example, each camera may capture at least 3 sets and more than 3 sets of data.
As an example, the zoom factor (zoom) value of the camera may be set to 1, and the angle of the camera 100 may be adjusted until a feature point in the display image where a specific location can be found in the satellite map appears. For example, the feature point may be a marker, a building edge, or the like, and the feature point is above the ground level. After the feature point is found, in order to calibrate the data more accurately, the zoom value of the camera may be enlarged (for example, enlarged by 10 times or more), the angle of the camera pan/tilt head may be finely adjusted until the feature point is located at the center of the displayed image, and the pan and tilt values of the camera pan/tilt head at that time and the coordinates (x, y) of the feature point on the satellite map are captured and recorded.
Where the coordinates (x, y) of the satellite map may refer to coordinates in a world coordinate system, in some examples, the coordinates (x, y) of the satellite map may be latitude and longitude coordinates.
And S602, solving the camera pose estimation model according to the multiple groups of calibration data to acquire pose parameters of six degrees of freedom of the camera.
The camera pose estimation model is used for acquiring pose parameters converted from a body coordinate system of the camera to a world coordinate system, and the pose parameters comprise an origin O of the body coordinate system of the camera b Relative to the origin O of the world coordinate system w Translation vector (x) 0 ，y 0 ，z 0 ) And the angular vector (roll) of the three-axis rotation of the body coordinate system of the camera with respect to the world coordinate system 0 ，pitch 0 ，yaw 0 )。
For a specific principle of establishing and solving the camera pose estimation model, reference may be made to the relevant descriptions in fig. 3 to fig. 5, and details are not repeated here for brevity.
Alternatively, in the S602 section, if the installation of the camera does not produce a tilt, the camera pose estimation model satisfies the following condition:
x＝x 0 +z 0 *tan(tilt)*cos(yaw 0 +pan)； (1)
y＝y 0 +z 0 *tan(tilt)*sin(yaw 0 +pan)； (2)
roll 0 ＝0；pitch 0 ＝0；
wherein, (x, y) represents the coordinates of the feature point in the world coordinate system, (tilt, pan) represents the pan value and tilt value of the corresponding camera when the feature point is located at the center of the camera image;
if the camera is installed to generate inclination, the camera pose estimation model meets the following conditions:
x＝x 0 +z 0 *tan(tilt')*cos(pan')； (5)
y＝y 0 +z 0 *tan(tilt')*sin(pan')； (6)
where (tilt ', pan') represents the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the feature point is located at the center of the camera image.
Alternatively, (tilt ', pan') meets the following condition:
q(roll',tilt',pan')＝q(roll 0 ,pitch 0 ,yaw 0 )*q(0,tilt,pan)； (3)
wherein (roll ', tilt ', pa) n ' represents a rotation angle of the body coordinate system of the camera relative to the world coordinate system when the feature point is located at the center of the camera image, (roll) 0 ,pitc 0 h,ya 0 w) angle vector (roll) representing the three-axis rotation of the body coordinate system of the tilted camera relative to the world coordinate system 0 ，pitch 0 ，yaw 0 )；
q (m, n, l) represents a quaternion (w, x, y, z) constructed from the three axes of rotation, which satisfies the following condition:
in the S602 section, the embodiment of the present application does not limit the manner of model solution. For example, the camera pose estimation model can be solved by a nonlinear equation set in a mode of interval iteration or fixed point method. Or, a function to be optimized of the least square method is constructed by using the camera pose estimation model, and the pose parameter is obtained by solving through an iteration method.
As an example, if the camera is not installed with a tilt, the function to be optimized of the least square method is constructed by using the following formula to solve:
as an example, if the installation of the camera generates inclination, the function to be optimized of the least square method is constructed by using the following formula to solve:
e 3 ＝∑(x-x 0 -z 0 *tan(tilt')*cos(pan')) 2 +(y-y 0 -z 0 *tan(tilt')*sin(pan')) 2 ； (9)
as an example, if the camera only controls the pan value change, the function to be optimized of the least square method is constructed using the following formula to solve:
wherein, (x, y) represents the coordinates of the feature points in the world coordinate system, (tilt, pan) represents the pan value and tilt value of the camera corresponding to the feature points; (tilt ', pan') represents the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the feature point is located at the center of the camera image.
Wherein the function e 1 、e 3 The method is suitable for scenes with high camera installation positions and high tilt degrees during data acquisition. As an example, it may be applicable to scenes where the tilt degree is greater than 1 degree. Function e 2 、e 4 The method is suitable for scenes with low camera installation positions or serious inclination or small tilt values during data acquisition, and for example, the method can be suitable for scenes with more tilt values smaller than 1 degree. Function e 5 The method is suitable for a scene that the installation position of the dome camera is very low and all tilt values are about 0 degree during data acquisition, or the tilt values are fixed, and only a plane scene of a pan value needs to be controlled or the tilt values are used as e 1 、e 3 、e 3 、e 4 And an initial value acquisition method of the function.
In the embodiment of the application, various optimization functions applied to the calibration process are designed, so that the camera erection scenes with different heights and postures can be flexibly adapted, and the calibration efficiency and the application flexibility are improved.
And S603, acquiring PTZ control parameters of the camera according to the coordinates and the pose parameters of the target to be observed in the world coordinate system.
Optionally, the PTZ control parameters of the camera include at least one of: and the pan value, the tilt value and the zoom value of the camera corresponding to the target to be observed.
Optionally, in S603, acquiring a PTZ control parameter of the camera according to the coordinate and pose parameter of the target to be observed in the world coordinate system, where the PTZ control parameter includes:
acquiring a pan value and a tilt value of a camera corresponding to a target to be observed according to the following formula:
q(roll w ',tilt w ,pan w )＝q -1 (roll 0 ,pitch 0 ,yaw 0 )*q(0,tilt w ',pan w ')； (14)
wherein (x) w ,y w ) The coordinate (tilt) of the target to be observed in the world coordinate system w ,pan w ) Representing the pan value and the tilt value of a camera corresponding to the target to be observed; (roll) w ',tilt w ',pan w ') indicates the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the object to be observed is located at the center of the camera image.
And acquiring a zoom value of a camera corresponding to the target to be observed according to the following formula:
wherein, zoom represents zoom value of camera corresponding to target to be observed, (x) w ,y w ) And the coordinate of the target to be observed in the world coordinate system is shown, s is a scale of the satellite map, and k is an empirical coefficient of magnification.
And S604, outputting the PTZ control parameters of the camera.
Finally, the computing device may adjust the PTZ control parameter (pan) of the camera according to a preset camera control protocol w 、tilt w Zoom) to control the camera to perform functions such as turning, zooming, etc., thereby realizing camera linkage of the target to be observed.
In the embodiment of the application, a scheme for controlling the camera is provided, the scheme does not depend on an external sensor as the input of camera calibration, but depends on a satellite map to calibrate and link the camera, the cost is low, the transverse and longitudinal full-coverage angle control of the camera can be guaranteed, and the variable magnification control of the maximum magnification range is realized according to the distance between the target to be observed and the camera. The method and the device fully consider the installation state of the camera, carry out modeling of the pose with six degrees of freedom, and can finish the calibration process by acquiring a small amount of calibration data, thereby improving the calibration efficiency and the control precision of the camera.
Fig. 7 is a schematic structural diagram of an apparatus 700 according to an embodiment of the present application. The apparatus 700 is configured to perform the method described above as being performed by the computing device 100.
The apparatus 700 includes a communication module 710 and a processing module 720. The apparatus 700 is configured to implement the operations performed by the computing device 100 in the above method embodiments.
For example, the communication module 710 is configured to obtain a plurality of sets of calibration data, each set of calibration data including: PTZ data and satellite map data output by the camera, wherein the PTZ data comprises a pan value and a tilt value of the camera corresponding to the characteristic point, and the satellite map data comprises coordinates of the characteristic point in a world coordinate system; the processing module 720 is configured to solve the camera pose estimation model according to the multiple sets of calibration data to obtain pose parameters of six degrees of freedom of the camera; the processing module 720 is further configured to obtain a PTZ control parameter of the camera according to the coordinate and pose parameter of the target to be observed in the world coordinate system; the communication module 710 is also used to output PTZ control parameters for the camera.
Fig. 8 is a schematic structural diagram of an apparatus 800 according to an embodiment of the present application. The apparatus 800 is configured to perform the method described above as being performed by the computing device 100.
The apparatus 800 includes a processor 810, and the processor 810 is configured to execute the computer program or instructions stored in the memory 820 or read data stored in the memory 820 to perform the method in the above embodiments of the method. Optionally, the processor 810 is one or more.
Optionally, as shown in fig. 8, the apparatus 800 further comprises a memory 820, the memory 820 being used for storing computer programs or instructions and/or data. The memory 820 may be integrated with the processor 810 or may be provided separately. Optionally, the memory 820 is one or more.
Optionally, as shown in fig. 8, the apparatus 800 further includes a communication interface 830, and the communication interface 830 is used for receiving and/or transmitting signals. For example, processor 810 may be used to control the reception and/or transmission of signals by communication interface 830.
Optionally, the apparatus 800 is configured to implement the operations performed by the computing device 100 in the above method embodiments.
For example, processor 810 is configured to execute computer programs or instructions stored by memory 820 to implement the operations associated with computing device 100 of the various method embodiments described above.
It should be noted that the apparatus 800 in fig. 8 may be the computing device 100 in the foregoing embodiment, or may be a component (e.g., a chip) of the computing device 100, and is not limited herein.
In the embodiment of the present application, the processor is a circuit having a signal processing capability, and in one implementation, the processor may be a circuit having an instruction reading and executing capability, such as a CPU, a microprocessor, a GPU (which may be understood as a kind of microprocessor), or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of hardware circuits, which may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In the reconfigurable hardware circuit, the process of loading the configuration document by the processor to implement the configuration of the hardware circuit may be understood as a process of loading instructions by the processor to implement the functions of some or all of the above units. Furthermore, it may also be a hardware circuit designed for artificial intelligence, which may be understood as an ASIC, such as an NPU, TPU, DPU, etc.
It is seen that the units in the above apparatus may be one or more processors (or processing circuits) configured to implement the above method, for example: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
In addition, all or part of the units in the above apparatus may be integrated together, or may be implemented independently. In one implementation, these units are integrated together, implemented in the form of a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any one of the above methods or implementing functions of the units of the apparatus, and the at least one processor may be of different types, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, and the like.
Accordingly, the present application also provides a computer readable storage medium storing a computer program, which when executed by a processor causes the processor to implement the steps of the method performed by the computing device 100 in fig. 2 to 6.
Accordingly, embodiments of the present application also provide a computer program product, which includes computer programs/instructions, when executed by a processor, cause the processor to implement the steps in the methods performed by the computing device 100 in fig. 2 to 6.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the same element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A method for controlling a camera, comprising:
acquiring multiple groups of calibration data of the camera, wherein each group of calibration data comprises: PTZ data and satellite map data output by the camera, wherein the PTZ data comprises a pan value and a tilt value of the camera corresponding to the characteristic point, and the satellite map data comprises coordinates of the characteristic point in a world coordinate system;
solving a camera pose estimation model according to the multiple groups of calibration data to acquire pose parameters of six degrees of freedom of the camera;
acquiring PTZ control parameters of the camera according to the coordinates of the target to be observed in the world coordinate system and the pose parameters;
outputting PTZ control parameters of the camera.
2. The method of claim 1, wherein the camera pose model is used to obtain the pose parameters for a transformation of a body coordinate system of a camera to a world coordinate system, the pose parameters including an origin O of the body coordinate system of the camera b Relative to the origin O of the world coordinate system w Translation vector (x) 0 ，y 0 ，z 0 ) And a three-axis rotational angle vector (roll) of the body coordinate system of the camera relative to a world coordinate system 0 ，pitch 0 ，yaw 0 )。
3. The method according to claim 2, wherein if the installation of the camera does not produce a tilt, the camera pose estimation model satisfies the following condition:
x＝x 0 +z 0 *tan(tilt)*cos(yaw 0 +pan)；
y＝y 0 +z 0 *tan(tilt)*sin(yaw 0 +pan)；
roll 0 ＝0；pitch 0 ＝0；
wherein (x, y) represents the coordinates of the feature point in the world coordinate system, and (tilt, pan) represents the pan value and tilt value of the corresponding camera when the feature point is located in the center of the camera image;
if the camera is installed to generate inclination, the camera pose estimation model meets the following conditions:
x＝x 0 +z 0 *tan(tilt')*cos(pan')；
y＝y 0 +z 0 *tan(tilt')*sin(pan')；
where (tilt ', pan') represents the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the feature point is located at the center of the camera image.
4. The method of claim 1 or 2, wherein solving the camera pose estimation model to obtain pose parameters for six degrees of freedom of the camera based on the plurality of sets of calibration data comprises:
and constructing a function to be optimized by using the camera pose estimation model, and solving by using an iteration method to obtain the pose parameters.
5. An apparatus for controlling a camera, comprising:
the communication module is used for acquiring multiple groups of calibration data of the camera, and each group of calibration data comprises: PTZ data and satellite map data output by the camera, wherein the PTZ data comprises a pan value and a tilt value of the camera corresponding to the characteristic point, and the satellite map data comprises coordinates of the characteristic point in a world coordinate system;
the processing module is used for solving the camera pose estimation model according to the multiple groups of calibration data so as to acquire pose parameters of six degrees of freedom of the camera;
the processing module is further used for acquiring PTZ control parameters of the camera according to the coordinates of the target to be observed in the world coordinate system and the pose parameters;
the communication module is also for outputting PTZ control parameters for the camera.
6. The apparatus of claim 5, wherein the camera pose model is to obtain the pose parameters for a transformation of a body coordinate system of a camera to a world coordinate system, the pose parameters comprising a body coordinate system origin O of the camera b Relative to the origin O of the world coordinate system w Translation vector (x) 0 ，y 0 ，z 0 ) And the angular vector (roll) of the three-axis rotation of the body coordinate system of the camera with respect to the world coordinate system 0 ，pitch 0 ，yaw 0 )。
7. The apparatus of claim 6, wherein if the installation of the camera does not produce a tilt, the camera pose estimation model satisfies the following condition:
x＝x 0 +z 0 *tan(tilt)*cos(yaw 0 +pan)；
y＝y 0 +z 0 *tan(tilt)*sin(yaw 0 +pan)；
roll 0 ＝0；pitch 0 ＝0；
wherein, (x, y) represents the coordinates of the feature point in the world coordinate system, (tilt, pan) represents the pan value and tilt value of the corresponding camera when the feature point is located at the center of the camera image;
if the camera is installed to generate inclination, the camera pose estimation model meets the following conditions:
x＝x 0 +z 0 *tan(tilt')*cos(pan')；
y＝y 0 +z 0 *tan(tilt')*sin(pan')；
where (tilt ', pan') represents the rotation angle of the body coordinate system of the camera with respect to the world coordinate system when the feature point is located at the center of the camera image.
8. The apparatus of claim 5 or 6, wherein in the solving of the camera pose estimation model to obtain the pose parameters for the six degrees of freedom of the camera based on the plurality of sets of calibration data, the processing module is specifically configured to: and constructing a function to be optimized by using the camera pose estimation model, and solving by using an iteration method to obtain the pose parameters.
9. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored by the memory to implement the method of any of claims 1 to 4.
10. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, are configured to implement the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210302903.XA CN114862959A (en) | 2022-03-24 | 2022-03-24 | Method and apparatus for controlling camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210302903.XA CN114862959A (en) | 2022-03-24 | 2022-03-24 | Method and apparatus for controlling camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114862959A true CN114862959A (en) | 2022-08-05 |
Family
ID=82629453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210302903.XA Pending CN114862959A (en) | 2022-03-24 | 2022-03-24 | Method and apparatus for controlling camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114862959A (en) |
-
2022
- 2022-03-24 CN CN202210302903.XA patent/CN114862959A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110310248B (en) | A kind of real-time joining method of unmanned aerial vehicle remote sensing images and system | |
| US10594941B2 (en) | Method and device of image processing and camera | |
| US7961980B2 (en) | Method for providing output image in either cylindrical mode or perspective mode | |
| CN111461994A (en) | Method for obtaining coordinate transformation matrix and positioning target in monitoring picture | |
| CN109523471B (en) | Method, system and device for converting ground coordinates and wide-angle camera picture coordinates | |
| KR20090078463A (en) | Distorted image correction apparatus and method | |
| CN105324791A (en) | Coordinate computation device and method, and image processing device and method | |
| CN108154558A (en) | A kind of augmented reality methods, devices and systems | |
| CN115841487B (en) | Hidden danger positioning method and terminal along power transmission line | |
| JP6804806B2 (en) | Positioning accuracy information calculation device and positioning accuracy information calculation method | |
| CN111693025A (en) | Remote sensing image data generation method, system and equipment | |
| CN110555813A (en) | rapid geometric correction method and system for remote sensing image of unmanned aerial vehicle | |
| CN114549666B (en) | AGV-based panoramic image splicing calibration method | |
| CN115439528B (en) | Method and equipment for acquiring image position information of target object | |
| CN111429529B (en) | Coordinate conversion calibration method, electronic equipment and computer storage medium | |
| CN113415433A (en) | Pod attitude correction method and device based on three-dimensional scene model and unmanned aerial vehicle | |
| CN116228888B (en) | Conversion method and system for geographic coordinates and PTZ camera coordinates | |
| CN116580097A (en) | Coordinate conversion method and device for video GIS | |
| CN115423863B (en) | Camera pose estimation method and device and computer readable storage medium | |
| CN113654528B (en) | Method and system for estimating target coordinates through unmanned aerial vehicle position and cradle head angle | |
| CN109377529A (en) | A kind of picture coordinate transformation method, system and the device of ground coordinate and Pan/Tilt/Zoom camera | |
| CN114862959A (en) | Method and apparatus for controlling camera | |
| CN115565092A (en) | Method and equipment for acquiring geographical position information of target object | |
| US11415990B2 (en) | Optical object tracking on focal plane with dynamic focal length | |
| CN114670981A (en) | Method and apparatus for controlling camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |