CN113538596B - Moving target tracking system based on trinocular vision - Google Patents

Moving target tracking system based on trinocular vision Download PDF

Info

Publication number
CN113538596B
CN113538596B CN202110800602.5A CN202110800602A CN113538596B CN 113538596 B CN113538596 B CN 113538596B CN 202110800602 A CN202110800602 A CN 202110800602A CN 113538596 B CN113538596 B CN 113538596B
Authority
CN
China
Prior art keywords
ptz
calibration
tracking
camera
executing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110800602.5A
Other languages
Chinese (zh)
Other versions
CN113538596A (en
Inventor
姜柯
李爱华
蔡艳平
王涛
苏延召
韩德帅
冯国彦
李庆辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rocket Force University of Engineering of PLA
Original Assignee
Rocket Force University of Engineering of PLA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rocket Force University of Engineering of PLA filed Critical Rocket Force University of Engineering of PLA
Priority to CN202110800602.5A priority Critical patent/CN113538596B/en
Publication of CN113538596A publication Critical patent/CN113538596A/en
Application granted granted Critical
Publication of CN113538596B publication Critical patent/CN113538596B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85Stereo camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20164Salient point detection; Corner detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30232Surveillance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30244Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a moving target tracking system based on binocular vision, which designs a binocular vision system of a gun camera and two PTZ ball machines, and compared with the existing binocular vision system, the moving target tracking system improves the types of data acquisition, thereby improving the accuracy of target tracking; the camera parameter calibration method assisted by the multi-degree-of-freedom mechanical arm is provided, the internal parameter calibration of a gun camera, the internal parameter calibration of a PTZ ball machine and the external parameter calibration among three cameras are realized, and compared with the existing method, the calibration speed and accuracy are improved.

Description

Moving target tracking system based on trinocular vision
Technical Field
The invention relates to a target tracking system, in particular to a moving target tracking system based on trinocular vision.
Background
In the key important parts of some military or civil facilities, such as airport docks, oil depots, chemical plants and the like, people have higher requirements on the performance of intelligent video monitoring.
In the prior art, a binocular vision method is generally adopted, a binocular vision system realized by combining a gun camera or an omnidirectional camera with a PTZ (pan/tilt/zoom) dome camera is adopted, a moving target is detected by using the gun camera or the omnidirectional camera, and tracking and amplified snapshot are realized by using the PTZ dome camera.
In a binocular vision system in the prior art, two cameras with too large focal length difference are generally adopted (a PTZ dome camera can enable the focal length to be far larger than that of a static camera during amplification tracking), a large error can be generated during stereo correction, and the difference of an effective size of an image pair obtained through correction is large, namely, the image of the PTZ dome camera can only correspond to a small area of a gunlock image, so that the loss of target details is serious, and the loss of a tracked target is caused.
In conclusion, the existing target tracking system based on binocular vision has the problem that the tracking result is inaccurate.
Disclosure of Invention
The invention aims to provide a moving target tracking system based on binocular vision, which is used for solving the problem that a tracking result of a target tracking system based on binocular vision in the prior art is inaccurate.
In order to realize the task, the invention adopts the following technical scheme:
a moving target tracking system based on trinocular vision comprises a trinocular vision module, a camera calibration module and a moving target tracking module;
the binocular vision module is used for acquiring images containing moving targets, and comprises a gun camera and two PTZ ball machines, wherein the gun camera and the PTZ ball machines are arranged on the same horizontal line, and the gun camera is positioned in the middle of the two PTZ ball machines; the optical axis of the PTZ dome camera at zero position is the same as the direction of the gunlock;
the camera calibration module is used for calibrating the trinocular vision module and comprises a calibration platform and a calibration server;
the calibration platform is used for driving the calibration plate to be at the position of the multi-group template;
the calibration server is used for controlling the calibration platform to a specified position and calibrating the gunlock and the PTZ ball machine by adopting a calibration algorithm;
the moving object tracking module has a first computer program stored therein, which when executed by a processor implements the steps of:
step a, controlling one gun (3) and two PTZ ball machines (2) in the trinocular vision module to respectively start tracking moving targets;
b, judging whether the speed of the moving target is greater than a speed threshold value or not according to the result of tracking the moving target by the gunlock (3), if so, executing the step d, otherwise, executing the step c;
step c, judging whether the rotation angle of the PTZ ball machine (2) is larger than an angle threshold, if so, executing the step d, otherwise, executing the step e;
d, setting a PTZ ball machine (2) to perform primary tracking by adopting a magnification x, and then executing a step e, wherein x is a positive integer;
e, judging whether the primary tracking is stable, if so, executing the step f, otherwise, returning to the step d;
step f, setting a PTZ ball machine (2) to perform standard tracking by adopting a magnification factor y, wherein y is a positive integer and is larger than x;
and g, completing tracking.
Further, the calibration server has a second computer program stored therein, and the second computer program, when executed by the processor, implements the following steps:
step 1, judging that the current calibration object is a gunlock or a PTZ ball machine, and if the current calibration object is the gunlock, executing step 2; if the ball machine is a PTZ ball machine, executing the step 3;
step 2, calibrating the bolt, which specifically comprises the following steps:
step 2.1, acquiring N groups of template positions, wherein N is a positive integer;
step 2.2, controlling the calibration platform to the position of the ith group of templates, wherein i =1,2, \8230; N;
step 2.3, controlling the lens to shoot an image containing the calibration plate;
step 2.4, carrying out corner point detection on the image obtained in the step 2.3;
step 2.5, judging whether the result of the angular point detection in the step 2.4 is accurate, if not, returning to the step 2.4, and if so, executing the step 2.6;
step 2.6, letting i = i +1, returning to step 2.2, and obtaining the internal parameters of the lens until i = N;
step 3, calibrating the PTZ dome camera, which specifically comprises the following steps:
step 3.1, obtaining the maximum value M of the to-be-calibrated magnification of the PTZ dome camera, wherein M is a positive integer greater than or equal to 1;
step 3.2, setting the magnification factor k =1 of the PTZ dome camera;
step 3.3, calibrating the PTZ dome camera under the magnification k by adopting the method of the step 2.1-2.6, and executing step 3.4 after obtaining the internal parameters of the PTZ dome camera under the current magnification k if successful; otherwise, returning to the step 3.3;
step 3.4, enabling k = k +1 until k = M, and obtaining internal parameters of the current PTZ dome camera under all magnification factors;
step 4, whether internal parameters of one gun camera and two PTZ ball machines are obtained or not is judged, and if yes, the step 5 is executed; otherwise, returning to the step 1;
and 5, obtaining the relative position relation between one gun camera and two PTZ ball machines in the trinocular vision module, and obtaining the external parameters of the trinocular vision module.
Further, in step 2.6, the inside parameters of the lens are obtained by adopting a Zhang-Yongyou calibration algorithm.
Compared with the prior art, the invention has the following technical effects:
1. the moving target tracking system based on the binocular vision provided by the invention designs a binocular vision system of a gun camera and two PTZ ball machines, and compared with the existing binocular vision system, the type of data acquisition is improved, so that the accuracy of target tracking is improved;
2. the moving target tracking system based on the trinocular vision, which is provided by the invention, provides a camera parameter calibration method assisted by a multi-degree-of-freedom mechanical arm, so that the internal parameter calibration of a gun camera, the internal parameter calibration of a PTZ ball machine and the external parameter calibration among three cameras are realized, and compared with the existing method, the speed and the accuracy of the calibration are improved;
3. the moving target tracking system based on the trinocular vision provided by the invention provides a secondary tracking strategy aiming at two conditions of quick target movement and large PTZ rotation angle, and the method provided by the invention has higher accuracy.
Drawings
Fig. 1 is a schematic structural diagram of a trinocular vision module provided by the present invention;
FIG. 2 is a flow chart of a bolt face calibration method provided in an embodiment of the present invention;
FIG. 3 is a flow chart of a PTZ dome camera calibration method provided in an embodiment of the present invention;
FIG. 4 is a diagram illustrating external reference calibration results provided in an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating the tracking result of a high-speed moving target provided in an embodiment of the present invention, where fig. 5 (a) shows the position of the moving target in the gun bolt, fig. 5 (b) shows the position of the moving target in the ball machine 1, and fig. 5 (c) shows the position of the moving target in the ball machine 2;
fig. 6 is a schematic diagram showing the target tracking result of a PTZ dome camera during large-angle rotation according to an embodiment of the present invention, where fig. 6 (a) shows the position of a moving target in a gun camera, fig. 6 (b) shows the position of the moving target in the dome camera 1, and fig. 6 (c) shows the position of the moving target in the dome camera 2;
FIG. 7 is a schematic diagram of a moving object tracking process of the secondary tracking strategy provided by the present invention;
fig. 8 is a comparison graph of tracking error results under different strategies, fig. 8 (a) is a tracking error under a standard tracking strategy, and fig. 8 (b) is a tracking error adopting a secondary tracking strategy provided by the present invention.
The reference numerals in the figures represent: 1-guide rail, 2-PTZ ball machine, and 3-gun machine.
Detailed Description
The present invention will be described in detail below with reference to the drawings and examples. So that those skilled in the art can better understand the present invention. It is to be expressly noted that in the following description, a detailed description of known functions and designs will be omitted when it may obscure the subject matter of the present invention.
The following definitions or conceptual connotations related to the present invention are provided for illustration:
a bolt face: the bolt face is one of the monitoring cameras. The bolt face is cuboid in appearance, and a C/CS lens interface is arranged in front of the bolt face.
PTZ ball machine: a Pan-Tilt-Zoom dome camera is a short hand of Pan/Tilt/Zoom in security monitoring application and represents a dome monitoring camera with omni-directional (left-right/up-down) movement of a holder, zoom of a lens and Zoom control.
Example one
The embodiment discloses a moving target tracking system based on trinocular vision, which comprises a trinocular vision module, a camera calibration module and a moving target tracking module;
the binocular vision module is used for acquiring images containing moving targets, and comprises a gun camera and two PTZ ball machines, wherein the gun camera and the PTZ ball machines are arranged on the same horizontal line, and the gun camera is positioned in the middle of the two PTZ ball machines; the optical axis of the PTZ dome camera at zero position is the same as the direction of the gunlock;
in the embodiment, the camera mainly uses the gunlock, and the speed of the moving target is obtained according to the position of the moving target in the image of the gunlock.
In this embodiment, as shown in fig. 1, an industrial rail 1 having a length of 1.5 m and manufactured by taiwan silver corporation is used as a mounting platform, and the rail can be fixed on a bracket or a wall. The guide rail 1 is provided with four positions A, C and D, wherein the positions C and D are fixedly provided with two PTZ ball machines 2 which are respectively positioned at two ends of the guide rail 1. The point a is located in the center of the guide rail 1 for mounting the bolt face 3. Wherein point a is the position of the bolt when the system is operating normally.
The optical axes of the two PTZ ball machines 2 in the zero positions are in the same direction as the gunlock 3, and the angle difference of the optical axes in the horizontal direction and the vertical direction is zero as much as possible. The PTZ ball machine (2) is arranged to cover 180-degree regions right in front of the system in a P (horizontal) direction (-90 degrees) and a T (vertical) direction (-20-90 degrees).
In the invention, the characteristics that the gunlock parameter A is fixed and the gunlock parameter A is fixed can be utilized to guide the dome camera to complete active PTZ tracking, and the PT rotation angle and the error correction of the focal length parameter of the PTZ dome camera are realized through matching point information;
the C point PTZ dome camera and the D point PTZ dome camera can form a binocular PTZ visual system for estimating the target size.
In this embodiment, the bolt 3 adopts a Haikang DS-2CD854F-E, the bolt is a 300 ten thousand pixel network gun type camera, adopts a 2.8mm wide-angle lens, works at 1920 × 1080 resolution, and can directly output network video data;
the PTZ dome camera 2 adopts a Sony EVI-D90P, has a rotation function of pitching-20 degrees to 90 degrees and horizontally-170 degrees to 170 degrees, has 28 times of optical zooming, and adopts VISCA control protocol and analog signal output;
in this embodiment, a video analysis terminal of a computer and a local area network are used to obtain video data to complete the functions of monitoring, tracking control, size estimation and display output of a target, and the analysis terminal uses a GPU graphics accelerator card (huashuo GTX560Ti DCII) to simultaneously complete the functions of GPU accelerated computation and dual-screen display driving. The system is provided with two displays, wherein one display is used for monitoring the video of the gunlock in real time, and the other display is used for displaying an operation interface for monitoring two PTZ ball machines in real time.
The camera calibration module is used for calibrating the trinocular vision module and comprises a calibration platform and a calibration server;
the calibration platform is used for driving the calibration plate to the position of the multiple groups of templates;
the calibration server is used for controlling the calibration platform to a specified position and calibrating the gunlock and the PTZ ball machine by adopting a calibration algorithm;
optionally, a second computer program is stored in the calibration server, and when executed by the processor, the second computer program implements the following steps:
step 1, judging that the current calibration object is a gunlock or a PTZ ball machine, and if the current calibration object is the gunlock, executing step 2; if the ball machine is a PTZ ball machine, executing the step 3;
step 2, calibrating the bolt, specifically comprising:
step 2.1, acquiring N groups of template positions, wherein N is a positive integer;
step 2.2, controlling the calibration platform to the position of the ith group of templates, wherein i =1,2, \8230, 8230and N;
step 2.3, controlling the gunlock to shoot an image containing the calibration plate;
step 2.4, carrying out corner point detection on the image obtained in the step 2.3;
step 2.5, judging whether the result of the angular point detection in the step 2.4 is accurate, if not, returning to the step 2.4, and if so, executing the step 2.6;
step 2.6, letting i = i +1, returning to the step 2.2, and obtaining the internal parameters of the lens of the bolt machine until i = N;
step 3, calibrating the PTZ ball machine, and specifically comprising the following steps:
step 3.1, obtaining the maximum value M of the to-be-calibrated magnification factor of the PTZ dome camera, wherein M is a positive integer greater than or equal to 1;
step 3.2, setting the magnification factor k =1 of the PTZ ball machine;
step 3.3, calibrating the PTZ dome camera under the magnification k by adopting the method of the step 2.1-2.6, and executing step 3.4 after obtaining the internal parameters of the PTZ dome camera under the current magnification k if successful; otherwise, returning to the step 3.3;
step 3.4, enabling k = k +1 until k = M, and obtaining internal parameters of the current PTZ dome camera under all magnification factors;
step 4, whether internal parameters of one gun camera 3 and two PTZ ball machines 2 are obtained or not is judged, and if yes, the step 5 is executed; otherwise, returning to the step 1;
and 5, obtaining the relative position relation between one gun camera 3 and two PTZ ball machines 2 in the trinocular vision module, and obtaining the external parameters of the trinocular vision module.
Since the calibration of the zoom lens needs to determine the corresponding relationship between the focal length parameters of the lens and the control parameters, the corresponding lens parameters under different control parameters need to be obtained, and the fitting relationship between the two parameters needs to be established. Therefore, calibration of lens images with multiple sets of different control parameters is required, which requires a lot of repetitive work. Assuming that N calibration images are used for each calibration, and there are Cz optical zoom multiples, the calibration staff needs to move the calibration template N × Cz times, for example, N =15, cz =20, and then 300 times are needed. In order to reduce manual operation, the calibration speed is increased. The invention provides a method for automatically controlling the movement of a calibration template.
The mechanical arm uses a steering engine with a controllable rotation angle as a joint movement assembly, and each joint can rotate by more than 180 degrees. The calibration template is fixed on the last joint of the mechanical arm, and can be rotated to any expected position by virtue of the multi-degree-of-freedom motion function of the mechanical arm, so that the problems of difficulty in operation and fixation and easiness in shaking of the conventional calibration template moved manually are solved. Because the relation between the relative positions of the calibration template and the lens does not need to be known in advance by the calibration algorithm of the Zhang Zhengyou, the precise motion parameters of the mechanical arm do not need to be known, and only the calibration template fixed on the calibration template can be completely positioned in the imaging and all the angular points can be correctly detected.
The control of arm is carried out the communication through the serial ports by dedicated drive plate, and this drive plate can provide the following three kinds of automation degree from low to high standard template motion scheme:
all manual operations control the movement of the calibration template, and different postures are realized by controlling the rotation angle of each steering engine, so that the aim of moving the calibration template is fulfilled;
using the action group function, obtaining the corresponding calibration template position through the pre-designed steering engine angle combination, and manually selecting the corresponding action group to obtain a calibration image during calibration;
the automatic calibration is realized by presetting an action group and directly communicating a calibration program with a steering engine control panel.
When the calibration is carried out specifically, which control mode is needed to be flexibly selected according to actual conditions, or three modes are switched according to the progress and the calibration conditions.
Since the bolt is responsible for monitoring the entire scene, the invention installs a wide-angle lens for it. Compared with a common lens, the wide-angle lens has obvious image distortion. Therefore, the calibration of the gunlock not only needs to obtain the focal length parameter and the principal point coordinate of the lens, but also needs to obtain the radial distortion parameter and the tangential distortion parameter, so that the parameters are utilized to carry out distortion correction on the video image, and the undistorted scene image is restored. The process of calibrating the bolt is shown in fig. 2, and the focal lengths fx and fy, the principal point coordinates cx and cy, the radial distortions k1 and k2 and the tangential distortions p1 and p2 of the bolt can be obtained through calibration.
Step 2, calibrating the bolt, which specifically comprises the following steps:
step 2.1, acquiring the positions of N groups of templates, wherein N is a positive integer, and adjusting the bolt and the mechanical arm to proper positions;
step 2.2, controlling the calibration platform to move to the position (preset position) of the ith group of templates, wherein i =1,2, \8230;
step 2.3, controlling the gunlock to shoot an image containing the calibration plate;
step 2.4, carrying out corner detection on the image obtained in the step 2.3, and comparing and judging according to the corner position stored in the original template and the detected corner position;
step 2.5, judging whether all the angular points on the template are correctly detected, namely whether the angular point detection result is accurate, if not, returning to the step 2.4, and if so, executing the step 2.6;
step 2.6, enabling i = i +1, returning to the step 2.2, and obtaining internal parameters of the gunlock lens until i = N;
and if the i is larger than the N, prompting that the calibration fails.
Preferably, the Zhang Zhengyou calibration algorithm is adopted in step 2.6 to obtain the internal parameters of the lens of the bolt machine.
The distortion of the PTZ dome camera is basically negligible except for slight radial distortion when the magnification is small. The binocular vision system of the invention mainly works in monitoring and tracking in a large range and a large visual field, and the installation position of a target is far away from the system, so that the PTZ dome camera usually works in a high magnification state. Therefore, neglecting the distortion effect of the PTZ dome camera does not bring loss of precision to the visual analysis of the system.
PTZ dome cameras typically have two modes of manual focus and auto focus, and when the PTZ lens is in the auto focus mode, the focus and aperture parameters will be automatically adjusted. In practical use, the invention selects to work in an automatic focusing mode in order to obtain the clearest image of the target. The focal length of the PTZ dome camera is influenced by the focal position and the aperture size, if the two additional control parameters are also included in calibration, the calibration process is quite complicated, and the variation relationship between the two additional control parameters is difficult to describe by a function and is not beneficial to quantitative analysis. Therefore, the invention uses the manual mode in the calibration stage, and fixes the focus and the aperture parameters at a constant value, thus the lens parameters under one Z control parameter can be kept unchanged, and the condition that the same Z parameter is not consistent with the focal length in the calibration process is prevented. The flow of PTZ parameter calibration with the assistance of a mechanical arm is shown in fig. 3, and the specific steps of calibrating the lens of the PTZ dome camera 2 in the figure are similar to those in fig. 2.
Step 3, calibrating the PTZ dome camera 2, which specifically comprises the following steps:
step 3.1, obtaining the maximum value M of the to-be-calibrated magnification of the PTZ dome camera 2, wherein M is a positive integer greater than or equal to 1;
step 3.2, setting the magnification factor k =1 of the PTZ ball machine 2;
step 3.3, calibrating the PTZ dome camera 2 under the magnification k by adopting the method of the steps 2.1-2.6, and executing step 3.4 after obtaining the internal parameters of the PTZ dome camera 2 under the current magnification k if successful; otherwise, returning to the step 3.3;
step 3.4, enabling k = k +1, and obtaining internal parameters of the current PTZ dome camera 2 under all magnification factors until k = M;
in this embodiment, it is equivalent to perform calibration of the lens internal reference once for each method multiple of the PTZ dome camera 2.
And (3) calibrating external parameters of the three cameras, namely acquiring the relative position relation between the two PTZ ball machines 2 and the gun camera 3, and describing by using a rotation matrix and a translation vector. The calibration is carried out after the system is fixedly installed, and the PTZ dome camera 2 needs to be reset before calibration, namely, all control parameters of the PTZ dome camera 2 are in zero positions. After about 10 Calibration template images are respectively collected, acquiring external parameters by using a GML MatLab Camera Calibration Toolbox.
Since the bolt 3 adopts a fixed parameter lens, the calibration of the internal parameters is relatively simple, and all the internal parameters of the bolt 3 obtained by calibration are shown in table 1 (where the outputs of fx and fy are pixel focal lengths and the unit is pixel).
TABLE 1 gun bolt internal parameter calibration results
Figure BDA0003164576500000121
Figure BDA0003164576500000131
The calibration task of the PTZ dome camera 2 is mainly to determine the functional relationship between the principal point coordinates and the focal length and the magnification control parameter Z, and the PTZ dome camera 2 works in a manual focusing mode during calibration. The principal point coordinates of the two ball machines are obtained by taking the average values of multiple calibrations, which are respectively (350.8, 285.3) and (348.3, 290.5).
Table 2 shows the fitting error analysis of the three fitting functions on the focal length and the Z control parameter, and it can be seen that the Exp2 fitting function obtains a better fitting effect only at the cost of 4 coefficients, and has the advantages of high precision and few parameters compared with the Poly4 fitting function. While Fourier3, while fitting performance is best, comes at the cost of a large number of coefficients (Fourier 3 requires 8 parameters). In this embodiment, an Exp2 fitting function is selected for the final focus distance. Table 3 shows the finally determined fitting parameters, and with the parameters of the fitting curve, the approximate value of the focal length can be obtained by controlling the parameter Z or the Z control parameter can be solved from the focal length value.
TABLE 2 comparison of three function fitting errors for PTZ dome camera focal parameters
Figure BDA0003164576500000132
TABLE 3 fitting parameters of PTZ dome camera focal length and Z control parameters when Exp2 fitting function is adopted
Figure BDA0003164576500000133
Through MATLAB calibration tool box, the external parameters of the two PTZ ball machines, namely a rotation vector R (rotation matrixes R1 and R2 can be obtained through Rodrigues transformation) and a translation vector t (unit is mm) which represent the relative positions between the two PTZ ball machines and the gun camera can be respectively obtained, as shown in Table 4. Wherein the camera coordinate system of the bolt is a reference coordinate system. The final positional relationship of the three cameras is shown in fig. 4.
TABLE 4 rotation vector and translation vector of two PTZ ball machines relative to the gun
Figure BDA0003164576500000141
The moving object tracking module has a first computer program stored therein, which when executed by a processor, performs the steps of:
step a, controlling one gun camera (3) and two PTZ ball machines (2) in a trinocular vision module to respectively start tracking a moving target;
b, judging whether the speed of the moving target is greater than a speed threshold value or not according to the result of tracking the moving target by the gunlock (3), if so, executing the step d, otherwise, executing the step c;
step c, judging whether the rotation angle of the PTZ dome camera (2) is larger than an angle threshold, if so, executing a step d, otherwise, executing a step e;
d, setting a PTZ ball machine (2) to perform primary tracking by adopting a magnification factor x, and then executing a step e, wherein x is a positive integer;
e, judging whether the primary tracking is stable, if so, executing the step f, otherwise, returning to the step d;
step f, setting a PTZ ball machine (2) to perform standard tracking by adopting a magnification factor y, wherein y is a positive integer and is more than x;
and g, completing tracking.
When the trinocular tracking system tracks the moving target, the gun camera 3 can effectively track the moving target provided by the scene and lock the moving target in the central area of the image. However, the tracking result of the PTZ dome camera is not accurate, fig. 5 is the tracking result of the high-speed moving target, fig. 5 (a) is the position of the moving target in the gun camera, fig. 5 (b) is the position of the moving target in the PTZ dome camera 1, and fig. 5 (c) is the position of the moving target in the PTZ dome camera 2, and it can be seen that, because the moving speed of the target is high, a large error occurs in the preset position of the target, so that the PTZ dome camera 2 cannot lock the high-speed car at the center of the screen. Fig. 6 shows the tracking result of the moving target after the PTZ dome camera performs large-angle movement, where fig. 6 (a) shows the position of the moving target in the gun camera, fig. 6 (b) shows the position of the moving target in the PTZ dome camera 1, and fig. 6 (c) shows the position of the moving target in the PTZ dome camera 2. Because the mechanical structure characteristic of the PTZ dome camera determines the PT movement speed, when the rotation angle is larger, the PT movement delay is increased, more uncertainty is brought to the position prediction of the moving target, and the moving target in the final tracking result is seriously deviated from the image center.
In order to solve the problem that the target speed is high or the tracking error is large when the current optical axis of the PTZ dome camera is far, the invention provides a secondary tracking strategy, namely, the target is firstly tracked for the first time by adopting a small magnification factor, more position redundancy is provided in a mode of reducing the proportion of the target in an image of the PTZ dome camera 2, and the target is firstly ensured to be positioned in the visual field range of the PTZ dome camera 2; and then, the second tracking is carried out by adopting a larger amplification factor, and the PT rotation angle of the tracking is smaller, so that the target can be accurately amplified and tracked in a short time. Fig. 7 shows the tracking flow of the PTZ dome 2 after the improvement using the quadratic tracking strategy.
In this embodiment, the speed threshold and the angle threshold may be adaptively set according to actual situations, for example, the speed threshold is set to 10 in km/h, and the angle threshold is set to 30 in °.
In order to further test the tracking performance, the invention respectively carries out 10 groups of moving object tracking tests on the tracking method adopting the standard tracking strategy and the secondary tracking strategy, namely 20 groups in total. As shown in fig. 8, fig. 8 (a) is a tracking error of a tracking method in the prior art, and fig. 8 (b) is a tracking error of a secondary tracking method provided by the present invention, where the test group with a star is a target tracking result with a fast moving target speed or a large rotation angle, it can be seen that the tracking error of the ordinary tracking method for these targets is large, and the tracking error of the secondary tracking strategy is relatively small and lower than the average level.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by software plus necessary general hardware, and certainly may also be implemented by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present invention may be substantially implemented or a part of the technical solutions contributing to the prior art may be embodied in the form of a software product, where the computer software product is stored in a readable storage medium, such as a floppy disk, a hard disk, or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

Claims (3)

1. A moving target tracking system based on the trinocular vision is characterized by comprising a trinocular vision module, a camera calibration module and a moving target tracking module;
the binocular vision module is used for acquiring images containing moving targets, and comprises a gun camera (3) and two PTZ (Pan/Tilt/zoom) ball machines (2), wherein the gun camera (3) and the PTZ ball machines (2) are arranged on the same horizontal line, and the gun camera (3) is positioned in the middle of the two PTZ ball machines (2); the optical axis of the PTZ dome camera (2) at the zero position is the same as the direction of the gunlock (3);
the camera calibration module is used for calibrating the trinocular vision module and comprises a calibration platform and a calibration server;
the calibration platform is used for driving the calibration plate to be at the position of the multi-group template;
the calibration server is used for controlling the calibration platform to a specified position and calibrating the gunlock (3) and the PTZ dome camera (2) by adopting a calibration algorithm;
the moving object tracking module has a first computer program stored therein, which when executed by a processor implements the steps of:
step a, controlling one gun (3) and two PTZ ball machines (2) in the trinocular vision module to respectively start tracking moving targets;
b, judging whether the speed of the moving target is greater than a speed threshold value or not according to the result of tracking the moving target by the gunlock (3), if so, executing the step d, otherwise, executing the step c;
step c, judging whether the rotation angle of the PTZ dome camera (2) is larger than an angle threshold, if so, executing a step d, otherwise, executing a step e;
d, setting a PTZ ball machine (2) to perform primary tracking by adopting a magnification factor x, and then executing a step e, wherein x is a positive integer;
e, judging whether the primary tracking is stable, if so, executing the step f, otherwise, returning to the step d;
step f, setting a PTZ ball machine (2) to perform standard tracking by adopting a magnification factor y, wherein y is a positive integer and is more than x;
and step g, completing tracking.
2. A system for tracking a moving object based on trinocular vision, as claimed in claim 1, characterized in that said calibration server has stored therein a second computer program which, when executed by a processor, carries out the steps of:
step 1, judging that a current calibration object is a gunlock (3) or a PTZ (pan/tilt/zoom) dome camera (2), and if the current calibration object is the gunlock (3), executing the step 2; if the PTZ ball machine (2) is adopted, executing the step 3;
step 2, calibrating the bolt (3), and specifically comprising the following steps:
step 2.1, acquiring N groups of template positions, wherein N is a positive integer;
step 2.2, controlling the calibration platform to the position of the ith group of templates, wherein i =1,2, \8230; N;
step 2.3, controlling the lens to shoot an image containing the calibration plate;
step 2.4, carrying out corner point detection on the image obtained in the step 2.3;
step 2.5, judging whether the result of the angular point detection in the step 2.4 is accurate, if not, returning to the step 2.4, and if so, executing the step 2.6;
step 2.6, letting i = i +1, returning to step 2.2, and obtaining the internal parameters of the lens until i = N;
step 3, calibrating the PTZ dome camera (2), which specifically comprises the following steps:
step 3.1, obtaining the maximum value M of the to-be-calibrated magnification of the PTZ dome camera (2), wherein M is a positive integer greater than or equal to 1;
step 3.2, setting the magnification factor k =1 of the PTZ ball machine (2);
step 3.3, calibrating the PTZ dome camera (2) under the magnification k by adopting the method of the steps 2.1-2.6, and executing step 3.4 after internal parameters of the PTZ dome camera (2) under the current magnification k are obtained if the calibration succeeds; otherwise, returning to the step 3.3;
step 3.4, enabling k = k +1 until k = M, and obtaining internal parameters of the current PTZ dome camera (2) under all magnification factors;
step 4, whether internal parameters of one gun (3) and two PTZ ball machines (2) are obtained or not is judged, and if yes, the step 5 is executed; otherwise, returning to the step 1;
and 5, obtaining the relative position relation between one gun (3) and two PTZ ball machines (2) in the trinocular vision module, and obtaining the external parameters of the trinocular vision module.
3. The trinocular vision-based moving object tracking system as claimed in claim 2, wherein the step 2.6 adopts a Zhang-Yong calibration algorithm to obtain the internal parameters of the lens.
CN202110800602.5A 2021-07-15 2021-07-15 Moving target tracking system based on trinocular vision Active CN113538596B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110800602.5A CN113538596B (en) 2021-07-15 2021-07-15 Moving target tracking system based on trinocular vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110800602.5A CN113538596B (en) 2021-07-15 2021-07-15 Moving target tracking system based on trinocular vision

Publications (2)

Publication Number Publication Date
CN113538596A CN113538596A (en) 2021-10-22
CN113538596B true CN113538596B (en) 2022-10-11

Family

ID=78099430

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110800602.5A Active CN113538596B (en) 2021-07-15 2021-07-15 Moving target tracking system based on trinocular vision

Country Status (1)

Country Link
CN (1) CN113538596B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115713565A (en) * 2022-12-16 2023-02-24 盐城睿算电子科技有限公司 Target positioning method for binocular servo camera

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102497543A (en) * 2012-01-06 2012-06-13 合肥博微安全电子科技有限公司 Multi-target tracking method based on DSP and system thereof
CN104754302B (en) * 2015-03-20 2017-08-08 安徽大学 A kind of target detection tracking method based on rifle ball linked system
CN110415278B (en) * 2019-07-30 2020-04-17 中国人民解放军火箭军工程大学 Master-slave tracking method of auxiliary binocular PTZ (Pan-Tilt-zoom) visual system of linear moving PTZ (pan-Tilt-zoom) camera

Also Published As

Publication number Publication date
CN113538596A (en) 2021-10-22

Similar Documents

Publication Publication Date Title
US8346071B2 (en) Camera system and method of correcting camera fitting errors
CN1761855B (en) Method and device for image processing in a geodetic measuring device
EP3389268B1 (en) Depth information acquisition method and apparatus, and image collection device
US20050069195A1 (en) Apparatus and method for establishing correspondence between images
EP3994875A1 (en) Camera-specific distortion correction
US9794483B1 (en) Video geolocation
CN113487683B (en) Target tracking system based on trinocular vision
CN113538596B (en) Moving target tracking system based on trinocular vision
KR20200081057A (en) Method and Apparatus for Center Calibration of Camera System
Ding et al. A robust detection method of control points for calibration and measurement with defocused images
JP2001221749A (en) Observation device and observation method
CN114820725A (en) Target display method and device, electronic equipment and storage medium
CN113781576A (en) Binocular vision detection system, method and device for multi-degree-of-freedom pose real-time adjustment
CN112598751A (en) Calibration method and device, terminal and storage medium
CN113470118B (en) Target size estimation system based on trinocular vision
CN114693807B (en) Method and system for reconstructing mapping data of power transmission line image and point cloud
CN113960564B (en) Laser comprehensive reference system for underwater detection and ranging and calibrating method
CN116051652A (en) Parameter calibration method, electronic equipment and storage medium
US11625836B2 (en) Trajectory calculation device, trajectory calculating method, and trajectory calculating program
CN115239816A (en) Camera calibration method, system, electronic device and storage medium
CN115103124A (en) Active alignment method for camera module
CN112233185A (en) Camera calibration method, image registration method, camera device and storage device
CN112465920A (en) Vision sensor calibration method and device
JP2001210261A (en) Transmission electron microscope
CN113489964B (en) Scene depth information acquisition system based on trinocular vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant