CN110490903B - Multi-target rapid capturing and tracking method in binocular vision measurement - Google Patents
Multi-target rapid capturing and tracking method in binocular vision measurement Download PDFInfo
- Publication number
- CN110490903B CN110490903B CN201910738024.XA CN201910738024A CN110490903B CN 110490903 B CN110490903 B CN 110490903B CN 201910738024 A CN201910738024 A CN 201910738024A CN 110490903 B CN110490903 B CN 110490903B
- Authority
- CN
- China
- Prior art keywords
- target
- beacon
- image
- tracking
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention provides a method for quickly capturing and tracking multiple targets in binocular vision measurement, and belongs to the field of computer vision measurement. The multi-target rapid capturing and tracking method comprises image preprocessing, target capturing, multi-target identification and multi-target tracking under the condition of a strong clutter background. The invention can realize real-time processing of mass data, can stably capture and track a plurality of beacon points on an object under the conditions of strong clutter such as solar illumination, sky cloud layer, imaging noise, sea level reflection and the like, and correspondingly matches each beacon point in an image with the space position thereof.
Description
Technical Field
The invention relates to a method for rapidly acquiring and tracking a plurality of weak and small targets under a complex background, in particular to a method for acquiring and tracking a plurality of cooperative target sources (namely beacons) in binocular vision measurement. Belonging to the field of image processing and computer vision measurement.
Background
With the development of the photoelectric technology and the integration of the technology and the subject intersection, the non-contact measurement by using a computer vision measurement method is becoming an important measurement method. The vision measurement technology can measure the distance of an object and the three-dimensional space position and attitude angle of the object. The vision measurement technology has the characteristics of low cost, long operating distance and the like, and can theoretically measure any far object position as long as a measuring camera can effectively extract a certain number of characteristic points on an object. At present, the vision measurement technology is widely applied to the fields of aerospace, aviation, navigation and the like, and is also widely applied to the fields of machine manufacturing, medicine, biology, architecture and the like.
The first problem to be solved is to quickly capture and accurately track a certain number of cooperative beacon points on an object in order to quickly measure the position and the posture of the object in space relative to a certain reference coordinate system in real time. By extracting coordinates of the beacon points in the image and combining the coordinates of the corresponding beacon points on the measured object, a projection mapping equation set can be established, and a rotation matrix and a translation matrix of the measured object relative to a reference coordinate system can be calculated through a photogrammetry principle, namely, the position and the attitude angle of the measured object in space are measured.
In the process of implementing the invention, the inventor finds that the prior art has at least the following defects:
target capturing and tracking devices in the binocular vision measurement system are mostly based on a PC (personal computer), but the PC is large in size, high in power consumption and poor in stability, and cannot meet engineering application, so that a stable high-speed embedded processing platform is urgently required to be developed for real-time processing of large data volume in a double camera; the traditional multi-target capturing and tracking method processes each beacon in an isolated manner, does not consider rigid connection between different beacon points on two sides of the same object in binocular vision measurement, and effectively utilizes result data specific to a binocular measurement system, namely position and attitude information of the measured object, so as to improve the stability of target capturing and tracking.
Disclosure of Invention
The invention aims to provide a multi-target rapid capturing and tracking method in binocular vision measurement, which is used for realizing real-time processing of mass data, stably capturing and tracking a plurality of beacon points on an object under the condition of strong clutter (sunlight, sky cloud layer, imaging noise, sea level reflection and the like), and correspondingly matching each tracked image beacon with the physical space position of the image beacon.
The invention relates to a multi-target capturing and tracking method in binocular measurement, which comprises the following steps: 1) Preprocessing an image under a strong clutter background condition; 2) Target capture; 3) Multi-target identification; 4) And (4) multi-target tracking, wherein each step is introduced below.
1) And (4) preprocessing the image under the strong clutter background condition. And (3) inhibiting the interference of background noise by adopting a filtering method and enhancing the target energy.
The background clutter mainly comprises solar illumination, sky cloud layer, sea level reflection and the like.
The filtering method is to adopt a 5-by-5 high-pass filter template to carry out convolution on an image so as to enhance the signal mark points in the image and inhibit background clutter.
2) And (4) target capture. And (4) forming a candidate target set by extracting all possible candidate targets in the image, and preparing for next target identification. The target capture includes target adaptive threshold segmentation and target search and region tagging.
The target self-adaptive threshold segmentation means that a segmentation threshold is automatically calculated according to the characteristics of the global image, and the image is segmented into a binary image with a foreground of 1 and a background of 0 by using the threshold.
The target searching and region marking means that all connected regions are calculated for the divided binary image, and attribute characteristics of each connected region are counted, including mathematical morphology processing; marking a multi-target area; merging and separating targets; and extracting multi-target features.
The mathematical morphology processing is to process a binary image by using an opening operation in the mathematical morphology, namely, the image is corroded firstly, then expansion operation is carried out, a false connected region with a small area is removed, and a connected region with a large area is filled.
The multi-target region marking means that pixels in the binary image, which accord with a certain connectivity rule (4 neighborhood connectivity and 8 neighborhood connectivity), are represented by the same mark, so that the connected regions are marked identically.
The target merging and separating means that if the two target areas are close to each other, the two target areas are merged to form a target area; otherwise, the two regions are separated.
The multi-target feature extraction refers to calculating the attribute features of each candidate target in the candidate target set, including the area, energy, length-width ratio and centroid point coordinates of the targets.
3) And (4) multi-target identification. Identifying a plurality of real beacon points from the candidate target set, including target attribute feature identification; identifying geometric relations among the targets; and identifying the motion continuity of the target.
The target attribute feature identification means that false targets are removed from the candidate target set according to certain attribute features of the targets, and a target set which is most likely to be a real beacon point is reserved.
The certain attribute features refer to the area, energy and aspect ratio of the candidate target, and if the area, energy and aspect ratio features of the candidate target exceed a reasonable range, the candidate target is considered as a false target and is removed.
The geometric relation identification between the targets refers to that a plurality of beacon points arranged on each side of a measured object are projected on an image to be approximately in a square relation, and a plurality of real beacon points can be found from a candidate target set by utilizing the geometric position relation among the plurality of beacon points.
The target motion continuity identification refers to the step of analyzing each target motion track so as to remove false target points from the candidate target set.
The target motion track is judged by counting the smoothness of the change of the target between adjacent image sequences and the continuity of the change of the target coordinate position by utilizing the characteristic information (including target area information and target position information) of each frame of image when the target moves.
4) And (4) multi-target tracking. Extracting the centroid coordinates of each beacon point from each frame of image to form the motion trail of the target, including multi-target initial tracking; predicting a multi-target track; multi-target locking tracking; target loss recapture.
The multi-target initial tracking is to judge whether a plurality of beacon points to be tracked are real beacon points or not through the motion continuity of a target between adjacent image sequences, and simultaneously construct the motion track of each beacon to prepare for target track prediction, which is a supplement of a multi-target identification step.
The multi-target track prediction means predicting the displacement of the target at the next moment according to the speed and the displacement of the target at the current moment.
The multi-target locking tracking means that the centroid coordinates of each beacon at the current moment are extracted in real time, and the motion track of each beacon is established.
The target loss recapture means that when the target is lost, the tracking algorithm can continuously lock and track the remaining target points, and photogrammetry is carried out by using the remaining target points to calculate the position and the posture of the object as long as the sum of the number of the remaining beacons in the two cameras is more than or equal to four; when the lost target reappears, the target loss recapturing method can capture the lost target immediately and add the captured target into the equation set of the photogrammetry, so that the equation solving precision is improved, and meanwhile, the calculation precision of the position and the posture of the object is improved.
Compared with the prior art, the invention has the advantages that:
(1) The invention can reduce background clutter, improve the signal-to-noise ratio of the image, and reduce the complexity and processing time of subsequent target capture and tracking by using the image preprocessing method under the strong clutter background condition;
(2) The invention utilizes a multi-layer progressive mode to carry out multi-target identification. Firstly, carrying out preliminary screening on candidate targets through target attribute feature identification; then, further identifying by using the geometric relationship between the targets, and removing a large number of candidate targets which do not meet the geometric relationship in a combined manner; and finally, performing time domain identification by using the target motion continuity, and performing final judgment by using the track continuity of the target motion. On one hand, the progressive identification mode improves the identification efficiency, reduces the calculated amount and can reduce the false identification rate;
(3) The method can stably track multiple targets, can still continuously track the remaining targets when a certain target is lost, and can still capture and add the lost target into a tracking list by a target loss recapturing method when the lost target appears again, and simultaneously does not change the tracking number of the target;
(4) The method can be widely applied to a binocular vision measurement system, and the reliability and stability of target capture and tracking in the system are improved.
Drawings
FIG. 1 is a schematic flow chart of a multi-target rapid capturing and tracking method in binocular vision measurement according to the present invention;
FIG. 2 is a schematic flow chart of a target capture method according to the present invention;
FIG. 3 is a schematic flow chart of a target searching and region labeling method according to the present invention;
FIG. 4 is a flow chart of a multi-target identification method of the present invention;
FIG. 5 is a diagram illustrating the numbering of multiple beacon points on an image according to the present invention;
FIG. 6 is a schematic view of a projection relationship of a plurality of landmark points on one side of an object to be measured in the present invention;
FIG. 7 is a flow chart of a multi-target tracking method of the present invention;
FIG. 8 is a diagram illustrating the state transitions of different constituent units of the multi-target rapid acquisition and tracking method of the present invention;
fig. 9 is a schematic view of the binocular vision measurement principle of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following will describe the method of the present invention in further detail with reference to the accompanying drawings.
The hardware environment for implementing the invention is: an image processing platform; two cameras; and (5) measuring the object. The image processing platform consists of 4 32-bit floating point ADSP-TS201 digital signal processors with the main frequency of 600MHz and 1 super-large-scale high-speed programmable logic device XC4VLX 80. As shown in fig. 9, two cameras 31 and 32 are distributed on both sides of the target to be measured, and 4 beacon lights are installed on both sides of the target to be measured 33. The implementation process of the invention comprises the following steps: firstly, two paths of cameras are used for simultaneously acquiring image data streams containing 4 beacon points on two sides of an object to be measured, and the two paths of data streams are transmitted to an image processing platform in real time through optical fibers; the image processing platform carries out preprocessing, multi-target capturing and tracking on the image and extracts the centroid coordinate of each beacon point; the centroid coordinates of the beacon points are then used for photogrammetry to resolve the position and attitude of the object relative to the reference coordinate system 34.
As shown in fig. 1, the invention relates to a method for rapidly capturing and tracking multiple targets in binocular vision measurement, which is used for rapidly capturing, identifying and tracking targets under the condition of strong clutter interference. The method comprises the following specific implementation steps: 1) Preprocessing an image under a strong clutter background condition; 2) Target capture; 3) Identifying multiple targets; 4) And (4) multi-target tracking. The detailed description of each step is as follows:
1) Image preprocessing under strong clutter background condition
For the correct implementation of the subsequent target capture and tracking, an effective preprocessing method must be selected to preprocess the image under the strong clutter background condition, so as to improve the energy of the target and remove or weaken the influence of the background clutter. Experimental data analysis shows that the main clutter interference is solar illumination, sky cloud layers, sea surface clutter and the like, the imaging sizes of the beacon points on two sides of an object in a certain measurement range are about 3 × 3-5 × 5 pixels, the energy of the beacon points on an image is far smaller than that of the solar illumination, and the imaging of the beacon points on the image is isolated singular points.
In view of the above, the present invention provides a 5 × 5 high pass filter template F to enhance the signal points in the image and suppress the background noise. The 5 x 5 high pass filter template F is:
by performing convolution operation on the image by using the high-pass filter template F, most of continuously distributed clutter interference in the image, such as continuously distributed cloud layers, solar illumination and the like, can be removed, but the discontinuous cloud layer interference cannot be removed, so that the filtered image contains useful target point data and a part of filtered noise point data, and needs to be processed in subsequent steps.
2) Target capture
As shown in fig. 2, the target capture includes target adaptive threshold segmentation and target search and region tagging, each of which is described in detail below.
The target adaptive threshold segmentation is to segment a current gray image into a binary image with a foreground of 1 and a background of 0, and is specifically performed by the following method:
a) Statistical mean of pre-processed images:
where I (x, y) is the pixel gray value with coordinates (x, y), M and N are the length and width of the image,
is the image mean.
b) Variance of statistical images:
wherein S is a variance value of the image; the segmentation threshold of the image is selected as follows:
where a is a regulatory factor, preferably between 1.75 and 2.45.
As shown in fig. 3, the target search and region labeling comprises the following steps: (a) mathematical morphology processing; (b) multi-target region markers; (c) target merging and separation; and (d) extracting multi-target features.
(a) And (5) carrying out mathematical morphology processing. And performing morphology opening operation on the binary image after the self-adaptive threshold segmentation, namely corroding the image firstly, and then performing expansion operation to remove the false connected region with a smaller area and fill the connected region with a larger area. The structural elements adopted by corrosion and expansion are all 3 x 3 templates, and the part of the operation is simple logic operation, and the parallelism of the algorithm is higher, so that the part of the operation is carried out in a field programmable logic device to improve the operation speed.
(b) And marking the multi-target area. Pixels in the binary image which accord with a certain connectivity rule (4 neighborhood connectivity and 8 neighborhood connectivity) are represented by the same mark, and the image after the mathematical morphology processing is marked by selecting an 8 neighborhood connectivity mode. In the present invention, the number of image lines is N, and a target segment is defined as a set of pixels in which pixels in one image line are continuously 1, and is labeled as L [ i ] = { Ms, me, line }, where Ms, me respectively represent a start position and an end position of the target segment in the line. The specific implementation steps of the algorithm are as follows:
step 0: label is initialized to 0, the label number of the connected region is recorded, and line is initialized to 0;
step 1: searching an unmarked target segment in the current line, if so, marking as { Ms, me, line }, adding 1 to label, assigning the pixel mark in the target segment as label, and performing the steps 2-4; if not, entering the step 4;
step 2: if the line is greater than 0, searching all unmarked target sections communicated with the current target section in the previous line, namely the line-1 line, if yes, marking pixels in each target section as label, and simultaneously repeating the steps 2-3 for each target section; if not, or line =0, entering step 3;
and 3, step 3: if the line + 1-woven fabric is N, searching all unmarked target sections communicated with the current target section in the next line, namely the line +1 line, if yes, marking the pixels in each target section as label, and simultaneously repeating the steps 2-3 for each target section; if not, or line = N-1, do nothing;
and 4, step 4: if the current line is finished, adding 1 to the line (if the line = N, finishing, otherwise, entering the step 1); if the current line is not finished, step 1 is entered.
And 5, completing multi-target area marking, wherein each connected target area is endowed with a unique number.
(c) Merging and separating targets. On the basis of the multi-target area mark, namely if two target areas are very close to each other, combining the two target areas to form a target area; otherwise, the two regions are separated. Wherein two target areas are adjacent means that the area edges of the two targets are less than the threshold T.
The threshold T is preferably between 3 and 5 pixels.
(d) And extracting multi-target features. And calculating the attribute characteristics of each candidate target in the candidate target set, and providing criteria for next target identification.
The candidate target set refers to each connected target area left after the target merging and separation in the previous step.
The attribute features of the candidate target include: area, energy, aspect ratio, centroid point coordinates for each candidate object. And establishing a feature structure for each candidate target, establishing a plurality of feature structures for a plurality of candidate targets, and finally outputting a feature structure array, namely a candidate target attribute feature set.
3) Multi-target identification
As shown in fig. 4, the multi-target identification includes the following steps: (a) target attribute feature identification; (b) identifying geometric relationships between the objects; and (c) identifying the motion continuity of the target.
(a) And identifying the target attribute features. And removing false targets from the candidate target set through the attribute characteristics of the targets, and keeping the target set which is most likely to be a real beacon point. The specific method comprises the following steps: first, the area and energy criteria of the target are used to determine whether the target in each candidate target set is a false target. According to the change range of the measured object relative to the distance of the camera when the measured object moves in the space and the actual size of the beacon point, the area and the energy change range of the beacon point on the image can be approximately judged through a pinhole imaging principle, if the area or the energy of a certain candidate target exceeds the range, the candidate target is considered as a false target, and the false target is intensively deleted from the candidate target; secondly, the aspect ratio attribute of the target is used for distinguishing the remaining candidate target set, and since the real laser target source is a circular spot, the imaging of the laser target source is also approximate to a circle or an ellipse on the image, if the aspect ratio of a certain candidate target exceeds a threshold value T, the certain candidate target is considered as a false target and is deleted from the candidate target set, and the threshold value T is preferably between 1.2 and 1.5.
The target attribute feature identification step is completed, and through the operation, a large number of false targets can be removed from the candidate feature set, so that the calculation amount of the next step of geometric relationship identification between the targets is reduced.
(b) And identifying the geometric relationship between the targets. The geometric relationship among a plurality of beacon points arranged on the measured object is used for identification.
Because at least 4 beacon points on the object are required to be provided to solve the spatial position of the object when the position and the posture of the object are measured, 4 beacon points are respectively arranged on the left side and the right side of the measured object in the method, and 8 beacon points are required to be matched with the spatial coordinate points on the object at the same time so as to ensure the accuracy and the robustness of measurement.
As shown in fig. 6, the 4 beacon points installed on each side of the measured object are approximately in a square relationship, and the 4 beacon points are also approximately in the same plane, so that the geometric relationship of the 4 beacon points projected on the image plane can be judged according to the distance and angle of the measured object relative to the movement of the camera and the shooting geometry knowledge. FIG. 5 shows the number projected on the image by 4 beacons, the number I projected on the image by the beacon point at the upper left corner of the measured object, and the number I projected on the upper right cornerThe corner is number II, the lower left corner is number III, and the lower right corner is number IV. Let candidate target set be O = { O = { (O) 1 ,o 2 ,...,o n Wherein n is the number of candidate targets in the target set, and the step of specifically identifying 4 corresponding points on the image of 4 beacon points in the measured object space is as follows:
step 1: for candidate target set O = { O = { (O) 1 ,o 2 ,...,o n Sorting the vertical coordinates of the mass centers of the targets from small to large, so as to ensure that the target points above the image view field appear in the front of the candidate target set, and simultaneously judging the number n of the candidate target set, if n =4, turning to the step 6;
step 2: selecting the ith candidate target point in the candidate target set, wherein I = 1.. And n, and assuming that the ith candidate target point is the beacon point numbered as I in FIG. 5, then judging whether the beacon points numbered as II, III and IV in FIG. 5 exist in the remaining candidate target set, and meanwhile, judging whether the beacon points numbered as II, III and IV in FIG. 5 exist in the remaining candidate target set
Then, the next step is carried out;
and 3, step 3: for j =1, n, and j ≠ i, it is determined whether the following conditions are satisfied:
①x i +t 2 ＞x j ＞x i +t 1
wherein, t 1 Preferably between 10 and 15 pixels; t is t 2 Preferably between 30 and 35 pixels; t is t 3 Preferably between 0.75 and 0.80. If the j candidate target point meets the two conditions, the j candidate target point is preliminarily judged to be the beacon point numbered II in the figure 5, the next step is carried out, and if all the candidate beacon points do not meet the two conditions, the step 2 is carried out;
and 4, step 4: for k = 1.. N and k ≠ i, k ≠ j, it is determined whether the following conditions are satisfied:
①y i +t 2 ＞y k ＞y i +t 1
if the kth candidate target point meets the two conditions, the kth candidate target point is preliminarily judged to be the beacon point numbered as III in the figure 5, the next step is carried out, and if all the candidate beacon points do not meet the two conditions, the step 2 is carried out;
and 5, step 5: for l = 1.., n and l ≠ i, l ≠ j, l ≠ k, it is determined whether the following condition is satisfied:
①x k +t 2 ＞x l ＞x k +t 1
②y j +t 2 ＞y l ＞y j +t 1
wherein t is 4 Preferably between 0.5 and 0.65. If the first candidate target point meets the three conditions, the first candidate target point is preliminarily judged to be the beacon point numbered IV in the figure 5, the next step is carried out, and if all the candidate beacon points do not meet the three conditions, the step 2 is carried out;
and 6, step 6: and judging the geometric imaging relation of the 4 candidate target points. According to the calculation of the geometrical knowledge of photography and the analysis of experimental data, the method comprises the following steps: the imaging of the 4 beacon points on the image cannot be a concave quadrangle and only can be a convex quadrangle, and meanwhile, the four beacon points are distributed in four quadrants of a coordinate system which takes the central points of the 4 beacon points as the original points, the horizontal rightward direction as the horizontal coordinate and the vertical downward method as the vertical coordinate. The specific judging steps are as follows: the center points of the 4 target candidate target points are first calculated, i.e.,
using the central point as the origin (X) 0 ,Y 0 ) Establishing a coordinate system, and firstly searching whether a point of a first quadrant exists in the 4 candidate target points, namely, if the coordinate of a certain candidate target point satisfies: x is the number of i ＜X 0 And y is i ＜Y 0 If the point is the beacon point numbered as I in FIG. 5; if the coordinates of a candidate target point satisfy: x is the number of i ＞X 0 And y is i ＜Y 0 If the point is the beacon point numbered as II in the figure 5; if the coordinates of a candidate target point satisfy: x is the number of i ＞X 0 And y is i ＞Y 0 If the point is the beacon point numbered as III in FIG. 5; if the coordinates of a candidate target point satisfy: x is a radical of a fluorine atom i ＞X 0 And y is i ＞Y 0 Then this point is numbered IV beacon point in fig. 5. If the 4 candidate target points all correspond to the 4 beacon points in the graph 5, the next step is carried out, otherwise, the step 2 is returned;
and 7, step 7: and judging the distance between every two 4 candidate beacon points. As shown in fig. 5, 4 beacon points may be connected in series to form a quadrangle, and by calculating the length and width of four sides of the quadrangle, it may be further determined whether the 4 candidate points to be identified are real beacon points. If the length and width of the quadrangle are smaller than the minimum threshold value and larger than the maximum threshold value, the 4 points are not all real beacon points, the step 2 is returned, otherwise, 4 real beacon points are found, and the judgment is finished.
The minimum threshold value is preferably 10 to 15, and the maximum threshold value is preferably 30 to 35.
(c) And identifying the motion continuity of the target. And analyzing the motion track of each target so as to eliminate false target points from the candidate target set. The track of the target movement is judged by counting the smoothness of the target change between adjacent image sequences and the continuity of the target coordinate position change by utilizing the characteristic information (including target area information and target position information) of each frame of image when the target moves.
The method for counting the smoothness of the change of the target area between the adjacent image sequences comprises the following steps: and (3) taking the current moment as t, and counting the mean value of the target areas in the previous n frames of images:
if the target area at the current moment t meets the following conditions:
and if so, considering that the area change of the target is not smooth, and removing the target from the candidate target set. Said n is preferably between 5 and 7.
The method for counting the continuity of the coordinate position change of the target comprises the steps that according to the characteristics of image data streams collected by a camera in real time, the coordinate position change of the target between adjacent image sequences cannot generate sudden change, meanwhile, the imaging area of the target on an image can also change when the target is close to the camera, if the target is close to the camera, the imaging area of the target on the image is larger, and the movement distance between the adjacent image sequences is larger; if the object is located relatively far from the camera, the area over the images is relatively small and the distance of movement between adjacent image sequences is also relatively small. By using the characteristic, the motion distance of the target between adjacent image sequences is divided by the area of the target, if the ratio exceeds a threshold value T, the coordinate displacement change of the target is considered to be unstable, and the target is removed from the candidate target set. The threshold value T is preferably between 20% and 30%.
4) Multi-target tracking
The multi-target tracking comprises the following steps: (a) multi-target initial tracking; (b) multi-target trajectory prediction; (c) multi-target lock tracking; (d) target loss recapture. The specific operation of each step is described below.
(a) The multi-target initial tracking is a supplement of the multi-target identification step, namely judging whether 4 beacon points to be tracked are real beacon points or not through the continuity of the motion of the target between the adjacent image sequences, and estimating the motion distance of the object between the adjacent sequences according to the real motion speed of the object to be measured in the space and the distance of the object to be measured relative to a camera, wherein the estimation method comprises the following steps:
wherein f is the focal length of the camera lens; l is the distance of the object to be measured relative to the camera; sigma is the pixel size of the CCD of the camera; d is the distance of the beacon movement between adjacent sequences, i.e. the value to be estimated; d is the moving speed of the measured object, and P is the frequency of the camera. The motion component of the object with respect to the direction of the optical axis of the camera is not taken into account because it is an approximate estimate. By determining the maximum value of D, the maximum value of D can be estimated. And if the moving distance of the initially tracked beacon point between the adjacent sequences exceeds the maximum value of d, judging the target as a false target. If the moving distance of the continuous T frame beacon between the adjacent sequences is within the maximum value, the next step is entered, wherein T is preferably 4.
(b) According to the invention, the multi-target track prediction means that the displacement of the target at the next moment is predicted according to the speed and the displacement of the target at the current moment. Due to real-time data sampling, the motion displacement of the target at adjacent moments is small, and the displacement of the target can be estimated by adopting a linear prediction method. The displacement prediction estimation formula of the target at the next moment k +1 is as follows:
wherein f (k), f (k-1) and f (k-2) are the displacement of the target in the k, k-1 and k-2 image sequences respectively.
(c) According to the invention, the multi-target locking tracking refers to extracting the centroid position of each beacon at the current moment in real time and establishing the motion track of each beacon. The specific calculation process is as follows: and respectively establishing a window area with the size of T x T by taking the predicted beacon position of the target track as the center, and calculating the centroid position of the area. Wherein T is preferably
Wherein S i Is the area of the ith beacon point. The centroid of each beacon point is calculated by:
wherein I (X, y) is the gray value of the image at position (X, y), (X) i ,Y i ) Is the centroid of the ith beacon point.
(d) According to the invention, the target loss recapture means that when the target is lost, the tracking algorithm can continuously lock and track the remaining target points, and photogrammetry is carried out by using the remaining target points, so as to calculate the position and the posture of the object, as long as the sum of the number of the remaining beacons in the two cameras is more than or equal to 4; when the lost target reappears, the target loss recapturing method can capture the lost target immediately and add the captured target into the equation set of the photogrammetry, so that the equation solving precision is improved, and the calculation precision of the position and the posture of the object is improved.
The basic principle of the target loss recapture method is based on projection error minimization. As shown in fig. 6, 4 beacon points on a certain side of the measured object 121 in the space can be projected into the image plane 122 through a photographic transformation according to the following transformation formula:
can calculate the rotation matrix R of the space three-dimensional coordinate point on the measured object relative to the camera coordinate system 1 R 2 And a translation vector R 1 T 2 + T, wherein R 1 And T 1 Is a rotation and translation matrix of the object's own coordinate system relative to the reference coordinate system;R 2 and T 2 Is a rotation and translation matrix of the reference coordinate system relative to the camera coordinate system; x O Is an object coordinate system; x H Is a reference coordinate system; x C Is the camera coordinate system. Due to R 1 And T 1 Is the final photogrammetric result, can be calculated when the target is lost by no more than 4, R 2 And T 2 Is the result of camera external parameter calibration, so the three-dimensional space coordinate of the measured object can be determined according to the pinhole imaging principle 1 ,Y 1 )，...，(X 8 ,Y 8 ) -transforming to image plane coordinates { (x) 1 ,y 1 )，...，(x 8 ,y 8 ) }. If a certain beacon point i is lost in the tracking process, the search is continued in the neighborhood where the lost beacon point i may appear while other unreleased beacon points are tracked, namely, the projection point (x) of the lost beacon point i is used i ,y i ) And taking the threshold T as a radius as a circle center, and searching whether the currently lost beacon point i appears again. The threshold T is preferably between 5 and 7 pixels.
FIG. 7 is a flow chart illustrating multi-target tracking. Firstly, target initial tracking is carried out, when the number of frames of initial tracking exceeds 4 frames, a target track prediction process is carried out, after the target track prediction predicts the position of a target at the next moment, a target locking tracking link is carried out, the target locking tracking process takes the target prediction position as the circle center, the centroid point of the area is calculated, the centroid point is the tracked target position, meanwhile, whether the target is lost or not is judged, if no target is lost, the target track prediction process is carried out, otherwise, a target loss recapture link is carried out, the process is repeated, and if 4 beacons on one side of a measured object are completely lost in the motion process, the target capture is carried out again.
FIG. 8 is a diagram illustrating state transitions of different constituent units in a multi-target rapid acquisition and tracking method. When the image data flow comes, firstly, the target capture is carried out, then, the target recognition is carried out, if the target recognition unit does not find a real beacon point, the target recognition unit returns to the target capture unit again to continue the target capture of the next frame, and if not, the target initial tracking unit is entered. And (3) establishing an initial motion track for each beacon point by target initial tracking, judging the continuity of the track, returning to a target capturing unit if the track is discontinuous, and entering a target locking tracking unit if the track is discontinuous. The target locking tracking finishes the extraction of the target centroid through a series of processes, judges whether the target is lost or not, and enters the target capturing unit again if the target is lost completely.
Claims (1)
1. A multi-target rapid capturing and tracking method in binocular vision measurement is characterized by comprising the following steps:
firstly, preprocessing an image under a strong clutter background condition;
secondly, target capture, wherein the target capture comprises target self-adaptive threshold segmentation, target search and region marking;
the target self-adaptive threshold segmentation is to automatically calculate a segmentation threshold according to the characteristics of a global image, and segment the image into a binary image with a foreground of 1 and a background of 0 by using the threshold;
the target searching and region marking means that all connected regions are calculated for the divided binary image, and the attribute characteristics of each connected region are counted, wherein the attribute characteristics comprise mathematical morphology processing, multi-target region marking, target merging and separation and multi-target characteristic extraction;
step three, multi-target identification, wherein the multi-target identification comprises target attribute feature identification, geometric relation identification between targets and target motion continuity identification;
the target attribute feature identification means that false targets are removed from a candidate target set through the attribute features of the targets, the target set which is most likely to be a real beacon point is reserved, and if the area and energy features of the candidate targets exceed reasonable ranges, the candidate targets are considered as false targets and are removed;
the geometric relation identification between the targets means that 4 beacon points arranged on each side of a measured object are projected on an image to show an approximately square relation, and the real 4 beacon points can be found from a candidate target set by utilizing the geometric position relation between the 4 beacon points;
the target motion continuity identification refers to analyzing each target motion track so as to eliminate false target points from the candidate target set;
the target motion track is judged by counting the smoothness of the change of the target between adjacent image sequences and the continuity of the change of the target coordinate position by utilizing the characteristic information of each frame of image when the target moves, including the target area information and the target position information;
step four, multi-target tracking, wherein the multi-target tracking comprises multi-target initial tracking, multi-target track prediction, multi-target locking tracking and target loss recapture;
the multi-target initial tracking is to judge whether 4 beacon points to be tracked are real beacon points or not through the continuity of the movement of a target between adjacent image sequences, and simultaneously construct the movement track of each beacon, which is the supplement of a multi-target identification step;
the multi-target track prediction means predicting the displacement of the target at the next moment according to the speed and the displacement of the target at the current moment;
the multi-target locking tracking means that the centroid coordinate of each beacon at the current moment is extracted in real time, and the motion track of each beacon is established;
the target loss recapture means that when the target is lost, the tracking algorithm can continuously lock and track the remaining target points, and the remaining target points are used for photogrammetry to calculate the three-dimensional space position and posture of the object, as long as the sum of the number of the remaining beacons in the two cameras is more than or equal to 4; when the lost beacon appears again in the image, the target loss recapture method can immediately capture the lost beacon and add the lost beacon into an equation set of the photogrammetry, so that the equation solving precision is improved, and meanwhile, the calculation precision of the position and the posture of the object is improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910738024.XA CN110490903B (en) | 2019-08-12 | 2019-08-12 | Multi-target rapid capturing and tracking method in binocular vision measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910738024.XA CN110490903B (en) | 2019-08-12 | 2019-08-12 | Multi-target rapid capturing and tracking method in binocular vision measurement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110490903A CN110490903A (en) | 2019-11-22 |
| CN110490903B true CN110490903B (en) | 2022-11-11 |
Family
ID=68550618
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910738024.XA Active CN110490903B (en) | 2019-08-12 | 2019-08-12 | Multi-target rapid capturing and tracking method in binocular vision measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110490903B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111724445B (en) * | 2020-05-08 | 2023-05-26 | 华中科技大学 | Method and system for identifying large-field small-size identification code |
| CN113327244A (en) * | 2021-06-25 | 2021-08-31 | 南京爱奇艺智能科技有限公司 | Handle controller LED lamp positioning method and system based on computer vision |
| CN114299112B (en) * | 2021-12-24 | 2023-01-13 | 萱闱(北京)生物科技有限公司 | Multi-target-based track identification method, device, medium and computing equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007008715A2 (en) * | 2005-07-07 | 2007-01-18 | Ingenious Targeting Laboratory, Inc. | System for 3d monitoring and analysis of motion behavior of targets |
| CN101609504A (en) * | 2009-07-21 | 2009-12-23 | 华中科技大学 | A kind of method for detecting, distinguishing and locating infrared imagery sea-surface target |
| CN102141398A (en) * | 2010-12-28 | 2011-08-03 | 北京航空航天大学 | Monocular vision-based method for measuring positions and postures of multiple robots |
| CN103839274A (en) * | 2014-03-25 | 2014-06-04 | 中国科学院光电技术研究所 | Extension target tracking method based on geometric proportion relation |
| CN107909604A (en) * | 2017-11-07 | 2018-04-13 | 武汉科技大学 | Dynamic object movement locus recognition methods based on binocular vision |
| CN108731587A (en) * | 2017-04-14 | 2018-11-02 | 中交遥感载荷（北京）科技有限公司 | A kind of the unmanned plane dynamic target tracking and localization method of view-based access control model |
| CN109978851A (en) * | 2019-03-22 | 2019-07-05 | 北京航空航天大学 | A kind of aerial weak moving target detection tracking of infrared video |
| CN110009682A (en) * | 2019-03-29 | 2019-07-12 | 北京理工大学 | A kind of object recognition and detection method based on monocular vision |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10204279B2 (en) * | 2015-07-30 | 2019-02-12 | Magna Electronics Inc. | Vehicle vision system with object detection |
-
2019
- 2019-08-12 CN CN201910738024.XA patent/CN110490903B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007008715A2 (en) * | 2005-07-07 | 2007-01-18 | Ingenious Targeting Laboratory, Inc. | System for 3d monitoring and analysis of motion behavior of targets |
| CN101609504A (en) * | 2009-07-21 | 2009-12-23 | 华中科技大学 | A kind of method for detecting, distinguishing and locating infrared imagery sea-surface target |
| CN102141398A (en) * | 2010-12-28 | 2011-08-03 | 北京航空航天大学 | Monocular vision-based method for measuring positions and postures of multiple robots |
| CN103839274A (en) * | 2014-03-25 | 2014-06-04 | 中国科学院光电技术研究所 | Extension target tracking method based on geometric proportion relation |
| CN108731587A (en) * | 2017-04-14 | 2018-11-02 | 中交遥感载荷（北京）科技有限公司 | A kind of the unmanned plane dynamic target tracking and localization method of view-based access control model |
| CN107909604A (en) * | 2017-11-07 | 2018-04-13 | 武汉科技大学 | Dynamic object movement locus recognition methods based on binocular vision |
| CN109978851A (en) * | 2019-03-22 | 2019-07-05 | 北京航空航天大学 | A kind of aerial weak moving target detection tracking of infrared video |
| CN110009682A (en) * | 2019-03-29 | 2019-07-12 | 北京理工大学 | A kind of object recognition and detection method based on monocular vision |
Non-Patent Citations (5)
| Title |
|---|
| Research on Target Tracking Algorithm Based on Parallel Binocular Camera;Ya’nan Xu等;《2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)》;20190805;第1483-1486页 * |
| 基于双目立体视觉的多目标检测跟踪算法研究;张芳冰;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20190215(第02期);I138-2096 * |
| 基于立体视觉的多目标跟踪方法研究;杨昱;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20110415(第04期);I138-1003 * |
| 复杂背景中的多目标检测与跟踪;刘雨辰;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20151215(第12期);I138-481 * |
| 特征融合的核相关滤波跟踪算法;吴昭童等;《国外电子测量技术》;20190215;第40卷(第01期);第108-111页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110490903A (en) | 2019-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Geiger et al. | Are we ready for autonomous driving? the kitti vision benchmark suite | |
| CN103325112B (en) | Moving target method for quick in dynamic scene | |
| Garcia et al. | Positioning an underwater vehicle through image mosaicking | |
| CN103149939B (en) | A kind of unmanned plane dynamic target tracking of view-based access control model and localization method | |
| CN110490903B (en) | Multi-target rapid capturing and tracking method in binocular vision measurement | |
| CN108981672A (en) | Hatch door real-time location method based on monocular robot in conjunction with distance measuring sensor | |
| CN101344965A (en) | Tracking system based on binocular camera shooting | |
| CN108597009A (en) | A method of objective detection is carried out based on direction angle information | |
| CN105160649A (en) | Multi-target tracking method and system based on kernel function unsupervised clustering | |
| CN113223045A (en) | Vision and IMU sensor fusion positioning system based on dynamic object semantic segmentation | |
| Kitt et al. | Detection and tracking of independently moving objects in urban environments | |
| Su et al. | Real-time dynamic SLAM algorithm based on deep learning | |
| Wang et al. | Monocular visual SLAM algorithm for autonomous vessel sailing in harbor area | |
| Tang et al. | Fmd stereo slam: Fusing mvg and direct formulation towards accurate and fast stereo slam | |
| Zhu et al. | A review of 6d object pose estimation | |
| CN112802196B (en) | Binocular inertia simultaneous positioning and map construction method based on dotted line feature fusion | |
| Chavez et al. | Adaptive navigation scheme for optimal deep-sea localization using multimodal perception cues | |
| CN116468786B (en) | Semantic SLAM method based on point-line combination and oriented to dynamic environment | |
| CN112393721A (en) | Camera pose estimation method | |
| CN116862832A (en) | Three-dimensional live-action model-based operator positioning method | |
| Cao et al. | Automatic geo-registration for port surveillance | |
| Yamashita et al. | Robust sensing against bubble noises in aquatic environments with a stereo vision system | |
| Omar et al. | Detection and localization of traffic lights using yolov3 and stereo vision | |
| CN115272904A (en) | System and method for real-time target tracking and distance measurement under multiple monocular cameras | |
| Fang et al. | Direct Monocular Visual Odometry Based on Lidar Vision Fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |