CN112070805B - Motor vehicle target real-time image tracking device and method - Google Patents

Motor vehicle target real-time image tracking device and method Download PDF

Info

Publication number
CN112070805B
CN112070805B CN202010947257.3A CN202010947257A CN112070805B CN 112070805 B CN112070805 B CN 112070805B CN 202010947257 A CN202010947257 A CN 202010947257A CN 112070805 B CN112070805 B CN 112070805B
Authority
CN
China
Prior art keywords
motor vehicle
target
determining
characteristic
actual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010947257.3A
Other languages
Chinese (zh)
Other versions
CN112070805A (en
Inventor
罗小平
童文超
蔡军
曾峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Longhorn Automobile Electron Equipment Co ltd
Original Assignee
Shenzhen Longhorn Automobile Electron Equipment Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Longhorn Automobile Electron Equipment Co ltd filed Critical Shenzhen Longhorn Automobile Electron Equipment Co ltd
Priority to CN202010947257.3A priority Critical patent/CN112070805B/en
Publication of CN112070805A publication Critical patent/CN112070805A/en
Application granted granted Critical
Publication of CN112070805B publication Critical patent/CN112070805B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/168Segmentation; Edge detection involving transform domain methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/66Analysis of geometric attributes of image moments or centre of gravity
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20048Transform domain processing
    • G06T2207/20061Hough transform
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20164Salient point detection; Corner detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30236Traffic on road, railway or crossing
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30241Trajectory

Abstract

The embodiment of the invention provides a device and a method for tracking a motor vehicle target in real time, wherein the device comprises: the target detection module is used for receiving original image frames acquired and transmitted by an image acquisition device of the motor vehicle, analyzing whether the original image frames contain motor vehicle targets frame by frame and determining model image frames and areas to be identified; the modeling module is used for calculating Zernike moments and Harris-Affine angular point characteristics in the area to be identified, determining an actual characteristic model of the motor vehicle target and calculating a target generalized Hough model of the contour of the motor vehicle target; the candidate region screening module is used for predicting a candidate target region of the motor vehicle target in each frame behind the model image frame; the target area determining module is used for judging the similarity between the actual generalized Hough model and the target generalized Hough model of the object outline in the candidate target area and determining the optimal target area; and the target track planning module is used for determining track points of the motor vehicle target, and planning and correcting a track route of the motor vehicle target in real time. The embodiment can accurately track the motor vehicle target.

Description

Motor vehicle target real-time image tracking device and method
Technical Field
The embodiment of the invention relates to the technical field of safe driving of motor vehicles, in particular to a real-time image tracking device and method for a motor vehicle target.
Background
In order to realize safe driving of the existing motor vehicle, a motor vehicle target is usually detected in an image frame of the environment around the motor vehicle, then the motor vehicle target is tracked in a real-time image to prompt a driver of the motor vehicle target around, so that traffic accidents are avoided, and the tracking of the motor vehicle target specifically means that the position of the motor vehicle target is determined in each image frame, and the same target in different image frames corresponds.
The existing target tracking method based on computer vision mainly comprises a visual target similarity measurement and a visual target motion track retrieval algorithm model, wherein the two methods firstly need to perform feature extraction in an image frame and then realize the tracking of a motor vehicle target by designing an effective feature detection model, but in an open environment, the image frame of the surrounding environment of a motor vehicle has a lot of interferences (such as illumination, shadow, shielding or imaging angles and the like), and the interferences influence the feature extraction process to a certain extent; in addition, the lens of the existing vehicle-mounted camera often adopts a wide-angle lens or a fisheye lens for obtaining a larger visual field, the lens causes a shot scene target to have larger compression and distortion due to an imaging principle, at present, the target tracking method often adopts image correction or defines a region with smaller distortion for target detection and tracking for correcting distortion, but the two modes lose a large visual angle scene of the fisheye or the wide-angle lens, and when the fisheye lens is used for shooting and acquiring image frames around a motor vehicle, the motor vehicle does not accord with rigid motion rules due to different distortion degrees of the motor vehicle target under different angles, finally, the target center is drifted in the image processing process, and the target detection and tracking accuracy is lower.
Disclosure of Invention
The technical problem to be solved by the embodiment of the invention is to provide a real-time image tracking device for a motor vehicle target, which can accurately and effectively track the motor vehicle target.
The embodiment of the invention further aims to solve the technical problem of providing a real-time image tracking method for a motor vehicle target, which can accurately and effectively track the motor vehicle target.
In order to solve the above technical problem, an embodiment of the present invention provides the following technical solutions: an apparatus for real-time image tracking of a motor vehicle target, comprising:
the target detection module is used for receiving original image frames acquired and transmitted by an image acquisition device of a motor vehicle and analyzing whether the original image frames contain motor vehicle targets or not frame by frame, determining the original image frames with the motor vehicle targets detected firstly as model image frames, and determining the minimum external quadrilateral area of the motor vehicle targets in the model image frames as an area to be identified;
the modeling module is used for calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be recognized of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie angular point characteristics and a motor vehicle model base trained in advance, and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model;
a candidate region screening module for predicting a candidate target region in which the motor vehicle target meets a predetermined occurrence probability threshold in each of subsequent frames of the model image frame by using a mean shift prediction model;
the target area determining module is used for calculating an actual generalized Hough model of all object outlines in each candidate target area, comparing and judging the similarity between the actual generalized Hough model and the target generalized Hough model, and determining the candidate target area meeting a preset screening rule as an optimal target area, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value; and
and the target track planning module is used for determining the central point of the optimal target area as a track point of the motor vehicle target, and planning and correcting a track route of the motor vehicle target in real time according to the track point.
Further, the motor vehicle model base is a characteristic function equation obtained by performing classification statistics on different motor vehicle databases based on a classifier, and the independent variables of the characteristic function equation are characteristic coefficients and characteristic values of characteristic vectors which satisfy a predetermined identification contribution rate for classified motor vehicles in the motor vehicle databases.
Further, the modeling module includes:
the pre-storage unit is pre-stored with a pre-trained motor vehicle model library;
the characteristic extraction unit is used for calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be identified of the model image frame;
the characteristic model determining unit is used for determining an actual characteristic model of the motor vehicle target according to the Zernike moment, the Harris-Affinie corner characteristic and the motor vehicle model base; and
and the contour model determining unit is used for calculating an object generalized Hough model of the contour of the motor vehicle object according to the actual characteristic model.
Further, the target area determination module includes:
the actual model calculation unit is used for calculating actual generalized Hough models of all object outlines in each candidate target area; and
and the comparison unit is used for comparing and judging the similarity between the actual generalized Hough model and the target generalized Hough model, and determining a candidate target region meeting a preset screening rule as a preferred target region, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value.
Further, the predetermined filtering rule further includes: the candidate target region satisfies a bias threshold for the motor vehicle target prediction.
On the other hand, in order to solve the further technical problem, the embodiment of the present invention provides the following technical solutions: a real-time image tracking method for a motor vehicle target comprises the following steps:
receiving original image frames acquired and transmitted by an image acquisition device of a motor vehicle and analyzing whether the original image frames contain a motor vehicle target frame by frame, determining the original image frames in which the motor vehicle target is detected at first as model image frames, and determining a minimum external quadrilateral area of the motor vehicle target in the model image frames as an area to be identified;
calculating Zernike moments and Harris-Affinie angular point characteristics in a region to be identified of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie angular point characteristics and a pre-trained motor vehicle model base, and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model;
predicting a candidate target region of the motor vehicle target satisfying a predetermined occurrence probability threshold in each of subsequent frames of the model image frame by using a mean shift prediction model;
calculating actual generalized Hough models of all objects in each candidate target region, comparing and judging the similarity between the actual generalized Hough models and the target generalized Hough models, and determining the candidate target region meeting a preset screening rule as a preferred target region, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value; and
and determining the central point of the optimal target area as a track point of the motor vehicle target, and planning and correcting a track route of the motor vehicle target in real time according to the track point.
Further, the motor vehicle model base is a characteristic function equation obtained by performing classification statistics on different motor vehicle databases based on a classifier, and the independent variables of the characteristic function equation are characteristic coefficients and characteristic values of characteristic vectors which satisfy a predetermined identification contribution rate for classified motor vehicles in the motor vehicle databases.
Further, the calculating Zernike moment and Harris-Affine angular point characteristics in the region to be recognized of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moment and Harris-Affine angular point characteristics and a motor vehicle model base trained in advance, and then calculating the target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model specifically includes:
prestoring a motor vehicle model library which is trained in advance;
calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be identified of the model image frame;
determining an actual characteristic model of the motor vehicle target according to the Zernike moment, the Harris-Affinie angular point characteristics and the motor vehicle model base; and
and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model.
Further, the calculating of the actual generalized Hough models of all object profiles in each candidate target region, comparing and judging the similarity between the actual generalized Hough models and the target generalized Hough models, and determining a candidate target region meeting a predetermined screening rule as a preferred target region, where the predetermined screening rule includes that the similarity is the largest and is greater than a preset similarity threshold specifically includes:
calculating actual generalized Hough models of all object contours in each candidate target region; and
and comparing and judging the similarity between the actual generalized Hough model and the target generalized Hough model, and determining a candidate target region meeting a preset screening rule as an optimal target region, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value.
Further, the predetermined filtering rule further includes: the candidate target region satisfies a bias threshold for the motor vehicle target prediction.
After the technical scheme is adopted, the embodiment of the invention at least has the following beneficial effects: the method comprises the steps of receiving an original image frame collected and transmitted by an image collecting device of a motor vehicle through a target detection module, analyzing whether the original image frame comprises a motor vehicle target or not frame by frame, firstly obtaining the original image frame of the surrounding environment of the motor vehicle in real time, detecting the motor vehicle target in the original image frame, then determining a model image frame and a to-be-identified area, then calculating Zernike moments and Harris-Afffine corner features in the to-be-identified area through a modeling module, thereby determining an actual feature model of the motor vehicle target, calculating a target generalized Hough model of the outline of the motor vehicle target, further predicting a candidate target area of the motor vehicle target in each subsequent frame of the model image frame through a candidate area screening module, and then judging the similarity between the actual generalized Hough model and the target generalized Hough model of the outline of an object in the candidate target area through a target area determination module, the optimal target area of the motor vehicle target is determined, the target track planning module determines track points of the motor vehicle target, the track route of the motor vehicle target is planned and corrected in real time, due to the fact that the Zernike moment has scale, rotation and distortion invariance, the influence on the tracking of the motor vehicle target due to the fact that a lens of an image acquisition device of the motor vehicle adopts wide-angle lens or fisheye lens to generate image distortion is avoided, the anti-reflection change of the Zernike moment can be eliminated through anti-reflection conversion normalization by the Harris-Affini angular point characteristic, the normalized angular point has anti-reflection invariance, the position of the motor vehicle target can be effectively and accurately positioned, and accurate and effective tracking can be achieved.
Drawings
Fig. 1 is a schematic block diagram of an alternative embodiment of the real-time tracking device for the target of the motor vehicle of the present invention.
FIG. 2 is a block diagram of a modeling module of an alternative embodiment of the real-time image tracking apparatus for a motor vehicle target according to the present invention.
Fig. 3 is a specific schematic block diagram of a target area determination module according to an alternative embodiment of the real-time image tracking device for the motor vehicle target of the present invention.
FIG. 4 is a flowchart illustrating steps of an alternative embodiment of a method for tracking a target of a motor vehicle in real time.
Fig. 5 is a detailed flowchart of step S2 of an alternative embodiment of the method for tracking a real-time image of a motor vehicle target according to the present invention.
Fig. 6 is a detailed flowchart of step S4 of an alternative embodiment of the method for tracking a real-time image of a motor vehicle target according to the present invention.
Detailed Description
The present application will now be described in further detail with reference to the accompanying drawings and specific examples. It should be understood that the following illustrative embodiments and description are only intended to explain the present invention, and are not intended to limit the present invention, and features of the embodiments and examples in the present application may be combined with each other without conflict.
As shown in fig. 1, an alternative embodiment of the present invention provides a real-time image tracking apparatus for a motor vehicle target, comprising: the target detection module 1 is used for receiving original image frames acquired and transmitted by an image acquisition device of a motor vehicle and analyzing whether the original image frames contain motor vehicle targets or not frame by frame, determining the original image frames with the motor vehicle targets detected firstly as model image frames, and determining the minimum external quadrilateral area of the motor vehicle targets in the model image frames as an area to be identified;
the modeling module 3 is used for calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be recognized of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie angular point characteristics and a motor vehicle model base trained in advance, and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model;
a candidate region screening module 5, configured to predict, by using a mean shift prediction model, a candidate target region where the motor vehicle target meets a predetermined occurrence probability threshold in each of subsequent frames of the model image frame;
the target area determining module 7 is configured to calculate an actual generalized Hough model of all object contours in each candidate target area, compare and judge similarity between the actual generalized Hough model and the target generalized Hough model, and determine a candidate target area meeting a predetermined screening rule as an optimal target area, where the predetermined screening rule includes that the similarity is the largest and is greater than a preset similarity threshold; and
and the target track planning module 9 is configured to determine the central point of the preferred target area as a track point of the motor vehicle target, and plan and correct a track route of the motor vehicle target in real time according to the track point.
The embodiment of the invention receives an original image frame acquired and transmitted by an image acquisition device of a motor vehicle through a target detection module and analyzes whether the original image frame contains a motor vehicle target or not frame by frame, the motor vehicle target is detected in the original image frame, then a model image frame and a region to be identified are determined, then a modeling module 3 calculates Zernike moment and Harris-Affine angular point characteristics in the region to be identified, thereby determining an actual characteristic model of the motor vehicle target, calculating a target generalized Hough model of the contour of the motor vehicle target, further a candidate region screening module 5 predicts a candidate target region of the motor vehicle target in each frame following the image frame, a target region determining module 7 judges the similarity of the actual generalized Hough model and the target generalized Hough model of the contour of an object in the candidate target region, determines a preferred target region of the motor vehicle target, and finally a target trajectory planning module 9 determines a trajectory point of the motor vehicle target, planning in real time and revising the orbit route of motor vehicle target, because the Zernike matrix possesses yardstick, rotation, distortion invariance, can not adopt wide-angle lens or the produced image distortion of fisheye lens and produce the influence to the tracking of motor vehicle target because the camera lens of the image acquisition device of motor vehicle, and Harris-Affini angular point characteristic can be through imitating the reflection change of converting normalization elimination Zernike matrix and imitating the reflection change for the angular point of normalization possesses the reflection invariance, can effective accurate location the position of motor vehicle target, the realization can be accurate, effectual tracking. In a specific implementation, the image acquisition device of the motor vehicle can be a vehicle-mounted camera.
In particular implementation, the Zernike moments are calculated as:x2+y21 or less, wherein A isn,mThe (n + m) -order Zernike moments, f (x, y) is the pixel value of the image projected onto the unit circle corresponding to the (x, y) coordinate,is the conjugate of Zernike polynomial, theta is the angle between the connecting line of the coordinate (X, y) and the origin of the coordinate and the positive direction of the X axis, the range is 0 DEG to 360 DEG, N is an integer greater than or equal to 0, m satisfies | m | to N, ρ is the distance between the coordinate (X, y) and the origin of the coordinate, the range is 0 DEG to 1, and the formula is as follows: vn,m(x,y)=Vn,m(ρ,θ)=Rn,m(ρ) × exp (jm θ), wherein, Rn,m(ρ) is a radius polynomial, which can be obtained according to the calculation formula:wherein the Zernike moment A is due to(m,n)Is a vector value, and thus, after taking the Zernike moment modulo, is Z(m,n)=|A(m,n)If it is, then it has rotation invariance, then, it is aligned againStandardizing the mould with a processing formula of Z(n,m)_normal=Z(n,m)/m(0,0)Wherein m is(0,0)The Zernike moment of (0, 0) order has scale invariance, and the Z is obtained by integrating the above operation formulas(n,m)_normalRepresenting Zernike moment features of the region to be identified.
In addition, the Harris-Affini angular point characteristics can be obtained by detecting angular point calculation based on a Harris-Laplace algorithm, then calculating angular point neighborhood secondary moment, estimating reflection modeling transformation, normalizing the field, then solving the angular point of the normalized field, finally calculating the characteristic value of the current angular point secondary moment matrix, if the characteristic values are not equal, recalculating the angular point neighborhood secondary moment, and circulating in sequence, so that a plurality of Harris-Affini angular points of a target can be obtained according to the method: (X0, Y0) - (Xn, Yn), then connecting all corner points of the tracked target sub-block region to form a maximum circumscribed polygon, and then calculating Zernike moment characteristic Z for the maximum circumscribed polygon as a contourHAFinally, Z isHAAnd original Zernike moment characteristics and the like are used as weights and input into a classifier for training.
Finally, when calculating the generalized Hough model, the reference center of the contour may be first set as (Xc, Yc), and the boundary point position of the contour (X _ edge, Y _ edge), so that the distance R from the center line to the boundary point when the contour radius of the current point is the center line can be represented by the following formula:setting the gradient vector of the contour of the current point as V _ edge, the vector of the center point and the contour point as V _ c2e, the included angle between the gradient vector and the vector of the center point and the contour point as theta, the boundary point of the contour can be expressed as: since (X _ edge, Y _ edge) ═ f (R, θ, Xc, Yc), all contour points can be represented as a look-up function (R-Table) of variables Xc, Yc, θ and R, which is a generalized Hough transform of the contour.
In an optional embodiment of the present invention, the vehicle model library is a feature function equation obtained by performing classification statistics on different vehicle databases based on a classifier, and the feature function equationIs the characteristic coefficient and the characteristic value of the characteristic vector in the motor vehicle database, which satisfies the preset identification contribution rate to the classified motor vehicles. In this embodiment, a classifier (e.g., AdaBoost classifier) may be first used to train a motor vehicle target database based on Zernike moments and Harris-Affine corner features, then the classifier is used to perform classification statistics on different motor vehicle databases, the classification effects of the classifiers of the different motor vehicle databases are counted, then according to the statistical results, and based on pca (principal component analysis) concept, feature vectors with contribution rates greater than a predetermined recognition contribution rate (e.g., 90%) to vehicle target recognition are selected as the independent variable features of a feature function equation, the combination factors of the features are normalized feature values, and finally a feature function equation is determined:wherein Si is the characteristic coefficient of the ith term characteristic vector, Ai is the characteristic value of the ith term characteristic vector, and b is the offset term.
In yet another alternative embodiment of the present invention, as shown in fig. 2, the modeling module 3 includes:
a pre-storing unit 30, in which a pre-trained model base of the motor vehicle is pre-stored;
the feature extraction unit 32 is configured to calculate Zernike moments and Harris-affinity corner features in the region to be identified of the model image frame;
a feature model determining unit 34, configured to determine an actual feature model of the vehicle target according to the Zernike moment, the Harris-Affine corner feature, and the vehicle model library; and
and the contour model determining unit 36 is configured to calculate an object generalized Hough model of the contour of the motor vehicle object according to the actual feature model.
According to the embodiment of the invention, a motor vehicle model base is pre-stored in a pre-storing unit 30, then a characteristic extracting unit 32 calculates Zernike moments and Harris-Affinie corner features in a region to be identified, a characteristic model determining unit 34 determines an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie corner features and the motor vehicle model base, and finally a contour model determining unit 36 calculates a target generalized Hough model of the contour of the motor vehicle target, so that the execution process is clear, and the data processing efficiency is higher.
In yet another alternative embodiment of the present invention, as shown in fig. 3, the target area determination module 7 includes:
an actual model calculating unit 70, configured to calculate actual generalized Hough models of all object contours in each candidate target region; and
and the comparison unit 72 is configured to compare and judge the similarity between the actual generalized Hough model and the target generalized Hough model, and determine a candidate target region meeting a predetermined screening rule as an optimal target region, where the predetermined screening rule includes that the similarity is the largest and is greater than a preset similarity threshold.
In this embodiment, the actual generalized Hough models of all object profiles in each candidate target region are calculated by the actual model calculation unit 70, the comparison unit 72 compares and judges the similarity between the actual generalized Hough models and the target generalized Hough models, and the candidate target region satisfying the predetermined screening rule is determined as the preferred target region, the predetermined screening rule includes that the similarity is the largest and is greater than the preset similarity threshold, and the candidate target region having the highest similarity is determined as the preferred target region, so that the matching of the template is realized, and the accuracy of target tracking and identification is ensured.
In another optional embodiment of the present invention, the predetermined filtering rule further comprises: the candidate target region satisfies a bias threshold for the motor vehicle target prediction. In the embodiment, by adding the screening condition, the candidate target region also needs to meet the offset threshold value of the motor vehicle target prediction, so that the candidate target region is prevented from deviating from the predicted offset threshold value seriously, and because the motor vehicle cannot be in instantaneous reversal, the screening accuracy is improved. In a specific implementation, the predicted deviation threshold value can be generated by combining historical motion data prediction of a current motor vehicle target, or can be artificially set according to motion habits of the motor vehicle.
On the other hand, as shown in fig. 4, an embodiment of the present invention provides a method for tracking a target of a motor vehicle in real time, including the following steps:
s1: receiving original image frames acquired and transmitted by an image acquisition device of a motor vehicle and analyzing whether the original image frames contain a motor vehicle target frame by frame, determining the original image frames in which the motor vehicle target is detected at first as model image frames, and determining a minimum external quadrilateral area of the motor vehicle target in the model image frames as an area to be identified;
s2: calculating Zernike moments and Harris-Affinie angular point characteristics in a region to be identified of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie angular point characteristics and a pre-trained motor vehicle model base, and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model;
s3: predicting a candidate target region of the motor vehicle target satisfying a predetermined occurrence probability threshold in each of subsequent frames of the model image frame by using a mean shift prediction model;
s4: calculating actual generalized Hough models of all objects in each candidate target region, comparing and judging the similarity between the actual generalized Hough models and the target generalized Hough models, and determining the candidate target region meeting a preset screening rule as a preferred target region, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value; and
s5: and determining the central point of the optimal target area as a track point of the motor vehicle target, and planning and correcting a track route of the motor vehicle target in real time according to the track point.
The embodiment of the invention firstly receives an original image frame collected and transmitted by an image collecting device of a motor vehicle, detects a motor vehicle target in the original image frame, then determines a model image frame and a region to be identified, then calculates Zernike moment and Harris-Affinie angular point characteristics in the region to be identified, thereby determining an actual characteristic model of the motor vehicle target, calculates a target generalized Hough model of the contour of the motor vehicle target, further predicts a candidate target region of the motor vehicle target in each frame of the model image frame, then judges the similarity between the actual generalized Hough model and the target generalized Hough model of the object contour in the candidate target region, determines a preferred target region of the motor vehicle target, finally determines track points of the motor vehicle target, plans and corrects a track route of the motor vehicle target in real time, because the Zernike moment has scale, rotation and distortion invariability, the lens of the image acquisition device of the motor vehicle does not influence the tracking of a motor vehicle target due to the fact that the wide-angle lens or the fisheye lens generates image distortion, and Harris-Affini angular point characteristics can eliminate the reflection-simulating change of a Zernike matrix through reflection-simulating transformation normalization, so that the normalized angular point has reflection-simulating invariance, the position of the motor vehicle target can be effectively and accurately positioned, and accurate and effective tracking can be achieved.
In yet another optional embodiment of the present invention, the vehicle model library is a feature function equation obtained by performing classification statistics on different vehicle databases based on a classifier, and the independent variables of the feature function equation are feature coefficients and feature values of feature vectors in the vehicle database that satisfy a predetermined identification contribution rate for classifying the vehicle. In this embodiment, a classifier (e.g., AdaBoost classifier) may be first used to train a motor vehicle target database based on Zernike moments and Harris-Affine corner features, then the classifier is used to perform classification statistics on different motor vehicle databases, the classification effects of the classifiers of the different motor vehicle databases are counted, then according to the statistical results, and based on pca (principal component analysis) concept, feature vectors with contribution rates greater than a predetermined recognition contribution rate (e.g., 90%) to vehicle target recognition are selected as the independent variable features of a feature function equation, the combination factors of the features are normalized feature values, and finally a feature function equation is determined:wherein Si is the characteristic coefficient of the ith term characteristic vector, Ai is the characteristic value of the ith term characteristic vector, and b is the offset term.
In yet another alternative embodiment of the present invention, as shown in fig. 5, the step S2 specifically includes:
s21: prestoring a motor vehicle model library which is trained in advance;
s22: calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be identified of the model image frame;
s23: determining an actual characteristic model of the motor vehicle target according to the Zernike moment, the Harris-Affinie angular point characteristics and the motor vehicle model base; and
s24: and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model.
According to the method, firstly, a motor vehicle model base is prestored, then Zernike moments and Harris-Affinie angular point characteristics in the area to be identified are calculated, then an actual characteristic model of the motor vehicle target is determined according to the Zernike moments and Harris-Affinie angular point characteristics and the motor vehicle model base, and finally a target generalized Hough model of the contour of the motor vehicle target is calculated, so that the execution process is clear, and the data processing efficiency is higher.
In another alternative embodiment of the present invention, as shown in fig. 6, the step S4 specifically includes:
s41: calculating actual generalized Hough models of all object contours in each candidate target region; and
s42: and comparing and judging the similarity between the actual generalized Hough model and the target generalized Hough model, and determining a candidate target region meeting a preset screening rule as an optimal target region, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value.
In this embodiment, by the method, the actual generalized Hough models of all object profiles in each candidate target region are calculated, the similarity between the actual generalized Hough models and the target generalized Hough models is compared and judged, the candidate target region meeting the predetermined screening rule is determined as the preferred target region, the predetermined screening rule includes that the similarity is the largest and is greater than the preset similarity threshold, the candidate target region with the highest similarity is determined as the preferred target region, the matching of the template is realized, and the accuracy of target tracking and identification is ensured.
In an optional embodiment of the present invention, the predetermined filtering rule further comprises: the candidate target region satisfies a bias threshold for the motor vehicle target prediction. In the embodiment, by adding the screening condition, the candidate target region also needs to meet the offset threshold value of the motor vehicle target prediction, so that the candidate target region is prevented from deviating from the predicted offset threshold value seriously, and because the motor vehicle cannot be in instantaneous reversal, the screening accuracy is improved. In a specific implementation, the predicted deviation threshold value can be generated by combining historical motion data prediction of a current motor vehicle target, or can be artificially set according to motion habits of the motor vehicle.
The functions described in the embodiments of the present invention may be stored in a storage medium readable by a computing device if they are implemented in the form of software functional modules or units and sold or used as independent products. Based on such understanding, part of the contribution of the embodiments of the present invention to the prior art or part of the technical solution may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device, a network device, or the like) to execute all or part of the steps of the method described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (8)

1. An apparatus for real-time image tracking of a motor vehicle target, the apparatus comprising:
the target detection module is used for receiving original image frames acquired and transmitted by an image acquisition device of a motor vehicle and analyzing whether the original image frames contain motor vehicle targets or not frame by frame, determining the original image frames with the motor vehicle targets detected firstly as model image frames, and determining the minimum external quadrilateral area of the motor vehicle targets in the model image frames as an area to be identified;
the modeling module is used for calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be recognized of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie angular point characteristics and a motor vehicle model base trained in advance, and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model;
a candidate region screening module for predicting a candidate target region in which the motor vehicle target meets a predetermined occurrence probability threshold in each of subsequent frames of the model image frame by using a mean shift prediction model;
the target area determining module is used for calculating an actual generalized Hough model of all object outlines in each candidate target area, comparing and judging the similarity between the actual generalized Hough model and the target generalized Hough model, and determining the candidate target area meeting a preset screening rule as an optimal target area, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value; and
and the target track planning module is used for determining the central point of the optimal target area as a track point of the motor vehicle target, and planning and correcting a track route of the motor vehicle target in real time according to the track point.
2. The device for tracking the target real-time image of the motor vehicle as claimed in claim 1, wherein the motor vehicle model library is a characteristic function equation obtained by performing classification statistics on different motor vehicle databases based on a classifier, and the independent variables of the characteristic function equation are characteristic coefficients and characteristic values of characteristic vectors in the motor vehicle databases which satisfy a predetermined identification contribution rate for classified motor vehicles.
3. The real-time image tracking device of an automotive target as claimed in claim 1, characterized in that said modeling module comprises:
the pre-storage unit is pre-stored with a pre-trained motor vehicle model library;
the characteristic extraction unit is used for calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be identified of the model image frame;
the characteristic model determining unit is used for determining an actual characteristic model of the motor vehicle target according to the Zernike moment, the Harris-Affinie corner characteristic and the motor vehicle model base; and
and the contour model determining unit is used for calculating an object generalized Hough model of the contour of the motor vehicle object according to the actual characteristic model.
4. The real-time image tracking device of the motor vehicle target according to claim 1, wherein the predetermined filtering rule further comprises: the candidate target region satisfies a bias threshold for the motor vehicle target prediction.
5. A method for real-time image tracking of a motor vehicle target, the method comprising the steps of:
receiving original image frames acquired and transmitted by an image acquisition device of a motor vehicle and analyzing whether the original image frames contain a motor vehicle target frame by frame, determining the original image frames in which the motor vehicle target is detected at first as model image frames, and determining a minimum external quadrilateral area of the motor vehicle target in the model image frames as an area to be identified;
calculating Zernike moments and Harris-Affinie angular point characteristics in a region to be identified of the model image frame, determining an actual characteristic model of the motor vehicle target according to the Zernike moments and Harris-Affinie angular point characteristics and a pre-trained motor vehicle model base, and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model;
predicting a candidate target region of the motor vehicle target satisfying a predetermined occurrence probability threshold in each of subsequent frames of the model image frame by using a mean shift prediction model;
calculating actual generalized Hough models of all objects in each candidate target region, comparing and judging the similarity between the actual generalized Hough models and the target generalized Hough models, and determining the candidate target region meeting a preset screening rule as a preferred target region, wherein the preset screening rule comprises that the similarity is maximum and is greater than a preset similarity threshold value; and
and determining the central point of the optimal target area as a track point of the motor vehicle target, and planning and correcting a track route of the motor vehicle target in real time according to the track point.
6. The method for tracking the target real-time image of the motor vehicle as claimed in claim 5, wherein the motor vehicle model library is a characteristic function equation obtained by performing classification statistics on different motor vehicle databases based on a classifier, and the independent variables of the characteristic function equation are characteristic coefficients and characteristic values of characteristic vectors in the motor vehicle databases which satisfy a predetermined identification contribution rate for classified motor vehicles.
7. The method for tracking the real-time image of the motor vehicle target according to claim 5, wherein the calculating Zernike moments and Harris-Affine angular point features in the region to be identified in the model image frame, determining an actual feature model of the motor vehicle target according to the Zernike moments and Harris-Affine angular point features and a pre-trained motor vehicle model library, and then calculating the target generalized Hough model of the contour of the motor vehicle target according to the actual feature model specifically comprises:
prestoring a motor vehicle model library which is trained in advance;
calculating Zernike moments and Harris-Affinie angular point characteristics in the region to be identified of the model image frame;
determining an actual characteristic model of the motor vehicle target according to the Zernike moment, the Harris-Affinie angular point characteristics and the motor vehicle model base; and
and calculating a target generalized Hough model of the contour of the motor vehicle target according to the actual characteristic model.
8. The method for tracking the real-time image of the target of the motor vehicle according to claim 5, wherein the predetermined filtering rule further comprises: the candidate target region satisfies a bias threshold for the motor vehicle target prediction.
CN202010947257.3A 2020-09-10 2020-09-10 Motor vehicle target real-time image tracking device and method Active CN112070805B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010947257.3A CN112070805B (en) 2020-09-10 2020-09-10 Motor vehicle target real-time image tracking device and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010947257.3A CN112070805B (en) 2020-09-10 2020-09-10 Motor vehicle target real-time image tracking device and method

Publications (2)

Publication Number Publication Date
CN112070805A CN112070805A (en) 2020-12-11
CN112070805B true CN112070805B (en) 2021-05-14

Family

ID=73663482

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010947257.3A Active CN112070805B (en) 2020-09-10 2020-09-10 Motor vehicle target real-time image tracking device and method

Country Status (1)

Country Link
CN (1) CN112070805B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102810250A (en) * 2012-07-31 2012-12-05 长安大学 Video based multi-vehicle traffic information detection method

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AT518152T (en) * 2006-12-21 2011-08-15 Selex Galileo Spa MULTI-TONE RADAR RECOGNITION METHOD AND DEVICE
US8170278B2 (en) * 2008-08-06 2012-05-01 Sri International System and method for detecting and tracking an object of interest in spatio-temporal space
CN101901354B (en) * 2010-07-09 2014-08-20 浙江大学 Method for detecting and tracking multi targets at real time in monitoring videotape based on characteristic point classification
CN101976504B (en) * 2010-10-13 2014-01-22 北京航空航天大学 Multi-vehicle video tracking method based on color space information
CN103065144A (en) * 2012-12-30 2013-04-24 信帧电子技术(北京)有限公司 Automobile logo identification method and device
CN106022280A (en) * 2016-05-26 2016-10-12 西安空间无线电技术研究所 Typical ship target identification method based on graded invariance features
CN110009663A (en) * 2019-04-10 2019-07-12 苏州大学 A kind of method for tracking target, device, equipment and computer readable storage medium
CN110516565A (en) * 2019-08-07 2019-11-29 青岛海之声科技有限公司 A kind of sea ship recognition methods based on Zernike and SVM
CN110517291A (en) * 2019-08-27 2019-11-29 南京邮电大学 A kind of road vehicle tracking based on multiple feature spaces fusion
CN111429481A (en) * 2020-03-18 2020-07-17 重庆邮电大学 Target tracking method, device and terminal based on adaptive expression

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102810250A (en) * 2012-07-31 2012-12-05 长安大学 Video based multi-vehicle traffic information detection method

Also Published As

Publication number Publication date
CN112070805A (en) 2020-12-11

Similar Documents

Publication Publication Date Title
Kim Robust lane detection and tracking in challenging scenarios
US9811732B2 (en) Systems and methods for object tracking
Gavrila Traffic sign recognition revisited
US8811744B2 (en) Method for determining frontal face pose
US20070058856A1 (en) Character recoginition in video data
CN107909600A (en) The unmanned plane real time kinematics target classification and detection method of a kind of view-based access control model
CN102567744B (en) Method for determining quality of iris image based on machine learning
CN104200495A (en) Multi-target tracking method in video surveillance
US8908921B2 (en) Object detection method and object detector using the method
US7162085B2 (en) Pattern recognition method and apparatus
CN105373135A (en) Method and system for guiding airplane docking and identifying airplane type based on machine vision
CN109558823B (en) Vehicle identification method and system for searching images by images
CN108446634B (en) Aircraft continuous tracking method based on combination of video analysis and positioning information
CN101320477B (en) Human body tracing method and equipment thereof
US20070223785A1 (en) Image processor and method
CN111311540A (en) Vehicle damage assessment method and device, computer equipment and storage medium
Liu et al. Dynamic vehicle detection with sparse point clouds based on PE-CPD
CN112070805B (en) Motor vehicle target real-time image tracking device and method
KR101050687B1 (en) Pedestrian recognition device and method using posterior probability and pedestrian protection device using it
Carneiro et al. The distinctiveness, detectability, and robustness of local image features
CN111862147A (en) Method for tracking multiple vehicles and multiple human targets in video
CN111932580A (en) Road 3D vehicle tracking method and system based on Kalman filtering and Hungary algorithm
CN103996207A (en) Object tracking method
Ying et al. Technique of measuring leading vehicle distance based on digital image processing theory
CN110569865A (en) Method and device for recognizing vehicle body direction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant