CN110825123A - Control system and method for automatic following loading vehicle based on motion algorithm - Google Patents
Control system and method for automatic following loading vehicle based on motion algorithm Download PDFInfo
- Publication number
- CN110825123A CN110825123A CN201910997269.4A CN201910997269A CN110825123A CN 110825123 A CN110825123 A CN 110825123A CN 201910997269 A CN201910997269 A CN 201910997269A CN 110825123 A CN110825123 A CN 110825123A
- Authority
- CN
- China
- Prior art keywords
- moving target
- point
- motion
- moving
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 97
- 238000000034 method Methods 0.000 title claims abstract description 71
- 239000011159 matrix material Substances 0.000 claims abstract description 15
- 230000003068 static effect Effects 0.000 claims abstract description 12
- 238000001914 filtration Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 8
- 230000009466 transformation Effects 0.000 claims description 24
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 22
- 238000001514 detection method Methods 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 10
- 238000000746 purification Methods 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 6
- 230000007797 corrosion Effects 0.000 claims description 5
- 238000005260 corrosion Methods 0.000 claims description 5
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 240000004282 Grewia occidentalis Species 0.000 claims description 4
- 238000005286 illumination Methods 0.000 claims description 3
- 238000000926 separation method Methods 0.000 claims description 3
- 230000000052 comparative effect Effects 0.000 claims description 2
- 238000002474 experimental method Methods 0.000 claims description 2
- 238000009499 grossing Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 230000004660 morphological change Effects 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 238000011410 subtraction method Methods 0.000 claims description 2
- 238000013519 translation Methods 0.000 claims description 2
- SAZUGELZHZOXHB-UHFFFAOYSA-N acecarbromal Chemical compound CCC(Br)(CC)C(=O)NC(=O)NC(C)=O SAZUGELZHZOXHB-UHFFFAOYSA-N 0.000 claims 1
- 238000002360 preparation method Methods 0.000 claims 1
- 230000007547 defect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 235000002566 Capsicum Nutrition 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/12—Target-seeking control
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
A control system and a method for an automatic following loading vehicle based on a motion algorithm belong to an automatic tracking method. The existing automatic tracking method has the defects of easy interference, poor tracking accuracy and the like, and a person to be tracked needs to carry a signal receiver. A control method of an automatic following carrier vehicle based on a motion algorithm is characterized in that a global motion model is established and unknown parameters in a motion model matrix are solved; performing motion compensation on the current frame image by using the obtained model to make the background static; solving a difference image to extract a moving target; obtaining a complete moving target area through an image processing means; carrying out threshold segmentation on the moving target and the background; by utilizing a method of combining a moving target tracking algorithm with a Kalman filtering algorithm, the carrier vehicle is adjusted along with the movement of the moving target so as to accurately follow the moving target. The invention tracks the moving target according to the moving condition of the moving target mass center, can realize automatic tracking of the target in a complex environment, can thoroughly liberate the hands of people and improve the working efficiency.
Description
Technical Field
The invention relates to a control system and method of an automatic following loading vehicle based on a motion algorithm.
Background
At present, the research on intelligent mobile vehicles in China gradually matures, and national intelligent vehicle competitions held in China every year, such as a Feishakaer intelligent vehicle competition, a RobotCup competition and the like, attract hundreds of representative teams of dozens of colleges to participate in the competition. The mobile robot is developing towards the direction of intellectualization, automation and diversification, and the research on documents shows that in recent years, the growth speed of the market of the intelligent mobile car in China is always in the first position in the world, and in 2014, China has become the country with the largest market sales volume of the intelligent mobile car, however, compared with the countries such as the U.S., the Sun and the Korean, the density of the intelligent mobile car in China is relatively low, most intelligent mobile car companies mainly consider the aspects of autonomous tracing, detection and rescue, high-altitude operation and the like, the research on the aspects of carrying and automatic following is less, the development space is huge, and therefore, the research on the automatic following carrying car with low cost, simple operation, easy installation, strong real-time performance and high tracking accuracy has great practical significance. The existing automatic tracking method has the defects of easy interference, poor tracking accuracy and the like, and a person to be tracked needs to carry a signal receiver.
Disclosure of Invention
The invention aims to solve the problems that the existing automatic tracking method is easy to interfere, poor in tracking accuracy and the like, and a follower needs to carry a signal receiver, and provides a control system and a method of an automatic following loading vehicle based on a motion algorithm.
A control system of an automatic following loading vehicle based on a motion algorithm comprises the following components: the device comprises a Cortex-A9 processor, a video acquisition circuit, an alarm circuit, a motor speed regulation driving circuit, a power supply circuit, a storage battery electric quantity detection circuit, a TF card interface circuit, SDRAM, NAND FLASH and NOR FLASH, wherein the Cortex-A9 processor is respectively connected with the video acquisition circuit, the alarm circuit, the motor speed regulation driving circuit, the power supply circuit, the storage battery electric quantity detection circuit, the TF card interface circuit, the SDRAM, NANDFLASH and NOR FLASH, the video acquisition circuit is connected with a camera, the power supply circuit is connected with the storage battery, the storage battery is connected with the storage battery electric quantity detection circuit, and the motor speed regulation driving circuit is connected with a group of motors.
A control method of an automatic following loading vehicle based on a motion algorithm is realized by the following steps:
firstly, an affine transformation model with six degrees of freedom is selected to establish a camera global motion model, and unknown parameters in a motion model matrix are solved according to the position relation of a characteristic region between two frames, namely the motion model is solved;
secondly, performing motion compensation on the current frame image by using the obtained affine transformation model to make the background static;
step three, solving a differential image by using four-frame differential under a static background and combining background subtraction to extract a moving target;
step four, obtaining a complete moving target area through morphological image processing means such as corrosion expansion and the like;
step five, carrying out threshold segmentation on the moving target and the background, solving a mass center according to the pixel information of the moving target, further determining four-corner coordinates of a rectangular frame of the moving target, and carrying out screenshot on the next frame according to the four-corner coordinates;
and sixthly, properly adjusting the carrier vehicle along with the movement of the moving target by utilizing a method of combining a moving target tracking algorithm with a Kalman filtering algorithm so as to accurately follow the moving target.
The invention has the beneficial effects that:
the invention relates to a control method of an automatic following loading vehicle based on a motion algorithm, which comprises the steps of establishing a global motion model and solving unknown parameters in a motion model matrix; performing motion compensation on the current frame image by using the obtained model to make the background static; solving a difference image to extract a moving target; obtaining a complete moving target area through an image processing means; carrying out threshold segmentation on the moving target and the background; by utilizing a method of combining a moving target tracking algorithm with a Kalman filtering algorithm, the carrier vehicle is adjusted along with the movement of the moving target so as to accurately follow the moving target. The invention tracks the moving target according to the moving condition of the moving target mass center, can realize automatic tracking of the target in a complex environment, can thoroughly liberate the hands of people and improve the working efficiency.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a diagram of an interframe difference method according to the present invention;
FIG. 3 is a diagram of the variation of the coordinates of the centroid of the moving object according to the present invention;
FIG. 4 is a plan view of the moving object of the present invention;
FIG. 5 is a schematic diagram of a control system of an automatic following loading vehicle based on a motion algorithm according to the present invention;
FIG. 6 is a power supply circuit according to the present invention;
FIG. 7 is a motor speed regulation driving circuit according to the present invention;
FIG. 8 is a circuit for detecting the amount of charge in a battery according to the present invention;
fig. 9 is a video capture circuit.
Detailed Description
The first embodiment is as follows:
the control system of the automatic following loading vehicle based on the motion algorithm in the embodiment is shown in fig. 5 to 9, and comprises: the device comprises a Cortex-A9 processor 1, a video acquisition circuit 2, an alarm circuit 3, a motor speed regulation driving circuit 4, a power supply circuit 5, a storage battery 6, a storage battery electric quantity detection circuit 7, a TF card interface circuit 8, SDRAM (the reference number in the figure is 9), NAND FLASH (the reference number in the figure is 10), and NOR FLASH (the reference number in the figure is 11), wherein the Cortex-A9 processor 1 is respectively connected with the video acquisition circuit 2, the alarm circuit 3, the motor speed regulation driving circuit 4, the power supply circuit 5, the storage battery electric quantity detection circuit 7, the TF card interface circuit 8, SDRAM (the reference number in the figure is 9), NAND FLASH (the reference number in the figure is 10), and the NOR FLASH (the reference number in the figure is 11), the video acquisition circuit 2 is connected with a camera 12, the power supply circuit 5 is connected with the storage battery 6, the storage battery 6 is connected with the storage battery electric quantity detection circuit 7, and the motor driving circuit 4 is.
The second embodiment is as follows:
in the embodiment, as shown in fig. 1, the method for controlling the automatic following loading vehicle based on the motion algorithm is implemented by the following steps:
firstly, an affine transformation model with six degrees of freedom is selected to establish a camera global motion model, and unknown parameters in a motion model matrix are solved according to the position relation of a characteristic region between two frames, namely the motion model is solved;
secondly, performing motion compensation on the current frame image by using the obtained affine transformation model to make the background static;
step three, solving a differential image by using four-frame differential under a static background and combining background subtraction to extract a moving target;
step four, obtaining a complete moving target area through morphological image processing means such as corrosion expansion and the like;
on the basis, effective threshold segmentation is carried out on the moving target and the background, the mass center is obtained according to the pixel information of the moving target, the coordinates of four corners of a rectangular frame of the moving target are further determined, and screenshot is carried out on the next frame according to the coordinates of the four corners, so that the interference of other moving targets except the moving target is avoided;
and step six, by utilizing a method of combining a moving target tracking algorithm with a Kalman filtering algorithm, the carrier vehicle is properly adjusted along with the movement of the moving target so as to achieve the purpose of accurately following the moving target.
The third concrete implementation mode:
different from the second specific embodiment, the method for controlling the automatic following loading vehicle based on the motion algorithm comprises the following steps that in the first step, an affine transformation model with six degrees of freedom is selected to establish a camera global motion model, unknown parameters in a motion model matrix are solved according to the position relation of a feature region between two frames, namely, the process of solving the motion model refers to the process that key points between adjacent images are paired by using FREAK feature points and feature descriptors thereof, and the matched key points are screened by a random sampling consistency method and then substituted into the affine transformation model with six degrees of freedom, so that the unknown parameters in the accurate global motion model are obtained;
the method specifically comprises the following steps:
step one, assuming that H is a FREAK descriptor of a certain feature point, then:
in the formula, PαRepresenting a pair of sample points; n is the number of feature vectors;
the intensity value of the sampling point after Gaussian smoothing is obtained;
and step two, in order to ensure that the algorithm has direction invariance, direction information needs to be added to each feature point. The direction of the feature point is determined by performing local gradient summation on the partial set with larger difference in the contrast set obtained in the above steps, and generally only long and symmetrical point pairs are extracted. Usually, 45 length and symmetrical sampling point pairs are selected to extract the direction of the characteristic point;
step three, calculating the angle through the formula (3):
in the formula: o is local gradient information, G is a set of pairs of feature points, M is the logarithm of the sample points at the points in the set G,is POThe area gray average value of the pixel point of the previous image is obtained by corresponding analysisIs POThe area gray level mean value of the next pixel point;
and
are respectively POThe two-dimensional vector of the space coordinate of the sampling characteristic point of the previous bit and the next bit, thereby obtaining the direction information of the characteristic point;
step four, generating a 512-bit binary descriptor H consisting of 0 and 1;
calculating the Hamming distance of the two binary descriptors through XOR operation;
when the Hamming distance between the two is smaller than a set threshold value, the matching is considered to be successful, and the next step is carried out; otherwise, the matching fails;
step one, regarding the target as moving on a plane, regarding the movement of the target between adjacent frames as the superposition of translation, rotation and scaling transformation, and based on the consideration of efficiency and accuracy, modeling the global movement by using a six-parameter affine transformation model, wherein the formula is as follows:
in the formula (4), (x, y) is pixel coordinates in a reference frame; (X, Y) is pixel coordinates in the current frame; define parameter vector P ═ (a, b, c, d, e, f)TWherein the components a, b, c, d are related to zooming, rotational movement, and the components e, f are related to translational movement; rewriting the matrix of the formula (4) to obtain an affine transformation model matrix:
and selecting three non-collinear feature point pairs from the obtained successfully matched feature point pairs to substitute in the formula (4), so that the global motion parameters can be obtained.
The fourth concrete implementation mode:
different from the third embodiment, the control method of the automatic following loading vehicle based on the motion algorithm of the third embodiment,
the first step further includes a step of purifying the obtained matching points due to the existence of mismatching of the feature points, and removing points with large errors, generally, the mismatching points are called as outliers, an affine model obtained by using the outliers has large errors, and the purification method of the matching points is divided into a comparative purification method and a consistency purification method, and the matching points are further purified by a random sampling consistency method in the consistency purification method in consideration of the real-time performance and stability of the system.
The specific method comprises the following steps:
(1) and screening the characteristic point pairs. And sorting the characteristic point pairs according to the Hamming distance, and selecting the first m characteristic point pairs.
(2) And 3 characteristic point pairs are selected from the m characteristic point pairs to establish an equation set, and 6 parameters of the affine model are solved.
(3) And (3) judging the remaining (m-3) feature point pairs by using the parameters obtained in the step (2), and if the Euclidean distance between the point (X, Y) passing through the affine transformation model matrix and the point (X, Y) is smaller than a set threshold, the matching is regarded as an inner point, otherwise, the matching is an outer point.
(4) And (3) randomly selecting 3 feature point pairs again, executing the step (2) and the step (3) k (k is 10 generally), then entering the step (5), and counting the number S of the inner points each time.
(5) And selecting the transformation matrix parameter with the maximum number of the interior points as a final transformation parameter, and taking the corresponding interior point set as a final matching point set.
The fifth concrete implementation mode:
different from the fourth specific embodiment, in the second step of the method for controlling an automatic following loading vehicle based on a motion algorithm of the present embodiment, the motion compensation is performed on the current frame image by using the obtained affine transformation model, so that the background is made static, specifically:
and (3) counteracting background motion between adjacent images due to the change of the camera posture by using a backward mapping method, so that the background between the two frames of images is staticized.
The sixth specific implementation mode:
different from the fifth embodiment, the control method of the automatic following loading vehicle based on the motion algorithm of the fifth embodiment,
in the third step, the difference image is obtained by combining the four-frame difference under the static background and background subtraction so as to extract the moving target, which specifically comprises the following steps: in an actual environment, the quality of an acquired image is poor due to the fact that the acquired image is limited by various external conditions and is influenced by noise, and the image cannot be well detected when the detection algorithm is directly applied to the image, so that the image must be preprocessed firstly after the image is acquired, interference of the noise and various external factors is reduced, and the accuracy of subsequent algorithm processing is improved. Common filtering modes in the spatial domain include median filtering and mean filtering, and salt and pepper noise and Gaussian noise in the image can be well eliminated.
And performing difference operation on two adjacent frames of images in the image sequence by using an interframe difference method:
firstly, setting T as a separation threshold value, setting the separation threshold value as a fixed value according to practical application, and often using an adaptive threshold value method;
then, selecting a proper threshold value T to binarize the difference image, approximately considering that the illumination is constant between two adjacent frames and the background change is not obvious, so that the position with a larger pixel value in the difference image is the gray value change caused by the movement of the foreground object, and expressing the change of the pixel value in a corresponding area caused by the movement of an object between the adjacent images by using the changed information, so that the moving object can be detected from the image sequence, namely the target area; the interframe difference method is shown in fig. 2.
Finally, a differential image D (m, n) is obtained by the following formula:
wherein, T1The binary threshold value can be obtained through experiments or a self-adaptive threshold value strategy; the point with the pixel value of 255 in the obtained image corresponds to the point moving between the two frames, and the point with the pixel value of 0 can be regarded as the point on the background; the frame difference method is sensitive to noise, and it is difficult to obtain the overall contour of the target when the inter-frame motion is not obvious.
The seventh embodiment:
different from the sixth embodiment, the control method of the automatic following loading vehicle based on the motion algorithm of the sixth embodiment,
in the fourth step, a complete process of obtaining the moving target area through morphological image processing means such as corrosion expansion and the like is specifically as follows: the moving target detection algorithm combining background subtraction and four-frame difference is adopted, the dynamic threshold T is respectively merged into the background subtraction method and the four-frame difference method, the problems of double images and the like are effectively solved while the influence caused by illumination mutation is effectively overcome, the moving target and the background can be effectively subjected to threshold segmentation, and the method for detecting connectivity is used for reducing or eliminating noise spots so as to obtain a more complete foreground image I (m, n).
The specific implementation mode is eight:
different from the seventh specific embodiment, in the fifth step, effective threshold segmentation is performed on the moving target and the background, the centroid is obtained according to the pixel information of the moving target, and then the coordinates of four corners of a rectangular frame of the moving target are determined, so that a screenshot is performed on the next frame according to the coordinates, and a process of avoiding interference of other moving targets except the moving target specifically includes:
after effective threshold segmentation is carried out on the moving target and the background, the centroid is obtained according to the pixel information of the moving target, and the foreground image is binarized, so that the centroid Z (p, q) of the moving target is calculated as follows:
wherein N, M is the number of pixel points in the length and width directions of the moving target contour, and p and q are the horizontal and vertical coordinates of the target centroid; and traversing the pixels around the coordinate as a center, stopping traversing when a point with a pixel value of 0 is detected due to the binary image, and recording the pixel coordinate at the moment.
The specific implementation method nine:
different from the eighth specific embodiment, in the sixth step, the moving target tracking algorithm is combined with the kalman filter algorithm, and the process that the carrier vehicle is properly adjusted along with the movement of the moving target to accurately follow the moving target means that the moving target is calculated and compared with the centroid coordinate difference between the target area map and the candidate area map by using the moving target tracking algorithm to judge the movement condition of the moving target, if the centroid of the candidate area is out of the centroid fluctuation range of normal walking, the moving target is shown to have moved on the basis of the target area, and according to the position relationship of the coordinates, the movement state of the moving target can be judged, so as to adjust the movement of the carrier vehicle. If the centroid of the candidate area is within the centroid fluctuation range of normal walking, it indicates that the moving target basically has no forward, backward, left-right turn and other motions on the basis of the target area, specifically:
sixthly, as shown in the diagram of the coordinate variation situation of the centroid of the moving object shown in fig. 3, setting: in the figure A0(x0,y0) As coordinates of the centroid of the moving object in the current frame, A1(x1,y1)、A2(x2,y2)、A3(x3,y3)、A4(x4,y4) As position coordinates to which the centroid of the moving object may vary, A5(x5,y5) As coordinates of the centroid of the moving object at a fixed distance, A6(x6,y6) As coordinates of the center of mass of the moving object at a safe distance, A6(x6,y6) At A5(x5,y5) And A0(x0,y0) A is0(x0,y0) Is a center, A6(x6,y6) The moving target is regarded as being in a tracking range and does not have huge morphological change in the normal fluctuation range, and further the moving state of the carrying vehicle does not need to be adjusted;
step six and two, when the center of mass is from A0(x0,y0) To a1(x1,y1) And analyzing the motion state of the moving target and the motion control mode of the loading vehicle. The moving object moving plan is shown in fig. 4.
In fig. 4, point a is the position of the camera, point C is the actual position of the moving object in the next frame, and point D is the abscissa x of the centroid of the moving object in the current frame in the image coordinate system0And the E point is the abscissa x of the centroid of the moving object in the next frame1Point E 'is a symmetric point of point E with point D as an axis, point F is a position of the moving object at a fixed distance, point G is a position of the moving object at a safe distance, length h of BC can be obtained by calibrating with a camera, and ED ═ E' D ═ E is a in fig. 3-8 a0Point and point A1The difference between the abscissas of the points is also | x0-x1I, the focal length AD of the camera is f, and the value of f is determined by the camera parameters. θ is the angle of the moving target deviating from the moving target in the current frame in the next frame, the actual vertical distance AB between the moving target and the camera in the next frame is g, the safe distance AF between the carrier vehicle and the moving target is i, and g is calculated by equation (7):
the angle θ can be obtained from equation (7):
if the distance g is greater than the distance i, the moving target exceeds a fixed distance, the carrying vehicle should pursue forwards, if the distance g is less than the distance i, the moving target moves backwards, the carrying vehicle should retreat to avoid, and the value x0-x1The positive and negative of the angle theta represent the left and right steering of the moving target when the moving target moves forwards and backwards, the turning direction of the loading vehicle is determined, and the angle theta represents the specific angle of the loading vehicle which should be turned when the loading vehicle turns; in conclusion, the carrying vehicle can be properly adjusted along with the movement of the moving target, so that the aim of automatically following the moving target is fulfilled;
sixthly, judging whether the target is lost or not by calculating a residual error between the target center coordinate estimated by the Kalman filter and the moving target centroid coordinate of the current frame measured by the moving algorithm; if the motion is lost, the system uses the predicted coordinates as the centroid of the moving object in the next frame; and conversely, the coordinate of the middle point of the moving object centroid coordinate in the next frame measured by the predicted coordinate and the motion algorithm is the final centroid coordinate. The system adopts a Kalman filtering algorithm to accurately predict the motion track, the Kalman filtering algorithm can predict the position of the target mass center possibly appearing in the next frame according to the position of the target mass center in the current frame, and the speed and the position of the target can be predicted more accurately by taking the minimum mean square error as the best recursion estimation criterion.
Claims (9)
1. The utility model provides a control system of automatic follow year thing car based on motion algorithm which characterized in that: the composition comprises: the device comprises a Cortex-A9 processor, a video acquisition circuit, an alarm circuit, a motor speed regulation driving circuit, a power supply circuit, a storage battery electric quantity detection circuit, a TF card interface circuit, SDRAM, NAND FLASH and NOR FLASH, wherein the Cortex-A9 processor is respectively connected with the video acquisition circuit, the alarm circuit, the motor speed regulation driving circuit, the power supply circuit, the storage battery electric quantity detection circuit, the TF card interface circuit, the SDRAM, NAND FLASH and the NOR FLASH, the video acquisition circuit is connected with a camera, the power supply circuit is connected with the storage battery, the storage battery is connected with the storage battery electric quantity detection circuit, and the motor speed regulation driving circuit is connected with a group of motors.
2. A control method of an automatic following loading vehicle based on a motion algorithm is characterized in that: the method is realized by the following steps:
firstly, an affine transformation model with six degrees of freedom is selected to establish a camera global motion model, and unknown parameters in a motion model matrix are solved according to the position relation of a characteristic region between two frames, namely the motion model is solved;
secondly, performing motion compensation on the current frame image by using the obtained affine transformation model to make the background static;
step three, solving a differential image by using four-frame differential under a static background and combining background subtraction to extract a moving target;
step four, obtaining a complete moving target area through morphological image processing means such as corrosion expansion and the like;
step five, carrying out threshold segmentation on the moving target and the background, solving a mass center according to the pixel information of the moving target, further determining four-corner coordinates of a rectangular frame of the moving target, and carrying out screenshot on the next frame according to the four-corner coordinates;
and sixthly, properly adjusting the carrier vehicle along with the movement of the moving target by utilizing a method of combining a moving target tracking algorithm with a Kalman filtering algorithm so as to accurately follow the moving target.
3. The control method of the automatic following loading vehicle based on the motion algorithm according to claim 1, characterized in that: in the first step, an affine transformation model with six degrees of freedom is selected to establish a camera global motion model, unknown parameters in a motion model matrix are solved according to the position relation of a feature region between two frames, namely the process of solving the motion model means that key points between adjacent images are paired by using FREAK feature points and feature descriptors thereof, and the matched key points are screened by a random sampling consistency method and then substituted into the affine transformation model with six degrees of freedom to further obtain the unknown parameters in the global motion model;
the method specifically comprises the following steps:
step one, assuming that H is a FREAK descriptor of a certain feature point, then:
in the formula, PαRepresenting a pair of sample points; n is the number of feature vectors;
the intensity value of the sampling point after Gaussian smoothing is obtained;
step two, selecting 45 length and symmetrical sampling point pairs to extract the direction of the characteristic points;
step three, calculating the angle through the formula (3):
in the formula: o is local gradient information, G is a set of pairs of feature points, M is the logarithm of the sample points at the points in the set G,is POThe area gray average value of the pixel point of the previous image is obtained by corresponding analysis
Is POThe area gray level mean value of the next pixel point;
and
are respectively POThe two-dimensional vector of the space coordinate of the sampling characteristic point of the previous bit and the next bit, thereby obtaining the direction information of the characteristic point;
step four, generating a 512-bit binary descriptor H consisting of 0 and 1;
calculating the Hamming distance of the two binary descriptors through XOR operation;
when the Hamming distance between the two is smaller than a set threshold value, the matching is considered to be successful, and the next step is carried out; otherwise, the matching fails;
step six, regarding the target as moving on a plane, regarding the movement of the target between adjacent frames as superposition of translation, rotation and scaling transformation, and modeling the global movement by using a six-parameter affine transformation model, wherein the formula is as follows:
in the formula (4), (x, y) is pixel coordinates in a reference frame; (X, Y) is pixel coordinates in the current frame; define parameter vector P ═ (a, b, c, d, e, f)TWherein the components a, b, c, d are related to zooming, rotational movement, and the components e, f are related to translational movement; rewriting the matrix of the formula (4) to obtain an affine transformation model matrix:
and selecting three non-collinear feature point pairs from the obtained successfully matched feature point pairs to substitute in the formula (4), so that the global motion parameters can be obtained.
4. The control method of the automatic following carrier vehicle based on the motion algorithm according to claim 3, characterized in that: the first step further includes a step of purifying the obtained matching points to remove points with large errors, generally, these points with mismatching are called as outliers, an affine model obtained by using these outliers will have large errors, the method for purifying the matching points is divided into a comparative purification method and a consistency purification method, and in consideration of real-time performance and stability of the system, a random sampling consistency method in the consistency purification method is adopted to further purify the matching points, and the specific method is as follows:
(1) screening the characteristic point pairs, sorting the characteristic point pairs according to the Hamming distance, selecting the first m characteristic point pairs,
(2) selecting 3 characteristic point pairs from the m characteristic point pairs to establish an equation set, solving 6 parameters of the affine model,
(3) judging the remaining (m-3) feature point pairs by using the parameters obtained in the step (2), if the Euclidean distance between the point (X, Y) passing through the affine transformation model matrix and the point (X, Y) is smaller than a set threshold value, the matching is regarded as an inner point, otherwise, the matching is an outer point,
(4) randomly selecting 3 feature point pairs again, executing step (2) and step (3) k (k is 10 generally), entering step (5), counting the number S of the inner points in each time,
(5) and selecting the transformation matrix parameter with the maximum number of the interior points as a final transformation parameter, and taking the corresponding interior point set as a final matching point set.
5. The control method of the automatic following carrier vehicle based on the motion algorithm according to claim 4, characterized in that: in the second step, the motion compensation is performed on the current frame image by using the obtained affine transformation model, so that the background is made to be static, specifically:
and (3) counteracting background motion between adjacent images due to the change of the camera posture by using a backward mapping method, so that the background between the two frames of images is staticized.
6. The control method of the automatic following loading vehicle based on the motion algorithm according to claim 5, characterized in that: in the third step, the difference image is obtained by combining the four-frame difference under the static background and background subtraction so as to extract the moving target, which specifically comprises the following steps:
and performing difference operation on two adjacent frames of images in the image sequence by using an interframe difference method:
firstly, setting T as a separation threshold value, and setting the T as a fixed value according to actual application;
then, selecting a proper threshold value T to binarize the difference image, approximately considering that the illumination is constant between two adjacent frames and the background change is not obvious, so that the position with a larger pixel value in the difference image is the gray value change caused by the movement of the foreground object, and expressing the change of the pixel value in a corresponding area caused by the movement of an object between the adjacent images by using the changed information, so that the moving object can be detected from the image sequence, namely the target area;
finally, a differential image D (m, n) is obtained by the following formula:
wherein, T1For binarizing threshold, the threshold can be set by experiment or self-adaptive thresholdSlightly obtaining; the point with the pixel value of 255 in the obtained image corresponds to the point moving between the two frames, and the point with the pixel value of 0 can be regarded as the point on the background; the frame difference method is sensitive to noise, and it is difficult to obtain the overall contour of the target when the inter-frame motion is not obvious.
7. The control method of the automatic following loading vehicle based on the motion algorithm according to claim 6, characterized in that: in the fourth step, a complete process of obtaining the moving target area through morphological image processing means such as corrosion expansion and the like is specifically as follows: and respectively integrating the dynamic threshold T into a background subtraction method and a four-frame difference method by adopting a moving object detection algorithm combining background subtraction and four-frame difference, thereby obtaining a more complete foreground image I (m, n).
8. The control method of the automatic following carrier vehicle based on the motion algorithm according to claim 7, characterized in that: in the fifth step, threshold segmentation is performed on the moving target and the background, the centroid is obtained according to the pixel information of the moving target, and then the coordinates of the four corners of the rectangular frame of the moving target are determined, and the process of capturing the next frame by taking the coordinates as the basis is specifically as follows:
after effective threshold segmentation is carried out on the moving target and the background, the centroid is obtained according to the pixel information of the moving target, and the foreground image is binarized, so that the centroid Z (p, q) of the moving target is calculated as follows:
wherein N, M is the number of pixel points in the length and width directions of the moving target contour, and p and q are the horizontal and vertical coordinates of the target centroid; and traversing the pixels around the coordinate as the center, stopping traversing when a point with the pixel value of 0 is detected, and recording the pixel coordinate at the moment.
9. The control method of the automatic following carrier vehicle based on the motion algorithm according to claim 8, characterized in that: in the sixth step, the process of utilizing the moving target tracking algorithm in combination with the kalman filter algorithm to properly adjust the carrier vehicle along with the movement of the moving target so as to accurately follow the moving target means that the moving target is judged by calculating and comparing the coordinate difference of the centroid of the target area map and the candidate area map through the moving target tracking algorithm, if the centroid of the candidate area is outside the normal walking centroid fluctuation range, it indicates that the moving target has moved on the basis of the target area, and according to the position relationship of the coordinates, the movement state of the moving target can be judged, and then the movement of the carrier vehicle is adjusted, if the centroid of the candidate area is within the normal walking centroid fluctuation range, it indicates that the moving target has not moved forward or backward basically on the basis of the target area, and the movement of the moving target such as left-right turn is specifically:
step six, setting A0(x0,y0) As coordinates of the centroid of the moving object in the current frame, A1(x1,y1)、A2(x2,y2)、A3(x3,y3)、A4(x4,y4) As position coordinates to which the centroid of the moving object may vary, A5(x5,y5) As coordinates of the centroid of the moving object at a fixed distance, A6(x6,y6) As coordinates of the center of mass of the moving object at a safe distance, A6(x6,y6) At A5(x5,y5) And A0(x0,y0) A is0(x0,y0) Is a center, A6(x6,y6) The moving target is regarded as being in a tracking range and does not have huge morphological change in the normal fluctuation range, and further the moving state of the carrying vehicle does not need to be adjusted;
step six and two, when the center of mass is from A0(x0,y0) To a1(x1,y1) Analyzing the motion state of the moving object and controlling the motion of the carrier vehicleA preparation mode;
let point A be the position of the camera, point C be the actual position of the moving object in the next frame, and point D be the abscissa x of the centroid of the moving object in the current frame in the image coordinate system0And the E point is the abscissa x of the centroid of the moving object in the next frame1Point E 'is a symmetric point of point E with point D as an axis, point F is a position of the moving object at a fixed distance, point G is a position of the moving object at a safe distance, length h of BC can be obtained by calibrating with a camera, and ED ═ E' D ═ E is a in fig. 3-8 a0Point and point A1The difference between the abscissas of the points is also | x0-x1I, the focal length AD of the camera is f, the value of f is determined by camera parameters, θ is the angle of the moving target deviating from the moving target in the current frame in the next frame, the actual vertical distance AB between the moving target and the camera in the next frame is g, the safe distance AF between the carrier vehicle and the moving target is i, and g is calculated by equation (7):
the angle θ can be obtained from equation (7):
if the distance g is greater than the distance i, the moving target exceeds a fixed distance, the carrying vehicle should pursue forwards, if the distance g is less than the distance i, the moving target moves backwards, the carrying vehicle should retreat to avoid, and the value x0-x1The positive and negative of the angle theta represent the left and right steering of the moving target when the moving target moves forwards and backwards, the turning direction of the loading vehicle is determined, and the angle theta represents the specific angle of the loading vehicle which should be turned when the loading vehicle turns; in conclusion, the carrying vehicle can be properly adjusted along with the movement of the moving target, so that the aim of automatically following the moving target is fulfilled;
sixthly, judging whether the target is lost or not by calculating a residual error between the target center coordinate estimated by the Kalman filter and the moving target centroid coordinate of the current frame measured by the moving algorithm; if the motion is lost, the system uses the predicted coordinates as the centroid of the moving object in the next frame; and conversely, the coordinate of the middle point of the moving object centroid coordinate in the next frame measured by the predicted coordinate and the motion algorithm is the final centroid coordinate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910997269.4A CN110825123A (en) | 2019-10-21 | 2019-10-21 | Control system and method for automatic following loading vehicle based on motion algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910997269.4A CN110825123A (en) | 2019-10-21 | 2019-10-21 | Control system and method for automatic following loading vehicle based on motion algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110825123A true CN110825123A (en) | 2020-02-21 |
Family
ID=69549713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910997269.4A Pending CN110825123A (en) | 2019-10-21 | 2019-10-21 | Control system and method for automatic following loading vehicle based on motion algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110825123A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232322A (en) * | 2020-12-14 | 2021-01-15 | 支付宝(杭州)信息技术有限公司 | Image generation method and device based on object state prediction |
| CN112381838A (en) * | 2020-11-14 | 2021-02-19 | 四川大学华西医院 | Automatic image cutting method for digital pathological section image |
| CN112995678A (en) * | 2021-02-22 | 2021-06-18 | 深圳创维－Rgb电子有限公司 | Video motion compensation method and device and computer equipment |
| CN113643355A (en) * | 2020-04-24 | 2021-11-12 | 广州汽车集团股份有限公司 | Method and system for detecting position and orientation of target vehicle and storage medium |
| WO2022217794A1 (en) * | 2021-04-12 | 2022-10-20 | 深圳大学 | Positioning method of mobile robot in dynamic environment |
| CN116441132A (en) * | 2023-04-18 | 2023-07-18 | 河北省体育科学研究所(河北省体育局反兴奋剂服务中心) | Snowboard waxing machine |
| CN117169887A (en) * | 2023-11-03 | 2023-12-05 | 武汉能钠智能装备技术股份有限公司 | SAR ground moving target positioning method based on direction determination |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102456225A (en) * | 2010-10-22 | 2012-05-16 | 深圳中兴力维技术有限公司 | Video monitoring system and moving target detecting and tracking method thereof |
| CN102750708A (en) * | 2012-05-11 | 2012-10-24 | 天津大学 | Affine motion target tracing algorithm based on fast robust feature matching |
| US20130099922A1 (en) * | 2011-10-24 | 2013-04-25 | Andrew Lohbihler | Motion and Area Monitoring System and Method |
| CN104637071A (en) * | 2015-01-28 | 2015-05-20 | 四川君逸易视科技有限公司 | People tracking method based on intelligent video analysis |
| CN106170676A (en) * | 2015-07-14 | 2016-11-30 | 深圳市大疆创新科技有限公司 | For determining the method for the movement of mobile platform, equipment and system |
| DE202016106187U1 (en) * | 2016-07-18 | 2016-12-13 | Dyaco International Inc. | Systems for the analysis of image-based movements |
| CN107450565A (en) * | 2017-09-18 | 2017-12-08 | 天津工业大学 | Intelligent movable tracks car |
| CN107909024A (en) * | 2017-11-13 | 2018-04-13 | 哈尔滨理工大学 | Vehicle tracking system, method and vehicle based on image recognition and infrared obstacle avoidance |
| CN207586755U (en) * | 2017-12-26 | 2018-07-06 | 侯亚辉 | A kind of intelligence following device and system |
| CN109102523A (en) * | 2018-07-13 | 2018-12-28 | 南京理工大学 | A kind of moving object detection and tracking |
| CN109559329A (en) * | 2018-11-28 | 2019-04-02 | 陕西师范大学 | A kind of particle filter tracking method based on depth denoising autocoder |
-
2019
- 2019-10-21 CN CN201910997269.4A patent/CN110825123A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102456225A (en) * | 2010-10-22 | 2012-05-16 | 深圳中兴力维技术有限公司 | Video monitoring system and moving target detecting and tracking method thereof |
| US20130099922A1 (en) * | 2011-10-24 | 2013-04-25 | Andrew Lohbihler | Motion and Area Monitoring System and Method |
| CN102750708A (en) * | 2012-05-11 | 2012-10-24 | 天津大学 | Affine motion target tracing algorithm based on fast robust feature matching |
| CN104637071A (en) * | 2015-01-28 | 2015-05-20 | 四川君逸易视科技有限公司 | People tracking method based on intelligent video analysis |
| CN106170676A (en) * | 2015-07-14 | 2016-11-30 | 深圳市大疆创新科技有限公司 | For determining the method for the movement of mobile platform, equipment and system |
| DE202016106187U1 (en) * | 2016-07-18 | 2016-12-13 | Dyaco International Inc. | Systems for the analysis of image-based movements |
| CN107632698A (en) * | 2016-07-18 | 2018-01-26 | 岱宇国际股份有限公司 | Motion analysis system and method based on image |
| CN107450565A (en) * | 2017-09-18 | 2017-12-08 | 天津工业大学 | Intelligent movable tracks car |
| CN107909024A (en) * | 2017-11-13 | 2018-04-13 | 哈尔滨理工大学 | Vehicle tracking system, method and vehicle based on image recognition and infrared obstacle avoidance |
| CN207586755U (en) * | 2017-12-26 | 2018-07-06 | 侯亚辉 | A kind of intelligence following device and system |
| CN109102523A (en) * | 2018-07-13 | 2018-12-28 | 南京理工大学 | A kind of moving object detection and tracking |
| CN109559329A (en) * | 2018-11-28 | 2019-04-02 | 陕西师范大学 | A kind of particle filter tracking method based on depth denoising autocoder |
Non-Patent Citations (6)
| Title |
|---|
| KANG FENG;HAN YONGHUA;ZHANG HUAXIONG: "Video Stabilization Based on Multi-scale Local Color Invariants", 《2013 FOURTH INTERNATIONAL CONFERENCE ON NETWORKING AND DISTRIBUTED COMPUTING》 * |
| 刘继辉: "载物车无线自动跟随系统设计", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 * |
| 向伟: "基于检测的目标跟踪算法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 夏雯娟: "独立式车辆跟踪控制系统研究", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 * |
| 谢冬: "基于双目视觉的智能跟踪行李车系统设计", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 郑启文: "基于无线通信网络的多智能小车协作控制系统研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113643355A (en) * | 2020-04-24 | 2021-11-12 | 广州汽车集团股份有限公司 | Method and system for detecting position and orientation of target vehicle and storage medium |
| CN113643355B (en) * | 2020-04-24 | 2024-03-29 | 广州汽车集团股份有限公司 | Target vehicle position and orientation detection method, system and storage medium |
| CN112381838A (en) * | 2020-11-14 | 2021-02-19 | 四川大学华西医院 | Automatic image cutting method for digital pathological section image |
| CN112381838B (en) * | 2020-11-14 | 2022-04-19 | 四川大学华西医院 | Automatic image cutting method for digital pathological section image |
| CN112232322A (en) * | 2020-12-14 | 2021-01-15 | 支付宝(杭州)信息技术有限公司 | Image generation method and device based on object state prediction |
| CN112995678A (en) * | 2021-02-22 | 2021-06-18 | 深圳创维－Rgb电子有限公司 | Video motion compensation method and device and computer equipment |
| WO2022217794A1 (en) * | 2021-04-12 | 2022-10-20 | 深圳大学 | Positioning method of mobile robot in dynamic environment |
| CN116441132A (en) * | 2023-04-18 | 2023-07-18 | 河北省体育科学研究所(河北省体育局反兴奋剂服务中心) | Snowboard waxing machine |
| CN117169887A (en) * | 2023-11-03 | 2023-12-05 | 武汉能钠智能装备技术股份有限公司 | SAR ground moving target positioning method based on direction determination |
| CN117169887B (en) * | 2023-11-03 | 2024-04-19 | 武汉能钠智能装备技术股份有限公司 | SAR ground moving target positioning method based on direction determination |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110825123A (en) | Control system and method for automatic following loading vehicle based on motion algorithm | |
| US11288818B2 (en) | Methods, systems, and computer readable media for estimation of optical flow, depth, and egomotion using neural network trained using event-based learning | |
| Mason et al. | Using histograms to detect and track objects in color video | |
| CN103325112B (en) | Moving target method for quick in dynamic scene | |
| Lou et al. | 3-D model-based vehicle tracking | |
| CN111563469A (en) | Method and device for identifying irregular parking behaviors | |
| Li et al. | Multiple object tracking with motion and appearance cues | |
| CN106863332B (en) | Robot vision positioning method and system | |
| CN106780560B (en) | Bionic robot fish visual tracking method based on feature fusion particle filtering | |
| CN104200495A (en) | Multi-target tracking method in video surveillance | |
| CN104408725A (en) | Target recapture system and method based on TLD optimization algorithm | |
| CN104036523A (en) | Improved mean shift target tracking method based on surf features | |
| CN107742306B (en) | Moving target tracking algorithm in intelligent vision | |
| CN107330922A (en) | Video moving object detection method of taking photo by plane based on movable information and provincial characteristics | |
| CN101324958A (en) | Method and apparatus for tracking object | |
| CN110110608A (en) | The fork truck speed monitoring method and system of view-based access control model under a kind of overall view monitoring | |
| Tian et al. | Research on multi-sensor fusion SLAM algorithm based on improved gmapping | |
| CN110490903B (en) | Multi-target rapid capturing and tracking method in binocular vision measurement | |
| Yi et al. | Ucmctrack: Multi-object tracking with uniform camera motion compensation | |
| Wen et al. | Dynamic slam: A visual slam in outdoor dynamic scenes | |
| Singh et al. | Fusing semantics and motion state detection for robust visual SLAM | |
| CN106780541B (en) | A kind of improved background subtraction method | |
| CN110415275B (en) | Point-to-point-based moving target detection and tracking method | |
| Mao et al. | Design of visual navigation system of farmland tracked robot based on raspberry pie | |
| Dong et al. | An automatic object detection and tracking method based on video surveillance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200221 |