CN111915649A - Strip steel moving target tracking method under shielding condition - Google Patents
Strip steel moving target tracking method under shielding condition Download PDFInfo
- Publication number
- CN111915649A CN111915649A CN202010733819.4A CN202010733819A CN111915649A CN 111915649 A CN111915649 A CN 111915649A CN 202010733819 A CN202010733819 A CN 202010733819A CN 111915649 A CN111915649 A CN 111915649A
- Authority
- CN
- China
- Prior art keywords
- target
- strip steel
- tracking
- color
- histogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 80
- 239000010959 steel Substances 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013528 artificial neural network Methods 0.000 claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 239000011159 matrix material Substances 0.000 claims description 30
- 230000001133 acceleration Effects 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 claims description 4
- 210000002569 neuron Anatomy 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 abstract description 2
- 230000007547 defect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 8
- 238000005096 rolling process Methods 0.000 description 8
- 238000004519 manufacturing process Methods 0.000 description 6
- 230000002093 peripheral effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000010008 shearing Methods 0.000 description 3
- 239000002184 metal Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010079 rubber tapping Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007730 finishing process Methods 0.000 description 1
- 238000005098 hot rolling Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 239000002904 solvent Substances 0.000 description 1
- 239000013076 target substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of machine vision, and particularly relates to a method for tracking a moving target of strip steel under a shielding condition. The method has the advantages that the color, texture and edge characteristics are fused by obtaining the motion video of the strip steel on the roller way and utilizing the Camshift motion target tracking algorithm with high calculation efficiency and good robustness, when water vapor or equipment shelters the strip steel occurs, the BP neural network and the Kalman filter are used for predicting the position of the strip steel, the detection and tracking tasks of the strip steel are completed, the defect that the tracking precision is low due to the fact that the traditional Camshift algorithm tracks through a single characteristic is effectively overcome, a solution is provided for the problem that the strip steel is sheltered, the loss of a tracked target is avoided, and the tracking stability and the tracking precision of the motion target under the complex background are improved.
Description
Technical Field
The invention relates to the field of machine vision, in particular to a method for tracking a moving target of strip steel under a shielding condition.
Background
The steel industry is an important basic industry of national economy and also one of the basic industries of all industrialized countries. The hot rolled strip steel is a main product of steel enterprises and is widely applied to various industrial fields of machinery, construction, national defense, aerospace and the like. The hot rolling processing of the strip steel is an important link in the steel processing and is also the production link which is most easy to cause quality problems. The strip steel moves on the roller way, and is conveyed to a rolling mill for reciprocating rolling and then conveyed to a finishing process after rolling. In the production process of the strip steel, the position and the state of the strip steel can be known only by observing and monitoring the strip steel in real time by an operator, and corresponding operation is carried out to prevent accidents.
At present, an HMD (Hot Metal Detector) is mostly adopted in a steel mill to detect strip steel and judge the movement position of the strip steel, however, the accuracy and stability of an HMD signal are closely related to the field environment, when water vapor and smoke exist in the field of view of the Detector, signal flicker can be caused, the sensitivity of an instrument is reduced, the head and tail positions of a rolled piece cannot be accurately positioned, and the problems of shearing error, steel piling and the like are easily caused.
With the increasing level of industrial automation, the continuous development of digital image processing technology and the increasing perfection of multimedia technology such as computers, visual tracking technology is produced and widely applied, the economic value and the social value of the visual tracking technology have been widely concerned by academia and enterprises all over the world, and many scholars are trying to involve in or have been engaged in the research in this aspect.
In the aspects of the existing papers and patents, the following methods for tracking the moving target of the strip steel in the prior art are available: in the hot rolled strip tracking control of Huangkeling, the tracking of the strip steel on the rolling line is realized by receiving signals of the rolling mill speed, the rolling mill biting steel, the flying shear shearing and the like from a Programmable Logic Controller (PLC) of the rolling line according to signals of on-site actual detection equipment. The university of northeast Jiachunying uses a multi-agent to optimize a strip steel tracking control system for judging tracking events according to signals of a detection instrument. The above papers all use signals of detecting instruments such as a hot metal detector, etc. to track the strip steel in combination with a PLC. However, the field detection equipment is complex, the signal is unstable, and many problems cannot be prevented and solved in time. In a converter tapping injection flow target tracking control system design part, Zhabin of northeast university uses a video image processing technology and adopts a centroid tracking algorithm to track a target. The patent 'a novel CBOCP on-line infrared converter tapping steel flow automatic detection and tracking method and system' adopts CBOCP (Camshift Based on Contour Features, Ca msshift algorithm) infrared steel flow image tracking method to automatically detect and track steel flow. The above papers and patents apply the visual tracking technology to the tracking of steel flow, and there is no mention about the detection and tracking of strip steel on the roller way.
In the production operation of an actual steel mill, the strip steel moving on a roller way is easily shielded by operators, machine equipment, water mist and the like, which has great influence on the tracking of a moving target and even can cause the tracking loss. The Camshift algorithm is a fast pattern matching algorithm based on kernel density without parameters, can effectively solve the problem of target deformation, is simple in calculation and high in real-time performance, and however, the traditional Camshift algorithm only realizes tracking of a moving target through a single characteristic of color constraint of the tracking target, so that the tracking precision is low.
Disclosure of Invention
Aiming at the technical problems, the invention provides a strip steel moving target tracking method under a shielding condition, in particular to a hot-rolled strip steel moving target tracking method combining a Camshift algorithm with multiple characteristics and a BP neural network and a Kalman filter prediction algorithm. The method integrates color, texture and edge features, and effectively overcomes the defect of low tracking precision caused by the fact that a traditional Camshift algorithm uses a single feature for tracking. The BP neural network and the Kalman filter are adopted to predict the state of the moving target, a solution is provided for the problem that the strip steel is shielded in actual production, the loss of the tracking target is avoided, and the tracking stability and the tracking precision of the moving target under a complex background are improved.
The invention is realized by the following technical scheme:
a method for tracking a moving target of strip steel under a shielding condition is characterized in that a moving video of the strip steel on a roller way is obtained, and color, texture and edge characteristics are fused by using a Camshift moving target tracking algorithm; when the strip steel is in a shielded condition, the BP neural network and the Kalman filter are used for predicting the position of the strip steel, so that the detection and tracking tasks of the strip steel are completed, the loss of a tracking target is avoided, and the tracking stability and the tracking precision of a moving target under a complex background are improved.
Further, the method comprises the steps of:
(1) collecting a video of the movement of the strip steel on a roller way, carrying out image preprocessing, determining the minimum external rectangular outline of the strip steel by outline detection by using a frame difference method, and determining the strip steel as a tracking starting target;
(2) using the minimum circumscribed rectangle contour determined in the step (1) as an initialization search frame, obtaining a two-dimensional histogram of color features, texture features and edge features of a target, and obtaining a color histogram of a search window;
(3) respectively calculating the two-dimensional histograms of the obtained color features, texture features and edge features, and fusing by different weights to obtain a reverse projection graph fused with multi-feature information;
(4) judging the shielding condition of the target: calculating the Bhattacharyya distance between the color feature histogram of the target and the color histogram of the search window by adopting the color feature histogram of the target and the color histogram of the search window, and judging that shielding occurs if the calculated Bhattacharyya distance is smaller than a threshold value; if the threshold value is larger than the threshold value, no shielding occurs;
(5) if the target is judged to be shielded, predicting the motion state of the target by adopting a prediction algorithm of a BP neural network and a Kalman filter, and using the predicted result for next frame Camshift algorithm iteration;
(6) and outputting a tracking result of the strip steel moving target.
Further, in step (2), the method for obtaining the two-dimensional histogram of the color feature includes: the image is converted to an HSV color space, and the two-dimensional histogram of the color feature is a histogram using chrominance components separated from the HSV color space.
Further, in the step (2), an LBP operator is used to describe texture features, the texture features are defined in a 3 × 3 window, the gray value of the central pixel point of the window is used as a threshold value, the gray value is compared with the gray values of the adjacent 8 pixel points, if the gray values of the peripheral pixel points are greater than the gray value of the central pixel point, the positions of the corresponding peripheral pixel points are marked as 1, otherwise, the positions of the peripheral pixel points are marked as 0; forming an 8-bit binary number from the binarization result, using the obtained 8-bit binary number as an LBP value of a window center pixel point, and reflecting texture information of a 3 × 3 window area by using the LBP value, that is:
wherein, LBP (x)c，yc) The representation is in pixels (x)c，yc) Texture information of a 3 × 3 window region as a center; p represents the p-th pixel point except the central pixel point in the 3 multiplied by 3 window; i (c) represents the gray value of the central pixel point, and I (p) represents the gray value of the p-th pixel point in the 3 multiplied by 3 window; s (x) formula is as follows:
further, in the step (2), the edge features are described by using a Canny operator, after the image is smoothed and filtered, the gradient strength and the gradient direction are calculated, then non-maximum value suppression is carried out, and the edge is detected and connected by using a dual-threshold algorithm to obtain the edge features of the image.
Further, in the step (4), the determination of the occlusion condition on the target specifically includes: judging whether the target is shielded or not by calculating the Bhattacharyya distance between the color feature histogram of the target and the color histogram of the search window:
wherein, p (p, p ') is an image similarity coefficient, the range is between 0 and 1, 0 represents extremely different, 1 represents extremely similar, p, p' respectively represent a color feature histogram of a target and a color histogram of a search window, i is a feature value of the image, and the value range is 1, 2.. multidot.N; p (i) and p' (i) are probability distributions of two images having a characteristic value i at the same position on the histogram.
Further, the step (5) is specifically as follows: if the target is judged to be shielded, sequentially subtracting the stored position point coordinates of the target at the first six moments to construct a difference matrix, inputting three rows and four columns of the sample difference matrix, and generating four pairs of difference values by five pairs of point coordinates, wherein each row comprises two pairs of difference values; outputting three rows and two columns of the sample difference matrix, wherein three pairs of difference values are generated by four pairs of coordinates, and one pair of difference values is generated in each row; training the constructed difference matrix by using a BP neural network according to rows, then predicting by using position difference values of three adjacent moments to obtain the possible position of the target at the next moment, and replacing an observed value in a Kalman filter with a predicted value of the BP neural network;
the constructed BP neural network model has three layers, the number of the neurons is 4, 9 and 2 in sequence, the activation function is a Sigmoid function, the highest iteration time is 1000 times, the predicted value of the BP neural network replaces the observed value in a Kalman filter, the state equation of the Kalman filter is three-order, the prediction is carried out through the state equation of the three-order Kalman filter to obtain the final optimal estimated value, and the result is used for the next frame of Camshift algorithm iteration;
because the strip steel moves on the roller way to change acceleration movement, a variable acceleration movement model is adopted; let the state vector of the target be:
X(k)＝[x(k)，y(k)，dx(k)，dy(k)，d(dx(k))，d(dy(k))]
wherein x (k), y (k), dx (k), dy (k), d (dx (k)), d (dy (k)) and d (dy (k)) are the coordinate, speed and acceleration of the target centroid in the x-axis and y-axis directions at the moment of kDegree; thus, the state transition matrix of the target
Comprises the following steps:
Δ t is the time interval between two adjacent frames of images; the observed state vector of the target, Y (k), is:
Y(k)＝[x(k)，y(k)]
the observation matrix h (k) is obtained from the relationship between the target state and the observation state of the target:
let the dynamic noise covariance matrix q (k) be an identity matrix of 6 × 6 dimensions, and the observation noise covariance matrix r (k) be an identity matrix of 2 × 2 dimensions.
The beneficial technical effects of the invention are as follows:
the method for tracking the moving target of the strip steel under the shielding condition overcomes the problems that the sensitivity of a hot metal detector is reduced, the detection is inaccurate, the fault is easy to occur and the like in the tracking method in the prior art. The Camshift algorithm is improved by fusing multiple features (color, texture and edge features), and the problem that the Camshift algorithm is low in tracking precision because the Camshift algorithm is only used for tracking according to color information of a target is solved. The prediction of the moving target state under the shielding condition is realized by combining the BP neural network and the prediction algorithm of the Kalman filter, the loss of the tracking target is avoided, and the tracking stability and the tracking precision of the moving target under the complex background are improved.
In addition, the method adopts a visual tracking technology to detect and track the strip steel on the roller way, can update the detection technology, and solves the problem of the reliability of the detection signal which is pending for long dragging with less investment and improvement time; the reliability of the shearing optimization control system is improved, the equipment accident and failure rate are reduced, and the requirements of steel rolling production can be met to the maximum extent; the maintenance is easy, the operation time of maintenance personnel in a dangerous area is reduced, and the labor intensity and the danger of the maintenance personnel are reduced; the method has profound significance for the steel mill in the aspects of product full-flow tracking, equipment operation and maintenance, product quality control, early warning of site abnormal conditions and the like.
Drawings
FIG. 1 is a flow chart of a hot-rolled strip steel moving target tracking method under a shielding condition in the embodiment of the invention.
Fig. 2 is a schematic diagram of the Camshift algorithm in the embodiment of the present invention.
Fig. 3 is a diagram of a BP neural network structure in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
On the contrary, the invention is intended to cover alternatives, modifications, equivalents and alternatives which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of the present invention, certain specific details are set forth in order to provide a better understanding of the present invention. It will be apparent to one skilled in the art that the present invention may be practiced without these specific details.
The embodiment of the invention provides a method for tracking a moving target of a strip steel under a shielding condition, and is a flow chart of the method for tracking the moving target of the hot-rolled strip steel under the shielding condition in the embodiment of the invention as shown in figure 1. The method comprises the steps of acquiring a motion video of strip steel on a roller way, and fusing color, texture and edge characteristics by using a Camshift motion target tracking algorithm; when the strip steel is in a shielded condition, predicting the position of the strip steel by using a BP neural network and a Kalman filter to complete the detection and tracking tasks of the strip steel; the tracking target is prevented from being lost, and the tracking stability and the tracking precision of the moving target under the complex background are improved.
In this embodiment, the method specifically includes the following steps:
(1) a camera is installed on a hot-rolled strip steel production line, a video of strip steel moving on a roller way is collected through the camera, image denoising processing is carried out, and a frame difference method is used for extracting a moving area in an image through pixel-based time difference and closed value conversion between two adjacent frames of the video. Determining the minimum external rectangular outline of the strip steel through outline detection, determining the strip steel as a moving target, and taking the moving target as a tracking starting target;
(2) and (2) using the minimum circumscribed rectangle outline determined in the step (1) as an initialization search frame, obtaining a two-dimensional histogram of the color feature, the texture feature and the edge feature of the target, and obtaining a color histogram of a search window. Wherein the color histogram is a histogram of image chrominance components; the texture features are that gray level images are obtained by calculating Local Binary Pattern (LBP) to represent texture information of the target, and then a histogram of the texture image is calculated; the edge features are edge images obtained by calculating Canny operators of the video images to describe edge information of the strip steel, and then calculating histograms of the edge images; specifically, the method comprises the following steps:
in this embodiment, the method for obtaining the two-dimensional histogram of the color feature includes: the image is converted to an HSV color space, and the two-dimensional histogram of the color feature is a histogram using chrominance components separated from the HSV color space.
In this embodiment, an LBP operator is used to describe texture features, the texture features are defined in a 3 × 3 window, the gray value of the central pixel point of the window is used as a threshold value, the threshold value is compared with the gray values of the adjacent 8 pixel points, if the gray values of the peripheral pixel points are greater than the gray value of the central pixel point, the positions of the corresponding peripheral pixel points are marked as 1, otherwise, the positions of the peripheral pixel points are marked as 0. Forming an 8-bit binary number from the binarization result, using the obtained 8-bit binary number as an LBP value of a window center pixel point, and reflecting texture information of a 3 × 3 window area by using the LBP value, that is:
wherein, LBP (x)c，yc) The representation is in pixels (x)c，yc) Texture information of a 3 × 3 window region as a center; p represents the p-th pixel point except the central pixel point in the 3 multiplied by 3 window; i (c) represents the gray value of the central pixel point, and I (p) represents the gray value of the p-th pixel point in the 3 multiplied by 3 window; s (x) formula is as follows:
in this embodiment, the edge feature is described by using a Canny operator, after the image is smoothed and filtered, the gradient strength and the direction are calculated, then non-maximum suppression is performed, and a dual-threshold algorithm is used to detect and connect edges, so as to obtain the edge feature of the image.
(3) And respectively calculating respective reverse projection drawings by using the obtained two-dimensional histograms of the color, the texture and the edge characteristics, and fusing the three reverse projection drawings by different weights to obtain a reverse projection drawing fused with multi-characteristic information. The color features are used as main feature information for tracking the strip steel, the distribution weight is maximum, and the distribution weight of the texture and the edge features is small.
(4) Judging the shielding condition of the target: calculating the Bhattacharyya distance between the color feature histogram of the target and the color histogram of the search window by adopting the color feature histogram of the target and the color histogram of the search window, and judging that shielding occurs if the calculated Bhattacharyya distance is smaller than a threshold value; if the threshold value is larger than the threshold value, no shielding occurs; in particular, the amount of the solvent to be used,
the method for judging the shielding condition of the target specifically comprises the following steps: judging whether the target is shielded or not by calculating the Bhattacharyya distance between the color feature histogram of the target and the color histogram of the search window:
where ρ (p, p') is an image similarity coefficient, ranging from 0 to 1, 0 representing an extreme difference, and 1 representing an extreme similarity. p and p' respectively represent a color feature histogram of the target and a color histogram of the search window, i is a feature value of the image, and the value range is 1, 2. p (i) and p' (i) are probability distributions of two images having a characteristic value i at the same position on the histogram.
(5) If the target is judged to be shielded, predicting the motion state of the target by adopting a BP (Back Propagation) neural network and a prediction algorithm of a Kalman filter, and using a predicted result for iteration of a Camshift (continuous adaptive mean shift) algorithm of the next frame;
specifically, if the target is judged to be shielded, the stored coordinates of the position points of the target at the first six moments are sequentially subjected to difference to construct a difference matrix, wherein three rows and four columns of the sample difference matrix are input, four pairs of difference values are input, and the difference values are generated by five pairs of point coordinates, wherein two pairs of difference values in each row are generated; three rows and two columns of the output sample difference matrix are provided, three pairs of difference values are generated by four pairs of point coordinates, and one pair of difference values is provided in each row. And training the constructed difference matrix by rows by using a BP neural network. And then, predicting by using the position difference values of the adjacent three moments to obtain the possible position of the target at the next moment, and replacing the observed value in the Kalman filter by using the predicted value of the BP neural network.
The constructed BP neural network model has three layers, the number of the neurons is 4, 9 and 2 in sequence, the activation function is a Sigmoid function, and the highest iteration time is 1000. And replacing the observed value in the Kalman filter with the predicted value of the neural network, wherein the state equation of the Kalman filter is three-order, predicting through the state equation of the three-order Kalman filter to obtain the final optimal estimated value, and using the result in the next frame Camshift algorithm iteration.
Because the strip steel moves on the roller way to change acceleration movement, a variable acceleration movement model is adopted; let the state vector of the target be:
X(k)＝[x(k)，y(k)，dx(k)，dy(k)，d(dx(k))，d(dy(k))]
wherein x (k), y (k), dx (k), dy (k), d (dx (k), d (dy (k)) are target substances respectivelyCoordinates, speed and acceleration of the heart in the directions of the x axis and the y axis when the moment is k; thus, the state transition matrix of the target
Comprises the following steps:
Δ t is the time interval between two adjacent frames of images; the observed state vector of the target, Y (k), is:
Y(k)＝[x(k)，y(k)]
the observation matrix h (k) is obtained from the relationship between the target state and the observation state of the target:
let the dynamic noise covariance matrix q (k) be an identity matrix of 6 × 6 dimensions, and the observation noise covariance matrix r (k) be an identity matrix of 2 × 2 dimensions.
In this embodiment, the adopted basic moving object tracking algorithm is a Camshift algorithm, a chromaticity color histogram, an LBP texture feature histogram and a Canny operator edge feature histogram of the object are obtained, the calculated multi-feature inverse projection graphs are fused, and Camshift search iteration is performed. FIG. 2 is a schematic diagram of the Camshift algorithm. Generally, an input image is converted into an HSV color space, a target area is an initially set search window range, and a chrominance component is separated to be used for calculating a color histogram of the area. Thus, the color histogram of the target template is obtained. The original input image is converted into a color probability distribution image based on the obtained color histogram, a process called "backprojection". And (5) iteratively searching in the reverse projection graph by using a Meanshift algorithm, and calculating the centroid position of the search box. And iterating until the obtained search box meets the convergence condition.
FIG. 3 is a BP neural network structure diagram constructed by the invention, the BP neural network model has three layers, and the number of neurons is 4, 9 and 2 in sequence; the activation function is a Sigmoid function.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (7)
1. A method for tracking a moving target of strip steel under a shielding condition is characterized in that the method is characterized in that the color, texture and edge characteristics are fused by acquiring a moving video of the strip steel on a roller way and utilizing a Camshift moving target tracking algorithm; when the strip steel is in a shielded condition, the BP neural network and the Kalman filter are used for predicting the position of the strip steel, so that the detection and tracking tasks of the strip steel are completed, the loss of a tracking target is avoided, and the tracking stability and the tracking precision of a moving target under a complex background are improved.
2. The method for tracking the strip steel moving target under the shielding condition according to claim 1, which is characterized by comprising the following steps:
(1) collecting a video of the movement of the strip steel on a roller way, carrying out image preprocessing, determining the minimum external rectangular outline of the strip steel by outline detection by using a frame difference method, and determining the strip steel as a tracking starting target;
(2) using the minimum circumscribed rectangle contour determined in the step (1) as an initialization search frame, obtaining a two-dimensional histogram of color features, texture features and edge features of a target, and obtaining a color histogram of a search window;
(3) respectively calculating the two-dimensional histograms of the obtained color features, texture features and edge features, and fusing by different weights to obtain a reverse projection graph fused with multi-feature information;
(4) judging the shielding condition of the target: calculating the Bhattacharyya distance between the color feature histogram of the target and the color histogram of the search window by adopting the color feature histogram of the target and the color histogram of the search window, and judging that shielding occurs if the calculated Bhattacharyya distance is smaller than a threshold value; if the threshold value is larger than the threshold value, no shielding occurs;
(5) if the target is judged to be shielded, predicting the motion state of the target by adopting a prediction algorithm of a BP neural network and a Kalman filter, and using the predicted result for next frame Camshift algorithm iteration;
(6) and outputting a tracking result of the strip steel moving target.
3. The method for tracking the strip steel moving target under the shielding condition according to claim 2, wherein in the step (2), the method for obtaining the two-dimensional histogram of the color features comprises the following steps: the image is converted to an HSV color space, and the two-dimensional histogram of the color feature is a histogram using chrominance components separated from the HSV color space.
4. The method for tracking the moving target of the strip steel under the shielding condition according to claim 2, wherein in the step (2), an LBP operator is adopted to describe the texture characteristics, the texture characteristics are defined in a 3 x 3 window, the gray value of a central pixel point of the window is taken as a threshold value, the gray value is compared with the gray values of adjacent 8 pixel points, if the gray values of surrounding pixel points are greater than the gray value of the central pixel point, the position of the corresponding surrounding pixel point is marked as 1, and if not, the position of the corresponding surrounding pixel point is marked as 0; forming an 8-bit binary number from the binarization result, using the obtained 8-bit binary number as an LBP value of a window center pixel point, and reflecting texture information of a 3 × 3 window area by using the LBP value, that is:
wherein, LBP (x)c，yc) The representation is in pixels (x)c，yc) 3 for centre3 texture information of the window area; p represents the p-th pixel point except the central pixel point in the 3 multiplied by 3 window; i (c) represents the gray value of the central pixel point, and I (p) represents the gray value of the p-th pixel point in the 3 multiplied by 3 window; s (x) formula is as follows:
5. the method according to claim 2, wherein in the step (2), the edge features are described by using a Canny operator, after the image is smoothed and filtered, gradient strength and direction are calculated, then non-maximum suppression is performed, and a dual-threshold algorithm is used for detecting and connecting edges to obtain the edge features of the image.
6. The method for tracking the strip steel moving target under the shielding condition according to claim 2, wherein in the step (4), the judgment of the shielding condition of the target is specifically as follows: judging whether the target is shielded or not by calculating the Bhattacharyya distance between the color feature histogram of the target and the color histogram of the search window:
wherein ρ (p, p ') is an image similarity coefficient, the range is between 0 and 1, 0 represents extremely different, 1 represents extremely similar, p, p' respectively represent a color feature histogram of a target and a color histogram of a search window, i is a feature value of an image, and the value range is 1, 2.. multidot.N; p (i) and p' (i) are probability distributions of two images having a characteristic value i at the same position on the histogram.
7. The method for tracking the strip steel moving target under the shielding condition according to claim 2, wherein the step (5) is specifically as follows: if the target is judged to be shielded, sequentially subtracting the stored position point coordinates of the target at the first six moments to construct a difference matrix, inputting three rows and four columns of the sample difference matrix, and generating four pairs of difference values by five pairs of point coordinates, wherein each row comprises two pairs of difference values; outputting three rows and two columns of the sample difference matrix, wherein three pairs of difference values are generated by four pairs of coordinates, and one pair of difference values is generated in each row; training the constructed difference matrix by using a BP neural network according to rows, then predicting by using position difference values of three adjacent moments to obtain the possible position of the target at the next moment, and replacing an observed value in a Kalman filter with a predicted value of the BP neural network;
the constructed BP neural network model has three layers, the number of the neurons is 4, 9 and 2 in sequence, the activation function is a Sigmoid function, the highest iteration time is 1000 times, the predicted value of the BP neural network replaces the observed value in a Kalman filter, the state equation of the Kalman filter is three-order, the prediction is carried out through the state equation of the three-order Kalman filter to obtain the final optimal estimated value, and the result is used for the next frame of Camshift algorithm iteration;
because the strip steel moves on the roller way to change acceleration movement, a variable acceleration movement model is adopted; let the state vector of the target be:
X(k)＝[x(k)，y(k)，dx(k)，dy(k)，d(dx(k))，d(dy(k))]
wherein x (k), y (k), dx (k), dy (k), d (dx (k)), and d (dy (k)) are coordinates, speed and acceleration of the target centroid in the x-axis and y-axis directions at the moment of k, respectively; thus, the state transition matrix of the target
Comprises the following steps:
Δ t is the time interval between two adjacent frames of images; the observed state vector of the target, Y (k), is:
Y(k)＝[x(k)，y(k)]
the observation matrix h (k) is obtained from the relationship between the target state and the observation state of the target:
let the dynamic noise covariance matrix q (k) be an identity matrix of 6 × 6 dimensions, and the observation noise covariance matrix r (k) be an identity matrix of 2 × 2 dimensions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010733819.4A CN111915649A (en) | 2020-07-27 | 2020-07-27 | Strip steel moving target tracking method under shielding condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010733819.4A CN111915649A (en) | 2020-07-27 | 2020-07-27 | Strip steel moving target tracking method under shielding condition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111915649A true CN111915649A (en) | 2020-11-10 |
Family
ID=73281892
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010733819.4A Pending CN111915649A (en) | 2020-07-27 | 2020-07-27 | Strip steel moving target tracking method under shielding condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111915649A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114937231A (en) * | 2022-07-21 | 2022-08-23 | 成都西物信安智能系统有限公司 | Method for improving target identification tracking accuracy |
| CN116523893A (en) * | 2023-05-16 | 2023-08-01 | 华院计算技术(上海)股份有限公司 | Flying steel detection system and method based on machine vision |
| CN117392179A (en) * | 2023-12-11 | 2024-01-12 | 四川迪晟新达类脑智能技术有限公司 | Target tracking method based on correlation filter and edge frame |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081801A (en) * | 2011-01-26 | 2011-06-01 | 上海交通大学 | Multi-feature adaptive fused ship tracking and track detecting method |
| CN103077539A (en) * | 2013-01-23 | 2013-05-01 | 上海交通大学 | Moving object tracking method under complicated background and sheltering condition |
| CN104200485A (en) * | 2014-07-10 | 2014-12-10 | 浙江工业大学 | Video-monitoring-oriented human body tracking method |
| US20150009323A1 (en) * | 2013-07-03 | 2015-01-08 | Zmodo Technology Shenzhen Corp. Ltd | Multi-target tracking method for video surveillance |
| US20150178568A1 (en) * | 2013-12-23 | 2015-06-25 | Canon Kabushiki Kaisha | Method for improving tracking using dynamic background compensation with centroid compensation |
| CN110472607A (en) * | 2019-08-21 | 2019-11-19 | 上海海事大学 | A kind of ship tracking method and system |
-
2020
- 2020-07-27 CN CN202010733819.4A patent/CN111915649A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081801A (en) * | 2011-01-26 | 2011-06-01 | 上海交通大学 | Multi-feature adaptive fused ship tracking and track detecting method |
| CN103077539A (en) * | 2013-01-23 | 2013-05-01 | 上海交通大学 | Moving object tracking method under complicated background and sheltering condition |
| US20150009323A1 (en) * | 2013-07-03 | 2015-01-08 | Zmodo Technology Shenzhen Corp. Ltd | Multi-target tracking method for video surveillance |
| US20150178568A1 (en) * | 2013-12-23 | 2015-06-25 | Canon Kabushiki Kaisha | Method for improving tracking using dynamic background compensation with centroid compensation |
| CN104200485A (en) * | 2014-07-10 | 2014-12-10 | 浙江工业大学 | Video-monitoring-oriented human body tracking method |
| CN110472607A (en) * | 2019-08-21 | 2019-11-19 | 上海海事大学 | A kind of ship tracking method and system |
Non-Patent Citations (3)
| Title |
|---|
| BIN ZHENG 等: "Object Tracking Algorithm Based on Combination of Dynamic Template Matching and Kalman Filter", 《2012 4TH INTERNATIONAL CONFERENCE ON INTELLIGENT HUMAN-MACHINE SYSTEMS AND CYBERNETICS》, pages 136 - 139 * |
| 仇男豪 等: "改进的Camshift与Kalman滤波联合跟踪算法", 《电子设计工程》, vol. 28, no. 14, pages 11 - 15 * |
| 郭世凯 等: "基于改进的Camshift算法的室内小车跟踪", 《电子测量技术》, vol. 42, no. 12, pages 105 - 111 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114937231A (en) * | 2022-07-21 | 2022-08-23 | 成都西物信安智能系统有限公司 | Method for improving target identification tracking accuracy |
| CN116523893A (en) * | 2023-05-16 | 2023-08-01 | 华院计算技术(上海)股份有限公司 | Flying steel detection system and method based on machine vision |
| CN116523893B (en) * | 2023-05-16 | 2023-12-26 | 华院计算技术(上海)股份有限公司 | Flying steel detection system and method based on machine vision |
| CN117392179A (en) * | 2023-12-11 | 2024-01-12 | 四川迪晟新达类脑智能技术有限公司 | Target tracking method based on correlation filter and edge frame |
| CN117392179B (en) * | 2023-12-11 | 2024-02-27 | 四川迪晟新达类脑智能技术有限公司 | Target tracking method based on correlation filter and edge frame |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Efficient detection model of steel strip surface defects based on YOLO-V7 | |
| KR102129893B1 (en) | Ship tracking method and system based on deep learning network and average movement | |
| CN111915649A (en) | Strip steel moving target tracking method under shielding condition | |
| KR20200007084A (en) | Ship detection method and system based on multi-dimensional features of scene | |
| CN106204484B (en) | Traffic target tracking method based on optical flow and local invariant features | |
| Chen et al. | An intelligent sewer defect detection method based on convolutional neural network | |
| CN118096815B (en) | Road abnormal event detection system based on machine vision | |
| CN113065431B (en) | Human body violation prediction method based on hidden Markov model and recurrent neural network | |
| CN111860143A (en) | Real-time flame detection method for inspection robot | |
| CN116486287A (en) | Target detection method and system based on environment self-adaptive robot vision system | |
| CN111986237A (en) | Real-time multi-target tracking algorithm irrelevant to number of people | |
| CN109558877B (en) | KCF-based offshore target tracking algorithm | |
| Chen et al. | Intrusion detection of specific area based on video | |
| CN113536934B (en) | Active hiding method for patrol robot during tracking task execution | |
| Xu et al. | Moving target tracking based on adaptive background subtraction and improved camshift algorithm | |
| Liao et al. | Multi-scale ship tracking based on maritime monitoring platform | |
| CN115409768A (en) | Water leakage disease deep learning detection method based on characteristic input enhancement | |
| Han et al. | Damage detection of quayside crane structure based on improved faster R-CNN | |
| Meng et al. | An extraction method of fish vector field for marine cage culture based on UAV vision | |
| Xiaojun et al. | Tracking of moving target based on video motion nuclear algorithm | |
| CN116343125B (en) | Container bottom lock head detection method based on computer vision | |
| Wu et al. | Detection of abnormal behavior of plant personnel based on YOLOv8 and HS optical flow method | |
| CN202720689U (en) | A motioning target detecting and tracking system based on a DSP | |
| Tianwei et al. | Feature Extraction and Accurate Localization Based on Hough Transform in Complex Scenes | |
| Yu et al. | Infrared Fault Detection Method for Dense Electrolytic Bath Polar Plate Based on YOLOv5s. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |