CN107909024B - Vehicle tracking system and method based on image recognition and infrared obstacle avoidance and vehicle - Google Patents
Vehicle tracking system and method based on image recognition and infrared obstacle avoidance and vehicle Download PDFInfo
- Publication number
- CN107909024B CN107909024B CN201711116535.5A CN201711116535A CN107909024B CN 107909024 B CN107909024 B CN 107909024B CN 201711116535 A CN201711116535 A CN 201711116535A CN 107909024 B CN107909024 B CN 107909024B
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- classifier
- obstacle avoidance
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Abstract
The invention provides a vehicle tracking system based on image recognition and infrared obstacle avoidance, which comprises: the system comprises an upper computer monitoring subsystem, a tracking obstacle avoidance subsystem and a vehicle control subsystem; the upper computer monitoring subsystem is used for monitoring the advancing speed and the advancing direction of the vehicle; the tracking obstacle avoidance subsystem is used for tracking and avoiding obstacles of the vehicle; the vehicle control subsystem is used for controlling the forward speed and the forward direction of the vehicle; the upper computer monitoring subsystem comprises a Wifi receiving module, a display module and a monitoring module; the tracking obstacle avoidance subsystem comprises: the system comprises a camera module, an infrared sensor module, a Wifi sending module, an embedded image processing development board and a tracking obstacle avoidance processing module; the method solves the problems that the method is easily influenced by illumination, shielding, object deformation and the like caused by a simple algorithm in the prior art; but also solves the technical problem that the real-time performance of a relatively complex algorithm is poor.
Description
Technical Field
The invention relates to the field of intelligent robots, in particular to a vehicle tracking system and method based on image recognition and infrared obstacle avoidance and a vehicle.
Background
The long-time real-time object tracking has great application prospect in face recognition, autonomous navigation, vehicle recognition and visual early warning. Although the real-time performance of the currently adopted simple algorithm is good, the algorithm is easily influenced by illumination, shielding, object deformation and the like; the adopted relatively complex algorithm is not easily influenced by the outside, has high precision, but has poor real-time performance. In addition, the two algorithms do not realize the process of synchronously transmitting the front-end camera information to the upper computer monitor.
Disclosure of Invention
The invention aims to solve the problems and provides a vehicle tracking system and method based on image recognition and infrared obstacle avoidance and a vehicle.
The invention can be realized by adopting the following system: a vehicle tracking system based on image recognition and infrared obstacle avoidance comprises: the system comprises an upper computer monitoring subsystem, a tracking obstacle avoidance subsystem and a vehicle control subsystem;
the upper computer monitoring subsystem is used for monitoring the advancing speed and the advancing direction of the vehicle; the tracking obstacle avoidance subsystem is used for tracking and avoiding obstacles of the vehicle; the vehicle control subsystem is used for controlling the forward speed and the forward direction of the vehicle;
the upper computer monitoring subsystem comprises a Wifi receiving module, a display module and a monitoring module;
the tracking obstacle avoidance subsystem comprises: the system comprises a camera module, an infrared sensor module, a Wifi sending module, an embedded image processing development board and a tracking obstacle avoidance processing module;
the tracking obstacle avoidance processing module comprises: a Linux operating system kernel submodule, an ROS submodule and a TLD calculation submodule;
the vehicle control subsystem includes: a controller and a drive motor;
the Wifi sending module is used for sending the real-time image of the vehicle to be tracked, which is acquired by the camera module, to the monitoring subsystem of the upper computer; the Wifi receiving module is used for receiving the real-time image of the vehicle to be tracked, which is acquired by the camera module; the embedded image processing development board is used for acquiring real-time image information of the vehicle to be tracked, which is acquired by the camera module; the tracking obstacle avoidance processing module is used for processing the data acquired by the embedded image processing development board;
the TLD calculation submodule comprises; the image processing system comprises a target model improving unit, a nearest neighbor image slice improving unit, a classifier optimizing unit and a GPU accelerating unit; the target model improving unit is used for improving a target model; the nearest neighbor image slice improving unit is used for improving the nearest neighbor image slice; the classifier optimizing unit is used for optimizing a classifier; the GPU acceleration unit is used for accelerating the calculation speed of the TLD calculation submodule;
the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, acquired by a camera module, as an offline training sample, and automatically reading parameters of a first frame image after offline training is completed to initialize a TLD calculation submodule; the parameters include: the position values of the image coordinate points of the nodes of the ten trees of the set classifier are obtained through all the patches of the set classifier, all the positive and negative samples of the nearest neighbor classifier are obtained through the position values of the image coordinate points, and the threshold value of each classifier of the cascade classifier is also obtained;
the controller is used for controlling the driving motor according to a real-time detection result of the infrared sensor module and a result of processing data of the tracking obstacle avoidance processing module, wherein the data refers to data acquired by the embedded image processing development board.
Further, the specific method for improving the nearest neighbor image slice is as follows: according to a time and reliability calculation method, taking out positive sample image slices and negative sample image slices with time and reliability within a preset value range from an off-line training sample, wherein the sum of the number of the positive sample image slices and the negative sample image slices is 20.
Further, the specific method for optimizing the classifier is as follows: eliminating the image frames of the vehicles exceeding the preset distance by a distance clustering method, clustering the image frames of the vehicles within the preset distance range, calculating a new target frame, and re-evaluating by using a nearest neighbor method; and judging that when the result of passing through the nearest neighbor classifier is 0 patch, re-aggregating the set classifier, and sending the aggregated patch to the nearest neighbor classifier for re-training by using the nearest neighbor classification threshold value.
Further, the GPU acceleration unit uses a GPU parallel acceleration method, specifically uses a CUDA platform to change serial operation into multi-thread parallel operation.
The invention is realized by adopting the following method: a vehicle tracking method based on image identification and infrared obstacle avoidance,
s201, acquiring a real-time image of a vehicle to be tracked through a camera;
s202, acquiring real-time image information of the vehicle to be tracked through the embedded image processing development board,
s203, processing the acquired real-time image information by using a TLD tracking method, and calculating the advancing speed and the advancing direction of the vehicle to be tracked;
s204, controlling a driving motor according to a real-time detection result of the infrared sensor and a result of tracking obstacle avoidance processing data, wherein the data is acquired by the embedded image processing development board; uploading the real-time image information of the tracked vehicle to an upper computer for monitoring through a WIFI module;
wherein the TLD tracking method comprises: improving an objective model, improving a nearest neighbor image slice and optimizing a classifier;
the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, acquired by a camera module, as an offline training sample, and automatically reading parameters of a first frame image after offline training is completed to initialize a TLD calculation submodule; the parameters include: the position values of the image coordinate points of the nodes of the ten trees of the set classifier are obtained through all the patches of the set classifier, all the positive and negative samples of the nearest neighbor classifier are obtained through the position values of the image coordinate points, and the threshold value of each classifier of the cascade classifier is also obtained;
further, the specific method for improving the nearest neighbor image slice is as follows: and according to the time and reliability calculation method, taking out the positive sample image slice and the negative sample image slice with the time and the reliability within the preset value range from the off-line training sample.
Further, the sum of the number of the positive sample image slices and the negative sample image slices is 20.
Further, the specific method for optimizing the classifier is as follows: eliminating the image frames of the vehicles exceeding the preset distance by a distance clustering method, clustering the image frames of the vehicles within the preset distance range, calculating a new target frame, and re-evaluating by using a nearest neighbor method; and judging that when the result of passing through the nearest neighbor classifier is 0 patch, re-aggregating the set classifier, and sending the aggregated patch to the nearest neighbor classifier for re-training by using the nearest neighbor classification threshold value.
Further, the GPU acceleration unit uses a GPU parallel acceleration method, specifically uses a CUDA platform to change serial operation into multi-thread parallel operation.
The invention also provides a vehicle based on image identification and infrared obstacle avoidance, which is provided with the vehicle tracking system based on image identification and infrared obstacle avoidance as claimed in any one of claims 1 to 5.
To sum up, a vehicle tracking system based on image recognition and infrared obstacle avoidance includes: the system comprises an upper computer monitoring subsystem, a tracking obstacle avoidance subsystem and a vehicle control subsystem; the upper computer monitoring subsystem is used for monitoring the advancing speed and the advancing direction of the vehicle; the tracking obstacle avoidance subsystem is used for tracking and avoiding obstacles of the vehicle; the vehicle control subsystem is used for controlling the forward speed and the forward direction of the vehicle; the upper computer monitoring subsystem comprises a Wifi receiving module, a display module and a monitoring module; the tracking obstacle avoidance subsystem comprises: the system comprises a camera module, an infrared sensor module, a Wifi sending module, an embedded image processing development board and a tracking obstacle avoidance processing module; the tracking obstacle avoidance processing module comprises: a Linux operating system kernel submodule, an ROS submodule and a TLD calculation submodule; the vehicle control subsystem includes: a controller and a drive motor; the Wifi sending module is used for sending the real-time image of the vehicle to be tracked, which is acquired by the camera module, to the monitoring subsystem of the upper computer; the Wifi receiving module is used for receiving the real-time image of the vehicle to be tracked, which is acquired by the camera module; the embedded image processing development board is used for acquiring real-time image information of the vehicle to be tracked, which is acquired by the camera module; the tracking obstacle avoidance processing module is used for processing the data acquired by the embedded image processing development board; the TLD calculation submodule comprises; the image processing system comprises a target model improving unit, a nearest neighbor image slice improving unit, a classifier optimizing unit and a GPU accelerating unit; the target model improving unit is used for improving a target model; the nearest neighbor image slice improving unit is used for improving the nearest neighbor image slice; the classifier optimizing unit is used for optimizing a classifier; the GPU acceleration unit is used for accelerating the calculation speed of the TLD calculation submodule; the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, which is acquired by a camera module, as an offline training sample, automatically reading parameters of a first frame image after offline training is completed, initializing a TLD calculation submodule, and only storing a nearest neighbor image slice and a variance value; the controller is used for controlling the driving motor according to a real-time detection result of the infrared sensor module and a result of processing data of the tracking obstacle avoidance processing module, wherein the data refers to data acquired by the embedded image processing development board.
The beneficial effects are that:
1. the vehicle tracking system can track the vehicle to be tracked in real time through the camera, and carry out obstacle avoidance detection on the surrounding environment through the infrared sensor;
2. the real-time image of the vehicle to be tracked acquired by the camera module is saved as an off-line training sample by using an improved target model algorithm, and the parameter of the first frame image after off-line training is automatically read to initialize the TLD calculation algorithm, so that the off-line detection and on-line learning capacity are realized; the occupied space of the model is reduced, so that the time for storing and loading the target model is greatly shortened;
drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a block diagram of an embodiment of a vehicle tracking system based on image recognition and infrared obstacle avoidance provided by the present invention;
FIG. 2 is a flowchart of an embodiment of a vehicle tracking method based on image recognition and infrared obstacle avoidance provided by the present invention;
fig. 3 is a structural diagram of a tracking obstacle avoidance processing module in the vehicle tracking system based on image recognition and infrared obstacle avoidance provided by the invention.
Detailed Description
The present invention provides an embodiment of a vehicle tracking system and method based on image recognition and infrared obstacle avoidance, and in order to make the technical solutions in the embodiment of the present invention better understood and make the above objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the present invention are further described in detail below with reference to the accompanying drawings:
the invention firstly provides a vehicle tracking system based on image recognition and infrared obstacle avoidance, as shown in fig. 1, an upper computer monitoring subsystem 10, a tracking obstacle avoidance subsystem 20 and a vehicle control subsystem 30;
the upper computer monitoring subsystem 10 is used for monitoring the forward speed and the forward direction of the vehicle; the tracking obstacle avoidance subsystem 20 is used for tracking and avoiding obstacles of the vehicle; the vehicle control subsystem 30 is used for controlling the forward speed and the forward direction of the vehicle;
the upper computer monitoring subsystem 10 comprises a Wifi receiving module 101, a display module 102 and a monitoring module 103;
the tracking obstacle avoidance subsystem 20 includes: the system comprises a camera module 201, an infrared sensor module 202, a Wifi sending module 203, an embedded image processing and developing board 204 and a tracking obstacle avoidance processing module 205;
the tracking and obstacle avoidance processing module 205 includes: a Linux operating system kernel submodule 2051, an ROS submodule 2052, and a TLD calculation submodule 2053;
the vehicle control subsystem 30 includes: a controller 301 and a drive motor 302;
the upper computer monitoring subsystem is a P2P server client receiving model based on a Linux system, and receives and displays real-time data collected by a camera transmitted by the WIFI module. The pictures stored in the specified path by the TLD calculation method can be read in real time through a daemon Client on the embedded image processing development board, and the pictures are sent to the host computer for real-time monitoring through a Linux network programming method.
Wherein, the controller has adopted the STM32 controller, and the controller uses driving motor drive mecanum omniwheel to remove according to handling the data result, reaches the effect of tracking, and meanwhile, infrared sensor surveys real-time all ring edge borders, returns the data that detect STM32 controller and keeps away the barrier and judges.
The working principle of the tracking obstacle avoidance algorithm is as follows: the automatic tracking of the target is realized by improving a TLD algorithm and an ROS system, and the automatic target tracking algorithm comprises the following steps: the target detection module, the target learning module and the target tracking module achieve the long-time target tracking effect through mutual supervision and learning.
The Wifi sending module 203 is used for sending the real-time image of the vehicle to be tracked, which is acquired by the camera module 201, to the upper computer monitoring subsystem 10; the Wifi receiving module 101 is configured to receive a real-time image of the vehicle to be tracked, which is acquired by the camera module 201; the embedded image processing and developing board 204 is used for acquiring real-time image information of the vehicle to be tracked, which is acquired by the camera module 201; the tracking obstacle avoidance processing module 205 is configured to process data acquired by the embedded image processing development board;
the TLD calculation sub-module 2053 includes; a target model improvement unit 20531, a nearest neighbor image slice improvement unit 20532, a classifier optimization unit 20533, and a GPU acceleration unit 20534; the object model improvement unit 20531 is configured to improve an object model; the nearest neighbor image patch improving unit 20532 is configured to improve the nearest neighbor image patch; the classifier optimizing unit 20533 is configured to optimize a classifier; the GPU acceleration unit 20534 is configured to accelerate the computation speed of the TLD computation submodule;
the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, acquired by a camera module, as an offline training sample, and automatically reading parameters of a first frame image after offline training is completed to initialize a TLD calculation submodule; the parameters include: the position values of the image coordinate points of the nodes of the ten trees of the set classifier are obtained through all the patches of the set classifier, all the positive and negative samples of the nearest neighbor classifier are obtained through the position values of the image coordinate points, and the threshold value of each classifier of the cascade classifier is also obtained;
wherein the parameters of the first frame image comprise all data of the first frame passing through the variance classifier; including but not limited to positive and negative samples through set classifiers, positive and negative samples through nearest neighbor classifiers, individual thresholds of cascaded classifiers, thresholds of variance classifiers, prior probabilities of set classifiers, thresholds of nearest neighbor classifiers, 20 online models, 10 set classifiers, image location of each node in each tree, etc.; the initialization process is that the vector data is stored in a file in a matrix form, and the data in the file is directly read when the computer is started for the second time, so that initialization is carried out; the number of the online models is limited by the adopted nearest neighbor classifier, the first 10 models are initialized and are not moved, and the last 10 models are circularly updated by a Round robin algorithm.
The controller is used for controlling the driving motor according to a real-time detection result of the infrared sensor module and a result of processing data of the tracking obstacle avoidance processing module, wherein the data refers to data acquired by the embedded image processing development board.
Preferably, the specific method for improving the nearest neighbor image slice is as follows: and according to the time and reliability calculation method, taking out the positive sample image slice and the negative sample image slice with the time and the reliability within the preset value range from the off-line training sample.
The improvement can reduce the image slices in the target model of nearest neighbor calculation and matching, thereby reducing the calculation amount and the occupied running memory
Preferably, the sum of the number of the positive sample image slices and the negative sample image slices is 20.
Preferably, the specific method for optimizing the classifier is as follows: eliminating the image frames of the vehicles exceeding the preset distance by a distance clustering method, clustering the image frames of the vehicles within the preset distance range, calculating a new target frame, and re-evaluating by using a nearest neighbor method; and judging that when the result of passing through the nearest neighbor classifier is 0 patch, re-aggregating the set classifier, and sending the aggregated patch to the nearest neighbor classifier for re-training by using the nearest neighbor classification threshold value. .
Preferably, the GPU acceleration unit uses a GPU parallel acceleration method, specifically, a CUDA platform is used to change serial operation into parallel operation of multiple threads, multiple scanning windows are processed simultaneously by multiple threads, and the processing result is returned for evaluation.
The invention also provides an embodiment of a vehicle tracking method based on image recognition and infrared obstacle avoidance, as shown in fig. 2, the method comprises the following steps:
the invention is realized by adopting the following method: a vehicle tracking method based on image identification and infrared obstacle avoidance,
s201, acquiring a real-time image of a vehicle to be tracked through a camera;
s202, acquiring real-time image information of a vehicle to be tracked through an embedded image processing development board;
s203, processing the acquired real-time image information by using a TLD tracking method, and calculating the advancing speed and the advancing direction of the vehicle to be tracked;
s204, controlling a driving motor according to a real-time detection result of the infrared sensor and a result of tracking obstacle avoidance processing data, wherein the data is acquired by the embedded image processing development board; uploading the real-time image information of the tracked vehicle to an upper computer for monitoring through a WIFI module;
wherein the TLD tracking method comprises: improving an objective model, improving a nearest neighbor image slice and optimizing a classifier;
the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, acquired by a camera module, as an offline training sample, and automatically reading parameters of a first frame image after offline training is completed to initialize a TLD calculation submodule; the parameters include: the position values of the image coordinate points of the nodes of the ten trees of the set classifier are obtained through all the patches of the set classifier, all the positive and negative samples of the nearest neighbor classifier are obtained through the position values of the image coordinate points, and the threshold value of each classifier of the cascade classifier is also obtained;
preferably, the specific method for improving the nearest neighbor image slice is as follows: according to a time and reliability calculation method, taking out positive sample image slices and negative sample image slices with time and reliability within a preset value range from an off-line training sample, wherein the sum of the number of the positive sample image slices and the negative sample image slices is 20.
The method reduces the image slices in the nearest neighbor calculation and matched target model, and reduces the calculation amount and the occupied running memory.
Preferably, the specific method for optimizing the classifier is as follows: eliminating the image frames of the vehicles exceeding the preset distance by a distance clustering method, clustering the image frames of the vehicles within the preset distance range, calculating a new target frame, and re-evaluating by using a nearest neighbor method; and judging that when the result of passing through the nearest neighbor classifier is 0 patch, re-aggregating the set classifier, and sending the aggregated patch to the nearest neighbor classifier for re-training by using the nearest neighbor classification threshold value.
Preferably, the GPU acceleration unit uses a GPU parallel acceleration method, specifically, a CUDA platform is used to change serial operation into parallel operation of multiple threads.
The invention also provides a vehicle based on image identification and infrared obstacle avoidance, which is provided with the vehicle tracking system based on image identification and infrared obstacle avoidance as claimed in any one of claims 1 to 5.
In conclusion, the invention utilizes the time and credibility compiling algorithm to take out the most representative positive image slices and negative image slices from the model and limits the number of the positive image slices and the negative image slices to be 20, thereby reducing the image slices in the target model for nearest neighbor calculation and matching, and reducing the calculation amount and the occupied running memory; the invention also eliminates the image frames far away from the target through a distance clustering algorithm, clusters the image frames near the target, synthesizes a new target frame, and re-evaluates by using nearest neighbor, so that the classification effect of the classifier is more accurate; in addition, the invention uses CUDA multithread parallel computing technology to optimize the algorithm, and changes the original serial computing method on the CPU into parallel computing, thereby improving the running speed of the algorithm and shortening the running time.
The invention provides a vehicle tracking system based on image recognition and infrared obstacle avoidance, which comprises: the system comprises an upper computer monitoring subsystem, a tracking obstacle avoidance subsystem and a vehicle control subsystem; the upper computer monitoring subsystem is used for monitoring the advancing speed and the advancing direction of the vehicle; the tracking obstacle avoidance subsystem is used for tracking and avoiding obstacles of the vehicle; the vehicle control subsystem is used for controlling the forward speed and the forward direction of the vehicle; the upper computer monitoring subsystem comprises a Wifi receiving module, a display module and a monitoring module; the tracking obstacle avoidance subsystem comprises: the system comprises a camera module, an infrared sensor module, a Wifi sending module, an embedded image processing development board and a tracking obstacle avoidance processing module; the tracking obstacle avoidance processing module comprises: a Linux operating system kernel submodule, an ROS submodule and a TLD calculation submodule; the vehicle control subsystem includes: a controller and a drive motor; the Wifi sending module is used for sending the real-time image of the vehicle to be tracked, which is acquired by the camera module, to the monitoring subsystem of the upper computer; the Wifi receiving module is used for receiving the real-time image of the vehicle to be tracked, which is acquired by the camera module; the embedded image processing development board is used for acquiring real-time image information of the vehicle to be tracked, which is acquired by the camera module; the tracking obstacle avoidance processing module is used for processing the data acquired by the embedded image processing development board; the TLD calculation submodule comprises; the image processing system comprises a target model improving unit, a nearest neighbor image slice improving unit, a classifier optimizing unit and a GPU accelerating unit; the target model improving unit is used for improving a target model; the nearest neighbor image slice improving unit is used for improving the nearest neighbor image slice; the classifier optimizing unit is used for optimizing a classifier; and the GPU acceleration unit is used for accelerating the calculation speed of the TLD calculation submodule. The method only can manually set an initialization target every time by changing the original algorithm, and then tracks the process, so that the information of the first frame of picture data is saved as initialization; the invention accelerates the variance classifier, an original algorithm runs on a cpu, and the computation of the variance classifier is accelerated by using an embedded board of a TK1 board of NVIDIA with a display card and by using the GPU for parallel computation. The number of online models is limited by the nearest neighbor classifier, the problems that the original algorithm is increased without limit and the embedded platform cannot support the number of classifiers passing through the cascade are solved, the aggregation algorithm is applied, multiple pictures are normalized, and the tracking accuracy is guaranteed.
The above examples are intended to illustrate but not to limit the technical solutions of the present invention. Any modification or partial replacement without departing from the spirit and scope of the present invention should be covered in the claims of the present invention.
Claims (5)
1. The utility model provides a vehicle tracking system based on image recognition and infrared obstacle avoidance which characterized in that includes: the system comprises an upper computer monitoring subsystem, a tracking obstacle avoidance subsystem and a vehicle control subsystem;
the upper computer monitoring subsystem is used for monitoring the advancing speed and the advancing direction of the vehicle; the tracking obstacle avoidance subsystem is used for tracking and avoiding obstacles of the vehicle; the vehicle control subsystem is used for controlling the forward speed and the forward direction of the vehicle;
the upper computer monitoring subsystem comprises a Wifi receiving module, a display module and a monitoring module;
the tracking obstacle avoidance subsystem comprises: the system comprises a camera module, an infrared sensor module, a Wifi sending module, an embedded image processing development board and a tracking obstacle avoidance processing module; the tracking obstacle avoidance processing module comprises: a Linux operating system kernel submodule, an ROS submodule and a TLD calculation submodule;
the vehicle control subsystem includes: a controller and a drive motor;
the Wifi sending module is used for sending the real-time image of the vehicle to be tracked, which is acquired by the camera module, to the monitoring subsystem of the upper computer; the Wifi receiving module is used for receiving the real-time image of the vehicle to be tracked, which is acquired by the camera module; the embedded image processing development board is used for acquiring real-time image information of the vehicle to be tracked, which is acquired by the camera module; the tracking obstacle avoidance processing module is used for processing the data acquired by the embedded image processing development board;
the TLD calculation submodule comprises: the image processing system comprises a target model improving unit, a nearest neighbor image slice improving unit, a classifier optimizing unit and a GPU accelerating unit; the target model improving unit is used for improving a target model; the nearest neighbor image slice improving unit is used for improving the nearest neighbor image slice; the classifier optimizing unit is used for optimizing a classifier; the GPU acceleration unit is used for accelerating the calculation speed of the TLD calculation submodule;
the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, acquired by a camera module, as an offline training sample, and automatically reading parameters of a first frame image after offline training is completed to initialize a TLD calculation submodule; the parameters include: all image slices passing through the set classifier, coordinate values of images of ten tree nodes of the set classifier, all positive and negative sample image slices passing through the nearest neighbor classifier and a threshold value of each classifier in the cascade classifier;
the specific method for improving the nearest neighbor image slice comprises the following steps: according to a time and reliability calculation method, taking out positive sample image slices and negative sample image slices with time and reliability within a preset value range from an offline training sample, wherein the sum of the number of the positive sample image slices and the negative sample image slices is 20;
the specific method for optimizing the classifier comprises the following steps: eliminating the image frames of the vehicles exceeding the preset distance by a distance clustering method, clustering the image frames of the vehicles within the preset distance range, calculating a new target frame, and re-evaluating by using a nearest neighbor method; when the result of passing through the nearest neighbor classifier is judged to be 0 image slice, the image slices which need to pass through the set classifier are aggregated again to obtain a nearest neighbor classification threshold value, and the aggregated image slices are sent to the nearest neighbor classifier for retraining;
the controller is used for controlling the driving motor according to a real-time detection result of the infrared sensor module and a result of processing data of the tracking obstacle avoidance processing module, wherein the data refers to data acquired by the embedded image processing development board.
2. The image recognition and infrared obstacle avoidance based vehicle tracking system of claim 1, wherein the GPU acceleration unit uses a GPU parallel acceleration method, in particular, a CUDA platform is used to change serial operation into multi-threaded parallel operation.
3. A vehicle tracking method based on image recognition and infrared obstacle avoidance is characterized by comprising the following steps:
s201, acquiring a real-time image of a vehicle to be tracked through a camera;
s202, acquiring real-time image information of the vehicle to be tracked through the embedded image processing development board,
s203, processing the acquired real-time image information by using a TLD tracking method, and calculating the advancing speed and the advancing direction of the vehicle to be tracked;
s204, controlling a driving motor according to a real-time detection result of the infrared sensor and a result of tracking obstacle avoidance processing data, wherein the data is acquired by the embedded image processing development board; uploading the real-time image information of the tracked vehicle to an upper computer for monitoring through a WIFI module;
wherein the TLD tracking method comprises: improving an objective model, improving a nearest neighbor image slice and optimizing a classifier;
the specific method for improving the target model comprises the following steps: the method comprises the steps of storing a real-time image of a vehicle to be tracked, acquired by a camera module, as an offline training sample, and automatically reading parameters of a first frame image after offline training is completed to initialize a TLD calculation submodule; accelerating the calculation speed of the TLD calculation submodule by using a GPU acceleration unit; the parameters include: all image slices passing through the set classifier, coordinate values of images of ten tree nodes of the set classifier, all positive and negative sample image slices passing through the nearest neighbor classifier and a threshold value of each classifier in the cascade classifier;
the specific method for improving the nearest neighbor image slice comprises the following steps: according to a time and reliability calculation method, taking out positive sample image slices and negative sample image slices with time and reliability within a preset value range from an offline training sample, wherein the sum of the number of the positive sample image slices and the negative sample image slices is 20;
the specific method for optimizing the classifier comprises the following steps: eliminating the image frames of the vehicles exceeding the preset distance by a distance clustering method, clustering the image frames of the vehicles within the preset distance range, calculating a new target frame, and re-evaluating by using a nearest neighbor method; and when the result of passing through the nearest neighbor classifier is 0 image slice, aggregating the image slices passing through the set classifier again to obtain a nearest neighbor classification threshold value, and sending the aggregated image slices to the nearest neighbor classifier for retraining.
4. The image recognition and infrared obstacle avoidance-based vehicle tracking method of claim 3, wherein the GPU acceleration unit uses a GPU parallel acceleration method, in particular uses a CUDA platform to change serial operation into multi-threaded parallel operation.
5. A vehicle based on image recognition and infrared obstacle avoidance, characterized in that a vehicle tracking system based on image recognition and infrared obstacle avoidance as claimed in any one of claims 1-2 is installed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711116535.5A CN107909024B (en) | 2017-11-13 | 2017-11-13 | Vehicle tracking system and method based on image recognition and infrared obstacle avoidance and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711116535.5A CN107909024B (en) | 2017-11-13 | 2017-11-13 | Vehicle tracking system and method based on image recognition and infrared obstacle avoidance and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107909024A CN107909024A (en) | 2018-04-13 |
| CN107909024B true CN107909024B (en) | 2021-11-05 |
Family
ID=61843896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711116535.5A Active CN107909024B (en) | 2017-11-13 | 2017-11-13 | Vehicle tracking system and method based on image recognition and infrared obstacle avoidance and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107909024B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825123A (en) * | 2019-10-21 | 2020-02-21 | 哈尔滨理工大学 | Control system and method for automatic following loading vehicle based on motion algorithm |
| CN112347953B (en) * | 2020-11-11 | 2021-09-28 | 上海伯镭智能科技有限公司 | Recognition device for road condition irregular obstacles of unmanned vehicle |
| CN112508865B (en) * | 2020-11-23 | 2024-02-02 | 深圳供电局有限公司 | Unmanned aerial vehicle inspection obstacle avoidance method, unmanned aerial vehicle inspection obstacle avoidance device, computer equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838244A (en) * | 2014-03-20 | 2014-06-04 | 湖南大学 | Portable target tracking method and system based on four-axis air vehicle |
| CN105631798A (en) * | 2016-03-04 | 2016-06-01 | 北京理工大学 | Low-power consumption portable real-time image target detecting and tracking system and method thereof |
| CN105844664A (en) * | 2016-03-21 | 2016-08-10 | 辽宁师范大学 | Monitoring video vehicle detection tracking method based on improved TLD |
| CN106303461A (en) * | 2016-09-08 | 2017-01-04 | 福建师范大学 | Movable-type intelligent safety device based on video analysis |
| WO2017044550A1 (en) * | 2015-09-11 | 2017-03-16 | Intel Corporation | A real-time multiple vehicle detection and tracking |
| CN106846362A (en) * | 2016-12-26 | 2017-06-13 | 歌尔科技有限公司 | A kind of target detection tracking method and device |
| CN106886748A (en) * | 2016-12-28 | 2017-06-23 | 中国航天电子技术研究院 | A kind of mutative scale method for tracking target suitable for unmanned plane based on TLD |
| CN107105159A (en) * | 2017-04-13 | 2017-08-29 | 山东万腾电子科技有限公司 | The real-time detecting and tracking system and method for embedded moving target based on SoC |
| WO2017185503A1 (en) * | 2016-04-29 | 2017-11-02 | 高鹏 | Target tracking method and apparatus |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7720580B2 (en) * | 2004-12-23 | 2010-05-18 | Donnelly Corporation | Object detection system for vehicle |
| US20170116488A1 (en) * | 2015-10-23 | 2017-04-27 | MAGNETI MARELLI S.p.A. | Method for identifying an incoming vehicle and corresponding system |
-
2017
- 2017-11-13 CN CN201711116535.5A patent/CN107909024B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838244A (en) * | 2014-03-20 | 2014-06-04 | 湖南大学 | Portable target tracking method and system based on four-axis air vehicle |
| WO2017044550A1 (en) * | 2015-09-11 | 2017-03-16 | Intel Corporation | A real-time multiple vehicle detection and tracking |
| CN105631798A (en) * | 2016-03-04 | 2016-06-01 | 北京理工大学 | Low-power consumption portable real-time image target detecting and tracking system and method thereof |
| CN105844664A (en) * | 2016-03-21 | 2016-08-10 | 辽宁师范大学 | Monitoring video vehicle detection tracking method based on improved TLD |
| WO2017185503A1 (en) * | 2016-04-29 | 2017-11-02 | 高鹏 | Target tracking method and apparatus |
| CN106303461A (en) * | 2016-09-08 | 2017-01-04 | 福建师范大学 | Movable-type intelligent safety device based on video analysis |
| CN106846362A (en) * | 2016-12-26 | 2017-06-13 | 歌尔科技有限公司 | A kind of target detection tracking method and device |
| CN106886748A (en) * | 2016-12-28 | 2017-06-23 | 中国航天电子技术研究院 | A kind of mutative scale method for tracking target suitable for unmanned plane based on TLD |
| CN107105159A (en) * | 2017-04-13 | 2017-08-29 | 山东万腾电子科技有限公司 | The real-time detecting and tracking system and method for embedded moving target based on SoC |
Non-Patent Citations (7)
| Title |
|---|
| An hybrid control for avoiding obstacles during a vision-based tracking task;V. Cadenat 等;《European Control Conference》;19991231;1114-1119 * |
| Claudio Caraffi等.A System for Real-time Detection and Tracking of Vehicles from a Single Car-mounted Camera.《IEEE Conference on Intelligent Transportation Systems 》.2012,1-8. * |
| Tracking-Learning-Detection;Zdenek Kalal 等;《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》;20120731;第34卷(第7期);1409-1422 * |
| 低分辨率条件下基于TLD的鲁棒车辆跟踪算法;刘阔 等;《计算机应用与软件》;20161231;第33卷(第12期);264-269 * |
| 基于红外避障的智能小车的设计;强彦 等;《微电子学与计算机》;20130228;第30卷(第2期);140-143 * |
| 检测区域动态调整的TLD目标跟踪算法;曲海成 等;《计算机应用》;20151010;第35卷(第10期);2985-2989 * |
| 红外传感器在智能车避障系统的应用;雷鹏飞 等;《电脑与信息技术》;20100831;第18卷(第4期);52-53 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107909024A (en) | 2018-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11216971B2 (en) | Three-dimensional bounding box from two-dimensional image and point cloud data | |
| CA3063011C (en) | System and method for detecting and tracking objects | |
| US11645765B2 (en) | Real-time visual object tracking for unmanned aerial vehicles (UAVs) | |
| US9990736B2 (en) | Robust anytime tracking combining 3D shape, color, and motion with annealed dynamic histograms | |
| CN110363058B (en) | Three-dimensional object localization for obstacle avoidance using one-shot convolutional neural networks | |
| US10733482B1 (en) | Object height estimation from monocular images | |
| WO2021134296A1 (en) | Obstacle detection method and apparatus, and computer device and storage medium | |
| US20180285697A1 (en) | Camera/object pose from predicted coordinates | |
| Sless et al. | Road scene understanding by occupancy grid learning from sparse radar clusters using semantic segmentation | |
| CN107909024B (en) | Vehicle tracking system and method based on image recognition and infrared obstacle avoidance and vehicle | |
| US10776941B2 (en) | Optimized neural network structure | |
| US11479213B1 (en) | Sensor obstruction detection and mitigation | |
| Huang et al. | Fish tracking and segmentation from stereo videos on the wild sea surface for electronic monitoring of rail fishing | |
| CN111913177A (en) | Method and device for detecting target object and storage medium | |
| CN116309719A (en) | Target tracking method, device, computer equipment and storage medium | |
| Patil et al. | A survey on joint object detection and pose estimation using monocular vision | |
| Junejo et al. | Dynamic scene modeling for object detection using single-class SVM | |
| Le et al. | Human detection and tracking for autonomous human-following quadcopter | |
| CN111814653A (en) | Method, device, equipment and storage medium for detecting abnormal behaviors in video | |
| Pandey et al. | Light-weight object detection and decision making via approximate computing in resource-constrained mobile robots | |
| Wang et al. | Online drone-based moving target detection system in dense-obstructer environment | |
| CN114792417B (en) | Model training method, image recognition method, device, equipment and storage medium | |
| CN117611592B (en) | Foreign matter detection method, device, electronic equipment and storage medium | |
| Patel | A Comparative study for object detection using UAV: Histogram vs YOLO | |
| Ping et al. | A parallel implementation of TLD algorithm using CUDA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |