CN112381857A - Brain-like target tracking method based on impulse neural network - Google Patents
Brain-like target tracking method based on impulse neural network Download PDFInfo
- Publication number
- CN112381857A CN112381857A CN202011259939.1A CN202011259939A CN112381857A CN 112381857 A CN112381857 A CN 112381857A CN 202011259939 A CN202011259939 A CN 202011259939A CN 112381857 A CN112381857 A CN 112381857A
- Authority
- CN
- China
- Prior art keywords
- target
- neural network
- tracking
- brain
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title claims abstract description 16
- 238000001514 detection method Methods 0.000 claims abstract description 44
- 210000002569 neuron Anatomy 0.000 claims abstract description 11
- 239000000284 extract Substances 0.000 claims description 2
- 239000012528 membrane Substances 0.000 claims 3
- 230000001537 neural effect Effects 0.000 claims 1
- 230000000284 resting effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 241000234295 Musa Species 0.000 description 6
- 235000018290 Musa x paradisiaca Nutrition 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 2
- 238000012421 spiking Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 235000021015 bananas Nutrition 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a brain-like target tracking method based on a pulse neural network. The target detection model constructed by the invention consists of a basic network and different superposed convolutional layers: the basic network, namely the impulse neural network, is used for extracting the characteristics of the input image; and then, the superposed different convolutional layers can obtain a multi-scale detection characteristic diagram, the large-scale characteristic diagram is used for detecting small objects, and the small-scale characteristic diagram is used for detecting large objects. And finally outputting a target coordinate position and a confidence probability based on the target detection model, and realizing the tracking of the moving end on the target by adopting PID control. The invention has the beneficial effects that: compared with the traditional target detection model, the LIF neuron-based impulse neural network is adopted as the basic network, so that the target detection model has better biological rationality.
Description
Technical Field
The invention relates to the field of target tracking, in particular to a brain-like target tracking method based on a pulse neural network.
Background
In recent years, with the development of artificial intelligence technology, target tracking technology has further advanced, and is widely applied to industries, medical treatment, home furnishing, transportation and the like. An effective target detection model is crucial to a target tracking technology, the target detection model mainly performs target detection on an input image and outputs information such as position coordinates of a detected target, and then tracking of the target by a moving end is achieved through a motion control algorithm.
Currently, deep learning is widely applied in the field of target tracking, wherein the traditional target detection models such as Faster R-CNN, SSD and YOLO are common. Although the target detection models can realize target detection on the input image by using the deep neural network, the deep neural network does not have good biological rationality, and further has the limiting conditions of large calculated amount, high power consumption, excessive dependence on hardware acceleration platforms such as a GPU and the like, and further causes the target detection speed to be slow and the accuracy to be not high enough.
At present, the impulse neural network is widely concerned by researchers because the impulse neural network better conforms to the structural characteristics of the biological brain and has the advantages of better biological rationality and the like, so that the brain-like target detection model which is more biologically releasable and is formed by combining the impulse neural network and the traditional target detection model has wider research prospect.
Disclosure of Invention
The invention aims to overcome the defects that the traditional target detection algorithm has low biological interpretability and the like, and provides a brain-like target tracking method based on LIF (LIF-like fuzzy inference) pulse neurons by combining with a pulse neural network. The method enables the target detection model to have better biological rationality and enables the target tracking method to be more effective.
The object detection algorithm essentially locates with a rectangular box and determines objects within the classified rectangular box. And positioning to frame the object by using a rectangular frame, marking the top left corner and the bottom right corner of the rectangular frame, and further identifying the maximum object in the rectangular frame. The algorithm cuts and covers the whole image by using default identification frames with different width-height ratios and sizes, performs single-target detection on image contents in different rectangular frames, and finally summarizes the object identification condition of the whole image. And marking the confidence coefficient of each object detected in the image and the coordinates of the top left corner and the bottom right corner of the recognition box. In the target detection model, the basic network is a pulse neural network and is mainly used for extracting features of an input image, different convolution layers are arranged behind the basic network, so that a multi-scale detection feature map (feature map) is obtained, therefore, default identification frames with different scales and aspect ratios and related class probabilities thereof can be predicted, and finally, a final result is generated through a non-maximum suppression step. And finally, carrying out PID control based on the coordinate information of the target to realize target tracking of the mobile terminal.
In order to solve the technical problems, the invention adopts the technical scheme that: a brain-like target tracking method based on a pulse neural network comprises the following steps:
s1, input image: inputting a target image to be tracked by a camera assembly;
s2, the target detection model carries out target detection on the input image:
firstly, extracting features of an input image by using a basic network pulse neural network to form a detection feature map, and then adding convolution layers with different scales to perform multi-scale detection;
s3, outputting coordinate position information: carrying out multi-scale default recognition frame prediction on the detection feature maps with different scales generated in the step S2, classifying the predicted default recognition frame types by utilizing softmax, carrying out boundary regression to obtain the position information of the real target, and outputting the vertex coordinates and the confidence coefficients of the upper left corner and the lower right corner of the target recognition frame to be tracked;
s4, target tracking: and changing the motion state of the mobile terminal based on PID control according to the target position information obtained in the step of S3, thereby realizing target tracking.
Step S1, the pulse neural network adopts LIF neuron as basic unit, and the dynamic model of LIF neuron is
The method has better biological reasonableness, so that the input image can be effectively extracted.
Step S2, the convolution layer with different scales further extracts the features of the feature map generated by the basic network, generates detection maps with different scales, and detects small target objects by adopting the feature map with large scale; and detecting a large target object by using the small-scale feature map.
The target coordinate position information in step S3 is the vertex coordinates [ x ] of the target object recognition box located at the upper left corner and the lower right corner of the whole image1，y1，x2，y2]Wherein softmax is calculated by
In the step S4, the object tracking is to determine the position information of the recognition object on the screen by using the vertex coordinates, and change the motion state of the recognition object according to the coordinate difference and the area difference of the adjusted recognition frame.
The invention has the beneficial effects that:
1. the invention adopts the pulse neural network as the basic network of the target detection model and the LIF neuron as the basic unit, thereby ensuring that the target detection model has more biological rationality.
2. The invention changes the motion state of the mobile terminal based on PID control based on the result output by the target detection algorithm model, thereby realizing the tracking of the identified target.
Drawings
FIG. 1 is a schematic diagram of target coordinate information output by a target detection model;
FIG. 2 is a schematic diagram of tracking an object using a target detection model;
FIG. 3 is a flow chart of a brain-like target tracking method based on a spiking neural network according to the present invention;
FIG. 4 is a flow chart of the spiking neural network of the present invention.
Detailed Description
The invention is described in further detail below with reference to the following figures and detailed description:
the drawings are for illustrative purposes only and are not to be construed as limiting the patent; for a better understanding of the present embodiments, some components of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted. The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Firstly, the experiment is trained on a COCO data set, a data set which can be used for image recognition is provided by a Microsoft team, and images in the data set are divided into a training set, a verification set and a test set, can be used for detecting 90 common objects and are common data sets of a target tracking algorithm.
Fig. 1 shows specific position information of the recognition target on the screen, wherein the object recognized by the target detection model is respectively marked by an image recognition box, and finally the target to be tracked is determined according to the matching tag.
Vertex [ x ] of upper left corner and lower right corner according to recognition coordinate1，y1，x2，y2]Wherein (x1, y1) is the top left corner vertex of the detection target, and (x2, y2) is the vertex with the lower corner. The coordinates [ (x) of the center position of the object are calculated1+x2)/2，(y1+y2)/2]If a plurality of targets to be tracked appear in the same picture, for example, a plurality of recognition frames are generated in fig. 1, a target closest to the center of the picture is selected as a final tracking target by calculating and comparing vector distance values of center positions, that is, the vector distance value is smaller and closer to the center position of the picture.
The deviation degree and the distance of the target in the picture are determined by calculating the position information, namely coordinate values, of the target, wherein the deviation degree is determined by the difference value of the abscissa and the coordinate of the center of the picture, and the distance is mapped by the area of the target recognition frame one by one. Using the PID controller, the set value is set according to a desired area (e.g., 30000) of the recognition frame, which corresponds one-to-one to the holding distance of the moving end tracking target. And comparing the actual recognition distance of the target object by the mobile terminal camera with an expected set value, and adjusting the speed value of the mobile terminal through PID control output so as to realize the adjustment of the distance. For the steering, a PID controller is also used, the set value is 0, namely the steering is carried out, the recognized object is kept at the center of the picture after the steering, PID control is carried out according to the comparison result of the set value and the actual coordinate difference, and the output result adjusts the difference of the left motor and the right motor of the moving end, so that the steering is realized.
As shown in fig. 3, the main innovation point of the present invention is to use an impulse neural network based on LIF neurons as a basic network of a target detection model for extracting features of an input image, and perform multi-scale detection on a feature map by adding convolutional layers of different scales, so as to realize multi-scale recognition frame to predict a target. And finally, tracking and controlling the target according to the target position information output by the target detection model by combining a PID control algorithm.
Fig. 4 shows the basic network of the target detection model: the impulse neural network adopts LIF neurons as the basic neurons, has better feature extraction capability on input images and better biological interpretability.
Examples
As shown in fig. 2, the experimental results are shown, where bananas and two bottles were selected as the picture recognition objects in the target tracking experiment. Firstly, the matching label is set as a banana, and the trolley is released at a certain distance. The images of the cameras can be analyzed, all objects in the visual field are identified by the trolley and are marked by the identification frames, meanwhile, the target to be tracked according to the set matching label is a banana, id in the COCO data set is 52, the banana is marked independently and serves as the target to be tracked, then the motor is automatically adjusted based on the image center object detection method, the trolley is controlled to place the banana identification frame in the images from the graph a to the graph b in the center, and the images from the graph b to the graph d are tracked until the banana identification frame stops in the graph d at a certain distance. At this time, if the matched label is replaced by the bottle, the visible trolley controls the motor to rotate right according to the matched target until the bottle is positioned in the pictures from e to g in the center of the picture and stops at a certain distance.
Claims (5)
1. A brain-like target tracking method based on a pulse neural network is characterized by comprising the following steps:
s1, input image: inputting a target image to be tracked by a camera assembly;
s2, the target detection model carries out target detection on the input image:
firstly, extracting features of an input image by using a basic network pulse neural network to form a detection feature map, and then adding convolution layers with different scales to perform multi-scale detection;
s3, outputting coordinate position information: carrying out multi-scale default recognition frame prediction on the detection feature maps with different scales generated in the step S2, classifying the predicted default recognition frame types by utilizing softmax, carrying out boundary regression to obtain the position information of the real target, and outputting the vertex coordinates and the confidence coefficients of the upper left corner and the lower right corner of the target recognition frame to be tracked;
s4, target tracking: and changing the motion state of the mobile terminal based on PID control according to the target position information obtained in the step of S3, thereby realizing target tracking.
2. The method for tracking the brain-like target based on the impulse neural network as claimed in claim 1, wherein the basic units of the impulse neural network in step S1 use LIF neurons, and the LIF neuron dynamic model can be approximated as a first order differential equation:
wherein u (t) the pulsed neuron membrane voltage; u. ofrestIs neuronal resting potential; r is a membrane resistance, I (t) is an external input current, taumIs the membrane time constant (τ)mRC). The method has better biological reasonableness, so that the input image can be effectively extracted.
3. The method for tracking the brain-like target based on the impulse neural network as claimed in claim 1, wherein the convolutional layers with different scales in step S2 further extract features of the feature map generated by the base network, generate detection maps with different scales, and detect the small target object by using the feature map with large scale; and detecting a large target object by using the small-scale feature map.
4. The method according to claim 1, wherein the target coordinate position information in step S3 is the vertex coordinates [ x ] of the target object recognition box located at the upper left corner and the lower right corner of the whole image1，y1，x2，y2]Wherein softmax is calculated by
Wherein, S is the probability of each output category of the last layer, and e is the input proportion of each category.
5. The method for tracking the brain-like object based on the impulse neural network as claimed in claim 1, wherein the step S4 is implemented by determining the position information of the recognition object on the screen by using the vertex coordinates, and changing the motion state of the recognition object according to the coordinate difference and the area difference of the adjustment recognition frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011259939.1A CN112381857A (en) | 2020-11-12 | 2020-11-12 | Brain-like target tracking method based on impulse neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011259939.1A CN112381857A (en) | 2020-11-12 | 2020-11-12 | Brain-like target tracking method based on impulse neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112381857A true CN112381857A (en) | 2021-02-19 |
Family
ID=74583190
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011259939.1A Pending CN112381857A (en) | 2020-11-12 | 2020-11-12 | Brain-like target tracking method based on impulse neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112381857A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023212857A1 (en) * | 2022-05-05 | 2023-11-09 | 中国科学院深圳先进技术研究院 | Brain-computer interface system and device based on brain-inspired intelligence |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160171330A1 (en) * | 2014-12-15 | 2016-06-16 | Reflex Robotics, Inc. | Vision based real-time object tracking system for robotic gimbal control |
| CN109816026A (en) * | 2019-01-29 | 2019-05-28 | 清华大学 | The fusion structure and method of convolutional neural networks and impulsive neural networks |
| CN110555523A (en) * | 2019-07-23 | 2019-12-10 | 中建三局智能技术有限公司 | short-range tracking method and system based on impulse neural network |
| CN110826437A (en) * | 2019-10-23 | 2020-02-21 | 中国科学院自动化研究所 | Intelligent robot control method, system and device based on biological neural network |
| CN111717390A (en) * | 2020-07-06 | 2020-09-29 | 湖北经济学院 | Multifunctional unmanned aerial vehicle system |
-
2020
- 2020-11-12 CN CN202011259939.1A patent/CN112381857A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160171330A1 (en) * | 2014-12-15 | 2016-06-16 | Reflex Robotics, Inc. | Vision based real-time object tracking system for robotic gimbal control |
| CN109816026A (en) * | 2019-01-29 | 2019-05-28 | 清华大学 | The fusion structure and method of convolutional neural networks and impulsive neural networks |
| CN110555523A (en) * | 2019-07-23 | 2019-12-10 | 中建三局智能技术有限公司 | short-range tracking method and system based on impulse neural network |
| CN110826437A (en) * | 2019-10-23 | 2020-02-21 | 中国科学院自动化研究所 | Intelligent robot control method, system and device based on biological neural network |
| CN111717390A (en) * | 2020-07-06 | 2020-09-29 | 湖北经济学院 | Multifunctional unmanned aerial vehicle system |
Non-Patent Citations (4)
| Title |
|---|
| WEI LIU等: ""SSD: Single Shot MultiBox Detector"", 《ARXIV》 * |
| 匡载波等: "基于脑启发视觉神经元网络输电线路部件识别的研究", 《电力系统及其自动化学报》 * |
| 易全政: ""基于脉冲神经网络的目标跟踪技术研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 邓小云等: ""基于脉冲神经网络人脸特征提取"", 《福建师范大学学报（自然科学版）》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023212857A1 (en) * | 2022-05-05 | 2023-11-09 | 中国科学院深圳先进技术研究院 | Brain-computer interface system and device based on brain-inspired intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ahmed et al. | Region and decision tree-based segmentations for Multi-objects detection and classification in Outdoor Scenes | |
| CN111798475B (en) | Indoor environment 3D semantic map construction method based on point cloud deep learning | |
| CN108564097B (en) | Multi-scale target detection method based on deep convolutional neural network | |
| CN111709310B (en) | Gesture tracking and recognition method based on deep learning | |
| CN111027493B (en) | Pedestrian detection method based on deep learning multi-network soft fusion | |
| CN111179217A (en) | Attention mechanism-based remote sensing image multi-scale target detection method | |
| CN112842690B (en) | Machine vision with dimension data reduction | |
| CN107545263B (en) | Object detection method and device | |
| CN110163187A (en) | Remote road traffic sign detection recognition methods based on F-RCNN | |
| CN113128558B (en) | Target detection method based on shallow space feature fusion and adaptive channel screening | |
| CN109800689A (en) | A kind of method for tracking target based on space-time characteristic fusion study | |
| CN111429418A (en) | Industrial part detection method based on YO L O v3 neural network | |
| CN112085024A (en) | Tank surface character recognition method | |
| CN110706252B (en) | Robot nuclear correlation filtering tracking algorithm under guidance of motion model | |
| CN111985385A (en) | Behavior detection method, device and equipment | |
| JP7439153B2 (en) | Lifted semantic graph embedding for omnidirectional location recognition | |
| CN112364931A (en) | Low-sample target detection method based on meta-feature and weight adjustment and network model | |
| CN109002752A (en) | A kind of complicated common scene rapid pedestrian detection method based on deep learning | |
| CN114693661A (en) | Rapid sorting method based on deep learning | |
| CN111582395B (en) | Product quality classification system based on convolutional neural network | |
| CN112766184B (en) | Remote sensing target detection method based on multi-level feature selection convolutional neural network | |
| Xin et al. | Deep learning for robust outdoor vehicle visual tracking | |
| CN112381857A (en) | Brain-like target tracking method based on impulse neural network | |
| CN111611869B (en) | End-to-end monocular vision obstacle avoidance method based on serial deep neural network | |
| CN112949635A (en) | Target detection method based on feature enhancement and IoU perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Yang Shuangming Inventor after: Yang Ming Inventor after: Hu Zhicai Inventor after: Wang Jiang Inventor after: Cheng Jian Inventor after: Deng Bin Inventor after: Li Huiyan Inventor before: Yang Shuangming Inventor before: Yang Ming Inventor before: Hu Zhicai Inventor before: Wang Jiang Inventor before: Deng Bin Inventor before: Li Huiyan |
|
| CB03 | Change of inventor or designer information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210219 |
|
| RJ01 | Rejection of invention patent application after publication |