CN110047090B - RGB-D target tracking method based on evolution feature learning - Google Patents

RGB-D target tracking method based on evolution feature learning Download PDF

Info

Publication number
CN110047090B
CN110047090B CN201910240994.7A CN201910240994A CN110047090B CN 110047090 B CN110047090 B CN 110047090B CN 201910240994 A CN201910240994 A CN 201910240994A CN 110047090 B CN110047090 B CN 110047090B
Authority
CN
China
Prior art keywords
rgb
chromosome
chromosomes
depth
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910240994.7A
Other languages
Chinese (zh)
Other versions
CN110047090A (en
Inventor
姜明新
王晓燕
贾银洁
王海燕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Longyuan Zhongchuang Information Technology Co ltd
Original Assignee
Huaiyin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huaiyin Institute of Technology filed Critical Huaiyin Institute of Technology
Priority to CN201910240994.7A priority Critical patent/CN110047090B/en
Publication of CN110047090A publication Critical patent/CN110047090A/en
Application granted granted Critical
Publication of CN110047090B publication Critical patent/CN110047090B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an RGB-D target tracking method based on evolution feature learning, which applies an evolution feature learning technology to an RGB-D video target tracking method, does not need prior knowledge of a target in the evolution feature learning, and can overcome the defect of manually designing the feature. Meanwhile, the number of parameters for the evolutionary feature learning is much smaller than that of the deep neural network, and a large amount of time is not needed for learning and training the parameters. Therefore, the method can overcome the technical defects of poor robustness of the manually designed features and the technical defects of low calculation efficiency and long time consumption of a deep learning method.

Description

RGB-D target tracking method based on evolution feature learning
Technical Field
The invention relates to the field of artificial intelligence and machine vision, in particular to an RGB-D target tracking method based on evolution feature learning.
Background
In recent years, RGB-D video target tracking is a hot research problem in the field of artificial intelligence, and has very wide application in the fields of virtual reality, human-computer interaction, unmanned driving and the like.
However, most of the existing target tracking methods only use RGB data to extract features, which causes a certain limitation on tracking performance, and when the illumination condition changes, the target is blocked, and the appearance of the target changes, the tracking accuracy decreases. In the other part of the method, RGB-D data is adopted for target tracking, and the method is roughly divided into two types, wherein one type is based on artificial design characteristics, and the other type is based on Deep learning characteristics; in which the manual design of features requires a lot of a priori knowledge, and the person who needs feature modeling has a very intensive study in this field, even though it is impossible to solve various possible difficulties in one model. The Deep learning characteristics can be used for learning and training a large amount of target characteristics automatically, so that the defects of artificial design characteristics are overcome to a great extent, but a large amount of parameters in a Depth neural network need to be learned and trained, which causes lower calculation efficiency and longer time consumption.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide an RGB-D target tracking method based on evolution feature learning, which has the advantages of no need of prior knowledge of targets and overcomes the defects of artificial design features and the technical defects of low calculation efficiency and long time consumption of a Deep learning method.
The technical scheme is as follows: the invention relates to an RGB-D target tracking method based on evolution feature learning, which comprises the following steps:
step 1, performing Depth image processing on candidate target image blocks in video target tracking, and performing HHA coding on candidate target images subjected to Depth image processing;
step 2, encoding chromosomes of candidate target images in RGB, and encoding chromosomes of candidate target images in Depth images; wherein one chromosome comprises 32 genes, each gene comprising 7 data items, one data item comprising 8 binary bits; in the RGB image, 7 data items respectively represent the center point coordinates (X, Y) of the candidate target image block, the width and height (W, H) of the image block, and the numerical values of the three channels R, G, B; in a Depth image, 7 data items respectively represent the central point coordinates (X, Y) of a candidate target image block, the width and height (W, H) of the image block, the horizontal difference, the height to the ground and the numerical values of three channels of the angle of a surface normal vector;
step 3, carrying out initialization candidate solution space on chromosomes of the candidate target image in the RGB and Depth image modes, wherein the population of the chromosomes in the initialization candidate solution space is represented as:
Figure BDA0002009639170000011
or
Figure BDA0002009639170000012
Figure BDA0002009639170000013
Is a chromosome, j =1,2, …, M is the number of chromosomes contained in the population, G =1,2, …, G represents the number of generations;
calculating the matching degree of each candidate target image and the last frame tracking result in the candidate solution space, wherein the matching degree is calculated as X for each input image in RGB or Depth mode RGB Or X depth In the evolution algorithm, there is one reconstructed image
Figure BDA0002009639170000021
Or
Figure BDA0002009639170000022
C j Representing the jth chromosome, for the candidate chromosomes, the corresponding degree of match equation is:
Figure BDA0002009639170000023
and taking the matched chromosome as a termination result, and simultaneously outputting the evolution characteristics of the chromosome in the RGB and depth modes.
Further, there is a chromosome with the highest matching degree among the chromosomes in the RGB and depth patterns, and the chromosome with the highest matching degree can be obtained by the following formula:
Figure BDA0002009639170000024
derived from
Figure BDA0002009639170000025
As X RGB Of the evolving characteristics of (2) and final results F RGB ，
Figure BDA0002009639170000026
As X depth Of the evolving characteristics of (2) and final results F depth 。
Further, in step 3, chromosomes which do not satisfy the termination result are subjected to further selection, including combinatorial crossing, mutation, generation of new populations, and the above process is repeated.
Further, in the selection process, each chromosome is ranked according to the matching degree of the chromosome, the worst is rank1, next rank 2 … … is preferably rank M, and M is the number of chromosomes contained in the population; calculating the selection probability according to the rank value of the chromosome:
Figure BDA0002009639170000027
P j to select the probability of the j-th chromosome,
Figure BDA0002009639170000028
is the probability of selecting the worst chromosome,
Figure BDA0002009639170000029
is the probability of selecting the best chromosome, m worst M is the number of chromosomes with the lowest degree of matching best M is the number of chromosomes with the highest degree of matching worst M is the number of chromosomes with the lowest degree of matching best The number of chromosomes with the highest matching degree.
Further, the matching chromosomes in the RGB and Depth modes can be used to analyze and obtain the error amount of the evolution features between two adjacent frames, which is according to the following formula:
Figure BDA00020096391700000210
wherein, F RGB Is X RGB End result of the evolving characteristics of (1), F depth Is X depth I, i is the number of sequences of the frame.
Has the advantages that: the method applies the evolution characteristic learning technology to the RGB-D video target tracking method, the evolution characteristic learning does not need the prior knowledge of the target, and the defect of manually designing the characteristic can be overcome. Meanwhile, the number of parameters for evolving feature learning is much smaller than that of a deep neural network, and a large amount of time is not needed for learning and training the parameters. Therefore, the method can overcome the technical defects of poor robustness of the manually designed features and the technical defects of low calculation efficiency and long time consumption of a deep learning method.
Drawings
FIG. 1 is a schematic representation of chromosome coding in RGB and Depth modes;
FIG. 2 is a flow chart of calculating evolution characteristics in RGB and Depth modes;
FIG. 3 is a flow chart of an RGB-D tracking method based on evolutionary feature learning;
fig. 4 is a diagram illustrating a part of tracking effects in the present invention.
Detailed Description
As shown in fig. 1 and 2, an RGB-D target tracking method based on evolutionary feature learning includes the following steps:
step 1, performing Depth image processing on candidate target image blocks in video target tracking, and performing HHA coding on candidate target images subjected to Depth image processing;
step 2, encoding chromosomes of candidate target images in RGB, and encoding chromosomes of candidate target images in Depth images; wherein one chromosome comprises 32 genes, each gene comprising 7 data items, one data item comprising 8 binary bits; in the RGB image, 7 data items respectively represent the center point coordinates (X, Y) of the candidate target image block, the width and height (W, H) of the image block, and the numerical values of the three channels R, G, B; in a Depth image, 7 data items respectively represent the central point coordinates (X, Y) of a candidate target image block, the width and height (W, H) of the image block, the horizontal difference, the height to the ground and the numerical values of three channels of the angle of a surface normal vector;
step 3, chromosome of candidate target image under RGB and Depth image mode is initialized to candidate solution space, and initialThe population of chromosomes in the solution space candidate is represented as:
Figure BDA0002009639170000031
or
Figure BDA0002009639170000032
Figure BDA0002009639170000033
Is a chromosome, j =1,2, …, M is the number of chromosomes contained in the population, G =1,2, …, G represents the number of generations;
calculating the matching degree of each candidate target image and the last frame tracking result in the candidate solution space, wherein the matching degree is calculated as X for each input image in an RGB or Depth mode RGB Or X depth In the evolution algorithm, there is one reconstructed image
Figure BDA0002009639170000034
Or alternatively
Figure BDA0002009639170000035
C j Representing the jth chromosome, for the candidate chromosomes, the corresponding degree of match equation is:
Figure BDA0002009639170000036
and taking the matched chromosome as a termination result, and simultaneously outputting the evolution characteristics of the chromosome in the RGB and depth modes.
Among chromosomes in the RGB and depth patterns, there is a chromosome with the highest matching degree, which can be obtained by the following formula:
Figure BDA0002009639170000037
derived from
Figure BDA0002009639170000041
As X RGB Of the evolving characteristics of (2) and final results F RGB ，
Figure BDA0002009639170000042
As X depth Of the evolving characteristics of (2) and final results F depth 。
In step 3, chromosomes which do not meet the termination result are further selected, including combination crossing and mutation, so as to generate a new population, and then the process is circulated.
In the selection process, each chromosome is sorted according to the matching degree of the chromosome, the worst is rank1, next rank 2 … … is most preferably rank M, and M is the number of chromosomes contained in the population; calculating the selection probability according to the rank value of the chromosome:
Figure BDA0002009639170000043
P j to select the probability of the j-th chromosome,
Figure BDA0002009639170000044
is the probability of selecting the worst chromosome,
Figure BDA0002009639170000045
is the probability of selecting the best chromosome, m worst M is the number of chromosomes with the lowest degree of matching best M is the number of chromosomes with the highest degree of matching worst M is the number of chromosomes with the lowest degree of matching best The number of chromosomes with the highest matching degree.
As shown in fig. 3, the matching chromosomes in RGB and Depth modes can be used to analyze and obtain the error amount of the evolution features between two adjacent frames, which is according to the following formula:
Figure BDA0002009639170000046
wherein, F RGB Is X RGB End result of the evolving characteristics of (1), F depth Is X depth T =1, 2.. I, i is the number of sequences of the frame.
The experimental results and analyses were as follows:
we used the Windows 10 operating system with MATLAB R2016a as the software platform. The main configuration of the computer is a 12-core processor, 54GB ram, nvidia GeForce GT650m GPU, the population contains the number of chromosomes M =30, the number of generations G =200, and the number of samples N =100, experimental performance was tested on a PTB data set, which contains 100 RGB-D videos, and the tracking success rate results are shown in table 1:
Figure BDA0002009639170000047
table 1.

Claims (5)

1. An RGB-D target tracking method based on evolution feature learning is characterized by comprising the following steps:
step 1, HHA coding is carried out on a Depth image of a candidate target in video target tracking;
step 2, encoding chromosomes of candidate target images in RGB (red, green and blue) and encoding chromosomes of candidate target images in a Depth image; wherein one chromosome comprises 32 genes, each gene comprising 7 data items, one data item comprising 8 binary bits; in the RGB image, 7 data items respectively represent the center point coordinates (X, Y) of the candidate target image block, the width and height (W, H) of the image block, and the numerical values of the three channels R, G, B; in a Depth image, 7 data items respectively represent central point coordinates (X, Y) of candidate target image blocks, width and height (W, H) of the image blocks, horizontal difference, ground height and the numerical values of three channels of angles of surface normal vectors;
step 3, carrying out initialization candidate solution space on chromosomes of the candidate target image in the RGB and Depth image modes, wherein the population of the chromosomes in the initialization candidate solution space is represented as:
Figure FDA0002009639160000011
or
Figure FDA0002009639160000012
Figure FDA0002009639160000013
Is a chromosome, j =1,2, …, M is the number of chromosomes contained in the population, G =1,2, …, G indicates the number of generations;
calculating the matching degree of each candidate target image and the last frame tracking result in the candidate solution space, wherein the matching degree is calculated as X for each input image in RGB or Depth mode RGB Or X depth In the evolution algorithm, there is one reconstructed image
Figure FDA0002009639160000014
Or
Figure FDA0002009639160000015
C j Representing the jth chromosome, for the candidate chromosomes, the corresponding degree of match equation is:
Figure FDA0002009639160000016
or alternatively
Figure FDA0002009639160000017
And taking the matched chromosome as a termination result, and simultaneously outputting the evolution characteristics of the chromosome in the RGB and depth modes.
2. The RGB-D target tracking method based on evolutionary feature learning as claimed in claim 1, wherein in step 3, there exists a chromosome with the highest matching degree among the chromosomes in RGB and depth modes, and the chromosome with the highest matching degree can be obtained by the following formula:
Figure FDA0002009639160000018
or alternatively
Figure FDA0002009639160000019
Derived from
Figure FDA00020096391600000110
As X RGB Of the evolving characteristics of (2) and final results F RGB ，
Figure FDA00020096391600000111
As X depth Of the evolving characteristics of (2) and final results F depth 。
3. The method for RGB-D object tracking based on evolutionary feature learning as claimed in claim 1, wherein in step 3, chromosomes that do not satisfy termination results are further selected, including combining crossover, mutation, generating new population, and then repeating the above process.
4. The RGB-D target tracking method based on evolutionary feature learning as claimed in claim 3, wherein in the selection process, each chromosome is ranked according to its matching degree, the worst is rank1, next rank 2 … … is the best is rank M, M is the number of chromosomes contained in the population; calculating the selection probability according to the rank value of the chromosome:
Figure FDA0002009639160000021
P j to select the probability of the j-th chromosome,
Figure FDA0002009639160000022
is the probability of selecting the worst chromosome,
Figure FDA0002009639160000023
is the probability of selecting the best chromosome, m worst M is the number of chromosomes with the lowest degree of matching best The number of chromosomes with the highest matching degree.
5. The RGB-D target tracking method based on evolving feature learning as claimed in claim 2, wherein the matching chromosomes in RGB and Depth modes can be used to analyze and obtain the error amount of evolving features between two adjacent frames, the error amount is according to the following formula:
Figure FDA0002009639160000024
wherein, F RGB Is X RGB End result of the evolving characteristics of (1), F depth Is X depth I, i is the number of sequences of the frame.
CN201910240994.7A 2019-03-28 2019-03-28 RGB-D target tracking method based on evolution feature learning Active CN110047090B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910240994.7A CN110047090B (en) 2019-03-28 2019-03-28 RGB-D target tracking method based on evolution feature learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910240994.7A CN110047090B (en) 2019-03-28 2019-03-28 RGB-D target tracking method based on evolution feature learning

Publications (2)

Publication Number Publication Date
CN110047090A CN110047090A (en) 2019-07-23
CN110047090B true CN110047090B (en) 2022-10-14

Family

ID=67275387

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910240994.7A Active CN110047090B (en) 2019-03-28 2019-03-28 RGB-D target tracking method based on evolution feature learning

Country Status (1)

Country Link
CN (1) CN110047090B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104809229A (en) * 2015-05-07 2015-07-29 北京京东尚科信息技术有限公司 Method and system for extracting text characteristic words
CN107992827A (en) * 2017-12-03 2018-05-04 湖南工程学院 A kind of method and device of the multiple mobile object tracking based on threedimensional model

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6957200B2 (en) * 2001-04-06 2005-10-18 Honeywell International, Inc. Genotic algorithm optimization method and network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104809229A (en) * 2015-05-07 2015-07-29 北京京东尚科信息技术有限公司 Method and system for extracting text characteristic words
CN107992827A (en) * 2017-12-03 2018-05-04 湖南工程学院 A kind of method and device of the multiple mobile object tracking based on threedimensional model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Genetic Algorithm for Depth Images;Danciu G，et al.;《2014 IEEE 20th International Symposium for Design and Technology in Electronic Packaging》;20141026;全文 *
Visual tracking with genetic algorithm augmented;Qu L，et a1．;《Signal Image&Video Processing》;20170601;全文 *

Also Published As

Publication number Publication date
CN110047090A (en) 2019-07-23

Similar Documents

Publication Publication Date Title
Man et al. Genetic algorithms: concepts and designs
CN101271572B (en) Image segmentation method based on immunity clone selection clustering
CN106529574B (en) Image classification method based on sparse autocoder and support vector machines
CN113469356A (en) Improved VGG16 network pig identity recognition method based on transfer learning
CN109446185B (en) Collaborative filtering missing data processing method based on user clustering
CN107330902B (en) Chaotic genetic BP neural network image segmentation method based on Arnold transformation
CN103942571B (en) Graphic image sorting method based on genetic programming algorithm
CN111898689A (en) Image classification method based on neural network architecture search
CN111832949A (en) Construction method of equipment combat test identification index system
de Oliveira et al. Image segmentation using multilevel thresholding and genetic algorithm: an approach
CN114219824A (en) Visible light-infrared target tracking method and system based on deep network
CN107451617B (en) Graph transduction semi-supervised classification method
CN114743027A (en) Weak supervision learning-guided cooperative significance detection method
Goel et al. Gray level enhancement to emphasize less dynamic region within image using genetic algorithm
CN114241267A (en) Structural entropy sampling-based multi-target architecture search osteoporosis image identification method
CN110047090B (en) RGB-D target tracking method based on evolution feature learning
CN113269617A (en) Large-scale multi-objective optimization commodity recommendation method based on image coding
CN110047088B (en) HT-29 image segmentation method based on improved teaching and learning optimization algorithm
CN116051924B (en) Divide-and-conquer defense method for image countermeasure sample
CN111814884A (en) Target detection network model upgrading method based on deformable convolution
CN116306919A (en) Large-scale multi-objective combination optimization method based on problem recombination and application
CN114581470B (en) Image edge detection method based on plant community behaviors
CN113469270B (en) Semi-supervised intuitive clustering method based on decomposition multi-target differential evolution superpixel
CN106709572A (en) Data processing method and equipment
CN108090514B (en) Infrared image identification method based on two-stage density clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20221220

Address after: Room 305 and 307, No. 68 Shengtai Road, Jiangning Development Zone, Nanjing, Jiangsu 211106

Patentee after: Nanjing Longyuan Zhongchuang Information Technology Co.,Ltd.

Address before: 223005 A12-2, high tech Industrial Park, three East seven street, Hongze District, Huaian, Jiangsu (Hongze technology transfer center Hongze sub center)

Patentee before: HUAIYIN INSTITUTE OF TECHNOLOGY

TR01 Transfer of patent right