CN110084108A - Pedestrian re-identification system and method based on GAN neural network - Google Patents
Pedestrian re-identification system and method based on GAN neural network Download PDFInfo
- Publication number
- CN110084108A CN110084108A CN201910208515.3A CN201910208515A CN110084108A CN 110084108 A CN110084108 A CN 110084108A CN 201910208515 A CN201910208515 A CN 201910208515A CN 110084108 A CN110084108 A CN 110084108A
- Authority
- CN
- China
- Prior art keywords
- image
- pedestrian
- network
- feature
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 26
- 230000002708 enhancing effect Effects 0.000 claims abstract description 29
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims description 22
- 230000004913 activation Effects 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 16
- 239000000284 extract Substances 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 11
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 6
- 238000000513 principal component analysis Methods 0.000 claims description 6
- 230000017105 transposition Effects 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 239000011248 coating agent Substances 0.000 claims description 5
- 238000000576 coating method Methods 0.000 claims description 5
- 230000004069 differentiation Effects 0.000 claims description 5
- 230000004083 survival effect Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000000536 complexating effect Effects 0.000 claims description 3
- 230000006855 networking Effects 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 6
- 238000012544 monitoring process Methods 0.000 abstract description 4
- 238000005259 measurement Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 230000001149 cognitive effect Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000011551 log transformation method Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 101100477520 Homo sapiens SHOX gene Proteins 0.000 description 1
- 102000048489 Short Stature Homeobox Human genes 0.000 description 1
- 108700025071 Short Stature Homeobox Proteins 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 230000035800 maturation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Abstract
The invention relates to a pedestrian re-identification system and method based on a GAN neural network in the technical field of pedestrian re-identification, which are divided into two stages of image reconstruction and enhancement and pedestrian re-identification. In the image reconstruction and enhancement stage, firstly, a generation network in a GAN neural network is utilized to generate a high-resolution image from a low-resolution image; and then judging whether the generated image or the original image is acquired through a judgment network. If the image is generated, the image is generated again until the network cannot be distinguished, and the image reconstruction is completed; then, enhancing the image by adopting a Retinex algorithm; and in the pedestrian re-identification stage, firstly, extracting color features by using an HSV histogram on the basis of image reconstruction and enhancement, extracting texture features by using SILTP, extracting LOMO features of an image, finally, reducing the dimension of a space by using an XQDA method, and carrying out pedestrian re-identification by using distance measurement. The method has the advantages of high accuracy, good robustness, good generalization capability and the like, and is suitable for video pedestrian monitoring, video pedestrian information retrieval and other applications.
Description
Technical field
The present invention relates to pedestrian's weight identification technology fields, and in particular, to a kind of pedestrian based on GAN neural network knows again
Other system and method.
Background technique
Pedestrian identifies that (Person re-identification) is also referred to as pedestrian and identifies again again, is to utilize computer vision skill
Art judges the technology that whether there is specific pedestrian in image or video sequence.Pedestrian identifies that mainly there are two committed steps again:
Firstly, feature extraction, the i.e. feature of extraction target pedestrian and candidate pedestrian.Then, metric learning, i.e. calculating target pedestrian and time
The characteristic similarity for selecting pedestrian judges whether candidate pedestrian is the target to be looked for.Originally researcher introduces some image procossings, mould
Formula identifies the maturation method in field, stresses to have studied the available feature of pedestrian, simple classification algorithm.Since two thousand and ten, pedestrian
The training library of weight identification technology tends to large-scale, and deep learning frame is widely used and method is trained and reasoning.
Domestic patent CN108520226A proposes a kind of decompose based on body and the pedestrian side of identification again of conspicuousness detection
Method.Pedestrian image is resolved into the semantic area with deep decomposition network first, pedestrian is separated from cluttered environment.
Next, pedestrian image is divided into fritter, according to the result and salient region of background deduction, effective picture region is automatically selected
Domain.LOMO feature is extracted from selected picture region and from whole image finally, extracting PHOG, HSV histogram and SIFT feature.
Domestic patent CN105718882A proposes a kind of resolution ratio self-adaptive feature extraction and pedestrian's recognition methods again for merging.This method
Biological characteristic and macroscopic features are merged, identifies pedestrian, the difference of Enhanced feature with the method that face characteristic and macroscopic features merge
Property, while the performance according to feature on different images scale, different pedestrian's features will be compared on different scale；Together
Shi Caiyong Filtering system is first screened with the fusion feature that color characteristic and contour feature obtain, then with face characteristic pair
The selection result is supplemented, finally the texture feature extraction on the pedestrian of screening, very big stable color provincial characteristics and weighting face
Color characteristic, and the global characteristics and local feature that are extracted using adaptive weighted method fusion obtain fusion feature.
Foreign patent US9396412B2 proposes a kind of recognition methods again of the pedestrian based on semantic color designation.This method is adopted
Character image is described with semantic color designation, in combination with traditional color characteristic, calculates the probability point of 11 kinds of basic colors
Cloth is as image descriptor.This method is given from rgb value to the mapping of the probability distribution of 11 kinds of colors, constructs semantic histogram conduct
Image descriptor.In order to which preferably by semantic color designation and other widely used features, (including color histogram, texture are straight
Side's figure and covariance matrix) it combines, this method will be defined as partial descriptor based on apparent affinity model, and pass through phase
It measures to form linear combination like property.Different from other conventional methods, it is used as special using the long original vector with antipode
Levy pond.Secondly, learning the weight of each similarity measurement by RankBoost algorithm.It is related in foreign patent US10108850B1
And a kind of recognition methods again of the pedestrian based on AlexNet neural network.This method extracts the various of pedestrian in different pictures first
Physical trait, such as trunk, foot etc., then by the pre-training of convolutional neural networks and the training of AlexNet, finally by feature
Than realizing that pedestrian identifies again.
Summary of the invention
In view of the drawbacks of the prior art, the object of the present invention is to provide a kind of pedestrians based on GAN neural network to identify again
System and method.The present invention establishes the neural network structure based on GAN, by image reconstruction and enhancing, solves difference and takes the photograph
Under camera and different perspectives, the problem of image definition, and the sensitivity of image under different illumination is reduced by Retinex algorithm
Degree；Realize that pedestrian identifies again by LOMO feature combination XQDA method simultaneously, with accuracy is high, robustness is good, generalization ability
The advantages that good, is suitable for the practical applications such as video pedestrian monitoring, video pedestrian's information retrieval.
Pedestrian's weight identifying system that the present invention relates to a kind of based on GAN neural network, including image reconstruction and enhancing module,
Pedestrian's weight identification module；
Described image, which is rebuild, to be built up with enhancing module for generating network, differentiation network and loss function calculating and counterweight
Image enhanced；
Pedestrian's weight identification module is used to rebuild described image the enhancing picture obtained with enhancing module and carries out pedestrian
It identifies again.
Preferably, the generation network includes multiple residual blocks, includes two 3 × 3 convolutional layers, volume in each residual block
Connection batch standardization layer after lamination chooses PReLU as activation primitive, reconnects two sub-pix convolutional layers and be used to increase feature
Size.
Preferably, the differentiation network includes 8 convolutional layers, and as the network number of plies is deepened, Characteristic Number is continuously increased, special
Sign size constantly reduces, and chooses LeakyReLU as activation primitive, activates letter eventually by two full articulamentums and sigmoid
Number obtains the probability for being predicted as original image.
Preferably, what the differentiation network judgement obtained is to generate image or original image, if image is generated, then again
It generates, until differentiating that network cannot be distinguished, then completes image reconstruction.
Preferably, the loss function calculating includes being weighted meter using different weights to content loss and confrontation loss
It calculates, the content loss be the Euclidean distance generated between image and original image, and the confrontation loses to generate and allow arbiter
Indistinguishable data distribution.
Preferably, the content loss includes the Minimum Mean Square Error Mseloss of pixel space and with 19 layers of Vgg net of pre-training
The Euclidean distance Vggloss between image and original image feature is sought survival into based on the ReLU active coating of network.
Preferably, described image, which is rebuild, carries out image enhancement using Retinex algorithm algorithm with enhancing module.
Preferably, the Retinex algorithm algorithm is first according to R, G of pixel, B component by the cromogram of input
As being decomposed into three width images, the intensity of the different reflected light of scene medium wavelength is represented；Calculate separately long wave, medium wave and short-wave band
Opposite relationship between light and dark between interior pixel, and then determine the color of each pixel, finally by the color line in Retinex chrominance space
Property be mapped to rgb space, obtain enhancing image.
Preferably, pedestrian's weight identification module extracts color with HSV histogram on the basis of image reconstruction and enhancing first
Feature with SILTP texture feature extraction, then extracts image LOMO feature, carries out dimensionality reduction to space finally by XQDA method, and
Pedestrian is carried out using distance metric to identify again.
A kind of recognition methods again of the pedestrian based on GAN neural network, which comprises the steps of:
Step 1, building generates network:
Generating network portion includes multiple residual blocks, includes two 3 × 3 convolutional layers in each residual block, after convolutional layer
Connection batch standardization layer chooses PReLU as activation primitive, reconnects two sub-pix convolutional layers for increased feature sizes；
Step 2, building differentiates network:
Differentiate that network portion includes 8 convolutional layers, as the network number of plies is deepened, Characteristic Number is continuously increased, characteristic size
Constantly reduce, chooses LeakyReLU as activation primitive, obtained eventually by two full articulamentums and sigmoid activation primitive
It is predicted as the probability of original image；
Step 3, loss function calculates:
Loss includes two parts: the content loss of super-resolution image (SR)Weighted sum generator (Gen) confrontation
Loss
Wherein X indicates one group of high-definition picture and low-resolution image.
Content loss indicates to generate the Euclidean distance between image and original image, including Mse loss and Vgg loss；
The Minimum Mean Square Error of Mse loss expression pixel space:
Wherein, x, y indicate image coordinate point；R indicates decimation factor；W indicates low resolution (LR) image ILRWidth；H
Indicate the height of low-resolution image；RW and rH respectively indicates r times of low-resolution image ILRWidth and height；IHRIt indicates
High-definition picture, i.e. original image；GθG(ILR)X, yThe high resolution graphics of networking of making a living complexing；GθGIndicate that generator, θ G indicate
The weight and biasing of L layer depth network；
Vgg loss: based on the ReLU active coating of 19 layers of Vgg network of pre-training, image and original image feature are sought survival into
Between Euclidean distance, extract the feature map of a certain layer on trained Vgg, image current layer will be generated
Feature map corresponding to feature map and original image is compared:
Wherein, WI, jAnd HI, jThe size of each characteristic pattern in VGG network is described；I, j indicate that i-th maximizes pooling
Jth time convolution after layer；φ corresponds to the characteristic pattern that certain convolutional layer exports after activation primitive among VGG network, φI, j
(IHR)X, yIndicate the characteristic pattern of high-definition picture, φI, j(GθG(ILR))X, yIndicate the characteristic pattern of generation network composograph,
The loss function can reflect the error on higher perception level, and mean square error loss item can only reflect between the pixel of low level
Error, therefore VGG loss item also known as perceive loss item；
Confrontation lossFor generating the data distribution for making arbiter indistinguishable:
Wherein, DθDFor arbiter, θ D is the weight of arbiter, the probability depending on candidate samples from data distribution；DθD
(GθG(ILR)) what is indicated is that arbiter will generate the probability that image prediction is original image；N is sample size；
Step 4, image enhancement is carried out using Retinex algorithm:
The color image of input is decomposed by three width images according to R, G of pixel, B component first, represents scene medium wavelength
The intensity of different reflected lights；It calculates separately long wave, the opposite relationship between light and dark in medium wave and short-wave band between pixel, and then determines
The color of each pixel obtains enhancing image finally, the color in Retinex chrominance space is linearly mapped to rgb space；
Step 5, feature extraction:
On the basis of step 1-4 carries out image reconstruction and enhancing, color characteristic is extracted with HSV histogram, is mentioned with SILTP
Take textural characteristics；
Step 6, LOMO extracts feature:
The down-sampled of 2 × 2average pooling twice is carried out to original image, LOMO is all made of to three images and is mentioned
Feature is taken, then by the merging features of three images at a feature vector, is finally used for the especially big value in feature vector
Log transformation is inhibited, renormalization later to unit-sized；
Step 7, dimensionality reduction and distance metric are carried out to space using XQDA method:
Enabling Δ=xi-xj indicates the feature difference between 2 samples, and P is Gaussian Profile, and P (Δ | Ω I) it is similar sample
Between discrepancy delta meet the Gaussian Profile of difference Ω I in class, the discrepancy delta of P (Δ | Ω E) between foreign peoples's sample meets between class
The Gaussian Profile of difference Ω E, and mean value is all 0:
Wherein, T is matrix transposition, and d (xi-xj) is distance function, ∑EAnd ∑IIt is similar sample respectively to collection and dissmilarity
Covariance matrix of the sample to collection sample；F (Δ) indicates the distance between 2 samples, if more than 0, then it represents that very in maximum probability
It is not similar；
XQDA method carries out feature principal component analysis and similarity-based learning simultaneously, by learning mapping matrix W ∈ Rd×r(r
< d), by primitive character xi, xi∈RdIt is mapped to multidimensional subspace, Feature Dimension Reduction is realized, improves the accuracy of characteristic matching,
Middle matrix W is made of the corresponding feature vector of preceding r maximum eigenvalue of principal component analysis, and d indicates dimension, therefore, formula (6)
Defined in distance function change conversion are as follows:
F (Δ)=(x-z)TW(∑′I -1-∑′E -1)WT(x-z) (7)
Wherein, T be matrix transposition, ∑ 'I -1=WT∑IW, indicate dissimilar sample to the covariance matrix of collection sample,
∑′E -1=WT∑EW indicates similar sample to the covariance matrix of collection；X is the sample point at a visual angle, and z is another visual angle
Sample point；
Step 8: output recognition result.
Compared with prior art, the present invention have it is following the utility model has the advantages that
1, the present invention establishes the neural network structure based on GAN, by image reconstruction and enhancing, solves different camera shootings
Under machine and different perspectives, the problem of image definition, and the sensitivity of image under different illumination is reduced by Retinex algorithm
Degree.
2, the present invention by LOMO feature combination XQDA method realize pedestrian identify again, have accuracy height, robustness it is good,
The advantages that generalization ability is good is suitable for the practical applications such as video pedestrian monitoring, video pedestrian's information retrieval.
Detailed description of the invention
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon:
Fig. 1 is the recognition methods flow diagram again of the pedestrian based on GAN neural network.
Specific embodiment
The present invention is described in detail combined with specific embodiments below.Following embodiment will be helpful to the technology of this field
Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill of this field
For personnel, without departing from the inventive concept of the premise, various modifications and improvements can be made.These belong to the present invention
Protection scope.
Embodiment
In view of the above demand, the present invention relates to a kind of recognition methods again of the pedestrian based on GAN neural network, are divided into image weight
It builds and identifies two stages again with enhancing and pedestrian.Image reconstruction and enhancing stage, first with the generation net in GAN neural network
The image of low resolution is generated high-definition picture by network.It then, is to generate image by differentiate network judgement acquisition, still
Original image.If generating image, then regenerate, until differentiating that network cannot be distinguished, then completes image reconstruction；It uses again
Retinex algorithm enhances image.Pedestrian's weight cognitive phase is straight with HSV on the basis of image reconstruction and enhancing first
Side's figure extracts color characteristic, with SILTP texture feature extraction, then image LOMO feature is extracted, finally by XQDA method to sky
Between carry out dimensionality reduction, and carry out pedestrian using distance metric and identify again.
As shown in Figure 1, being divided into image reconstruction the present invention relates to a kind of recognition methods again of the pedestrian based on GAN neural network
Two stages are identified again with enhancing stage and pedestrian.
The step of image reconstruction is with the enhancing stage is as follows:
Step 1: building generates network
Generating network portion includes multiple residual blocks, includes two 3 × 3 convolutional layers in each residual block, after convolutional layer
Connection batch standardization layer chooses PReLU as activation primitive, reconnects two sub-pix convolutional layers for increased feature sizes.
Step 2: building differentiates network
Differentiate that network portion includes 8 convolutional layers, as the network number of plies is deepened, Characteristic Number is continuously increased, characteristic size
Constantly reduce, chooses LeakyReLU as activation primitive, obtained eventually by two full articulamentums and sigmoid activation primitive
It is predicted as the probability of original image.
Step 3: loss function calculates
Loss function calculates:
Loss includes two parts: the content loss of super-resolution image (SR)Weighted sum generator (Gen) confrontation
Loss
Wherein X indicates one group of high-definition picture and low-resolution image.
Content loss indicates to generate the Euclidean distance between image and original image, including Mse loss and Vgg loss；
The Minimum Mean Square Error of Mse loss expression pixel space:
Wherein, x, y indicate image coordinate point；R indicates decimation factor；W indicates low resolution (LR) image ILRWidth；H
Indicate the height of low-resolution image；RW and rH respectively indicates r times of low-resolution image ILRWidth and height；IHRIt indicates
High-definition picture, i.e. original image；GθG(ILR)X, yThe high resolution graphics of networking of making a living complexing；GθGIndicate that generator, θ G indicate
The weight and biasing of L layer depth network；
Vgg loss: based on the ReLU active coating of 19 layers of Vgg network of pre-training, image and original image feature are sought survival into
Between Euclidean distance, extract the feature map of a certain layer on trained Vgg, image current layer will be generated
Feature map corresponding to feature map and original image is compared:
Wherein, WI, jAnd HI, jThe size of each characteristic pattern in VGG network is described；I, j indicate that i-th maximizes pooling
Jth time convolution after layer；φ corresponds to the characteristic pattern that certain convolutional layer exports after activation primitive among VGG network, φI, j
(IHR)X, yIndicate the characteristic pattern of high-definition picture, φI, j(GθG(ILR))X, yIndicate the characteristic pattern of generation network composograph, it should
Loss function can reflect the error on higher perception level, and mean square error loss item can only reflect between the pixel of low level
Error, therefore VGG loss item also known as perceives loss item；
Confrontation lossFor generating the data distribution for making arbiter indistinguishable:
Wherein, DθDFor arbiter, θ D is the weight of arbiter, the probability depending on candidate samples from data distribution；DθD
(GθG(ILR)) what is indicated is that arbiter will generate the probability that image prediction is original image；N is sample size；
Step 4: carrying out image enhancement using Retinex algorithm
The color image of input is decomposed by three width images according to R, G of pixel, B component first, represents scene medium wavelength
The intensity of different reflected lights；It calculates separately long wave, the opposite relationship between light and dark in medium wave and short-wave band between pixel, and then determines
The color of each pixel obtains enhancing image finally, the color in Retinex chrominance space is linearly mapped to rgb space.
Pedestrian's weight cognitive phase step:
Step 1: feature extraction
Color characteristic is extracted with HSV histogram on the basis of image reconstruction and enhancing, with SILTP texture feature extraction.
In view of multi-scale information, the down-sampled of 2 × 2average pooling twice is carried out to original image, three images are adopted
Feature is extracted with LOMO, then by the merging features of three images at a feature vector, finally, for the spy in feature vector
Big value is inhibited using log transformation, renormalization later to unit-sized；
Step 2: dimensionality reduction and distance metric are carried out to space using XQDA method:
Enabling Δ=xi-xj indicates the feature difference between 2 samples, and P is Gaussian Profile, and P (Δ | Ω I) it is similar sample
Between discrepancy delta meet the Gaussian Profile of difference Ω I in class, the discrepancy delta of P (Δ | Ω E) between foreign peoples's sample meets between class
The Gaussian Profile of difference Ω E, and mean value is all 0:
Wherein, T is matrix transposition, and d (xi-xj) is distance function, ∑EAnd ∑IIt is similar sample respectively to collection and dissmilarity
Covariance matrix of the sample to collection sample；F (Δ) indicates the distance between 2 samples, if more than 0, then it represents that very in maximum probability
It is not similar；
XQDA method carries out feature principal component analysis and similarity-based learning simultaneously, by learning mapping matrix W ∈ Rd×r(r
< d), by primitive character xi, xj∈RdIt is mapped to multidimensional subspace, Feature Dimension Reduction is realized, improves the accuracy of characteristic matching,
Middle matrix W is made of the corresponding feature vector of preceding r maximum eigenvalue of principal component analysis, and d indicates dimension, therefore, formula (6)
Defined in distance function change conversion are as follows:
F (Δ)=(x-z)TW(∑′I -1-∑′E -1)WT(x-z) (7)
Wherein, T be matrix transposition, ∑ 'I -1=WT∑IW, indicate dissimilar sample to the covariance matrix of collection sample,
∑′E -1=WT∑EW indicates similar sample to the covariance matrix of collection；X is the sample point at a visual angle, and z is another visual angle
Sample point；
Step 3: output recognition result.
In conclusion the present invention establishes the neural network structure based on GAN, by image reconstruction and enhancing, solve
Under different cameras and different perspectives, the problem of image definition, and image under different illumination is reduced by Retinex algorithm
Susceptibility；It realizes that pedestrian identifies again by LOMO feature combination XQDA method simultaneously, has accuracy height, robustness good, general
The advantages that change ability is good is suitable for the practical applications such as video pedestrian monitoring, video pedestrian's information retrieval.
Specific embodiments of the present invention are described above.It is to be appreciated that the invention is not limited to above-mentioned
Particular implementation, those skilled in the art can make a variety of changes or modify within the scope of the claims, this not shadow
Ring substantive content of the invention.In the absence of conflict, the feature in embodiments herein and embodiment can any phase
Mutually combination.
Claims (10)
1. it is a kind of based on GAN neural network pedestrian weight identifying system, which is characterized in that including image reconstruction and enhancing module,
Pedestrian's weight identification module；
Described image, which is rebuild, can generate network with enhancing module, differentiate network and the figure that loss function calculates and counterweight is built up
As being enhanced；
Pedestrian's weight identification module can rebuild the enhancing picture progress pedestrian obtained with enhancing module to described image to be known again
Not.
2. pedestrian's weight identifying system according to claim 1 based on GAN neural network, characterized in that the generation net
Network includes multiple residual blocks, includes two 3 × 3 convolutional layers in each residual block, connection batch standardization layer, chooses after convolutional layer
PReLU reconnects two sub-pix convolutional layers as activation primitive for increased feature sizes.
3. pedestrian's weight identifying system according to claim 1 based on GAN neural network, characterized in that the differentiation net
Network includes 8 convolutional layers, and as the network number of plies is deepened, Characteristic Number is continuously increased, and characteristic size constantly reduces, and is chosen
LeakyReLU obtains being predicted as original image as activation primitive eventually by two full articulamentums and sigmoid activation primitive
Probability.
4. pedestrian's weight identifying system according to claim 3 based on GAN neural network, characterized in that the differentiation net
What network judgement obtained is to generate image or original image, if image is generated, is then regenerated, until differentiating that network can not area
Point, then complete image reconstruction.
5. pedestrian's weight identifying system according to claim 1 based on GAN neural network, characterized in that the loss letter
It includes that content loss and confrontation loss are weighted using different weights that number, which calculates, and the content loss is to generate image
Euclidean distance between original image, the confrontation loss are used to generate the data distribution for making arbiter indistinguishable.
6. pedestrian's weight identifying system according to claim 5 based on GAN neural network, characterized in that the content damage
It loses the Minimum Mean Square Error Mseloss including pixel space and is sought survival based on the ReLU active coating of 19 layers of Vgg network of pre-training
At the Euclidean distance Vggloss between image and original image feature.
7. pedestrian's weight identifying system according to claim 1 based on GAN neural network, characterized in that described image weight
It builds and carries out image enhancement using Retinex algorithm algorithm with enhancing module.
8. pedestrian's weight identifying system according to claim 7 based on GAN neural network, characterized in that the Retinex
The color image of input is decomposed into three width images according to R, G of pixel, B component first by algorithm algorithm, is represented in scene
The intensity of the different reflected light of wavelength；Calculate separately long wave, the opposite relationship between light and dark in medium wave and short-wave band between pixel, in turn
It determines the color of each pixel, the color in Retinex chrominance space is linearly finally mapped to rgb space, obtain enhancing figure
Picture.
9. pedestrian's weight identifying system according to claim 1 based on GAN neural network again, characterized in that pedestrian identifies
Module extracts color characteristic with HSV histogram on the basis of image reconstruction and enhancing first, with SILTP texture feature extraction,
Image LOMO feature is extracted again, dimensionality reduction is carried out to space finally by XQDA method, and carry out pedestrian using distance metric and know again
Not.
10. a kind of recognition methods again of the pedestrian based on GAN neural network, which comprises the steps of:
Step 1, building generates network:
Generating network portion includes multiple residual blocks, includes two 3 × 3 convolutional layers in each residual block, connects after convolutional layer
Standardization layer is criticized, PReLU is chosen as activation primitive, reconnects two sub-pix convolutional layers for increased feature sizes；
Step 2, building differentiates network:
Differentiate that network portion includes 8 convolutional layers, as the network number of plies is deepened, Characteristic Number is continuously increased, and characteristic size is continuous
Reduce, chooses LeakyReLU as activation primitive, predicted eventually by two full articulamentums and sigmoid activation primitive
For the probability of original image；
Step 3, loss function calculates:
Loss includes two parts: the content loss of super-resolution image (SR)Weighted sum generator (Gen) confrontation loss
Wherein X indicates one group of high-definition picture and low-resolution image.
Content loss indicates to generate the Euclidean distance between image and original image, including Mse loss and Vgg loss；
The Minimum Mean Square Error of Mse loss expression pixel space:
Wherein, x, y indicate image coordinate point；R indicates decimation factor；W indicates low resolution (LR) image ILRWidth；H is indicated
The height of low-resolution image；RW and rH respectively indicates r times of low-resolution image ILRWidth and height；IHRIndicate high score
Resolution image, i.e. original image；GθG(ILR)X, yThe high resolution graphics of networking of making a living complexing；GθGIndicate that generator, θ G indicate L layers
The weight and biasing of depth network；
Vgg loss: it based on the ReLU active coating of 19 layers of Vgg network of pre-training, seeks survival between image and original image feature
Euclidean distance, extract the feature map of a certain layer on trained Vgg, image current layer will be generated
Feature map corresponding to feature map and original image is compared:
Wherein, WI, jAnd HI, jThe size of each characteristic pattern in VGG network is described；After i, j indicate that i-th maximizes pooling layers
Jth time convolution；φ corresponds to the characteristic pattern that certain convolutional layer exports after activation primitive among VGG network, φI, j(IHR)X, yTable
Show the characteristic pattern of high-definition picture, φI, j(GθG(ILR))X, yIndicate the characteristic pattern of generation network composograph, the loss function
Can error on the higher perception level of reflection, and mean square error loss item can only reflect the error between the pixel of low level, because
This VGG loses item and also known as perceives loss item；
Confrontation lossFor generating the data distribution for making arbiter indistinguishable:
Wherein, DθDFor arbiter, θ D is the weight of arbiter, the probability depending on candidate samples from data distribution；DθD(GθG
(ILR)) what is indicated is that arbiter will generate the probability that image prediction is original image；N is sample size；
Step 4, image enhancement is carried out using Retinex algorithm:
The color image of input is decomposed by three width images according to R, G of pixel, B component first, it is different to represent scene medium wavelength
Reflected light intensity；It calculates separately long wave, the opposite relationship between light and dark in medium wave and short-wave band between pixel, and then determines each
The color of pixel obtains enhancing image finally, the color in Retinex chrominance space is linearly mapped to rgb space；
Step 5, feature extraction:
On the basis of step 1-4 carries out image reconstruction and enhancing, color characteristic is extracted with HSV histogram, extracts line with SILTP
Manage feature；
Step 6, LOMO extracts feature:
The down-sampled of 2 × 2 average pooling twice is carried out to original image, to three images be all made of LOMO extract it is special
Sign is finally become for the especially big value in feature vector using log then by the merging features of three images at a feature vector
Swap-in row inhibits, renormalization later to unit-sized；
Step 7, dimensionality reduction and distance metric are carried out to space using XQDA method:
Enabling Δ=xi-xj indicates the feature difference between 2 samples, and P is Gaussian Profile, and P (Δ | Ω I) between similar sample
Discrepancy delta meet the Gaussian Profile of difference Ω I in class, the discrepancy delta of P (Δ | Ω E) between foreign peoples's sample meets class inherited
The Gaussian Profile of Ω E, and mean value is all 0:
Wherein, T is matrix transposition, and d (xi-xj) is distance function, ∑EAnd ∑IIt is similar sample respectively to collection and dissimilar sample
To the covariance matrix of collection sample；F (Δ) indicates the distance between 2 samples, if more than 0, then it represents that be not in maximum probability very much
It is similar；
XQDA method carries out feature principal component analysis and similarity-based learning simultaneously, by learning mapping matrix W ∈ Rd×r(r <
D), by primitive character xi, xj∈RdIt is mapped to multidimensional subspace, Feature Dimension Reduction is realized, improves the accuracy of characteristic matching, wherein
Matrix W is made of the corresponding feature vector of preceding r maximum eigenvalue of principal component analysis, and d indicates dimension, therefore, in formula (6)
The distance function of definition changes conversion are as follows:
F (Δ)=(x-z)TW(∑′I -1-∑′E -1)WT(x-z) (7)
Wherein, T be matrix transposition, ∑ 'I -1=WT∑IW, indicate dissimilar sample to the covariance matrix of collection sample, ∑ 'E -1=
WT∑EW indicates similar sample to the covariance matrix of collection；X is the sample point at a visual angle, and z is the sample point at another visual angle；
Step 8: output recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910208515.3A CN110084108A (en) | 2019-03-19 | 2019-03-19 | Pedestrian re-identification system and method based on GAN neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910208515.3A CN110084108A (en) | 2019-03-19 | 2019-03-19 | Pedestrian re-identification system and method based on GAN neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110084108A true CN110084108A (en) | 2019-08-02 |
Family
ID=67413310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910208515.3A Pending CN110084108A (en) | 2019-03-19 | 2019-03-19 | Pedestrian re-identification system and method based on GAN neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110084108A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110688966A (en) * | 2019-09-30 | 2020-01-14 | 华东师范大学 | Semantic-guided pedestrian re-identification method |
| CN110728238A (en) * | 2019-10-12 | 2020-01-24 | 安徽工程大学 | Personnel re-detection method of fusion type neural network |
| CN111178178A (en) * | 2019-12-16 | 2020-05-19 | 汇纳科技股份有限公司 | Multi-scale pedestrian re-identification method, system, medium and terminal combined with region distribution |
| CN111275694A (en) * | 2020-02-06 | 2020-06-12 | 电子科技大学 | Attention mechanism guided progressive division human body analytic model and method |
| CN111340700A (en) * | 2020-02-21 | 2020-06-26 | 北京中科虹霸科技有限公司 | Model generation method, resolution improvement method, image identification method and device |
| CN111402143A (en) * | 2020-06-03 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN111460891A (en) * | 2020-03-01 | 2020-07-28 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Automatic driving-oriented vehicle-road cooperative pedestrian re-identification method and system |
| CN111476353A (en) * | 2020-04-07 | 2020-07-31 | 中国科学院重庆绿色智能技术研究院 | Super-resolution method of GAN image introducing significance |
| CN111539382A (en) * | 2020-05-22 | 2020-08-14 | 支付宝(杭州)信息技术有限公司 | Image recognition model privacy risk assessment method and device and electronic equipment |
| CN111582126A (en) * | 2020-04-30 | 2020-08-25 | 浙江工商大学 | Pedestrian re-identification method based on multi-scale pedestrian contour segmentation fusion |
| CN112016402A (en) * | 2020-08-04 | 2020-12-01 | 杰创智能科技股份有限公司 | Unsupervised learning-based pedestrian re-identification field self-adaption method and device |
| CN112200106A (en) * | 2020-10-16 | 2021-01-08 | 中国计量大学 | Cross-camera pedestrian re-identification and tracking method |
| CN112348766A (en) * | 2020-11-06 | 2021-02-09 | 天津大学 | Progressive feature stream depth fusion network for surveillance video enhancement |
| CN112446361A (en) * | 2020-12-16 | 2021-03-05 | 上海芯翌智能科技有限公司 | Method and equipment for cleaning training data |
| CN112507617A (en) * | 2020-12-03 | 2021-03-16 | 青岛海纳云科技控股有限公司 | Training method of SRFlow super-resolution model and face recognition method |
| CN112906614A (en) * | 2021-03-08 | 2021-06-04 | 中南大学 | Pedestrian re-identification method and device based on attention guidance and storage medium |
| CN113128441A (en) * | 2021-04-28 | 2021-07-16 | 安徽大学 | System and method for identifying vehicle weight by embedding structure of attribute and state guidance |
| CN113344798A (en) * | 2020-02-18 | 2021-09-03 | 济南浪潮高新科技投资发展有限公司 | Retinex-based dark image enhancement method |
| CN113536995A (en) * | 2021-06-30 | 2021-10-22 | 河南大学 | Pedestrian re-identification method based on feature mapping space and sample judgment |
| CN112016402B (en) * | 2020-08-04 | 2024-05-17 | 杰创智能科技股份有限公司 | Self-adaptive method and device for pedestrian re-recognition field based on unsupervised learning |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520199A (en) * | 2018-03-04 | 2018-09-11 | 天津大学 | Based on radar image and the human action opener recognition methods for generating confrontation model |
-
2019
- 2019-03-19 CN CN201910208515.3A patent/CN110084108A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520199A (en) * | 2018-03-04 | 2018-09-11 | 天津大学 | Based on radar image and the human action opener recognition methods for generating confrontation model |
Non-Patent Citations (4)
| Title |
|---|
| LEDIG C等: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network", 《ARXIV:1609.04802V5 [CS.CV] 25 MAY 2017》 * |
| SHENGCAI LIAO等: "Person Re-identification by Local Maximal Occurrence Representation and Metric Learning", 《2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
| 李学明: "基于Retinex理论的图像增强算法", 《万方学术期刊数据库》 * |
| 金翠等: "基于随机擦除行人对齐网络的行人重识别方法", 《山东大学学报（工学版）》 * |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110688966B (en) * | 2019-09-30 | 2024-01-09 | 华东师范大学 | Semantic guidance pedestrian re-recognition method |
| CN110688966A (en) * | 2019-09-30 | 2020-01-14 | 华东师范大学 | Semantic-guided pedestrian re-identification method |
| CN110728238A (en) * | 2019-10-12 | 2020-01-24 | 安徽工程大学 | Personnel re-detection method of fusion type neural network |
| CN111178178A (en) * | 2019-12-16 | 2020-05-19 | 汇纳科技股份有限公司 | Multi-scale pedestrian re-identification method, system, medium and terminal combined with region distribution |
| CN111178178B (en) * | 2019-12-16 | 2023-10-10 | 汇纳科技股份有限公司 | Multi-scale pedestrian re-identification method, system, medium and terminal combined with region distribution |
| CN111275694B (en) * | 2020-02-06 | 2020-10-23 | 电子科技大学 | Attention mechanism guided progressive human body division analysis system and method |
| CN111275694A (en) * | 2020-02-06 | 2020-06-12 | 电子科技大学 | Attention mechanism guided progressive division human body analytic model and method |
| CN113344798A (en) * | 2020-02-18 | 2021-09-03 | 济南浪潮高新科技投资发展有限公司 | Retinex-based dark image enhancement method |
| CN111340700A (en) * | 2020-02-21 | 2020-06-26 | 北京中科虹霸科技有限公司 | Model generation method, resolution improvement method, image identification method and device |
| CN111340700B (en) * | 2020-02-21 | 2023-04-25 | 北京中科虹霸科技有限公司 | Model generation method, resolution improvement method, image recognition method and device |
| CN111460891A (en) * | 2020-03-01 | 2020-07-28 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Automatic driving-oriented vehicle-road cooperative pedestrian re-identification method and system |
| CN111476353B (en) * | 2020-04-07 | 2022-07-15 | 中国科学院重庆绿色智能技术研究院 | Super-resolution method of GAN image introducing significance |
| CN111476353A (en) * | 2020-04-07 | 2020-07-31 | 中国科学院重庆绿色智能技术研究院 | Super-resolution method of GAN image introducing significance |
| CN111582126B (en) * | 2020-04-30 | 2024-02-27 | 浙江工商大学 | Pedestrian re-recognition method based on multi-scale pedestrian contour segmentation fusion |
| CN111582126A (en) * | 2020-04-30 | 2020-08-25 | 浙江工商大学 | Pedestrian re-identification method based on multi-scale pedestrian contour segmentation fusion |
| CN111539382A (en) * | 2020-05-22 | 2020-08-14 | 支付宝(杭州)信息技术有限公司 | Image recognition model privacy risk assessment method and device and electronic equipment |
| CN111402143A (en) * | 2020-06-03 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN111402143B (en) * | 2020-06-03 | 2020-09-04 | 腾讯科技（深圳）有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN112016402B (en) * | 2020-08-04 | 2024-05-17 | 杰创智能科技股份有限公司 | Self-adaptive method and device for pedestrian re-recognition field based on unsupervised learning |
| CN112016402A (en) * | 2020-08-04 | 2020-12-01 | 杰创智能科技股份有限公司 | Unsupervised learning-based pedestrian re-identification field self-adaption method and device |
| CN112200106A (en) * | 2020-10-16 | 2021-01-08 | 中国计量大学 | Cross-camera pedestrian re-identification and tracking method |
| CN112348766B (en) * | 2020-11-06 | 2023-04-18 | 天津大学 | Progressive feature stream depth fusion network for surveillance video enhancement |
| CN112348766A (en) * | 2020-11-06 | 2021-02-09 | 天津大学 | Progressive feature stream depth fusion network for surveillance video enhancement |
| CN112507617A (en) * | 2020-12-03 | 2021-03-16 | 青岛海纳云科技控股有限公司 | Training method of SRFlow super-resolution model and face recognition method |
| CN112446361A (en) * | 2020-12-16 | 2021-03-05 | 上海芯翌智能科技有限公司 | Method and equipment for cleaning training data |
| CN112906614A (en) * | 2021-03-08 | 2021-06-04 | 中南大学 | Pedestrian re-identification method and device based on attention guidance and storage medium |
| CN113128441B (en) * | 2021-04-28 | 2022-10-14 | 安徽大学 | System and method for identifying vehicle weight by embedding structure of attribute and state guidance |
| CN113128441A (en) * | 2021-04-28 | 2021-07-16 | 安徽大学 | System and method for identifying vehicle weight by embedding structure of attribute and state guidance |
| CN113536995B (en) * | 2021-06-30 | 2022-11-18 | 河南大学 | Pedestrian re-identification method based on feature mapping space and sample judgment |
| CN113536995A (en) * | 2021-06-30 | 2021-10-22 | 河南大学 | Pedestrian re-identification method based on feature mapping space and sample judgment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110084108A (en) | Pedestrian re-identification system and method based on GAN neural network | |
| Jiang et al. | Edge-enhanced GAN for remote sensing image superresolution | |
| CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
| CN112507997B (en) | Face super-resolution system based on multi-scale convolution and receptive field feature fusion | |
| CN108573276B (en) | Change detection method based on high-resolution remote sensing image | |
| Ahmad | Deep image retrieval using artificial neural network interpolation and indexing based on similarity measurement | |
| Xiao et al. | Making of night vision: Object detection under low-illumination | |
| CN106599854B (en) | Automatic facial expression recognition method based on multi-feature fusion | |
| CN111625608B (en) | Method and system for generating electronic map according to remote sensing image based on GAN model | |
| CN112560831B (en) | Pedestrian attribute identification method based on multi-scale space correction | |
| CN107679250A (en) | A kind of multitask layered image search method based on depth own coding convolutional neural networks | |
| CN109558806A (en) | The detection method and system of high score Remote Sensing Imagery Change | |
| CN110929593A (en) | Real-time significance pedestrian detection method based on detail distinguishing and distinguishing | |
| CN109635726B (en) | Landslide identification method based on combination of symmetric deep network and multi-scale pooling | |
| CN109543632A (en) | A kind of deep layer network pedestrian detection method based on the guidance of shallow-layer Fusion Features | |
| CN111652273B (en) | Deep learning-based RGB-D image classification method | |
| CN115423734B (en) | Infrared and visible light image fusion method based on multi-scale attention mechanism | |
| Liang et al. | An improved DualGAN for near-infrared image colorization | |
| CN113870160B (en) | Point cloud data processing method based on transformer neural network | |
| Cho et al. | Semantic segmentation with low light images by modified CycleGAN-based image enhancement | |
| CN115359372A (en) | Unmanned aerial vehicle video moving object detection method based on optical flow network | |
| CN114724155A (en) | Scene text detection method, system and equipment based on deep convolutional neural network | |
| CN111428758A (en) | Improved remote sensing image scene classification method based on unsupervised characterization learning | |
| CN113052200A (en) | Sonar image target detection method based on yolov3 network | |
| Guo et al. | Dual graph U-Nets for hyperspectral image classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190802 |
|
| RJ01 | Rejection of invention patent application after publication |