CN113627272A - Serious misalignment pedestrian re-identification method and system based on normalization network - Google Patents

Serious misalignment pedestrian re-identification method and system based on normalization network Download PDF

Info

Publication number
CN113627272A
CN113627272A CN202110812700.0A CN202110812700A CN113627272A CN 113627272 A CN113627272 A CN 113627272A CN 202110812700 A CN202110812700 A CN 202110812700A CN 113627272 A CN113627272 A CN 113627272A
Authority
CN
China
Prior art keywords
network
processing
feature
characteristic diagram
acquiring
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110812700.0A
Other languages
Chinese (zh)
Other versions
CN113627272B (en
Inventor
杨华
何远航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN202110812700.0A priority Critical patent/CN113627272B/en
Publication of CN113627272A publication Critical patent/CN113627272A/en
Application granted granted Critical
Publication of CN113627272B publication Critical patent/CN113627272B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a serious misalignment pedestrian re-identification method and a serious misalignment pedestrian re-identification system based on a normalization network, wherein the method comprises the following steps: according to each input image IwCollecting corresponding randomly clipped non-aligned images I with the same identityu(ii) a For input image Iw、IuProcessing and obtaining characteristic diagram
Figure DDA0003169020450000011
For input image Iw、IuProcessing and acquiring a posture graph Pw、Pu(ii) a Using a gesture guidance generator, from an input feature map
Figure DDA0003169020450000012
And attitude map Pw、PuGenerating a feature map
Figure DDA0003169020450000013
Using feature maps
Figure DDA0003169020450000014
And
Figure DDA0003169020450000015
training the posture guide generator and the posture guide discriminator; for characteristic diagram
Figure DDA0003169020450000016
Processing and acquiring high-order characteristic diagram
Figure DDA0003169020450000017
For high-order characteristic diagram
Figure DDA0003169020450000018
Processing to obtain feature vector
Figure DDA0003169020450000019
And
Figure DDA00031690204500000110
based on the attitude map Pw、PuComputing feature vectors
Figure DDA00031690204500000111
(ii) visibility of; acting on the feature vector with classification loss and distance loss
Figure DDA00031690204500000112
And
Figure DDA00031690204500000113
thereby enabling supervision of network training. The method of the invention is robust and generates alignments for similarity measurementsThe feature map enhances the robustness under complex situations.

Description

Serious misalignment pedestrian re-identification method and system based on normalization network
Technical Field
The invention relates to the technical field of computer vision, in particular to a serious misalignment pedestrian re-identification method and system based on a normalization network.
Background
People Re-identification (Re-ID) is intended to match people images in different camera views. Most research has focused on the traditional Re-ID task. Sometimes, the pedestrian is blocked by an object or other person. Partial Re-ID studied this problem, making it closer to a real scene. It is still assumed that the pedestrian detection is perfect and that the detected pedestrian images are spatially aligned. In fact, a large number of real person images are not cropped out of the exact bounding box, especially when the detector is affected by different camera heights, complex lighting conditions or severe occlusion, resulting in meaningless background filling or severe body part loss. We call this task a severely misaligned pedestrian Re-ID.
When the traditional integral Re-ID tasks (such as Tianlong Chen, Shaojin Ding, Jingyi Xie, Y e Y uon, Wuyang Chen, Y ang Y ang, Zhou Ren, and Zhangyang Wang, Abd-net: extensive but reverse person Re-identification, "in ICCV,2019, pp. 8351-8361.) are directly applied to Re-ID with serious misalignment, the existing methods often cannot work well due to weak alignment capability. Existing partial Re-ID task methods (e.g., Guan' an Wang, Shuo Y ang, Huang u Liu, Zherching Wang, Y ang Y ang, Shuliang Wang, Gang Y u, Erjin Zhou, and Jian Sun, "High-order information models: Learning registration and topology for encapsulated person Re-identification," in CVPR,2020, pp.6449-6458) typically use an attention-based convolution sub-network to segment a feature map into different parts to improve anti-occlusion capability. However, due to the lack of segmentation labels, inaccuracy of pose key points, non-correspondence of spaces between different hierarchical feature maps, and huge data deviation between different scenes, a subnetwork is generally difficult to train well. These limit the stability of the segmentation, especially in complex and realistic scenes where there is severe misalignment.
Disclosure of Invention
The invention provides a serious misalignment pedestrian re-identification method and system based on a normalization network aiming at the problems in the prior art, which summarize the defects of the existing pedestrian re-identification method, do not adopt a mode of dynamically dividing a feature map to align parts, but combine the stability of the whole method and generate an aligned feature map for similarity measurement, thereby enhancing the robustness under a complex situation.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the invention provides a serious misalignment pedestrian re-identification method based on a normalization network, which comprises the following steps:
s11: according to each input image IwCollecting corresponding randomly clipped non-aligned images I with the same identityu；
S12: for the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w；
S13, for the input image Iw、IuProcessing and acquiring a posture graph Pw、Pu；
S14: using the gesture guidance generator, from the input feature map Fi uAnd attitude map Pw、PuGenerating a feature map
Figure BDA0003169020430000021
S15: using feature maps Fi wAnd
Figure BDA0003169020430000022
training the posture guide generator and the posture guide discriminator;
s16: for the characteristic diagram Fi u、Fi wProcessing and acquiring a high-order characteristic diagram Fo u、Fo w；
S17: for the high-order characteristic diagram
Figure BDA0003169020430000023
Processing to obtain a feature vector f1 w...fn wAnd f1 u...fn u；
S18: based on the attitude map Pw、PuCalculating the feature vector f1 u...fn u(ii) visibility of;
s19: by using the effects of classification loss and distance lossThe feature vector f1 w...fn wAnd f1 u...fn uThereby enabling supervision of network training.
Preferably, the S12 includes: using a convolutional neural network to the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w(ii) a Further, the air conditioner is provided with a fan,
the convolutional neural network in S12 is a ResNet50 front three-layer network.
Preferably, the S13 includes: for the processed input image I by using attitude estimation algorithmw、IuProcessing and acquiring a posture graph Pw、Pu(ii) a Further, the air conditioner is provided with a fan,
the attitude estimation algorithm is OpenPose.
Preferably, the posture guidance generator in S14 includes: t blocks, wherein each block contains a spatial recomposition operation.
Preferably, the training process in S15 is applied to the L2 distance loss and discrimination loss; further, the air conditioner is provided with a fan,
the calculation method of the L2 distance loss is as follows:
Figure BDA0003169020430000031
preferably, the S16 includes: using a convolutional neural network to the feature map Fi u、Fi wProcessing and acquiring a high-order characteristic diagram Fo u、Fo w(ii) a Further, the air conditioner is provided with a fan,
the convolutional neural network in S16 is a ResNet50 layer four network.
Preferably, the S17 includes: using convolution and pooling operations on the high-order feature map Fo u、Fo wProcessing to obtain a feature vector f1 w...fn wAnd f1 u...fn u(ii) a Further, the air conditioner is provided with a fan,
the convolution is a convolution with a kernel size of 1 × 1, and the pooling operation is global average pooling.
Preferably, the classification loss in S19 is a cross-entropy loss, which is structured as follows:
Figure BDA0003169020430000032
wherein [ a, b ] is the visible interval in [1, n ]; the Class represents that the scale of the feature vector is scaled to a classification scale through a full-connection layer network, and the cross entropy represents cross entropy calculation;
the distance loss is:
Figure BDA0003169020430000033
preferably, the step S19 is followed by:
s110: the network re-identification effectiveness after S19 was tested using CMC and mag as characterizations.
The invention also provides a serious misalignment pedestrian re-identification system based on the normalization network, which comprises the following components: the system comprises a data collection module, a feature map acquisition module, a posture guide generator, a posture guide training module, a high-order feature acquisition module, a feature vector acquisition module, a component visibility calculation module and a network training supervision module; wherein the content of the first and second substances,
the data gathering module is used for collecting I according to each input imagewCollecting corresponding randomly clipped non-aligned images I with the same identityu；
The characteristic diagram acquisition module is used for acquiring the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w；
The attitude map acquisition module is used for acquiring the input image Iw、IuProcessing and acquiring a posture graph Pw、Pu；
The above-mentionedThe gesture guidance generator is used for utilizing the gesture guidance generator according to the input feature diagram Fi uAnd attitude map Pw、PuGenerating a feature map
Figure BDA0003169020430000041
The posture guide training module is used for utilizing a feature map Fi wAnd
Figure BDA0003169020430000042
training the posture guide generator and the posture guide discriminator;
the high-order characteristic acquisition module is used for acquiring the characteristic diagram Fi u、Fi wProcessing and acquiring high-order characteristic diagram
Figure BDA0003169020430000043
The feature vector acquisition module is used for acquiring the high-order feature map
Figure BDA0003169020430000044
Processing to obtain a feature vector f1 w...fn wAnd f1 u...fn u；
The component visibility calculation module is to calculate a component visibility based on the pose graph Pw、PuCalculating the feature vector f1 u...fn u(ii) visibility of;
the network training supervision module is used for acting on the feature vector f by using classification loss and distance loss1 w...fn wAnd f1 u...fn uThereby enabling supervision of network training.
Compared with the prior art, the embodiment of the invention has at least one of the following advantages:
(1) according to the serious misalignment pedestrian re-identification method and system based on the normalization network, the aligned characteristic diagram is obtained through the generation mode, and the variable part positions are positioned in the non-segmentation mode, so that the robustness under the complex environment is enhanced;
(2) according to the serious non-alignment pedestrian re-identification method and system based on the normalization network, the network can be trained through the alignment/non-alignment image pair subjected to random cutting processing, and an accurate human body segmentation data label is not needed;
(3) the serious misalignment pedestrian re-identification method and system based on the normalization network, provided by the invention, are used for feature filtering and alignment through the space reconstruction operation of the attitude guide generator, so that the space dependence among modules is reduced; and the credibility of different generation areas is distinguished through visibility calculation;
(4) according to the serious non-alignment pedestrian re-recognition method and system based on the normalization network, provided by the invention, the training process does not depend on the attitude point information excessively, the attitude point is higher in tolerance, and the training process does not need additional segmentation information to participate in training.
Drawings
Embodiments of the invention are further described below with reference to the accompanying drawings:
FIG. 1 is a flowchart of a method for re-identifying severely misaligned pedestrians based on a normalization network according to an embodiment of the present invention;
FIG. 2 is a block diagram of a preferred embodiment of a gesture guidance generator;
fig. 3 is a schematic diagram illustrating a specific framework and a generation manner of the spatial recomposition module according to a preferred embodiment of the invention.
Detailed Description
The following examples are given for the detailed implementation and specific operation of the present invention, but the scope of the present invention is not limited to the following examples.
Fig. 1 is a flowchart illustrating a method for re-identifying severely misaligned pedestrians based on a normalization network according to an embodiment of the present invention.
Referring to fig. 1, the method for re-identifying severely misaligned pedestrians based on a normalization network of the present embodiment includes:
the method for collecting and primarily processing the paired data specifically comprises the following steps:
s11: according to each input image IwCollecting corresponding randomly clipped non-aligned images I with the same identityu；
In particular, from the input image IwIn the same data set, other images of the person are collected and randomly cut to obtain an image Iu；
S12: for input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w；
S13 for input image Iw、IuProcessing and acquiring a posture graph Pw、Pu；
Applying supervision to the gesture generator, specifically comprising:
s14: using the gesture guidance generator, from the input feature map Fi uAnd attitude map Pw、PuGenerating a feature map
Figure BDA0003169020430000051
S15: using feature maps Fi wAnd
Figure BDA0003169020430000052
training the posture guide generator and the posture guide discriminator;
applying supervision to the network end, specifically comprising:
s16: for feature map Fi u、Fi wProcessing and acquiring high-order characteristic diagram
Figure BDA0003169020430000053
S17: for high-order characteristic diagram
Figure BDA0003169020430000054
The treatment is carried out, and the treatment is carried out,obtain the feature vector f1 w...fn wAnd f1 u...fn u；
S18: based on the attitude map Pw、PuCalculating the feature vector f1 u...fn u(ii) visibility of;
s19: applying classification loss and distance loss to feature vector f1 w...fn wAnd f1 u...fn uThereby enabling supervision of network training.
According to the embodiment of the invention, a mode of dynamically segmenting the feature map to align the parts is not adopted, but the stability of the whole method is combined, and the aligned feature map is generated for the similarity measurement, so that the robustness under a complex situation is enhanced.
In one embodiment, S12 includes: using a convolutional neural network to the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w. Preferably, the convolutional neural network in S12 is a ResNet50 top three-layer network. The selection is beneficial to considering the corresponding relation between the feature expression capability and the position of the spatial feature map in the subsequent feature map comparison process.
In one embodiment, S13 includes: using attitude estimation algorithm to input image Iw、IuProcessing and acquiring a posture graph Pw、Pu. Preferably, the pose estimation algorithm is openpos.
In a preferred embodiment, the gesture guidance generator in S14 includes: and T blocks, as shown in fig. 2, are a specific framework and a schematic diagram of a generation manner of the gesture guidance generator in an embodiment. Each block includes a spatial recomposition module, and fig. 3 is a schematic diagram illustrating a specific framework and a generation manner of the spatial recomposition module in an embodiment. In the attitude guidance generator, two-way characteristic modes of attitude point characteristics and images are used. In the spatial recombination module, the features of a certain height at each stage are obtained by weighting the features of all the heights from the previous stage according to independent weight coefficients. The operation breaks up the dependency relationship between the front and the rear stages in the height direction, and is more beneficial to the spatial recombination in the height direction.
In one embodiment, the training process in S15 applies to the L2 distance loss and discrimination loss. Preferably, the L2 distance loss is calculated by:
Figure BDA0003169020430000061
the L2 distance loss shortens the space distance of the same ID personnel image, and identifies the characteristic expression capability of the loss constraint to make the loss constraint meet the original classification characteristic.
In one embodiment, S16 includes: using a convolutional neural network, for the feature map Fi u、Fi wProcessing and acquiring high-order characteristic diagram
Figure BDA0003169020430000062
Preferably, the convolutional neural network in S16 is a ResNet50 layer four network. Similar to the selection in S12, this selection is beneficial to considering the feature expression ability and the spatial feature map position correspondence relationship in the subsequent feature map comparison process.
In one embodiment, S17 includes: using convolution and pooling operations on high-order feature maps
Figure BDA0003169020430000063
Processing to obtain a feature vector f1 w...fn wAnd f1 u...fn u. Preferably, the convolution is a convolution with a kernel size of 1 × 1, and the pooling operation is a global average pooling.
In the preferred embodiment, the classification penalty in S19 is a cross-entropy penalty, which is structured as follows:
Figure BDA0003169020430000071
wherein [ a, b ] is a visible interval in [1, n ], Class represents scaling of the feature vector to a classification scale through a full-connection layer network, and cross entropy represents cross entropy calculation. The classification loss constrains the feature expression ability to satisfy the original classification characteristics.
The distance loss shortens the space distance of the images of the persons with the same ID, and is as follows:
Figure BDA0003169020430000072
in a preferred embodiment, S19 is followed by:
s110: the network re-identification effectiveness after S19 was tested using CMC and mag as characterizations.
The technical solution in the above embodiment of the present invention is further described in detail below with reference to a specific example.
The images used in this particular example were from database Market1501 (see: Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian, "Scalable person re-identification: Abenchmark," in ICCV,2015, pp.1116-1124.) for anomaly detection performance evaluation.
As shown in Table 1, the method is a schematic diagram of the pedestrian re-identification result under the final serious misalignment based on the method of the above embodiment of the present invention, and CMC values (Rank-1, Rank-5, Rank-10) and mAP are used as the characteristics. Conventional systemic methods, such as PCB (see Yifang Sun, Huang Zheng, Yi Yang, Qi Tian, and Shengjin Wang, "Beyond and part models: Person regenerative with regenerative partial discharge (and a linear coherent bank)," in ECCV,2018, pp.480-496 "), MGN (see Guianshu Wang, Yoefeng Yuan, Xiong Chen, Jiweli, and Xi Zhou," guiding discrete reactive feeds with multiple nuclear discharge-identification, "in ACM, 2018, 274-282 ], st-Reid (see Guingcon, Jian Yang Cheng, Lang Huang, variance, ye Yuan, Wuyang Chen, Yang Yang Yang, Zhou Ren, and Zhang Yang Wang, "Abd-net: attention but not both sides of the person who is walking, in ICCV,2019, pp.8351-8361.), resolution adaptive algorithm (see Lingxiao He, Jian Liang, Haiqing Li, and Zhang Sun," Deep specific feature acquisition for partial person re-identification: Alignment-free adaptation, "in CVPR,2018, pp.7073-7082)," Lingxiao He, Zhang Sun, Yuha Zhu, and Yunbo, "recording medicine for diagnosis, Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang Yang, Yang Ren, Zhang Yang Ren, Yang Yang Xin, Yang Zhu, and Yang for strengthening, Yang Yang Yang Yang Yang Yang Yang, Yang Yang, Yang Yigan, and Yang Yigan, J-Pose, J-Yang Gui-Pose, J-Yang-Pose, J-P3, J-Pose, J-Yigan, J-Pose, ping Liu, Yuhang Ding, and Yi Yang, "position-defined feature alignment for enclosed person re-identification," in ICCV,2019, pp.542-551 "), posture guidance part pedestrian re-identification method (see Shang Gao, Jingya Wang, Huchuan Lu, and Zimo Liu," position-defined visual feature alignment for enclosed person re, "in CVPR,2020, pp.11744-11752.), (see Guan' Wang, Shuo Yang, Huang Liu, Zhing Wang, Yang Yang Yang Yang Yang, Shuliang Wang, Gang Yu, Erjin Zhou, and Jian Sun, and" High-order features: laser alignment, and CVJ found in the invention, 6449, which is more specific than the other methods.
Table 1 comparison of the performance of the method proposed by the present invention with existing algorithms
Figure BDA0003169020430000081
Figure BDA0003169020430000091
In another embodiment of the present invention, a system for re-identifying severely misaligned pedestrians based on a normalized network is further provided, which is used to implement the method for re-identifying severely misaligned pedestrians based on a normalized network of the foregoing embodiment, and includes: the device comprises a data collection module, a feature map acquisition module, a posture guide generator, a posture guide training module, a high-order feature acquisition module, a feature vector acquisition module, a component visibility calculation module and a network training supervision module.
Wherein the data collecting module is used for collecting the data according to each input image IwCollecting corresponding randomly clipped non-aligned images I with the same identityu；
The characteristic diagram acquisition module is used for inputting an image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w；
The attitude map acquisition module is used for acquiring an input image Iw、IuProcessing and acquiring a posture graph Pw、Pu；
The gesture guidance generator is used for utilizing the gesture guidance generator according to the input feature diagram Fi uAnd attitude map Pw、PuGenerating a feature map
Figure BDA0003169020430000092
The posture guiding training module is used for utilizing the feature map Fi wAnd
Figure BDA0003169020430000093
training the posture guide generator and the posture guide discriminator;
the high-order characteristic acquisition module is used for acquiring a characteristic diagram Fi u、Fi wProcessing and acquiring a high-order characteristic diagram Fo u、Fo w；
The feature vector acquisition module is used for carrying out high-order feature map Fo u、Fo wProcessing to obtain a feature vector f1 w...fn wAnd
Figure BDA0003169020430000094
the component visibility calculation module is used for calculating the visibility of the component based on the attitude map Pw、PuComputing feature vectors
Figure BDA0003169020430000095
(ii) visibility of;
the network training supervision module is used for acting on the feature vector f by using classification loss and distance loss1 w...fn wAnd
Figure BDA0003169020430000096
thereby enabling supervision of network training.
According to the severe misalignment pedestrian re-identification method and system based on the normalized network, the low-order alignment feature map is generated, and cross-image supervision and pre-training integral model supervision is used for carrying out antagonistic training on the network, so that the universality and the robustness of the network are improved; the network can be trained by randomly clipping the aligned/non-aligned image pairs without the need for accurate human segmentation data labels.
In the above embodiments of the present invention, the description of each embodiment has a respective emphasis, and reference may be made to related descriptions of other embodiments for parts that are not described in detail in a certain embodiment.
In the embodiments provided in the present invention, it should be understood that the disclosed technical contents can be implemented in other manners. The above-described embodiments of the system are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed, another point, and the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software function and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to execute all or part of the steps of the method according to the embodiments of the present invention. The second aforementioned storage medium includes: u disk, ROM, RAM, removable hard disk, magnetic or optical disk, etc. for storing program codes.
The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, and not to limit the invention. Any modifications and variations within the scope of the description, which may occur to those skilled in the art, are intended to be within the scope of the invention.

Claims (10)

1. A serious misalignment pedestrian re-identification method based on a normalization network is characterized by comprising the following steps:
s11: according to each input image IwCollecting corresponding randomly cropped non-aligned images of the same identityIu；
S12: for the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w；
S13: for the input image Iw、IuProcessing and acquiring a posture graph Pw、Pu；
S14: using the gesture guidance generator, from the input feature map Fi uAnd attitude map Pw、PuGenerating a feature map
Figure FDA0003169020420000011
S15: using feature maps Fi wAnd
Figure FDA0003169020420000012
training the posture guide generator and the posture guide discriminator;
s16: for the characteristic diagram Fi u、Fi wProcessing and acquiring high-order characteristic diagram
Figure FDA0003169020420000013
S17: for the high-order characteristic diagram
Figure FDA0003169020420000014
Processing to obtain feature vector
Figure FDA0003169020420000015
And
Figure FDA0003169020420000016
s18: based on the attitude map Pw、PuCalculating the feature vector
Figure FDA0003169020420000017
Can seeSex;
s19: acting on the feature vector with classification loss and distance loss
Figure FDA0003169020420000018
And
Figure FDA0003169020420000019
thereby enabling supervision of network training.
2. The method for re-identifying severely misaligned pedestrians based on a normalized network as claimed in claim 1, wherein the S12 includes: using a convolutional neural network to the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w(ii) a The convolutional neural network is a ResNet50 top three-layer network.
3. The method for re-identifying severely misaligned pedestrians based on a normalized network as claimed in claim 1, wherein the S13 includes: using an attitude estimation algorithm on the input image Iw、IuProcessing and acquiring a posture graph Pw、Pu(ii) a The attitude estimation algorithm is OpenPose.
4. The normalization network based severe misalignment pedestrian re-recognition method according to claim 1, wherein the gesture guidance generator in S14 comprises: t blocks, wherein each block contains a spatial recomposition operation.
5. The method of claim 1, wherein the training in S15 applies to L2 distance loss and discrimination loss; the calculation method of the L2 distance loss is as follows:
Figure FDA00031690204200000110
6. the method for re-identifying severely misaligned pedestrians based on a normalized network as claimed in claim 1, wherein the S16 includes: using a convolutional neural network to the feature map Fi u、Fi wProcessing and acquiring high-order characteristic diagram
Figure FDA0003169020420000021
The convolutional neural network in S16 is a ResNet50 layer four network.
7. The method for re-identifying severely misaligned pedestrians based on a normalized network as claimed in claim 1, wherein the S17 includes: using convolution and pooling operations on the high-order feature map
Figure FDA0003169020420000022
Processing to obtain feature vector
Figure FDA0003169020420000023
And
Figure FDA0003169020420000024
the convolution is a convolution with a kernel size of 1 × 1, and the pooling operation is global average pooling.
8. The method of claim 1, wherein the classification penalty in S19 is a cross entropy penalty as follows:
Figure FDA0003169020420000025
wherein [ a, b ] is a visible interval in [1, n ], Class represents that the scale of the feature vector is scaled to a classification scale through a full-connection layer network, and cross entry represents cross entropy calculation;
the distance loss is:
Figure FDA0003169020420000026
9. the method for re-identifying severely misaligned pedestrians based on a normalized network as claimed in claim 1, wherein said S19 is followed by further comprising:
s110: the network re-identification effectiveness after S19 was tested using CMC and mag as characterizations.
10. A normalization network based severe misalignment pedestrian re-identification system, comprising: the system comprises a data collection module, a feature map acquisition module, a posture guide generator, a posture guide training module, a high-order feature acquisition module, a feature vector acquisition module, a component visibility calculation module and a network training supervision module; wherein the content of the first and second substances,
the data gathering module is used for collecting I according to each input imagewCollecting corresponding randomly clipped non-aligned images I with the same identityu；
The characteristic diagram acquisition module is used for acquiring the input image Iw、IuProcessing and obtaining a characteristic diagram Fi u、Fi w；
The attitude map acquisition module is used for acquiring the input image Iw、IuProcessing and acquiring a posture graph Pw、Pu；
The gesture guidance generator is used for utilizing the gesture guidance generator to generate the input feature diagram Fi uAnd attitude map Pw、PuGenerating a feature map
Figure FDA0003169020420000031
The posture guide training module is used for utilizing a feature map Fi wAnd
Figure FDA0003169020420000032
training the posture guide generator and the posture guide discriminator;
the high-order characteristic acquisition module is used for acquiring the characteristic diagram Fi u、Fi wProcessing and acquiring high-order characteristic diagram
Figure FDA0003169020420000033
The feature vector acquisition module is used for acquiring the high-order feature map
Figure FDA0003169020420000034
Processing to obtain feature vector
Figure FDA0003169020420000035
And
Figure FDA0003169020420000036
the component visibility calculation module is to calculate a component visibility based on the pose graph Pw、PuCalculating the feature vector
Figure FDA0003169020420000037
(ii) visibility of;
the network training supervision module is used for acting on the feature vector by using classification loss and distance loss
Figure FDA0003169020420000038
And
Figure FDA0003169020420000039
thereby enabling supervision of network training.
CN202110812700.0A 2021-07-19 2021-07-19 Serious misalignment pedestrian re-identification method and system based on normalization network Active CN113627272B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110812700.0A CN113627272B (en) 2021-07-19 2021-07-19 Serious misalignment pedestrian re-identification method and system based on normalization network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110812700.0A CN113627272B (en) 2021-07-19 2021-07-19 Serious misalignment pedestrian re-identification method and system based on normalization network

Publications (2)

Publication Number Publication Date
CN113627272A true CN113627272A (en) 2021-11-09
CN113627272B CN113627272B (en) 2023-11-28

Family

ID=78380050

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110812700.0A Active CN113627272B (en) 2021-07-19 2021-07-19 Serious misalignment pedestrian re-identification method and system based on normalization network

Country Status (1)

Country Link
CN (1) CN113627272B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110135366A (en) * 2019-05-20 2019-08-16 厦门大学 Pedestrian's recognition methods again is blocked based on multiple dimensioned generation confrontation network
CN111783736A (en) * 2020-07-23 2020-10-16 上海高重信息科技有限公司 Pedestrian re-identification method, device and system based on human body semantic alignment
CN112101318A (en) * 2020-11-17 2020-12-18 深圳市优必选科技股份有限公司 Image processing method, device, equipment and medium based on neural network model
CN112200111A (en) * 2020-10-19 2021-01-08 厦门大学 Global and local feature fused occlusion robust pedestrian re-identification method
CN112434654A (en) * 2020-12-07 2021-03-02 安徽大学 Cross-modal pedestrian re-identification method based on symmetric convolutional neural network
WO2021092600A2 (en) * 2020-12-14 2021-05-14 Innopeak Technology, Inc. Pose-over-parts network for multi-person pose estimation
US20210150268A1 (en) * 2017-07-13 2021-05-20 Peking University Shenzhen Graduate School Method of using deep discriminate network model for person re-identification in image or video

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210150268A1 (en) * 2017-07-13 2021-05-20 Peking University Shenzhen Graduate School Method of using deep discriminate network model for person re-identification in image or video
CN110135366A (en) * 2019-05-20 2019-08-16 厦门大学 Pedestrian's recognition methods again is blocked based on multiple dimensioned generation confrontation network
CN111783736A (en) * 2020-07-23 2020-10-16 上海高重信息科技有限公司 Pedestrian re-identification method, device and system based on human body semantic alignment
CN112200111A (en) * 2020-10-19 2021-01-08 厦门大学 Global and local feature fused occlusion robust pedestrian re-identification method
CN112101318A (en) * 2020-11-17 2020-12-18 深圳市优必选科技股份有限公司 Image processing method, device, equipment and medium based on neural network model
CN112434654A (en) * 2020-12-07 2021-03-02 安徽大学 Cross-modal pedestrian re-identification method based on symmetric convolutional neural network
WO2021092600A2 (en) * 2020-12-14 2021-05-14 Innopeak Technology, Inc. Pose-over-parts network for multi-person pose estimation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
程杨: "基于深度混合卷积神经网络的行人重识别研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 1 - 69 *
范楼苗: "基于图注意力网络的群组行为识别", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 1 - 49 *

Also Published As

Publication number Publication date
CN113627272B (en) 2023-11-28

Similar Documents

Publication Publication Date Title
Vieira et al. Stop: Space-time occupancy patterns for 3d action recognition from depth map sequences
Gou et al. Cascade learning from adversarial synthetic images for accurate pupil detection
CN111241989A (en) Image recognition method and device and electronic equipment
WO2013091370A1 (en) Human body part detection method based on parallel statistics learning of 3d depth image information
CN108875586B (en) Functional limb rehabilitation training detection method based on depth image and skeleton data multi-feature fusion
Yang et al. Facial expression recognition based on dual-feature fusion and improved random forest classifier
Liu et al. Attentive cross-modal fusion network for RGB-D saliency detection
Liu et al. A Fusion Face Recognition Approach Based on 7‐Layer Deep Learning Neural Network
CN112668550A (en) Double-person interaction behavior recognition method based on joint point-depth joint attention RGB modal data
CN115018999A (en) Multi-robot-cooperation dense point cloud map construction method and device
Li et al. Human action recognition based on 3D body mask and depth spatial-temporal maps
Lu et al. JHPFA-Net: Joint head pose and facial action network for driver yawning detection across arbitrary poses in videos
Ming et al. A unified 3D face authentication framework based on robust local mesh SIFT feature
Xu et al. A novel method for hand posture recognition based on depth information descriptor
Prakash False mapped feature removal in spin images based 3D ear recognition
Zhao et al. Human pose regression through multiview visual fusion
CN113627272A (en) Serious misalignment pedestrian re-identification method and system based on normalization network
Schneider Visual hull
CN113239861B (en) Method for determining head motion of driver, storage medium, and electronic device
Lin et al. Human action recognition using motion history image based temporal segmentation
Yu et al. Mental fatigue testing based on deep learning
Said et al. Wavelet networks for facial emotion recognition
Colombo et al. Face^ 3 a 2D+ 3D Robust Face Recognition System
Dai et al. Iris center localization using energy map synthesis based on gradient and isophote
CN113705304A (en) Image processing method and device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant