CN114445852A - Brain-inspired long-duration continuous pedestrian re-identification method and device - Google Patents
Brain-inspired long-duration continuous pedestrian re-identification method and device Download PDFInfo
- Publication number
- CN114445852A CN114445852A CN202111588641.XA CN202111588641A CN114445852A CN 114445852 A CN114445852 A CN 114445852A CN 202111588641 A CN202111588641 A CN 202111588641A CN 114445852 A CN114445852 A CN 114445852A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- parameter
- model
- value
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000006870 function Effects 0.000 claims abstract description 78
- 230000015654 memory Effects 0.000 claims abstract description 68
- 238000012549 training Methods 0.000 claims abstract description 44
- 238000013528 artificial neural network Methods 0.000 claims abstract description 10
- 210000004556 brain Anatomy 0.000 claims description 18
- 230000007774 longterm Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 230000035945 sensitivity Effects 0.000 claims description 7
- 239000000126 substance Substances 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000037007 arousal Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000036992 cognitive tasks Effects 0.000 description 1
- 238000007596 consolidation process Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005056 memory consolidation Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a brain-inspired long-duration continuous pedestrian re-identification method and device, wherein the method comprises the following steps: training the deep neural network by using initial training data to obtain an initial pedestrian re-recognition model; extracting scene memory buffer data in the initial training data, and acquiring preset parameters in an initial pedestrian re-identification model; retraining the initial pedestrian re-recognition model according to the newly added training data and the scene memory buffer data, and simultaneously constraining the preset parameters so that the preset parameters meet constraint conditions; and calculating a loss function of the retrained initial pedestrian re-recognition model, and iteratively updating the scene memory buffer data and the preset parameters according to the loss function until the current pedestrian re-recognition model meets an iteration termination condition to obtain a final pedestrian re-recognition model. The neural network model parameter updating method and device can update the neural network model parameters without storing a large amount of old data, and maintain the performance of the model on the old data while adapting to the new data.
Description
Technical Field
The application relates to the technical field of pedestrian re-identification of deep learning, in particular to a brain-inspired long-duration continuous pedestrian re-identification method and device.
Background
The pedestrian re-identification technology is a computer vision technology, and aims to search data shot by different cameras for pedestrians with specified identities, namely, images with similar characteristics are selected from a pedestrian candidate image library shot by a plurality of different cameras according to inquired pedestrian images with specific identities. The pedestrian re-identification technology plays a crucial role in the fields of intelligent monitoring, security and the like.
The deep learning method is widely applied to the field of pedestrian re-identification, and a general process is that a pedestrian image to be inquired is input into a deep neural network model, a feature vector of the pedestrian image is extracted, similarity calculation is carried out on the feature vector and a feature vector of a candidate pedestrian image library, and an image with high similarity is used as a retrieval result with the same identity as the inquired pedestrian image. At present, the pedestrian re-identification method based on deep learning has high performance on a fixed pedestrian re-identification data set, however, in a real scene, pedestrian data is likely to be continuously increased and the distribution of the pedestrian data is continuously changed, and obviously, the precondition of data set fixation is not satisfied. If the added data has large domain differences from the previous data, severe performance degradation may occur for models trained on the previous data set.
Most of the existing pedestrian re-recognition systems are static, that is, new information is not learned after training on a fixed data set, and the existing pedestrian re-recognition systems cannot be directly applied to new data with great style difference with training data. One possible idea for solving this problem is to add new data to the training set and then retrain the model, which is impractical, because the camera is constantly collecting a large amount of new data, if the model is trained from the beginning each time, it will cause serious waste of computing resources and time, and simultaneously storing all the collected data will occupy a large amount of storage resources, and because of privacy protection in the field of pedestrian re-recognition, the old data set may not be available when the model needs to be updated. On the other hand, the current deep learning method for solving the cross-domain pedestrian re-identification problem has a catastrophic forgetting phenomenon, which is also a general problem of a deep neural network, namely, the learning of a new task changes parameters of a neural network model, and interferes with a previously learned task, so that the performance of the model on an old task is seriously reduced.
Continuous learning is a new direction of machine learning, aiming to alleviate the catastrophic forgetting phenomenon, i.e. to be able to retain the knowledge of old data when learning information of new data. If a pedestrian re-recognition system is expected to be effective in a real scene for a long time, the pedestrian re-recognition system must have the capability of continuous learning, and the continuous learning problem in the field of pedestrian re-recognition is not fully researched at present.
Disclosure of Invention
The application provides a brain-inspired long-duration continuous pedestrian re-identification method and device, and aims to solve the problem that in the related art, the pedestrian re-identification method based on deep learning has catastrophic forgetting when new data is continuously learned.
The embodiment of the first aspect of the application provides a brain-inspired long-duration continuous pedestrian re-identification method, which comprises the following steps: s1, training the deep neural network by using the initial training data to obtain an initial pedestrian re-recognition model; s2, extracting scene memory buffer data in the initial training data, and acquiring preset parameters in the initial pedestrian re-recognition model; s3, retraining the initial pedestrian re-recognition model according to newly added training data and the scene memory buffer data, and simultaneously constraining the preset parameters to enable the preset parameters to meet constraint conditions; s4, calculating a loss function of the retrained initial pedestrian re-identification model, updating the scene memory buffer data and the preset parameters according to the loss function, and executing S3 until the current pedestrian re-identification model meets an iteration termination condition to obtain a final pedestrian re-identification model.
According to an embodiment of the application, the obtaining of the preset parameter in the initial pedestrian re-identification model includes: calculating the sensitivity of the model parameters according to the pedestrian feature function output by the initial pedestrian re-identification model to obtain the parameter importance values of a plurality of parameters in the initial pedestrian re-identification model, and taking the model parameters with the parameter importance values larger than a preset threshold value as the preset parameters.
According to an embodiment of the present application, the calculation formula of the parameter importance value is:
wherein omegaiFor the parameter importance value, θ represents the current parameter value of the model, N represents the number of samples of the input model between two updating parameter importance, xjDenotes the jth pedestrian sample, F (x)jTheta) represents the pedestrian characteristics of the jth sample extracted by the current model, and represents the square L2And (4) norm.
According to an embodiment of the present application, the calculation formula of the loss function is:
wherein the content of the first and second substances,
a loss function representing an initial pedestrian re-identification model,
the parameter theta representing the importance of the parameter when the preset parameter is updated last timeiValue of (A)1And λ2Are tradeoffs of parameters.
According to an embodiment of the present application, the constraint condition of the preset parameter includes: and the difference value between the parameter value after updating the preset parameter and the parameter value before updating is smaller than the preset difference value.
According to an embodiment of the present application, the updating the scene memory buffer data and the preset parameter according to the loss function includes: detecting the mean value and the variance of the loss function by using a sliding window, and updating the preset parameters when the mean value of the loss function is greater than a first threshold value and the mean value and the variance are both less than a second threshold value; and updating the scene memory buffer data according to the size of the loss function value.
According to an embodiment of the application, the iteration termination condition comprises: the current iteration round reaches a preset round; or the current loss function value is smaller than a preset error value.
The second aspect embodiment of the present application provides a long-term continuous pedestrian re-identification device with brain inspiration, including: the pre-training module is used for training the deep neural network by using initial training data to obtain an initial pedestrian re-recognition model; the processing module is used for extracting the scene memory buffer data in the initial training data and acquiring preset parameters in the initial pedestrian re-recognition model; the retraining module is used for retraining the initial pedestrian re-recognition model according to newly-added training data and the scene memory buffer data and simultaneously constraining the preset parameters so that the preset parameters meet constraint conditions; and the iteration module is used for calculating a loss function of the retrained initial pedestrian re-identification model, updating the scene memory buffer data and the preset parameters according to the loss function, and training by using the retraining module until the current pedestrian re-identification model meets an iteration termination condition to obtain a final pedestrian re-identification model.
According to an embodiment of the application, the processing module is specifically configured to calculate sensitivity of a model parameter according to a pedestrian feature function output by the initial pedestrian re-identification model to obtain parameter importance values of multiple parameters in the initial pedestrian re-identification model, and use the model parameter of which the parameter importance value is greater than a preset threshold as the preset parameter.
According to an embodiment of the present application, the calculation formula of the parameter importance value is:
wherein omegaiFor the parameter importance value, theta represents the current parameter value of the model, N represents the number of samples of the input model between two updating parameter importance values, xjDenotes the jth pedestrian sample, F (x)jTheta) represents the pedestrian characteristics of the jth sample extracted by the current model, and represents the square L2And (4) norm.
According to an embodiment of the present application, the calculation formula of the loss function is:
wherein the content of the first and second substances,
a loss function representing an initial pedestrian re-identification model,
the parameter theta representing the importance of the parameter when the preset parameter is updated last timeiValue of (A)1And λ2Are tradeoffs of parameters.
According to an embodiment of the present application, the constraint condition of the preset parameter includes: and the difference value between the parameter value after updating the preset parameter and the parameter value before updating is smaller than the preset difference value.
According to an embodiment of the present application, the updating the scene memory buffer data and the preset parameter according to the loss function includes: detecting the mean value and the variance of the loss function by using a sliding window, and updating the preset parameters when the mean value of the loss function is greater than a first threshold value and the mean value and the variance are both less than a second threshold value; and updating the scene memory buffer data according to the size of the loss function value.
According to an embodiment of the application, the iteration termination condition comprises: the current iteration round reaches a preset round; or the current loss function value is smaller than a preset error value.
An embodiment of a third aspect of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the brain-inspired long-duration continuous pedestrian re-identification method as described in the above embodiments.
A fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program, where the computer program is executed by a processor to implement the brain-inspired long-term continuous pedestrian re-identification method according to the foregoing embodiment.
The brain-inspired long-duration continuous pedestrian re-identification method and device have the following beneficial effects:
1) the continuous learning method inspired by brain science is applied to the pedestrian re-identification technology, so that the catastrophic forgetting problem of a pedestrian re-identification system is relieved;
2) by maintaining a scene memory buffer with limited capacity for storing difficult samples, the knowledge of the model about old data can be kept without storing a large amount of old data;
3) and (3) considering the importance of model parameters to old data when updating the model, and balancing the stability and plasticity of the model.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a flowchart of a brain-inspired long-duration continuous pedestrian re-identification method according to an embodiment of the present application;
fig. 2 is an exemplary diagram of a brain-inspired long-duration continuous pedestrian re-identification apparatus according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
Fig. 1 is a flowchart of a brain-inspired long-duration continuous pedestrian re-identification method according to an embodiment of the present application.
As shown in fig. 1, the brain-inspired long-duration continuous pedestrian re-identification method includes the following steps:
in step S1, the deep neural network is trained using the initial training data to obtain an initial pedestrian re-identification model.
Specifically, the existing training data is input into a deep neural network, and an initial pedestrian re-recognition model is obtained by training with a mainstream pedestrian re-recognition algorithm.
In step S2, the scene memory buffer data in the initial training data is extracted, and the preset parameters in the initial pedestrian re-identification model are obtained.
It is understood that the long-term memory of the human brain can be divided into semantic memory and contextual memory, the semantic memory relates to abstract concepts and knowledge, and can be analogous to features deep in a neural network, and the contextual memory contains richer detailed information, such as images input into the network. In addition, in the brains of some living beings, neurons involved in memory can spontaneously replay past contextual memory, thereby performing a memory consolidation process. Based on the above teaching, the method is provided with a fixed-capacity scenario memory buffer area in the model training process, the fixed-capacity scenario memory buffer area is used for storing the difficult sample images in the old data, and the samples in the buffer area are replayed in the subsequent model updating process, namely are input into the pedestrian re-recognition network as part of the training set. Replaying these images with detailed information in the network facilitates the consolidation of the knowledge of the model about old data.
In one embodiment of the present application, obtaining preset parameters in an initial pedestrian re-identification model includes: calculating the sensitivity of the model parameters according to the pedestrian characteristic function output by the initial pedestrian re-identification model to obtain parameter importance values of a plurality of parameters in the initial pedestrian re-identification model, and taking the model parameters with the parameter importance values larger than a preset threshold value as preset parameters.
Specifically, the importance of the parameters of the model refers to the importance of each parameter for maintaining the knowledge of the old data, and the important parameters are constrained to be close to the old value when the new data is trained later, so that the model is prevented from forgetting the old knowledge.
In one embodiment of the present application, the parameter θiImportance of omegaiAnd calculating the sensitivity of the pedestrian characteristic function output by the model to the parameter, wherein the calculation formula of the parameter importance value is as follows:
wherein omegaiFor the parameter importance value, theta represents the current parameter value of the model, N represents the number of samples of the input model between two updating parameter importance values, xjDenotes the jth pedestrian sample, F (x)jTheta) represents the pedestrian characteristics of the jth sample extracted by the current model, and represents the square L2And (4) norm.
In a specific implementation process, the current Ω may be calculated by using an accumulative moving average method each time the importance of the parameter is updatediThe value is obtained.
In step S3, the initial pedestrian re-recognition model is retrained based on the newly added training data and the scene memory buffer data, and the preset parameters are constrained such that the preset parameters satisfy the constraint conditions.
Specifically, in order to enable the model to learn information of new data and simultaneously consolidate memory of old data, the new data collected by the camera and the old data in the scene memory buffer are input into the initial pedestrian re-recognition model as a training set for retraining when the model is updated.
In one embodiment of the present application, the constraint condition of the preset parameter includes: and the difference value between the parameter value after updating the preset parameter and the parameter value before updating is smaller than the preset difference value.
In the retraining process, the updated values of the selected preset parameter values are limited to be close to the old parameters, so that the knowledge of the model about the old data is kept, and the catastrophic forgetting is avoided.
It should be noted that the size of the preset difference may be adjusted according to actual situations, and is not particularly limited.
In step S4, a loss function of the retrained initial pedestrian re-identification model is calculated, the scene memory buffer data and the preset parameters are updated according to the loss function, and S3 is executed until the current pedestrian re-identification model meets the iteration termination condition, so as to obtain a final pedestrian re-identification model.
In an embodiment of the present application, a parameter importance regularization term is added to the loss function, and a calculation formula of the loss function is:
wherein, therein
And represents the loss functions in the original pedestrian re-identification algorithm, such as identity classification loss and triple loss,
the parameter theta representing the importance of the last time the parameter was updated at the present timeiValue of (A)1And λ2For balancing the plasticity and stability of the parameters.
Specifically, the last item of the loss function refers to a memory allocation mechanism of the brain, neurons in the brain have different possibilities of forming memory, and neurons with stronger activation are more likely to participate in subsequent memory, so that memories at different times are related, and new knowledge is better acquired by using past experience. Two regular terms in the loss function can play a role in constraining important or large-absolute-value parameters during training and limit the important or large-absolute-value parameters to be close to old parameter values, so that the knowledge of the model about old data is kept, and catastrophic forgetting is avoided.
In an embodiment of the present application, updating the scene memory buffer data and the preset parameters according to the loss function includes: detecting the mean value and the variance of the loss function by using a sliding window, and updating the preset parameters when the mean value of the loss function is greater than a first threshold value and the mean value and the variance are both less than a second threshold value; and updating the scene memory buffer data according to the size of the loss function value.
Specifically, pedestrian data in a real scene are continuous, the moment when the data change in the field often cannot be accurately judged, and the importance of the parameters is determined when to be updated according to the change condition of the loss function in the model in the training process of the application. Generally, when the model is trained on data with little difference in the field, the loss function presents a downward trend on the whole, which indicates that the model has learned knowledge related to the data, and some difficult samples with big difference in style from the previous data or containing new pedestrian identities may increase the value of the loss function, which indicates that the model needs to learn new information. The human brain has a post-error slowing effect in performing cognitive tasks, i.e., the response is significantly prolonged upon the occurrence of an error. Similarly, in the present application, after the neural network model has a large error, a phase is needed to consolidate the newly learned knowledge and then to continue learning new data. In the application, the sliding window is used for detecting the mean and the variance of the loss function, when the mean of the loss function exceeds a set threshold, it indicates that the importance of the parameter needs to be updated later, and the updating time is when the model learns the information of the hard samples after the loss function is regressed to be stable, namely, the mean and the variance in the sliding window are smaller than the set threshold.
The scene memory buffer area in the application does not store all old data, but selects a fixed number of difficult samples with the maximum loss function value from the data and the new data of the current buffer area according to the size of the loss function value of the model, and continuously updates the data in the buffer area with the limited sample capacity. During the subsequent training process, the samples in the scene memory buffer are replayed, i.e. input into the model as a training set together with new data, so as to consolidate the knowledge of the model for old data under the condition of low storage requirement.
In one embodiment of the present application, the iteration termination condition includes: the current iteration round reaches a preset round; or the current loss function value is smaller than a preset error value.
In a practical scenario where the model needs to be deployed for a long time, in the face of a continuous new data flow, repeating steps S3 and S4 can obtain a long-term continuous pedestrian re-identification model with continuous learning capability suitable for old data and new data.
According to the long-term continuous pedestrian re-identification method based on brain elicitation provided by the embodiment of the application, a long-term continuous pedestrian re-identification technology based on brain elicitation is provided by taking the brain science theory as reference, parameters of a neural network model can be updated under the condition that a large amount of old data does not need to be stored, and the performance of the model on the old data is maintained while the new data is adapted.
Next, a long-term continuous pedestrian re-recognition apparatus of brain elicitation proposed according to an embodiment of the present application is described with reference to the drawings.
Fig. 2 is a block diagram of a brain-inspired long-duration continuous pedestrian re-identification apparatus according to an embodiment of the present application.
As shown in fig. 2, the brain-inspired long-duration continuous pedestrian re-recognition device 10 includes: pre-training module 100, processing module 200, retraining module 300, and iteration module 400.
The pre-training module 100 is configured to train the deep neural network by using initial training data to obtain an initial pedestrian re-recognition model. And the processing module 200 is configured to extract the scene memory buffer data in the initial training data, and obtain a preset parameter in the initial pedestrian re-recognition model. And the retraining module 300 is configured to retrain the initial pedestrian re-recognition model according to the newly added training data and the context memory buffer data, and simultaneously constrain the preset parameters so that the preset parameters meet constraint conditions. The iteration module 400 is configured to calculate a loss function of the retrained initial pedestrian re-recognition model, update the scene memory buffer data and the preset parameters according to the loss function, and train by using the retraining module until the current pedestrian re-recognition model meets an iteration termination condition, so as to obtain a final pedestrian re-recognition model.
According to an embodiment of the application, the processing module is specifically configured to calculate sensitivity of a model parameter according to a pedestrian feature function output by the initial pedestrian re-identification model to obtain parameter importance values of a plurality of parameters in the initial pedestrian re-identification model, and use a model parameter of which the parameter importance value is greater than a preset threshold as a preset parameter.
According to one embodiment of the present application, the calculation formula of the parameter importance value is:
wherein omegaiFor the parameter importance value, theta represents the current parameter value of the model, N represents the number of samples of the input model between two updating parameter importance values, xjDenotes the jth pedestrian sample, F (x)jTheta) represents the pedestrian characteristics of the jth sample extracted by the current model, and represents the square L2And (4) norm.
According to one embodiment of the present application, the formula for the calculation of the loss function is:
wherein, the first and the second end of the pipe are connected with each other,
a loss function representing an initial pedestrian re-identification model,
parameter theta representing the importance of the parameter when the preset parameter was last updatediValue of (A)1And λ2Are tradeoffs of parameters.
According to one embodiment of the present application, the constraint condition of the preset parameter includes: and the difference value between the parameter value after updating the preset parameter and the parameter value before updating is smaller than the preset difference value.
According to an embodiment of the present application, updating the scene memory buffer data and the preset parameters according to the loss function includes: detecting the mean value and the variance of the loss function by using a sliding window, and updating the preset parameters when the mean value of the loss function is greater than a first threshold value and the mean value and the variance are both less than a second threshold value; and updating the scene memory buffer data according to the size of the loss function value.
According to one embodiment of the application, the iteration termination condition includes: the current iteration round reaches a preset round; or the current loss function value is smaller than a preset error value.
It should be noted that the explanation of the embodiment of the brain-inspired long-duration continuous pedestrian re-identification method is also applicable to the brain-inspired long-duration continuous pedestrian re-identification device of the embodiment, and is not repeated herein.
According to the long-term continuous pedestrian re-identification device for brain elicitation provided by the embodiment of the application, a long-term continuous pedestrian re-identification technology for brain elicitation is provided by taking the brain science theory as reference, parameters of a neural network model can be updated under the condition that a large amount of old data does not need to be stored, and the performance of the model on the old data is maintained while the new data is adapted.
Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
a memory 301, a processor 302, and a computer program stored on the memory 301 and executable on the processor 302.
The processor 302, when executing the program, implements the long-term continuous pedestrian re-identification method of brain elicitation provided in the above-described embodiments.
Further, the electronic device further includes:
a communication interface 303 for communication between the memory 301 and the processor 302.
A memory 301 for storing computer programs executable on the processor 302.
The memory 301 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 301, the processor 302 and the communication interface 303 are implemented independently, the communication interface 303, the memory 301 and the processor 302 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended EISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.
Optionally, in a specific implementation, if the memory 301, the processor 302, and the communication interface 303 are integrated on a chip, the memory 301, the processor 302, and the communication interface 303 may complete communication with each other through an internal interface.
The processor 302 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present Application.
The present embodiment also provides a computer-readable storage medium on which a computer program is stored, characterized in that the program, when executed by a processor, implements the long-term continuous pedestrian re-recognition method of brain arousal as above.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more N executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of implementing the embodiments of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
Claims (16)
1. A long-term continuous pedestrian re-identification method based on brain elicitation is characterized by comprising the following steps:
s1, training the deep neural network by using the initial training data to obtain an initial pedestrian re-recognition model;
s2, extracting scene memory buffer data in the initial training data, and acquiring preset parameters in the initial pedestrian re-recognition model;
s3, retraining the initial pedestrian re-recognition model according to newly added training data and the scene memory buffer data, and simultaneously constraining the preset parameters to enable the preset parameters to meet constraint conditions;
s4, calculating a loss function of the retrained initial pedestrian re-identification model, updating the scene memory buffer data and the preset parameters according to the loss function, and executing S3 until the current pedestrian re-identification model meets an iteration termination condition to obtain a final pedestrian re-identification model.
2. The method according to claim 1, wherein the obtaining of the preset parameters in the initial pedestrian re-identification model comprises:
and calculating the sensitivity of the pedestrian characteristic function output by the initial pedestrian re-identification model to the model parameters to obtain parameter importance values of a plurality of parameters in the initial pedestrian re-identification model, and taking the model parameters with the parameter importance values larger than a preset threshold value as the preset parameters.
3. The method of claim 2, wherein the parameter significance value is calculated by the formula:
wherein omegaiFor the parameter importance value, theta represents the current parameter value of the model, N represents the number of samples of the input model between two updating parameter importance values, xjDenotes the jth pedestrian sample, F (x)jTheta) represents the pedestrian characteristics of the jth sample extracted by the current model, and represents the square L2And (4) norm.
4. The method of claim 3, wherein the loss function is calculated by:
wherein the content of the first and second substances,
a loss function representing an initial pedestrian re-identification model,
the parameter theta representing the importance of the parameter when the preset parameter is updated last timeiValue of (A)1And λ2Are tradeoffs of parameters.
5. The method according to claim 1, wherein the constraint condition of the preset parameter comprises:
and the difference value between the parameter value after updating the preset parameter and the parameter value before updating is smaller than the preset difference value.
6. The method of claim 1, wherein the updating the contextual memory buffer data and the preset parameters according to the loss function comprises:
detecting the mean value and the variance of the loss function by using a sliding window, and updating the preset parameters when the mean value of the loss function is greater than a first threshold value and the mean value and the variance are both less than a second threshold value;
and updating the scene memory buffer data according to the size of the loss function value.
7. The method according to any of claims 1-6, wherein the iteration termination condition comprises:
the current iteration round reaches a preset round;
or the current loss function value is smaller than a preset error value.
8. A long-term continuous pedestrian re-identification device with brain inspiration, comprising:
the pre-training module is used for training the deep neural network by using initial training data to obtain an initial pedestrian re-recognition model;
the processing module is used for extracting the scene memory buffer data in the initial training data and acquiring preset parameters in the initial pedestrian re-recognition model;
the retraining module is used for retraining the initial pedestrian re-recognition model according to newly-added training data and the scene memory buffer data and simultaneously constraining the preset parameters so that the preset parameters meet constraint conditions;
and the iteration module is used for calculating a loss function of the retrained initial pedestrian re-identification model, updating the scene memory buffer data and the preset parameters according to the loss function, and training by using the retraining module until the current pedestrian re-identification model meets an iteration termination condition to obtain a final pedestrian re-identification model.
9. The apparatus according to claim 8, wherein the processing module is specifically configured to calculate, according to the sensitivity of the pedestrian feature function output by the initial pedestrian re-identification model to the model parameters, parameter importance values of a plurality of parameters in the initial pedestrian re-identification model, and use the model parameters whose parameter importance values are greater than a preset threshold as the preset parameters.
10. The apparatus of claim 9, wherein the formula for calculating the importance value of the parameter is:
wherein omegaiFor the parameter importance value, theta represents the current parameter value of the model, N represents the number of samples of the input model between two updating parameter importance values, xjDenotes the jth pedestrian sample, F (x)jTheta) represents the pedestrian characteristics of the jth sample extracted by the current model, and represents the square L2And (4) norm.
11. The apparatus of claim 10, wherein the loss function is calculated by:
wherein the content of the first and second substances,
a loss function representing an initial pedestrian re-identification model,
the parameter theta representing the importance of the parameter when the preset parameter is updated last timeiValue of (A)1And λ2Are tradeoffs of parameters.
12. The apparatus of claim 8, wherein the constraint condition of the preset parameter comprises:
and the difference value between the parameter value after updating the preset parameter and the parameter value before updating is smaller than the preset difference value.
13. The apparatus of claim 8, wherein the updating the contextual memory buffer data and the preset parameters according to the loss function comprises:
detecting the mean value and the variance of the loss function by using a sliding window, and updating the preset parameters when the mean value of the loss function is greater than a first threshold value and the mean value and the variance are both less than a second threshold value;
and updating the scene memory buffer data according to the size of the loss function value.
14. The apparatus according to any of claims 8-13, wherein the iteration termination condition comprises:
the current iteration round reaches a preset round;
or the current loss function value is smaller than a preset error value.
15. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the brain-inspired long-duration continuous pedestrian re-identification method of any one of claims 1-7.
16. A computer-readable storage medium, on which a computer program is stored, characterized in that the program is executed by a processor for implementing a brain-inspired long-term continuous pedestrian re-identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111588641.XA CN114445852B (en) | 2021-12-23 | 2021-12-23 | Long-time continuous pedestrian re-identification method and device for brain inspiring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111588641.XA CN114445852B (en) | 2021-12-23 | 2021-12-23 | Long-time continuous pedestrian re-identification method and device for brain inspiring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114445852A true CN114445852A (en) | 2022-05-06 |
| CN114445852B CN114445852B (en) | 2024-06-28 |
Family
ID=81362905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111588641.XA Active CN114445852B (en) | 2021-12-23 | 2021-12-23 | Long-time continuous pedestrian re-identification method and device for brain inspiring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114445852B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090157625A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for identifying an avatar-linked population cohort |
| WO2020098158A1 (en) * | 2018-11-14 | 2020-05-22 | 平安科技（深圳）有限公司 | Pedestrian re-recognition method and apparatus, and computer readable storage medium |
| CN111967429A (en) * | 2020-08-28 | 2020-11-20 | 清华大学 | Pedestrian re-recognition model training method and device based on active learning |
| CN113111814A (en) * | 2021-04-20 | 2021-07-13 | 合肥学院 | Regularization constraint-based semi-supervised pedestrian re-identification method and device |
-
2021
- 2021-12-23 CN CN202111588641.XA patent/CN114445852B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090157625A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for identifying an avatar-linked population cohort |
| WO2020098158A1 (en) * | 2018-11-14 | 2020-05-22 | 平安科技（深圳）有限公司 | Pedestrian re-recognition method and apparatus, and computer readable storage medium |
| CN111967429A (en) * | 2020-08-28 | 2020-11-20 | 清华大学 | Pedestrian re-recognition model training method and device based on active learning |
| CN113111814A (en) * | 2021-04-20 | 2021-07-13 | 合肥学院 | Regularization constraint-based semi-supervised pedestrian re-identification method and device |
Non-Patent Citations (2)
| Title |
|---|
| AL-QADERI, MK 等: "A Brain-Inspired Multi-Modal Perceptual System for Social Robots: An Experimental Realization", IEEE ACCESS, vol. 6, 9 August 2018 (2018-08-09) * |
| 张永飞等: "行人再识别技术研究进展", 中国图像图形学报, vol. 28, no. 6, 6 March 2023 (2023-03-06) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114445852B (en) | 2024-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633745B (en) | Image classification training method and device based on artificial intelligence and storage medium | |
| CN109697434B (en) | Behavior recognition method and device and storage medium | |
| CN107529650B (en) | Closed loop detection method and device and computer equipment | |
| CN108681746B (en) | Image identification method and device, electronic equipment and computer readable medium | |
| CN109816032B (en) | Unbiased mapping zero sample classification method and device based on generative countermeasure network | |
| CN111161315B (en) | Multi-target tracking method and system based on graph neural network | |
| KR20200022739A (en) | Method and device to recognize image and method and device to train recognition model based on data augmentation | |
| CN112561027A (en) | Neural network architecture searching method, image processing method, device and storage medium | |
| CN107945210B (en) | Target tracking method based on deep learning and environment self-adaption | |
| CN107240087B (en) | Object detection system and method | |
| WO2022007867A1 (en) | Method and device for constructing neural network | |
| CN110096617B (en) | Video classification method and device, electronic equipment and computer-readable storage medium | |
| CN111797970B (en) | Method and device for training neural network | |
| CN112257738A (en) | Training method and device of machine learning model and classification method and device of image | |
| CN112101114B (en) | Video target detection method, device, equipment and storage medium | |
| CN111126249A (en) | Pedestrian re-identification method and device combining big data and Bayes | |
| CN111027610B (en) | Image feature fusion method, apparatus, and medium | |
| CN115797735A (en) | Target detection method, device, equipment and storage medium | |
| CN116994021A (en) | Image detection method, device, computer readable medium and electronic equipment | |
| CN113158904B (en) | Twin network target tracking method and device based on double-mask template updating | |
| CN114445684A (en) | Method, device and equipment for training lane line segmentation model and storage medium | |
| CN115705706A (en) | Video processing method, video processing device, computer equipment and storage medium | |
| CN114528990A (en) | Neural network searching method and system | |
| CN111931747A (en) | Remote sensing image classification method suitable for multitask iterative learning and memory | |
| CN109492124B (en) | Method and device for detecting bad anchor guided by selective attention clue and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |