CN113033415B - Data queue dynamic updating method and device, electronic equipment and storage medium - Google Patents
Data queue dynamic updating method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113033415B CN113033415B CN202110327593.2A CN202110327593A CN113033415B CN 113033415 B CN113033415 B CN 113033415B CN 202110327593 A CN202110327593 A CN 202110327593A CN 113033415 B CN113033415 B CN 113033415B
- Authority
- CN
- China
- Prior art keywords
- group
- image
- model
- sample data
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 94
- 238000012549 training Methods 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims abstract description 47
- 238000000605 extraction Methods 0.000 claims abstract description 45
- 230000008569 process Effects 0.000 claims abstract description 33
- 230000006870 function Effects 0.000 claims description 31
- 238000013075 data extraction Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The disclosure provides a method, a device, electronic equipment and a storage medium for dynamically updating a data queue, and relates to the technical field of computer vision and deep learning. The specific implementation scheme is as follows: extracting features of the image data according to the first model to obtain a first image feature queue; extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to the second model to obtain first group of image features; extracting features of the second group of images according to the first model to obtain second group of image features; and in the process of model training, dynamically updating the first image characteristic queue. By adopting the method and the device, the accuracy and the data comprehensiveness of the data queues containing various scene characteristics can be ensured, so that the processing speed and the processing accuracy of hardware are improved.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence. The present disclosure relates particularly to the field of computer vision and deep learning techniques.
Background
With the development of technology, the hardware acceleration processing can be realized through artificial intelligence, and the applicable application scenarios are various, for example, the artificial intelligence technology can be adopted in the hardware design of application scenarios such as image-text processing, image processing, video processing, face recognition and the like, so as to improve the processing speed and the processing accuracy of the hardware.
However, the accuracy of the data queues including various scene features and the integrity of the data affect the processing speed and the processing accuracy of the hardware. In this regard, there is no effective solution in the related art.
Disclosure of Invention
The disclosure provides a data queue dynamic updating method, a device, electronic equipment and a storage medium.
According to an aspect of the present disclosure, there is provided a method for dynamically updating a data queue, including:
extracting features of the image data according to the first model to obtain a first image feature queue;
extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features;
extracting a second group of images corresponding to the first group of image Identifications (IDs) from the image data, and carrying out feature extraction on the second group of images according to the first model to obtain a second group of image features;
and in the process of performing model training by taking the first group of image features, the second group of image features and the first image feature queue as sample data, dynamically updating the first image feature queue.
According to another aspect of the present disclosure, there is provided a face recognition method, including:
Acquiring a face image to obtain face image data;
the face image data is subjected to recognition processing through a face recognition model to obtain a face recognition result, wherein the face recognition model is a trained second model obtained based on dynamic updating of a first face image feature queue;
the dynamic updating of the first face image feature queue includes:
extracting features of the face image data according to a first model to obtain a first face image feature queue, extracting a first group of face images from the face image data, and extracting features of the first group of face images according to a second model to obtain first group of face image features;
extracting a second group of face images corresponding to the first group of face image IDs from the face image data, and extracting features of the second group of face images according to the first model to obtain second group of face image features;
and dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features.
According to another aspect of the present disclosure, there is provided a data queue dynamic updating apparatus, including:
The first feature extraction module is used for carrying out feature extraction on the image data according to the first model to obtain a first image feature queue;
the second feature extraction module is used for extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features;
the third feature extraction module is used for extracting a second group of images corresponding to the first group of image IDs from the image data, and extracting features of the second group of images according to the first model to obtain second group of image features;
and the queue updating module is used for dynamically updating the first image feature queue in the process of taking the first group of image features, the second group of image features and the first image feature queue as sample data to perform model training.
According to another aspect of the present disclosure, there is provided a face recognition apparatus including:
the acquisition module is used for acquiring the face image to obtain face image data;
the recognition module is used for recognizing the face image data through a face recognition model to obtain a face recognition result, wherein the face recognition model is a trained second model obtained by dynamically updating a first face image feature queue by adopting a queue updating module;
The queue updating module is configured to:
extracting features of the face image data according to a first model to obtain a first face image feature queue, extracting a first group of face images from the face image data, and extracting features of the first group of face images according to a second model to obtain first group of face image features;
extracting a second group of face images corresponding to the first group of face image IDs from the face image data, and extracting features of the second group of face images according to the first model to obtain second group of face image features;
and dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods provided by any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method provided by any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising computer instructions which, when executed by a processor, implement the method provided by any one of the embodiments of the present disclosure.
By adopting the method and the device, the image data can be subjected to feature extraction according to the first model to obtain a first image feature queue; extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features; extracting a second group of images corresponding to the first group of image IDs from the image data, and extracting features of the second group of images according to the first model to obtain second group of image features; in the process of performing model training by taking the first group of image features, the second group of image features and the first image feature queue as sample data, the first image feature queue is dynamically updated, so that the accuracy and the data comprehensiveness of the data queues containing various scene features can be ensured, and the processing speed and the processing accuracy of hardware are improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow diagram of a method for dynamically updating a data queue according to an embodiment of the present disclosure;
FIG. 2 is a flow diagram of a method for dynamically updating a data queue according to an embodiment of the present disclosure;
fig. 3 is a flow chart of a face recognition method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of the composition and structure of a data queue dynamic update device according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of the composition of a data queue dynamic update device according to an embodiment of the disclosure;
fig. 6 is a flow diagram of a face recognition device according to an embodiment of the present disclosure;
fig. 7 is a block diagram of an electronic device for implementing a data queue dynamic update method and a face recognition method according to an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. The term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, e.g., including at least one of A, B, C, may mean including any one or more elements selected from the group consisting of A, B and C. The terms "first" and "second" herein mean a plurality of similar technical terms and distinguishes them, and does not limit the meaning of the order, or only two, for example, a first feature and a second feature, which means that there are two types/classes of features, the first feature may be one or more, and the second feature may be one or more.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
According to an embodiment of the present disclosure, a method for dynamically updating a data queue is provided, and fig. 1 is a schematic flow diagram of a method for dynamically updating a data queue according to an embodiment of the present disclosure, where the method may be applied to a device for dynamically updating a data queue, for example, where the device may be deployed in a terminal or a server or other processing device for execution, where feature extraction, dynamic update of a data queue may be performed, and so on. The terminal may be a User Equipment (UE), a mobile device, a cellular phone, a cordless phone, a personal digital assistant (PDA, personal Digital Assistant), a handheld device, a computing device, a vehicle mounted device, a wearable device, etc. In some possible implementations, the method may also be implemented by way of a processor invoking computer readable instructions stored in a memory. As shown in fig. 1, includes:
s101, extracting features of the image data according to a first model to obtain a first image feature queue.
S102, extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features.
S103, extracting a second group of images corresponding to the first group of image IDs from the image data, and extracting features of the second group of images according to the first model to obtain second group of image features.
S104, dynamically updating the first image feature queue in the process of performing model training by taking the first group of image features, the second group of image features and the first image feature queue as sample data.
In an example of S101-S104, in the face recognition scenario, the first model may be denoted as a face recognition model k, the second model may be denoted as a face recognition model q, the face recognition model q may be trained to obtain a required target model, and feature extraction is performed on image data according to the face recognition model k to obtain a first image feature queue with a queue length N, where N is an integer greater than 1; and extracting a first group of images (such as a group of images comprising B images, wherein B is an integer greater than 1) from the image data, and performing feature extraction on the first group of images according to the face recognition model q to obtain a first group of image features (such as a group of B features, and B is an integer greater than 1). And extracting a second group of images (such as another group of images containing B images, B being an integer greater than 1) corresponding to the first group of image IDs from the image data, and performing feature extraction on the second group of images according to the face recognition model k to obtain a second group of image features (such as another group of B features, B being an integer greater than 1). And in the process of taking the first group of image features, the second group of image features and the first image feature queue as sample data to perform model training, dynamically updating the first image feature queue.
According to the method and the device, the image features are extracted through the two models respectively to obtain the first group of image features, the second group of image features and the first image feature queue (such as an initial feature queue), and in the process of taking the first group of image features, the second group of image features and the first image feature queue as sample data to carry out model training, the accuracy and the data comprehensiveness of the data queue containing various scene features can be ensured by dynamically updating the first image feature queue, so that the processing speed and the processing accuracy of hardware are improved.
According to an embodiment of the present disclosure, there is provided a method for dynamically updating a data queue, and fig. 2 is a schematic flow chart of the method for dynamically updating a data queue according to an embodiment of the present disclosure, as shown in fig. 2, including:
and S201, extracting features of the image data according to the first model to obtain a first image feature queue.
S202, extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features.
S203, extracting a second group of images corresponding to the first group of image IDs from the image data, and extracting features of the second group of images according to the first model to obtain second group of image features.
S204, positive sample data and negative sample data are extracted from the sample data.
S205, calculating the loss of the positive sample data and the negative sample data based on constraint conditions to obtain a total loss function, training the second model according to the back propagation of the total loss function, and dynamically updating the first image feature queue in the training process.
In an example of S201-S205, in the face recognition scenario, the first model may be denoted as a face recognition model k, the second model may be denoted as a face recognition model q, the face recognition model q may be trained to obtain a required target model, and feature extraction is performed on image data according to the face recognition model k to obtain a first image feature queue with a queue length of N, where N is an integer greater than 1; and extracting a first group of images (such as a group of images comprising B images, wherein B is an integer greater than 1) from the image data, and performing feature extraction on the first group of images according to the face recognition model q to obtain a first group of image features (such as a group of B features, and B is an integer greater than 1). And extracting a second group of images (such as another group of images containing B images, B being an integer greater than 1) corresponding to the first group of image IDs from the image data, and performing feature extraction on the second group of images according to the face recognition model k to obtain a second group of image features (such as another group of B features, B being an integer greater than 1). In the process of performing model training by taking the first group of image features, the second group of image features and the first image feature queue as sample data, the loss can be calculated based on constraint conditions according to positive sample data and negative sample data in the sample number, so that a total loss function is obtained, the face recognition model q is trained according to the back propagation of the total loss function, and the first image feature queue is dynamically updated in the training process. Wherein the constraint is used to characterize: the distance between the positive sample data is sufficiently close and meets the first constraint value and the distance between the negative sample data is sufficiently far and meets the second constraint value.
According to the method and the device, the image features are extracted through the two models respectively to obtain the first group of image features, the second group of image features and the first image feature queue (such as an initial feature queue), and in the process of performing model training by taking the first group of image features, the second group of image features and the first image feature queue as sample data, losses are calculated through positive sample data, negative sample data and the constraint condition so as to be reversely propagated according to a total loss function obtained by calculating the losses, so that training of the second model is realized, the first image feature queue is dynamically updated in the training process, the model is trained according to the losses obtained by the constraint condition, the accuracy is higher, and the accuracy of the data queue containing various scene features can be ensured, thereby improving the processing speed and the processing accuracy of hardware.
In one embodiment, the extracting positive sample data and negative sample data from the sample data includes: extracting the first group of image features and the second group of image features from the sample data, and taking feature pairs corresponding to the IDs in the first group of image features and the second group of image features as the positive sample data; the first image feature queue and the second set of image features extracted from the sample data are taken as negative sample data. According to the embodiment, the pair of pictures can be adopted to respectively pass through the first model and the second model to extract the features, the feature pairs are obtained, the feature pairs are used as positive sample data, the first image feature queue and the features extracted by the first model, such as the second group of image features, are used as negative sample data, the positive and negative sample data are considered to be more comprehensive, the data comprehensiveness of the data queues containing various scene features can be ensured, and therefore the processing speed and the processing accuracy of hardware are improved.
In an embodiment, the training the first model according to the back propagation of the total loss function, and dynamically updating the first image feature queue in the training process, includes: and in the process of training the second model according to the back propagation of the total loss function, replacing corresponding image features in the first image feature queue according to the second group of image features to obtain an updated first image feature queue. By adopting the embodiment, in the model training of the second model according to the back propagation of the total loss function, corresponding image features in the first image feature queue are replaced according to the second group of image features, for example, B features (B is an integer greater than 1) are extracted through the first model, and the B features replace the last B features in the first image feature queue, so that an updated first image feature queue is obtained, manual update is not needed, real-time data update and data comprehensiveness of a data queue containing various scene features can be ensured, and therefore the processing speed and processing accuracy of hardware are improved.
In one embodiment, the method further includes updating parameters of the first model until a preset update iteration number is satisfied; extracting image features according to the first model after parameter updating iteration to obtain a third group of image features; adding the third group of image features to the sample data to obtain updated sample data; and training the model according to the updated sample data until the model converges, and obtaining a trained second model. According to the embodiment, the parameters are optimized by updating and iterating the parameters of the first model, so that image feature extraction is performed on the basis of the first model after parameter updating and iterating, a more accurate third group of image features can be obtained, the third group of image features are added into sample data, updated sample data can be obtained, model training is performed according to the updated sample data until the models are converged, a trained second model is more accurate, image-text processing, image processing, video processing and the like are performed through the trained second model, and the processing speed and the processing accuracy of hardware are improved.
It should be noted that, in the foregoing embodiments and implementation manners, regarding the dynamic update, in addition to dynamically updating the first image feature queue during the training process, the method may further include: and after the second model training is finished, extracting the corresponding image features from the image again, and dynamically updating the first image feature queue according to the re-extracted image features. For example, after training the second model according to the back propagation of the total loss function, a trained second model is obtained, a fourth group of images corresponding to the first group of image IDs are extracted from the image data according to the trained second model, feature extraction is performed on the fourth group of images according to the second model, a fourth group of image features are obtained, and corresponding image features in the first image feature queue are replaced according to the fourth group of image features, so that an updated first image feature queue is obtained.
According to an embodiment of the present disclosure, there is provided a face recognition method, and fig. 3 is a schematic flow chart of a method for dynamically updating a data queue according to an embodiment of the present disclosure, as shown in fig. 3, including:
s301, acquiring a face image to obtain face image data.
S302, the face image data is subjected to recognition processing through a face recognition model to obtain a face recognition result, wherein the face recognition model is a trained second model obtained based on dynamic updating of a first face image feature queue.
And S303, in the dynamic updating process of the first face image feature queue, feature extraction is carried out on the face image data according to a first model to obtain the first face image feature queue, a first group of face images are extracted from the face image data, and feature extraction is carried out on the first group of face images according to a second model to obtain first group of face image features.
And S304, extracting a second group of face images corresponding to the first group of face image IDs from the face image data in the dynamic updating process of the first face image feature queue, and extracting features of the second group of face images according to the first model to obtain second group of face image features.
And S305, in the dynamic updating process of the first face image feature queue, dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features.
By adopting the method and the device, the face image is acquired, the face image data can be obtained, and the face recognition model for recognizing the face image data is the trained second model obtained based on the dynamic update of the first face image feature queue, and the data features of the dynamic update of the first face image feature queue are more accurate, so that a more accurate face recognition result can be obtained.
Application example:
in application scenes such as image-text processing, image processing, video processing, public cloud face recognition, authentication and the like, the extraction of features can be trained by relying on a full-connection (FC) layer, and the application scenes such as image-text processing, image processing, video processing and the like comprise faces as an example, and for a face recognition model with 1000 ten thousand IDs, a 1000 ten thousand FC layer is needed. In the case of massive IDs, the parameter amount of the FC layer may be far greater than the parameter amount of the face recognition model backbone network (backbone), so that: 1) The face recognition model has very slow convergence speed and can not converge in some cases; 2) The FC layer of the face recognition model is easy to be fitted excessively, so that the generalization capability of the model is poor; 3) The FC layer single machine cannot be stored and needs to be stored on a plurality of machines; 4) Since the class of newly added reflow data changes, the FC needs to be retrained, which takes a long time. The model effect of training the face recognition model based on the global FC layer is not good enough, so that the hardware performance (such as face recognition precision and face recognition processing speed) of deploying the face recognition model in the hardware design of the application scene is also not good enough. The face recognition model is trained by adopting a loss function (pair loss) based on feature pairs, and the scheme has the defects that the final precision of the model is influenced by the locality of the features due to the limited quantity of positive and negative sample data in the quantity of image pairs in each batch, so that the model performance is too poor to use, and the hardware performance (such as face recognition precision and face recognition processing speed) of the face recognition model deployed in the hardware design of the application scene is also not good.
In summary, the method for realizing face recognition based on the dynamic queue in model training is provided pertinently, a global FC layer is not needed, and face feature extraction can be well trained only by a dynamic queue mode, so that the method can be widely applied to various recognition scenes containing faces, the training speed of the model is accelerated, the precision of the model is improved, and the FC layer does not need to be retrained when reflow data is newly added. Under the condition of keeping the same precision, the same precision can be achieved by using a smaller model through a training mode of a dynamic queue (namely, under the condition of unchanged precision, a face recognition model with higher speed of processing images (such as face images) on specific hardware can be obtained through the method, and then the face recognition model is deployed in hardware design of various recognition application scenes, so that hardware performance (such as face recognition precision and face recognition processing speed) is better.
The first processing flow of the embodiment of the application disclosure comprises the following contents:
1) The first model (denoted as face recognition model k) and the second model (denoted as face recognition model q) are initialized.
2) Initializing a feature Queue (Queue), wherein the length of the Queue is N (N is an integer larger than 1), and features in the feature Queue can be obtained by extracting features from K pictures (K is an integer larger than 1) through the face recognition model K during initialization.
3) B pictures of one batch (B is an integer larger than 1) are used, and B groups of features (B is an integer larger than 1) are obtained through the face recognition model q.
4) And obtaining a picture corresponding to the ID in the step 3) by using an enhancement mode such as a translation, turnover, noise adding method and the like or by using a sampling mode, and obtaining the B group of characteristics through the face recognition model k. Here, consider that: to avoid overfitting, a batch of "new" data can be created from the existing data by means of translation, flipping, noise addition, etc., to artificially increase the size of the training set.
5) And calculating a loss function, and updating parameters of the face recognition model q in a back propagation mode.
Taking one ID of B IDs as an example, and taking the characteristics obtained by respectively passing a pair of pictures through a face recognition model q and a face recognition model k as positive samples for the ID; and taking the extracted features of the face recognition model q and the N features of the Queue as negative samples. The pair of pictures can be subjected to authentication or sampling and then are respectively subjected to the characteristics obtained by the face recognition model q and the face recognition model k, and the characteristics are used as positive samples.
In the training process, consider that: the face recognition task essentially obtains features by training such that the distance between features of the same person is close enough and the distance between features of different persons is far enough, thus placing constraints such as the distance between positive samples being close enough and the distance between negative samples being far enough. The loss corresponding to the B IDs may be finally superimposed by satisfying the above constraint as a loss function, to obtain a total loss function.
6) The last B features in the Queue are replaced with the B features extracted by the face recognition model q.
7) The parameters of the face recognition model k are updated offline using the momentum approach.
The updating mode is as follows: parameter_k=m+parameter_k+ (1-m) parameter_q, where parameter_k is a parameter of face recognition model k, parameter_q is a parameter of face recognition model q, and m is the number of Queue queues.
8) And if the iteration number of the face recognition model k does not reach the preset value, returning to 3).
9) After training, the face recognition model q is output as a final target model.
According to an embodiment of the present disclosure, there is provided a data queue dynamic updating apparatus, and fig. 4 is a schematic diagram of a composition structure of the data queue dynamic updating apparatus according to an embodiment of the present disclosure, and as shown in fig. 4, a data queue dynamic updating apparatus 400 includes: a first feature extraction module 401, configured to perform feature extraction on image data according to a first model, so as to obtain a first image feature queue; a second feature extraction module 402, configured to extract a first set of images from the image data, and perform feature extraction on the first set of images according to a second model, so as to obtain a first set of image features; a third feature extraction module 403, configured to extract a second set of images corresponding to the first set of image identifiers ID from the image data, and perform feature extraction on the second set of images according to the first model, so as to obtain a second set of image features; the queue updating module 404 is configured to dynamically update the first image feature queue in a process of performing model training using the first set of image features, the second set of image features, and the first image feature queue as sample data.
According to an embodiment of the present disclosure, there is provided a data queue dynamic updating apparatus, and fig. 5 is a schematic diagram of a composition structure of the data queue dynamic updating apparatus according to an embodiment of the present disclosure, as shown in fig. 5, a data queue dynamic updating apparatus 500 includes: a first feature extraction module 501, configured to perform feature extraction on image data according to a first model, so as to obtain a first image feature queue; a second feature extraction module 502, configured to extract a first set of images from the image data, and perform feature extraction on the first set of images according to a second model to obtain a first set of image features; a third feature extraction module 503, configured to extract a second set of images corresponding to the first set of image IDs from the image data, and perform feature extraction on the second set of images according to the first model, so as to obtain a second set of image features; the queue updating module 504 is configured to dynamically update the first image feature queue in a process of performing model training using the first set of image features, the second set of image features, and the first image feature queue as sample data. Wherein the queue update module 504 further comprises: a data extraction submodule 5041 for extracting positive sample data and negative sample data from the sample data; a loss operator module 5042, configured to calculate a loss of the positive sample data and the negative sample data based on constraint conditions, so as to obtain a total loss function; a queue update sub-module 5043, configured to train the second model according to the back propagation of the total loss function, and dynamically update the first image feature queue during the training process.
In an embodiment, the data extraction submodule is configured to extract the first set of image features and the second set of image features from the sample data, and use a feature pair corresponding to the ID in the first set of image features and the second set of image features as the positive sample data; the first image feature queue and the second set of image features extracted from the sample data are taken as negative sample data.
In one embodiment, the constraint is used to characterize: the distance between the positive sample data is sufficiently close and meets a first constraint value and the distance between the negative sample data is sufficiently far and meets a second constraint value.
In an embodiment, the queue updating sub-module is configured to replace, in training the second model according to the back propagation of the total loss function, corresponding image features in the first image feature queue according to the second set of image features, to obtain an updated first image feature queue.
In one embodiment, the method further includes an update iteration module, configured to update parameters of the first model until a preset update iteration number is satisfied; extracting image features according to the first model after parameter updating iteration to obtain a third group of image features; adding the third group of image features to the sample data to obtain updated sample data; and training the model according to the updated sample data until the model converges, and obtaining a trained second model.
According to an embodiment of the present disclosure, there is provided a face recognition apparatus, and fig. 6 is a schematic flow diagram of the face recognition apparatus according to an embodiment of the present disclosure, as shown in fig. 6, including: the acquisition module 601 is configured to acquire a face image, so as to obtain face image data; the recognition module 602 is configured to perform recognition processing on the face image data through a face recognition model to obtain a face recognition result, where the face recognition model is a trained second model obtained by dynamically updating the first face image feature queue by using the queue updating module 603. The queue updating module 603 is configured to perform feature extraction on the face image data according to a first model to obtain the first face image feature queue, extract a first group of face images from the face image data, and perform feature extraction on the first group of face images according to a second model to obtain a first group of face image features; extracting a second group of face images corresponding to the first group of face image IDs from the face image data, and extracting features of the second group of face images according to the first model to obtain second group of face image features; and dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features.
The functions of each module in each apparatus of the embodiments of the present disclosure may be referred to the corresponding descriptions in the above methods, which are not repeated herein.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 7 is a block diagram of an electronic device for implementing a data queue dynamic update method and a face recognition method according to an embodiment of the present disclosure. The electronic device may be the aforementioned deployment device or proxy device. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the electronic device 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 may also be stored. The computing unit 701, the ROM702, and the RAM 703 are connected to each other through a bus 704. An input output (I/O) interface 705 is also connected to bus 704.
Various components in the electronic device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information/data with other devices through a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the respective methods and processes described above, such as a data queue dynamic update method and a face recognition method. For example, in some embodiments, the data queue dynamic update method and face recognition method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 700 via the ROM 702 and/or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the data queue dynamic updating method and the face recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the data queue dynamic update method and the face recognition method in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (14)
1. A method of dynamically updating a data queue, comprising:
extracting features of the image data according to the first model to obtain a first image feature queue;
extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features;
extracting a second group of images corresponding to the first group of image identification IDs from the image data, and extracting features of the second group of images according to the first model to obtain second group of image features;
in the process of taking the first group of image features, the second group of image features and the first image feature queue as sample data to perform model training, dynamically updating the first image feature queue;
in the process of performing model training by using the first group of image features, the second group of image features and the first image feature queue as sample data, dynamically updating the first image feature queue includes:
Extracting positive sample data and negative sample data from the sample data;
calculating loss of the positive sample data and the negative sample data based on constraint conditions to obtain a total loss function;
training the second model according to the back propagation of the total loss function, and dynamically updating the first image characteristic queue in the training process.
2. The method of claim 1, wherein the extracting positive and negative sample data from the sample data comprises:
extracting the first group of image features and the second group of image features from the sample data, and taking feature pairs corresponding to the IDs in the first group of image features and the second group of image features as the positive sample data;
the first image feature queue and the second set of image features extracted from the sample data are taken as negative sample data.
3. The method of claim 1, wherein the constraints are used to characterize: the distance between the positive sample data is sufficiently close and meets a first constraint value and the distance between the negative sample data is sufficiently far and meets a second constraint value.
4. The method of claim 1, wherein the training the second model according to the back propagation of the total loss function and dynamically updating the first image feature queue during the training process comprises:
and in the process of training the second model according to the back propagation of the total loss function, replacing corresponding image features in the first image feature queue according to the second group of image features to obtain an updated first image feature queue.
5. The method of any of claims 1-4, further comprising:
updating parameters of the first model until the preset updating iteration times are met;
extracting image features according to the first model after parameter updating iteration to obtain a third group of image features;
adding the third group of image features to the sample data to obtain updated sample data;
and training the model according to the updated sample data until the model converges, and obtaining a trained second model.
6. A face recognition method, comprising:
acquiring a face image to obtain face image data;
the face image data is subjected to recognition processing through a face recognition model to obtain a face recognition result, wherein the face recognition model is a trained second model obtained based on dynamic updating of a first face image feature queue;
The dynamic updating of the first face image feature queue includes:
extracting features of the face image data according to a first model to obtain a first face image feature queue, extracting a first group of face images from the face image data, and extracting features of the first group of face images according to a second model to obtain first group of face image features;
extracting a second group of face images corresponding to the first group of face image identification IDs from the face image data, and extracting features of the second group of face images according to the first model to obtain second group of face image features;
dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features;
the dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features includes:
extracting positive sample data and negative sample data from sample data, wherein the sample data comprises the first group of face image features, the second group of face image features and the first face image feature queue;
Calculating loss of the positive sample data and the negative sample data based on constraint conditions to obtain a total loss function;
training the second model according to the back propagation of the total loss function, and dynamically updating the first face image feature queue in the training process.
7. A data queue dynamic update apparatus, the apparatus comprising:
the first feature extraction module is used for carrying out feature extraction on the image data according to the first model to obtain a first image feature queue;
the second feature extraction module is used for extracting a first group of images from the image data, and carrying out feature extraction on the first group of images according to a second model to obtain first group of image features;
the third feature extraction module is used for extracting a second group of images corresponding to the first group of image identification IDs from the image data, and extracting features of the second group of images according to the first model to obtain second group of image features;
the queue updating module is used for dynamically updating the first image feature queue in the process of taking the first group of image features, the second group of image features and the first image feature queue as sample data to perform model training;
Wherein, the queue updating module comprises:
a data extraction sub-module for extracting positive sample data and negative sample data from the sample data;
the loss operation sub-module is used for calculating loss of the positive sample data and the negative sample data based on constraint conditions to obtain a total loss function;
and the queue updating sub-module is used for training the second model according to the back propagation of the total loss function and dynamically updating the first image characteristic queue in the training process.
8. The apparatus of claim 7, wherein the data extraction sub-module is to:
extracting the first group of image features and the second group of image features from the sample data, and taking feature pairs corresponding to the IDs in the first group of image features and the second group of image features as the positive sample data;
the first image feature queue and the second set of image features extracted from the sample data are taken as negative sample data.
9. The apparatus of claim 7, wherein the constraints are used to characterize: the distance between the positive sample data is sufficiently close and meets a first constraint value and the distance between the negative sample data is sufficiently far and meets a second constraint value.
10. The apparatus of claim 7, wherein the queue update sub-module is to:
and in the process of training the second model according to the back propagation of the total loss function, replacing corresponding image features in the first image feature queue according to the second group of image features to obtain an updated first image feature queue.
11. The apparatus of any of claims 7-10, further comprising an update iteration module to:
updating parameters of the first model until the preset updating iteration times are met;
extracting image features according to the first model after parameter updating iteration to obtain a third group of image features;
adding the third group of image features to the sample data to obtain updated sample data;
and training the model according to the updated sample data until the model converges, and obtaining a trained second model.
12. A face recognition device, comprising:
the acquisition module is used for acquiring the face image to obtain face image data;
the recognition module is used for recognizing the face image data through a face recognition model to obtain a face recognition result, wherein the face recognition model is a trained second model obtained by dynamically updating a first face image feature queue by adopting a queue updating module;
The queue updating module is configured to:
extracting features of the face image data according to a first model to obtain a first face image feature queue, extracting a first group of face images from the face image data, and extracting features of the first group of face images according to a second model to obtain first group of face image features;
extracting a second group of face images corresponding to the first group of face image identification IDs from the face image data, and extracting features of the second group of face images according to the first model to obtain second group of face image features;
dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features;
the dynamically updating the first face image feature queue according to the first group of face image features and the second group of face image features includes:
extracting positive sample data and negative sample data from sample data, wherein the sample data comprises the first group of face image features, the second group of face image features and the first face image feature queue;
Calculating loss of the positive sample data and the negative sample data based on constraint conditions to obtain a total loss function;
training the second model according to the back propagation of the total loss function, and dynamically updating the first face image feature queue in the training process.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110327593.2A CN113033415B (en) | 2021-03-26 | 2021-03-26 | Data queue dynamic updating method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110327593.2A CN113033415B (en) | 2021-03-26 | 2021-03-26 | Data queue dynamic updating method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113033415A CN113033415A (en) | 2021-06-25 |
| CN113033415B true CN113033415B (en) | 2023-11-28 |
Family
ID=76474402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110327593.2A Active CN113033415B (en) | 2021-03-26 | 2021-03-26 | Data queue dynamic updating method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113033415B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117801A (en) * | 2018-08-20 | 2019-01-01 | 深圳壹账通智能科技有限公司 | Method, apparatus, terminal and the computer readable storage medium of recognition of face |
| CN112257689A (en) * | 2020-12-18 | 2021-01-22 | 北京京东尚科信息技术有限公司 | Training and recognition method of face recognition model, storage medium and related equipment |
| WO2021017261A1 (en) * | 2019-08-01 | 2021-02-04 | 平安科技（深圳）有限公司 | Recognition model training method and apparatus, image recognition method and apparatus, and device and medium |
| CN112507833A (en) * | 2020-11-30 | 2021-03-16 | 北京百度网讯科技有限公司 | Face recognition and model training method, device, equipment and storage medium |
-
2021
- 2021-03-26 CN CN202110327593.2A patent/CN113033415B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117801A (en) * | 2018-08-20 | 2019-01-01 | 深圳壹账通智能科技有限公司 | Method, apparatus, terminal and the computer readable storage medium of recognition of face |
| WO2020037937A1 (en) * | 2018-08-20 | 2020-02-27 | 深圳壹账通智能科技有限公司 | Facial recognition method and apparatus, terminal, and computer readable storage medium |
| WO2021017261A1 (en) * | 2019-08-01 | 2021-02-04 | 平安科技（深圳）有限公司 | Recognition model training method and apparatus, image recognition method and apparatus, and device and medium |
| CN112507833A (en) * | 2020-11-30 | 2021-03-16 | 北京百度网讯科技有限公司 | Face recognition and model training method, device, equipment and storage medium |
| CN112257689A (en) * | 2020-12-18 | 2021-01-22 | 北京京东尚科信息技术有限公司 | Training and recognition method of face recognition model, storage medium and related equipment |
Non-Patent Citations (1)
| Title |
|---|
| 应用于课堂场景人脸验证的卷积网络方法研究;吴江林;刘堂友;刘钊;;信息技术与网络安全(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113033415A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4033453A1 (en) | Training method and apparatus for target detection model, device and storage medium | |
| CN114550177B (en) | Image processing method, text recognition method and device | |
| CN113033566B (en) | Model training method, recognition method, device, storage medium, and program product | |
| CN112560874A (en) | Training method, device, equipment and medium for image recognition model | |
| CN114020950B (en) | Training method, device, equipment and storage medium for image retrieval model | |
| CN114186632A (en) | Method, device, equipment and storage medium for training key point detection model | |
| CN113627536B (en) | Model training, video classification method, device, equipment and storage medium | |
| CN113033408B (en) | Data queue dynamic updating method and device, electronic equipment and storage medium | |
| CN113705362B (en) | Training method and device of image detection model, electronic equipment and storage medium | |
| CN113705515B (en) | Training of semantic segmentation model and generation method and device of high-precision map lane line | |
| CN115456167B (en) | Lightweight model training method, image processing device and electronic equipment | |
| CN113378855A (en) | Method for processing multitask, related device and computer program product | |
| CN115147680B (en) | Pre-training method, device and equipment for target detection model | |
| CN114549904B (en) | Visual processing and model training method, device, storage medium and program product | |
| CN113379592B (en) | Processing method and device for sensitive area in picture and electronic equipment | |
| CN114494747A (en) | Model training method, image processing method, device, electronic device and medium | |
| CN113379750A (en) | Semi-supervised learning method of semantic segmentation model, related device and product | |
| CN113344213A (en) | Knowledge distillation method, knowledge distillation device, electronic equipment and computer readable storage medium | |
| CN116402914B (en) | Method, device and product for determining stylized image generation model | |
| CN113033415B (en) | Data queue dynamic updating method and device, electronic equipment and storage medium | |
| CN116452861A (en) | Target model training method and device and electronic equipment | |
| CN115496916A (en) | Training method of image recognition model, image recognition method and related device | |
| CN113361575B (en) | Model training method and device and electronic equipment | |
| CN113887435A (en) | Face image processing method, device, equipment, storage medium and program product | |
| CN115019057A (en) | Image feature extraction model determining method and device and image identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |