CN112307247B - Distributed face retrieval system and method - Google Patents
Distributed face retrieval system and method Download PDFInfo
- Publication number
- CN112307247B CN112307247B CN202011094648.1A CN202011094648A CN112307247B CN 112307247 B CN112307247 B CN 112307247B CN 202011094648 A CN202011094648 A CN 202011094648A CN 112307247 B CN112307247 B CN 112307247B
- Authority
- CN
- China
- Prior art keywords
- retrieval
- face
- service module
- feature
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Library & Information Science (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a distributed face retrieval system, which relates to the technical field of rapid identification of face data and comprises a database, a message queue, a distributed file system, a cold start service module, a feature extraction service module, a feature slicing service module, a retrieval service module, a hot update service module, a backup and recovery service module and a log service module. In addition, a distributed face retrieval method is also disclosed, which comprises the following steps: s100, training a model; step S200, establishing a face feature library; step S300, writing into a distributed file system; step S400, processing image information and retrieval request; step S500, updating the information of the retrieval service module in real time; and step S600, regularly backing up or restoring. The invention solves the problem of high concurrency, improves the retrieval speed, reduces the complexity of the face retrieval system and promotes the high expansibility of the face retrieval system.
Description
Technical Field
The invention relates to the technical field of rapid identification of face data, in particular to a distributed face retrieval system and a distributed face retrieval method.
Background
The rapid retrieval service based on the large-scale face data shows the important value in many application scenes. For example, in the aspect of identity verification, the cost for purchasing and maintaining the identity card reading terminal equipment can be effectively reduced by using the face quick retrieval service, and meanwhile, the burden of a verified person is reduced, so that the verification can be realized without additional operation. In the application of personnel trajectory tracking, the personnel positioning efficiency can be greatly improved by carrying out face recognition and retrieval on the videos in each monitoring camera. However, the huge face data associated with the face search method becomes one of the main bottlenecks that the search service is restricted to be close to real-time loss in a massive face database to accurately query a plurality of pieces of face information most similar to the face information.
For a face retrieval system, the face retrieval system can be simply divided into two stages of face recognition, feature extraction and feature retrieval. For the first stage, with the continuous development and progress of face recognition technology based on deep learning in recent years, the recognition and feature extraction of face targets in real time can be realized at present. However, the data set used for deep learning network model training has great differences in distribution of race, age, gender, and the like and in practical application scenarios, which may cause that the recognition and feature extraction model cannot effectively recognize faces and generate feature information with discrimination, and then search noise is indirectly introduced. How to change the deep learning model to be more suitable for the application scene is also a problem which needs to be solved urgently at present.
In addition, the huge face database also causes that a large amount of feature comparison calculation needs to be carried out in the process of searching and querying the searched feature target. In practical applications, it is not practical to compare the retrieved feature objects with each of the face features in the face database. In order to improve the face retrieval speed, a large-scale face database generally divides all face features into sub-databases according to certain rules, such as the distance between the face features and a central point and other slicing rules, so that the calculation amount is reduced during retrieval, namely slicing processing is performed, one or more feature slices needing to be inquired can be quickly positioned by comparing the slicing rules in the retrieval process, and the calculation amount in the matching process is effectively reduced. How to establish valid slice index rules will directly affect the accuracy and real-time of the service. At present, establishing a feature index by using clustering is a common way, and specifically, it can be understood that a face feature library is subjected to clustering operation by using certain clustering algorithms, a plurality of clustering clusters are finally obtained, and a central vector of each cluster is regarded as a feature comparison standard. When the searched features are input, the potential candidate clusters are obtained by comparing the searched features with the central vector of each cluster, the operation is continuously repeated until the minimum cluster granularity is reached, and finally all candidate cluster members are compared one by one to obtain a final matching result, so that the aim of acceleration is fulfilled.
Although the method effectively reduces the time consumption during retrieval, the face information stored in the face library is always in a dynamic updating state, and the index center cluster vector deviates from the real center cluster vector along with the continuous execution of the operations of adding, deleting and changing the face feature library, so that the retrieval error is gradually increased, and the reliability of service is reduced. In order to effectively alleviate the phenomenon, the hierarchical index generation operation needs to be executed again after the operations of face feature addition, deletion and modification are completed, and the original cluster members are regrouped, which directly results in that the complexity of the retrieval service is improved and the expandability is greatly reduced.
The current common human face retrieval platform on the market has the following problems: firstly, the retrieval matching speed of million-level and ten million-level massive face data is slow and the accuracy is poor: with the continuous advance of intelligent construction, people can find the figure of face retrieval matching service in various application scenes such as schools, parks and the like. However, the retrieved face data in the scene is relatively closed and the number of faces is relatively small, so that the performance requirement for the retrieval service is not very high. If the face data set is applied to the government affair environment, the matched face data set reaches the order of millions or even tens of millions, and the multiple network points simultaneously request the retrieval service, so that a high concurrency phenomenon is caused. Therefore, how to design an effective service architecture and accurately and rapidly search matching algorithms become the crucial factors. In order to solve the above problems, most of the current mainstream retrieval matching platforms divide features by performing routing combination in a slice structure, so as to implement a multi-feature sub-graph node (feature information of a node, i.e. certain face data, which is used for feature comparison to determine similar face data.) service request, rather than directly processing a retrieval graph with ten million nodes. Under the condition of high feature dimension, due to the natural defect of the distance calculation method in slice retrieval, the slice structure cannot guarantee that the model structure can accurately find the matching target.
Secondly, the complexity of updating nodes of the face retrieval platform is high, the expandability is insufficient, and along with the continuous increase of the number of face data, higher and higher requirements are put forward on the expandability and the stability of the platform. However, for a general face retrieval platform, due to the limitation of the traditional hierarchical retrieval algorithm, the requirement of the added and changed nodes on the input features is high, and a partition where a new node of the input features is located needs to be effectively judged. As the node deletion and addition operations are increased continuously, slice reference deviation is finally caused, so that the retrieval result is inaccurate, a slice cutting machine needs to be updated regularly, the platform complexity is increased, and expansion is not easy.
Therefore, those skilled in the art are dedicated to developing a distributed face retrieval system and method.
Disclosure of Invention
In view of the above defects in the prior art, the technical problems to be solved by the present invention are that the retrieval matching speed of the mass face data is slow, the accuracy is not good, and the problems of high complexity and poor expandability of the update node of the face retrieval platform are solved.
In order to directly search a face library with millions and millions of nodes during face search and avoid the problem of inaccurate search caused by hierarchical routing, the inventor establishes a search graph for face input characteristics by using a nearest neighbor network structure. The nearest neighbor network structure can easily cope with nodes with the number of millions and millions, extra levels do not need to be divided, the complexity of the model is reduced, and meanwhile, the structure can easily and transversely expand the retrieval service module.
The invention provides a distributed face retrieval system, which comprises a database, a message queue, a distributed file system, a cold start service module, a feature extraction service module, a feature slicing service module, a retrieval service module, a hot update service module, a backup and recovery service module and a log service module, wherein the database is used for storing a plurality of face data;
the database comprises a face image library, a face feature library and a log library, wherein the face image library stores face image information, the face feature library stores face feature information, and the log library stores logs;
a message queue is a container that holds messages, i.e. information that needs to be transmitted between two computers, during the transmission of the messages, the messages being sent into the message queue, which acts as a man-in-the-middle role in relaying the message from its source to its destination; the message queue comprises an image message queue, a characteristic message queue, a slice message queue and a retrieval message queue, wherein the image message queue is a container for face image information subjected to characteristic extraction, the characteristic message queue is a container for characteristic information after the face characteristic extraction, the slice message queue is a container for characteristic slice information when the characteristic extraction is finished, and the retrieval message queue is a container for retrieval request information;
The distributed file system adopts a distributed mode to deploy storage resources so as to meet the requirements of continuous amplification of storage capacity and node migration;
the cold start service module sends a start beacon to other modules in the system;
the feature extraction service module extracts features of the face image by using a pre-trained feature extraction model, and sends a feature extraction completion beacon and a slicing rule to the feature slicing service module after completing the feature extraction;
in response to the slicing rule, the feature slicing service module performs grouping slicing on the extracted features, writes the extracted features into a distributed file system, and sends a slicing completion beacon to the retrieval service module after the slicing is completed;
responding to the slicing completion beacon, the retrieval service module establishes a retrieval image to prepare for receiving a retrieval request and informs the hot updating service module to start;
the log service module monitors the feature extraction process of the face image and the recovery and backup of the retrieval image;
responding to the updating request, the hot updating service module schedules the feature extraction service module and the retrieval service module to update the retrieval map;
and in response to the persistence request sent by the recovery and backup service module at regular time, the retrieval service module writes the retrieval graph into the distributed file system.
Optionally, in the distributed face retrieval system in the embodiment, there are a plurality of feature extraction service modules, which are used to improve concurrency performance.
Optionally, in the distributed face retrieval system in the above embodiment, the distributed file system is connected to the nodes through a computer network using physical storage resources managed by the file system.
Optionally, in the distributed face retrieval system in the above embodiment, the distributed file system uses a logical storage resource, and a plurality of different logical disk partitions or volume labels are combined together to form a complete hierarchical system.
Optionally, in the distributed face retrieval system in any embodiment of the foregoing, the retrieval map is a feature retrieval map generated by using a KNN algorithm, and is used for target quick retrieval.
Optionally, in the distributed face retrieval system in any of the embodiments, there are a plurality of retrieval service modules, which are used to improve concurrency performance.
Optionally, in the distributed face retrieval system in any of the above embodiments, the slicing rule is at most 100 records in a single file.
Based on any one of the above embodiments, in another embodiment of the present invention, a distributed face retrieval method is provided, including the following steps:
S100, training a model, a face target recognition model and a face feature extraction model;
step S200, establishing a face feature library, extracting corresponding face features from face images in the face image library by using a deep learning neural network model, and writing the extracted face features into the face feature library;
step S300, writing the face features in the face feature library into a file by writing the face features in a distributed file system to ensure that the face features can be read quickly;
step S400, processing image information and a retrieval request, responding to the image information, establishing a retrieval graph by the retrieval service module, and responding to the retrieval request, and retrieving by the retrieval service module;
step S500, updating information of a retrieval service module in real time, and scheduling the retrieval service module to update a retrieval map in real time by the hot update service module;
step S600, performing timed backup or recovery, and in response to the backup or recovery request periodically distributed by the backup and recovery service module, the retrieval service module performs backup or recovery operation.
Optionally, in the distributed face retrieval method in the above embodiment, step S100 specifically includes:
s110, reading an open source data set, and reading face image information in the open source data set;
S120, training a face target recognition model, namely, finishing the training of the whole deep learning frame by taking a face target frame and a target classification loss function as targets until the precision of the targets is not obviously improved any more, and storing corresponding neural network parameters;
s130, preprocessing a face recognition result, namely preprocessing the size of a face image, which is an input face recognition result, and adjusting all face images to be the same in width and height so as to generate face features with the same dimensionality;
s140, training a face feature extraction model, namely, finishing the training of the whole deep learning frame by taking a face classification loss function as a target until the precision is not obviously improved any more, and storing corresponding neural network parameters;
s150, reasoning acceleration is achieved, namely model conversion is achieved, after training is completed, the face target recognition model and the face feature extraction model are converted into an onnx or trt file structure which can be read by a TensorRT framework, and the reasoning speed of the model is improved.
Optionally, in the distributed face retrieval method in any embodiment, step S200 specifically includes:
and S210, starting the system, wherein the cold start service module reads the face image information in the face image library and sends a start beacon to each module in the distributed face retrieval system.
S220, pushing face image information, and pushing the face image information in the face image library to the image message queue one by one;
s230, feature extraction, wherein the feature extraction service module acquires the face image information through subscription and performs feature extraction to obtain corresponding face feature information, and records the result of the feature extraction to a log; the feature extraction service module can be continuously and transversely expanded under the condition of resource permission;
s240, pushing face feature information, and pushing the face feature information to the feature message queue by the feature extraction service module;
s250, acquiring face feature information, wherein the cold start service acquires the face feature information through subscription;
and S260, establishing a face feature library, and storing the face feature information into the face feature library by the cold start service module.
Optionally, in the distributed face retrieval method in any embodiment above, step S300 specifically includes:
s310, image feature extraction is monitored and detected, the log service module monitors and detects the image feature extraction in real time, and when all face feature information extraction is confirmed to be completed, the face feature information is pushed to the feature slicing service module;
S320, inquiring the face feature data in batch, wherein the feature slicing service module inquires the face feature data in batch from the face feature library according to a slicing rule;
s330, storing the human face features into the distributed file system, wherein the feature slicing service module stores the feature files of the human face feature data read in batches into the distributed file system according to a slicing rule;
s340, pushing a file storage path and slice information, wherein the characteristic slice service module pushes the file storage path and the slice information to the slice message queue;
and S350, generating a retrieval image, wherein the retrieval service module retrieves according to the acquired human face features and returns a retrieval result, acquires slice information and a feature file address according to subscription, acquires a feature file through the address and generates a nearest neighbor retrieval image.
Further, in the distributed face retrieval method in any of the above embodiments, the retrieval service module interacts with the user in step S400, and the user sends an add, delete, modify, and view request through a web page.
Optionally, in the distributed face retrieval method in any of the above embodiments, step S400 specifically includes:
S410, receiving image information and a retrieval request, wherein the retrieval service module receives the image information and the retrieval request and stores an operation log into the log library;
s420, pushing a query request, wherein the query request is pushed by the retrieval service module and pushed to different subscription topics according to the slices during pushing; the subscription topic is a mode of mutual communication between message queues, and one-to-many content broadcasting is realized by subscribing the same topic
S430, feature extraction is carried out, and the feature extraction service module acquires the face image information to be processed according to subscription and carries out feature extraction;
s440, pushing the face feature information to a feature message queue, wherein the feature extraction service module pushes the face feature information to the feature message queue;
s450, pushing the face feature retrieval information to a retrieval message queue, and pushing the face feature retrieval information to the retrieval message queue by the retrieval service module to initiate a retrieval request;
s460, obtaining the retrieval request according to the subscription, wherein the retrieval service module obtains the retrieval request according to the subscription;
s470, returning a retrieval result, wherein the retrieval service module returns the retrieval result;
s480, acquiring a retrieval result of each slice, wherein the retrieval service module acquires the retrieval result of each slice;
And S490, returning a retrieval result, wherein the retrieval service module returns the retrieval result.
Optionally, in the distributed face retrieval method in any embodiment, step S500 specifically includes:
s510, the hot update service module receives a request for adding, deleting and modifying;
s520, storing the operation request to be updated into a log library;
s530, responding to the deletion request, executing S540, otherwise, the hot update service module subscribes to acquire image information and extracts features, and pushes the face features to the feature message queue;
s540, the hot updating service module acquires the characteristic information through subscription and determines a subgraph which needs to be updated;
s550, the hot update service module broadcasts the subgraph to be updated;
s560, the retrieval service module receives the broadcast for updating;
s570, the retrieval service module pushes the update result;
and S580, if the updating of all the retrieval modules is completed, identifying that the updating is completed, and otherwise, returning to the step S570.
Optionally, in the distributed face retrieval method in any embodiment above, step S600 specifically includes:
s610, the backup and recovery service module distributes backup or recovery requests to the retrieval service module at regular time according to the slicing rule;
S620, if the request is a backup request, the retrieval service module writes corresponding feature slices and sub-graph information into the distributed file system; if not, the retrieval service module pulls the latest feature slice and the sub-graph information to the file system, checks whether the data is latest, and if not, executes the updating operation to update; if it is up to date, no action is taken.
The invention solves the problems of low retrieval matching speed and poor accuracy of mass human face data by constructing the nearest neighbor retrieval network graph, can continuously expand the retrieval module under the condition of resource permission, effectively solves the problem of high concurrency, improves the response speed of the whole retrieval service, promotes the utilization rate of system resources, realizes the high-precision retrieval with the highest speed, reduces the complexity of a human face retrieval system and promotes the high expansibility of the human face retrieval system.
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Drawings
FIG. 1 is a block diagram illustrating a distributed face retrieval system in accordance with an illustrative embodiment;
FIG. 2 is a flow diagram illustrating a distributed face retrieval method according to an exemplary embodiment;
FIG. 3 is a flow diagram illustrating training an extraction model in accordance with an illustrative embodiment;
FIG. 4 is a flow diagram illustrating the creation of a face feature library in accordance with an illustrative embodiment;
FIG. 5 is a flowchart illustrating a write distributed file system in accordance with an illustrative embodiment;
FIG. 6 is a flowchart illustrating processing image information and retrieval requests according to an exemplary embodiment;
FIG. 7 is a flow diagram illustrating updating module information in real-time in accordance with an illustrative embodiment;
FIG. 8 is a flowchart illustrating a timed backup or restore according to an example embodiment.
Detailed Description
The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.
In the drawings, structurally identical elements are represented by like reference numerals, and structurally or functionally similar elements are represented by like reference numerals throughout the several views. The size and thickness of each component shown in the drawings are arbitrarily illustrated, and the present invention is not limited to the size and thickness of each component. The thickness of the components is exaggerated somewhat schematically and appropriately in order to make the illustration clearer.
The inventor establishes a retrieval graph for the face input characteristics by utilizing a nearest neighbor network structure in order to directly retrieve a face library with millions and millions of nodes during face retrieval and avoid the problem of inaccurate retrieval caused by hierarchical routing. In addition, the nearest neighbor network structure can easily deal with nodes with the number of millions and millions, extra levels do not need to be divided, the complexity of the model is reduced, and meanwhile, the structure can easily and transversely expand the retrieval service module.
The inventor designs a distributed face retrieval system, as shown in fig. 1, which comprises a database, a message queue, a distributed file system, a cold start service module, a feature extraction service module, a feature slicing service module, a retrieval service module, a hot update service module, a backup and recovery service module and a log service module; wherein
The database comprises a face image library, a face feature library and a log library, wherein the face image library stores face image information, the face feature library stores face feature information, and the log library stores logs;
a message queue is a container that holds messages, i.e. information that needs to be transmitted between two computers, during the transmission of the messages, which are sent into the message queue, which acts as a man-in-the-middle role in relaying the message from its source to its destination; the message queue comprises an image message queue, a characteristic message queue, a slice message queue and a retrieval message queue, wherein the image message queue is a container for face image information subjected to characteristic extraction, the characteristic message queue is a container for characteristic information after the face characteristic extraction, the slice message queue is a container for characteristic slice information when the characteristic extraction is finished, and the retrieval message queue is a container for retrieval request information;
The distributed file system deploys storage resources in a distributed mode, and the embodiment uses physical storage resources managed by the file system and is connected with nodes through a computer network so as to meet the requirements of continuous storage capacity expansion and node migration;
the cold start service module sends a start beacon to other modules in the system;
the feature extraction service module extracts features of the face image by using a pre-trained feature extraction model, and sends a feature extraction completion beacon and a slicing rule to the feature slicing service module after the feature extraction is completed; in order to improve concurrency performance, the inventor designs a plurality of feature extraction service modules; the inventor sets a slicing rule to a maximum of 100 records in a single file;
in response to the slicing rule, the feature slicing service module performs grouping slicing on the extracted features, writes the extracted features into a distributed file system, and sends a slicing completion beacon to the retrieval service module after the slicing is completed;
responding to the slicing completion beacon, the retrieval service module establishes a retrieval image to prepare for receiving a retrieval request and informs the hot update service module of starting; the retrieval map is a feature retrieval map generated by a KNN algorithm and used for target quick retrieval. The number of the retrieval service modules is multiple, and the retrieval service modules are used for improving concurrency performance.
The log service module monitors the feature extraction process of the face image and the recovery and backup of the retrieval image;
responding to the updating request, and scheduling the feature extraction service module and the retrieval service module by the hot updating service module to update the retrieval graph;
and in response to the persistence request sent by the recovery and backup service module at regular time, the retrieval service module writes the retrieval graph into the distributed file system.
Based on the above embodiment, the inventor has designed a distributed face retrieval method, as shown in fig. 2, including the following steps:
s100, training a model, a face target recognition model and a face feature extraction model; as shown in fig. 3, the method specifically includes:
s110, reading an open source data set, and reading face image information in the open source data set;
s120, training a face target recognition model, namely, taking a face target frame and a target classification loss function as targets, ending the training of the whole deep learning frame until the precision is not obviously improved any more, and storing corresponding neural network parameters;
s130, preprocessing a face recognition result, namely preprocessing the size of a face image, of an input face recognition result, and adjusting all face images to be the same in width and height so as to generate face features with the same dimensionality;
S140, training a face feature extraction model, namely finishing the training of the whole deep learning frame by taking a face classification loss function as a target until the precision is not obviously improved any more, and storing corresponding neural network parameters;
and S150, reasoning and accelerating, namely model conversion, are realized, after training is finished, the face target recognition model and the face feature extraction model are converted into an onnx or trt file structure which can be read by a TensorRT framework, and the reasoning speed of the model is improved.
Step S200, establishing a face feature library, extracting corresponding face features from face images in the face image library by using a deep learning neural network model, and writing the extracted face features into the face feature library; as shown in fig. 4, the method specifically includes:
and S210, starting the system, wherein the cold start service module reads the face image information in the face image library and sends a start beacon to each module in the distributed face retrieval system.
S220, pushing face image information, and pushing the face image information in the face image library to the image message queue one by one;
s230, extracting features, wherein the feature extraction service module acquires the face image information through subscription and performs feature extraction to obtain corresponding face feature information, and records the result of the feature extraction to a log; the feature extraction service module can be continuously and transversely expanded under the condition of resource permission;
S240, pushing face feature information, and pushing the face feature information to the feature message queue by the feature extraction service module;
s250, acquiring face feature information, wherein the cold start service acquires the face feature information through subscription;
and S260, establishing a face feature library, and storing the face feature information into the face feature library by the cold start service module.
Step S300, writing the face features in the face feature library into a file by writing the face features into a distributed file system, and ensuring that the face features can be quickly read; as shown in fig. 5, the method specifically includes:
s310, image feature extraction is monitored and detected, the log service module monitors and detects the image feature extraction in real time, and when all face feature information extraction is confirmed to be completed, the face feature information is pushed to the feature slicing service module;
s320, inquiring the human face feature data in batch, wherein the feature slicing service module inquires the human face feature data in batch from the human face feature library according to a slicing rule;
s330, storing the human face features into the distributed file system, wherein the feature slicing service module stores the feature files of the human face feature data read in batches into the distributed file system according to a slicing rule;
S340, pushing a file storage path and slice information, wherein the characteristic slice service module pushes the file storage path and the slice information to the slice message queue;
and S350, generating a retrieval image, wherein the retrieval service module retrieves according to the acquired human face features and returns a retrieval result, acquires slice information and a feature file address according to subscription, acquires a feature file through the address and generates a nearest neighbor retrieval image.
Step S400, processing the image information and the retrieval request, responding to the image information, establishing a retrieval graph by the retrieval service module, and responding to the retrieval request, and retrieving by the retrieval service module; the retrieval service module interacts with the user, and the user sends an adding, deleting, modifying and viewing request through a webpage. As shown in fig. 6, the method specifically includes:
s410, receiving image information and a retrieval request, wherein the retrieval service module receives the image information and the retrieval request and stores an operation log into the log library;
s420, pushing a query request, wherein the retrieval service module pushes the query request to different subscription topics according to the slices during pushing; the subscription topic is a mode of mutual communication between message queues, and one-to-many content broadcasting is realized by subscribing the same topic
S430, feature extraction is carried out, and the feature extraction service module acquires face image information to be processed according to subscription and carries out feature extraction;
s440, pushing the face feature information to a feature message queue, wherein the feature extraction service module pushes the face feature information to the feature message queue;
s450, pushing the face feature retrieval information to a retrieval message queue, and pushing the face feature retrieval information to the retrieval message queue by the retrieval service module to initiate a retrieval request;
s460, obtaining the retrieval request according to the subscription, wherein the retrieval service module obtains the retrieval request according to the subscription;
s470, returning a retrieval result, wherein the retrieval service module returns the retrieval result;
s480, acquiring a retrieval result of each slice, wherein the retrieval service module acquires the retrieval result of each slice;
and S490, returning a retrieval result, wherein the retrieval service module returns the retrieval result.
Step S500, updating information of a retrieval service module in real time, and scheduling the retrieval service module to update a retrieval map in real time by the hot update service module; as shown in fig. 7, the method specifically includes:
s510, the hot update service module receives a request for adding, deleting and modifying;
S520, storing the operation request to be updated into a log library
S530, responding to the deletion request, executing S540, otherwise, the hot update service module subscribes to acquire image information and extracts features, and pushes the face features to the feature message queue;
s540, the hot updating service module acquires the characteristic information through subscription and determines a subgraph which needs to be updated;
s550, the hot update service module broadcasts the subgraph to be updated;
s560, the retrieval service module receives the broadcast for updating;
s570, the retrieval service module pushes the update result;
and S580, if the updating of all the retrieval modules is completed, identifying that the updating is completed, and otherwise, returning to the step S570.
Step S600, backing up or recovering at regular time, wherein the retrieval service module executes backing up or recovering operation in response to the backing up or recovering request distributed by the backing up and recovering service module at regular time; as shown in fig. 8, the method specifically includes:
s610, the backup and recovery service module distributes backup or recovery requests to the retrieval service module at regular time according to the slicing rule;
s620, if the request is a backup request, the retrieval service module writes corresponding feature slices and sub-graph information into the file system; if not, the retrieval service module pulls the latest feature slice and the sub-graph information to the file system, checks whether the data is latest, and if not, executes the updating operation to update; if it is up to date, no action is taken.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (10)
1. A distributed face retrieval system is characterized by comprising a database, a message queue, a distributed file system, a cold start service module, a feature extraction service module, a feature slicing service module, a retrieval service module, a hot update service module, a backup and recovery service module and a log service module;
the database comprises a face image library, a face feature library and a log library, wherein the face image library stores face image information, the face feature library stores face feature information, and the log library stores logs;
the message queue comprises an image message queue, a feature message queue, a slice message queue and a retrieval message queue, wherein the image message queue is a container for face image information subjected to feature extraction, the feature message queue is a container for feature information after the face feature extraction, the slice message queue is a container for feature slice information when the feature extraction is finished, and the retrieval message queue is a container for retrieval request information;
The distributed file system adopts a distributed mode to deploy storage resources so as to meet the requirements of continuous amplification of storage capacity and node migration;
the cold start service module sends a start beacon to other modules in the distributed face retrieval system;
the feature extraction service module extracts features of the face image by using a pre-trained feature extraction model, and sends a feature extraction completion beacon and a slicing rule to the feature slicing service module after the feature extraction is completed;
in response to the slicing rule, the feature slicing service module performs grouping slicing on the extracted human face features, writes the human face features into the distributed file system, and sends a slicing completion beacon to the retrieval service module after the slicing is completed;
responding to the slice completion beacon, the retrieval service module establishes a retrieval image to be prepared for receiving a retrieval request and informs the hot update service module of starting;
the log service module monitors the face feature extraction process and the recovery and backup of the retrieval image;
responding to an updating request, and scheduling the feature extraction service module and the retrieval service module to update the retrieval map by the hot updating service module;
And responding to a persistence request sent by the recovery and backup service module at regular time, and writing the retrieval graph into the distributed file system by the retrieval service module.
2. The distributed face retrieval system of claim 1, wherein the plurality of feature extraction service modules are configured to improve concurrency.
3. The distributed face retrieval system of claim 1, wherein there are a plurality of retrieved service modules for improving concurrency performance.
4. A distributed face retrieval method based on the distributed face retrieval system according to any one of claims 1 to 3, comprising the steps of:
step S100, training a model, a face target recognition model and a face feature extraction model;
step S200, establishing a face feature library, extracting corresponding face features from face images in the face image library by using a deep learning neural network model, and writing the extracted face features into the face feature library;
step S300, writing the face features in the face feature library into a file by writing the face features into a distributed file system, and ensuring that the face features can be quickly read;
step S400, processing image information and a retrieval request, responding to the image information, establishing a retrieval graph by the retrieval service module, and responding to the retrieval request, and retrieving by the retrieval service module;
Step S500, updating information of a retrieval service module in real time, and scheduling the retrieval service module to update a retrieval graph in real time by the hot updating service module;
step S600, performing timed backup or recovery, and in response to the backup or recovery request periodically distributed by the backup and recovery service module, the retrieval service module performs backup or recovery operation.
5. The distributed face retrieval method of claim 4, wherein the step S100 comprises:
s110, starting a source data set to read face image information in the source data set;
s120, training a face target recognition model, namely, taking a face target frame and a target classification loss function as targets, ending the training of the whole deep learning frame until the precision is not obviously improved any more, and storing corresponding neural network parameters;
s130, preprocessing a face recognition result, namely preprocessing the size of a face image, of an input face recognition result, and adjusting all face images to be the same in width and height so as to generate face features with the same dimensionality;
s140, training a face feature extraction model, namely, finishing the training of the whole deep learning frame by taking a face classification loss function as a target until the precision is not obviously improved any more, and storing corresponding neural network parameters;
S150, reasoning acceleration is achieved, namely model conversion is achieved, after training is completed, the face target recognition model and the face feature extraction model are converted into an onnx or trt file structure which can be read by a TensorRT framework, and the reasoning speed of the model is improved.
6. The distributed face retrieval method of claim 4, wherein the step S200 comprises:
s210, starting a system, wherein the cold start service module reads face image information in the face image library and sends a start beacon to each module in the distributed face retrieval system;
s220, pushing face image information, and pushing the face image information in the face image library to the image message queue one by one;
s230, extracting features, wherein the feature extraction service module acquires the face image information through subscription and performs feature extraction to obtain corresponding face feature information, and records the result of the feature extraction to a log;
s240, pushing face feature information, and pushing the face feature information to the feature message queue by the feature extraction service module;
s250, acquiring face feature information, wherein the cold start service acquires the face feature information through subscription;
And S260, establishing a face feature library, and storing the face feature information into the face feature library by the cold start service module.
7. The distributed face retrieval method of claim 4, wherein the step S300 comprises:
s310, image feature extraction is monitored and detected, the log service module monitors and detects the image feature extraction in real time, and when all face feature information extraction is confirmed to be completed, the face feature information is pushed to the feature slicing service module;
s320, inquiring the face feature data in batch, wherein the feature slicing service module inquires the face feature data in batch from the face feature library according to a slicing rule;
s330, storing the human face features into the distributed file system, wherein the feature slicing service module stores the feature files of the human face feature data read in batches into the distributed file system according to a slicing rule;
s340, pushing a file storage path and slice information, wherein the characteristic slice service module pushes the file storage path and the slice information to the slice message queue;
and S350, generating a retrieval image, wherein the retrieval service module retrieves according to the acquired human face features and returns a retrieval result, acquires slice information and a feature file address according to subscription, acquires a feature file through the address and generates a nearest neighbor retrieval image.
8. The distributed face retrieval method of claim 4, wherein the step S400 comprises:
s410, receiving image information and a retrieval request, wherein the retrieval service module receives the image information and the retrieval request and stores an operation log into the log library;
s420, pushing a query request, wherein the retrieval service module pushes the query request to different subscription topics according to the slices during pushing;
s430, feature extraction is carried out, and the feature extraction service module acquires the face image information to be processed according to subscription and carries out feature extraction;
s440, pushing the face feature information to a feature message queue, wherein the feature extraction service module pushes the face feature information to the feature message queue;
s450, pushing the face feature retrieval information to a retrieval message queue, and pushing the face feature retrieval information to the retrieval message queue by the retrieval service module to initiate a retrieval request;
s460, acquiring a retrieval request according to subscription, wherein the retrieval service module acquires the retrieval request according to subscription;
s470, returning a retrieval result, wherein the retrieval service module returns the retrieval result;
s480, acquiring a retrieval result of each slice, wherein the retrieval service module acquires the retrieval result of each slice;
And S490, returning a retrieval result, wherein the retrieval service module returns the retrieval result.
9. The distributed face retrieval method of claim 4, wherein the step S500 comprises:
s510, the hot update service module receives the request of adding, deleting and modifying;
s520, storing the operation request to be updated into a log library;
s530, responding to the deletion request, executing S540, otherwise, the hot update service module subscribes to acquire image information and extracts features, and pushes the face features to the feature message queue;
s540, the hot updating service module acquires the characteristic information through subscription and determines a subgraph which needs to be updated;
s550, the hot update service module broadcasts the subgraph to be updated;
s560, the retrieval service module receives the broadcast for updating;
s570, the retrieval service module pushes the update result;
and S580, if the updating of all the retrieval modules is completed, identifying that the updating is completed, and otherwise, returning to the step S570.
10. The distributed face retrieval method of claim 4, wherein the step S600 comprises:
s610, the backup and recovery service module distributes backup or recovery requests to the retrieval service module at regular time according to the slicing rule;
S620, if the request is a backup request, the retrieval service module writes corresponding feature slices and sub-graph information into the file system; if not, the retrieval service module pulls the latest feature slice and the sub-graph information to the file system, checks whether the data is latest, and if not, executes the updating operation to update; if it is up to date, no action is taken.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011094648.1A CN112307247B (en) | 2020-10-14 | 2020-10-14 | Distributed face retrieval system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011094648.1A CN112307247B (en) | 2020-10-14 | 2020-10-14 | Distributed face retrieval system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112307247A CN112307247A (en) | 2021-02-02 |
| CN112307247B true CN112307247B (en) | 2022-05-24 |
Family
ID=74489376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011094648.1A Active CN112307247B (en) | 2020-10-14 | 2020-10-14 | Distributed face retrieval system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112307247B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875016A (en) * | 2018-06-20 | 2018-11-23 | 上海百林通信网络科技服务股份有限公司 | A kind of sample technology of sharing and evaluation method based on face recognition application |
| CN116682167B (en) * | 2023-08-01 | 2023-10-27 | 山东威尔数据股份有限公司 | Cluster type IoT-based face feature extraction method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8489872B1 (en) * | 2009-10-16 | 2013-07-16 | Vikas Kapoor | Apparatuses, methods and systems for a real-time desktop configurer utilizing a user identifier or an initialization request to retrieve a data-structure-tracking syntax-configured string |
| CN105373626A (en) * | 2015-12-09 | 2016-03-02 | 深圳融合永道科技有限公司 | Distributed face recognition track search system and method |
| CN107251098A (en) * | 2015-03-23 | 2017-10-13 | 英特尔公司 | The true three-dimensional virtual for promoting real object using dynamic 3 D shape is represented |
-
2020
- 2020-10-14 CN CN202011094648.1A patent/CN112307247B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8489872B1 (en) * | 2009-10-16 | 2013-07-16 | Vikas Kapoor | Apparatuses, methods and systems for a real-time desktop configurer utilizing a user identifier or an initialization request to retrieve a data-structure-tracking syntax-configured string |
| CN107251098A (en) * | 2015-03-23 | 2017-10-13 | 英特尔公司 | The true three-dimensional virtual for promoting real object using dynamic 3 D shape is represented |
| CN105373626A (en) * | 2015-12-09 | 2016-03-02 | 深圳融合永道科技有限公司 | Distributed face recognition track search system and method |
Non-Patent Citations (1)
| Title |
|---|
| 一种分布式人脸识别方法及性能优化;闵卫东等;《光学精密工程》;20170315;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112307247A (en) | 2021-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210158219A1 (en) | Method and system for an end-to-end artificial intelligence workflow | |
| CN112199375B (en) | Cross-modal data processing method and device, storage medium and electronic device | |
| CN108897784B (en) | Emergency multidimensional analysis system based on social media | |
| CN112307247B (en) | Distributed face retrieval system and method | |
| CN110909182A (en) | Multimedia resource searching method and device, computer equipment and storage medium | |
| CN109739975B (en) | Hot event extraction method and device, readable storage medium and electronic equipment | |
| JP2022020070A (en) | Information processing, information recommendation method and apparatus, electronic device and storage media | |
| CN113298197B (en) | Data clustering method, device, equipment and readable storage medium | |
| CN114357197B (en) | Event reasoning method and device | |
| US20230056760A1 (en) | Method and apparatus for processing graph data, device, storage medium, and program product | |
| CN108509545B (en) | Method and system for processing comments of article | |
| CN114330703A (en) | Method, device and equipment for updating search model and computer-readable storage medium | |
| CN109241315B (en) | Rapid face retrieval method based on deep learning | |
| CN113590898A (en) | Data retrieval method and device, electronic equipment, storage medium and computer product | |
| CN114490923A (en) | Training method, device and equipment for similar text matching model and storage medium | |
| US20230326185A1 (en) | Object recognition method and apparatus, device, and storage medium | |
| CN110209895B (en) | Vector retrieval method, device and equipment | |
| CN107291875B (en) | Metadata organization management method and system based on metadata graph | |
| Firmino et al. | Automatic and semi-automatic annotation of people in photography using shared events | |
| CN113873025B (en) | Data processing method and device, storage medium and electronic equipment | |
| Aggarwal et al. | Spatio-temporal frequent itemset mining on web data | |
| CN115146103A (en) | Image retrieval method, image retrieval apparatus, computer device, storage medium, and program product | |
| Zheng et al. | Towards edge-cloud collaborative machine learning: A quality-aware task partition framework | |
| WO2022148108A1 (en) | Systems, devices and methods for distributed hierarchical video analysis | |
| Wu et al. | Neist: a neural-enhanced index for spatio-temporal queries |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 315048 room 5-1-1, building 22, East Zone, Ningbo new material innovation center, high tech Zone, Ningbo, Zhejiang Province Applicant after: Ningbo Boden Intelligent Technology Co.,Ltd. Address before: Room 1002, 288, 296 Baihe Road, hi tech Zone, Ningbo, Zhejiang 315048 Applicant before: NINGBO BODEN INTELLIGENT TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |