CN111241915B - Multi-analysis algorithm fusion application service platform method based on micro-service - Google Patents
Multi-analysis algorithm fusion application service platform method based on micro-service Download PDFInfo
- Publication number
- CN111241915B CN111241915B CN201911347718.7A CN201911347718A CN111241915B CN 111241915 B CN111241915 B CN 111241915B CN 201911347718 A CN201911347718 A CN 201911347718A CN 111241915 B CN111241915 B CN 111241915B
- Authority
- CN
- China
- Prior art keywords
- analysis algorithm
- analysis
- library
- algorithm
- management module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 285
- 230000004927 fusion Effects 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000008569 process Effects 0.000 claims abstract description 9
- 238000012544 monitoring process Methods 0.000 claims description 49
- 238000010276 construction Methods 0.000 claims description 23
- 238000000605 extraction Methods 0.000 claims description 15
- 239000000284 extract Substances 0.000 claims description 10
- 238000011156 evaluation Methods 0.000 claims description 7
- 230000035945 sensitivity Effects 0.000 claims description 5
- 238000012552 review Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 15
- 230000000694 effects Effects 0.000 description 9
- 230000008901 benefit Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 238000010191 image analysis Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 238000002790 cross-validation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000014616 translation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a multiservice-based multi-analysis algorithm fusion application service platform method, which can mobilize global resource to process burst tasks, realize load balancing and realize failover: when the single algorithm has operation faults, the analysis task is automatically distributed to the algorithm which operates normally, so that the normal execution of the analysis task is ensured. The method is based on the scheduling management of various analysis algorithms, realizes the unified scheduling of computing resources, realizes the scheduling issuing of analysis tasks according to needs, and can more effectively utilize global computing resources.
Description
Technical Field
The invention belongs to the technical field of intelligent analysis of videos, and particularly relates to a multi-analysis algorithm fusion application service platform method based on micro-services.
Background
Abbreviations and key term definitions of the present invention, i.e., the english abbreviations terms presented herein provide corresponding english full and chinese translations, or detailed interpretations of chinese technical terms, are as follows:
representational state transfer (Representational State Transfer), REST is a method of creating a service by abstracting all information into resources in a unified way, any information that can be named can be used as a resource.
REST uses a resource identifier (URI) to identify the specific resource involved in interactions between components. The REST component performs actions on one resource by: a resource representation is used to capture the current or expected state of the resource, to pass the representation between components, a representation is a sequence of bytes, and representation metadata describing the sequence of bytes. REST services are implemented by the method verbs of HTTP.
At present, intelligent video analysis is an important application research in the field of computer vision, and is based on video monitoring, video retrieval, event detection and target positioning, and the analysis system automatically completes partial functions of human vision under the condition of no need of supervision by a computer and video acquisition equipment. Such as: personnel positioning, personnel identification and vehicle retrieval. Intelligent video analysis technology is actually a special technology of information data mining and perception, which analyzes and mines relevant information of interest from video data sources, and the process is actually an information perception process. Video intelligent analysis has been applied locally in areas of dense personnel and vehicles on roads, squares or hot spots. The intelligent video analysis algorithms of various manufacturers are all long, and the algorithms of specific manufacturers have better analysis effects for specific types of target groups. However, in the existing video analysis system, a single manufacturer algorithm is adopted, the advantage of each manufacturer algorithm is not fully volatilized in the application level of the video image, and more ideal practical application effects are obtained by using the analysis results of multiple manufacturer algorithms.
As shown in the single-vendor analysis algorithm calling mode technical scheme of fig. 1, the prior art schemes are all modes in which an application system calls a single-vendor analysis algorithm for analysis. The analysis algorithm adopts a private data interface to access video image data, and the analysis algorithm provides a private application interface to support the application of the application system. The implementation flow is as follows:
1, a snapshot warehouse building process: the analysis algorithm obtains video image data through the private data interface for analysis, extracts characteristic values and realizes database construction.
2, static library and focus library building flow: the application system transmits static or focused image data to the analysis algorithm through the private application interface, and the analysis algorithm extracts characteristic values to realize database construction.
3, image comparison flow: and the application system calls an analysis algorithm through the private application interface to perform image comparison.
4, monitoring a notification flow: the application system calls an analysis algorithm through a private application interface to monitor. The analysis algorithm compares the image data of the snapshot library with the image data of the attention library, and a notification is sent to the application system in the comparison.
Disadvantages of the prior art: the analysis algorithms of each manufacturer are long, and the algorithm of a specific manufacturer has more accurate analysis effect on specific types of target groups or scenes, but has limitation on certain types of target groups or scenes, so that the effect is poor or even can not be analyzed. Only a single manufacturer analysis algorithm is selected, and the limitation of the algorithm cannot be avoided.
1, under the same computing resource condition, the operation speeds of the analysis algorithms of all manufacturers are greatly different for different analysis functions, and the overall computing resource cannot be effectively utilized and the optimal computing performance cannot be obtained only by selecting the analysis algorithm of a single manufacturer. The reliability of the analysis capability of the system is completely dependent on the running stability of a single analysis algorithm only by selecting the analysis algorithm of a single manufacturer, and when the single analysis algorithm fails, the failure transfer cannot be realized.
2, only a single manufacturer analysis algorithm is adopted, an interface of the application and the analysis algorithm is basically a private interface of the analysis algorithm, when the application needs to replace the analysis algorithm of another manufacturer, the interface adapting to the analysis algorithm of another manufacturer needs to be redeveloped, the expandability and flexibility are low, and the development cost and period are long.
In order to obtain better analysis effect, in video parsing application, the comprehensive analysis from single-manufacturer algorithm analysis to multi-manufacturer algorithm analysis is beginning to be explored. The multi-manufacturer algorithm comprehensive analysis refers to the fact that the same video analysis function is conducted through a pointer, analysis algorithms of a plurality of manufacturers are deployed and applied at the same time, and secondary comprehensive analysis is conducted on analysis results of the analysis algorithms. The products of each manufacturer have different using effects on different scenes and have advantages and disadvantages due to different using algorithm principles and different realizing schemes, so that respective solutions with differences are formed. The secondary analysis is carried out by adopting a multi-algorithm mode, so that the advantages of each algorithm are utilized comprehensively, the respective special short plates of a single algorithm are avoided, and the advantages and the disadvantages are avoided; and the intelligent analysis effect of each algorithm can be checked through cross-validation among the algorithms, so that a basis is provided for evaluating the actual performance of the algorithm.
In order to more effectively utilize global computing resources, in video analysis application, unified scheduling of computing resources is realized, and realization of full-network distributed intelligent analysis also becomes a trend: the video analysis task is dispatched according to the requirement, and the IT capacity is moved according to the requirement; local analysis and storage, result aggregation and whole network intelligence; the overall resource processing burst task can be mobilized, and load balancing and fault transfer are realized.
In order to realize standardized integration of various analysis algorithms, an integrated interface of the analysis algorithms with unified specification is established, and each analysis algorithm can be integrated very conveniently only by providing service according to a standard interface, so that heterogeneous analysis algorithms of various manufacturers can be packaged for service conveniently.
Disclosure of Invention
The invention aims to provide a multi-analysis algorithm fusion application service platform method based on micro-services, which can overcome the technical problems.
The method of the invention comprises the following steps:
step 1, library building flow:
step 1.1, an application system issues a library building request to a multi-analysis algorithm scheduling management module;
step 1.1.1, requesting to carry parameters of extracting task identifiers, image data required by library construction, image library identifiers, analysis algorithms used for library construction and library construction types;
step 1.1.2, the analysis algorithm used for library construction is optional, and a plurality of analysis algorithms can be appointed for library construction without selection;
step 1.1.3, library building types are divided into: only the input image, the whole extraction of the appointed image library and the partial extraction of the appointed image library are carried out;
step 1.2, an analysis algorithm scheduling management module performs task and resource scheduling according to a scheduling strategy and the current resource condition, and distributes a database building task to an analysis algorithm;
step 1.2.1, selecting an analysis algorithm according to a scheduling strategy and a resource condition: when the request carries an analysis algorithm used for library construction, selecting an analysis algorithm appointed in the request; if the request is not carried, selecting all analysis algorithms with normal operation states of the library building service;
step 1.2.2, selecting image data according to the library building type:
step 1.2.2.1, when the library establishment type is that only the image is transmitted, selecting image data carried in the request, and storing the image data into a designated image library;
step 1.2.2.2, when the library building type is that the designated image library is extracted completely, selecting all images in the library;
step 1.2.2.3, when the library building type is that the designated image library is partially extracted, selecting the images which are not extracted by the designated analysis algorithm in the library;
step 1.2.3, sending the selected image data to a selected analysis algorithm in batches, wherein the selected analysis algorithm carries an extraction task identifier and the image data;
and 1.3, after the analysis algorithm finishes extracting the characteristics of the image data required by the database construction, asynchronously returning the characteristic data to the multi-analysis algorithm scheduling management module. The returned data comprises a feature extraction task identifier, an image identifier, feature data and an algorithm identifier;
step 1.4, the multi-analysis algorithm scheduling management module stores the received characteristic data into a video image information database;
step 1.4.1, according to a multi-analysis algorithm, a scheduling management module returns an extraction task identifier and an image identifier of characteristic data according to a certain analysis algorithm to find corresponding image data in an image library;
step 1.4.2, when the corresponding image data in the image does not have the characteristic data of the analysis algorithm, directly storing the characteristic data of the analysis algorithm, otherwise, deleting the characteristic data of the existing analysis algorithm, and then storing the characteristic data of the newly received analysis algorithm;
step 1.5, the multi-analysis algorithm scheduling management module returns the comprehensive library construction result to the application system according to the fusion strategy;
step 1.5.1, a multi-analysis algorithm scheduling management module transmits all image data in a database creation task to an analysis algorithm, and after feature data returned by the analysis algorithm are stored, the database creation result, namely the number of images with successful feature extraction and the number of images with failed feature extraction, is counted;
and step 1.5.2, the multi-analysis algorithm scheduling management module asynchronously returns the library construction result to the application system.
Step 2, image comparison flow:
step 2.1, an application system issues a target comparison request to a multi-analysis algorithm scheduling management module, wherein the request carries parameters of target images to be compared, an image library range to be compared, a similarity threshold, an analysis algorithm (optional ) and whether fusion results are required;
step 2.2, the analysis algorithm scheduling management module performs task and resource scheduling according to the scheduling strategy and the current resource condition, and distributes the comparison task to the analysis algorithm;
step 2.2.1, selection of analysis algorithm: selecting according to an analysis algorithm specified in the comparison request;
step 2.2.1.1, selecting according to the matching degree of the service scene and the algorithm characteristics;
step 2.2.1.2, selecting the running state of the service according to the analysis algorithm comparison;
2.2.1.3, selecting a better algorithm according to the comprehensive evaluation of the analysis algorithm;
step 2.2.1.4, selecting according to the space-time distribution of the built database image data in the analysis algorithm;
step 2.2.2, distributing the comparison task to a plurality of selected analysis algorithms simultaneously;
step 2.3, the analysis algorithm extracts the characteristics of the target image, compares the characteristics of the target with the characteristics of all targets in the designated image library, and returns the comparison result, namely, the data with the similarity exceeding the set threshold value to the multi-analysis algorithm scheduling management module;
and 2.4, returning the fused comparison result to the application system by the multi-analysis algorithm scheduling management module according to the fusion strategy:
step 2.4.1, after the comparison results of all analysis algorithms are returned, recording all the comparison results, and when part of the algorithms do not return the comparison results within a set time, considering the comparison results as overtime, and not waiting for the return results;
step 2.4.2, when the comparison request designates that the fusion result is not needed, the fusion is not carried out, the comparison result of each analysis algorithm is directly returned to the application system, the image comparison process is finished, otherwise, the steps 2.4.3 to 2.4.5 are continuously executed;
step 2.4.3, N analysis algorithms E i The comparison results (target ID, similarity Ei) returned by (i=1, 2 … N) are reordered and combined and summed to form a result set (target ID, similarity E) 1 Similarity E 2 . . . Similarity E N ) Form of (c);
step 2.4.4, calculating the comprehensive similarity, and setting a certain target F j (j=1, 2 … M) there is E in the results returned by the k analysis algorithms 1 ,E 2 …E k The verification similarity threshold values (similarity threshold values which can be considered as the same target) corresponding to the k analysis algorithms are respectively T 1 ,T 2 …T k Wherein engine E i The returned target object is F ji (j=1, 2 … M), the corresponding similarity is S ji (j＝1,2…M)
1) Normalized threshold
2) Standard similarity(when S ji ≥T i )
(when S ji ＜T i )
3) Comprehensive similarity
Wherein alpha is a multi-algorithm influence coefficient;
and 2.4.5, reordering all targets according to the comprehensive similarity from high to low, and returning the fusion comparison result to the application system.
Step 3, monitoring a notification flow:
step 3.1, an application system issues a monitoring request to a multi-analysis algorithm scheduling management module, wherein the request carries images required by monitoring or information of a target library to be monitored, a monitoring time range, a monitoring space range, a monitoring reason, a monitoring level, a monitoring person and a notification sensitivity, namely a similarity threshold level;
step 3.2, the multi-analysis algorithm scheduling management module schedules tasks and resources according to a scheduling strategy and the current resource condition, and distributes monitoring tasks to an analysis algorithm;
step 3.3, the video image information database collects the snapshot target image data in real time;
step 3.4, the video image information database distributes the snapshot target image data to each analysis algorithm;
step 3.5, the analysis algorithm extracts characteristics of the snap target image data, compares the characteristics with the target image characteristics of the attention library, exceeds a set similarity threshold value, generates a notification, and pushes the notification to the multi-analysis algorithm scheduling management module;
step 3.6, the analysis algorithm transmits the characteristic data of the snap target back to the video image information database;
and 3.7, returning the fused notification result to the application system by the multi-analysis algorithm scheduling management module according to the fusion strategy:
step 3.7.1, identifying duplicate notifications: when the same monitoring task designates a plurality of algorithms, aiming at the same target which is captured by the same equipment at the same time point in the same monitoring task, notifications generated by different analysis algorithms are regarded as repeated notifications;
step 3.7.2, filter duplicate notifications: forwarding only the first received notification, and the subsequent repeated notifications are not forwarded any more;
step 3.7.3, notifying automatic review: calling other analysis algorithms to compare the monitoring target with the notification target, regarding the comparison result to pass when the comparison result exceeds a similarity threshold corresponding to the corresponding notification sensitivity level, comparing whether the comparison result passes according to the other analysis algorithms, and finally determining whether the notification passes according to a voting strategy;
step 3.7.4, forwarding the notification confirmed as non-duplicate notification and through review to the application system.
The method has the advantages that:
(1) The better analysis effect is obtained: the analysis results fused by the multiple analysis algorithms are adopted, so that the advantages of each algorithm are utilized comprehensively, the respective special short plates of a single algorithm are avoided, and the advantages and the disadvantages are avoided; the intelligent analysis effect of each algorithm is checked through cross-validation among the algorithms, and a basis is provided for evaluating the actual performance of the algorithms.
(2) The utilization rate of computing resources is improved: based on the scheduling management of various analysis algorithms, the unified scheduling of the computing resources is realized, the scheduling issuing of analysis tasks according to needs is realized, and the global computing resources can be more effectively utilized.
(3) Load balancing is achieved: the overall resource processing burst task can be mobilized, and load balancing is realized.
(4) Failover is achieved: when the single algorithm has operation faults, the analysis task is automatically distributed to the algorithm which operates normally, so that the normal execution of the analysis task is ensured.
(5) The expandability is stronger: when the platform needs to integrate a new analysis algorithm, the new analysis algorithm can be integrated conveniently only by providing service according to a standard interface, the multi-analysis algorithm fusion application service platform does not need to be changed, and the application system does not need to be changed. When a new application system needs to be supported, the new application system does not need to adapt the analysis algorithm one by one with a plurality of analysis algorithms, and only needs to call application services.
Drawings
FIG. 1 is a schematic diagram of a single vendor analysis algorithm call mode technique of the prior art;
FIG. 2 is a schematic diagram of the overall architecture of a multi-analysis algorithm fusion application service platform according to the method of the present invention;
FIG. 3 is a schematic diagram of a multi-analysis algorithm fusion application service platform software architecture of the method of the present invention;
FIG. 4 is a schematic diagram of a library building process of a multi-analysis algorithm fusion application service platform according to the method of the present invention;
FIG. 5 is a schematic diagram of image comparison flow of a multi-analysis algorithm fusion application service platform according to the method of the present invention;
FIG. 6 is a schematic diagram of a monitoring notification flow of a multi-analysis algorithm fusion application service platform according to the method of the present invention;
FIG. 7 is a schematic diagram of the overall architecture of a multi-analysis algorithm fusion face application service platform of the method of the present invention;
FIG. 8 is a schematic diagram of a face database creation process of a multi-analysis algorithm fusion face application service platform of the method of the invention;
FIG. 9 is a schematic diagram of a face comparison flow of a multi-analysis algorithm fusion face application service platform of the method of the present invention;
FIG. 10 is a schematic diagram of a face monitoring notification flow of a multi-analysis algorithm fusion face application service platform of the method of the present invention;
FIG. 11 is a schematic diagram of the overall architecture of a multi-analysis algorithm fusion vehicle application service platform of the method of the present invention;
FIG. 12 is a schematic diagram of a vehicle library building process of a multi-analysis algorithm fusion vehicle application service platform according to the method of the present invention;
FIG. 13 is a schematic diagram of a vehicle-to-vehicle flow diagram for a multi-analysis algorithm fusion vehicle application service platform of the method of the present invention;
FIG. 14 is a schematic diagram of a vehicle monitoring notification flow for a multi-analysis algorithm fusion vehicle application service platform according to the method of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
The multi-analysis algorithm fusion application service platform of the method is composed of a multi-analysis algorithm dispatching management module and a video image information database, as shown in figure 2, wherein the multi-analysis algorithm dispatching management module is used for integrating the multi-analysis algorithm, task dispatching, analysis result fusion and algorithm evaluation, the video image information database is used for gathering, processing, storing and distributing video image information data, and a unified dispatching management interface is used for integrating the multi-analysis algorithm dispatching management module and the analysis algorithms of all manufacturers. The multi-analysis algorithm scheduling management module and the analysis algorithm adopt a data service interface to acquire video image data from a video image information database, the multi-analysis algorithm scheduling management module provides an application service interface for each application system to support various video image analysis application functions of the application system, and the multi-analysis algorithm fusion application service platform software frame component of the method is shown in fig. 3, adopts a layered architecture and is respectively a data layer, a service layer and an application layer. Data layer: the system comprises an original image library, an image feature library, an algorithm management library and a system management library, and is responsible for storing original image information, image feature value information and related algorithm management information in a system; the service layer comprises scheduling management of a plurality of analysis algorithms, and provides video image analysis of the multiple analysis algorithms and management service of the multiple analysis algorithms for the application layer. The system is in butt joint with each analysis algorithm through an analysis algorithm management scheduling interface, and performs unified scheduling and management on a plurality of analysis algorithms through the modules of algorithm management, operation monitoring, task scheduling, result fusion and algorithm evaluation. And the analysis application service of the video image is realized through library building service, comparison service and monitoring notification service. The application layer comprises management functions of algorithm information management, algorithm state management, algorithm configuration, algorithm evaluation and system management.
The technical scheme of the embodiment 1 of the method comprises the following steps:
step 1, a multi-analysis algorithm fusion face application service platform is composed of a face multi-analysis algorithm dispatching management module and a video image information database, as shown in fig. 7, wherein the face multi-analysis algorithm dispatching management module is used for integrating multi-analysis algorithms, task dispatching, analysis result fusion and algorithm evaluation, the video image information database is used for gathering, processing, storing and distributing video image information data, the face multi-analysis algorithm dispatching management module is integrated with face analysis algorithms of all manufacturers by adopting a unified face dispatching management interface, the face multi-analysis algorithm dispatching management module and the face analysis algorithms adopt face data service interfaces to acquire video image data from the video image information database, and the face multi-analysis algorithm dispatching management module provides face application service interfaces for all application systems to support face image analysis application functions of the application systems.
Step 2, library building flow:
step 2.1, an application system issues a library establishment request to a face multi-analysis algorithm scheduling management module, wherein the request carries face image data required by library establishment;
step 2.2, the dispatching management module of the face multi-analysis algorithm dispatches tasks and resources according to the dispatching strategy and the current resource condition, and distributes the library building tasks to the analysis algorithm;
2.3, extracting features from the face image data required by the database construction by the face analysis algorithm, and returning the face feature data to the face multi-analysis algorithm scheduling management module;
step 2.4, the facial feature data are saved to a video image information database by the facial multi-analysis algorithm scheduling management module; .
And 2.5, the dispatching management module of the face multi-analysis algorithm returns the comprehensive library building result to the application system according to the fusion strategy.
Step 3, image comparison flow:
step 3.1, an application system issues a face comparison request to a face multi-analysis algorithm scheduling management module, wherein the request carries a target face image to be compared, a face library range to be compared and a set similarity threshold;
step 3.2, the scheduling management module of the face multi-analysis algorithm performs task and resource scheduling according to the scheduling strategy and the current resource condition, and distributes the comparison task to the analysis algorithm;
step 3.3, the face analysis algorithm extracts the characteristics of the target face image, compares the target face characteristics with the face characteristics of the appointed face library, and returns the comparison result, namely the face data with the similarity exceeding the set threshold value, to the face multi-analysis algorithm scheduling management module;
and 3.4, the face multi-analysis algorithm scheduling management module returns the fused comparison result to the application system according to the fusion strategy.
Step 4, monitoring a notification flow:
step 4.1, an application system issues a monitoring request to a face multi-analysis algorithm scheduling management module, wherein the request carries face images required by monitoring or information of a face library to be monitored, a monitoring time range, a monitoring space range, a cloth monitoring reason, a monitoring level, a monitoring person and a notification sensitivity, namely a similarity threshold level;
step 4.2, the dispatching management module of the face multi-analysis algorithm dispatches tasks and resources according to the dispatching strategy and the current resource condition, and distributes the monitoring tasks to the analysis algorithm;
step 4.3, the video image information database collects the snapshot face image data in real time;
step 4.4, the video image information database distributes the snap-shot face image data to each face analysis algorithm;
step 4.5, the face analysis algorithm extracts characteristics of the snap-shot face image data, compares the characteristics with the face image characteristics of the attention library, generates a notification when the characteristics exceed a set similarity threshold, and pushes the notification to a face multi-analysis algorithm scheduling management module;
step 4.6, the face analysis algorithm transmits the characteristic data of the snap face back to the video image information database;
and 4.7, the dispatching management module of the face multi-analysis algorithm returns the fused notification result to the application system according to the fusion strategy.
The technical scheme of the embodiment 2 of the method comprises the following steps:
step 1, a multi-analysis algorithm fusion vehicle application service platform consists of a vehicle multi-analysis algorithm scheduling management module and a video image information database, as shown in fig. 11; the vehicle multi-analysis algorithm scheduling management module is used for integrating the multi-analysis algorithm, task scheduling, analysis result fusion and algorithm evaluation, the video image information database is used for gathering, processing, storing and distributing video image information data, the vehicle multi-analysis algorithm scheduling management module is integrated with the vehicle analysis algorithms of all manufacturers by adopting a unified and standard vehicle scheduling management interface, the vehicle multi-analysis algorithm scheduling management module and the vehicle analysis algorithm acquire the video image data from the video image information database by adopting a vehicle data service interface, and the vehicle multi-analysis algorithm scheduling management module provides a vehicle application service interface for all application systems so as to support the vehicle image analysis application functions of the application systems.
Step 2, library building flow:
step 2.1, an application system issues a library establishment request to a vehicle multi-analysis algorithm scheduling management module, wherein the request carries vehicle image data required by library establishment;
step 2.2, the vehicle multi-analysis algorithm scheduling management module schedules tasks and resources according to a scheduling strategy and the current resource condition, and distributes the library building tasks to an analysis algorithm;
step 2.3, extracting features from vehicle image data required by database construction by a vehicle analysis algorithm, and returning the vehicle feature data to a vehicle multi-analysis algorithm scheduling management module;
step 2.4, the vehicle multi-analysis algorithm scheduling management module stores the vehicle characteristic data into a video image information database;
and 2.5, returning the comprehensive library construction result to the application system by the vehicle multi-analysis algorithm scheduling management module according to the fusion strategy.
Step 3, image comparison flow:
step 3.1, an application system issues a vehicle comparison request to a vehicle multi-analysis algorithm scheduling management module, wherein the request carries a target vehicle image to be compared, a vehicle library range to be compared and a set similarity threshold;
step 3.2, the vehicle multi-analysis algorithm scheduling management module schedules tasks and resources according to a scheduling strategy and the current resource condition, and distributes the comparison tasks to an analysis algorithm;
step 3.3, the vehicle analysis algorithm extracts the characteristics of the target vehicle image, compares the characteristics of the target vehicle with the characteristics of the vehicles in the appointed vehicle library, and returns the comparison result, namely the vehicle data with the similarity exceeding the set threshold value, to the vehicle multi-analysis algorithm scheduling management module;
and 3.4, the vehicle multi-analysis algorithm scheduling management module returns the fused comparison result to the application system according to the fusion strategy.
Step 4, monitoring a notification flow:
step 4.1, an application system issues a monitoring request to a vehicle multi-analysis algorithm scheduling management module, wherein the request carries information of a vehicle number plate required for monitoring or a vehicle library to be monitored, a monitoring time range, a monitoring space range, a monitoring reason, a monitoring level and a monitoring person;
step 4.2, the vehicle multi-analysis algorithm scheduling management module schedules tasks and resources according to a scheduling strategy and the current resource condition, and distributes monitoring tasks to an analysis algorithm;
step 4.3, the video image information database collects the snapshot vehicle image data in real time;
step 4.4, the video image information database distributes the snapshot vehicle image data to each vehicle analysis algorithm;
step 4.5, the vehicle analysis algorithm extracts characteristics from the snapshot vehicle image data, compares the characteristics with the vehicle characteristics of the attention library, generates a notification after the characteristics are matched, and pushes the notification to the vehicle multi-analysis algorithm scheduling management module;
step 4.6, the vehicle analysis algorithm transmits the characteristic data of the snap-shot vehicle back to the video image information database;
and 4.7, returning the fused notification result to the application system by the vehicle multi-analysis algorithm scheduling management module according to the fusion strategy.
The above description is only specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions easily come within the scope of the present invention as those skilled in the art can easily come within the scope of the present invention defined by the appended claims.
Claims (3)
1. A method for integrating a multi-analysis algorithm based on micro-services into an application service platform, comprising the following steps:
step 1, library building flow:
step 1.1, an application system issues a library building request to a multi-analysis algorithm scheduling management module;
step 1.1.1, requesting to carry parameters of extracting task identifiers, image data required by library construction, image library identifiers, analysis algorithms used for library construction and library construction types;
step 1.1.2, the analysis algorithm used for library construction is optional, and a plurality of analysis algorithms can be appointed for library construction without selection;
step 1.1.3, library building types are divided into: only the input image, the whole extraction of the appointed image library and the partial extraction of the appointed image library are carried out;
step 1.2, an analysis algorithm scheduling management module performs task and resource scheduling according to a scheduling strategy and the current resource condition, and distributes a database building task to an analysis algorithm;
step 1.2.1, selecting an analysis algorithm according to a scheduling strategy and a resource condition: when the request carries an analysis algorithm used for library construction, selecting an analysis algorithm appointed in the request; if the request is not carried, selecting all analysis algorithms with normal operation states of the library building service;
step 1.2.2, selecting image data according to the library building type;
step 1.2.3, sending the selected image data to a selected analysis algorithm in batches, wherein the selected analysis algorithm carries an extraction task identifier and the image data;
step 1.3, after the analysis algorithm finishes extracting features from image data required by database construction, asynchronously returning the feature data to the multi-analysis algorithm scheduling management module, wherein the returned data comprises a feature extraction task identifier, an image identifier, feature data and an algorithm identifier;
step 1.4, the multi-analysis algorithm scheduling management module stores the received characteristic data into a video image information database;
step 1.4.1, according to a multi-analysis algorithm, a scheduling management module returns an extraction task identifier and an image identifier of characteristic data according to a certain analysis algorithm to find corresponding image data in an image library;
step 1.4.2, when the corresponding image data in the image does not have the characteristic data of the analysis algorithm, directly storing the characteristic data of the analysis algorithm, otherwise, deleting the characteristic data of the existing analysis algorithm, and then storing the characteristic data of the newly received analysis algorithm;
step 1.5, the multi-analysis algorithm scheduling management module returns the comprehensive library construction result to the application system according to the fusion strategy;
step 1.5.1, a multi-analysis algorithm scheduling management module transmits all image data in a database creation task to an analysis algorithm, and after feature data returned by the analysis algorithm are stored, the database creation result, namely the number of images with successful feature extraction and the number of images with failed feature extraction, is counted;
step 1.5.2, the multi-analysis algorithm dispatching management module asynchronously returns the library construction result to the application system;
step 2, image comparison flow;
step 2.1, an application system issues a target comparison request to a multi-analysis algorithm scheduling management module, wherein the request carries parameters of target images to be compared, an image library range to be compared, a similarity threshold, an analysis algorithm and whether fusion results are required;
step 2.2, the analysis algorithm scheduling management module performs task and resource scheduling according to the scheduling strategy and the current resource condition, and distributes the comparison task to the analysis algorithm;
step 2.2.1, selection of analysis algorithm: selecting according to an analysis algorithm specified in the comparison request:
step 2.2.1.1, selecting according to the matching degree of the service scene and the algorithm characteristics;
step 2.2.1.2, selecting the running state of the service according to the analysis algorithm comparison;
2.2.1.3, selecting a better algorithm according to the comprehensive evaluation of the analysis algorithm;
step 2.2.1.4, selecting according to the space-time distribution of the built database image data in the analysis algorithm;
step 2.2.2, distributing the comparison task to a plurality of selected analysis algorithms simultaneously;
step 2.3, the analysis algorithm extracts the characteristics of the target image, compares the characteristics of the target with the characteristics of all targets in the designated image library, and returns the comparison result, namely, the data with the similarity exceeding the set threshold value to the multi-analysis algorithm scheduling management module;
and 2.4, returning the fused comparison result to the application system by the multi-analysis algorithm scheduling management module according to the fusion strategy:
step 2.4.1, after the comparison results of all analysis algorithms are returned, recording all the comparison results, and when part of the algorithms do not return the comparison results within a set time, considering the comparison results as overtime, and not waiting for the return results;
step 2.4.2, when the comparison request designates that the fusion result is not needed, the fusion is not carried out, the comparison result of each analysis algorithm is directly returned to the application system, the image comparison process is finished, otherwise, the steps 2.4.3 to 2.4.5 are continuously executed;
step 2.4.3, N analysis algorithms E i Wherein i=1, 2 … N, and the returned comparison results are reordered and combined and summed to form a result set form;
step 2.4.4, calculating the comprehensive similarity, and setting a certain target F j E exists in the results returned by the k analysis algorithms 1 ,E 2 …E k The verification similarity threshold values corresponding to the k analysis algorithms are T respectively 1 ,T 2 …T k Wherein engine E i The returned target object is F ji The corresponding similarity is S ji Wherein j=1, 2 … M;
1) Normalized threshold
2) Standard similarity is S ji ≥T i In the time-course of which the first and second contact surfaces,
when S is ji ＜T i In the time-course of which the first and second contact surfaces,
3) Comprehensive similarity
Wherein alpha is a multi-algorithm influence coefficient;
step 2.4.5, reordering the targets from high to low according to the comprehensive similarity, and returning the fusion comparison result to the application system;
step 3, monitoring a notification flow;
step 3.1, an application system issues a monitoring request to a multi-analysis algorithm scheduling management module, wherein the request carries images required by monitoring or information of a target library to be monitored, a monitoring time range, a monitoring space range, a monitoring reason, a monitoring level, a monitoring person and a notification sensitivity, namely a similarity threshold level;
step 3.2, the multi-analysis algorithm scheduling management module schedules tasks and resources according to a scheduling strategy and the current resource condition, and distributes monitoring tasks to an analysis algorithm;
step 3.3, the video image information database collects the snapshot target image data in real time;
step 3.4, the video image information database distributes the snapshot target image data to each analysis algorithm;
step 3.5, the analysis algorithm extracts characteristics of the snap target image data, compares the characteristics with the target image characteristics of the attention library, exceeds a set similarity threshold value, generates a notification, and pushes the notification to the multi-analysis algorithm scheduling management module;
step 3.6, the analysis algorithm transmits the characteristic data of the snap target back to the video image information database;
and 3.7, returning the fused notification result to the application system by the multi-analysis algorithm scheduling management module according to the fusion strategy.
2. The method of claim 1, wherein the step 1.2.2 comprises the steps of:
step 1.2.2.1, when the library establishment type is that only the image is transmitted, selecting image data carried in the request, and storing the image data into a designated image library;
step 1.2.2.2, when the library building type is that the designated image library is extracted completely, selecting all images in the library;
and 1.2.2.3, when the library building type is the partial extraction of the appointed image library, selecting the images of which the characteristics are not extracted by using the appointed analysis algorithm in the library.
3. The method of claim 1, wherein the step 3.7 comprises the steps of:
step 3.7.1, identifying duplicate notifications: when the same monitoring task designates a plurality of algorithms, aiming at the same target which is captured by the same equipment at the same time point in the same monitoring task, notifications generated by different analysis algorithms are regarded as repeated notifications;
step 3.7.2, filter duplicate notifications: forwarding only the first received notification, and the subsequent repeated notifications are not forwarded any more;
step 3.7.3, notifying automatic review: calling other analysis algorithms to compare the monitoring target with the notification target, regarding the comparison result to pass when the comparison result exceeds a similarity threshold corresponding to the corresponding notification sensitivity level, comparing whether the comparison result passes according to the other analysis algorithms, and finally determining whether the notification passes according to a voting strategy;
step 3.7.4, forwarding the notification confirmed as non-duplicate notification and through review to the application system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911347718.7A CN111241915B (en) | 2019-12-24 | 2019-12-24 | Multi-analysis algorithm fusion application service platform method based on micro-service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911347718.7A CN111241915B (en) | 2019-12-24 | 2019-12-24 | Multi-analysis algorithm fusion application service platform method based on micro-service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111241915A CN111241915A (en) | 2020-06-05 |
| CN111241915B true CN111241915B (en) | 2024-02-09 |
Family
ID=70874173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911347718.7A Active CN111241915B (en) | 2019-12-24 | 2019-12-24 | Multi-analysis algorithm fusion application service platform method based on micro-service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111241915B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112333442B (en) * | 2020-11-03 | 2023-06-27 | 成都华栖云科技有限公司 | Ultrahigh-definition content technical quality evaluation method based on micro-service architecture |
| CN112434711B (en) * | 2020-11-27 | 2023-10-13 | 杭州海康威视数字技术股份有限公司 | Data management method and device and electronic equipment |
| CN114625901B (en) * | 2022-05-13 | 2022-08-05 | 南京维数软件股份有限公司 | Multi-algorithm integration method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573611A (en) * | 2013-10-11 | 2015-04-29 | 杭州海康威视数字技术股份有限公司 | Distributed face recognition trunking system |
| CN105959138A (en) * | 2016-04-29 | 2016-09-21 | 深圳前海大数点科技有限公司 | Micro-service dynamic disposition system and method based on cloud calculation |
| CN106033371A (en) * | 2015-03-13 | 2016-10-19 | 杭州海康威视数字技术股份有限公司 | Method and system for dispatching video analysis task |
| CN108091384A (en) * | 2017-11-09 | 2018-05-29 | 南京耐久软件科技有限公司 | A kind of medical image processing system based on micro services frame |
| CN109815839A (en) * | 2018-12-29 | 2019-05-28 | 深圳云天励飞技术有限公司 | Hover personal identification method and Related product under micro services framework |
| CN109840446A (en) * | 2017-11-24 | 2019-06-04 | 华平信息技术股份有限公司 | Management control method, the apparatus and system of plurality of human faces identification engine |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090025004A1 (en) * | 2007-07-16 | 2009-01-22 | Microsoft Corporation | Scheduling by Growing and Shrinking Resource Allocation |
-
2019
- 2019-12-24 CN CN201911347718.7A patent/CN111241915B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573611A (en) * | 2013-10-11 | 2015-04-29 | 杭州海康威视数字技术股份有限公司 | Distributed face recognition trunking system |
| CN106033371A (en) * | 2015-03-13 | 2016-10-19 | 杭州海康威视数字技术股份有限公司 | Method and system for dispatching video analysis task |
| CN105959138A (en) * | 2016-04-29 | 2016-09-21 | 深圳前海大数点科技有限公司 | Micro-service dynamic disposition system and method based on cloud calculation |
| CN108091384A (en) * | 2017-11-09 | 2018-05-29 | 南京耐久软件科技有限公司 | A kind of medical image processing system based on micro services frame |
| CN109840446A (en) * | 2017-11-24 | 2019-06-04 | 华平信息技术股份有限公司 | Management control method, the apparatus and system of plurality of human faces identification engine |
| CN109815839A (en) * | 2018-12-29 | 2019-05-28 | 深圳云天励飞技术有限公司 | Hover personal identification method and Related product under micro services framework |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111241915A (en) | 2020-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111241915B (en) | Multi-analysis algorithm fusion application service platform method based on micro-service | |
| CN101652999B (en) | System and method for managing live video data | |
| CN109150572B (en) | Method, device and computer readable storage medium for realizing alarm association | |
| CN106980627A (en) | The display methods and device of log content | |
| US20220222134A1 (en) | Distributed object identification system and method, and edge computing device | |
| CN104966006A (en) | Intelligent face identification system based on cloud variation platform | |
| CN108664914B (en) | Face retrieval method, device and server | |
| KR101612483B1 (en) | Recording server, integrated control system having the same analyzing camera images in real time and operating method thereof | |
| CN106533782A (en) | Method and system for discovering service faults of offline places in real time | |
| CN111192459A (en) | Video monitoring deployment and control method and device | |
| CN109284212A (en) | It is a kind of tracking, monitoring software behavior method, device and equipment | |
| US10009220B2 (en) | In-vehicle information system and information processing method thereof | |
| CN104834911A (en) | Portrait dynamic comparison system based on distributed cloud transformation platform | |
| CN102916830B (en) | Implement system for resource service optimization allocation fault-tolerant management | |
| CN109672721B (en) | Media file pushing method and device, server and computer readable storage medium | |
| CN107632885A (en) | The readjustment management method and device of inter-module | |
| US20230412769A1 (en) | Scalable Visual Computing System | |
| CN109409685A (en) | Information O&M operation area management system | |
| CN111291063B (en) | Master and backup copy election method, system, computer equipment and storage medium | |
| CN112182295B (en) | Service processing method and device based on behavior prediction and electronic equipment | |
| CN111917885A (en) | System for managing cloud computing service | |
| CN111210150A (en) | Intelligent park system based on edge calculation | |
| CN116319810A (en) | Flow control method, device, equipment, medium and product of distributed system | |
| CN111061916B (en) | Video sharing system based on multi-target library image recognition | |
| CN110705912A (en) | Park state monitoring method, device, equipment and medium based on park cockpit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |