CN110245267A - Multi-user's video flowing deep learning is shared to calculate multiplexing method - Google Patents
Multi-user's video flowing deep learning is shared to calculate multiplexing method Download PDFInfo
- Publication number
- CN110245267A CN110245267A CN201910413748.7A CN201910413748A CN110245267A CN 110245267 A CN110245267 A CN 110245267A CN 201910413748 A CN201910413748 A CN 201910413748A CN 110245267 A CN110245267 A CN 110245267A
- Authority
- CN
- China
- Prior art keywords
- deep learning
- request
- precision
- multiplexing
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
- G06F16/735—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/74—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The present invention relates to computers, video processing, to realize under multi-user scene, the service of query video is provided by technologies such as target identification and target detections, the present invention, multi-user's video flowing deep learning is shared to calculate multiplexing method, firstly, when the request with detection operation or identification operation arrives, according to the relevance of Spatial Dimension, request is merged；Then, according to the relevance of time dimension, whether inquiry has suitable data for multiplexing, and the deep learning model for having configured parameter is called to be analyzed；For not reusable part, first according to the equilibrium relation of velocity accuracy, find most suitable parameter configuration in analytic process, then difference detector and deep learning model is called to carry out video analysis accordingly, analysis result is finally exported and is stored in data warehouse, hoisting module is used to be promoted in database original result precision then in order to be multiplexed to high-precision inquiry request.Present invention is mainly applied to videos to handle occasion.
Description
Technical field
The present invention relates to computers, video processing, and in particular to multi-user's video flowing deep learning is shared to calculate multiplexing side
Method.
Background technique
Currently, deep learning, which has become, pushes artificial intelligence application landing and universal important engine.Especially counting
Calculation machine visual field, the fast development of deep learning is that the field brings deep change, wherein the most representative
It is the continuous development of the technologies such as image analysis technology such as target detection and target identification.Target detection can be used to divide target
Class, such as identify whether there is dog in image, someone has desk, but can not identify the name of the people；Target identification then may be used
Identify the specific identity of the people.For an image, application target detection model can quickly identify all in image
Target, this brings huge variation for video analysis process.
During traditional video analysis, query service mainly is provided for user by way of labelling manually.It borrows
The development for helping deep learning can automatically analyze video content by technologies such as target identification, target detections, provide more high-quality
With query service abundant.For example, when user's editing video certain in video can be automatically found by Target Recognition Algorithms
All segments that people occurs, this is bringing great convenience property of user.Therefore, it is analyzed using the video flowing based on deep learning
It substitutes traditional manual label type query video mode and has been increasingly becoming a kind of trend.But, because deep learning model provides
The features such as source demand is high, and the training time is long, it will usually allow one computing platform of multiple user sharings by the way of resource-sharing,
To improve the utilization rate of deep learning system resources in computation and reduce entreprise cost.
The shortcomings that prior art
The existing Video Analysis Technology based on deep learning has the shortcomings that there are two aspects mostly: first is that query result list
One, second is that not solving the locality that multi-user inquires under shared platform.Query result is single in noscope, chameleon etc.
That embodies in system is particularly evident.In general, target detection model can identify thousands of kinds of different target categories, but know
Other speed is slower.Therefore, to solve the problems, such as that deep learning model treatment video data is excessively slow, noscope proposition passes through training
The network number of plies less proprietary model identifies vehicle, substantially increases recognition speed.But this mode abandons
The versatility of deep learning model is just unable to satisfy its demand when user needs to inquire multiple targets.
On the other hand, these models of noscope, chameleon, videostorm do not solve multi-user under shared platform
The limitation of inquiry.Limitation is embodied in the following aspects:
The locality of time dimension.The locality of time dimension shows that different periods inquiry has weight in inquiry data
Renaturation, when it is new inquire come when, before there are some users to carry out inquiry concern to identical content.This is in some heat
It is showed on door video particularly evident.The locality of time dimension is also embodied in the similitude on continuous content frame.Due to video counts
According to exclusive feature itself, the continuity of video is kept between successive frame with high similitude.In addition, in some prisons
The camera head monitor in video such as park is controlled, changing very little in part-time (such as night) content in the video.When passing through depth
When spending the target in learning model identification video frame, the recognition result between these successive frames also can be essentially identical, therefore these
Successive frame, which is brought, largely to be computed repeatedly.
The locality of Spatial Dimension.Be mainly manifested in multiple reasonings request that same one piece of data can be arrived simultaneously simultaneously into
Row query processing.Often there is identical and similitudes for these inquiries.These inquiries are calculated, it can often be cut out
Union operation is cut, to avoid unnecessary redundant computation.
Locality between data result logic.Be embodied in that data result between different models has a multiplexing in logic can
It can property.Such as often there is multiplexing relationships in logic each other for object detection and object identification.For example, for video
A certain frame, when object detection shows nobody, object recognition algorithm can not then can recognize that Zhang San, so as to avoid object
The calculating of body identification people.Conversely, the result of object identification also can be directly multiplexed into object detection in any case.As led to
Object recognition algorithm is crossed to show then to be shown to be someone there are Zhang San, directly avoid the object detection for people.
Summary of the invention
In order to overcome the deficiencies of the prior art, the present invention is directed to propose a kind of shared calculating of multi-user's video flowing deep learning is multiple
Use method.It realizes under multi-user scene, provides the service of query video by technologies such as target identification and target detections.And needle
Locality existing for multi-user's query video under shared calculating environment is optimized, so that depth can be directed between multi-user
It practises the reasoning results to be shared, to improve the speed of service, solution deep learning model is spent low in processing video data speed per hour
The problem of.Meanwhile solving the equilibrium problem because of shared bring speed and precision.For this reason, the technical scheme adopted by the present invention is that
Multi-user's video flowing deep learning is shared to calculate multiplexing method, firstly, when the request with detection operation or identification operation arrives
When, according to the relevance of Spatial Dimension, request is merged, crops the part that there is overlapping in request；Then, according to when
Between dimension relevance, request retrieved into object detection database or object identification database, inquiry whether have conjunction
Suitable data are for multiplexing, when reusable data are not present, according to the relevance between logic, using object identification database
Or data operate detection in object detection database or identification operation is filtered reduction invalid computation, recall the good ginseng of configuration
Several deep learning models are analyzed；For not reusable part, first according to the equilibrium relation of velocity accuracy, find point
Then most suitable parameter configuration during analysis calls difference detector and deep learning model to carry out video analysis, most accordingly
Analysis result exports and is stored in data warehouse at last, hoisting module be then used to be promoted in database original result precision in order to
It is multiplexed to high-precision inquiry request.
It needs to carry out reasonable disposition to relevant parameter such as resolution ratio, the selection of deep learning model, frame-skipping rate.
The multiplexing specific steps of time dimension:
1) similarity between successive frame is detected by the way of Difference test, difference detector obtains the histogram of successive frame
Scheme and calculate Histogram distance, further calculate out similarity, then judges whether data can be multiplexed according to similarity；
2) when new request arrives, two parts be split as to request first, reusable part with can not multiplexing part,
The data of reusable part are present in database, directly can obtain knot by the faster database query operations of runing time
Fruit；For not reusable part, processing acquisition request is carried out as a result, the knot that then will be requested yet by deep learning model
Fruit feeds back to database, in order to which subsequent query is multiplexed.
The multiplexing of Spatial Dimension: request is merged in such a way that request cuts and merges, crops overlapping therein
Multiple repetitive requests are reduced in part.
The multiplexing of logical dimension: being associated with the data foundation of object detection and object identification, finds between different models and exists
The data of relevance are to be filtered mutually.
The multiplexing specific steps of logical dimension:
Contain m frame image in one video, shared l type objects are detectable and k personage can recognize, use Dl*m and Rk*m
Two matrixes store data, and Dl*m is to represent object matrix, and Rk*m is representative figure's matrix, when new inquiring comes
When, it is inquired in Dl*m and Rk*m first whether with the presence of corresponding data, if thening be used directly, Dij is 1 the i-th frame picture of expression
In there are j object, indicate that the object is not present for 0；
Theorem one: if D1j is 0, as 0 < j≤m, R*j is also 0；
Theorem two: if Rij is 1, as 0 < j≤m, D1j is also 1；
Theorem one indicates, when jth frame detects nobody, then Model of Target Recognition will can't detect in jth frame
Anyone；Theorem two indicates that, when identifying personage i in jth frame, target detection matrix can automatically update someone in jth frame, by
This dynamically updates database data according to theorem one and theorem two when carrying out logical multiplexing, is target detection and target identification
Establish association.
Balance the precision and speed specific steps of inquiry:
Step 1: being fitted to the precision of the different models under different parameters with length velocity relation, each model speed is obtained
The corresponding relationship of degree and precision can select optimal parameter to carry out video analysis, when inquiring and to reach user demand；
Step 2:, by the diversity factor between the precision and two frames of former frame, adding phase according to Markov Chain rule
The adjustment parameter answered constantly corrects the numerical value of adjustment parameter by confirmatory experiment to predict the precision of a later frame, so as to
Accurate precision evaluation is carried out, specifically used δ diff represents the diversity factor between two frames, and the essence of i-1 frame is represented using A (fi-1)
Degree, uses k as adjustment parameter, then according to Markov Chain rule, the precision of the i-th frame are as follows:
A (fi-1)=k* δ diff*A (fi-1)
Finally, the result multiplexing between reasoning request is conditional multiplexing；
Step 3: retain deep learning model as a result, to the partial results of difference detector using high-precision model into
Row detects again, and sufficiently calculated result of the multiplexing difference detector about similarity between two frames, reappraise successive frame it
Between precision, to improve detection accuracy on the whole.
Conditional multiplexing is specifically included using the frequency increased using deep learning model.
The features of the present invention and beneficial effect are:
The present invention has built the deep learning model of multiplicity, it is thus possible to realize under multi-user scene, be known by target
The service of the technologies such as other target detection offer query video.And office existing for multi-user's query video under environment is calculated for shared
It is sex-limited to optimize, so that can be shared for deep learning the reasoning results between multi-user, to improve the speed of service, solve
Certainly deep learning model spends low problem in processing video data speed per hour.Meanwhile it solving because of shared bring speed and precision
Equilibrium problem.
Detailed description of the invention:
Fig. 1: the request results multiplexing of same type data.
Fig. 2: the merging treatment of inquiry.
Fig. 3: the multiplexing of logical dimension.
Fig. 4: more reasoning request results are multiplexed overall architecture.
Specific embodiment
The present invention relates to computer visions and high-performance computing sector, and it is total to propose a kind of multi-user's video flowing deep learning
Enjoy calculating multiplexing method.The method achieve under multi-user scene, video is provided by technologies such as target identification target detections
The service of inquiry.And optimized for locality existing for multi-user's query video under shared calculating environment, so that multi-user
Between can be shared for deep learning the reasoning results, to improve the speed of service, solve deep learning model and regarded in processing
Frequency spends low problem according to speed per hour.Meanwhile solving the equilibrium problem because of shared bring speed and precision.
For the limitation of the prior art, it is proposed that a new video flowing analysis method, supports multiple users share ring
Data sharing under border is to improve analysis speed.It is calculated in environment shared, the reasoning request content that user submits includes several
A aspect: request type (object detection, object identification, object tracking etc.), request data and range, demand precision etc..It is full
The sufficient diversified reasoning request content of user need to build various deep learning model in shared calculate, arrive in request in environment
Suitable model is dynamically selected when coming to meet user demand.In more reasoning request results multiplying questions, research emphasis is just
It is to find the connection between more reasoning requests among this diversified request content, the relevance constructed between multi-request is closed
System carries out multiplexing to accelerate runing time, while solving the problems such as bring precision, to reach due to multiplexing and meet user demand
Effect.
This method mainly includes two big modules, first is that the relevance between building request, second is that solving because of multiplexing bring essence
The equilibrium problem of degree and speed.
Relevance between building request
Step 1, the multiplexing of time dimension.For video data, the relevance of time dimension is embodied in terms of two: one
It is in video analysis, in terms of content with the similitude of height between successive frame；Second is that being had largely not for hot data
Inquiry with the period can carry out retrieval analysis to it.A possibility that the two aspects bring a large amount of multiplexings.Firstly for continuous
The similitude of frame, since deep learning model reasoning request time is longer, two frame similar for height can be by former frame
The result of (referred to as reference frame) is multiplexed and uses to a later frame, and the detection so as to avoid deep learning model to a later frame mentions
At high speed.For this purpose, we detect the similarity between successive frame by the way of Difference test, difference detector obtains successive frame
Histogram and calculate Histogram distance, further calculate out similarity.Then judge whether data can carry out according to similarity
Multiplexing.
Repeatability of the different periods inquiry in inquiry data equally also brings a large amount of redundant computation for system.Due to
It is not associated between inquiry, therefore inquiry each time all independently can make requests analysis to video data, bring
The huge wasting of resources.In response to this problem, it is stored for request results, when new finds out duplicate portion when inquiring and
Divide and is directly multiplexed.As shown in Figure 1, when new request arrives, be split as two parts to request first, reusable part with not
Reusable part.The data of reusable part are present in database, can directly be looked by the faster database of runing time
It askes operation and obtains result.For not reusable part, processing acquisition request result is carried out yet by deep learning model.Then
The result requested is fed back into database, in order to which subsequent query is multiplexed.It is assumed that contain m frame image in a video,
Shared l type objects are detectable and k personage can recognize that we store data using two matrixes of Dl*m and Rk*m,
When new inquiring comes, whether with the presence of corresponding data in inquiry Dl*m and Rk*m first, if thening be used directly, avoid
It computes repeatedly.Dij is there are j object to indicate that the object is not present for 0 in 1 the i-th frame picture of expression.
Step 2, the multiplexing of Spatial Dimension.It is calculated in environment shared, there is a large amount of concurrent inquiry operations, when this
When concurrent inquiry request is interested in same partial data a bit, just results in and largely compute repeatedly.The present invention is used and is asked
The mode for asking cutting combined merges request, crops lap therein, reduces multiple repetitive requests.Such as Fig. 2
It is shown.To partial data, there is overlapping with Q2 by two request inquiry Q1 to arrive simultaneously.In such a way that inquiry merges, rationally
Two requests are merged into a request Q1 ', avoid weight part from computing repeatedly.It is also one when multiple queries arrive simultaneously
Sample merges multiple queries, finds the lap between multi-request to the maximum extent to reduce repetition detection.
Step 3, the multiplexing of logical dimension.Diversified analysis mode is provided in video analytic system, including in video
Data carry out object detection, object identification, the various aspects such as object tracking, this just needs the deep learning model of multiplicity to provide not
Same detectability.However between these different models, a possibility that there is also multiplexings mutually.Most intuitively a bit,
If object detection can't detect people in certain frames, it can directly remove object identification model from the process of these frames identification people.Instead
It, if object identification has recognized someone, object detection can also remove to the detection in these frames for people.As shown in figure 3,
We be associated withs the data foundation of object detection and object identification, find between different models there are the data of relevance to carry out
Filtering mutually.For example, when object identification model recognizes people, it, also can be by data while updating object identification database
It feeds back to object detection database to be updated, will test result queue is someone.When new reasoning request inquiry whether someone
When can be directly filtered according to this result.We using in D1j record jth frame whether presence of people, use Rij record the
Whether there are personage i, R*j to indicate all people's object in jth frame in j frame.According to these properties, we have concluded that following two fixed
Reason.
Theorem one: if D1j is 0, as 0 < j≤m, R*j is also 0.
Theorem two: if Rij is 1, as 0 < j≤m, D1j is also 1.
Theorem one indicates, when jth frame detects nobody, then Model of Target Recognition will can't detect in jth frame
Anyone.Theorem two indicates that, when identifying personage i in jth frame, target detection matrix can automatically update someone in jth frame.By
This, when carrying out logical multiplexing, we can dynamically update database data according to theorem one and theorem two, be target detection and
Target identification establishes association.
Balance the precision and speed of inquiry
In current deep learning model, although analysis precision is stepping up, the bring with precision improvement
Resource consumption is also increasing, and this requires longer runing times.In general, model with high accuracy, the speed of service is slower,
The low model of precision, the speed of service are very fast.On the other hand, the photo resolution in video analysis process, the jump of difference detector
Frame number etc. also can all have an impact precision and speed.This is just that parameter configuration in analytic process brings challenge.Because
In practical application, different reasonings requests the requirement for precision and speed different.Component requests are as plundered kidnapping
Detection needs higher precision to keep correctness, can sacrifice partial velocity thus；And the request such as traffic lights detection
Higher timeliness is then needed, precision reaches suitable horizontal.For this purpose, during how reasonably selecting video analysis
Model, resolution ratio, the parameters such as frame-skipping number are the matters of utmost importance faced.
Step 1: being fitted to the precision of the different models under different parameters with length velocity relation, each model speed is obtained
The corresponding relationship of degree and precision.Optimal parameter can be selected to carry out video analysis, when inquiring and to reach user demand.
On the other hand, difference detector is that the measurement of precision brings uncertainty.Difference detector detect present frame with
The similarity of former frame (reference frame) is directly multiplexed the analysis of former frame as a result, to avoid generation in the similar situation of height
The high reasoning and calculation of valence.But directly multiplexing results in multiplexed result and actual result there are deviation, reduces and detects essence
Degree.So need to carry out effective accuracy evaluation to difference detector, it is accurately full so as to calculate whole effective accuracy
The accuracy requirement of sufficient user.Since difference detector is detected to continuous two frame of front and back, it is evident that the essence of a later frame
Degree and diversity factor between the precision and two frames of former frame are closely related.This characteristic between successive frame meets Markov
Chain rule.
Step 2:, by the diversity factor between the precision and two frames of former frame, adding phase according to Markov Chain rule
The adjustment parameter answered, to predict the precision of a later frame.The numerical value that adjustment parameter is constantly corrected by a large amount of confirmatory experiment, from
And it is able to carry out accurate precision evaluation.We represent the diversity factor between two frames using δ diff, represent i-1 using A (fi-1)
The precision of frame, uses k as adjustment parameter, then according to Markov Chain rule, the precision of the i-th frame are as follows:
A (fi-1)=k* δ diff*A (fi-1)
Finally, the result multiplexing between reasoning request is conditional multiplexing, and it is not complete multiplexing without a moment's thought, essence
The influence spent in multiplexing is even more important.For example, when one requires the request of high-precision result to arrive, already present low precision
As a result it obviously can not be multiplexed and be requested to high-precision.The problem of can not being multiplexed accordingly, there exist the data of different accuracy.Obviously most intuitive
Method be to abandon low precision as a result, detected again using high-precision model, but this can unquestionably bring larger hold
Pin.In view of the data of multiplexing are by deep learning model and the common bring of difference detector as a result, such as deep learning mould
Type detects first frame, and difference detector is for four frames after handling, and wherein deep learning model bring result precision is compared to difference
Different detector bring result precision wants high.For example, for same section of video data, it is assumed that inquiry request Q1 requires 90% essence
Degree, according to this requirement, precision velocity balance part selects frame-skipping step number to be detected for 5 frequency.That is, passing through depth every 5 frames
Degree learning model is once identified that 5 frames skipped obtain diversity factor by difference detector.After a period of time, new inquiry
Request Q2 arrival requires 95% precision, it is clear that the result of Q1 can not be directly used in Q2, at this point, learning that frame-skipping walks by calculating
Number can reach accuracy requirement when being 2.Firstly, the data in multiplexing Q1, that is, be multiplexed the frame detected, to using difference detector
5 frames skipped can reach the requirement that frame-skipping step number is 2 using the 3rd frame of deep learning model inspection in this way.It has been finally reached
The effect of condition multiplexing.
Step 3: retain deep learning model as a result, the partial results to difference detector are (a certain in such as rear four frame
Frame) it is detected again using high-precision model.And sufficiently it is multiplexed calculating knot of the difference detector about similarity between two frames
Fruit reappraises the precision between successive frame, to improve detection accuracy on the whole.
In summary several parts, it is as shown in Figure 4 for the data-reusing model general frame of more reasonings request.The system
Middle to provide diversified video analysis request, there are two main classes: object identification for the deep learning model that these requests are finally used
Model and object detection model.First.When the request with detection operation (or identification operation) arrives, according to Spatial Dimension
Relevance, system first merge request, crop the part that there is overlapping in request.Then, the Multiplexing module in figure
Part, according to the relevance of time dimension, request is retrieved in object detection database (or object identification database), is looked into
It askes and whether has suitable data for multiplexing.In addition, according to the relevance between logic, using object identification database (or object
Body Test database) in data reduction invalid computation is filtered to detection operation (or identification operate).When there is no reusables
Data when, call and configured the deep learning model of parameter and analyzed.Before calling deep learning model, root is first had to
Demand according to user about precision and speed configures relevant parameter by speed-precision balance portion, is meeting user
While accuracy requirement, reach speed as high as possible.The part inference in figure, need to relevant parameter such as resolution ratio,
Selection, frame-skipping rate of deep learning model etc. carry out reasonable disposition, and difference detector and deep learning model is then called to carry out
Analysis result is finally exported and is stored in data warehouse by video analysis.Precision improvement module then constantly updates the knot in database
Fruit precision, so that high-precision inquiry request can be multiplexed.
Claims (8)
1. a kind of multi-user's video flowing deep learning is shared to calculate multiplexing method, characterized in that firstly, when with detection operation or
When the request of identification operation arrives, according to the relevance of Spatial Dimension, request is merged, crops and there is overlapping in request
Part；Then, according to the relevance of time dimension, request is examined into object detection database or object identification database
Whether rope, inquiry have suitable data for multiplexing, when reusable data are not present, according to the relevance between logic,
It is invalid to operate or identify that operation is filtered reduction to detection using data in object identification database or object detection database
It calculates, recalls and configured the deep learning model of parameter and analyzed；For not reusable part, first according to speed essence
The equilibrium relation of degree finds most suitable parameter configuration in analytic process, then calls difference detector and deep learning accordingly
Model carries out video analysis, finally exports and be stored in data warehouse for analysis result, hoisting module is then used to be promoted in database
Original result precision is in order to being multiplexed to high-precision inquiry request.
2. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that need to phase
It closes parameter such as resolution ratio, the selection of deep learning model, frame-skipping rate and carries out reasonable disposition.
3. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that time dimension
Multiplexing specific steps:
1) similarity between successive frame is detected by the way of Difference test, difference detector obtains the histogram of successive frame simultaneously
Histogram distance is calculated, similarity is further calculated out, then judges whether data can be multiplexed according to similarity；
2) when new request arrives, two parts be split as to request first, reusable part with can not multiplexing part, can answer
It is present in database with the data of part, directly can obtain result by the faster database query operations of runing time；
For not reusable part, processing acquisition request is carried out as a result, the result that then will be requested yet by deep learning model
Database is fed back to, in order to which subsequent query is multiplexed.
4. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that Spatial Dimension
Multiplexing: using request cut merge by the way of request is merged, crop lap therein, reduce multiple weight
Multiple request.
5. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that logical dimension
Multiplexing: the data foundation of object detection and object identification be associated with, find between different models there are the data of relevance thus
It is filtered mutually.
6. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that logical dimension
Multiplexing specific steps:
Contain m frame image in one video, shared l type objects are detectable and k personage can recognize, use Dl*m and Rk*m two
Matrix stores data, and Dl*m is to represent object matrix, and Rk*m is representative figure matrix, when new inquiring comes,
It is inquired in Dl*m and Rk*m first whether with the presence of corresponding data, if thening be used directly, Dij is to deposit in 1 the i-th frame picture of expression
In j object, indicate that the object is not present for 0；
Theorem one: if D1j is 0, as 0 < j≤m, R*j is also 0；
Theorem two: if Rij is 1, as 0 < j≤m, D1j is also 1；
Theorem one indicate, when jth frame detects nobody, then Model of Target Recognition will can't detect in jth frame it is any
People；Theorem two indicates that, when identifying personage i in jth frame, target detection matrix can automatically update someone in jth frame, as a result,
When carrying out logical multiplexing, database data is dynamically updated according to theorem one and theorem two, is built for target detection and target identification
Vertical association.
7. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that balance inquiry
Precision and speed specific steps:
Step 1: be fitted to the precision of the different models under different parameters with length velocity relation, obtain each model velocity with
The corresponding relationship of precision can select optimal parameter to carry out video analysis, when inquiring and to reach user demand；
Step 2: according to Markov Chain rule, by the diversity factor between the precision and two frames of former frame, along with corresponding
Adjustment parameter is constantly corrected the numerical value of adjustment parameter by confirmatory experiment, thus allowed for predict the precision of a later frame
Accurate precision evaluation, specifically used δ diff represent the diversity factor between two frames, and the precision of i-1 frame is represented using A (fi-1),
Use k as adjustment parameter, then according to Markov Chain rule, the precision of the i-th frame are as follows:
A (fi-1)=k* δ diff*A (fi-1)
Finally, the result multiplexing between reasoning request is conditional multiplexing；
Step 3: retain deep learning model as a result, carrying out weight using high-precision model to the partial results of difference detector
New detection, and sufficiently calculated result of the multiplexing difference detector about similarity between two frames, reappraise between successive frame
Precision, to improve detection accuracy on the whole.
8. multi-user's video flowing deep learning as described in claim 1 is shared to calculate multiplexing method, characterized in that conditional
Multiplexing is specifically included using the frequency increased using deep learning model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910413748.7A CN110245267B (en) | 2019-05-17 | 2019-05-17 | Multi-user video stream deep learning sharing calculation multiplexing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910413748.7A CN110245267B (en) | 2019-05-17 | 2019-05-17 | Multi-user video stream deep learning sharing calculation multiplexing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110245267A true CN110245267A (en) | 2019-09-17 |
| CN110245267B CN110245267B (en) | 2023-08-11 |
Family
ID=67884171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910413748.7A Active CN110245267B (en) | 2019-05-17 | 2019-05-17 | Multi-user video stream deep learning sharing calculation multiplexing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110245267B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI776462B (en) * | 2020-04-30 | 2022-09-01 | 大陸商深圳市商湯科技有限公司 | Image processing method, electronic device and computer readable storage medium |
| US11488117B2 (en) * | 2020-08-27 | 2022-11-01 | Mitchell International, Inc. | Systems and methods for managing associations between damaged parts and non-reusable parts in a collision repair estimate |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009150425A2 (en) * | 2008-06-10 | 2009-12-17 | Half Minute Media Ltd | Automatic detection of repeating video sequences |
| CN102918531A (en) * | 2010-05-28 | 2013-02-06 | 甲骨文国际公司 | Systems and methods for providing multilingual support for data used with a business intelligence server |
| US9176987B1 (en) * | 2014-08-26 | 2015-11-03 | TCL Research America Inc. | Automatic face annotation method and system |
| CN107480178A (en) * | 2017-07-01 | 2017-12-15 | 广州深域信息科技有限公司 | A kind of pedestrian's recognition methods again compared based on image and video cross-module state |
| CN108235001A (en) * | 2018-01-29 | 2018-06-29 | 上海海洋大学 | A kind of deep-sea video quality objective assessment model based on space-time characteristic |
| CN108549846A (en) * | 2018-03-26 | 2018-09-18 | 北京航空航天大学 | A kind of movement character combined and head and shoulder structure pedestrian detection and statistical method |
| CN108846384A (en) * | 2018-07-09 | 2018-11-20 | 北京邮电大学 | Merge the multitask coordinated recognition methods and system of video-aware |
-
2019
- 2019-05-17 CN CN201910413748.7A patent/CN110245267B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009150425A2 (en) * | 2008-06-10 | 2009-12-17 | Half Minute Media Ltd | Automatic detection of repeating video sequences |
| CN102918531A (en) * | 2010-05-28 | 2013-02-06 | 甲骨文国际公司 | Systems and methods for providing multilingual support for data used with a business intelligence server |
| US9176987B1 (en) * | 2014-08-26 | 2015-11-03 | TCL Research America Inc. | Automatic face annotation method and system |
| CN107480178A (en) * | 2017-07-01 | 2017-12-15 | 广州深域信息科技有限公司 | A kind of pedestrian's recognition methods again compared based on image and video cross-module state |
| CN108235001A (en) * | 2018-01-29 | 2018-06-29 | 上海海洋大学 | A kind of deep-sea video quality objective assessment model based on space-time characteristic |
| CN108549846A (en) * | 2018-03-26 | 2018-09-18 | 北京航空航天大学 | A kind of movement character combined and head and shoulder structure pedestrian detection and statistical method |
| CN108846384A (en) * | 2018-07-09 | 2018-11-20 | 北京邮电大学 | Merge the multitask coordinated recognition methods and system of video-aware |
Non-Patent Citations (6)
| Title |
|---|
| ROBERTO CAMACHO BARRANCO: "Enhancing Yelp Data with Deep Learning and Information Reuse", 《2017 IEEE INTERNATIONAL CONFERENCE ON INFORMATION REUSE AND INTEGRATION (IRI)》 * |
| 万南洋: "数据仓库事实表的建模理论与方法", 《计算机工程》 * |
| 文孟飞等: "网络媒体大数据流异构多模态目标识别策略", 《计算机研究与发展》 * |
| 汤善江等: "层级式可视化并行程序建模系统研究", 《计算机工程》 * |
| 高洁等: "基于增量式Bayes的中文网页自动分类技术", 《电脑知识与技术(学术交流)》 * |
| 高翔等: "基于视频场景深度学习的人物语义识别模型", 《计算机技术与发展》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI776462B (en) * | 2020-04-30 | 2022-09-01 | 大陸商深圳市商湯科技有限公司 | Image processing method, electronic device and computer readable storage medium |
| US11488117B2 (en) * | 2020-08-27 | 2022-11-01 | Mitchell International, Inc. | Systems and methods for managing associations between damaged parts and non-reusable parts in a collision repair estimate |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110245267B (en) | 2023-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106295594B (en) | A kind of across camera method for tracking target and device based on dynamic route tree | |
| CN110245267A (en) | Multi-user's video flowing deep learning is shared to calculate multiplexing method | |
| CN107239203A (en) | A kind of image management method and device | |
| CN107423754B (en) | Automatic radiation source identification system based on parameter multi-attribute autonomous intelligent decision | |
| CN109117856A (en) | People and object method for tracing, apparatus and system based on intelligent edge cloud | |
| JP7143260B2 (en) | Methods and systems for assisting users in creating and selecting images | |
| CN102136147A (en) | Target detecting and tracking method, system and video monitoring device | |
| CN111739056B (en) | Track tracking system | |
| CN108287550A (en) | The method of SLAM systems and construction data correlation based on data correlation and error detection | |
| CN114153795A (en) | Method and device for intelligently calling electronic archive, electronic equipment and storage medium | |
| CN111107319A (en) | Target tracking method, device and system based on regional camera | |
| CN110232133A (en) | A kind of image of clothing search method and system classified based on Fusion Features and style | |
| CN110992124A (en) | House resource recommendation method and system | |
| CN109411093A (en) | A kind of intelligent medical treatment big data analysis processing method based on cloud computing | |
| CN114879891A (en) | Multi-mode man-machine interaction method under self-supervision multi-target tracking | |
| CN112288530B (en) | Resource sharing intelligent dining method, device, system, medium and equipment | |
| Wang et al. | You only need two detectors to achieve multi-modal 3d multi-object tracking | |
| CN111061911B (en) | Target detection and tracking method, device and equipment for multi-video monitoring data | |
| CN110059658B (en) | Remote sensing satellite image multi-temporal change detection method based on three-dimensional convolutional neural network | |
| US10956741B2 (en) | System for automatic monitoring of birds and related method of use | |
| US20230076241A1 (en) | Object detection systems and methods including an object detection model using a tailored training dataset | |
| CN115062676A (en) | Data processing method, device and computer readable storage medium | |
| CN115129659A (en) | Main and auxiliary kernel computer processing platform | |
| US11610080B2 (en) | Object detection improvement based on autonomously selected training samples | |
| CN109389053A (en) | High performance vehicle detection system based on vehicle perpendicular type feature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |