CN107729901A - Method for building up, device and the image processing method and system of image processing model - Google Patents
Method for building up, device and the image processing method and system of image processing model Download PDFInfo
- Publication number
- CN107729901A CN107729901A CN201610652942.7A CN201610652942A CN107729901A CN 107729901 A CN107729901 A CN 107729901A CN 201610652942 A CN201610652942 A CN 201610652942A CN 107729901 A CN107729901 A CN 107729901A
- Authority
- CN
- China
- Prior art keywords
- image
- model
- matrix
- provincial characteristics
- image processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
This application provides a kind of method for building up of image processing model, device and a kind of image processing method and system, the technological thought of this four schemes is consistent, wherein, image processing system includes：Image processing equipment and matrix multiplier；Image processing equipment is configured with the image processing model pre-established, and the image processing model includes the first model and the second model；The image processing equipment, pending image is handled for handling model by described image, obtains the provincial characteristics matrix and provincial characteristics weight matrix of the pending image；The matrix multiplier, provincial characteristics matrix and provincial characteristics weight matrix for the pending image make matrix multiple processing, obtain the feature of the pending image.The system can rapidly realize the feature extraction end to end of " image to target signature ", and wherein, image processing model establishes process, greatly reduces the demand to sample size, so as to reduce the artificial mark cost of early stage.
Description
Technical field
The application is related to technical field of image processing, more particularly to a kind of method for building up of image processing model, Yi Zhongtu
Device, a kind of image processing method and a kind of image processing system are established as processing model.
Background technology
In recent years, with the continuous development of development of Mobile Internet technology, search entrance is from traditional text search to figure
As search transformation, picture search has been developed well at present, also has become a kind of way of search that user commonly uses.Example
Such as：In e-commerce field, user can be by " to scheme to search figure " way of search come search commercial articles.
The realization of picture search rely primarily on iamge description algorithm, at present, conventional iamge description algorithm mainly includes
Two modules, detection module and characteristic extracting module；Wherein, detection module is used to from original image to need to extract feature
Target part, which plucks out, to be come, to remove ambient interferences.Characteristic extracting module, carried for being plucked out from monitoring modular in the target part come
Take feature.When realizing, it is necessary to ad hoc train detection module and characteristic extracting module, and the training process of the two modules is
Separation, but but be sample correlation.
, it is necessary to manually (typically all must Seeking Truth hundred for the substantial amounts of training sample of training mark of detection module in training
Ten thousand grades of sample sizes for arriving millions)；Detection module is first trained, recycles the training result of detection module, artificial mark feature
Training sample needed for extraction module, for artificial, mark workload is larger, also, the training of characteristic extracting module is
Inchoate after detection module training is completed, therefore, whole training flow is longer, and time cost is higher.
The content of the invention
Technical problems to be solved in this application are to provide a kind of method for building up of image processing model, can be rapidly real
Now model is established end to end, reduces the workload of artificial mark sample, can either improve the efficiency of model foundation, and and can is enough protected
The reliability of model of a syndrome.
In addition, establish device, a kind of image processing method and one kind that the application also provides a kind of image processing model are schemed
As processing system, to ensure the realization and application of the above method in practice.
A kind of method for building up of image processing model is provided in the application first aspect, this method includes：
The sample image marked in advance is inputted to the first model and learnt, obtains each image in the sample image
Provincial characteristics matrix, the provincial characteristics matrix is used to characterize target signature situation in the image in each region；
The provincial characteristics Input matrix of each image is learnt into the second model, obtains each image
Provincial characteristics weight matrix, the provincial characteristics weight matrix is used to characterize the power in each region in the provincial characteristics matrix
Weight, the weight characterize the conspicuousness of target signature in the region；
According to the provincial characteristics matrix of each image and corresponding provincial characteristics weight matrix, it is calculated described every
The feature of individual image；
The essence of the feature of the image in each feature of image and the sample image marked in advance is obtained after study
When degree tends towards stability, it is determined that the image processing model for learning to obtain includes：First model and second model.
Device is established what the application second aspect provided a kind of image processing model, the device includes：
First model learning module, is learnt for the sample image marked in advance to be inputted to the first model, is obtained
The provincial characteristics matrix of each image in the sample image, the provincial characteristics matrix are used to characterize each region in the image
Interior target signature situation；
Second model learning module, for the provincial characteristics Input matrix of each image to be carried out into the second model
Study, obtains the provincial characteristics weight matrix of each image, and the provincial characteristics weight matrix is used to characterize region spy
The weight in each region in matrix is levied, the weight characterizes the conspicuousness of target signature in the region；
Computing module, for the provincial characteristics matrix according to each image and corresponding provincial characteristics weight matrix,
The feature of each image is calculated；
Determining module, for obtaining the figure in each feature of image and the sample image marked in advance after study
When the precision of the feature of picture tends towards stability, it is determined that the image processing model for learning to obtain includes：First model and described
Second model.
A kind of image processing system is provided in the application third aspect, the system includes：
Image processing equipment and matrix multiplier；
Wherein, described image processing equipment communicates with the matrix multiplier；
Described image processing equipment is configured with the image processing model pre-established, and described image processing model includes first
Model and the second model；First model is that image can be learnt to obtain the provincial characteristics matrix norm type of image；
Second model is that the provincial characteristics matrix of image can be learnt to obtain the mould of the provincial characteristics weight matrix of image
Type；
Described image processing equipment, pending image is handled for handling model by described image, obtains institute
The provincial characteristics matrix and provincial characteristics weight matrix of pending image are stated, and the provincial characteristics matrix and provincial characteristics are weighed
Output matrix is weighed to the matrix multiplier；
The matrix multiplier, provincial characteristics matrix and provincial characteristics for the pending image to receiving are weighed
Weight matrix makees matrix multiple processing, obtains the feature of the pending image.
In the application fourth aspect, there is provided a kind of image processing method, methods described include：
Pending image, which is inputted to the image processing model pre-established, described image processing model, includes the first model
With the second model；
The first model in model is handled by described image the pending image is handled to obtain and described wait to locate
The provincial characteristics matrix of image is managed, and the provincial characteristics Input matrix to second model is obtained into the pending image
Provincial characteristics weight matrix；
According to the provincial characteristics matrix of the pending image and the provincial characteristics weight matrix of the pending image, meter
Calculation obtains the feature of the pending image.
Compared with prior art, the technical scheme that the application provides has advantages below：
The technical scheme that the application provides proposes the mode of establishing of image processing model end to end, and mould end to end
Type study only needs the sample image with characteristic attribute marked in advance.And it is understood that prior art when realizing, both needed
The training that be directed to detection module marks the sample image largely with target information in advance, needs to be directed to feature extraction mould again
The training of block marks the sample image largely with characteristic attribute in advance, therefore, compared with prior art, the technology of the application
Scheme does not need so much sample image, reduces the markers work amount of nearly half.
In addition, the image processing model of prior art includes：Detection module and characteristic extracting module；Wherein, detection module
The target area being mainly capable of detecting when in image；And characteristic extracting module is that the target of image is extracted for target area
Feature.With prior art except that, the model of image processing model first and the second model that the application provides, wherein, the
One model is mainly the model that the feature for the regional location that can be directed to image is learnt；And the second model mainly being capable of pin
The model learnt to the feature weight of the regional location of image.
It can be seen that：Being combined for the two models can just learn the target signature situation for obtaining image, can realize from " figure
As directly to feature " " end-to-end " learn, compared with prior art from " image to target area, from target area to spy
The Layered Learning of sign ", this mode of learning of the application have the advantages of reliability of learning efficiency height, learning model.
Brief description of the drawings
In order to illustrate more clearly of the technical scheme in the embodiment of the present application, make required in being described below to embodiment
Accompanying drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the present application, for
For those of ordinary skill in the art, without having to pay creative labor, it can also be obtained according to these accompanying drawings
His accompanying drawing.
Fig. 1 is a kind of structure chart for image processing system embodiment 1 that the application provides；
Fig. 2 is a kind of structure chart for image processing system embodiment 2 that the application provides；
Fig. 3 is a kind of flow chart of the method for building up for image processing model that the application provides；
Fig. 4 is a kind of flow chart for image processing method that the application provides；
Fig. 5 is a kind of structure chart for establishing device for image processing model that the application provides.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present application, the technical scheme in the embodiment of the present application is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only some embodiments of the present application, rather than whole embodiments.It is based on
Embodiment in the application, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of the application protection.
The application can be used in numerous general or special purpose computing device environment or configuration.Such as：Personal computer, service
Device computer, handheld device or portable set, laptop device, multi-processor device including any of the above device or equipment
DCE etc..
The application can be described in the general context of computer executable instructions, such as program
Module.Usually, program module includes performing particular task or realizes routine, program, object, the group of particular abstract data type
Part, data structure etc..The application can also be put into practice in a distributed computing environment, in these DCEs, by
Task is performed and connected remote processing devices by communication network.In a distributed computing environment, program module can be with
In the local and remote computer-readable storage medium including storage device.
The technical scheme that the application provides mainly there is provided a kind of new image processing model, and institute is different from prior art
, this new image processing model of the application is that the first model and the second model are combined to be formed, and the first model
It is entirely different with the respective function of the second model and measurement module of the prior art and characteristic extracting module；One side the application
The method for building up (it can be appreciated that training method or learning method) for this new image processing model is provided, should
Method and prior art except that, the no longer training for each module marks substantial amounts of sample image respectively, and at this
During the whole foundation of application, it is only necessary to the sample image with characteristic attribute marked in advance.This just subtracts significantly
The demand of sample size is lacked, so as to reduce the artificial mark cost of early stage.Another aspect the application is based on this new image
Processing model additionally provides a kind of image processing system, and the end of " image to target signature " can be rapidly realized based on the system
To the feature extraction at end, can be well adapted for great amount of images needs in business scenario to be processed, such as can be preferably
It is adapted to e-commerce platform.On the other hand in order to ensure the application and realization of the method for building up of the application in practice, this Shen
Please additionally provide a kind of image processing model establishes device.
The technical scheme of the application is best understood by order to facilitate those skilled in the art, next, first being carried to the application
The image processing system of confession is explained.
Referring to Fig. 1, Fig. 1 is a kind of structure chart for image processing system that the application provides, as shown in figure 1, the system 100
Including：
Image processing equipment 101 and matrix multiplier 102；
Wherein, described image processing equipment 101 communicates with the matrix multiplier 102；
Described image processing equipment is configured with the image processing model pre-established, and described image processing model includes first
Model and the second model；
Wherein, first model is that image can be learnt to obtain the provincial characteristics matrix norm type of image；Institute
It is that the provincial characteristics matrix of image can be learnt to obtain the model of the provincial characteristics weight matrix of image to state the second model；
When realizing, described image processing model can be the method for building up of the image processing model provided using the application
The image processing model established.
Described image processing equipment 101, pending image is handled for handling model by described image, obtained
The provincial characteristics matrix and provincial characteristics weight matrix of the pending image, and by the provincial characteristics matrix and provincial characteristics
Weight matrix is exported to the matrix multiplier；
The matrix multiplier 102, it is special for the provincial characteristics matrix of the pending image to receiving and region
Sign weight matrix makees matrix multiple processing, obtains the feature of the pending image.
Based on the system 100 shown in Fig. 1, when there is image to need to extract target signature, then the image is inputted to system
100 image processing equipment 101, then, it is trained to obtain the image by the first model 1011 in image processing equipment 101
Provincial characteristics matrix；Then, obtained provincial characteristics Output matrix will be trained to the He of the second model 1012 by the first model 101
Matrix multiplier 102, it is trained to obtain the provincial characteristics weight matrix A of the image by the second model 1012, and region is special
Sign weight matrix B is exported to matrix multiplier 102.Finally, by matrix multiplier 102 to the provincial characteristics of the image received
Matrix A and provincial characteristics weight matrix B are handled to obtain target signature Matrix C as matrix multiple, and the target signature Matrix C just can
Characterize the target signature situation that the image is included.
In addition, on the basis of above-mentioned system shown in Figure 1, present invention also provides a kind of more practical system, referring to Fig. 2
The system 200 shown, the difference of the system 200 and system shown in Figure 1 100 are, in the system shown in Fig. 2, image procossing
The image processing model of configuration gives more practical in equipment, more specifically model structure, it is proposed that uses full convolutional Neural net
Network module merges the model structure of attention model, can either preferably play full convolutional neural networks model learning ability height,
The fast advantage of pace of learning, and can enough preferably play the advantage of attention model Fast Learning region significance so that whole
Image processing model plays preferable treatment effect.
As shown in Fig. 2 the system 200 includes：
Image processing equipment 201 and matrix multiplier 202；
Wherein, described image processing equipment 201 communicates with the matrix multiplier 202；
Described image processing equipment 201 is configured with the image processing model pre-established, and described image processing model includes
Full convolutional neural networks model and attention model；
Described image processing equipment 201, pending image is handled for handling model by described image, obtained
The provincial characteristics matrix and provincial characteristics weight matrix of the pending image, and by the provincial characteristics matrix and provincial characteristics
Weight matrix is exported to the matrix multiplier；
The matrix multiplier 202, it is special for the provincial characteristics matrix of the pending image to receiving and region
Sign weight matrix makees matrix multiple processing, obtains the feature of the pending image.
Next, with reference to the practical application scene of e-commerce platform, the operation principle of said system 200 is explained
Explanation.
Need to carry out feature extraction to clothing image on e-commerce platform, the specific clothes extracted in image are special
Sign.Based on such business demand, on the electronic goods platform processed clothing image can will be needed to be separately input into
Image processing equipment 101 in said system 100, then, due to the full convolutional neural networks model in image processing equipment 101
2011 pairs of clothing images are learnt to obtain the provincial characteristics matrix of the image, then the provincial characteristics matrix can just characterize this
The feature situation of the regional location of class image is serviced, feature here can include：Collar shape, collar color, cuff shape,
The characteristic information related to the attribute of specific clothes such as cuff color, coat-sleeve length.
Then, the provincial characteristics matrix of clothing image is learnt by attention model 2012 to obtain the clothing figure
The provincial characteristics weight matrix of picture, the provincial characteristics weight matrix just can the making the grade service class image regional location target
The conspicuousness of feature, mainly embodied with weight size corresponding to regional location.
Based on this, the provincial characteristics matrix of the clothing image and corresponding provincial characteristics are weighed by matrix multiplier 202
Weight matrix obtains target signature matrix as matrix multiple, and the target signature matrix can just obtain what the clothing image was included
Target signature situation.
Above-mentioned example is only explained by taking clothing product image in e-commerce platform as an example, and the application is implemented
The system that example provides goes in any required platform that feature extraction is carried out to image, can be to other kinds of image
Handled, it is not limited to the processing of above-mentioned service class product image.
Explanation is needed exist for, in order to be adapted to different application scenarios, the image processing equipment in said system can
To be pointedly configured with the image processing model pre-established using the sample image of concrete application scene；Set in image procossing
An image processing model can be configured with standby, multiple images processing model can also be configured with.Certainly, set with image procossing
What standby function matched, we can be configured with a matrix multiplier in systems, can also be directed to each image procossing
Model configures a corresponding matrix multiplier.Certainly, in systems, multiple images processing model can also be directed to and only configure one
Individual matrix multiplier, then multiple images, which handle model, can be total to same matrix multiplier.So understand：Shown in Fig. 1, Fig. 2
Structure chart is for only for ease of the understanding of those skilled in the art, and the structure for the system that the application provides is not limited to Fig. 1
And Fig. 2.
In addition, when realizing, image processing equipment and matrix multiplier in Fig. 1, Fig. 2 can be integrated in same hardware and set
In standby, independent hardware device can also be deployed to.
Its work of said system 100,200 depends on the image processing model configured, and how the application is for build
This image processing model is found, corresponding method for building up and device are additionally provided, next, first to this image processing model
Method for building up is introduced.
Referring to Fig. 3, Fig. 3 shows a kind of flow chart of the method for building up for image processing model that the application provides, such as Fig. 3 institutes
Show, this method can include：Step 301 is to step 304：
Step 301：The sample image marked in advance is inputted to the first model and learnt, is obtained in the sample image
The provincial characteristics matrix of each image, the provincial characteristics matrix are used to characterize the target signature feelings in the image in each region
Condition.
When realizing, a number of sample marked in advance can be collected according to the demand of the model learning of reality
Image, then these sample images being collected into are inputted to the first model and learnt, the first model, can when training beginning
With random initializtion, then, these sample images are learnt successively, obtain each self-corresponding provincial characteristics square of each image
Battle array.
In the embodiment of the present application, the first model can use existing model structure, as long as the model can be directed to figure
As being learnt, study obtains the provincial characteristics situation of image.
Inventor has been proposed that in the embodiment of the present application that the first model is using full volume for the specific implementation of the first model
Product Artificial Neural Network Structures, full convolutional neural networks only have process of convolution, non-full connection, can guarantee that the independence of provincial characteristics
Property；In addition, full convolutional neural networks have simple in construction, training parameter few and the characteristics of strong adaptabilities.
When realizing, the embodiment of the present application can support the image of any form, these forms include but is not limited to JPG,
PNG, TIF, BMP etc..Certainly, when realizing, in order to ensure the uniformity of image procossing and processing speed, can also receive
During sample image, sample image is first converted into unified a kind of form that system supported, then carries out respective handling again.When
So, for the process performance of adaptive system, different size of sample image can also be directed to, is first cut into system support
The image of fixed size, respective handling then is carried out to image again.
When realizing, in order to be adapted to e-commerce platform, each image is marked in the sample image marked in advance
Target signature includes：The product attribute feature of the affiliated product classification of the image.For example, clothes class image, its described product classification
It is clothing, then the target signature that this kind of image is marked includes clothes attributive character.
Further, in order to provide more, more effectively sample characteristics, each image in the sample image marked in advance
The target signature being marked can also include：The similarity relation feature of the image and another image.
For example, image A is 90% with image B similarities, image A and image C similarity is 0%；These images are similar
Relationship characteristic, the relation of its internal feature can be characterized indirectly from the entirety of two images.On the basis of specific product feature
On, it can also be characterized as that sample data carries out model learning with the similarity relation of image so that the model learnt is more reliable, property
Can be more high-quality.
Step 302：The provincial characteristics Input matrix of each image is learnt into the second model, obtained described
The provincial characteristics weight matrix of each image, the provincial characteristics weight matrix are used to characterize each area in the provincial characteristics matrix
The weight in domain, the weight characterize the conspicuousness of target signature in the region.
In the embodiment of the present application, the second model can use existing model structure, as long as the model can be directed to figure
As being learnt, study obtains the notable implementations of the provincial characteristics of phenogram picture.
Inventor has been proposed that in the embodiment of the present application that the second model is using attention for the specific implementation of the second model
Power model structure, attention model are also referred to as visual attention model, and the model is one kind computer simulation human vision
Pay attention to the model of Force system, in piece image extract human eye it is observed that attractive feature, for computer
Speech, it is exactly the salient region feature of the image.Attention model is mainly capable of determining that the area that conspicuousness is higher in image
Domain, carry out the height of the conspicuousness in region in phenogram picture with weight size in this application.It can be passed through using attention model
The weight size of regional position, reaches rejection image ambient interferences, and reduces the purpose of unessential information weight, so as to
Being capable of the great small objectives characteristic extracted in image of exploitation right.
Explanation is needed exist for, for the ease of calculating, improves learning efficiency, when realizing, first can be pre-set
Model and the matrix size specification of the second model output are identical, and the provincial characteristics matrix size that such as the first model learning arrives is N*M,
Then the second model learning to provincial characteristics weight matrix size be similarly N*M, wherein, N and M are just whole more than or equal to 2
Number, N and M can be identical with value, can also value difference.
Certainly, the matrix size of output of the realization of the embodiment of the present application to the two models can not also be made specifically
Ask.But if the provincial characteristics matrix of the first model output and the provincial characteristics weight matrix that the second model exports are of different sizes
When, the matrix of formed objects can be converted into by way of matrixing, so, is easy to the calculating in step 303 to grasp
Make.
Step 303：According to the provincial characteristics matrix of each image and corresponding provincial characteristics weight matrix, calculate
To the feature of each image.
When realizing, step 203 can be realized using matrix multiplier, so that the features of each image are calculated
Point.Wherein, matrix multiplier is the multiplier for referring to realize matrix multiple.
Handled, can be obtained for the multiple study of sample image corresponding by the multiplicating of above-mentioned steps 301 to 303
Characteristic.After study, decide whether to stop image processing model of the study to obtain succeeding in school by step 304.
Step 304：The image in each feature of image and the sample image marked in advance is obtained after study
When the precision of feature tends towards stability, it is determined that the image processing model for learning to obtain includes：First model and described second
Model.
When realizing, the embodiment of the present application can also judge each figure obtained after the study in the following manner
Whether the precision of the feature of the figure tends towards stability in the feature of picture and the sample image marked in advance：
Calculate study every time and obtain the feature of the image in each feature of image and the sample image marked in advance
Precision between the two, the precision using the precision as this learning model；
Judge whether the amplitude of variation of the precision of multiple learning model is less than default amplitude of variation threshold value, if it is, really
Both features of the image in each feature of image and the sample image marked in advance are obtained after the fixed repeatedly study
Precision tend towards stability.
When the precision for judging multiple learning model tends towards stability, when being no longer mutated, then illustrate the first mould now learnt
Type and the second model have reached stable state, without being further continued for learning, now, it is possible to it is determined that the image procossing mould succeeded in school
Type includes：First model and the second model.This judgment mode, effectively Schistosomiasis control it can either obtain a performance and preferably scheme
As processing model, and can enough saves the sample learning time as far as possible.
When realizing, a learning cycle can be pre-set (such as learning cycle is iterations 1000 times), then, when
When the iterations of model learning reaches 1000 times, just all sample images marked in advance are judged by above-mentioned steps 304
Learn precision, if precision is lifted very small in learning process, then the model for being considered as currently learning has reached stable
State, it can stop learning.
In addition, present inventor is additionally contemplates that the sample image marked in advance is all handmarking, and handmarking
Sample image mark error sometimes occurs, the problems such as feature marking error, image is marked for the ease of artificial correction, with
The sample image of better quality is provided for model training next time, based on the above method, following steps can also be increased：
When both features of the image in the feature of some sample image that study obtains and the sample image marked in advance
Between precision when being less than default precision threshold, then used to the artificial monitoring system feeder alert information in backstage, the warning information
Mark be present in some described sample image of prompting.
Based on the warning information, backstage personnel pointedly can repair to the mark situation of suspicious sample image
Just, to ensure the accuracy of image tagged.
The method for building up for the image processing model that the application provides, it is proposed that a kind of image processing model end to end is built
Cube formula, and model learning only needs the sample image with characteristic attribute marked in advance end to end.And it is understood that existing
There is technology when realizing, both need the training for detection module to mark the sample image largely with target information in advance,
Mark largely has the sample image of characteristic attribute in advance for the training for needing again for characteristic extracting module, therefore, and existing
Technology is compared, and the technical scheme of the application does not need so much sample image, reduces the markers work amount of nearly half.
In addition, the image processing model of prior art includes：Detection module and characteristic extracting module；Wherein, detection module
The target area being mainly capable of detecting when in image；And characteristic extracting module is that the target of image is extracted for target area
Feature.Except that, the image processing model that the application provides includes with prior art：First model and the second model, its
In, the first model is mainly the model that the feature for the regional location that can be directed to image is learnt；And the second model is mainly
The model that the feature weight for the regional location that image can be directed to is learnt.
It can be seen that：First model and the second model, being combined for the two models can just learn the target for obtaining image spy
Sign situation, can realize and learn from " end-to-end " of " image is directly to feature ", compared with prior art from " image to target
The Layered Learning in region, from target area to feature ", this mode of learning of the application have learning efficiency height, learning model
Reliability the advantages of.
Based on the method for building up of above-mentioned image processing model, present invention also provides a kind of image processing method, this method
Feature extraction mainly is carried out to image using the image processing model, is mainly used in the system shown in above-mentioned Fig. 1, Fig. 2,
The image processing method is explained below.
Referring to Fig. 4, Fig. 4 shows a kind of flow chart for image processing method that the application provides, and this method is applied to profit
In the image processing model that the method shown in Fig. 2 is established, this method can include：Step 401 is to step 403：
Step 401：Pending image, which is inputted to the image processing model pre-established, described image processing model, to be included
First model and the second model；
Step 402：The first model in model is handled by described image the pending image is handled to obtain
The provincial characteristics matrix of the pending image, and the provincial characteristics Input matrix to second model is obtained into described treat
Handle the provincial characteristics weight matrix of image；
Step 403：Weighed according to the provincial characteristics of the provincial characteristics matrix of the pending image and the pending image
Weight matrix, the feature of the pending image is calculated.
When the above method is suitable for the system shown in above-mentioned Fig. 2, first model in the image processing model is adopted
With full convolutional neural networks model；Second model in the image processing model uses attention model.That is, at the image
Reason model includes full convolutional neural networks model and attention model.
In addition, corresponding with the method for building up of above-mentioned image processing model, the embodiment of the present application additionally provides corresponding
Image processing model establishes device, for realizing the above method.The device is explained with reference to Fig. 5.
Fig. 5 is that a kind of image processing model that the application provides establishes device, as shown in figure 5, the device can include：
First model learning module 501, the second model learning module 502, computing module 503 and determining module 504, below based on this
The function and annexation of its inside modules are explained the operation principle of device.
First model learning module 501, is learnt for the sample image marked in advance to be inputted to the first model, is obtained
The provincial characteristics matrix of each image into the sample image, the provincial characteristics matrix are used to characterize each area in the image
Target signature situation in domain；
Second model learning module 502, for by the provincial characteristics Input matrix of each image into the second model
Learnt, obtain the provincial characteristics weight matrix of each image, the provincial characteristics weight matrix is used to characterize the area
The weight in each region, the weight characterize the conspicuousness of target signature in the region in characteristic of field matrix；
Computing module 503, for the provincial characteristics matrix according to each image and corresponding provincial characteristics weight square
Battle array, the feature of each image is calculated；
Determining module 504, for being obtained after study in each feature of image and the sample image marked in advance
When the precision of the feature of the image tends towards stability, it is determined that the image processing model for learning to obtain includes：First model and
Second model.
When realizing, the first model can be full convolutional neural networks mould used by the first model learning module
Type.
When realizing, the second model can be attention model used by the second model learning module.
When realizing, the target signature that each image is marked in the sample image marked in advance includes：The image
The product attribute feature of affiliated product classification.
Further, the target signature that each image is marked in the sample image marked in advance also includes：The figure
As the similarity relation feature with another image.
When realizing, described device can also include：Judge module, it is described each for judging to obtain after the study
Whether the precision of the feature of the figure tends towards stability in the feature of image and the sample image marked in advance；
The judge module includes：
Accuracy computation submodule, the sample for obtaining the feature of each image for calculating study every time and marking in advance
The precision of the feature of the image between the two in image, the precision using the precision as this learning model；
Determination sub-module, for judging whether the amplitude of variation of precision of multiple learning model is less than default amplitude of variation threshold
It is worth, if it is, obtaining being somebody's turn to do in each feature of image and the sample image marked in advance after determining the repeatedly study
The precision of both features of image tends towards stability.
When realizing, described device can also include：
Alarm module, for when the figure in the feature for some sample image for learning to obtain and the sample image marked in advance
When the precision of the feature of picture between the two is less than default precision threshold, then to the artificial monitoring system feeder alert information in backstage, institute
Warning information is stated to be used to prompt some described sample image mark to be present.
The image processing model that the application provides establishes device, it is proposed that a kind of image processing model end to end is built
Cube formula, and model learning only needs the sample image with characteristic attribute marked in advance end to end.And it is understood that existing
There is technology when realizing, both need the training for detection module to mark the sample image largely with target information in advance,
Mark largely has the sample image of characteristic attribute in advance for the training for needing again for characteristic extracting module, therefore, and existing
Technology is compared, and the technical scheme of the application does not need so much sample image, reduces the markers work amount of nearly half.
In addition, the image processing model of prior art includes：Detection module and characteristic extracting module；Wherein, detection module
The target area being mainly capable of detecting when in image；And characteristic extracting module is that the target of image is extracted for target area
Feature.Except that, the image processing model that the application provides includes with prior art：First model and the second model, its
In, the first model is mainly the model that the feature for the regional location that can be directed to image is learnt；And the second model is mainly
The model that the feature weight for the regional location that image can be directed to is learnt.
It can be seen that：First model and the second model, being combined for the two models can just learn the target for obtaining image spy
Sign situation, can realize and learn from " end-to-end " of " image is directly to feature ", compared with prior art from " image to target
The Layered Learning in region, from target area to feature ", this mode of learning of the application have learning efficiency height, learning model
Reliability the advantages of.
It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment weight
Point explanation is all difference with other embodiment, between each embodiment identical similar part mutually referring to.
For device class embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is joined
See the part explanation of embodiment of the method.
Finally, it is to be noted that, herein, such as first, second, third, fourth or the like relational terms
It is used merely to make a distinction an entity or operation with another entity or operation, and not necessarily requires or imply these
Any this actual relation or order be present between entity or operation.Moreover, term " comprising ", "comprising" or its is any
Other variants are intended to including for nonexcludability, so that process, method, article or equipment including a series of elements
Not only include those key elements, but also the other element including being not expressly set out, or also include for this process, side
Method, article or the intrinsic key element of equipment.In the absence of more restrictions, limited by sentence "including a ..."
Key element, it is not excluded that other identical element in the process including the key element, method, article or equipment also be present.
Method for building up to a kind of image processing model provided herein, a kind of foundation of image processing model above
Device, a kind of image processing method and a kind of image processing system are described in detail, specific case used herein
The principle and embodiment of the application are set forth, the explanation of above example is only intended to help the side for understanding the application
Method and its core concept；Meanwhile for those of ordinary skill in the art, according to the thought of the application, in embodiment
And there will be changes in application, in summary, this specification content should not be construed as the limitation to the application.
Claims (10)
1. a kind of method for building up of image processing model, it is characterised in that methods described includes：
The sample image marked in advance is inputted to the first model and learnt, obtains the area of each image in the sample image
Characteristic of field matrix, the provincial characteristics matrix are used to characterize the target signature situation in the image in each region；
The provincial characteristics Input matrix of each image is learnt into the second model, obtains the area of each image
Characteristic of field weight matrix, the provincial characteristics weight matrix are used for the weight for characterizing each region in the provincial characteristics matrix, should
Weight characterizes the conspicuousness of target signature in the region；
According to the provincial characteristics matrix of each image and corresponding provincial characteristics weight matrix, each figure is calculated
The feature of picture；
The precision that the feature of the image in each feature of image and the sample image marked in advance is obtained after study becomes
When stablizing, it is determined that the image processing model for learning to obtain includes：First model and second model.
2. the method for building up of image processing model according to claim 1, it is characterised in that
First model uses full convolutional neural networks model；
Second model uses attention model.
3. the method for building up of image processing model according to claim 1, it is characterised in that
The target signature that each image is marked in the sample image marked in advance includes：The affiliated product classification of the image
Product attribute feature.
4. the method for building up of image processing model according to claim 3, it is characterised in that
The target signature that each image is marked in the sample image marked in advance also includes：The image and another image
Similarity relation feature.
5. the method for building up of image processing model according to claim 1, it is characterised in that judge institute in the following manner
State the feature of each image obtained after study and the sample image that marks in advance in the figure feature precision whether
Tend towards stability：
Calculate study every time and obtain both features of the image in each feature of image and the sample image marked in advance
Between precision, the precision using the precision as this learning model；
Judge whether the amplitude of variation of the precision of multiple learning model is less than default amplitude of variation threshold value, if it is, determining institute
The essence of both features of the image in each feature of image and the sample image marked in advance is obtained after stating repeatedly study
Degree tends towards stability.
6. the method for building up of image processing model according to claim 1, it is characterised in that methods described also includes：
When the image in the feature of some sample image that study obtains and the sample image marked in advance feature between the two
Precision when being less than default precision threshold, then be used to carry to the artificial monitoring system feeder alert information in backstage, the warning information
Show that some described sample image has mark.
7. a kind of image processing method, it is characterised in that established applied to method any one of the claims 1 to 6
Image processing model in, methods described includes：Pending image is inputted to the image processing model pre-established, the figure
As processing model includes the first model and the second model；
The first model in model is handled by described image the pending image is handled to obtain the pending figure
The provincial characteristics matrix of picture, and the provincial characteristics Input matrix to second model is obtained into the area of the pending image
Characteristic of field weight matrix；
According to the provincial characteristics matrix of the pending image and the provincial characteristics weight matrix of the pending image, calculate
To the feature of the pending image.
8. image processing method according to claim 7, it is characterised in that
First model uses full convolutional neural networks model；
Second model uses attention model.
9. a kind of image processing system, it is characterised in that the system includes：
Image processing equipment and matrix multiplier；
Wherein, described image processing equipment communicates with the matrix multiplier；
Described image processing equipment is configured with the image processing model pre-established, and described image processing model includes the first model
With the second model；First model is that image can be learnt to obtain the provincial characteristics matrix norm type of image；It is described
Second model is that the provincial characteristics matrix of image can be learnt to obtain the model of the provincial characteristics weight matrix of image；
Described image processing equipment, pending image is handled for handling model by described image, obtain described treat
The provincial characteristics matrix and provincial characteristics weight matrix of image are handled, and by the provincial characteristics matrix and provincial characteristics weight square
Battle array is exported to the matrix multiplier；
The matrix multiplier, provincial characteristics matrix and provincial characteristics weight square for the pending image to receiving
Battle array makees matrix multiple processing, obtains the feature of the pending image.
10. a kind of image processing model establishes device, it is characterised in that described device includes：
First model learning module, is learnt for the sample image marked in advance to be inputted to the first model, is obtained described
The provincial characteristics matrix of each image in sample image, the provincial characteristics matrix are used to characterize in the image in each region
Target signature situation；
Second model learning module, for the provincial characteristics Input matrix of each image to be learned into the second model
Practise, obtain the provincial characteristics weight matrix of each image, the provincial characteristics weight matrix is used to characterize the provincial characteristics
The weight in each region, the weight characterize the conspicuousness of target signature in the region in matrix；
Computing module, for the provincial characteristics matrix according to each image and corresponding provincial characteristics weight matrix, calculate
Obtain the feature of each image；
Determining module, for obtaining the image in each feature of image and the sample image marked in advance after study
When the precision of feature tends towards stability, it is determined that the image processing model for learning to obtain includes：First model and described second
Model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610652942.7A CN107729901B (en) | 2016-08-10 | 2016-08-10 | Image processing model establishing method and device and image processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610652942.7A CN107729901B (en) | 2016-08-10 | 2016-08-10 | Image processing model establishing method and device and image processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107729901A true CN107729901A (en) | 2018-02-23 |
| CN107729901B CN107729901B (en) | 2021-04-27 |
Family
ID=61200152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610652942.7A Active CN107729901B (en) | 2016-08-10 | 2016-08-10 | Image processing model establishing method and device and image processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107729901B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106934397A (en) * | 2017-03-13 | 2017-07-07 | 北京市商汤科技开发有限公司 | Image processing method, device and electronic equipment |
| CN108903913A (en) * | 2018-05-31 | 2018-11-30 | 中南大学湘雅医院 | The postoperative care monitoring equipment of skin flap transplantation, system, method, product and server |
| CN109117781A (en) * | 2018-08-07 | 2019-01-01 | 北京飞搜科技有限公司 | Method for building up, device and the more attribute recognition approaches of more attribute Recognition Models |
| CN109145816A (en) * | 2018-08-21 | 2019-01-04 | 北京京东尚科信息技术有限公司 | Commodity recognition method and system |
| CN110210535A (en) * | 2019-05-21 | 2019-09-06 | 北京市商汤科技开发有限公司 | Neural network training method and device and image processing method and device |
| CN110503151A (en) * | 2019-08-26 | 2019-11-26 | 北京推想科技有限公司 | A kind of processing method and system of image |
| CN110688509A (en) * | 2018-06-19 | 2020-01-14 | 新智数字科技有限公司 | Sample data storage method and device |
| CN112686185A (en) * | 2021-01-05 | 2021-04-20 | 北京地平线机器人技术研发有限公司 | Relationship feature extraction method and device and electronic equipment |
| CN113222167A (en) * | 2020-02-06 | 2021-08-06 | 浙江大学 | Image processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104063872A (en) * | 2014-07-04 | 2014-09-24 | 西安电子科技大学 | Method for detecting salient regions in sequence images based on improved visual attention model |
| CN104077609A (en) * | 2014-06-27 | 2014-10-01 | 河海大学 | Saliency detection method based on conditional random field |
| CN104732534A (en) * | 2015-03-18 | 2015-06-24 | 中国人民公安大学 | Method and system for matting conspicuous object in image |
| US20150317800A1 (en) * | 2014-05-05 | 2015-11-05 | Xiaomi Inc. | Method and device for image segmentation |
| CN105224963A (en) * | 2014-06-04 | 2016-01-06 | 华为技术有限公司 | The method of changeable degree of depth learning network structure and terminal |
| CN105426919A (en) * | 2015-11-23 | 2016-03-23 | 河海大学 | Significant guidance and unsupervised feature learning based image classification method |
-
2016
- 2016-08-10 CN CN201610652942.7A patent/CN107729901B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150317800A1 (en) * | 2014-05-05 | 2015-11-05 | Xiaomi Inc. | Method and device for image segmentation |
| CN105224963A (en) * | 2014-06-04 | 2016-01-06 | 华为技术有限公司 | The method of changeable degree of depth learning network structure and terminal |
| CN104077609A (en) * | 2014-06-27 | 2014-10-01 | 河海大学 | Saliency detection method based on conditional random field |
| CN104063872A (en) * | 2014-07-04 | 2014-09-24 | 西安电子科技大学 | Method for detecting salient regions in sequence images based on improved visual attention model |
| CN104732534A (en) * | 2015-03-18 | 2015-06-24 | 中国人民公安大学 | Method and system for matting conspicuous object in image |
| CN105426919A (en) * | 2015-11-23 | 2016-03-23 | 河海大学 | Significant guidance and unsupervised feature learning based image classification method |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106934397B (en) * | 2017-03-13 | 2020-09-01 | 北京市商汤科技开发有限公司 | Image processing method and device and electronic equipment |
| WO2018166438A1 (en) * | 2017-03-13 | 2018-09-20 | 北京市商汤科技开发有限公司 | Image processing method and device and electronic device |
| CN106934397A (en) * | 2017-03-13 | 2017-07-07 | 北京市商汤科技开发有限公司 | Image processing method, device and electronic equipment |
| US10943145B2 (en) | 2017-03-13 | 2021-03-09 | Beijing Sensetime Technology Development Co., Ltd. | Image processing methods and apparatus, and electronic devices |
| CN108903913A (en) * | 2018-05-31 | 2018-11-30 | 中南大学湘雅医院 | The postoperative care monitoring equipment of skin flap transplantation, system, method, product and server |
| CN108903913B (en) * | 2018-05-31 | 2020-12-01 | 中南大学湘雅医院 | Skin flap transplantation postoperative care monitoring equipment, system, method, product and server |
| CN110688509A (en) * | 2018-06-19 | 2020-01-14 | 新智数字科技有限公司 | Sample data storage method and device |
| CN109117781A (en) * | 2018-08-07 | 2019-01-01 | 北京飞搜科技有限公司 | Method for building up, device and the more attribute recognition approaches of more attribute Recognition Models |
| CN109145816A (en) * | 2018-08-21 | 2019-01-04 | 北京京东尚科信息技术有限公司 | Commodity recognition method and system |
| CN109145816B (en) * | 2018-08-21 | 2021-01-26 | 北京京东尚科信息技术有限公司 | Commodity identification method and system |
| CN110210535A (en) * | 2019-05-21 | 2019-09-06 | 北京市商汤科技开发有限公司 | Neural network training method and device and image processing method and device |
| CN110210535B (en) * | 2019-05-21 | 2021-09-10 | 北京市商汤科技开发有限公司 | Neural network training method and device and image processing method and device |
| CN110503151A (en) * | 2019-08-26 | 2019-11-26 | 北京推想科技有限公司 | A kind of processing method and system of image |
| CN113222167A (en) * | 2020-02-06 | 2021-08-06 | 浙江大学 | Image processing method and device |
| CN112686185A (en) * | 2021-01-05 | 2021-04-20 | 北京地平线机器人技术研发有限公司 | Relationship feature extraction method and device and electronic equipment |
| CN112686185B (en) * | 2021-01-05 | 2024-05-07 | 北京地平线机器人技术研发有限公司 | Relation feature extraction method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107729901B (en) | 2021-04-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107729901A (en) | Method for building up, device and the image processing method and system of image processing model | |
| CN107679522B (en) | Multi-stream LSTM-based action identification method | |
| CN105046277B (en) | Robust mechanism study method of the feature significance in image quality evaluation | |
| CN110909630B (en) | Abnormal game video detection method and device | |
| CN108230278B (en) | Image raindrop removing method based on generation countermeasure network | |
| CN110188720A (en) | A kind of object detection method and system based on convolutional neural networks | |
| CN107657249A (en) | Method, apparatus, storage medium and the processor that Analysis On Multi-scale Features pedestrian identifies again | |
| CN110188611A (en) | A kind of pedestrian recognition methods and system again introducing visual attention mechanism | |
| CN104915673B (en) | A kind of objective classification method and system of view-based access control model bag of words | |
| CN108961220B (en) | Image collaborative saliency detection method based on multilayer convolution feature fusion | |
| CN105574550A (en) | Vehicle identification method and device | |
| CN106022273A (en) | Handwritten form identification system of BP neural network based on dynamic sample selection strategy | |
| CN105469376A (en) | Method and device for determining picture similarity | |
| CN107301376B (en) | Pedestrian detection method based on deep learning multi-layer stimulation | |
| CN104063686B (en) | Crop leaf diseases image interactive diagnostic system and method | |
| CN105139450B (en) | A kind of three-dimensional personage construction method and system based on face simulation | |
| CN109410211A (en) | The dividing method and device of target object in a kind of image | |
| CN110084165A (en) | The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations | |
| CN109410190B (en) | Tower pole reverse-breaking detection model training method based on high-resolution remote sensing satellite image | |
| CN104978586A (en) | Trademark identifying method and apparatus | |
| CN110147745A (en) | A kind of key frame of video detection method and device | |
| CN108960015A (en) | A kind of vehicle system automatic identifying method and device based on deep learning | |
| CN107633225A (en) | Information obtaining method and device | |
| CN109272044A (en) | A kind of image similarity determines method, apparatus, equipment and storage medium | |
| CN110163117A (en) | A kind of pedestrian's recognition methods again based on autoexcitation identification feature learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |