CN109657799A - A kind of model tuning method and apparatus based on scene adaptation - Google Patents
A kind of model tuning method and apparatus based on scene adaptation Download PDFInfo
- Publication number
- CN109657799A CN109657799A CN201811381246.2A CN201811381246A CN109657799A CN 109657799 A CN109657799 A CN 109657799A CN 201811381246 A CN201811381246 A CN 201811381246A CN 109657799 A CN109657799 A CN 109657799A
- Authority
- CN
- China
- Prior art keywords
- submodel
- test
- model
- obtains
- algorithm model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 74
- 230000006978 adaptation Effects 0.000 title claims abstract description 24
- 238000012360 testing method Methods 0.000 claims abstract description 197
- 238000012549 training Methods 0.000 claims abstract description 127
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 107
- 238000002790 cross-validation Methods 0.000 claims description 7
- 238000011161 development Methods 0.000 abstract description 13
- 230000000694 effects Effects 0.000 abstract description 8
- 238000012790 confirmation Methods 0.000 abstract description 6
- 238000012795 verification Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 238000010276 construction Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 5
- 235000013399 edible fruits Nutrition 0.000 description 4
- 238000012163 sequencing technique Methods 0.000 description 4
- 238000010998 test method Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012854 evaluation process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Abstract
The present invention provides a kind of model tuning method and apparatus based on scene adaptation to determine the corresponding N number of algorithm model of the set scene by the special scenes information provided based on user；Wherein, N is positive integer；The data set provided the user divides, and obtains training set and test set；Each algorithm model is trained respectively by the training set, obtains one or more corresponding submodel of each algorithm model；By the test set, one or more submodel corresponding to each algorithm model is tested respectively, determines the corresponding optimal models of the special scenes information.It solves actual scene in the prior art and is difficult to the technical issues of choosing suitable algorithm, providing power-assisted for industry development.The indices and generalization ability for guaranteeing model are reached, and have reached the ability of final confirmation optimal models, convenient and efficient, accuracy is high, obtains extensive accuracy, performance, the technical effect of the horizontal high optimal models of suitability.
Description
Technical field
The present invention relates to data analysis technique field more particularly to a kind of model tuning methods based on scene adaptation.
Background technique
When industry development needs to introduce algorithm system, has miscellaneous scene and need to be adapted to respective algorithms to solve
Practical problem.Each scene can have oneself feature and focus, for the people for being unfamiliar with algorithm field, need very high
Learning cost and the time investment algorithm is really landed in industry scene.In this entire process of construction, the choosing of algorithm
Selecting the foundation with model is a vital step.
By priori knowledge, the algorithm of several possible adaptations can be preferentially selected for scene, passes through the data scale of construction of scene
And type, select several model evaluation methods mutually agreed with.Indices value in evaluation process will be used as final choice model
Foundation.
But present invention applicant during technical solution, has found the above-mentioned prior art extremely in realizing the embodiment of the present application
It has the following technical problems less:
Actual scene is difficult to the technical issues of choosing suitable algorithm, providing power-assisted for industry development in the prior art.
Summary of the invention
The embodiment of the invention provides a kind of model tuning method based on scene adaptation, solve practical in the prior art
Scene is difficult to the technical issues of choosing suitable algorithm, providing power-assisted for industry development.
In view of the above problems, propose the embodiment of the invention provides it is a kind of based on scene adaptation model tuning method and
Device.
In a first aspect, the present invention provides a kind of model tuning methods based on scene adaptation, which comprises be based on
The special scenes information that user provides, determines the corresponding N number of algorithm model of the set scene；Wherein, N is positive integer；To institute
The data set for stating user's offer divides, and obtains training set and test set；By the training set respectively to each algorithm model
It is trained, obtains one or more corresponding submodel of each algorithm model；By the test set respectively to each algorithm mould
One or more corresponding submodel of type is tested, and determines the corresponding optimal models of the special scenes information.
Preferably, the data set provided the user divides, before obtaining training set and test set, packet
It includes: the data set is pre-processed；The pretreatment includes: to carry out characteristic point mark to each data in the data set
Note.
Preferably, the data set provided the user divides, and obtains training set and test set, specific to wrap
It includes: the data set being divided using appraisal procedure, determines the training set and the test set；The appraisal procedure,
Including reserving method, K folding cross-validation method, bootstrap.
Preferably, the data set provided the user divides, and obtains training set and test set, specific to wrap
It includes: M division being carried out to the data set, obtains M training set and corresponding test set；Wherein, each training set and correspondence
Test set intersection be the data set, M is positive integer.
Preferably, described that each algorithm model is trained respectively by the training set, it is corresponding to obtain each algorithm model
One or more submodel, specifically include: each algorithm model being trained respectively using the M training set, obtain
The corresponding M submodel of each algorithm model.
It is preferably, described that by the test set, one or more submodel corresponding to each algorithm model is carried out respectively
Test, determines the corresponding optimal models of the special scenes information, specifically includes: using M test set respectively to corresponding
Submodel is tested, and M test result is obtained；According to the test result, the optimal models in the submodel are determined.
Preferably, described that M division, after obtaining M training set and corresponding test set, tool are carried out to the data set
Body includes: that the M training set and corresponding test set are divided into N group training set and corresponding test set；Wherein, every group of instruction
Practicing collection includes present count L different training sets, and L is the positive integer less than M；Using the N group training set respectively to N number of calculation
Method model is trained, and obtains the corresponding L submodel of each algorithm model；Using the N group test set respectively to corresponding
Submodel is tested, and L test result of each submodel is obtained；According to the test result, determine in the submodel
Optimal models.
Preferably, described according to the test result, it determines the optimal models in the submodel, specifically includes: setting
The special scenes information corresponds to the default result of model, and the default result is the mesh of the special scenes information test result
Scale value；The test result is compared with the default result, obtains the deviation of each test result and the default result
Value；The smallest corresponding submodel of the deviation is determined as the corresponding optimal models of the special scenes information.
Preferably, described according to the test result, it determines the optimal models in the submodel, specifically includes: by institute
Test result is stated to be ranked up according to accuracy；Wherein, accuracy is directly proportional to the test result numerical values recited；Described in selection
The maximum corresponding submodel of test result numerical value is the corresponding optimal models of the special scenes information.
Second aspect, the present invention provides a kind of model tuning device based on scene adaptation, described device includes:
First determination unit, first determination unit are used for the special scenes information that provides based on user, described in determination
The corresponding N number of algorithm model of set scene；Wherein, N is positive integer；
First obtains unit, the data set that the first obtains unit is used to provide the user divide, and obtain
Training set and test set；
Second obtaining unit, second obtaining unit is for respectively instructing each algorithm model by the training set
Practice, obtains one or more corresponding submodel of each algorithm model；
Second determination unit, second determination unit are used for corresponding to each algorithm model respectively by the test set
One or more submodel is tested, and determines the corresponding optimal models of the special scenes information.
Preferably, described device further include:
First processing units, the first processing units are for pre-processing the data set；The pretreatment packet
It includes: characteristic point label is carried out to each data in the data set.
Preferably, described device further include:
First division unit, first division unit is for dividing the data set using appraisal procedure, really
The fixed training set and the test set；The appraisal procedure, including reserve method, K folding cross-validation method, bootstrap.
Preferably, described device further include:
Third obtaining unit, the third obtaining unit are used to carry out the data set M division, obtain M training
Collection and corresponding test set；Wherein, the intersection of each training set and corresponding test set is the data set, and M is positive integer.
Preferably, described device further include:
4th obtaining unit, the 4th obtaining unit be used for using the M training set respectively to each algorithm model into
Row training, obtains the corresponding M submodel of each algorithm model.
Preferably, described device further include:
First test unit, the first test unit utilize M test set respectively to corresponding submodel for the 5th
It is tested, obtains M test result；
Third determination unit, the third determination unit are used to be determined in the submodel according to the test result
Optimal models.
Preferably, described device further include:
Second division unit, second division unit are used to the M training set and corresponding test set being divided into N
Group training set and corresponding test set；Wherein, every group of training set includes present count L different training sets, and L is just whole less than M
Number；
5th obtaining unit, the 5th obtaining unit are used for using the N group training set respectively to N number of algorithm mould
Type is trained, and obtains the corresponding L submodel of each algorithm model；
Second test cell, second test cell are used for using the N group test set respectively to corresponding submodel
It is tested, obtains L test result of each submodel；
4th determination unit, the 4th determination unit are used to be determined in the submodel according to the test result
Optimal models.
Preferably, described device further include:
First setup unit, first setup unit is for setting the default knot that the special scenes information corresponds to model
Fruit, the default result are the target value of the special scenes information test result；
6th obtaining unit, the 6th obtaining unit is for comparing the test result and the default result
Compared with obtaining the deviation of each test result Yu the default result；
5th determination unit, the 5th determination unit are used to the smallest corresponding submodel of the deviation being determined as institute
State the corresponding optimal models of special scenes information.
Preferably, described device further include:
First sequencing unit, first sequencing unit is for the test result to be ranked up according to accuracy；Its
In, accuracy is directly proportional to the test result numerical values recited；
First selecting unit, the first selecting unit is for selecting the maximum corresponding submodel of the test result numerical value
For the corresponding optimal models of the special scenes information.
Said one or multiple technical solutions in the embodiment of the present application at least have following one or more technology effects
Fruit:
A kind of model tuning method based on scene adaptation provided in an embodiment of the present invention, which comprises based on use
The special scenes information that family provides is that scene is adapted to many algorithms by priori knowledge, so that it is determined that the set scene is corresponding
N number of algorithm model；Wherein, N is positive integer, then to the user provide data set divide, obtain training set and
Test set；Each algorithm model is trained respectively by the training set, obtains each algorithm mould by training set training
One or more corresponding submodel of type, finally by the corresponding test set of the training set respectively to each algorithm mould
One or more corresponding submodel of type is tested, and the test check results of each submodel are obtained, mainly to submodel into
The extensive accuracy of row, performance, suitability carry out test verification, select wherein that test result is most from the test result of each submodel
This submodel is determined as the corresponding optimal models of the special scenes information, is selected by indices value by excellent submodel
Most suitable model solves actual scene in the prior art and is difficult to choose suitable algorithm, provides power-assisted for industry development
The technical issues of, reach the indices and generalization ability for guaranteeing model, and reach the ability of final confirmation optimal models, side
Just quick, accuracy is high, obtains extensive accuracy, performance, the horizontal high optimal models of suitability, provides power-assisted for industry development
Technical effect.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
Fig. 1 is a kind of flow diagram of the model tuning method based on scene adaptation in the embodiment of the present invention；
Fig. 2 is a kind of structural schematic diagram of the model tuning device based on scene adaptation in the embodiment of the present invention.
Description of symbols: the first determination unit 11, first obtains unit 12, the second obtaining unit 13, second determines list
Member 14.
Specific embodiment
The embodiment of the invention provides a kind of model tuning methods based on scene adaptation, real in the prior art for solving
Border scene is difficult to the technical issues of choosing suitable algorithm, providing power-assisted for industry development.
Technical solution general thought provided by the invention is as follows:
Based on the special scenes information that user provides, the corresponding N number of algorithm model of the set scene is determined；Wherein, N is
Positive integer；The data set provided the user divides, and obtains training set and test set；It is right respectively by the training set
Each algorithm model is trained, and obtains one or more corresponding submodel of each algorithm model；Distinguished by the test set
One or more submodel corresponding to each algorithm model is tested, and determines that the special scenes information is corresponding optimal
Model.Reach the indices and generalization ability for guaranteeing model, and reaches the ability of final confirmation optimal models, it is convenient fast
Victory, accuracy is high, obtains extensive accuracy, performance, the horizontal high optimal models of suitability, provides the skill of power-assisted for industry development
Art effect.
Technical solution of the present invention is described in detail below by attached drawing and specific embodiment, it should be understood that the application
Specific features in embodiment and embodiment are the detailed description to technical scheme, rather than to present techniques
The restriction of scheme, in the absence of conflict, the technical characteristic in the embodiment of the present application and embodiment can be combined with each other.
The terms "and/or", only a kind of incidence relation for describing affiliated partner, indicates that there may be three kinds of passes
System, for example, A and/or B, can indicate: individualism A exists simultaneously A and B, these three situations of individualism B.In addition, herein
Middle character "/" typicallys represent the relationship that forward-backward correlation object is a kind of "or".
Embodiment one
Fig. 1 is a kind of flow diagram of the model tuning method based on scene adaptation in the embodiment of the present invention.Such as Fig. 1 institute
Show, the embodiment of the invention provides a kind of model tuning methods based on scene adaptation, which comprises
Step 110: the special scenes information provided based on user determines the corresponding N number of algorithm model of the set scene；
Wherein, N is positive integer.
Specifically, model tuning method through the embodiment of the present invention, user provides specific in model Operation Optimization Systerm
Scene information, for example, the scenes such as input recognition of face, Defect Search are carried out according to the special scenes information and algorithm
Matching, can have many algorithms according to the various scenes of priori knowledge, according to the feature of the special scenes and the special scenes
Adaptation algorithm collection match polyalgorithm model.Model tuning method described in the embodiment of the present invention is mainly used in face knowledge
Not, Defect Search, but it is not limited only to this, the image procossing being also suitable in remaining special scenes.
Step 120: the data set provided the user divides, and obtains training set and test set.
Further, the data set provided the user divides, before obtaining training set and test set, packet
It includes: the data set is pre-processed；The pretreatment includes: to carry out characteristic point mark to each data in the data set
Note.
Further, the data set provided the user divides, and obtains training set and test set, specific to wrap
It includes: the data set being divided using appraisal procedure, determines the training set and the test set；The appraisal procedure,
Including reserving method, K folding cross-validation method, bootstrap.
Further, the data set provided the user divides, and obtains training set and test set, specific to wrap
It includes: M division being carried out to the data set, obtains M training set and corresponding test set；Wherein, each training set and correspondence
Test set intersection be the data set, M is positive integer.
Specifically, user after providing the special scenes information, also provides the described specific of oneself corresponding input
The data set of scene information, the data set are the data acquisition system for meeting the special scenes information.It first has to provide user
The data set pre-processed according to scene characteristics, it is described pretreatment for the data set each data carry out feature
Point label is for example recognition of face as user provides, and is shone in the scene of the recognition of face the face that user provides
Piece feature locations are marked, and the characteristic point that eyes, nose, mouth etc. are used to carry out recognition of face is marked, and are convenient for
Later period analyzes the identifying processing of face picture in data set.It then will be by the pretreated data set according to its body
Amount, type carry out the matching of model evaluation algorithm, i.e., the feature selectings pair such as data set quantity, photo type provided according to user
The appraisal procedure that should be suitble to, the assessment algorithm includes the method that reserves, K folding cross-validation method, bootstrap common method, but is not limited to
This three kinds of methods.Finally according to the scale of construction of the data set, type and suitable appraisal procedure to the data in the data set
It is trained the division of collection and test set.For example, the data set D that user provides, if the appraisal procedure is when reserving method,
The data set D is divided into two parts, training set S and test set T, wherein S ∪ T=D, S ∩ T=Φ；If data set A is corresponding
Appraisal procedure be that K rolls over cross-validation method, data set A is divided for training set B and test set C, in the insufficient situation of sample size
Under, in order to make full use of data set to test algorithm effect, data set A is randomly divided into k packet, every time by one of them
Packet is used as test set, and remaining k-1 packet is trained as training set；If the appraisal procedure of given data set Y corresponding selection
The instruction of d sample is generated if, comprising d sample, data set Y samples d times with putting back in the data set Y for bootstrap
Practice collection, certain samples in data sample former in this way are likely to occur repeatedly, not entering into the training set in the sample set
Sample ultimately forms inspection set (test set), it is clear that the selected probability of each sample is 1/d, therefore not selected probability is just
It is (1-1/d) that the probability that such a sample does not occur in training set is exactly d all not selected probability, i.e., (1-1/d)
^d, when d tends to infinity, this probability will just level off to e^-1=0.368, so staying in sample in training set probably
Account for the 63.2% of original data set.In order to which the accuracy of assessment result generally takes the random division several times to data, repeat
Test assessment is carried out to guarantee assessment result, when the data set scale of construction that user provides is too small, can also use and replicate the number
According to collection to increase the scale of construction of data set, then assess a certain number of data sets are reached.
Step 130: each algorithm model being trained respectively by the training set, obtains each algorithm model corresponding one
A or multiple submodels.
Specifically, by the training set determined by being divided to the data set that user provides to step
Each algorithm model obtained in 110 is trained respectively, obtains the corresponding submodel of each algorithm model, i.e., N number of calculation by training
Method model corresponds to respective one or more submodels.
Step 140: by the test set, one or more submodel corresponding to each algorithm model is surveyed respectively
Examination, determines the corresponding optimal models of the special scenes information.
Further, described according to the test result, it determines the optimal models in the submodel, specifically includes: setting
The fixed special scenes information corresponds to the default result of model, and the default result is the special scenes information test result
Target value；The test result is compared with the default result, obtains the inclined of each test result and the default result
Difference；The smallest corresponding submodel of the deviation is determined as the corresponding optimal models of the special scenes information.
Further, described according to the test result, it determines the optimal models in the submodel, specifically includes: will
The test result is ranked up according to accuracy；Wherein, accuracy is directly proportional to the test result numerical values recited；Selection institute
Stating the maximum corresponding submodel of test result numerical value is the corresponding optimal models of the special scenes information.
Specifically, after the completion of passing through the training set model training, by the test set respectively to obtained son
Model carries out test verification, and verification object mainly has: extensive accuracy, performance, suitability.Extensive accuracy: model is not
See the estimated performance in data；Performance: the model best from given hypothesis spatial choice performance, Lai Gaishan estimated performance；Adaptation
Property: determine the learning algorithm for being most suitable for problem to be solved.The final evaluation index value of model is mainly bat, Cha Quan
Rate, precision ratio can determine whether a model is adapted to the scene with problem to be solved according to these three values.Pass through the training
Collect the test result for verifying and obtaining to the test of each submodel, test result is therefrom selected most according to the test result
Good submodel is as the corresponding optimal models of the special scenes information.The optimum value in the test result is wherein selected,
The method that method or test result of the test result compared with default result sort according to accuracy can be used.The test result
Method compared with default result is to preset the corresponding objective result of the special scenes information, i.e., in the scene information
The test result of each submodel and the objective result of the presupposed information are compared, calculate it by ideal test result
Deviation, deviation is bigger to illustrate that result is more undesirable, and larger with target value gap, deviation is smaller, illustrates that the test verifies
As a result more ideal, then select wherein the corresponding submodel of the smallest test result of deviation be optimal models.The test result
By the method that accuracy is sorted, the correspondence test result of each submodel acquisition is verified according to it for the test set test will be passed through
The descending sequence of accuracy selects the first detection data i.e. corresponding submodel of the highest testing result of accuracy the most most
Excellent model selects most suitable model by the indices value in evaluation process, finally solves actual scene and is difficult to select
The problem of to suitable algorithm, the technical issues of providing power-assisted for industry development.To reach the indices for guaranteeing model
And generalization ability, and reach the ability of final confirmation optimal models, convenient and efficient, accuracy is high, obtains extensive accuracy, property
The horizontal high optimal models of energy, suitability, effectively avoid overfitting problem, solve in actual scene, use algorithm mould
Overfitting problem can occur for type, it is difficult to the technical issues of choosing suitable algorithm, providing power-assisted for industry development.
Further, described that each algorithm model is trained respectively by the training set, obtain each algorithm model pair
One or more submodel answered, specifically includes: being trained, is obtained to each algorithm model respectively using the M training set
Obtain the corresponding M submodel of each algorithm model.
Further, it is described by the test set respectively one or more submodel corresponding to each algorithm model into
Row test, determines the corresponding optimal models of the special scenes information, specifically includes: using M test set respectively to correspondence
Submodel tested, obtain M test result；According to the test result, the optimal models in the submodel are determined.
Further, described that M division is carried out to the data set, after obtaining M training set and corresponding test set,
It specifically includes: the M training set and corresponding test set is divided into N group training set and corresponding test set；Wherein, every group
Training set includes present count L different training sets, and L is the positive integer less than M；Using the N group training set respectively to described N number of
Algorithm model is trained, and obtains the corresponding L submodel of each algorithm model；Using the N group test set respectively to correspondence
Submodel tested, obtain L test result of each submodel；According to the test result, determine in the submodel
Optimal models.
Specifically, by the training set to the training of each algorithm model, M training sets obtained and right will be divided
The test set answered, a kind of test method are that each algorithm model is respectively trained in each training set, show that each algorithm model is corresponding
The corresponding submodel of each training set, i.e. M training set is trained each algorithm model, to obtain each algorithm
Then the corresponding M submodel of model recycles the corresponding test set of the M training set to carry out the correspondence submodel of acquisition
Test verification, obtains its corresponding submodel test result, is selected according to the submodel test result, to obtain most
Excellent model, for example, M training set, wherein M is 3, has respectively included the first training set, the second training set, third training
Collection and corresponding first test set, the second test set, third test set, the special scenes model that user provides are looked into for defect
It looks for, searches the defects of steel plate image steel plate, algorithm model in 2, respectively model one and model two are matched altogether, using each
When the test method of each algorithm model is respectively trained in training set, first training set is respectively to the model one and the model
Two are trained respectively, obtain submodel 1.1, first instruction that the first training set training model one obtains respectively
Practice the submodel 2.1 that the collection training model two obtains；Recycle the second training set that the model one and the mould is respectively trained
Type two obtains corresponding submodel 1.2 and submodel 2.2；The mould finally is respectively trained using the third training set
Type one and the model two obtain corresponding submodel 1.3 and submodel 2.3；To obtain the model one corresponding 3
A submodel is respectively 1.1,1.2 and 1.3；Two corresponding 3 submodels of the model are respectively 2.1,2.2,2.3, then sharp
Test verification is carried out to obtained each submodel with each training set corresponding test set, to obtain each submodel
Test result, if wherein the test result of submodel 1.2 is most ideal, it is determined that the submodel 1.2 user provides
The optimal models of Defect Search.
Another test method is to choose the different training sets of predetermined number to remove to train N number of algorithm model wherein one
The M training set and corresponding test set are combined grouping according to predetermined number altogether, are divided into N group by a algorithm model
N number of algorithm model is trained respectively, to obtain more under the corresponding training set group of each algorithm model
A submodel recycles each submodel that obtains for the corresponding test set pair of the training set for training it to carry out test verification, most
The test result for obtaining each submodel afterwards, according to the test result therefrom according to selecting excellent mode to select the wherein test result
Optimal submodel is as optimal models, and for example, the special scenes that user provides are recognition of face, when for the test method
When, if M is 4, algorithm model is model one and model two, is grouped according to predetermined number 2, by the first training set and second
Training set is one group and is trained to the model one, by one group of third training set and the 4th training set position to the model two into
Row training, obtains the corresponding submodel 1.1 of the model one and submodel 1.2, the corresponding submodel 2.3 of the model two respectively
With submodel 2.4, test verification finally is carried out to the submodel 1.1 using the first test set, using the second test set to institute
It states submodel 1.2 and carries out test verification, test verification is carried out to the submodel 2.3 using third test set, utilize the 4th instruction
Practice collection and test verification is carried out to the submodel 2.4, to obtain the test check results of each submodel, it is most accurate therefrom to select
Highest submodel is spent as optimal models.By the indices value in test checking procedure, most suitable model is selected, from
And the corresponding optimal models of the special scenes information of user's offer are provided, reach and has guaranteed the indices of model and general
Change ability, and reach the ability of final confirmation optimal models, convenient and efficient, accuracy is high, obtains extensive accuracy, performance, fits
With the horizontal high optimal models of property, the technical effect of overfitting problem is effectively avoided, to solve actual field in the prior art
Scape is difficult to the technical issues of choosing suitable algorithm, providing power-assisted for industry development.
Embodiment two
Based on the same inventive concept of model tuning method being adapted to based on scene a kind of in previous embodiment, the present invention
A kind of model tuning device based on scene adaptation is provided, as shown in Fig. 2, described device includes:
First determination unit 11, first determination unit 11 are used for the special scenes information provided based on user, determine
The corresponding N number of algorithm model of the set scene；Wherein, N is positive integer；
First obtains unit 12, the data set that the first obtains unit 12 is used to provide the user divide,
Obtain training set and test set；
Second obtaining unit 13, second obtaining unit 13 be used for by the training set respectively to each algorithm model into
Row training, obtains one or more corresponding submodel of each algorithm model；
Second determination unit 14, second determination unit 14 are used for through the test set respectively to each algorithm model pair
One or more submodel answered is tested, and determines the corresponding optimal models of the special scenes information.
Further, described device further include:
First processing units, the first processing units are for pre-processing the data set；The pretreatment packet
It includes: characteristic point label is carried out to each data in the data set.
Further, described device further include:
First division unit, first division unit is for dividing the data set using appraisal procedure, really
The fixed training set and the test set；The appraisal procedure, including reserve method, K folding cross-validation method, bootstrap.
Further, described device further include:
Third obtaining unit, the third obtaining unit are used to carry out the data set M division, obtain M training
Collection and corresponding test set；Wherein, the intersection of each training set and corresponding test set is the data set, and M is positive integer.
Further, described device further include:
4th obtaining unit, the 4th obtaining unit be used for using the M training set respectively to each algorithm model into
Row training, obtains the corresponding M submodel of each algorithm model.
Further, described device further include:
First test unit, the first test unit utilize M test set respectively to corresponding submodel for the 5th
It is tested, obtains M test result；
Third determination unit, the third determination unit are used to be determined in the submodel according to the test result
Optimal models.
Further, described device further include:
Second division unit, second division unit are used to the M training set and corresponding test set being divided into N
Group training set and corresponding test set；Wherein, every group of training set includes present count L different training sets, and L is just whole less than M
Number；
5th obtaining unit, the 5th obtaining unit are used for using the N group training set respectively to N number of algorithm mould
Type is trained, and obtains the corresponding L submodel of each algorithm model；
Second test cell, second test cell are used for using the N group test set respectively to corresponding submodel
It is tested, obtains L test result of each submodel；
4th determination unit, the 4th determination unit are used to be determined in the submodel according to the test result
Optimal models.
Further, described device further include:
First setup unit, first setup unit is for setting the default knot that the special scenes information corresponds to model
Fruit, the default result are the target value of the special scenes information test result；
6th obtaining unit, the 6th obtaining unit is for comparing the test result and the default result
Compared with obtaining the deviation of each test result Yu the default result；
5th determination unit, the 5th determination unit are used to the smallest corresponding submodel of the deviation being determined as institute
State the corresponding optimal models of special scenes information.
Further, described device further include:
First sequencing unit, first sequencing unit is for the test result to be ranked up according to accuracy；Its
In, accuracy is directly proportional to the test result numerical values recited；
First selecting unit, the first selecting unit is for selecting the maximum corresponding submodel of the test result numerical value
For the corresponding optimal models of the special scenes information.
The various change mode for the model tuning method that one of 1 embodiment one of earlier figures is adapted to based on scene and specific
Example is equally applicable to a kind of model tuning device based on scene adaptation of the present embodiment, is based on scene to one kind by aforementioned
The detailed description of the model tuning method of adaptation, those skilled in the art are clear that in the present embodiment a kind of based on field
The implementation method of the model tuning device of scape adaptation, so this will not be detailed here in order to illustrate the succinct of book.
Said one or multiple technical solutions in the embodiment of the present application at least have following one or more technology effects
Fruit:
A kind of model tuning method based on scene adaptation provided in an embodiment of the present invention, which comprises based on use
The special scenes information that family provides is that scene is adapted to many algorithms by priori knowledge, so that it is determined that the set scene is corresponding
N number of algorithm model；Wherein, N is positive integer, then to the user provide data set divide, obtain training set and
Test set；Each algorithm model is trained respectively by the training set, obtains each algorithm mould by training set training
One or more corresponding submodel of type, finally by the corresponding test set of the training set respectively to each algorithm mould
One or more corresponding submodel of type is tested, and the test check results of each submodel are obtained, mainly to submodel into
The extensive accuracy of row, performance, suitability carry out test verification, select wherein that test result is most from the test result of each submodel
This submodel is determined as the corresponding optimal models of the special scenes information, is selected by indices value by excellent submodel
Most suitable model solves actual scene in the prior art and is difficult to choose suitable algorithm, provides power-assisted for industry development
The technical issues of, reach the indices and generalization ability for guaranteeing model, and reach the ability of final confirmation optimal models, side
Just quick, accuracy is high, obtains extensive accuracy, performance, the horizontal high optimal models of suitability, provides power-assisted for industry development
Technical effect.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of model tuning method based on scene adaptation, which is characterized in that the described method includes:
Based on the special scenes information that user provides, the corresponding N number of algorithm model of the set scene is determined；Wherein, N is positive whole
Number；
The data set provided the user divides, and obtains training set and test set；
Each algorithm model is trained respectively by the training set, obtains one or more corresponding son of each algorithm model
Model；
By the test set, one or more submodel corresponding to each algorithm model is tested respectively, is determined described
The corresponding optimal models of special scenes information.
2. the method as described in claim 1, which is characterized in that the data set provided the user divides, and obtains
Before training set and test set, comprising:
The data set is pre-processed；The pretreatment includes: to carry out characteristic point to each data in the data set
Label.
3. the method as described in claim 1, which is characterized in that the data set provided the user divides, and obtains
Training set and test set are obtained, is specifically included:
The data set is divided using appraisal procedure, determines the training set and the test set；The appraisal procedure,
Including reserving method, K folding cross-validation method, bootstrap.
4. the method as described in claim 1, which is characterized in that the data set provided the user divides, and obtains
Training set and test set are obtained, is specifically included:
M division is carried out to the data set, obtains M training set and corresponding test set；Wherein, each training set and correspondence
Test set intersection be the data set, M is positive integer.
5. method as claimed in claim 4, which is characterized in that described to be carried out respectively to each algorithm model by the training set
Training, obtains one or more corresponding submodel of each algorithm model, specifically includes:
Each algorithm model is trained respectively using the M training set, obtains the corresponding M submodel of each algorithm model.
6. method as claimed in claim 5, which is characterized in that described corresponding to each algorithm model respectively by the test set
One or more submodel tested, determine the corresponding optimal models of the special scenes information, specifically include:
Corresponding submodel is tested respectively using M test set, obtains M test result；
According to the test result, the optimal models in the submodel are determined.
7. method as claimed in claim 4, which is characterized in that it is described that M division is carried out to the data set, obtain M instruction
After practicing collection and corresponding test set, specifically include:
The M training set and corresponding test set are divided into N group training set and corresponding test set；Wherein, every group of training
Collection includes present count L different training sets, and L is the positive integer less than M；
N number of algorithm model is trained respectively using the N group training set, it is L corresponding to obtain each algorithm model
Submodel；
Corresponding submodel is tested respectively using the N group test set, obtains L test result of each submodel；
According to the test result, the optimal models in the submodel are determined.
8. method according to claim 6 or 7, which is characterized in that it is described according to the test result, determine the submodel
In optimal models, specifically include:
The default result that the special scenes information corresponds to model is set, the default result is special scenes information test
As a result target value；
The test result is compared with the default result, obtains the deviation of each test result and the default result
Value；
The smallest corresponding submodel of the deviation is determined as the corresponding optimal models of the special scenes information.
9. method according to claim 6 or 7, which is characterized in that it is described according to the test result, determine the submodel
In optimal models, specifically include:
The test result is ranked up according to accuracy；Wherein, accuracy is directly proportional to the test result numerical values recited；
Select the maximum corresponding submodel of the test result numerical value for the corresponding optimal models of the special scenes information.
10. a kind of model tuning device based on scene adaptation, which is characterized in that described device includes:
First determination unit, first determination unit are used for the special scenes information provided based on user, determine the setting
The corresponding N number of algorithm model of scene；Wherein, N is positive integer；
First obtains unit, the data set that the first obtains unit is used to provide the user are divided, are trained
Collection and test set；
Second obtaining unit, second obtaining unit are used to respectively be trained each algorithm model by the training set,
Obtain one or more corresponding submodel of each algorithm model；
Second determination unit, second determination unit are used to pass through the test set one corresponding to each algorithm model respectively
Or multiple submodels are tested, and determine the corresponding optimal models of the special scenes information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811381246.2A CN109657799A (en) | 2018-11-20 | 2018-11-20 | A kind of model tuning method and apparatus based on scene adaptation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811381246.2A CN109657799A (en) | 2018-11-20 | 2018-11-20 | A kind of model tuning method and apparatus based on scene adaptation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109657799A true CN109657799A (en) | 2019-04-19 |
Family
ID=66111312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811381246.2A Pending CN109657799A (en) | 2018-11-20 | 2018-11-20 | A kind of model tuning method and apparatus based on scene adaptation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109657799A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110333994A (en) * | 2019-07-08 | 2019-10-15 | 深圳乐信软件技术有限公司 | A kind of matched model of data set determines method, apparatus, equipment and storage medium |
| CN110598586A (en) * | 2019-08-27 | 2019-12-20 | 电子科技大学 | Target detection method and system |
| WO2021004198A1 (en) * | 2019-07-10 | 2021-01-14 | 江苏金恒信息科技股份有限公司 | Plate performance prediction method and apparatus |
| CN112487699A (en) * | 2019-09-11 | 2021-03-12 | 北京国双科技有限公司 | Working fluid level determining method, working fluid level determining model obtaining method and related equipment |
| CN113065658A (en) * | 2021-03-30 | 2021-07-02 | 山东英信计算机技术有限公司 | Method and system for improving accuracy of artificial intelligence inference result |
| CN114331238A (en) * | 2022-03-17 | 2022-04-12 | 北京航天晨信科技有限责任公司 | Intelligent model algorithm optimization method, system, storage medium and computer equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101203883A (en) * | 2005-06-21 | 2008-06-18 | 皇家飞利浦电子股份有限公司 | Progressive model-based adaptation |
| CN105117611A (en) * | 2015-09-23 | 2015-12-02 | 北京科技大学 | Determining method and system for traditional Chinese medicine tongue diagnosis model based on convolution neural networks |
| CN107886503A (en) * | 2017-10-27 | 2018-04-06 | 重庆金山医疗器械有限公司 | A kind of alimentary canal anatomical position recognition methods and device |
| CN107886064A (en) * | 2017-11-06 | 2018-04-06 | 安徽大学 | A kind of method that recognition of face scene based on convolutional neural networks adapts to |
| CN108231201A (en) * | 2018-01-25 | 2018-06-29 | 华中科技大学 | A kind of construction method, system and the application of disease data analyzing and processing model |
| CN108830380A (en) * | 2018-04-11 | 2018-11-16 | 开放智能机器（上海）有限公司 | A kind of training pattern generation method and system based on cloud service |
-
2018
- 2018-11-20 CN CN201811381246.2A patent/CN109657799A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101203883A (en) * | 2005-06-21 | 2008-06-18 | 皇家飞利浦电子股份有限公司 | Progressive model-based adaptation |
| CN105117611A (en) * | 2015-09-23 | 2015-12-02 | 北京科技大学 | Determining method and system for traditional Chinese medicine tongue diagnosis model based on convolution neural networks |
| CN107886503A (en) * | 2017-10-27 | 2018-04-06 | 重庆金山医疗器械有限公司 | A kind of alimentary canal anatomical position recognition methods and device |
| CN107886064A (en) * | 2017-11-06 | 2018-04-06 | 安徽大学 | A kind of method that recognition of face scene based on convolutional neural networks adapts to |
| CN108231201A (en) * | 2018-01-25 | 2018-06-29 | 华中科技大学 | A kind of construction method, system and the application of disease data analyzing and processing model |
| CN108830380A (en) * | 2018-04-11 | 2018-11-16 | 开放智能机器（上海）有限公司 | A kind of training pattern generation method and system based on cloud service |
Non-Patent Citations (1)
| Title |
|---|
| 杨晶东等: "基于迁移学习的全连接神经网络舌象分类方法", 《第二军医大学学报》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110333994A (en) * | 2019-07-08 | 2019-10-15 | 深圳乐信软件技术有限公司 | A kind of matched model of data set determines method, apparatus, equipment and storage medium |
| CN110333994B (en) * | 2019-07-08 | 2023-06-06 | 深圳乐信软件技术有限公司 | Data set matching model determination method, device, equipment and storage medium |
| WO2021004198A1 (en) * | 2019-07-10 | 2021-01-14 | 江苏金恒信息科技股份有限公司 | Plate performance prediction method and apparatus |
| GB2600213A (en) * | 2019-07-10 | 2022-04-27 | Jiangsu Jinheng Information Tech Co Ltd | Plate performance prediction method and apparatus |
| GB2600213B (en) * | 2019-07-10 | 2023-06-14 | Jiangsu Jinheng Information Tech Co Ltd | Method and apparatus for predicitng plate performance |
| CN110598586A (en) * | 2019-08-27 | 2019-12-20 | 电子科技大学 | Target detection method and system |
| CN112487699A (en) * | 2019-09-11 | 2021-03-12 | 北京国双科技有限公司 | Working fluid level determining method, working fluid level determining model obtaining method and related equipment |
| CN113065658A (en) * | 2021-03-30 | 2021-07-02 | 山东英信计算机技术有限公司 | Method and system for improving accuracy of artificial intelligence inference result |
| CN114331238A (en) * | 2022-03-17 | 2022-04-12 | 北京航天晨信科技有限责任公司 | Intelligent model algorithm optimization method, system, storage medium and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109657799A (en) | A kind of model tuning method and apparatus based on scene adaptation | |
| JP6701367B2 (en) | Method, device, and computer program for providing personalized educational content | |
| CN110009171B (en) | User behavior simulation method, device, equipment and computer readable storage medium | |
| CN104346802B (en) | A kind of personnel leave the post monitoring method and equipment | |
| CN110348580A (en) | Construct the method, apparatus and prediction technique, device of GBDT model | |
| CN104850818A (en) | Face detector training method, face detection method and device | |
| CN107203467A (en) | The reference test method and device of supervised learning algorithm under a kind of distributed environment | |
| CN108921569A (en) | A kind of method and device of determining customer complaint type | |
| CN109919252A (en) | The method for generating classifier using a small number of mark images | |
| CN105488055B (en) | The generation method and system of Individualized computer study and evaluation and test product | |
| CN109242825A (en) | A kind of steel surface defect identification method and device based on depth learning technology | |
| CN109983482A (en) | Learning model generation method, learning model generating means, signal data method of discrimination, signal data discriminating gear and signal data discriminating program | |
| CN109142374A (en) | Method and system based on the efficient Checking model of extra small sample training | |
| Conati et al. | Further Results on Predicting Cognitive Abilities for Adaptive Visualizations. | |
| CN109903242A (en) | A kind of image generating method and device | |
| CN109033262A (en) | Question and answer knowledge base update method and device | |
| CN110472802A (en) | A kind of data characteristics appraisal procedure, device and equipment | |
| Benedetto et al. | On the application of transformers for estimating the difficulty of multiple-choice questions from text | |
| CN103927530A (en) | Acquiring method, application method and application system of final classifier | |
| CN109919941A (en) | Internal screw thread defect inspection method, device, system, equipment and medium | |
| CN110490056A (en) | The method and apparatus that image comprising formula is handled | |
| CN108875445B (en) | Pedestrian re-identification method and device | |
| CN106611021A (en) | Data processing method and equipment | |
| KR101745874B1 (en) | System and method for a learning course automatic generation | |
| CN109086790A (en) | A kind of alternative manner of disaggregated model, device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190419 |