CN109102078A - A kind of optimization method of machine learning, system, computer storage medium and electronic equipment - Google Patents
A kind of optimization method of machine learning, system, computer storage medium and electronic equipment Download PDFInfo
- Publication number
- CN109102078A CN109102078A CN201810759470.4A CN201810759470A CN109102078A CN 109102078 A CN109102078 A CN 109102078A CN 201810759470 A CN201810759470 A CN 201810759470A CN 109102078 A CN109102078 A CN 109102078A
- Authority
- CN
- China
- Prior art keywords
- characteristic pattern
- machine learning
- classification
- data
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of optimization method of machine learning, system, computer storage medium and electronic equipments, wherein the acquisition methods include: to obtain at least one learning data；It identifies the learning data, obtains recognition result, the recognition result includes: classification belonging to the characteristic pattern and/or the learning data of the learning data；When the recognition result meets preset selection rule, using the corresponding learning data of the recognition result as the optimization data of the machine learning model.The present invention, which can targetedly be got, has the data being obviously improved to machine learning model performance.
Description
Technical field
The invention belongs to field of artificial intelligence, and in particular to a kind of optimization method of machine learning, system, computer
Storage medium and electronic equipment.
Background technique
Machine learning is an important branch of artificial intelligence, in recent years, due to the rapid development of artificial intelligence, so that machine
Device study is also required to that its learning ability is continuously improved.Machine learning be refer to by continue on new training data to its into
Row learning training, to constantly promote its performance.For example, the performance of neural network model can be with the increasing of training data quantity
It is promoted more, therefore, is based on sufficient amount of training data, neural network model can obtain the promotion of expected performance.This
It is because training data is enough, so that can traverse most features of data when learning training.
However, in reality under certain application scenarios, training data be not it is very sufficient, result in existing training data past
Toward the acquisition that there is the feature namely training data that do not cover it cannot be guaranteed that all features are adequately covered.Therefore, how
So that the acquisition of training data can cover and all be characterized in a urgent problem to be solved.
Summary of the invention
(1) goal of the invention
The object of the present invention is to provide a kind of optimization sides of the big machine learning of a kind of pair of machine learning model performance boost
Method, system, computer storage medium and electronic equipment.
(2) technical solution
To solve the above problems, the first aspect of the present invention provides a kind of optimization method of machine learning, comprising: obtain
At least one learning data；It identifies the learning data, obtains recognition result, the recognition result includes: the learning data
Characteristic pattern and/or the learning data belonging to classification；It, will when the recognition result meets preset selection rule
Optimization data of the corresponding learning data of the recognition result as the machine learning model.
Optionally, the preset selection rule is resolution and/or the study number based on the characteristic pattern
It is set according to the quantity of affiliated classification.
Optionally, the resolution of the characteristic pattern includes: low resolution and/or high resolution.
Optionally, the resolution of the characteristic pattern obtains with the following method: will at least one described machine learning data
Input trained machine learning model in advance；The machine learning model is obtained to propose at least one described machine learning data
Take the characteristic pattern that feature obtains；Identification probability based on the characteristic pattern under each classification of the machine learning model, meter
Calculate the resolution of the characteristic pattern.
Optionally, the resolution of the characteristic pattern is to seek standard deviation to identification probability of the characteristic pattern under all categories
Difference.
Optionally, if the resolution of a certain characteristic pattern is lower than the first resolution threshold value, this feature figure is low resolution
Characteristic pattern；And/or if the resolution of a certain characteristic pattern is higher than the second resolution threshold value, this feature figure is the spy of high resolution
Sign figure.
Optionally, if the quantity of classification belonging to the learning data is less than predetermined quantity, meet preset choosing
Take rule.
Optionally, further includes: count the quantity of classification belonging at least one described learning data；Wherein, the statistics
The quantity of classification belonging at least one described learning data, comprising: learning data will have been trained to input trained machine in advance
Device learning model；It obtains the machine learning model and has trained learning data to extract the characteristic pattern that feature obtains to described；Statistics
The classification that each characteristic pattern is got；Based on the classification that each characteristic pattern is got, the quantity of the characteristic pattern under each classification is calculated.
Optionally, further includes: if classification belonging to new learning data is the classification less than predetermined quantity, meet preparatory
The selection rule of setting.
According to another aspect of the present invention, a kind of optimization system of machine learning is also provided, comprising: obtain module, use
In at least one learning data of acquisition；Identification module, the learning data, obtains recognition result, the identification knot for identification
Fruit includes: classification belonging to the characteristic pattern and/or the learning data of the learning data, and is met in the recognition result
When preset selection rule, using the corresponding learning data of the recognition result as the excellent of the machine learning model
Change data.
Optionally, the preset selection rule is resolution and/or the study number based on the characteristic pattern
It is set according to the quantity of affiliated classification.
Optionally, the resolution of the characteristic pattern includes: low resolution and/or high resolution.
Optionally, further includes: characteristic pattern obtains module, for obtaining preparatory trained machine learning model at least one
A machine learning data extract the characteristic pattern that feature obtains；Resolution computing module, for being based on the characteristic pattern in institute
The identification probability under each classification of machine learning model is stated, the resolution of the characteristic pattern is calculated.
Optionally, the resolution computing module includes: resolution computing unit, is used for the characteristic pattern in all classes
Identification probability under not seeks standard deviation, the resolution as characteristic pattern.
Optionally, the resolution computing module is also used to the resolution in a certain characteristic pattern lower than the first resolution threshold
When value, determine that this feature figure is the characteristic pattern of low resolution；And/or the resolution computing module, it is also used in a certain feature
When the resolution of figure is higher than the second resolution threshold value, determine that this feature figure is the characteristic pattern of high resolution.
Optionally, the identification module is also used to, and the quantity of the classification belonging to the learning data is less than predetermined quantity
When, then it is identified as meeting preset selection rule.
Optionally, further includes: statistical module, for counting the quantity of classification belonging at least one described learning data；
Wherein, the statistical module includes: input submodule, for learning data will to have been trained to input trained machine learning in advance
Model；Characteristic pattern acquisition submodule obtains the machine learning model and has trained learning data to extract what feature obtained to described
Characteristic pattern；Classification statistic submodule, the classification got for counting each characteristic pattern；Characteristic pattern quantity calculating submodule, is used for
Based on the classification that each characteristic pattern is got, the quantity of the characteristic pattern under each classification is calculated.
Optionally, it is the classification less than predetermined quantity that the identification module, which is also used to the classification belonging to new learning data,
When, then it is identified as meeting preset selection rule.
Another aspect according to an embodiment of the present invention provides a kind of computer storage medium, stores on the storage medium
The step of having computer program, any one of the above the method is realized when described program is executed by processor.
Another aspect according to an embodiment of the present invention, provides a kind of electronic equipment, which is characterized in that including memory, place
It manages device and is stored in the computer program that can be run on the memory and on the processor, described in the processor execution
The step of any one of the above the method is realized when program.
(3) beneficial effect
Above-mentioned technical proposal of the invention has following beneficial technical effect: by obtaining at least one learning data；
Identification learning data, obtain include classification belonging to the characteristic pattern and/or learning data of learning data recognition result, identifying
When as a result meeting preset selection rule, using the corresponding learning data of recognition result as the optimization number of machine learning model
According to, to be used to hoisting machine learning model performance, further, visualization processing is carried out to the data finally got, thus
Data mark personnel's quick obtaining is labeled the apparent data of machine learning model performance boost, avoids existing
There is no all being labeled all new datas of orientation in technology, improves annotating efficiency, and in hoisting machine learning model
When performance also more targetedly.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of the convolutional neural networks model of the embodiment of the present invention one；
Fig. 2 is the schematic diagram of the probability distribution of the classification layer of convolutional neural networks model；
Fig. 3 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention one；
Fig. 4 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention two；
Fig. 5 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention three；
Fig. 6 is the 20 class probability distribution results schematic diagrames testing picture and getting in all characteristic patterns of layer 3 network；
Fig. 7 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention four；
Fig. 8 is the 20 class probability distribution results schematic diagrames testing picture and getting in all characteristic patterns of layer 3 network；
Fig. 9 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention five；
Figure 10 is the statistical result that 1000 pictures are got with classification in all characteristic patterns of layer 3 network；
Figure 11 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention six；
Figure 12 is a kind of schematic diagram of the optimization method of machine learning of the embodiment of the present invention；
Figure 13 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention seven；
Figure 14 is the schematic diagram for the classification that new test picture is got in layer 3 network；
Figure 15 is the schematic diagram for the classification that another new test picture is got in layer 3 network；
Figure 16 (a) is pilotless automobile training picture or test picture；
Figure 16 (b) is the training picture or test picture of wood grain detection；
Figure 17 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention eight；
Figure 18 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention nine；
Figure 19 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention ten；
Figure 20 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention 11；
Figure 21 is the structural schematic diagram of a kind of electronic equipment of the embodiment of the present invention.
Specific embodiment
In order to make the objectives, technical solutions and advantages of the present invention clearer, With reference to embodiment and join
According to attached drawing, the present invention is described in more detail.It should be understood that these descriptions are merely illustrative, and it is not intended to limit this hair
Bright range.In addition, in the following description, descriptions of well-known structures and technologies are omitted, to avoid this is unnecessarily obscured
The concept of invention.
Technical term:
Convolution kernel: being contained in convolutional layer, and doing convolution with image can extract the feature of the image, and different convolution kernels can be with
Different characteristics of image is extracted, extracted characteristics of image is referred to as characteristic pattern in embodiments of the present invention.
Convolutional layer: each convolutional layer is then progressive by the extraction that convolution kernel carries out corresponding topical feature.Close to defeated
Enter that the convolutional layer of layer successively extracts is the features such as texture, boundary, profile, and the feature close to the extraction of output layer is more abstract, example
Such as cat and dog gross feature.All convolution kernel sizes of each convolutional layer are identical, convolution kernel sizes between different convolutional layers
It can be different.The characteristic pattern that each convolutional layer extracts can use several classifications, and each classification is got with some probability.
Parameter setting: the model parameters such as the convolutional network number of plies, every layer of convolution kernel size for including and characteristic pattern quantity according to
Adjust the setting of ginseng experience.
Depth convolutional neural networks: the small and deep convolution kernel of preference, i.e. convolution kernel size are small, convolution number of layers is more, in this way
The characteristics of image of arbitrarily complicated shape can either be portrayed, and can reduce the number and computation complexity of parameter.
Characteristic pattern size: with to convolved image size and convolution kernel size it is related, such as: the size to convolved image is
28x28, by convolutional layer include 6 5x5 convolution kernel, then output be 6 features having a size of (28-5+1) x (28-5+1)
Figure.
The structure and each convolutional layer for introducing convolutional neural networks model respectively by taking Fig. 1 and Fig. 2 as an example below are extracted
Characteristic pattern and characteristic pattern take the probability distribution of each classification.
Fig. 1 is the structural schematic diagram of a convolution neural network model.
As shown in Figure 1, the convolutional neural networks model includes 8 layers, specifically: convolutional layer (layer 1) -> convolutional layer (layer
2) -> convolutional layer (layer 3) -> convolutional layer (layer 4) -> convolutional layer (layer 5) -> full articulamentum (layer 6) -> full articulamentum (layer 7) ->
Softmax classifies layer (layer 8).
Wherein, each layer of layer 1 to layer 5 can be regarded as sensu lato convolutional layer, but it is comprising convolutional layer in the narrow sense, swashs
Encourage layer and pond layer three.Preceding 5 layers of every layer network all can extract the feature of the data of a certain input, and it is (every to obtain multiple characteristic patterns
The quantity that layer network extracts characteristic pattern can rule of thumb be set).
Characteristic pattern in the embodiment of the present invention is divided according to each convolutional layer, and characteristic pattern is represented by A.X, wherein A table
Showing that convolutional layer is numbered, X indicates that the convolutional layer characteristic pattern numbers (each convolutional layer has multiple characteristic patterns), such as: layer 1 to layer 5 mentions
Characteristic pattern is taken to be denoted as 1.x~5.x.Assuming that current layer is layer 3, then the characteristic pattern that layer 3 extracts is denoted as 3.1~3.4.
Fig. 2 is the schematic diagram of the probability distribution of the classification layer of convolutional neural networks model.
As shown in Fig. 2, being to present in table form, specifically, at this for the probability distribution under each classification
In convolutional neural networks model structure, Softmax classification layer (layer 8) of definition includes 20 classifications.
Inventor has found in following two points when to learning training is carried out through mass data input convolutional neural networks model
Hold:
First point: there is certain close in the performance boost space of machine learning model, the feature for being included with new training data
System, i.e., if new training data includes the feature that lacks of old training data, performance boost will be larger, and if new
The feature that training data includes and old training data are more similar, then performance boost will be smaller.Therefore, that trained is all
The statistical nature for the characteristic pattern that labeled data is exported in all convolutional layers of machine learning model can be used to infer new training number
According to type demand.
Second point: if judging which class data is machine learning model lack by the resolution of test sample, from low
Resolution or the test picture of high resolution itself, can only obtain the data requirements classified on an equal basis therewith.Further, low identification
Why the test picture of degree has obtained lower resolution in the identification process of machine learning model, is because in the picture
The sample that certain category feature possesses during model training is less, i.e., model lacks training to this category feature；And high resolution
Then it is usually because certain category feature more obviously or in model forms fixed mode, (thinking similar to people is fixed for test picture
Gesture), obvious feature can be used to quickly training and fixed mode then most probably needs to correct.Therefore, different convolutional layer outputs
Profile information, can be used to infer the type demand of new training data.
Based on two above-mentioned starting points, The present invention gives the acquisition sides of the training data type demand of machine learning
Method refers to following figure 3-Figure 16 embodiment and is introduced.
It should be noted that the embodiment of the present invention be the neural network model shown in Fig. 1 structure for be illustrated, but
The method of the embodiment of the present invention is not intended to limit the number of plies of neural network model, can be adapted for the neural network mould of arbitrary structures
Type.
Fig. 3 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention one.
As shown in figure 3, a kind of optimization method of machine learning, includes the following steps S1-S3:
S1 obtains at least one learning data；
In step S1, learning data includes training data and/or test data, and training data refers to can use after marking
In the sample data of machine learning model learning training；Test data refers to that is do not marked learns for test machine learning model
The data of training result.
S2, identification learning data obtain recognition result, recognition result include: learning data characteristic pattern and/or
Practise classification belonging to data；
S3, when recognition result meets preset selection rule, using the corresponding learning data of recognition result as machine
The optimization data of device learning model.These optimization data can be used to improve the performance of machine learning model.
As an implementation, preset Feature Selection rule is resolution and/or study based on characteristic pattern
The quantity of classification belonging to data is set.
Wherein, the resolution of characteristic pattern includes: low resolution and/or high resolution.Wherein, low resolution and high resolution
It is that is, low resolution characteristic pattern and high resolution characteristic pattern for characteristic pattern, low resolution characteristic pattern and high resolution are special
The division of sign figure can be found in following embodiment introduction:
In step S2, machine learning model identification learning data can be used to obtain characteristic pattern, wherein machine learning mould
Type is to have trained in advance, and have turned on what classification mode can classify to learning data.
Fig. 4 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention two.
As shown in figure 4, the characteristic pattern of identification learning data, includes the following steps S401-S403:
At least one machine learning data is inputted trained machine learning model in advance by S401；
S402 obtains machine learning model and extracts the characteristic pattern that feature obtains at least one machine learning data；
Wherein, machine learning model is when carrying out learning training to training data or test data, each layer can extract it is more
A characteristic pattern can get multiple characteristic patterns in each layer for each training data or test data, wherein every
One layer of characteristic pattern quantity got is determined by the convolution nuclear volume set.
S403, the identification probability based on characteristic pattern under each classification of machine learning model, calculates the identification of characteristic pattern
Degree；
Wherein, resolution is accuracy of the machine learning model to the recognition result of characteristic pattern.Specifically to characteristic pattern
Identification probability under all categories asks standard deviation to obtain.Specifically, standard deviation refers to measure characteristic pattern each defeated
The probability value under classification deviates the degree of probability average out.Standard deviation is smaller, represents characteristic pattern under each output classification
It is fewer that probability value deviates probability average, conversely, standard deviation is bigger, then represents probability of the characteristic pattern under each output classification
It is more that value deviates probability average.Therefore, standard deviation is able to reflect machine learning model to the resolution of characteristic pattern.Standard
Deviation includes standard deviation in population and sample standard deviation, and the embodiment of the present invention can be calculated discrete using standard deviation in population
Degree can also calculate dispersion using sample standard deviation, be not limited thereto.
Fig. 5 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention three.
As shown in figure 5, including the following steps S404a after step S403:
Whether S404, the resolution of judging characteristic figure are lower than the first resolution threshold value；
S405a identifies that this feature figure is low resolution if the resolution of a certain characteristic pattern is lower than the first resolution threshold value
Characteristic pattern.
Before above-mentioned steps S404a is executed, the resolution and first for the characteristic pattern for needing to be calculated in step S403
Resolution threshold value is compared, if being lower than the first resolution threshold value, then it is assumed that it is low resolution characteristic pattern, is known if being higher than first
It does not spend, then needs to carry out judgement as the following examples:
Fig. 5 described embodiment of the present invention is described in detail below by citing:
Example one:
Still by taking convolutional neural networks model shown in FIG. 1 as an example, obtains a new picture (hereinafter referred to as test picture) and make
For input, it is input in the machine learning model for opening classification mode after training, all spies of the test picture in layer 3 network
20 class probability distribution results that sign figure is got as shown in Figure 6 (Fig. 6 illustrates only the characteristic pattern 3.2 distinct with Fig. 2,
Other characteristic patterns are referring to fig. 2), the standard deviation of characteristic pattern 3.1 and characteristic pattern 3.2 is calculated based on probability distribution result shown in fig. 6
Difference:
The standard deviation of characteristic pattern 3.1 are as follows:
Std Dev(0.005,0,0.2,0,0.01,0,0,0.3,0,0.01,0,0.45,0,0,0.005,0.01,0,0,
0.01,0)=0.12203
The standard deviation of characteristic pattern 3.2 are as follows:
Std Dev(0.06,0.05,0.04,0.06,0.05,0.05,0.04,0.06,0.04,0.05,0.05,0.05,
0.06,0.05,0.05,0.05,0.06,0.04,0.05,0.04)=0.00725
Assuming that resolution threshold value is 0.1, then by the above calculated result, it can be seen that the high resolution of characteristic pattern 3.1, feature
Fig. 3 .2 is low resolution.
Fig. 7 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention four.
As shown in fig. 7, including the following steps S404b after step S403:
Whether S404, the resolution of judging characteristic figure are higher than the second resolution threshold value；
S405b identifies that this feature figure is high resolution if the resolution of a certain characteristic pattern is higher than the second resolution threshold value
Characteristic pattern.
Before above-mentioned steps S404a is executed, the resolution and second for the characteristic pattern for needing to be calculated in step S403
Resolution threshold value is compared, if being higher than the second resolution threshold value, then it is assumed that it is high resolution characteristic pattern, if knowing lower than first
It does not spend, then it is assumed that be low resolution characteristic pattern.
It should be noted that the implementation of above-mentioned steps S404a and S404b can be execution arranged side by side, it is also possible to wherein
The judgement of another embodiment is carried out in one ungratified situation, it is specific such as above embodiment introduction.
Corresponding to step S404a and step S404b, the first resolution threshold value is the first standard deviation threshold, if a certain spy
Levy figure standard deviation less than the first standard deviation threshold, then represent its deviate probability average it is larger, therefore, can be determined
For the characteristic pattern of low resolution.Similarly, corresponding to step S404b, the second resolution threshold value is the second standard deviation threshold, if
The standard deviation of a certain characteristic pattern be greater than the second standard deviation threshold, then represent its deviate probability average it is larger, therefore, can will
Its characteristic pattern for being determined as low resolution.
Since resolution picture placed in the middle is for machine learning model, certain category feature therein is during model training
The sample possessed is more, therefore, this part picture no too big use for the performance of hoisting machine learning model, and wrap
The learning data of the characteristic pattern of characteristic pattern or high resolution containing low resolution is construed as during model training
The sample possessed is less, then it is assumed that is the sample data big to machine learning model performance boost, data mark personnel can should
Type data are labeled as sample data, to help machine learning model to carry out learning training and then improving performance.
Fig. 7 described embodiment is described in detail below by an example:
Example two:
By taking convolutional neural networks model shown in FIG. 1 as an example, opened after the new test picture 3 in addition obtained is input to training
Open the network of classification mode, (Fig. 8 illustrates only that there are areas with Fig. 2 for available characteristic pattern probability distribution as shown in Figure 8
Another characteristic Fig. 3 .2, other characteristic patterns are referring to fig. 2):
Specifically, 3.2 probability distribution of characteristic pattern according to figure 8 calculates standard deviation, specific as follows:
StdDev(0,0.005,0.2,0,0.01,0,0,0.3,0.45,0,0.01,0,0,0,0.005,
0.01,0,0,0.01,0)=0.12203
Pass through the comparison of result above and resolution threshold value, it can be deduced that: to the resolution of the characteristic pattern 3.2 of low resolution
It is higher, therefore new test picture 3 is considered that the apparent sample data of model improving performance can be trained.
It as another embodiment, is based on classification belonging to learning data in preset Feature Selection rule
In the embodiment of quantity setting, if the quantity of classification belonging to learning data is less than predetermined quantity, the category will be included
Learning data is as optimization data, otherwise it is assumed that it is the data little to machine learning model performance boost.Before this,
By counting classification belonging to a large amount of learning data, if statistics obtains the negligible amounts of a certain classification, then it is assumed that
Learning data comprising the less classification is the sample data big to machine learning model performance boost, and data mark personnel and can incite somebody to action
The type data are labeled as sample data, to help machine learning model to be trained and then improving performance.Specifically
Counting can also be realized by the way of manual identified and statistics by the way of machine learning identification and statistics,
Specific such as following embodiment introduction:
Fig. 9 is that statistical nature figure goes out under each output classification in a kind of optimization method of machine learning of the embodiment of the present invention
The flow chart of existing number.
As shown in figure 9, including the following steps S1101-S1104:
Learning data is inputted trained machine learning model in advance by S1101；
S1102 obtains machine learning model and extracts the characteristic pattern that feature obtains at least one learning data；
S1103 counts the classification that each characteristic pattern is got；
S1104 calculates the quantity of the characteristic pattern under each classification based on the classification that each characteristic pattern is got.
The embodiment of the present invention is identified by the way that a large amount of learning data to be input in machine learning model, to obtain
To the characteristic pattern about these learning datas, these characteristic patterns have a probability results under each classification, if probability knot
Fruit is 0, then it represents that this feature figure does not get the category, if probability results are not 0, then it is assumed that this feature figure has got the category,
The classification finally got to all characteristic patterns counts, and obtains having characteristic pattern quantity under each classification, as study number
According to the number got under the category, the i.e. quantity of classification belonging to learning data.
In another optional embodiment, if the characteristic pattern of learning data is not belonging to the characteristic pattern of training data
One of, it may be considered that it is also to belong to the data big to machine learning model performance boost, training data is machine
Device learning model has carried out the data after learning training to it.
Fig. 9 described embodiment of the present invention is described in detail below by citing:
Example three:
Assuming that training data is 1000 pictures trained, the picture that 1000 had been trained is sequentially input into training
In the machine learning model for opening classification mode afterwards, i.e., machine learning model shown in FIG. 1, and count 1000 pictures and correspond to machine
Each classification that all characteristic patterns that every layer network extracts in 5 layer networks before device learning model are got.
With first trained picture (hereinafter referred to as training picture 1), in the 3rd layer of all characteristic patterns (characteristic pattern 3.1, spy
Sign Fig. 3 .2, characteristic pattern 3.3 and characteristic pattern 3.4) each classification for being got is as shown in Figure 2:
The classification that characteristic pattern 3.1 is got includes: classification 1,3,5,8,10,12,15,16 and 19；
The classification that characteristic pattern 3.2 is got includes: classification 2,3,10,11,14,15,16,18 and 19；
Does the classification that characteristic pattern 3.3 is got include: classification?,?,?,?,?,?,? with?；
Does the classification that characteristic pattern 3.4 is got include: classification?,?,?,?,?,?,?,? with?；
Figure 10 is the statistical result that 1000 pictures are got with classification in all characteristic patterns of layer 3 network, it is assumed that predetermined
Quantity is 100, and as seen in Figure 10, the classification 1,9,11 and 20 quantity that 1000 pictures are got in layer 3 network are small
In predetermined quantity 100, then it is assumed that the characteristic pattern comprising classification 1,9,11 and 20 meets preset Feature Selection rule.
Similarly, 1000 pictures can be counted according to the statistical method of the classification for the characteristic pattern that above-mentioned layer 3 network is extracted
In the classification for the characteristic pattern that layer 1, layer 2, layer 4 and layer 5 extract, details are not described herein.
After embodiment above-mentioned obtains the optimization data big to machine learning model performance boost, can also carry out
Following processing: the characteristic pattern that will meet the learning data of preset selection rule carries out visualization processing.Wherein, due to machine
Device learning model includes multiple layers, and each layer in multiple layer includes multiple sublayers, and there are execute sequence between multiple sublayers.
Therefore, visualization processing is successively to carry out corresponding inverse processing operation to each sublayer using reversed execution sequence.Such as: if
The a certain layer of machine learning model is followed successively by convolutional layer and excitation layer from front to back, then above-mentioned steps are successively to carry out flyback to encourage place
Reason and deconvolution processing.
After the characteristic pattern to the learning data for meeting preset selection rule carries out visualization processing, enable to
Mark personnel are labeled the picture in the unlabeled data newly obtained with close feature according to visual characteristic pattern,
And then as training sample help neural network improving performance, avoid do not orient in the prior art by all new acquisitions
Unlabeled data is all labeled, and improves annotating efficiency, and be capable of the performance of targeted hoisting machine learning model, together
When also save mark cost.
Figure 11 is a kind of flow chart of embodiment of visualization processing in a kind of optimization method of machine learning of the present invention.
Figure 12 is the schematic illustration of visualization processing in a kind of optimization method of machine learning of the present invention.
Wherein, by taking machine learning model structure shown in FIG. 1 as an example, multiple layers of machine learning model refer to layer 1 to layer 5,
It from front to back successively again include convolutional layer, excitation layer and pond (i.e. from input layer to the direction of output layer) by taking layer 3 as an example
Layer.
As is illustrated by figs. 11 and 12, the step of visualization processing includes:
S110 carries out anti-pondization to the characteristic pattern for the learning data for meeting preset selection rule and handles；
Wherein, the operation of anti-pondization processing is set according to pond mode.Maximum value pondization can pass through each pond of record
Position that domain maximum value occurs and approximate reconstruction goes out the structure before pond, the pondization of various other modes can also be using similar to just
Method approximate reconstruction goes out the structure before pond.
S111 carries out anti-energized process to the characteristic pattern of anti-pondization treated learning data；
Wherein, the operation of anti-energized process is depending on energisation mode.Such as: Relu excitation can be again by Relu weight
Structure goes out the structure before motivating.
S112 carries out deconvolution processing to the characteristic pattern of flyback learning data of encouraging that treated.
Wherein, deconvolution processing reconstructs the structure before convolution using Deconvolution Filters, and Deconvolution Filters are convolution
The transposition of filter.
For the visualization processing operation that Figure 11 and Figure 12 are described, be on the basis of example one and example three into
Row, namely visualization processing is carried out to the characteristic pattern for meeting preset selection rule that embodiment one obtains, at visualization
Managing, which can help to mark personnel, quickly selects the picture comprising the low resolution characteristic pattern or high resolution characteristic pattern to be labeled,
Or selection is labeled comprising the classification got lower than the characteristic pattern of pre-determined number, or is to the test picture newly obtained
In with aforementioned two kinds of characteristic patterns have close feature picture be labeled, as training sample help neural network enhancing
Energy.Compared with the prior art do not orient all new datas are all labeled more efficiently, and also can more have needle
To the promotion network performance of property, while cost can also be saved.
As a preferred embodiment, in the step S4 of the embodiment of the present invention one: if a certain training data or survey
The characteristic pattern of examination data meets preset Feature Selection rule, can not also directly carry out to training data or test data
Visualization processing, but obtain in new test data input machine learning model, pass through the identification knot to new test data
Fruit determines whether it can be used as the data big to machine learning model performance boost.It is specific as follows:
Figure 13 is a kind of flow chart of the optimization method of machine learning of the embodiment of the present invention seven.
As shown in figure 13, further include following steps:
S1301 obtains new learning data；
S1302 meets preset choosing if classification belonging to new learning data is the classification less than predetermined quantity
Take rule.
The embodiment of the present invention is executed on the basis of previous embodiment, that is, is obtaining the classification less than predetermined quantity
Afterwards, visualization processing is not carried out to characteristic pattern, but new learning data is directly inputted into machine learning model, judge that it is got
Classification in whether comprising less than predetermined quantity classification, if comprising directly the data are labeled, is advantageous in that and is not required to
Visualization processing is carried out, is directly identified by machine learning model it may determine that whether it can be used as optimization data, is subtracted
The step for having lacked visualization processing.
Such as: if being less than classification belonging to the learning data of predetermined quantity is classification 1,3,8 and 9, and new learning data
Affiliated classification is 1 and 5, then the classification 1 of new learning data belongs to the class of the characteristic pattern of the training data less than predetermined quantity
Not, classification 5 is not belonging to the classification of the characteristic pattern of the training data less than predetermined quantity.Therefore, include in new learning data
Classification 1 can be used as optimization data.
Below by citing Figure 13 described embodiment of the present invention is described in detail, example third is that example one it
After execute, i.e., get the number for taking neural network classification lower than pre-determined number characteristic pattern after, can also be directly it is right
Characteristic pattern carries out visualization processing, but executes and operate shown in following example three:
Example four:
Assuming that the data obtained be one it is new do not mark picture, hereinafter referred to as new test picture 1, after being inputted training
In the machine learning model (network of machine learning model shown in FIG. 1) for opening classification mode, it is assumed that this sublevel 1 obtains feature
Fig. 1 .1 and 1.2, layer 2 obtain characteristic pattern 2.1~2.4, and layer 3 obtains characteristic pattern 3.1~3.3, and layer 4 obtains characteristic pattern 4.1 and 4.2,
Layer 5 obtains characteristic pattern 5.1~5.3.
Recognition result is as shown in figure 14, it can be seen that characteristic pattern 3.1 has got classification 9 and classification 11, and characteristic pattern 3.2 takes
Classification 1 and classification 20 are arrived.Belong in the classification 1,9,11 and 20 that layer 3 is got less than the taken classification of the characteristic pattern of predetermined quantity
One of them (shown in one Fig. 2 of example), it is therefore contemplated that newly test picture 1 is that can significantly improve machine learning model performance
Data, mark personnel can mark for test chart piece 1.
Continue for new test picture 2 to be input to the network for opening classification mode after training, it is assumed that this sublevel 1 obtains characteristic pattern
1.1~1.4, layer 2 obtains characteristic pattern 2.1~2.3, and layer 3 obtains characteristic pattern 3.1 and 3.2, and layer 4 obtains characteristic pattern 4.1~4.3, layer
5 obtain characteristic pattern 5.1~5.4.Each classification that the characteristic pattern that layer 3 obtains is got the case where, as shown in figure 15, it can see
It arrives, does not get one of classification 1,9,11 and 20.Therefore, that is, think that newly testing picture 2 not is that can significantly improve machine
The data of learning model performance.
The method of above embodiments of the present invention is described in detail below by two complete examples:
Example five:
By taking pilotless automobile as an example, as shown in Figure 16 (a), under a scene, front detection zone image is four vehicles
Road, two automobiles, several trees.The characteristic pattern that black car is exported at each layer of convolutional neural networks is tracked, specific as follows:
The characteristic pattern of 1 output of layer: the texture information of automobile；
The characteristic pattern of 2 output of layer: the boundary information of automobile；
The characteristic pattern of 3 output of layer: the profile information of automobile；
The characteristic pattern of 4 output of layer: the component information of automobile, such as tire, vehicle window, car light, bumper；
The characteristic pattern of 5 output of layer: automobile.
After opening trained picture by counting this scene 1000, it is assumed that the tire characteristics figure for obtaining the output of layer 4 may return
The 20 category feature statistical magnitudes belonged to are respectively as follows: classification 1: automobile tire -500, and classification 2: classification 3: cable tyre -40 rubs
Motorcycle tire -80, classification 4: turntable -90, classification 5:? ... ... classification 20:?, then convolutional neural networks think cable tyre feature
It is less, it needs to find the picture with cable tyre feature and is labeled as training data.
Example six:
By taking wood grain identifies as an example, as shown in Figure 16 (b), this wood grain is tracked in each convolutional layer output of convolutional neural networks
Characteristic pattern, specific as follows:
The characteristic pattern of 1 output of layer: the fiber information of wood grain；
The characteristic pattern of 2 output of layer: the colouring information of wood grain；
The characteristic pattern of 3 output of layer: the envelope information of wood grain；
The characteristic pattern of 4 output of layer: the defect information of wood grain；
The characteristic pattern of 5 output of layer: wood grain is integrally moved towards.
After counting 1000 wood grain pictures, 20 category feature statistical magnitudes may belonging to of characteristic pattern that layer 4 exports
It is respectively as follows: classification 1: rotten -100, classification 2: crackle -80, classification 3: movable joint -50, classification 4: dying for the sake of honour -8, classification 5:? ... ... class
Other 20:?, then the convolutional neural networks feature that thinks to die for the sake of honour is less, needs to find the picture with feature of dying for the sake of honour as training data
It is labeled.
Figure 17 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention 11.
As shown in figure 17, a kind of optimization system of machine learning, comprising: module is obtained, for obtaining at least one study
Data；Identification module, the learning data, obtains recognition result for identification, and the recognition result includes: the learning data
Characteristic pattern and/or the learning data belonging to classification, and meet preset selection rule in the recognition result
When, using the corresponding learning data of the recognition result as the optimization data of the machine learning model.It is preset
Selection rule is the quantity setting of classification belonging to resolution and/or the learning data based on the characteristic pattern.Characteristic pattern
Resolution include: low resolution and/or high resolution.
Figure 18 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention 12.
As shown in figure 18, further includes: characteristic pattern obtains module, for obtaining preparatory trained machine learning model to extremely
Few machine learning data extract the characteristic pattern that feature obtains；Resolution computing module, for being based on the characteristic pattern
Identification probability under each classification of the machine learning model, calculates the resolution of the characteristic pattern.
Further, as shown in figure 19, resolution computing module includes: resolution computing unit, for the characteristic pattern
Identification probability under all categories seeks standard deviation, the resolution as characteristic pattern.Resolution computing module, is also used at certain
When the resolution of one characteristic pattern is lower than the first resolution threshold value, determine that this feature figure is the characteristic pattern of low resolution.Resolution meter
Module is calculated, is also used to determine that this feature figure is high resolution when the resolution of a certain characteristic pattern is higher than the second resolution threshold value
Characteristic pattern.
Further, identification module is also used to, and when the quantity of the classification belonging to learning data is less than predetermined quantity, is then identified
To meet preset selection rule.
Figure 20 is a kind of structural schematic diagram of the optimization system of machine learning of the embodiment of the present invention 12.
As shown in figure 20, optimization system further include: statistical module, for counting belonging at least one described learning data
The quantity of classification；Wherein, the statistical module includes: input submodule, for will train learning data input training in advance
Good machine learning model；Characteristic pattern acquisition submodule obtains the machine learning model and has trained learning data to mention to described
Take the characteristic pattern that feature obtains；Classification statistic submodule, the classification got for counting each characteristic pattern；Characteristic pattern quantity calculates
Submodule, the classification for being got based on each characteristic pattern, calculates the quantity of the characteristic pattern under each classification.
Further, identification module be also used to the classification belonging to new learning data be less than predetermined quantity classification when,
It is then identified as meeting preset selection rule.
It should be noted that a kind of optimization system of machine learning of the present invention is and is related to one kind of computer program process
The one-to-one system of the optimization method of machine learning, due to having been flowed to a kind of the step of optimization method of machine learning preceding
Journey is described in detail, and no longer repeats herein a kind of implementation process of the optimization system of machine learning.
The embodiment of the invention also provides a kind of non-transient computer readable storage medium, non-transient computer readable storages
Medium storing computer instruction, the method that computer instruction is used to that computer to be made to execute any of the above-described a embodiment.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
As shown in figure 21, it is a kind of execute preceding method electronic equipment, including one or more processors 2701 and with
The memory 2702 of one or more processors communication connection, takes a processor as an example in Figure 21.
Electronic equipment can also include: input unit 2703 and output device 2704, and input unit 2703 is obtained for inputting
At least image taken, output device 2704 is for exporting the data for meeting scheduled Feature Selection rule got.
Processor 2701, memory 2702, input unit 2703 and output device 2704 can by bus or other
Mode connects, in Figure 21 for being connected by bus.
Memory 2702 is used as a kind of non-transient computer readable storage medium.Can be used for storing non-transient software program,
Non-transient computer executable program, such as the corresponding software journey of the optimization method of one of embodiment of the present invention machine learning
Sequence, instruction and module.Processor 2701 by operation be stored in memory 2702 non-transient software program, instruction and
Module executes the various function application and data processing of a kind of optimization system of machine learning, i.e. the realization above method is implemented
The method and step of example.
Memory 2702 may include storing program area and storage data area, wherein storing program area can store operation system
Application program required for system, at least one function；Storage data area can be stored according to the optimization system of machine learning a kind of
Use created data etc..In addition, memory 2702 may include high-speed random access memory, it can also include non-transient
Memory, for example, at least a disk memory, flush memory device or other non-transient solid-state memories.In some implementations
In example, optional memory 2702 includes the memory remotely located relative to processor 2701, these remote memories can lead to
Cross a kind of optimization system of network connection to machine learning.The example of above-mentioned network includes but is not limited to internet, enterprises
Net, local area network, mobile radio communication and combinations thereof.
Input unit 2703 can receive an at least image for input, and generate a kind of engineering with the image of input
The related key signals input of the user setting and function control of the optimization system of habit.Input unit 2703 may include touching
Screen, keyboard etc. also may include wireline interface, wireless interface etc..Output device 2704 may include that display screen etc. shows equipment.
One or more software programs, instruction are stored in memory 2702, are held when by one or more processors 2701
When row, the optimization method of one of above-mentioned any means embodiment machine learning is executed.
In embodiments of the present invention, one or more processors can: execute a kind of machine of aforementioned any embodiment
The optimization method of study.
The present invention is intended to provide a kind of data capture method and system that can be obviously improved machine learning performance, has
Below the utility model has the advantages that
(1) by obtaining at least one training data or test data, and it is input to preparatory trained machine learning mould
Type weight, and then the characteristic pattern obtained when machine learning model extracts feature at least one training data or test data is obtained,
After obtaining characteristic pattern, selected in all characteristic patterns according to preset Feature Selection rule, in selection course, such as
The a certain characteristic pattern of fruit meets preset Feature Selection rule, then makees the corresponding training data of this feature figure or test data
For the data big to machine learning model performance boost, do not orient in the prior art all carry out all new datas is avoided
Mark, and it is also more targeted in hoisting machine learning model performance.
(2) visualization processing is carried out to the data finally got, so that data mark personnel can be quickly to machine
The apparent data of device learning model performance boost are labeled, and improve annotating efficiency.
(3) intermediate data obtained in machine learning model process is analyzed so that do not need manually to data into
Row analysis, and for manual analysis, it is also more accurate.
It should be understood that above-mentioned specific embodiment of the invention is used only for exemplary illustration or explains of the invention
Principle, but not to limit the present invention.Therefore, that is done without departing from the spirit and scope of the present invention is any
Modification, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.In addition, appended claims purport of the present invention
Covering the whole variations fallen into attached claim scope and boundary or this range and the equivalent form on boundary and is repairing
Change example.
Claims (10)
1. a kind of optimization method of machine learning characterized by comprising
Obtain at least one learning data；
Identify the learning data, obtain recognition result, the recognition result include: the learning data characteristic pattern and/or
Classification belonging to the learning data；
When the recognition result meets preset selection rule, the corresponding learning data of the recognition result is made
For the optimization data of the machine learning model.
2. optimization method according to claim 1, which is characterized in that the preset selection rule is based on described
The quantity of classification belonging to the resolution of characteristic pattern and/or the learning data is set.
3. optimization method according to claim 2, which is characterized in that the resolution of the characteristic pattern includes: low resolution
And/or high resolution.
4. optimization method according to claim 2 or 3, which is characterized in that the resolution of the characteristic pattern uses such as lower section
Method obtains:
At least one described machine learning data are inputted into trained machine learning model in advance；
It obtains the machine learning model and the characteristic pattern that feature obtains is extracted at least one described machine learning data；
Identification probability based on the characteristic pattern under each classification of the machine learning model, calculates the knowledge of the characteristic pattern
It does not spend.
5. optimization method according to claim 4, which is characterized in that the resolution of the characteristic pattern is to the characteristic pattern
Identification probability under all categories seeks standard deviation.
6. optimization method according to claim 4, which is characterized in that if the resolution of a certain characteristic pattern is lower than the first identification
Threshold value is spent, then this feature figure is the characteristic pattern of low resolution；And/or
If the resolution of a certain characteristic pattern is higher than the second resolution threshold value, this feature figure is the characteristic pattern of high resolution.
7. optimization method according to claim 2, which is characterized in that if the quantity of classification belonging to the learning data is small
In predetermined quantity, then meet preset selection rule.
8. optimization method according to claim 7, which is characterized in that further include: count at least one described learning data
The quantity of affiliated classification；
Wherein, the quantity for counting classification belonging at least one described learning data, comprising:
Learning data will have been trained to input trained machine learning model in advance；
It obtains the machine learning model and has trained learning data to extract the characteristic pattern that feature obtains to described；
Count the classification that each characteristic pattern is got；
Based on the classification that each characteristic pattern is got, the quantity of the characteristic pattern under each classification is calculated.
9. optimization method according to claim 8, which is characterized in that further include:
If classification belonging to new learning data is the classification less than predetermined quantity, meet preset selection rule.
10. a kind of electronic equipment, which is characterized in that including memory, processor and be stored on the memory and can be in institute
The computer program run on processor is stated, the processor realizes any one of claim 1-9 when executing described program
The step of the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810759470.4A CN109102078A (en) | 2018-07-11 | 2018-07-11 | A kind of optimization method of machine learning, system, computer storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810759470.4A CN109102078A (en) | 2018-07-11 | 2018-07-11 | A kind of optimization method of machine learning, system, computer storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109102078A true CN109102078A (en) | 2018-12-28 |
Family
ID=64845977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810759470.4A Pending CN109102078A (en) | 2018-07-11 | 2018-07-11 | A kind of optimization method of machine learning, system, computer storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109102078A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070542A (en) * | 2019-04-30 | 2019-07-30 | 王智华 | Machine learning method, device and the computer readable storage medium of intuition physics |
| CN111401526A (en) * | 2020-03-20 | 2020-07-10 | 厦门渊亭信息科技有限公司 | Model-universal deep neural network representation visualization method and device |
-
2018
- 2018-07-11 CN CN201810759470.4A patent/CN109102078A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070542A (en) * | 2019-04-30 | 2019-07-30 | 王智华 | Machine learning method, device and the computer readable storage medium of intuition physics |
| CN111401526A (en) * | 2020-03-20 | 2020-07-10 | 厦门渊亭信息科技有限公司 | Model-universal deep neural network representation visualization method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111400930B (en) | Power equipment small sample fault diagnosis method and system based on virtual and real twin space | |
| Kynkäänniemi et al. | Improved precision and recall metric for assessing generative models | |
| CN109800736B (en) | Road extraction method based on remote sensing image and deep learning | |
| CN109344736B (en) | Static image crowd counting method based on joint learning | |
| CN108154105B (en) | Underwater biological detection and identification method and device, server and terminal equipment | |
| D'Elia et al. | A tree-structured Markov random field model for Bayesian image segmentation | |
| US11468664B2 (en) | Machine learning to predict cognitive image composition | |
| CN109559295A (en) | Image analysis system, method, computer readable storage medium and electric terminal | |
| CN108831161A (en) | A kind of traffic flow monitoring method, intelligence system and data set based on unmanned plane | |
| TW201947463A (en) | Model test method and device | |
| CN114758081A (en) | Pedestrian re-identification three-dimensional data set construction method and device based on nerve radiation field | |
| CN109635825A (en) | Vehicle attribute detection method, device and storage medium | |
| CN109102078A (en) | A kind of optimization method of machine learning, system, computer storage medium and electronic equipment | |
| CN110826581A (en) | Animal number identification method, device, medium and electronic equipment | |
| CN105868102A (en) | Computer vision based mobile terminal application testing system and method | |
| CN112329815A (en) | Model training method, device and medium for detecting travel track abnormality | |
| CN114387270B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN109101922A (en) | Operating personnel device, assay, device and electronic equipment | |
| CN114942951A (en) | Fishing vessel fishing behavior analysis method based on AIS data | |
| CN110378952A (en) | A kind of image processing method and device | |
| CN106935038A (en) | One kind parking detecting system and detection method | |
| CN109583741A (en) | Driving trace methods of risk assessment and system | |
| CN110163140A (en) | Crowd density picture capturing method and device | |
| CN113838076A (en) | Method and device for labeling object contour in target image and storage medium | |
| JP2023044674A (en) | Visual analytics system to diagnose and improve deep learning model for movable object in autonomous driving |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181228 |
|
| RJ01 | Rejection of invention patent application after publication |