CN111382191A - Machine learning identification method based on deep learning - Google Patents
Machine learning identification method based on deep learning Download PDFInfo
- Publication number
- CN111382191A CN111382191A CN202010224110.1A CN202010224110A CN111382191A CN 111382191 A CN111382191 A CN 111382191A CN 202010224110 A CN202010224110 A CN 202010224110A CN 111382191 A CN111382191 A CN 111382191A
- Authority
- CN
- China
- Prior art keywords
- training
- data
- model
- parameters
- machine learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000013135 deep learning Methods 0.000 title claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 107
- 238000012360 testing method Methods 0.000 claims abstract description 16
- 238000003062 neural network model Methods 0.000 claims abstract description 13
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 12
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000007418 data mining Methods 0.000 claims abstract description 8
- 238000013528 artificial neural network Methods 0.000 claims abstract description 7
- 238000001514 detection method Methods 0.000 claims abstract description 7
- 238000005065 mining Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a machine learning identification method based on deep learning, which comprises the following steps: s1, acquiring index data related to the training parameter set based on the BP neural network model according to the identification target parameter; s2, calling a corresponding data mining module and/or a Faster R-CNN model according to the acquired related index data to realize automatic detection and acquisition of the training parameter set; s3, calling a corresponding data preprocessing model according to the identification target parameters and the acquired related index parameters to realize preprocessing of the training parameter set, and acquiring training set data and test set data; and S4, inputting the training set data into the corresponding machine learning model for learning training, then updating the parameters of the neural network according to forward propagation and backward propagation until the model converges, and storing the trained model. The whole process of the invention is automatically completed depending on different neural network models, and the machine learning efficiency can be greatly improved.
Description
Technical Field
The invention relates to the field of machine learning, in particular to a machine learning identification method based on deep learning.
Background
With the advent of the big data era, the technology of classifying and mining mass data is particularly important. In mass data mining, how to guide classification and mining of new data by using information classified and mined from existing data has become a new research hotspot. Particularly, when the number of samples of some tasks is small, the time cost for classifying and mining mass data can be effectively reduced and the information acquisition accuracy can be improved by utilizing multi-task learning.
The deep learning-based method is proved to be an effective and robust information classification method in practice. Deep neural networks (e.g., deep convolutional neural networks) are the most representative machine learning methods. Deep learning models typically have tens of learnable data processing layers, hundreds of thousands, or even millions, of learnable parameters. Since a large number of parameters constitutes an extremely large learning space, a large amount of training data is usually required in order to obtain optimal model parameters. However, in order to train the deep learning model, a training data set with a large number of samples must be constructed, and the number of training samples is usually tens of thousands or more. However, it is very difficult to construct such a training set in practical applications.
When the deep learning method is used for a classification task, the traditional deep learning method requires that the class of a comparison sample of a classification model is the same as the class of a production sample, namely, the model can only classify the learned class, and if a new class of samples needs to be classified, the machine learning model needs to be retrained, or some adaptive training learning is carried out on the machine learning model. This also results in the training of machine learning models based on deep learning methods, which consumes a lot of training computational resources and long training learning time, limiting the convenience and versatility of use in practical applications.
Disclosure of Invention
In order to solve the problems, the invention provides a machine learning identification method based on deep learning.
In order to achieve the purpose, the invention adopts the technical scheme that:
a machine learning identification method based on deep learning comprises the following steps:
s1, acquiring index data related to the training parameter set based on the BP neural network model according to the identification target parameter;
s2, calling a corresponding data mining module and/or a Faster R-CNN model according to the acquired related index data to realize automatic detection and acquisition of the training parameter set;
s3, calling a corresponding data preprocessing model according to the identification target parameters and the acquired related index parameters to realize preprocessing of the training parameter set, and acquiring training set data and test set data;
s4, inputting the training set data into a corresponding machine learning model for learning training, then updating the parameters of the neural network according to forward propagation and backward propagation until the model converges, and storing the trained model;
and S5, predicting the model by using the trained model and the test set data, analyzing the model according to the training log and the prediction log of the model, drawing training and testing Accracy and loss curves, and judging whether the model can effectively perform target parameter identification.
Further, when new training data is found, the incidence relation between the new training data and the relevant index parameters is firstly established, then the new training data is converted into parameters expressed by the relevant index parameters, and the corresponding trained model can be input for training.
Further, the data preprocessing model in the step S3 adopts an inclusion _ V3 neural network model.
Further, when the data to be recognized cannot be recognized, firstly, feature data of the data to be recognized are extracted based on the convolutional neural network, then, an incidence relation between feature parameters and relevant index parameters is established, then, the data to be recognized are converted into parameters expressed by the relevant index parameters, and then, the corresponding trained model can be input for training.
Further, the relevant index data of the training parameters at least comprises types of the training parameters, and features or feature sets contained in the training parameters, and the field to which the training parameters belong.
In the scheme, the method comprises the following steps:
the training parameter set is obtained based on the relevant index data of the training parameter set, the pertinence of the training parameter set can be improved, the number of the training parameters can be reduced as much as possible, and the model training time is shortened.
The automatic detection and acquisition of the training parameter set are realized through the data mining module and/or the Faster R-CNN model, so that the automatic generation of the training parameter set can be quickly realized.
The method realizes fine adjustment of the trained model by constructing the incidence relation between the new training data and the related index parameters and then converting the new training data into the parameters expressed by the related index parameters, so that the trained model can quickly have the function of identifying the new parameters.
The whole process is automatically completed by depending on different neural network models, so that the machine learning efficiency is greatly improved.
Drawings
FIG. 1 is a flow chart of example 1 of the present invention.
Fig. 2 is a flowchart of embodiment 2 of the present invention.
Fig. 3 is a flowchart of embodiment 3 of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.
Example 1
A machine learning identification method based on deep learning comprises the following steps:
s1, acquiring index data related to the training parameter set based on the BP neural network model according to the identification target parameter; the related index data of the training parameters at least comprises the types (pictures/texts) of the training parameters, and the characteristics or characteristic sets contained in the training parameters, and the fields to which the training parameters belong, such as the financial field and the multimedia field;
s2, calling a corresponding data mining module and/or a Faster R-CNN model according to the acquired related index data to realize automatic detection and acquisition of the training parameter set; namely, all the obtained training parameter sets comprise the features or the feature sets in the related index data of the training parameter sets;
s3, calling a corresponding inclusion _ V3 neural network model according to the recognition target parameters and the obtained related index parameters to realize the preprocessing of a training parameter set, and obtaining training set data and test set data;
s4, inputting the training set data into a corresponding machine learning model for learning training, then updating the parameters of the neural network according to forward propagation and backward propagation until the model converges, and storing the trained model;
and S5, predicting the model by using the trained model and the test set data, analyzing the model according to the training log and the prediction log of the model, drawing training and testing Accracy and loss curves, and judging whether the model can effectively perform target parameter identification.
Example 2
A machine learning identification method based on deep learning comprises the following steps:
s1, acquiring index data related to the training parameter set based on the BP neural network model according to the identification target parameter; the training parameter related index data at least comprises the type of the training parameter, the characteristics or the characteristic set contained in the training parameter and the field to which the training parameter belongs;
s2, calling a corresponding data mining module and/or a Faster R-CNN model according to the acquired related index data to realize automatic detection and acquisition of the training parameter set;
s3, calling a corresponding inclusion _ V3 neural network model according to the recognition target parameters and the obtained related index parameters to realize the preprocessing of a training parameter set, and obtaining training set data and test set data;
s4, inputting the training set data into a corresponding machine learning model for learning training, then updating the parameters of the neural network according to forward propagation and backward propagation until the model converges, and storing the trained model;
and S5, predicting the model by using the trained model and the test set data, analyzing the model according to the training log and the prediction log of the model, drawing training and testing Accracy and loss curves, and judging whether the model can effectively perform target parameter identification.
S6, converting the new training data into parameters expressed by the relevant index parameters, inputting the parameters into the corresponding trained model, and training to obtain a new model; specifically, when new training data is found, feature data of the new training data is extracted based on the convolutional neural network, an association relationship between the feature data and relevant index parameters (here, feature or feature set parameters included in the training parameters) is constructed, then the new training data is converted into parameters expressed by the relevant index parameters (the feature or feature set parameters included in the training parameters), and then the corresponding trained model can be input for training.
Example 3
A machine learning identification method based on deep learning comprises the following steps:
s1, acquiring index data related to the training parameter set based on the BP neural network model according to the identification target parameter; the training parameter related index data at least comprises the type of the training parameter, the characteristics or the characteristic set contained in the training parameter and the field to which the training parameter belongs;
s2, calling a corresponding data mining module and/or a Faster R-CNN model according to the acquired related index data to realize automatic detection and acquisition of the training parameter set;
s3, calling a corresponding inclusion _ V3 neural network model according to the recognition target parameters and the obtained related index parameters to realize the preprocessing of a training parameter set, and obtaining training set data and test set data;
s4, inputting the training set data into a corresponding machine learning model for learning training, then updating the parameters of the neural network according to forward propagation and backward propagation until the model converges, and storing the trained model;
and S5, predicting the model by using the trained model and the test set data, analyzing the model according to the training log and the prediction log of the model, drawing training and testing Accracy and loss curves, and judging whether the model can effectively perform target parameter identification.
S6, when the data to be recognized cannot be recognized, converting the data to be recognized into a trained model corresponding to parameter input expressed by related index parameters for training, and acquiring a new model; specifically, feature data of data to be recognized is extracted based on a convolutional neural network, then an incidence relation between feature parameters and related index parameters (here, feature or feature set parameters included in training parameters) is established, then the data to be recognized is converted into parameters expressed by the related index parameters (the feature or feature set parameters included in the training parameters), and then a corresponding trained model can be input for training.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (5)
1. A machine learning identification method based on deep learning is characterized in that: the method comprises the following steps:
s1, acquiring index data related to the training parameter set based on the BP neural network model according to the identification target parameter;
s2, calling a corresponding data mining module and/or a Faster R-CNN model according to the acquired related index data to realize automatic detection and acquisition of the training parameter set;
s3, calling a corresponding data preprocessing model according to the identification target parameters and the acquired related index parameters to realize preprocessing of the training parameter set, and acquiring training set data and test set data;
s4, inputting the training set data into a corresponding machine learning model for learning training, then updating the parameters of the neural network according to forward propagation and backward propagation until the model converges, and storing the trained model;
and S5, predicting the model by using the trained model and the test set data, analyzing the model according to the training log and the prediction log of the model, drawing training and testing Accracy and loss curves, and judging whether the model can effectively perform target parameter identification.
2. The deep learning-based machine learning identification method of claim 1, wherein: when new training data are found, the incidence relation between the new training data and the relevant index parameters is firstly established, then the new training data are converted into parameters expressed by the relevant index parameters, and the corresponding trained model can be input for training.
3. The deep learning-based machine learning identification method of claim 1, wherein: the data preprocessing model in the step S3 adopts an inclusion _ V3 neural network model.
4. The deep learning-based machine learning identification method of claim 1, wherein: when the data to be recognized cannot be recognized, firstly, feature data of the data to be recognized are extracted based on a convolutional neural network, then, an incidence relation between feature parameters and related index parameters is established, then, the data to be recognized are converted into parameters expressed by the related index parameters, and then, the corresponding trained model can be input for training.
5. The deep learning-based machine learning identification method of claim 1, wherein: the training parameter related index data at least comprises the type of the training parameter, the characteristics or the characteristic set contained in the training parameter, and the field to which the training parameter belongs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010224110.1A CN111382191A (en) | 2020-03-26 | 2020-03-26 | Machine learning identification method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010224110.1A CN111382191A (en) | 2020-03-26 | 2020-03-26 | Machine learning identification method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111382191A true CN111382191A (en) | 2020-07-07 |
Family
ID=71219986
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010224110.1A Pending CN111382191A (en) | 2020-03-26 | 2020-03-26 | Machine learning identification method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111382191A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115793490A (en) * | 2023-02-06 | 2023-03-14 | 南通弈匠智能科技有限公司 | Intelligent household energy-saving control method based on big data |
-
2020
- 2020-03-26 CN CN202010224110.1A patent/CN111382191A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115793490A (en) * | 2023-02-06 | 2023-03-14 | 南通弈匠智能科技有限公司 | Intelligent household energy-saving control method based on big data |
| CN115793490B (en) * | 2023-02-06 | 2023-04-11 | 南通弈匠智能科技有限公司 | Intelligent household energy-saving control method based on big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109582793B (en) | Model training method, customer service system, data labeling system and readable storage medium | |
| CN109471938B (en) | Text classification method and terminal | |
| CN103617435B (en) | Image sorting method and system for active learning | |
| WO2021057133A1 (en) | Method for training document classification model, and related apparatus | |
| US20220092359A1 (en) | Image data classification method, device and system | |
| CN117056834A (en) | Big data analysis method based on decision tree | |
| CN111582341A (en) | User abnormal operation prediction method and device | |
| CN116823793A (en) | Device defect detection method, device, electronic device and readable storage medium | |
| CN106484913A (en) | Method and server that a kind of Target Photo determines | |
| CN117523218A (en) | Label generation, training of image classification model and image classification method and device | |
| CN111382191A (en) | Machine learning identification method based on deep learning | |
| CN117876018A (en) | Method, device, electronic equipment and storage medium for identifying and predicting potential customers | |
| CN117372144A (en) | Wind control strategy intelligent method and system applied to small sample scene | |
| CN111488950A (en) | Classification model information output method and device | |
| CN110879821A (en) | Method, device, equipment and storage medium for generating rating card model derivative label | |
| CN115269855A (en) | Paper fine-grained multi-label labeling method and device based on pre-training encoder | |
| CN112948251B (en) | Automatic software testing method and device | |
| CN111798237B (en) | Abnormal transaction diagnosis method and system based on application log | |
| CN114842301A (en) | Semi-supervised training method of image annotation model | |
| CN112613521A (en) | Multilevel data analysis system and method based on data conversion | |
| CN115587297A (en) | Method, apparatus, device and medium for constructing image recognition model and image recognition | |
| CN112633344B (en) | Quality inspection model training method, device, equipment and readable storage medium | |
| CN115237739B (en) | Analysis method, device and equipment for board card running environment and readable storage medium | |
| CN109214411B (en) | Verification method and system for identifying typical pictures to newly added entities based on training model | |
| CN110728615B (en) | Steganalysis method based on sequential hypothesis testing, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200707 |