CN107563496A - A kind of deep learning mode identification method of vectorial core convolutional neural networks - Google Patents
A kind of deep learning mode identification method of vectorial core convolutional neural networks Download PDFInfo
- Publication number
- CN107563496A CN107563496A CN201710665575.9A CN201710665575A CN107563496A CN 107563496 A CN107563496 A CN 107563496A CN 201710665575 A CN201710665575 A CN 201710665575A CN 107563496 A CN107563496 A CN 107563496A
- Authority
- CN
- China
- Prior art keywords
- neural networks
- convolutional neural
- mtr
- mtd
- core
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
The invention discloses a kind of deep learning mode identification method of vectorial core convolutional neural networks, methods described includes：(1) structure of design vector core convolutional neural networks, including input layer, convolutional layer, at least one full articulamentum and soft maximum grader output layer；Wherein, input is the image that size is m × n, and the core of l layers is all plThe vector of × 1 size is all 1 × qlThe vector of size, NlRepresent the convolution kernel number of l layers；Rule of thumb select pl(or ql) and NlValue, the convolution algorithm step-length of l layers is sl(sl>=1), activation primitive all elects correction linear unit as；(2) iterations of convolutional neural networks is set, cost function is selected, utilizes training sample { (x1,y1),…,(xD,yD), according to back-propagation algorithm, the parameter of vectorial core convolutional neural networks is learnt；(3) judge whether to complete iterations, if it is not complete, then continuing to train；If completing iterations, tested in the network model that test sample input is trained, obtain test result.
Description
Technical field
The invention belongs to the technical field in pattern-recognition and deep learning.Particular content is a kind of vectorial core convolutional Neural
The deep learning mode identification method of network.
Background technology
Deep learning is one new research direction of artificial intelligence field, in recent years in image recognition, computer vision etc.
The progress of making a breakthrough property in multiclass application.With the continuous development of deep learning, autocoder is generated
(AutoEncoder, AE), degree of deeply convinceing neutral net (Deep Belief Networks, DBNs), convolutional neural networks
Deep-neural-network models such as (Convolutional Neural Networks, CNN).Wherein because there are CNN weights to be total to
Enjoy, local receptor field, dimensionality reduction the features such as, be widely used.But CNN weight data amount is larger, with the number of plies
Increase, the training time is more, and consumption calculations resource is larger.
The content of the invention
The main object of the present invention is to provide a kind of deep learning mode identification method of vectorial core convolutional neural networks, purport
The data volume of weights and training time are being reduced, is reducing the loss of computing resource.
To achieve the above object, the technical solution adopted by the present invention is a kind of deep learning of vectorial core convolutional neural networks
Mode identification method, this method comprise the following steps：
(1) structure of design vector core convolutional neural networks, the structure include input layer L, 1≤L≤2000 convolutional layer,
At least one full articulamentum and soft maximum (Softmax) grader output layer；Wherein, include input layer L inputs is that size is
M × n, m, the image of n >=1,1≤l of l layers≤L core is all pl× 1,1≤plThe vector of≤m sizes is all 1 × ql, 1
≤qlThe vector of≤n sizes, NlRepresent the convolution kernel number of l layers, 1≤Nl≤5000；Rule of thumb select plAnd NlOr qlWith
NlValue, the convolution algorithm step-length of l layers is sl, sl>=1, activation primitive all elects correction linear unit (Rectified as
Linear Units, ReLU)；
(2) the iterations Epochs of convolutional neural networks is set, 1≤Epochs≤20000000, selects cost function,
Utilize training sample { (x1,y1),…,(xD,yD), according to back-propagation algorithm, the parameter of vectorial core convolutional neural networks is entered
Row study；
(3) judge whether to complete iterations, if it is not complete, then continuing to train；, will if completing iterations
Tested in the network model that test sample input trains, obtain test result.
If the input of Softmax grader output layers is U=(u1,…um)T, export as O=(o1,…om)T, then O=
Softmax (U), i.e.,
Wherein k is labeling number.Activation primitive ReLu expression formulas are：F (x)=max (0, x), wherein x are the defeated of each layer
Go out value, f (x) represents the output valve of activation primitive.
The present invention has the following advantages that compared with prior art：
Present invention employs Vector convolution core, reduces data volume and the training time of weights, reduces computed losses, add
Fast convergence rate.For the present invention in experiment test, the index of experimental results is better than existing convolutional neural networks, energy
It is enough that preferable recognition effect is obtained in image recognition.
Brief description of the drawings
Fig. 1 is the network structure of the present invention.
Fig. 2 is the network training method flow chart of the present invention.
Embodiment
Below in conjunction with the accompanying drawings and specific implementation case the invention will be further described.It should be understood that following case study on implementation
Described embodiment does not represent the consistent all embodiments of the present invention.In addition, it is to be understood that reading the present invention
After the content of instruction, those skilled in the art can make various changes or modifications to the present invention, and these equivalent form of values equally fall
In the application appended claims limited range.
The present invention provides a kind of deep learning mode identification method of vectorial core convolutional neural networks, applied to handwritten numeral
Identify in case study on implementation, comprise the following steps：
(1) structure of design vector core convolutional neural networks, including input layer, individual convolutional layers of L (1≤L≤2000), at least
One full articulamentum and soft maximum (Softmax) grader output layer；Wherein, input is that size is m × n (m, n >=1)
Image, the core of l layers (1≤l≤L) is all pl×1(1≤pl≤ m) size vector or be all 1 × ql(1≤ql≤ n) it is big
Small vector, Nl(1≤Nl≤ 5000) the convolution kernel number of l layers is represented；Rule of thumb select pl(or ql) and NlValue, l
The convolution algorithm step-length of layer is sl(sl>=1), activation primitive all elect as correction linear unit (Rectified Linear Units,
ReLU)；
Specifically, the vectorial core convolutional neural networks of the implementation case design include：One input layer, five convolutional layers,
One full articulamentum, a Softmax grader output layer.0th layer is input layer, and input is the gray-scale map that size is 28 × 28
Picture；1st layer of core size is all 6 × 1, and core number is 32, step-length 1；2nd layer of core size is all 1 × 6, and core number is 32,
Step-length is 1；3rd layer of core size is all 1 × 5, and core number is 48, step-length 2；4th layer of core size is all 1 × 5, core number
For 48, step-length 1；5th layer of core size is all 1 × 1, and core number is 64, step-length 2；If Softmax grader output layers
Input as U=(u1,…um)T, export as O=(o1,…om)T, then O=Softmax (U), i.e.,
Wherein k is labeling number.Activation primitive ReLu expression formulas are：F (x)=max (0, x), wherein x are the defeated of each layer
Go out value, f (x) represents the output valve of activation primitive.
(2) the iterations Epochs, 1≤Epochs of setting convolutional neural networks≤20000000 select cost function,
Utilize training sample { (x1,y1),…,(xD,yD), according to back-propagation algorithm, the parameter of vectorial core convolutional neural networks is entered
Row study；
Specifically, the iterations in the implementation case is set to 10000 times；The implementation case uses MNIST data sets, it
It is to be built by the CorinnaCortes in Google laboratories and the YannLeCun of Ke Lang research institutes of New York University, wherein 60000
Individual training sample, 10000 test samples.For the input picture of input layer as described in (1), output classification is 10 classes (digital 0-9 marks
Know).The test environment of the present invention is windows7, and 64 bit manipulation systems, program code is to use python3.5+tensorflow
Write；Hardware environment is NVIDIATeslaK40C GPU (two pieces), and interior save as is run in the environment of 128G.In network mould
In type training process, cost function elects softmax_cross_entropy_with_logits () as, is calculated using backpropagation
The stochastic gradient descent version of method is updated to weights and biasing；And in order to prevent over-fitting, used in full articulamentum
Dropout methods abandon neuron according to 0.25 probability.
(3) judge whether to complete iterations, if it is not complete, then continuing to train；, will if completing iterations
Tested in the network model that test sample input trains, obtain test result.
Emulation experiment shows：The present invention compares traditional convolutional neural networks, has the smaller (present invention of weight data amount：
928, CNN:3952), the training time less (present invention：0.69H,CNN:0.87H), the higher (present invention of discrimination：99.56%,
CNN:99.55%) the characteristics of.
The experiment effect of the present invention is as shown in table 1：
Parameter and the Experimental comparison results of the implementation case network model of table 1 and traditional convolutional neural networks
Remarks：Numeral " 300 " in full articulamentum refers to neuron number, and " H " represents hour.
Claims (2)
- A kind of 1. deep learning mode identification method of vectorial core convolutional neural networks, it is characterised in that：This method includes following Step, the structure of (1) design vector core convolutional neural networks, the structure include input layer L, 1≤L≤2000 convolutional layer, extremely A few full articulamentum and soft maximum grader output layer；Wherein, input layer L inputs is that size is m × n, m, n >=1 Image, 1≤l of l layers≤L core is all pl× 1,1≤plThe vector of≤m sizes is all 1 × ql, 1≤ql≤ n sizes Vector, NlRepresent the convolution kernel number of l layers, 1≤Nl≤5000；Rule of thumb select plAnd NlOr qlAnd NlValue, l layers Convolution algorithm step-length is sl, sl>=1, activation primitive all elects correction linear unit as；(2) the iterations Epochs of convolutional neural networks is set, 1≤Epochs≤2000000, selects cost function, is utilized Training sample { (x1,y1),…,(xD,yD), according to back-propagation algorithm, to the parameter of vectorial core convolutional neural networks Practise；(3) judge whether to complete iterations, if it is not complete, then continuing to train；, will test if completing iterations Tested in the network model that sample input trains, obtain test result.
- 2. a kind of deep learning mode identification method of vectorial core convolutional neural networks according to claim 1, its feature It is：If the input of Softmax grader output layers is U=(u1,…um)T, export as O=(o1,…om)T, then O= Softmax (U), i.e.,<mrow> <mi>O</mi> <mo>=</mo> <mfrac> <mn>1</mn> <mrow> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <msup> <mi>e</mi> <msub> <mi>u</mi> <mi>j</mi> </msub> </msup> </mrow> </mfrac> <mfenced open = "[" close = "]"> <mtable> <mtr> <mtd> <msup> <mi>e</mi> <msub> <mi>u</mi> <mn>1</mn> </msub> </msup> </mtd> </mtr> <mtr> <mtd> <msup> <mi>e</mi> <msub> <mi>u</mi> <mn>2</mn> </msub> </msup> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <msup> <mi>e</mi> <msub> <mi>u</mi> <mi>j</mi> </msub> </msup> </mtd> </mtr> </mtable> </mfenced> <mo>,</mo> </mrow>Wherein k is labeling number；Activation primitive ReLu expression formulas are：F (x)=max (0, x), wherein x are the output of each layer Value, f (x) represent the output valve of activation primitive.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710665575.9A CN107563496A (en) | 2017-08-07 | 2017-08-07 | A kind of deep learning mode identification method of vectorial core convolutional neural networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710665575.9A CN107563496A (en) | 2017-08-07 | 2017-08-07 | A kind of deep learning mode identification method of vectorial core convolutional neural networks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107563496A true CN107563496A (en) | 2018-01-09 |
Family
ID=60975035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710665575.9A Pending CN107563496A (en) | 2017-08-07 | 2017-08-07 | A kind of deep learning mode identification method of vectorial core convolutional neural networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107563496A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111798935A (en) * | 2019-04-09 | 2020-10-20 | 南京药石科技股份有限公司 | Universal compound structure-property correlation prediction method based on neural network |
| CN113574544A (en) * | 2019-03-15 | 2021-10-29 | 浜松光子学株式会社 | Convolutional neural network judgment basis extraction method and device |
| CN113780553A (en) * | 2021-09-09 | 2021-12-10 | 中山大学 | Deep learning model optimization method and system based on high-level comprehensive tool |
-
2017
- 2017-08-07 CN CN201710665575.9A patent/CN107563496A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113574544A (en) * | 2019-03-15 | 2021-10-29 | 浜松光子学株式会社 | Convolutional neural network judgment basis extraction method and device |
| CN111798935A (en) * | 2019-04-09 | 2020-10-20 | 南京药石科技股份有限公司 | Universal compound structure-property correlation prediction method based on neural network |
| CN113780553A (en) * | 2021-09-09 | 2021-12-10 | 中山大学 | Deep learning model optimization method and system based on high-level comprehensive tool |
| CN113780553B (en) * | 2021-09-09 | 2023-11-07 | 中山大学 | Deep learning model optimization method and system based on high-level comprehensive tool |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lin et al. | Research on convolutional neural network based on improved Relu piecewise activation function | |
| CN109271522B (en) | Comment emotion classification method and system based on deep hybrid model transfer learning | |
| CN104850890B (en) | Instance-based learning and the convolutional neural networks parameter regulation means of Sadowsky distributions | |
| CN109829541A (en) | Deep neural network incremental training method and system based on learning automaton | |
| CN107562784A (en) | Short text classification method based on ResLCNN models | |
| CN108197736A (en) | A kind of Air Quality Forecast method based on variation self-encoding encoder and extreme learning machine | |
| CN109325547A (en) | Non-motor vehicle image multi-tag classification method, system, equipment and storage medium | |
| CN108304826A (en) | Facial expression recognizing method based on convolutional neural networks | |
| CN107358293A (en) | A kind of neural network training method and device | |
| CN107885853A (en) | A kind of combined type file classification method based on deep learning | |
| CN107330514A (en) | A kind of Air Quality Forecast method based on integrated extreme learning machine | |
| CN105205448A (en) | Character recognition model training method based on deep learning and recognition method thereof | |
| CN107330446A (en) | A kind of optimization method of depth convolutional neural networks towards image classification | |
| CN108960301B (en) | Ancient Yi-nationality character recognition method based on convolutional neural network | |
| CN107729993A (en) | Utilize training sample and the 3D convolutional neural networks construction methods of compromise measurement | |
| CN107705556A (en) | A kind of traffic flow forecasting method combined based on SVMs and BP neural network | |
| CN109063719A (en) | A kind of image classification method of co-ordinative construction similitude and category information | |
| CN109766995A (en) | The compression method and device of deep neural network | |
| CN107563496A (en) | A kind of deep learning mode identification method of vectorial core convolutional neural networks | |
| WO2020143253A1 (en) | Method employing sparse autoencoder to cluster power system operation modes | |
| CN107563430A (en) | A kind of convolutional neural networks algorithm optimization method based on sparse autocoder and gray scale correlation fractal dimension | |
| CN109783887A (en) | A kind of intelligent recognition and search method towards Three-dimension process feature | |
| CN110390017A (en) | Target sentiment analysis method and system based on attention gate convolutional network | |
| CN109165699A (en) | Fine granularity image classification method | |
| CN109754079A (en) | A method of catastrophic forgetting is overcome based on parameter importance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180109 |