CN113988181B - Target classification method based on adaptive feedforward neural network - Google Patents
Target classification method based on adaptive feedforward neural network Download PDFInfo
- Publication number
- CN113988181B CN113988181B CN202111263644.6A CN202111263644A CN113988181B CN 113988181 B CN113988181 B CN 113988181B CN 202111263644 A CN202111263644 A CN 202111263644A CN 113988181 B CN113988181 B CN 113988181B
- Authority
- CN
- China
- Prior art keywords
- neural network
- weight
- feedforward neural
- adaptive
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a target classification method based on a self-adaptive feedforward neural network, which is characterized by comprising the following steps of: constructing a feedforward neural network model: reconstructing the nonlinear feedforward neural network system into a nonlinear system with linear weight value through a Taylor series expansion; and (3) taking the weight estimation error as a drive to update the weight: extracting a weight estimation error information variable according to the constructed auxiliary variable, and correspondingly designing the self-adaptive rate of weight updating; and (3) target classification: and classifying the target images according to the feedforward neural network model updated by the weight. The invention solves the limitation existing in the traditional neural network weight updating algorithm through an improved weight self-adaptive estimation method, and realizes the quick identification of the target in the image.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a target classification method based on a self-adaptive feedforward neural network.
Background
The target classification can realize target identification in the image and provide information data support for a decision-making system. In recent years, a deep learning method is widely researched by target classification, but a traditional target classification algorithm is not ideal in accuracy and robustness, the output error is mostly minimized through a gradient algorithm in the existing result to derive a classical neural network weight updating law, a large amount of data and time training of a neural network can be caused in the slow convergence process of the classical neural network weight updating law, and the time cost and the consumption cost of neural network learning are increased. A new approach is needed to solve the problem of neural networks.
Disclosure of Invention
The invention aims to: aiming at the existing problems, a target classification method based on an adaptive feedforward neural network is provided, and the technical problems of convergence limited by a traditional gradient algorithm, massive data training and time cost consumption in the traditional neural network are solved.
The technical scheme adopted by the invention is as follows:
the invention relates to a target classification method based on a self-adaptive feedforward neural network, which comprises the following steps of:
constructing a feedforward neural network model: reconstructing the nonlinear feedforward neural network system into a nonlinear system with linear weight value through a Taylor series expansion;
and (3) taking the weight estimation error as a drive to update the weight: extracting a weight estimation error information variable according to the constructed auxiliary variable, and correspondingly designing the self-adaptive rate of weight updating;
and (3) target classification: and classifying the target images according to the feedforward neural network model updated by the weight.
Preferably, the constructing the feedforward neural network model specifically includes:
step S100: collecting image data, labeling the training data of deep learning, and carrying out normalization processing on the collected image data;
step S200: setting an input layer, a hidden layer and an output layer of a feedforward neural network according to image data to be input and an image label;
step S300: introducing the input and output of the feedforward neural network and the gradient of each layer of weight, and reconstructing an adaptive feedforward neural network model:
wherein f (x, theta) is epsilon to R t For the constructed feedforward neural network, x ∈ R n×n Is the input matrix of the feedforward neural network, theta belongs to R r Is a vector after the weight matrix of the feedforward neural network is flattened, and the feedforward neural network estimates the weight
In the first order taylor series expansion of (b),
is the first order gradient of the feedforward neural network with respect to the weight theta, and v is the first order taylor expansion remainder.
Preferably, the step S200: determining the batch size of matrixes to be input by a neural network according to image data to be input, setting the image data as a matrix transformation method for neural network input, and selecting the number of nodes and an activation function of an input layer and a hidden layer of a feedforward neural network; and determining the number of nodes and an activation function of the output layer of the neural network according to the image label to be input, and initializing the weight of the neural network.
Preferably, the step S200 specifically includes the following steps:
step S210: based on the image data after normalization processing, performing matrix transformation on an image matrix, and converting the image matrix into a one-dimensional vector input by a neural network through one-dimensional projection or matrix flattening;
step S220: based on the image label, the number of output nodes of the neural network is set, the weight matrix of each corresponding layer is designed, the output dimension of the network layer is equal to the input dimension of the next network layer, and the nonzero matrix of each layer of weight is initialized.
Preferably, the feedforward neural network designed after step S220 is as follows:
wherein the content of the first and second substances,
is an activation function of the output layer,
is the input of the output layer, w t Is the weight matrix of the layer and,
activation function and input of hidden layer, g (x) matrix transformation, i.e. one-dimensional projection or matrix flattening, of image matrix, theta is weight of neural network, w 1 ,w 2 ,…,w t Is the vector after flattening.
Preferably, the method for updating the weight by using the weight estimation error as a drive specifically includes:
step S410: according to the neural network reconstructed in the step S300, the following auxiliary variable P epsilon R is constructed r×r ,Q∈R r ：
Wherein l >0 is a leakage term and κ >0 is a gain;
step S420: constructing a variable W epsilon R according to the auxiliary variable obtained in the step S410 r ,H∈R r ：
Wherein W is a weight estimation error variable
Is bounded, i.e.
The derivative of the variable W is as follows:
step S430: according to the variable W, H obtained in step S420, calculating the adaptive rate of the neural network weight update
Wherein gamma is>0 is the learning gain.
Preferably, the object classification method includes: and under the condition of reaching the test precision, realizing the stable convergence of the weight according to the self-adaption rate updated by the weight, and classifying the subsequent target images.
Preferably, the object classification method specifically includes the following steps:
step S510: continuously loading batch image data for the adaptive neural network, stopping image data loading under the condition that the test precision is continuously reduced for 5 times, wherein the weight under the condition of the highest test precision is the optimal weight after training;
step S520: based on the Lyapunov stability theory, the self-adaptive rate
Under the driving of the neural network, the weight of the neural network is updated, stable convergence is achieved under the test stopping condition in the step S510, the weight estimation error is consistent and finally bounded, the weight of the neural network is obtained, and the subsequent target images are classified.
Preferably, the data of the adaptive feedforward neural network are respectively stored and operated on a CPU and a GPU, and image data, label data, batch size, weight, nodes and gradients of weights of network layers are stored in the CPU; auxiliary variables P and Q, a weight estimation error variable W and a weight theta are stored in the GPU.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. the method solves the problems that the traditional target classification algorithm is not ideal in accuracy and robustness and the neural network of the traditional target classification algorithm needs a large amount of data and time for training, and improves the efficiency of the target classification algorithm.
2. The function of target recognition in the image is realized. A new weight updating adaptive law is developed, the extracted weight estimation error is used as a drive, the adaptive rate of weight updating is designed, and the convergence of the weight of the feedforward neural network is ensured based on the Lyapunov stability theory, so that the problem of unknown weight updating in the neural network is solved.
Drawings
The invention will now be described, by way of example, with reference to the accompanying drawings, in which:
FIG. 1 is a model of an object classification method based on an adaptive feedforward neural network according to the present invention.
FIG. 2 is a model of the adaptive update of weights of the adaptive feedforward neural network in the present invention.
FIG. 3 is a flow chart of the adaptive feedforward neural network setup according to the present invention.
Detailed Description
All of the features disclosed in this specification, or all of the steps in any method or process so disclosed, may be combined in any combination, except combinations of features and/or steps that are mutually exclusive.
Any feature disclosed in this specification (including any accompanying claims, abstract) may be replaced by alternative features serving equivalent or similar purposes, unless expressly stated otherwise. That is, unless expressly stated otherwise, each feature is only an example of a generic series of equivalent or similar features.
Referring to fig. 1, the present invention provides a target classification method based on an adaptive feedforward neural network, including the following steps:
step S100: after image data are collected, labeling the training data of deep learning, and carrying out normalization processing on the collected image data;
step S200: determining the batch size of matrixes to be input by a neural network according to image data to be input, designing a matrix transformation method for inputting the data as the neural network, reasonably selecting the node number and the activation function of an input layer and a hidden layer of the feedforward neural network, determining the node number and the activation function of an output layer of the neural network according to an image label to be input, and initializing the weight of the neural network;
the step S200 specifically includes the following steps:
step S210: and based on the image data after the normalization processing, performing matrix transformation on the image matrix, and converting the image matrix into a one-dimensional vector input by the neural network through one-dimensional projection or matrix flattening.
Step S220: based on the image label, the number of output nodes of the neural network is set, the weight matrix of each corresponding layer is designed, the output dimension of the network layer is equal to the input dimension of the next network layer, the non-zero matrix of each layer of weight is initialized, and the feedforward neural network is designed as follows:
in the formula
Is an activation function of the output layer,
is the input of the output layer, w t Is the weight matrix of the layer and,
respectively, activation function and input of hidden layer, g (x) is to perform matrix transformation, i.e. one-dimensional projection or matrix flattening, on image matrix, theta is to weight w of neural network 1 ,w 2 ,…,w t Flattening the vector;
step S300: reconstructing an adaptive feedforward neural network model as the following formula by introducing the input and output of the neural network and the gradient of each layer weight:
wherein f (x, theta) is formed by R t A feedforward neural network designed according to step S200, x ∈ R n×n Is a neural network input matrix, theta ∈ R r The vector is a vector after the weight matrix of the neural network is flattened;
feed-forward neural network weight estimation
In the first order taylor series expansion of (b),
is the first order gradient of the neural network with respect to the weight θ, v is the first orderTaylor expansion remainder;
step S400: constructing an auxiliary variable according to the reconstructed neural network, extracting and representing weight estimation error information, and designing a self-adaptive rate based on the weight estimation error;
the step S400 specifically includes the following steps:
step S410: according to the neural network reconstructed in the step S300, the following auxiliary variable P epsilon R is constructed r×r ,Q∈R r ,H∈R r ：
Wherein l >0 is a leakage term and κ >0 is a gain;
step S420: constructing a variable W epsilon R according to the auxiliary variable obtained in the step S410 r ,H∈R r ：
Variable in the formula
Is bounded, i.e.
The derivative of the variable W is as follows:
step S430: according to the auxiliary variables obtained in step S420, the adaptive rate of updating the weight of the neural network is designed as
In the formula of gamma>0 is the learning gain;
step S500: and under the condition of reaching the test precision, realizing the stable convergence of the weight according to the self-adaption rate updated by the weight, and classifying the subsequent target images.
The step S500 specifically includes the following steps:
step S510: the continuity is that the adaptive neural network loads the image data in batches, the image data loading can be stopped under the condition that the test precision is continuously reduced for 5 times, and the weight under the condition of the highest test precision is the optimal weight after training.
Step S520: based on the Lyapunov stability theory, the self-adaptive rate
Is driven by taking the Lyapunov function
According to the young inequality, the corresponding derivative can be expressed as follows:
where μ ═ 2(σ -1/m)/λ max (Γ -1 ) Is about m>Positive number 1/sigma, due to being bounded
The term ρ is also a positive number. Therefore, the updated image of the neural network weight is a smooth curve, and under the test stop condition in step S510, the curve is stably converged, and the weight estimation error is consistent and finally bounded, and meanwhile, the obtained neural network weight can classify the subsequent target images.
In a specific embodiment, the design concept of the adaptive feedforward neural network is as follows:
as shown in fig. 2, the feed-forward neural network continuously reads the batch image data and outputs the classification result of the target image. The weight value self-adaptive estimation method provided by the invention is driven by weight value estimation errors, namely auxiliary variables P and Q are constructed according to the target label, the image classification label output by the feedforward neural network and the gradient of the neural network; then, according to the constructed auxiliary variables, weight estimation error information variables are extracted, and the adaptive rate of weight updating is designed correspondingly.
As shown in fig. 3, constructing the feedforward neural network first requires constructing a data class, which stores image and tag data, defines the batch size, and includes a normalization function for data preprocessing. Then, a network class is designed, and the class stores the weight of each layer of neural network, the output of nodes and an activation function.
The weight updating method of the invention is different from the traditional weight gradient updating, auxiliary variables P and Q need to be constructed, and a complex neural network can cause complex weight parameters, so that parallel operation needs to be carried out on GPU equipment during matrix operation. In the process of building the neural network, data classes and network classes, specifically image data, label data, batch sizes, weights, nodes and gradients of weights of network layers, are applied to a CPU memory. Auxiliary variables P, Q, a weight estimation error variable W and a weight theta are also applied to the GPU memory. The self-adaptive updating of the weight of the neural network is realized by reading the image data and mutually accessing the CPU memory and the GPU memory.
The invention is not limited to the foregoing embodiments. The invention extends to any novel feature or any novel combination of features disclosed in this specification, and to any novel method or process steps or any novel combination of steps disclosed.
Claims (4)
1. A target classification method based on an adaptive feedforward neural network is characterized by comprising the following steps:
constructing a feedforward neural network model: reconstructing the nonlinear feedforward neural network system into a nonlinear system with linear weight value through a Taylor series expansion; the constructing of the feedforward neural network model specifically includes:
step S100: collecting image data, labeling the training data of deep learning, and carrying out normalization processing on the collected image data;
step S200: setting an input layer, a hidden layer and an output layer of a feedforward neural network according to image data to be input and an image label;
step S300: introducing the input and output of the feedforward neural network and the gradient of each layer of weight, and reconstructing an adaptive feedforward neural network model:
wherein the content of the first and second substances,
in order to construct a feed-forward neural network,
is the input matrix of the feed-forward neural network,
is a vector after the weight matrix of the feedforward neural network is flattened, and the feedforward neural network estimates the weight
In the first order taylor series expansion of (b),
is a feed-forward neural network on weights
The first-order gradient of the gradient (c),
is a first order taylor expansion remainder;
and (3) taking the weight estimation error as a drive to update the weight: extracting a weight estimation error information variable according to the constructed auxiliary variable, and correspondingly designing the self-adaptive rate of weight updating;
and (3) target classification: classifying the target images according to the feedforward neural network model updated by the weight;
the target classification method comprises the following steps: and under the condition of reaching the test precision, realizing the stable convergence of the weight according to the self-adaption rate updated by the weight, and classifying the subsequent target images.
2. The method for classifying the target based on the adaptive feedforward neural network as claimed in claim 1, wherein the step S200 specifically comprises the following steps:
step S210: based on the image data after normalization processing, performing matrix transformation on an image matrix, and converting the image matrix into a one-dimensional vector input by a neural network through one-dimensional projection or matrix flattening;
step S220: based on the image label, the number of output nodes of the neural network is set, the weight matrix of each corresponding layer is designed, the output dimension of the network layer is equal to the input dimension of the next network layer, and the nonzero matrix of each layer of weight is initialized.
3. The target classification method based on the adaptive feedforward neural network as claimed in claim 1, wherein the method for updating the weight using the weight estimation error as a driver specifically comprises:
step S410: constructing the following auxiliary variables according to the neural network reconstructed in the step S300
：
Wherein the content of the first and second substances,
in order to be able to make a leak,
is the gain;
step S420: constructing variables according to the auxiliary variables obtained in step S410
：
Wherein W is a weight estimation error variable
Is bounded, i.e.
Is the weight estimation error;
step S430: calculating the self-adaptive rate of the neural network weight update according to the variables W and H obtained in the step S420
Wherein
In order to learn the gain, the learning unit,
is a gain matrix.
4. The method for classifying an object based on an adaptive feedforward neural network as claimed in claim 3, wherein the method for classifying an object specifically comprises the steps of:
step S510: continuously loading batch image data for the adaptive neural network, stopping image data loading under the condition that the test precision is continuously reduced for 5 times, wherein the weight under the highest test precision condition is the optimal weight after training;
step S520: based on the Lyapunov stability theory, the self-adaptive rate
Under the driving of the neural network, the weight of the neural network is updated, stable convergence is achieved under the test stopping condition in the step S510, the weight estimation error is consistent and finally bounded, the weight of the neural network is obtained, and the subsequent target images are classified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111263644.6A CN113988181B (en) | 2021-10-26 | 2021-10-26 | Target classification method based on adaptive feedforward neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111263644.6A CN113988181B (en) | 2021-10-26 | 2021-10-26 | Target classification method based on adaptive feedforward neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113988181A CN113988181A (en) | 2022-01-28 |
| CN113988181B true CN113988181B (en) | 2022-09-20 |
Family
ID=79743451
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111263644.6A Active CN113988181B (en) | 2021-10-26 | 2021-10-26 | Target classification method based on adaptive feedforward neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113988181B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20180037558A (en) * | 2016-10-04 | 2018-04-12 | 한국전자통신연구원 | Convolutional neural network system using adaptive pruning and weight sharing and operation method thererof |
| CN110009528A (en) * | 2019-04-12 | 2019-07-12 | 杭州电子科技大学 | A kind of parameter adaptive update method based on optimum structure multidimensional Taylor net |
| CN113093771A (en) * | 2021-04-07 | 2021-07-09 | 安徽大学 | Neural network-based underwater robot-manipulator system modeling method and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915445B (en) * | 2012-09-17 | 2015-08-19 | 杭州电子科技大学 | A kind of follow-on neural network Hyperspectral Remote Sensing Imagery Classification method |
| CN104463846B (en) * | 2014-11-04 | 2017-05-17 | 浙江捷尚视觉科技股份有限公司 | Parameter adjustment method used for digital image processing |
| US10762427B2 (en) * | 2017-03-01 | 2020-09-01 | Synaptics Incorporated | Connectionist temporal classification using segmented labeled sequence data |
| CN107301456B (en) * | 2017-05-26 | 2020-05-12 | 中国人民解放军国防科学技术大学 | Deep neural network multi-core acceleration implementation method based on vector processor |
-
2021
- 2021-10-26 CN CN202111263644.6A patent/CN113988181B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20180037558A (en) * | 2016-10-04 | 2018-04-12 | 한국전자통신연구원 | Convolutional neural network system using adaptive pruning and weight sharing and operation method thererof |
| CN110009528A (en) * | 2019-04-12 | 2019-07-12 | 杭州电子科技大学 | A kind of parameter adaptive update method based on optimum structure multidimensional Taylor net |
| CN113093771A (en) * | 2021-04-07 | 2021-07-09 | 安徽大学 | Neural network-based underwater robot-manipulator system modeling method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113988181A (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110020682B (en) | Attention mechanism relation comparison network model method based on small sample learning | |
| CN111241952B (en) | Reinforced learning reward self-learning method in discrete manufacturing scene | |
| CN110263227B (en) | Group partner discovery method and system based on graph neural network | |
| CN110288030B (en) | Image identification method, device and equipment based on lightweight network model | |
| WO2020019236A1 (en) | Loss-error-aware quantization of a low-bit neural network | |
| CN103838836B (en) | Based on discriminant multi-modal degree of depth confidence net multi-modal data fusion method and system | |
| WO2018016608A1 (en) | Neural network apparatus, vehicle control system, decomposition device, and program | |
| CN112987664B (en) | Flow shop scheduling method based on deep reinforcement learning | |
| CN112115998B (en) | Method for overcoming catastrophic forgetting based on anti-incremental clustering dynamic routing network | |
| CN114386694A (en) | Drug molecule property prediction method, device and equipment based on comparative learning | |
| CN116503676B (en) | Picture classification method and system based on knowledge distillation small sample increment learning | |
| CN111723914A (en) | Neural network architecture searching method based on convolution kernel prediction | |
| CN112685504A (en) | Production process-oriented distributed migration chart learning method | |
| EP4341862A1 (en) | Low-rank adaptation of neural network models | |
| CN111695011B (en) | Tensor expression-based dynamic hypergraph structure learning classification method and system | |
| CN113112013A (en) | Optimized quantization for reduced resolution neural networks | |
| CN110473195B (en) | Medical focus detection framework and method capable of being customized automatically | |
| Chang et al. | Differentiable architecture search with ensemble gumbel-softmax | |
| CN114463596A (en) | Small sample image identification method, device and equipment of hypergraph neural network | |
| Gabrijel et al. | On-line identification and reconstruction of finite automata with generalized recurrent neural networks | |
| CN113988181B (en) | Target classification method based on adaptive feedforward neural network | |
| CN111753995A (en) | Local interpretable method based on gradient lifting tree | |
| CN115661546A (en) | Multi-objective optimization classification method based on feature selection and classifier joint design | |
| CN115063374A (en) | Model training method, face image quality scoring method, electronic device and storage medium | |
| WO2022162839A1 (en) | Learning device, learning method, and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |