CN115359337A - Searching method, system and application of pulse neural network for image recognition - Google Patents

Searching method, system and application of pulse neural network for image recognition Download PDF

Info

Publication number
CN115359337A
CN115359337A CN202211011023.3A CN202211011023A CN115359337A CN 115359337 A CN115359337 A CN 115359337A CN 202211011023 A CN202211011023 A CN 202211011023A CN 115359337 A CN115359337 A CN 115359337A
Authority
CN
China
Prior art keywords
snn
population
module
image recognition
male
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211011023.3A
Other languages
Chinese (zh)
Other versions
CN115359337B (en
Inventor
孙亚楠
宋孝天
冯诚喆
吕泽琼
刘芸
李思毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan University
Original Assignee
Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan University filed Critical Sichuan University
Priority to CN202211011023.3A priority Critical patent/CN115359337B/en
Publication of CN115359337A publication Critical patent/CN115359337A/en
Application granted granted Critical
Publication of CN115359337B publication Critical patent/CN115359337B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/086—Learning methods using evolutionary algorithms, e.g. genetic algorithms or genetic programming
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Physiology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a searching method, a searching system and application of a pulse neural network for image recognition, wherein the searching method comprises the steps of S1 initializing a plurality of SNN frameworks as a population; s2, training SNN frameworks in the population by adopting an image recognition data set, and calculating the fitness value of each SNN framework; s3, selecting an SNN framework to form an SNN male parent set by adopting a binary tournament algorithm according to the fitness value of the SNN framework; s4, carrying out cross mutation operation on the SNN male parents of the SNN male parent set, and then carrying out environment selection on all filial generation SNNs after the mutation operation; s5, judging whether the evolution algebra of the environment-selected offspring SNN meets a preset threshold, if so, entering a step S6, otherwise, updating the population by adopting the environment-selected offspring SNN, and returning to the step S2; and S6, adopting the individuals with the highest fitness in the population after environment selection as the final SNN network.

Description

Searching method, system and application of pulse neural network for image recognition
Technical Field
The invention relates to an image recognition technology, in particular to a searching method, a searching system and an application of a pulse neural network for image recognition.
Background
The image identification is to distinguish different types of images according to semantic information of the images, is an important basic problem in computer vision, and is the basis of other high-level vision tasks such as object detection, image segmentation, object tracking, behavior analysis, face recognition and the like. Image recognition is widely applied in many fields, and part of field data processing has high requirements on safety, needs to ensure timeliness, and needs to perform calculation on a small embedded device.
At present, a traditional Artificial Neural Network (ANN) model represented by a convolutional neural network has been successfully applied to the deep learning field (e.g., computer vision, natural language processing), is often applied to small embedded devices (e.g., mobile phones, watches, cameras, etc.) for image recognition, and has high recognition accuracy in recognition.
However, because a large amount of high-power consumption floating-point number multiplication operations exist in the conventional ANN network, the ANN network has huge energy consumption, and when the embedded device performs image recognition, the embedded device has a problem of huge power consumption, and the embedded device is severely restricted in terms of time duration of use under single charging and high power consumption, and simultaneously generates a large amount of heat, which also has a certain influence on the battery life of the embedded device.
Disclosure of Invention
Aiming at the defects in the prior art, the searching method, the searching system and the searching application of the impulse neural network for image recognition provided by the invention solve the problem that the image recognition network constructed by the existing method has high energy consumption when in use.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
in a first aspect, a searching method of a spiking neural network for image recognition is provided, which includes:
s1, initializing a plurality of SNN frameworks as a population;
s2, training SNN frameworks in the population by adopting an image recognition data set, and calculating the fitness value of each SNN framework:
f(x)＝w 1 ×ACC+w 2 ×SPIKES
wherein f (x) is a fitness value of the SNN framework; ACC is the accuracy of the SNN architecture; spike is the total number of pulses in the SNN architecture; w is a 1 And w 2 Weights corresponding to ACC and SPIKES respectively;
s3, selecting an SNN framework to form an SNN male parent set by adopting a binary tournament algorithm according to the fitness value of the SNN framework;
s4, carrying out cross mutation operation on the SNN male parents of the SNN male parent set, and then carrying out environment selection on all filial generation SNNs after the mutation operation;
s5, judging whether the evolution algebra of the environment-selected offspring SNN meets a preset threshold, if so, entering a step S6, otherwise, updating the population by adopting the environment-selected offspring SNN, and returning to the step S2;
and S6, adopting the individuals with the highest fitness in the population after environment selection as the final SNN network.
Further, the cross mutation operation of the SNN male parents of the SNN male parent set further comprises the following steps:
a1, adopting a twice binary championship algorithm, randomly selecting two non-overlapping SNN male parents from an SNN male parent set each time, and pairing the SNN male parents with optimal fitness values selected twice;
a2, continuously executing the step A1 on unpaired SNN male parents in the SNN male parents until the SNN male parents in the SNN male parent set are all matched, and then entering the step A3;
a3, generating a random number r for each pair of SNN male parents, and judging whether the random number r is larger than a preset cross operation probability, if so, entering the step A5, otherwise, entering the step A4;
a4, randomly dividing each SNN male parent of the SNN male parents into two parts, performing cross operation on the two parts of the two SNN male parents to generate filial generation SNN, and then entering the step A5;
a5, generating a random number y for each pair of SNN male parent/filial generation SNN, and judging whether the random number y is smaller than a preset mutation probability, if so, entering the step A6, otherwise, entering an environment selection operation;
and A6, selecting each SNN male parent/filial generation SNN with a higher fitness value, randomly selecting a mutation position, selecting a mutation operation according to a preset probability to be executed at the mutation position, and then entering an environment selection operation.
Further, the method for performing environment selection on all the offspring SNNs after mutation operation comprises the following steps:
b1, combining the SNN male parent set in the step S3 with all filial generation SNNs after cross variation to form a next generation population, and arranging in a descending order according to the fitness value;
b2, selecting individuals with optimal fitness values to the next generation population according to the percentage ratio of elite in the population trained in the step S2 by utilizing an elite strategy;
b3, selecting individuals from the rest individuals of the population by using a binary tournament selection algorithm until the number of the individuals in the next generation population is the same as that of the population in the step S2;
and B4, forming the offspring SNN after environment selection by adopting the next generation population.
Further, the mutation operation comprises at least four types, namely adding a layer of network with random parameters at the selected position, deleting a layer of network at the selected position, randomly modifying the parameter value of the node at the selected position and adding reverse connection between the nodes at the selected position; the probabilities of the four mutation operations are set to 0.7,0.1,0.1,0.1, respectively.
Further, the step S1 further includes automatically generating a variable-length neural network architecture by using a variable-length coding method in evolution calculation for the multiple LIF base nodes, and accumulating data information over multiple time steps to finally generate the SNN architecture.
Further, the image recognition comprises object detection, image segmentation, object tracking, behavior analysis and face recognition;
when the image recognition task is vehicle violation recognition, the image recognition data set is all vehicle photos collected by a traffic department; when the image recognition task is a photo of a person when the smart phone is unlocked, the image recognition data set is a photo of the person with all user tags; when the image recognition task is a CT scanning picture shot by the intelligent medical equipment, the image recognition training set is the CT scanning picture with the disease condition label at the same part.
In a second aspect, a system for constructing a spiking neural network for image recognition is provided, which includes:
the initialization module is used for initializing a plurality of SNN frameworks as a population;
the training module is used for training the SNN frameworks in the population by adopting an image recognition data set and calculating the fitness value of each SNN framework;
the male parent set construction module is used for selecting the SNN framework to construct the SNN male parent set by adopting a binary tournament algorithm according to the fitness value of the SNN framework;
the generation module of filial generation population, is used for carrying on the cross mutation operation to SNN male parent of SNN male parent set, carry on the environment selection to all filial generation SNN of the mutation operation afterwards;
the judgment module is used for judging whether the evolution algebra of the filial generation SNN after the environment selection meets a preset threshold value, if so, the model generation module updates the population by adopting the filial generation SNN after the environment selection, and otherwise, the population is returned to the training module;
and the model generation module is used for constructing and forming a final SNN network by adopting the offspring SNN after environment selection.
Further, the offspring population generation module comprises a variation crossover module and an environment selection module, wherein the variation crossover module comprises:
the first pairing module is used for randomly selecting two non-overlapping SNN male parents from the SNN male parent set each time by adopting a binary championship algorithm twice and pairing the SNN male parents with optimal fitness values selected twice;
the second pairing module is used for continuing executing the first pairing module on the unpaired SNN male parents in the SNN male parents until all the SNN male parents in the SNN male parent set are paired, and then entering the random number judgment module;
the random number judgment module is used for generating a random number r for each pair of SNN male parents and judging whether the random number r is greater than the preset cross operation probability, if so, the random number r enters the mutation execution module, and otherwise, the random number r enters the cross execution module;
the cross execution module is used for randomly dividing each SNN male parent of the current SNN male parents into two parts, performing cross operation on the two parts of the two SNN male parents to generate filial generation SNN, and then performing mutation;
a mutation operation judging module for generating a random number y for each pair of SNN male parent/son SNN, and judging whether the random number y is smaller than a preset mutation probability, if so, performing mutation, otherwise, entering an environment selecting module;
and the mutation execution module selects the SNN male parent/son SNN with a higher fitness value in each pair, randomly selects a mutation position, selects a mutation operation according to a preset probability to execute at the mutation position, and then enters the environment selection module.
In a third aspect, an application method of a spiking neural network is provided, where the spiking neural network is searched by using a searching method of the spiking neural network for image recognition, and when performing image recognition, the method includes:
acquiring an identification image of an object to be identified, and preprocessing the identification image;
and inputting the preprocessed identification image into a pulse neural network for identification to obtain an identification result.
Compared with the prior art, the invention has the beneficial effects that:
(1) According to the scheme, when the self-adaptive value is calculated, the total number of pulses is introduced to evaluate the SNN performance, so that the total number of pulses and the accuracy rate are jointly used as performance evaluation indexes of the SNN framework, and the searched SNN network can take energy consumption and accuracy rate into account when being applied.
(2) The scheme applies a natural selection process to search optimization by using a Genetic Algorithm (GA), so that the SNN framework design is free from dependence of professional knowledge and manual operation, an optimal SNN framework is continuously searched through a series of evolution operators, a high-performance SNN framework is automatically designed, and manual intervention is reduced to the maximum extent.
(3) When the pulse neural network is searched, a specific selected image recognition data set can be selected for network training according to a recognition object finally applied to the searched SNN network, and the searched SNN network can be directly applied to embedded equipment for image recognition without retraining.
Drawings
Fig. 1 is a flowchart of a searching method of a pulse neural network for image recognition according to the present invention.
Fig. 2 is a schematic diagram of the LIF base node.
Fig. 3 is a schematic diagram of an architecture for automatically generating SNNs.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
Referring to fig. 1, fig. 1 shows a flowchart of a searching method of the present solution for an impulse neural network for image recognition; as shown in fig. 1, the method S includes steps S1 to S6.
In step S1, initializing a plurality of SNN frameworks as a population; there are N individuals in the population, each individual (chromosome) representing an SNN framework; during initialization, each SNN framework is generated by adopting a variable length coding mode:
a plurality of LIF base nodes (a schematic diagram of which may refer to fig. 2) automatically generate a variable-length neural network architecture by using a variable-length coding method in evolutionary computation, and accumulate data information in a plurality of time steps to finally generate an SNN architecture, where a network architecture of the SNN architecture may refer to fig. 3.
Each SNN architecture simulates biological neurons using integrated release LIF models with leakage; the LIF neuron is represented by the following formula, and the current node voltage value is calculated by the formula:
Figure BDA0003810865360000071
in the formula u i 、u i-1 The voltage values of the nodes of the current layer and the previous layer after LIF activation; r is m J is the leakage constant, w is the node number of the current layer m To the connection weight, o is the current node pulse value. When u is i When the threshold voltage is exceeded, the current node is activated, and the pulse signal is 1, otherwise, the current node fails to be activated, and the pulse signal is 0.
In step S2, the SNN frameworks in the population are trained using the image recognition dataset, and a fitness value of each SNN framework is calculated:
f(x)＝w 1 ×ACC+w 2 ×SPIKES
wherein f (x) is the fitness value of the SNN framework; ACC is the accuracy of the SNN architecture; spike is the total number of pulses of the SNN architecture; w is a 1 And w 2 Are weights corresponding to ACC and SPIKES, respectively.
According to the scheme, the total number of pulses is introduced to evaluate the SNN performance when the fitness value is calculated, so that the total number of pulses and the accuracy rate are jointly used as performance evaluation indexes of the SNN framework, and the searched SNN network can give consideration to both energy consumption and accuracy rate when being applied.
The image recognition of the scheme comprises object detection, image segmentation, object tracking, behavior analysis and face recognition. When the image recognition task is vehicle violation recognition, the image recognition data set is all vehicle photos collected by a traffic department; during identification, the traffic pictures acquired by the intelligent traffic camera are acquired, and the searched SNN is adopted to carry out image identification on the shot illegal vehicle pictures and the vehicle information in the database, so that the vehicle information can be accurately identified.
When the image recognition task is a photo of a person when the smart phone is unlocked, the image recognition data set is a photo of the person with all user tags; during identification, one or a series of face images containing undetermined identities are input into the SNN for searching, the SNN is compared with the face images of the owner users in the face database, the input face images are identified, and the identities of the faces to be identified are accurate.
When the image recognition task is a CT scanning picture shot by intelligent medical equipment, the image recognition training set is a CT scanning picture with a disease condition label at the same part; during identification, the searched SNN can identify the CT images and match with corresponding disease types to help doctors to preliminarily judge the disease conditions.
The image identification data set can also adopt a CIFAR-10 data set, wherein the CIFAR-10 data set comprises 60000 pictures and 10 classes (including airplanes, automobiles, cats, dogs and the like), and 6000 pictures in each class, wherein 50000 pictures are used as a training set, and 10000 pictures are used as a testing set.
In step S3, according to the fitness value of the SNN framework, selecting the SNN framework by adopting a binary tournament algorithm to form an SNN male parent set;
in step S4, performing cross mutation operation on the SNN male parents of the SNN male parent set, and then performing environment selection on all the progeny SNNs after the mutation operation;
in one embodiment of the present invention, the cross mutation operation of the SNN male parents of the SNN male parent set further comprises:
a1, adopting a twice binary championship algorithm, randomly selecting two non-overlapping SNN male parents from an SNN male parent set each time, and pairing the SNN male parents with optimal fitness values selected twice;
the specific implementation process of the step A1 is as follows:
by using the binary tournament algorithm twice, each time randomly selecting two individuals from the population (two individuals cannot be repeated), selecting the individual with the greater fitness of the two individuals as the male parent using the following formula:
max(w r11 ×ACC r1 +w r21 ×SPIKES r1 ，w r12 ×ACC r2 +w r22 ×SPIKES r2 )
in the formula, w r11 And w r21 For randomly selecting the weight corresponding to the first individual, ACC r1 And spike r1 The corresponding accuracy and total number of pulses for that individual; w is a r12 And w r22 For randomly selecting the weight corresponding to the second individual, ACC r2 And SPIKES r2 The corresponding accuracy and total number of pulses for that individual;
and (3) marking the individuals selected by the two-time binary championship algorithm as a male parent 1 and a male parent 2, and forming a pair by the male parent 1 and the male parent 2 to perform subsequent cross variation.
A2, continuously executing the step A1 on unpaired SNN male parents in the SNN male parents until the SNN male parents in the SNN male parent set are all matched, and then entering the step A3;
a3, generating a random number r for each pair of SNN male parents, and judging whether the random number r is larger than a preset cross operation probability, if so, entering the step A5, otherwise, entering the step A4;
a4, randomly dividing each SNN male parent of the current SNN male parents into two parts, performing cross operation on the two parts of the two SNN male parents to generate filial generation SNN, and then entering the step A5;
a5, generating a random number y for each pair of SNN male parent/sub-parent SNNs, and judging whether the random number y is smaller than a preset variation probability, if so, entering the step A6, otherwise, entering environment selection operation;
and A6, selecting each pair of SNN male parent/sub-parent SNNs with larger fitness value, randomly selecting variant positions, selecting variant operation to be executed at the variant positions according to preset probability, and then entering environment selection operation.
In step S4, the method for performing environment selection on all the progeny SNNs after mutation operation includes:
b1, combining the SNN male parent set in the step S3 with all filial generation SNNs after cross variation to form a next generation population, and arranging in a descending order according to the fitness value;
b2, selecting individuals with optimal fitness values to the next generation population according to the percentage ratio of elite in the population trained in the step S2 by utilizing an elite strategy;
b3, selecting individuals from the rest individuals of the population by using a binary tournament selection algorithm until the number of the individuals in the next generation population is the same as that of the population in the step S2;
and B4, forming the filial generation SNN after environment selection by adopting the next generation population.
According to the scheme, the individuals with the best performance are directly added into the next generation of population, so that the population elegance can be kept; individuals are selected by using a binary tournament algorithm, so that slightly inferior individuals are selected while the optimal individuals are selected, and the population diversity can be effectively maintained.
In the step S5, judging whether the evolution algebra of the filial generation SNN after the environment selection meets a preset threshold value, if so, entering the step S6, otherwise, adopting the filial generation SNN after the environment selection to update the population, and returning to the step S2;
in step S6, the individual with the highest fitness in the population after environment selection is used as the final SNN network.
Compared with the traditional ANN, the impulse neural network SNN constructed by the scheme can make up the difference between neuroscience and machine learning, a model which is most fit with a biological neuron mechanism is used for calculation, input information is encoded into binary pulses, discrete binary pulse addition operation is used for replacing traditional ANN floating point number multiplication operation, and the purpose of saving energy consumption is achieved by utilizing the processing capacity of the biological neuron mechanism on discrete data. Taking 32-bit floating point as an example, floating point Multiplication and Addition (MAC) operations in the traditional ANN consume 4.6pJ once, and addition operations in the SNN consume only 0.9pJ once, so that the energy is saved by about 5.1 times.
The mutation operation of the scheme comprises at least four types, namely adding a layer of network with random parameters at a selected position, deleting a layer of network at the selected position, randomly modifying the parameter value of a node at the selected position and adding reverse connection between nodes at the selected position; the probabilities of the four mutation operations are set to 0.7,0.1,0.1,0.1, where the probabilities are also preset probabilities in step A6.
The network with random parameters is added at the selected position to have the highest probability, and the operation can effectively increase the network depth, so that the model prediction accuracy is improved. The depth of the network can be reduced by deleting one layer of network at the selected position, and the network which is too deep can possibly return to a reasonable layer number range by reducing the layer number so as to ensure the accuracy in subsequent identification.
The scheme also provides a construction system of the impulse neural network for image recognition, which comprises the following steps:
the initialization module is used for initializing a plurality of SNN frameworks as a population;
the training module is used for training the SNN frameworks in the population by adopting an image recognition data set and calculating the fitness value of each SNN framework;
the male parent set construction module is used for selecting the SNN framework to construct the SNN male parent set by adopting a binary tournament algorithm according to the fitness value of the SNN framework;
the generation module of filial generation population, is used for carrying on the cross mutation operation to SNN male parent of SNN male parent set, carry on the environment selection to all filial generation SNN of the mutation operation afterwards;
the judging module is used for judging whether the evolution algebra of the offspring SNN after the environment selection meets a preset threshold value, if so, the model generating module is used, otherwise, the offspring SNN after the environment selection is used for updating the population and returning to the training module;
and the model generation module is used for constructing and forming a final SNN network by adopting the offspring SNN after environment selection.
In one embodiment of the invention, the offspring population generation module comprises a variant crossing module and an environment selection module, the variant crossing module comprising:
the first pairing module is used for randomly selecting two non-overlapping SNN male parents from the SNN male parent set each time by adopting a binary championship algorithm twice and pairing the SNN male parents with optimal fitness values selected twice;
the second pairing module is used for continuing the execution of the first pairing module on the unpaired SNN male parents in the SNN male parents until the SNN male parents in the SNN male parent set are paired, and then entering the random number judgment module;
the random number judgment module is used for generating a random number r for each pair of SNN male parents and judging whether the random number r is greater than the preset cross operation probability, if so, entering the mutation execution module, and otherwise, entering the cross execution module;
the cross execution module is used for randomly dividing each SNN male parent of the current SNN male parents into two parts, performing cross operation on the two parts of the two SNN male parents to generate filial generation SNN, and then performing mutation;
the mutation operation judging module is used for generating a random number y for each pair of SNN male parent/child SNN and judging whether the random number y is smaller than a preset mutation probability, if so, the mutation execution module is executed, and if not, the environment selection module is started;
and the mutation execution module selects the SNN male parent/son SNN with a higher fitness value in each pair, randomly selects a mutation position, selects a mutation operation according to a preset probability to execute at the mutation position, and then enters the environment selection module.
Finally, the present invention also provides an application method of a spiking neural network, wherein the spiking neural network is obtained by searching with a spiking neural network search method for image recognition, and the method comprises the following steps of:
acquiring an identification image of an object to be identified, and preprocessing the identification image; and inputting the preprocessed identification image into a pulse neural network for identification to obtain an identification result.
The following specific examples are combined to evaluate the performance of the impulse neural network searched by the scheme:
the CIFAR-10 data set is widely used in the field of image recognition to carry out model training and evaluation, and the CIFAR-10 data set comprises 60000 pictures, is divided into 10 classes (including airplanes, automobiles, cats, dogs and the like), and comprises 6000 pictures in each class, wherein 50000 pictures are used as a training set, and 10000 pictures are used as a testing set.
When the impulse neural network is constructed, all parameter designs of the evolution stage follow the convention of the genetic algorithm, and particularly, the number of virtual generations and the number of evolution generations in a population are set to be 20. The probabilities of crossover and mutation were designated 0.9 and 0.2, respectively. The probabilities of adding, deleting and modifying layers in the mutation operation are 0.7,0.15 and 0.15, respectively. The range of the number of layers of the impulse neural network is 5 to 15 layers for generating the number of layers randomly at the beginning.
When the optimal SNN architecture is selected after the evolution process, in order to train a model by using Stochastic Gradient Descent (SGD), a discontinuous and indistinguishable pulse value is simulated by using a substitute gradient, and a triangle-like function in the following formula is used for simulating the SNN gradient to train the model:
Figure BDA0003810865360000131
wherein the content of the first and second substances,
Figure BDA0003810865360000132
representing the non-differentiable functional gradient in the SNN, and gamma represents an activation factor for determining the range of the activation gradient; u (t) is the present node voltage, V th Is a set threshold voltage.
In the training parameter setting, the initial learning rate (learning rate) is set to 0.001 and the pulse time step (spike time) is set to 2. On the CIFAR-10 dataset, the blocksize is set to 64 and 600 iterations of training are performed on the evolved best model to obtain the final result.
The result of comparing SNN architecture (SNN-GA) performance generated automatically based on evolution with the existing manually constructed networks (CIFARNet-Wu and CIFARNet-Fang), ANN-SNN conversion networks (ResNet 11-Lee and ResNet 19-Zheng) and NAS searching method (AutoSNN) is shown in table 1.
Table 1 shows SNN framework performance comparison data constructed by SNN framework of the scheme and other means
Figure BDA0003810865360000133
Figure BDA0003810865360000141
The results in table 1 show that, compared with the manually well-designed algorithm, for three indexes, the SNN architecture (SNN-GA) automatically generated by the method has obvious improvement in accuracy and total number of pulses, the accuracy is improved by 2% compared with the manually designed SNN network, the total number of pulses is reduced by 3.5 times compared with CIFARNet-Fang having the least number of pulses, and is improved by nearly 20 times compared with ResNet11-Lee having the most number of pulses, and the reduction of the total number of pulses represents that the energy consumption of the network generated by the method can be saved by a corresponding proportion under the same condition when the network is applied.
Compared with the current latest SNN architecture algorithm (Auto SNN) automatically designed by the NAS method, the SNN network searched by the method is kept accurate, the model parameters are slightly reduced, particularly for the total number of pulses index, the method reduces the total number of pulses by 3.5 times, and the power consumption of the SNN is obviously reduced.
Advantages in the actual scene:
when the SNN network obtained by searching is applied to a specific scene, particularly in the fields of image classification and picture identification in an embedded system (a mobile phone, industrial control software and the like), the embedded system is limited by the performance of a battery, so that the system has higher requirements on power consumption.
Specifically, for example, photo album pictures are classified in a mobile phone system, and things such as people, animals, flowers and the like are required to be identified, and the SNN network model constructed by the method is deployed in the mobile phone system, so that power consumption can be significantly reduced in scenes such as photo album picture classification.
Compared with the traditional manually designed SNN framework, the method can help the personnel lacking relevant knowledge of the SNN framework design, and the SNN framework meeting the actual use is designed in a full-automatic mode, and has the characteristics of low power consumption and high precision. For example, in the aspect of wire foreign matter identification, a wire foreign matter picture is used as an image set when the SNN is constructed by the method, so that a power system engineer is helped to quickly and automatically evolve a high-performance SNN framework and deploy the framework in an embedded system in a power system.
Because the SNN searched by the method has both accuracy and power consumption, when the SNN is applied to an actual scene, the SNN plays a significant role in reducing the power consumption of a system in the actual application scene.

Claims (9)

1. The searching method of the impulse neural network for image recognition is characterized by comprising the following steps:
s1, initializing a plurality of SNN frameworks as a population;
s2, training SNN frameworks in the population by adopting an image recognition data set, and calculating the fitness value of each SNN framework:
f(x)＝w 1 ×ACC+w 2 ×SPIKES
wherein f (x) is the fitness value of the SNN framework; ACC is the accuracy of the SNN architecture; spike is the total number of pulses in the SNN architecture; w is a 1 And w 2 Weights corresponding to ACC and SPIKES respectively;
s3, selecting an SNN framework to form an SNN male parent set by adopting a binary tournament algorithm according to the fitness value of the SNN framework;
s4, carrying out cross mutation operation on the SNN male parents of the SNN male parent set, and then carrying out environment selection on all filial generation SNNs after the mutation operation;
s5, judging whether the evolution algebra of the environment-selected offspring SNN meets a preset threshold, if so, entering a step S6, otherwise, updating the population by adopting the environment-selected offspring SNN, and returning to the step S2;
and S6, adopting the individuals with the highest fitness in the population after environment selection as the final SNN network.
2. The method of searching for the spiking neural network for image recognition according to claim 1, wherein the cross mutation operation of the SNN male parents of the SNN male parent set further comprises:
a1, adopting a twice binary championship algorithm, randomly selecting two non-overlapping SNN male parents from an SNN male parent set each time, and pairing the SNN male parents with optimal fitness values selected twice;
a2, continuously executing the step A1 on unpaired SNN male parents in the SNN male parents until the SNN male parents in the SNN male parent set are all matched, and then entering the step A3;
a3, generating a random number r for each pair of SNN male parents, and judging whether the random number r is larger than a preset cross operation probability, if so, entering the step A5, otherwise, entering the step A4;
a4, randomly dividing each SNN male parent of the SNN male parents into two parts, performing cross operation on the two parts of the two SNN male parents to generate filial generation SNN, and then entering the step A5;
a5, generating a random number y for each pair of SNN male parent/sub-parent SNNs, and judging whether the random number y is smaller than a preset variation probability, if so, entering the step A6, otherwise, entering environment selection operation;
and A6, selecting each SNN male parent/filial generation SNN with a higher fitness value, randomly selecting a mutation position, selecting a mutation operation according to a preset probability to be executed at the mutation position, and then entering an environment selection operation.
3. The method for searching the spiking neural network for image recognition according to claim 2, wherein the method for selecting the environment of all the offspring SNNs after mutation operation comprises:
b1, combining the SNN male parent set in the step S3 with all filial generation SNNs after cross variation to form a next generation population, and arranging in a descending order according to the fitness value;
b2, selecting individuals with optimal fitness values to the next generation population according to the percentage ratio of elite in the population trained in the step S2 by utilizing an elite strategy;
b3, selecting individuals from the rest individuals of the population by using a binary tournament selection algorithm until the number of the individuals in the next generation population is the same as that of the population in the step S2;
and B4, forming the offspring SNN after environment selection by adopting the next generation population.
4. The method of claim 3, wherein the mutation operation comprises at least four of adding a layer of random networks at the selected location, deleting a layer of networks at the selected location, randomly modifying node parameter values at the selected location, and adding reverse connections between nodes at the selected location; the probabilities of the four mutation operations are set to 0.7,0.1,0.1,0.1, respectively.
5. The method for searching the spiking neural network for image recognition according to claim 1, wherein the step S1 further comprises: and (3) automatically generating a neural network framework with variable length by adopting a variable length coding method in evolution calculation for a plurality of LIF basic nodes, and accumulating data information on a plurality of time steps to finally generate the SNN framework.
6. The method for searching the impulse neural network for image recognition according to any one of claims 1 to 5, wherein the image recognition includes object detection, image segmentation, object tracking, behavior analysis and face recognition;
when the image recognition task is vehicle violation recognition, the image recognition data set is all vehicle photos collected by a traffic department; when the image recognition task is a human face photo when the smart phone is unlocked, the image recognition data set is a human face photo with all user tags; when the image recognition task is a CT scanning picture shot by the intelligent medical equipment, the image recognition training set is the CT scanning picture with the disease condition label at the same part.
7. A system for constructing a spiking neural network for image recognition, comprising:
the initialization module is used for initializing a plurality of SNN frameworks as a population;
the training module is used for training the SNN frameworks in the population by adopting an image recognition data set and calculating the fitness value of each SNN framework;
the male parent set construction module is used for selecting the SNN framework to construct the SNN male parent set by adopting a binary tournament algorithm according to the fitness value of the SNN framework;
the generation module of filial generation population, is used for carrying on the cross mutation operation to SNN male parent of SNN male parent set, carry on the environment selection to all filial generation SNN of the mutation operation afterwards;
the judgment module is used for judging whether the evolution algebra of the filial generation SNN after the environment selection meets a preset threshold value, if so, the model generation module updates the population by adopting the filial generation SNN after the environment selection, and otherwise, the population is returned to the training module;
and the model generation module is used for constructing and forming a final SNN network by adopting the offspring SNN after environment selection.
8. The construction system of claim 7, wherein the offspring population generation module comprises a variant crossing module and a context selection module, the variant crossing module comprising:
the first pairing module is used for randomly selecting two non-overlapping SNN male parents from the SNN male parent set each time by adopting a binary tournament algorithm twice and pairing the SNN male parents with optimal fitness values selected twice;
the second pairing module is used for continuing executing the first pairing module on the unpaired SNN male parents in the SNN male parents until all the SNN male parents in the SNN male parent set are paired, and then entering the random number judgment module;
the random number judgment module is used for generating a random number r for each pair of SNN male parents and judging whether the random number r is greater than the preset cross operation probability, if so, entering the mutation execution module, and otherwise, entering the cross execution module;
the cross execution module is used for randomly dividing each SNN male parent of the current SNN male parents into two parts, performing cross operation on the two parts of the two SNN male parents to generate filial generation SNN, and then performing mutation;
the mutation operation judging module is used for generating a random number y for each pair of SNN male parent/child SNN and judging whether the random number y is smaller than a preset mutation probability, if so, the mutation execution module is executed, and if not, the environment selection module is started;
and the mutation execution module selects the SNN male parent/son SNN with a higher fitness value in each pair, randomly selects a mutation position, selects a mutation operation according to a preset probability to execute at the mutation position, and then enters the environment selection module.
9. An application method of a spiking neural network, which is obtained by searching according to the searching method of the spiking neural network for image recognition in any one of claims 1 to 6, comprising:
acquiring an identification image of an object to be identified, and preprocessing the identification image; and inputting the preprocessed identification image into a pulse neural network for identification to obtain an identification result.
CN202211011023.3A 2022-08-23 2022-08-23 Searching method, system and application of pulse neural network for image recognition Active CN115359337B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211011023.3A CN115359337B (en) 2022-08-23 2022-08-23 Searching method, system and application of pulse neural network for image recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211011023.3A CN115359337B (en) 2022-08-23 2022-08-23 Searching method, system and application of pulse neural network for image recognition

Publications (2)

Publication Number Publication Date
CN115359337A true CN115359337A (en) 2022-11-18
CN115359337B CN115359337B (en) 2023-04-18

Family

ID=84002147

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211011023.3A Active CN115359337B (en) 2022-08-23 2022-08-23 Searching method, system and application of pulse neural network for image recognition

Country Status (1)

Country Link
CN (1) CN115359337B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116910594A (en) * 2023-09-14 2023-10-20 青岛明思为科技有限公司 Rolling bearing fault diagnosis method based on impulse neural network

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004097733A2 (en) * 2003-04-30 2004-11-11 Darwinian Neural Network Industries Ltd Neural networks
CN108875846A (en) * 2018-05-08 2018-11-23 河海大学常州校区 A kind of Handwritten Digit Recognition method based on improved impulsive neural networks
CN112561039A (en) * 2020-12-26 2021-03-26 上海悠络客电子科技股份有限公司 Improved search method of evolutionary neural network architecture based on hyper-network
CN113189553A (en) * 2021-01-28 2021-07-30 北京理工大学 Multi-modal optimization-based micro-maneuvering multi-target rapid detection method
CN113204743A (en) * 2021-05-19 2021-08-03 成都大学 Neutron-gamma discrimination method based on genetic algorithm
CN113947705A (en) * 2021-10-25 2022-01-18 四川大学 Target detection method based on evolutionary neural network under constraint condition
CN114241267A (en) * 2021-12-21 2022-03-25 安徽大学 Structural entropy sampling-based multi-target architecture search osteoporosis image identification method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004097733A2 (en) * 2003-04-30 2004-11-11 Darwinian Neural Network Industries Ltd Neural networks
CN108875846A (en) * 2018-05-08 2018-11-23 河海大学常州校区 A kind of Handwritten Digit Recognition method based on improved impulsive neural networks
CN112561039A (en) * 2020-12-26 2021-03-26 上海悠络客电子科技股份有限公司 Improved search method of evolutionary neural network architecture based on hyper-network
CN113189553A (en) * 2021-01-28 2021-07-30 北京理工大学 Multi-modal optimization-based micro-maneuvering multi-target rapid detection method
CN113204743A (en) * 2021-05-19 2021-08-03 成都大学 Neutron-gamma discrimination method based on genetic algorithm
CN113947705A (en) * 2021-10-25 2022-01-18 四川大学 Target detection method based on evolutionary neural network under constraint condition
CN114241267A (en) * 2021-12-21 2022-03-25 安徽大学 Structural entropy sampling-based multi-target architecture search osteoporosis image identification method

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
HE WANG ET AL: "Compact Graphene-Based Spiking Neural Network With Unsupervised Learning Capabilities" *
JAMES FITZGERALD AND KONGFATT WONG-LIN: "Multi-Objective Optimisation of Cortical Spiking Neural Networks With Genetic Algorithms" *
栾庆林和卢辉斌: "改进遗传算法在神经网络权值优化中的应用研究" *
王宁: "一种基于遗传算法的脉冲神经网络学习新算法" *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116910594A (en) * 2023-09-14 2023-10-20 青岛明思为科技有限公司 Rolling bearing fault diagnosis method based on impulse neural network
CN116910594B (en) * 2023-09-14 2023-12-01 青岛明思为科技有限公司 Rolling bearing fault diagnosis method based on impulse neural network

Also Published As

Publication number Publication date
CN115359337B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
Gomez et al. Learning sparse networks using targeted dropout
Zhang et al. A return-cost-based binary firefly algorithm for feature selection
CN109948029A (en) Based on the adaptive depth hashing image searching method of neural network
CN109241317A (en) Based on the pedestrian's Hash search method for measuring loss in deep learning network
CN111753189A (en) Common characterization learning method for few-sample cross-modal Hash retrieval
CN107832835A (en) The light weight method and device of a kind of convolutional neural networks
CN112685504B (en) Production process-oriented distributed migration chart learning method
CN112905997B (en) Method, device and system for detecting poisoning attack facing deep learning model
CN104156464A (en) Micro-video retrieval method and device based on micro-video feature database
CN111179244A (en) Automatic crack detection method based on cavity convolution
CN115359337B (en) Searching method, system and application of pulse neural network for image recognition
CN112016617B (en) Fine granularity classification method, apparatus and computer readable storage medium
CN112949620B (en) Scene classification method and device based on artificial intelligence and electronic equipment
CN109214426A (en) A kind of method and deep neural network model of the detection of object appearance
CN112148997B (en) Training method and device for multi-modal countermeasure model for disaster event detection
CN116524183A (en) Camouflage target detection method based on multitask adapter fine adjustment
CN113378706A (en) Drawing system for assisting children in observing plants and learning biological diversity
CN110633689B (en) Face recognition model based on semi-supervised attention network
Tian et al. Genetic algorithm based deep learning model selection for visual data classification
CN113553918B (en) Machine ticket issuing character recognition method based on pulse active learning
CN110728186A (en) Fire detection method based on multi-network fusion
CN116434347B (en) Skeleton sequence identification method and system based on mask pattern self-encoder
CN111783688B (en) Remote sensing image scene classification method based on convolutional neural network
CN117197451A (en) Remote sensing image semantic segmentation method and device based on domain self-adaption
CN112329539A (en) Social adaptation model-based crowd behavior identification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant