CN111275742A - Target identification method, device and system and computer readable storage medium - Google Patents
Target identification method, device and system and computer readable storage medium Download PDFInfo
- Publication number
- CN111275742A CN111275742A CN202010062434.XA CN202010062434A CN111275742A CN 111275742 A CN111275742 A CN 111275742A CN 202010062434 A CN202010062434 A CN 202010062434A CN 111275742 A CN111275742 A CN 111275742A
- Authority
- CN
- China
- Prior art keywords
- pulse
- layer
- neurons
- excitation
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a target identification method, a device, a system and a computer readable storage medium, wherein the method comprises the following steps: acquiring original pulse data, determining a pulse sampling window, inputting pulses in the pulse sampling window into a pulse neural network, mapping the pulses input into the pulse neural network, and enabling the pulses to be sequentially transmitted along each layer of excitation neurons except the final layer of excitation neurons, the pulses to be sequentially transmitted along the final layer of excitation neurons to a reasoning layer and the pulses to be sequentially transmitted along each layer of reasoning neurons except the final layer of reasoning neurons under the condition of meeting the condition to determine a recognition result; the device comprises a pulse acquisition module, a sampling window module, a pulse mapping module, a neuron excitation module, a neuron reasoning module and an identification result determining module; the system comprises the device; the method can realize accurate identification and quick identification of the target to be identified, is well suitable for the target with higher movement speed, and can give consideration to identification accuracy and calculated amount.
Description
Technical Field
The present invention relates to the field of object recognition technologies, and in particular, to an object recognition method, an object recognition device, an object recognition system, and a computer-readable storage medium.
Background
In recent years, the field of artificial intelligence is developed rapidly, breakthroughs are made in various aspects such as algorithms, hardware, chips and the like, particularly in the field of images represented by computer vision, a plurality of research results are widely commercialized and civilized, and great convenience is brought to daily life of people. However, the current artificial intelligence algorithm is still the second generation artificial neural network as the main core, the algorithm is proposed as early as eighty years of the last century, and with continuous research for many years, the current artificial intelligence algorithm falls into the research bottleneck in the academic world. Therefore, people shift the emphasis to the third generation artificial neural network (namely, the impulse neural network), but due to the limitation of the prior art, for computer vision applications, especially in the aspect of target identification, the problems of low identification accuracy, low identification speed, large calculation amount, high requirement on hardware, excessive power consumption and the like often exist.
Therefore, how to effectively improve the accuracy and speed of target identification and reduce the requirements of target identification on the calculation amount and power consumption becomes a key point for technical problems to be solved and research all the time by technical personnel in the field.
Disclosure of Invention
In order to solve the problems of poor recognition effect, incapability of effectively recognizing a target moving at a high speed and the like widely existing in the existing target recognition technology, the invention innovatively provides a target recognition method, a device, a system and a computer readable storage medium, wherein the original input pulse is subjected to noise filtering and pulse enhancement through a pulse neural network, so that the accurate recognition and high-efficiency recognition of the target to be recognized can be realized, and the calculated amount and the power consumption can be obviously reduced.
In order to achieve the technical purpose, the invention discloses a target identification method, and the method comprises the following steps;
acquiring original pulse data of a shot target to be identified, wherein the original pulse data comprises at least one pulse sequence;
searching the area where the target to be identified is located by screening each pulse in the pulse sequence, determining a pulse sampling window according to the area where the target to be identified is located, and inputting the pulse in the pulse sampling window into a pulse neural network, wherein the pulse neural network comprises an excitation layer and a reasoning layer;
mapping the pulse input to the pulse neural network so as to enable the number of first-layer excitation neurons of an excitation layer to correspond to the size of a pulse sampling window, wherein the excitation layer comprises a plurality of layers of excitation neurons which are sequentially connected;
under the condition of meeting a first preset condition, enabling the pulse to be transmitted along each layer of the excited neurons except the final layer of the excited neurons in sequence, and under the condition of meeting a second preset condition, enabling the pulse to be transmitted to a reasoning layer along the final layer of the excited neurons, wherein the reasoning layer comprises a plurality of layers of reasoning neurons connected in sequence;
under the condition of meeting a third preset condition, the pulse is transmitted along each layer of reasoning neurons except the last layer of reasoning neurons in sequence;
and determining a recognition result according to the activity of the last layer inference neuron.
Further, the acquired original pulse data of the target to be recognized is derived from the bionic vision sensor, and the target to be recognized is shot by the bionic vision sensor.
Further, the first preset condition is that the membrane potential variation of each layer of firing neurons except the last layer of firing neurons exceeds the first firing threshold of the firing neurons.
Further, the second preset condition is that the membrane potential variation of the terminal layer firing neuron exceeds a second firing threshold and the ratio of the number of activated neurons in the terminal layer firing neuron is greater than the first preset ratio.
Further, the third preset condition is that the membrane potential variation of each layer of inference neurons except the last layer of inference neurons exceeds the third excitation threshold of the inference neurons.
Further, after the pulse sampling window is determined, the method also comprises the step of recording the time interval of two adjacent occurrences of the target to be identified in the pulse sampling window.
Furthermore, the original pulse data includes a plurality of pulse sequences, and the identification results are output only when the identification results obtained by simultaneously performing the above-mentioned processing on all the pulse sequences are consistent.
In order to achieve the technical purpose, the invention also discloses a target identification device, and the device comprises a pulse acquisition module, a sampling window module, a pulse mapping module, a neuron excitation module, a neuron reasoning module and an identification result determining module;
the pulse acquisition module is used for acquiring original pulse data of a shot target to be identified; the original pulse data comprises at least one pulse sequence;
the sampling window module is used for searching the area where the target to be identified is located by screening each pulse in the pulse sequence, determining a pulse sampling window according to the area where the target to be identified is located, and inputting the pulse in the pulse sampling window to the pulse neural network; the impulse neural network comprises an excitation layer and an inference layer;
the pulse mapping module is used for mapping the pulse input to the pulse neural network so as to enable the number of first-layer excitation neurons of the excitation layer to correspond to the size of a pulse sampling window; the excitation layer comprises a plurality of layers of excitation neurons which are connected in sequence;
the neuron excitation module is used for enabling the pulse to be transmitted in sequence along each layer of the excitation neurons except the final layer of the excitation neurons under the condition that a first preset condition is met, and is used for enabling the pulse to be transmitted to the inference layer along the final layer of the excitation neurons under the condition that a second preset condition is met; the reasoning layer comprises a plurality of layers of reasoning neurons which are connected in sequence;
the neuron reasoning module is used for enabling the pulse to be transmitted in sequence along each layer of reasoning neurons except the last layer of reasoning neurons under the condition that a third preset condition is met;
and the identification result determining module is used for determining an identification result according to the activity of the last layer inference neuron.
In order to achieve the technical purpose, the invention also discloses a target recognition system, which comprises a bionic vision sensor and the target recognition device, wherein the bionic vision sensor is used for shooting the target to be recognized so as to obtain the original pulse data of the target to be recognized.
To achieve the above technical object, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement any one of the object recognition methods as described above.
The invention has the beneficial effects that: compared with the prior art, the method can realize accurate identification and quick identification of the target to be identified, is well suitable for the target with higher motion speed, and can simultaneously take accurate identification and the problems of calculated amount and power consumption into consideration. The invention can also realize the rotation speed measurement of the high-speed rotating target, thereby providing a better solution for the rotation speed calibration of the rotating body.
The invention can record the dynamic change information of a high-speed moving target through the bionic vision sensor with high time resolution, process the pulse sequence output by the bionic vision sensor through the impulse neural network, can realize noise filtering and impulse enhancement to the original input impulse, so as to better realize accurate identification and rapid identification of the target, and particularly can reduce the impulse input quantity of a reasoning layer by controlling the activation proportion value of impulse neurons of an excitation layer, thereby reducing the calculated quantity and power consumption of the whole impulse processing algorithm; the invention can obviously improve the identification accuracy of the impulse neural network through a multiple measurement mechanism; the invention realizes the rotating speed measurement of the high-speed rotating target by analyzing the characteristics of circular motion and adopting a method of fixing a sampling window, thereby providing an implementation method for the rotating speed calibration of a rotating body.
Drawings
Fig. 1 is a flow chart illustrating a target identification method according to some embodiments of the present invention.
FIG. 2 is a block diagram of a target recognition system in some embodiments of the invention.
Fig. 3 is a schematic diagram illustrating the operation of a pulse identification process with multiple excitation neurons in a local connection relationship according to some embodiments of the present invention.
Fig. 4 is a schematic diagram illustrating the operation of the pulse recognition process according to another embodiment of the present invention.
Fig. 5 shows the effect of filtering and enhancing the pulse sequence of the region where the target-character "P" to be recognized is located through the excitation layer.
Fig. 6 shows the effect of noise filtering and pulse enhancement on the pulse sequence of the region where the target-character "C" to be recognized is located through the excitation layer.
Fig. 7 shows the effect of noise filtering and pulse enhancement on the pulse sequence of the region where the target-character "L" to be recognized is located through the excitation layer.
Detailed Description
The following describes and explains a method, an apparatus, a system and a computer readable storage medium for object recognition in detail with reference to the drawings of the specification.
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a target identification method according to some embodiments of the present invention. The embodiment provides an object identification method, and particularly, the object identification method can comprise the following steps.
Step 100, in this embodiment, a target to be recognized may be photographed by a bionic vision sensor, so as to obtain original pulse data of the target to be recognized, where the bionic vision sensor used is an electronic device manufactured by simulating a vision sensing principle of an eye of a living body, and may provide information input for a subsequent brain-like neural network algorithm, and the bionic vision sensor often has a very high time resolution and may record dynamic change information of a target moving at a high speed, and the bionic vision sensor used in the present invention includes, but is not limited to, the following three types: a differential sensor, an integral sensor, and an event sensor; among them, the differential Sensor is most typically a Dynamic Vision Sensor (DVS), and the DVS outputs only a pixel address and information whose light intensity changes, and thus is only sensitive to a moving object but not to a static object; the integral sensor is mainly represented by a simulated retina sensor (Vidar) which is researched by professor of yellow iron force at Beijing university at present, simulates the cell connection structure of the fovea region of retina and the principle of integrating and releasing pulses, and outputs the pulses released by all pixels at each moment in an array form, has the characteristics of full time, asynchrony, high speed and the like, and has sensitive sensing capability on dynamic objects and static objects; the event Sensor combines DVS with a conventional camera, and outputs a frame Image while outputting a pulse data stream, which may be ATIS (Asynchronous Time-based Image Sensor), DAVIS (Dynamic and active-pixel Vision Sensor), or CeleX. Therefore, the invention can also be a target identification method based on the bionic vision sensor and can also be a method for quickly identifying the target. In the target identification scheme, the invention can enable the bionic vision sensor to take the role of 'eyes'.
Step 101, acquiring original pulse data of a shot target to be identified, wherein the original pulse data comprises at least one pulse sequence, in this embodiment, one pulse sequence may comprise one frame of pulse or multiple frames of pulses, and multiple pulse sequences or multiple frames of pulses may be used for subsequent multiple measurement mechanisms; in a preferred embodiment, the acquired original pulse data of the target to be recognized is derived from a bionic vision sensor, and the target to be recognized is photographed by using the bionic vision sensor.
102, searching the area of the target to be identified by screening each pulse in the pulse sequence, and determining a pulse sampling window according to the area of the target to be identified so as to take the area of the target as the range of the pulse sampling window. In some preferred embodiments, after the pulse sampling window is determined, the step of recording a time interval between two adjacent times of the target to be identified appearing in the pulse sampling window is further included, it should be understood that the time interval here may be an average value of a plurality of time intervals or a time interval between two adjacent times at random. The invention uses the impulse Neural Network, which relates to the brain-like computing field, the impulse Neural Network (SNN) is called as the third generation artificial Neural Network, the neuron model of the impulse Neural Network is closer to the biological neuron characteristic, the brain is used for reference in the connection mode and the information processing mechanism, the impulse is used as the medium of information transmission, the time information and the space information are both contained in the impulse, so the impulse Neural Network is the most representative algorithm in the brain-like computing field, and along with the development of deep learning, the impulse Neural Network also changes in the connection mode and the application scene, for example, in the connection mode, the impulse Neural Network develops deep simple connection from early shallow complex connection, for example, in the application scene, the impulse Neural Network can be applied to various pattern recognition problems, in particular to the recognition of high-speed rotating characters in the invention, compared with algorithms such as deep learning and machine learning, the impulse neural network has higher biological likelihood, and is lower in calculation energy consumption and higher in processing speed by virtue of an asynchronous Response and processing mechanism of the impulse neural network, the impulse neuron (spiking neuron) is a basic component unit and an information processing unit of the impulse neural network and is a mathematical formula expression after the electrochemical characteristics of the biological neuron are simplified, and the neuron Model in the impulse neural network can comprise an integral excitation Model and Fire (IF), a leakage current integral excitation Model and Fire (LIF), an impulse Response Model (SRM) and the like. In this step, the pulse in the pulse sampling window is input to the impulse neural network, which in the present invention may include an excitation layer and an inference layer.
Step 103, processing the input pulse through the spiking neural network, specifically, in this embodiment, mapping the pulse input to the spiking neural network so that the number of first-layer firing neurons of the firing layer corresponds to the size of the pulse sampling window, the mapping manner may adopt a hash mapping manner, a binary conversion manner, and the like, the firing layer of this embodiment includes multiple layers of firing neurons sequentially connected through synapses, the connection manner between adjacent firing neurons includes, but is not limited to, local connection, full connection, or dynamic random connection, the firing layer of this embodiment is composed of (leak integration and Fire) LIF neurons, the first-layer firing neurons of the firing layer are input spiking neurons, and the other-layer firing neurons are firing spiking neurons. In a specific operation, the membrane potential of the excitation neuron changes after receiving the pulse.
104, sequentially transmitting pulses along each layer of the excitation neurons except the last layer of the excitation neurons under the condition that a first preset condition is met, and resetting the membrane potential of each layer of the excitation neurons except the last layer of the excitation neurons to a resting state after the current excitation neurons sends the pulses to the next layer of the excitation neurons or the inference layer, wherein the first preset condition is that the membrane potential variation of each layer of the excitation neurons except the last layer of the excitation neurons exceeds a first excitation threshold of each layer of the excitation neurons, and the first excitation threshold is 1.0 in the embodiment, so that noise filtering and pulse enhancement of the pulses input into the impulse neural network are realized; and transmitting the pulse to the inference layer along the last layer excitation neuron when a second preset condition is met, in this embodiment, the second preset condition may be set as two types: the first condition is that the membrane potential variation of the peripheral excitatory neuron exceeds the second excitation threshold, the membrane potential variation of the second and peripheral excitatory neurons exceeds the second excitation threshold, and the ratio of the number of activated neurons in the peripheral excitatory neurons (i.e. the ratio of the number of excited neurons to the number of all neurons in the peripheral layer) is greater than a first preset ratio, the first preset ratio in the embodiment is 15%. In some preferred embodiments of the present invention, please refer to fig. 3 and 4, fig. 3 is a schematic diagram illustrating an operation principle of a pulse recognition process in some embodiments of the present invention, for example, a plurality of excitation neurons having a local connection relationship, and fig. 4 is a schematic diagram illustrating an operation principle of a pulse recognition process in other embodiments of the present invention. Only when the number of the activated pulse neurons in the excitation layer exceeds a certain proportional value, the issued pulses are tiled into a single dimension and then are uniformly input into the inference layer, when no target exists in the sampling window, if background noise is input into the sampling window, the excitation layer can not issue the pulses to the inference layer by adjusting the proportional value when only the background noise is input, the calculated amount of the whole pulse recognition algorithm is further reduced, and the calculation energy consumption is greatly reduced.
The inference layer in this embodiment is composed of (integrated and Fire) IF neurons, on the basis of the disclosure of the present invention, the number of layers of the inference layer and the number of neurons in each layer may be reasonably and wisely adjusted according to actual situations, the inference neurons in each layer are connected by synapses, the synapse connection mode includes, but is not limited to, convolutional connection, local connection, or full connection, the synapse strength may be obtained by using an existing neural network training mode, the training method includes, but is not limited to, a gradient-based back propagation learning algorithm, a reinforcement learning algorithm, and a pulse time dependent synapse Plasticity algorithm (STDP), and the present invention is not repeated. After receiving the pulse, the inference neuron of the invention changes the membrane potential of the inference neuron.
And 105, sequentially transmitting pulses along each layer of the inference neurons except the last layer of the inference neurons under the condition that a third preset condition is met, wherein in the embodiment, the third preset condition is that the membrane potential variation of each layer of the inference neurons except the last layer of the inference neurons exceeds a third excitation threshold of the inference neurons, and the third preset value can be 1.0. In the various embodiments provided herein, it should be understood that "sequentially delivering" refers to a preceding neuron delivering a pulse to a following neuron, as shown in fig. 3 and 4.
And step 106, determining a recognition result according to the activity of the last layer inference neuron. For example, the IF neuron in the last layer obtains the recognition result by comparing the respective activity degrees, that is, the activity degrees of the inference neurons in the last layer respectively reach various set threshold values, the recognition can be determined to be successful, otherwise, the recognition fails, or, in this embodiment, the recognition can be determined whether the recognition is successful by whether the sum of the activity degrees of the inference neurons in the last layer reaches a preset value. As some preferred technical solutions, in some embodiments of the present invention, a multiple measurement mechanism may be used, that is, the above steps 101 to 106 may be executed multiple times, preferably, synchronously, a segment of pulse sequence output by the artificial vision sensor is acquired in step 101 (the bionic vision sensor breaks the concept of a traditional camera frame and generally has a high time resolution), then the original pulse data includes multiple pulse sequences, the result calculated by the pulse recognition algorithm is also more than one but continuous multiple, in this embodiment, the recognition result is finally output only when multiple recognition results obtained by performing the above processing on all pulse sequences are consistent, and the improved scheme can significantly improve the recognition accuracy. Therefore, the target identification method provided in the above embodiments of the present invention can complete quick identification of the target by using the time information and the spatial information at the same time.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a target recognition system according to some embodiments of the present disclosure. The invention provides a target recognition system, which comprises a bionic vision sensor and a target recognition device, wherein the target recognition device can be integrated on a pulse calculation platform and used for realizing a pulse processing algorithm, the bionic vision sensor is used for shooting a target to be recognized so as to obtain original pulse data (namely recorded data) of the target to be recognized, and then the recorded data can be output to the pulse calculation platform in real time.
Pulse computing platforms used in the present invention include, but are not limited to, the following types: (1) a server, a workstation, a desktop host or a mobile computer, (2) an embedded computing platform, such as a Multi-Processor System-on-Chip (MPSoC) development board, a Central Processing Unit (CPU) development board, a Graphics Processing Unit (GPU) development board, a Single-Chip Microcomputer (SCM), etc., (3) a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), etc., (4) a cloud computing platform, etc. The pulse calculation platform is used for running pulse processing algorithms, which may include system control algorithms and pulse recognition algorithms.
Specifically, some embodiments of the present invention provide a target recognition apparatus, which includes a pulse acquisition module, a sampling window module, a pulse mapping module, a neuron excitation module, a neuron inference module, and a recognition result determination module.
The pulse acquisition module is used for acquiring original pulse data of a shot target to be identified; the original pulse data comprises at least one pulse sequence, wherein one pulse sequence comprises one frame pulse or multiple frames of pulses, and multiple pulse sequences or multiple frames of pulses can be used for subsequent multiple measurement mechanisms.
The sampling window module is used for searching the area where the target to be identified is located in a mode of screening each pulse in the pulse sequence, determining a pulse sampling window according to the area where the target to be identified is located, and inputting the pulse in the pulse sampling window to the pulse neural network; the impulse neural network includes an excitation layer and an inference layer. The pulse data in the sampling window is screened out by the sampling window module after the pulse data is received from the algorithm processing platform and is input into the excitation layer, a tiny time difference is generated in the process, and the tiny time difference can be measured by the current time recording module and is marked as delta t 2.
The pulse mapping module is used for mapping the pulse input to the pulse neural network so as to enable the number of the first layer of excitation neurons of the excitation layer to correspond to the size of a pulse sampling window, can be understood as a pulse event mapping layer of the pulse neural network in the pulse identification algorithm used by the invention, and can map the pulse event to the input pulse neurons of the excitation layer by adopting methods such as Hash mapping, binary conversion and the like; the excitation layer includes a plurality of excitation neurons sequentially connected by synapse, and the synapse connection may be local connection, full connection, or dynamic random connection.
Firstly, the number of input pulse neurons of the excitation layer needs to be consistent with the size of a pulse sampling window, and assuming that the size of the sampling window is N × N, the input pulses in the sampling window are actually generated by exciting the photosensitive neurons at the corresponding positions of the N × N bionic vision sensors, and a time delay (Δ t1+ Δ t2) is generated when the input pulse neurons reach the excitation layer. As shown in fig. 3, each input pulse neuron is locally connected to a plurality of excitation pulse neurons in a manner that: pulse neurons in the peripheral neighborhood range with the radius of R are connected by taking the position of the input pulse neuron as the center (if the input neuron is positioned at the boundary of the sampling window, only pulse neurons positioned in the range of N x N are connected).
The neuron excitation module is used for enabling the pulse to be transmitted in sequence along each layer of the excitation neurons except the final layer of the excitation neurons under the condition that a first preset condition is met, and is used for enabling the pulse to be transmitted to the inference layer along the final layer of the excitation neurons under the condition that a second preset condition is met; the inference layer comprises a plurality of layers of inference neurons which are connected in sequence. Specifically, the first preset condition is that the membrane potential variation of each layer of the firing neurons except the last layer of the firing neurons exceeds the first firing threshold, and the second preset condition is that the membrane potential variation of the last layer of the firing neurons exceeds the second firing threshold and the ratio of the number of activated neurons in the last layer of the firing neurons is greater than the first preset ratio. Specifically, when a pulse neuron fires a pulse, the membrane potential of the firing pulse neurons connected to and located at the same position increases by Δ 1 (i.e., the synaptic strength of the connection is Δ 1), the membrane potential of the remaining firing pulse neurons connected to and located at the same position increases by Δ 2 (i.e., the synaptic strength of the connection is Δ 2), where Δ 1> Δ 2, and when no pulse is input, the membrane potential of the firing pulse neurons connected to and located at the same position attenuates by Δ 3 (i.e., the membrane potential attenuation factor of the LIF neuron is Δ 3); the values of Δ 1, Δ 2, and Δ 3 may be preset to fixed values or may be dynamically adjusted. When the membrane potential of a firing pulse neuron exceeds its firing threshold, the neuron activates and fires a pulse, while at the same time resetting its membrane potential to a resting state.
And the neuron reasoning module is used for enabling the pulse to be transmitted in sequence along each layer of reasoning neurons except the last layer of reasoning neurons under the condition that a third preset condition is met. Specifically, the third preset condition is that the membrane potential variation of each layer of inference neurons except the last layer of inference neurons exceeds the third excitation threshold of the inference neurons. Specifically, unlike the LIF neurons of the excitation layer, the IF neurons of the inference layer are characterized by no attenuation of the membrane potential, and the IF neurons of the inference layer increase their own membrane potential after receiving the pulse input, the increased value being equal to the sum of the synaptic strength values of all the pulse inputs; IF the membrane potential of the IF neuron exceeds the excitation threshold of the IF neuron, a pulse is sent out and transmitted to the next layer of connected IF neurons, meanwhile, the membrane potential of the IF neuron resets, and the pulse propagates backwards layer by layer until the last layer.
And the recognition result determining module is used for determining a recognition result according to the activity of the last layer inference neuron. For example, the IF neuron in the last layer obtains the recognition result by comparing the respective activity degrees, that is, the activity degrees of the inference neurons in the last layer respectively reach various set threshold values, the recognition can be determined to be successful, otherwise, the recognition fails, or, in this embodiment, the recognition can be determined whether the recognition is successful by whether the sum of the activity degrees of the inference neurons in the last layer reaches a preset value.
In some improved embodiments of the present invention, the target recognition apparatus may further include a rotation speed measurement module and a current time recording module, the rotation speed measurement module is configured to record a time interval, in which the target to be recognized appears in the pulse sampling window twice, after the pulse sampling window is determined, the rotation speed measurement module fixes the sampling window at a position in the field of view of the camera according to the rotation radius of the rotating target, then accumulates and records a time interval, in which the same target appears in the sampling window multiple times in succession, and may reduce an error by a statistical method, calculate a time required for the target to rotate for one cycle, and further implement measurement of the rotation speed of the high-speed rotating target. The current time recording module is used for accurately acquiring the current time, the specific form of the time can be reasonably and judiciously selected according to the needs, and the time precision delay can be as small as possible. The invention adopts the pulse neural network to process the pulse sequence output by the bionic vision sensor, and the processing method simultaneously utilizes the time information and the space information, can realize the quick identification of the target and is still suitable under the condition of higher movement speed.
In some embodiments of the present invention, a computer-readable storage medium may be provided, and the computer-readable storage medium stores thereon a computer program, which can be executed by a processor, so as to implement the object identification method in the above embodiments, and the present invention enables the computer program to be run on a pulse computing platform.
The working process of the present invention and the significant effects brought by the practice thereof will be further described in detail by specific experimental procedures. In the embodiment, the integral bionic vision sensor Vidar is adopted, the time resolution of the sensor is 25 microseconds, and the display resolution is 400 × 250 pixels. The sensor is used for shooting at a clear noon directly opposite to an industrial fan rotating at a high speed, the center distance between the fan and the sensor is about 60cm, the radius of the fan is about 20cm, 3 black-bottom and white English characters ('P', 'C', 'L') are pasted on blades of the fan, the rotating speed of the blades is about 2500R/min, the shot pulse number data are packaged on a data acquisition card and then transmitted to a desktop computer through a USB3.0 bus for processing, the desktop computer receives and acquires data output by a Vidar in real time, a sampling window setting module screens the pulse data, a pulse event mapping layer maps the screened pulse data to input pulse neurons of an excitation layer one by one, the position of a sampling window is set according to the position of the English characters in the Vidar field, and the size of the sampling window is set according to the pixel size of the characters in the Vidar field. In this embodiment, the coordinates (y, x) of the upper left corner of the sampling window are (50,180), the size of the sampling window is 40 × 40 pixels, that is, the excitation layer includes 40 × 40 input pulse neurons, the pulse processing algorithm processes the pulse data input by the biomimetic visual sensor, as shown in fig. 4, the number of pulse input neurons of the excitation layer is consistent with the size of the sampling window and is 40 × 40, the pulse event mapping module implements one-to-one mapping of the pulses input in the sampling window and the pulse input neurons, each pulse input neuron is connected with 9 excitation pulse neurons by synapses, and the synapses are connected in the following manner: the excitation pulse neurons in the range of eight surrounding neighborhoods centered on the input pulse neuron's own position are connected (if the input pulse neuron is located at the boundary of the sampling window, only the excitation pulse neurons located in the range of 40 × 40 are connected). When the input pulse neuron sends out a pulse, the membrane potential of the connected excitation pulse neuron at the same position is increased by 1.4 (delta 1), and the membrane potential of the rest connected excitation pulse neurons is increased by 0.5 (delta 2). When no pulse is input, the membrane potential of the excitation pulse neuron connected and located at the same position decays by 0.25(Δ 3). When the membrane potential of the excitation pulse neuron exceeds the excitation threshold (1.0) of the excitation pulse neuron, the excitation pulse neuron activates and emits a pulse, please refer to fig. 5 to 7, which show the effect of performing noise filtering and pulse enhancement on a pulse sequence by using the excitation layer provided by the present invention, the left graph is the pulse input of the bionic visual sensor, the right graph is the effect of performing filtering and enhancement by using the excitation layer of the present invention, specifically, the white part represents the position with pulse input, the black part represents the position without pulse input, and the square area represents the pulse sampling window; fig. 5 shows the effect of filtering and enhancing the pulse sequence in the region where the target-character "P" is to be recognized through the excitation layer, fig. 6 shows the effect of performing noise filtering and pulse enhancement on the pulse sequence in the region where the target-character "C" is to be recognized through the excitation layer, fig. 7 shows the effect of performing noise filtering and pulse enhancement on the pulse sequence in the region where the target-character "L" is to be recognized through the excitation layer, and only when the number of activated excitation pulse neurons exceeds a certain proportional value (15%), the excited pulses are tiled into one dimension (1 × 1600) and then are uniformly input to the inference layer. When the rotating character does not appear in the sampling window, the background noise is input in the sampling window; as shown in fig. 4, the inference layer is composed of 3 layers of IF neurons, the number of the IF neurons in each layer is 512, 1024, and 3, and the IF neurons in each layer realize synaptic connection in a full connection manner. Upon receipt of a pulse input, the IF neuron increases its membrane potential by an amount equal to the sum of the synaptic strength values of all pulse inputs. IF the membrane potential of the IF neuron exceeds the excitation threshold (1.0) of the IF neuron, a pulse is sent out and transmitted to the next connected IF neuron, and the membrane potential of the IF neuron is reset to the resting state (0). The pulse propagates backwards layer by layer until reaching the last layer, and the last layer of IF neurons obtains a recognition result by comparing the activity of each neuron. According to the invention, a multiple measurement mechanism is adopted, for example, only when the recognition results output by the inference layer for 5 times are consistent, the final recognition result is obtained, the mode can obviously improve the recognition accuracy and the recognition speed, and the measurement of the rotating speed of the high-speed rotating character is realized. The invention obviously improves the recognition speed by an optimization method of multi-thread parallel computation, divides the computation task into a plurality of processors of a desktop computer for simultaneous computation, and can calculate the time required by one circle of character rotation by recording the time interval of the same character appearing in a sampling window twice continuously, thereby realizing the measurement of the rotating speed of a high-speed rotating fan.
In the description herein, references to the description of the term "the present embodiment," "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and simplifications made in the spirit of the present invention are intended to be included in the scope of the present invention.
Claims (10)
1. A method of object recognition, characterized by: the method comprises the following steps;
acquiring original pulse data of a shot target to be identified, wherein the original pulse data comprises at least one pulse sequence;
searching the area where the target to be identified is located by screening each pulse in the pulse sequence, determining a pulse sampling window according to the area where the target to be identified is located, and inputting the pulse in the pulse sampling window into a pulse neural network, wherein the pulse neural network comprises an excitation layer and a reasoning layer;
mapping the pulse input to the pulse neural network so as to enable the number of first-layer excitation neurons of an excitation layer to correspond to the size of a pulse sampling window, wherein the excitation layer comprises a plurality of layers of excitation neurons which are sequentially connected;
under the condition of meeting a first preset condition, enabling the pulse to be transmitted along each layer of the excited neurons except the final layer of the excited neurons in sequence, and under the condition of meeting a second preset condition, enabling the pulse to be transmitted to a reasoning layer along the final layer of the excited neurons, wherein the reasoning layer comprises a plurality of layers of reasoning neurons connected in sequence;
under the condition of meeting a third preset condition, the pulse is transmitted along each layer of reasoning neurons except the last layer of reasoning neurons in sequence;
and determining a recognition result according to the activity of the last layer inference neuron.
2. The object recognition method of claim 1, wherein: the acquired original pulse data of the target to be recognized is derived from the bionic vision sensor, and the target to be recognized is shot by the bionic vision sensor.
3. The object recognition method according to claim 1 or 2, characterized in that: the first preset condition is that the membrane potential variation of each layer of excitation neurons except the last layer of excitation neurons exceeds the first excitation threshold of the excitation neurons.
4. The object recognition method of claim 3, wherein: the second preset condition is that the membrane potential variation of the last layer excitatory neuron exceeds a second excitatory threshold and the proportion of the number of activated neurons in the last layer excitatory neuron is larger than the first preset proportion.
5. The object recognition method of any one of claims 1, 2 or 4, wherein: the third preset condition is that the membrane potential variation of each layer of reasoning neurons except the last layer of reasoning neurons exceeds the third excitation threshold of the each layer of reasoning neurons.
6. The object recognition method of claim 1, wherein: after the pulse sampling window is determined, the method further comprises the step of recording the time interval of two adjacent occurrences of the target to be identified in the pulse sampling window.
7. The object recognition method of any one of claims 1, 2, 4 or 6, wherein: the original pulse data comprises a plurality of pulse sequences, and the identification results are output only when the identification results of all the pulse sequences after the processing are consistent.
8. An object recognition apparatus characterized by: the device comprises a pulse acquisition module, a sampling window module, a pulse mapping module, a neuron excitation module, a neuron reasoning module and an identification result determining module;
the pulse acquisition module is used for acquiring original pulse data of a shot target to be identified; the original pulse data comprises at least one pulse sequence;
the sampling window module is used for searching the area where the target to be identified is located by screening each pulse in the pulse sequence, determining a pulse sampling window according to the area where the target to be identified is located, and inputting the pulse in the pulse sampling window to the pulse neural network; the impulse neural network comprises an excitation layer and an inference layer;
the pulse mapping module is used for mapping the pulse input to the pulse neural network so as to enable the number of first-layer excitation neurons of the excitation layer to correspond to the size of a pulse sampling window; the excitation layer comprises a plurality of layers of excitation neurons which are connected in sequence;
the neuron excitation module is used for enabling the pulse to be transmitted in sequence along each layer of the excitation neurons except the final layer of the excitation neurons under the condition that a first preset condition is met, and is used for enabling the pulse to be transmitted to the inference layer along the final layer of the excitation neurons under the condition that a second preset condition is met; the reasoning layer comprises a plurality of layers of reasoning neurons which are connected in sequence;
the neuron reasoning module is used for enabling the pulse to be transmitted in sequence along each layer of reasoning neurons except the last layer of reasoning neurons under the condition that a third preset condition is met;
and the identification result determining module is used for determining an identification result according to the activity of the last layer inference neuron.
9. An object recognition system, characterized by: the target recognition system comprises a bionic vision sensor and the target recognition device of claim 8, wherein the bionic vision sensor is used for shooting a target to be recognized so as to obtain original pulse data of the target to be recognized.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program is executed by a processor to implement the object recognition method as claimed in any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010062434.XA CN111275742B (en) | 2020-01-19 | 2020-01-19 | Target identification method, device and system and computer readable storage medium |
| PCT/CN2020/099443 WO2021143066A1 (en) | 2020-01-19 | 2020-06-30 | Target recognition method, device, and system, and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010062434.XA CN111275742B (en) | 2020-01-19 | 2020-01-19 | Target identification method, device and system and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275742A true CN111275742A (en) | 2020-06-12 |
| CN111275742B CN111275742B (en) | 2022-01-11 |
Family
ID=71003474
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010062434.XA Active CN111275742B (en) | 2020-01-19 | 2020-01-19 | Target identification method, device and system and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111275742B (en) |
| WO (1) | WO2021143066A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021143066A1 (en) * | 2020-01-19 | 2021-07-22 | 北京大学 | Target recognition method, device, and system, and computer readable storage medium |
| CN113554726A (en) * | 2021-06-04 | 2021-10-26 | 北京大学 | Image reconstruction method and device based on pulse array, storage medium and terminal |
| CN114202068A (en) * | 2022-02-17 | 2022-03-18 | 浙江大学 | Self-learning implementation system for brain-like computing chip |
| WO2023202313A1 (en) * | 2022-04-18 | 2023-10-26 | 北京大学 | Position prediction method and apparatus, electronic device, and storage medium |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112734012B (en) * | 2021-01-07 | 2024-03-05 | 北京灵汐科技有限公司 | Impulse neural network training method, data processing method, electronic equipment and medium |
| CN114723038B (en) * | 2022-03-03 | 2024-04-23 | 北京大学 | Model conversion method, device, equipment and storage medium based on target detection |
| CN115048979A (en) * | 2022-04-29 | 2022-09-13 | 贵州大学 | Robot touch pulse data classification method based on regularization |
| CN115100458A (en) * | 2022-06-02 | 2022-09-23 | 西安电子科技大学 | Image classification method and related device |
| CN114998996B (en) * | 2022-06-14 | 2024-04-05 | 中国电信股份有限公司 | Signal processing method, device and equipment with motion attribute information and storage |
| CN115841142B (en) * | 2023-02-20 | 2023-06-06 | 鹏城实验室 | Visual cortex simulation method and related equipment based on deep pulse neural network |
| CN115862338B (en) * | 2023-03-01 | 2023-05-16 | 天津大学 | Airport traffic flow prediction method, airport traffic flow prediction system, electronic equipment and medium |
| CN116011563B (en) * | 2023-03-28 | 2023-07-21 | 之江实验室 | High-performance pulse transmission simulation method and device for pulse relay |
| CN116663631B (en) * | 2023-07-28 | 2024-05-14 | 之江实验室 | Task execution method and device, storage medium and electronic equipment |
| CN117252239A (en) * | 2023-09-18 | 2023-12-19 | 北京邮电大学 | Method and system for detecting target of pulse signal |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130325766A1 (en) * | 2012-06-04 | 2013-12-05 | Csaba Petre | Spiking neuron network apparatus and methods |
| CN106295799A (en) * | 2015-05-12 | 2017-01-04 | 核工业北京地质研究院 | A kind of implementation method of degree of depth study multilayer neural network |
| CN108470190A (en) * | 2018-03-09 | 2018-08-31 | 北京大学 | The image-recognizing method of impulsive neural networks is customized based on FPGA |
| CN109871940A (en) * | 2019-01-31 | 2019-06-11 | 清华大学 | A kind of multilayer training algorithm of impulsive neural networks |
| CN110210563A (en) * | 2019-06-04 | 2019-09-06 | 北京大学 | The study of pattern pulse data space time information and recognition methods based on Spike cube SNN |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11853875B2 (en) * | 2017-10-23 | 2023-12-26 | Samsung Electronics Co., Ltd. | Neural network apparatus and method |
| CN110659666B (en) * | 2019-08-06 | 2022-05-13 | 广东工业大学 | Image classification method of multilayer pulse neural network based on interaction |
| CN111275742B (en) * | 2020-01-19 | 2022-01-11 | 北京大学 | Target identification method, device and system and computer readable storage medium |
-
2020
- 2020-01-19 CN CN202010062434.XA patent/CN111275742B/en active Active
- 2020-06-30 WO PCT/CN2020/099443 patent/WO2021143066A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130325766A1 (en) * | 2012-06-04 | 2013-12-05 | Csaba Petre | Spiking neuron network apparatus and methods |
| CN106295799A (en) * | 2015-05-12 | 2017-01-04 | 核工业北京地质研究院 | A kind of implementation method of degree of depth study multilayer neural network |
| CN108470190A (en) * | 2018-03-09 | 2018-08-31 | 北京大学 | The image-recognizing method of impulsive neural networks is customized based on FPGA |
| CN109871940A (en) * | 2019-01-31 | 2019-06-11 | 清华大学 | A kind of multilayer training algorithm of impulsive neural networks |
| CN110210563A (en) * | 2019-06-04 | 2019-09-06 | 北京大学 | The study of pattern pulse data space time information and recognition methods based on Spike cube SNN |
Non-Patent Citations (1)
| Title |
|---|
| 徐桂芝 等；: "《基于忆阻器的脉冲神经网络研究综述》", 《生物医学工程学杂志》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021143066A1 (en) * | 2020-01-19 | 2021-07-22 | 北京大学 | Target recognition method, device, and system, and computer readable storage medium |
| CN113554726A (en) * | 2021-06-04 | 2021-10-26 | 北京大学 | Image reconstruction method and device based on pulse array, storage medium and terminal |
| CN113554726B (en) * | 2021-06-04 | 2023-10-20 | 北京大学 | Image reconstruction method and device based on pulse array, storage medium and terminal |
| CN114202068A (en) * | 2022-02-17 | 2022-03-18 | 浙江大学 | Self-learning implementation system for brain-like computing chip |
| CN114202068B (en) * | 2022-02-17 | 2022-06-28 | 浙江大学 | Self-learning implementation system for brain-like computing chip |
| WO2023202313A1 (en) * | 2022-04-18 | 2023-10-26 | 北京大学 | Position prediction method and apparatus, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275742B (en) | 2022-01-11 |
| WO2021143066A1 (en) | 2021-07-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111275742B (en) | Target identification method, device and system and computer readable storage medium | |
| Kim et al. | Optimizing deeper spiking neural networks for dynamic vision sensing | |
| EP4074247A1 (en) | Health status test method and device, and computer storage medium | |
| EP3971684A1 (en) | Vision-based tactile measurement method, apparatus, chip and storage medium | |
| US10832039B2 (en) | Facial expression detection method, device and system, facial expression driving method, device and system, and storage medium | |
| CN111274916B (en) | Face recognition method and face recognition device | |
| US20140365414A1 (en) | Spiking network apparatus and method with bimodal spike-timing dependent plasticity | |
| WO2022116423A1 (en) | Object posture estimation method and apparatus, and electronic device and computer storage medium | |
| CN106407889A (en) | Video human body interaction motion identification method based on optical flow graph depth learning model | |
| CN111611851B (en) | Model generation method, iris detection method and device | |
| CN109033945A (en) | A kind of human body contour outline extracting method based on deep learning | |
| CN113326735B (en) | YOLOv 5-based multi-mode small target detection method | |
| CN116250021A (en) | Training method of image generation model, new view angle image generation method and device | |
| CN114067166A (en) | Apparatus and method for determining physical properties of a physical object | |
| CN114091554A (en) | Training set processing method and device | |
| CN116524183A (en) | Camouflage target detection method based on multitask adapter fine adjustment | |
| CN114842270A (en) | Target image classification method and device, electronic equipment and medium | |
| CN110517345A (en) | A kind of method and device of threedimensional model rendering | |
| Li et al. | Novel biological visual attention mechanism via Gaussian harmony search | |
| Chen et al. | Fast retinomorphic event-driven representations for video gameplay and action recognition | |
| US20210056247A1 (en) | Pose detection of objects from image data | |
| Banerjee et al. | Gaze Detection Using Encoded Retinomorphic Events | |
| Wang et al. | Research on an Effective Human Action Recognition Model Based on 3D CNN | |
| CN112699730B (en) | Machine room character re-identification method based on YOLO and convolution-cycle network | |
| Kerr et al. | A novel approach to robot vision using a hexagonal grid and spiking neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220125 Address after: 100084 b501b-2, 5th floor, building 8, yard 1, Zhongguancun East Road, Haidian District, Beijing Patentee after: Pulse vision (Beijing) Technology Co.,Ltd. Address before: 100871 No. 5, the Summer Palace Road, Beijing, Haidian District Patentee before: Peking University |