CN114240802A - Visual perception method and system based on biological neuron network and stochastic resonance - Google Patents
Visual perception method and system based on biological neuron network and stochastic resonance Download PDFInfo
- Publication number
- CN114240802A CN114240802A CN202111605009.1A CN202111605009A CN114240802A CN 114240802 A CN114240802 A CN 114240802A CN 202111605009 A CN202111605009 A CN 202111605009A CN 114240802 A CN114240802 A CN 114240802A
- Authority
- CN
- China
- Prior art keywords
- image
- information matrix
- enhanced
- hsv
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000002569 neuron Anatomy 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 73
- 230000016776 visual perception Effects 0.000 title claims abstract description 35
- 230000008569 process Effects 0.000 claims abstract description 27
- 239000011159 matrix material Substances 0.000 claims description 103
- 239000012528 membrane Substances 0.000 claims description 29
- 238000005286 illumination Methods 0.000 claims description 16
- 230000008447 perception Effects 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 8
- 230000001965 increasing effect Effects 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 7
- 230000036982 action potential Effects 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 3
- 210000005036 nerve Anatomy 0.000 claims description 3
- 239000004576 sand Substances 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000001537 neural effect Effects 0.000 claims description 2
- 238000004422 calculation algorithm Methods 0.000 abstract description 19
- 210000004027 cell Anatomy 0.000 abstract description 10
- 230000000007 visual effect Effects 0.000 abstract description 8
- 210000001525 retina Anatomy 0.000 abstract description 5
- 230000000946 synaptic effect Effects 0.000 abstract description 4
- 230000002079 cooperative effect Effects 0.000 abstract description 2
- 230000015572 biosynthetic process Effects 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000010304 firing Methods 0.000 description 3
- 230000004297 night vision Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000000857 visual cortex Anatomy 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011953 bioanalysis Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 210000001153 interneuron Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000002856 peripheral neuron Anatomy 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Neurology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a visual perception method and a system based on a biological neuron network and stochastic resonance, which combine the stochastic resonance principle of an integral discharge neuron network of synaptic conductance and a basic biophysical process of visual formation. Two types of nerve cells are known to be distributed predominantly on the retina: rod and cone cells; the rods are mainly responsible for distinguishing the approximate outline of the object but cannot distinguish the color, while the cones are sensitive to light and have high color distinguishing capability. In order to enhance the contrast of a color image, a biological neuron network is used for simulating the cooperative action of rod cell clusters in the visual perception process, and therefore, a novel color image enhancement method is developed. The invention can obviously improve the contrast and brightness of the image, obviously enhance the details of the dark area, obviously improve the edge information and achieve better visual effect; compared with the classical single-scale Retinex method and the HE algorithm, the method also shows obvious superiority.
Description
Technical Field
The invention belongs to the technical field of image enhancement, and particularly relates to a visual perception method and system based on a biological neuron network and stochastic resonance.
Background
The visual perception technology or image enhancement technology is a hot research subject in the current image processing field, and is widely applied in the engineering fields related to weak signal detection, such as military night vision, road traffic, surveillance video, brain-computer interfaces and the like.
Most of the traditional image enhancement methods mainly use 'denoising', but under strong background noise, some useful information of a weak image is weakened in the denoising process, so that the image enhancement algorithm aiming at denoising usually has inevitable defects. The stochastic resonance principle provides a new idea of highlighting weak image information characteristics by using noise, and breaks through the inherent concept that image enhancement can only be performed by eliminating noise.
However, the existing stochastic resonance algorithm for image enhancement mainly considers the enhancement of gray scale images, and most of the existing algorithms lack details for algorithm implementation and have insufficient bioanalysis. For example, a visual perception algorithm based on a simple threshold model or an over-damping bistable model does not give key parameters such as selection details of a threshold and evaluation indexes of an optimal target image; the image enhancement algorithm based on the combination of singular value decomposition and the over-damped bistable model also lacks key details, while the stochastic resonance algorithm based on the total variation regularization and the over-damped bistable model is inconvenient to use and understand due to the excessively complex algorithm structure.
Disclosure of Invention
The invention aims to solve the technical problem of providing a color image sensing method and system based on biological neuron network and stochastic resonance aiming at the defects in the prior art, wherein the stochastic resonance principle in the integral discharge neuron network of synaptic conductance is combined with the physiological background of human eye imaging, and the low-illumination color image is enhanced by adjusting the noise intensity, so that the visual sensing algorithm of the color image with better biological interpretability is provided.
The invention adopts the following technical scheme:
a visual perception method based on biological neuron networks and stochastic resonance comprises the following steps:
s1, converting the low-illumination color image from RGB color space to HSV color space through nonlinear transformation, and recording hue information matrix, saturation information matrix and brightness information matrix corresponding to three channels of the HSV color space as HM×N,SM×NAnd VM×NM and N respectively represent the number of rows and columns of the image, and an output brightness information matrix VM×NTone information matrix HM×NAnd saturation information matrix SM×N；
S2, setting Gaussian white noise intensity D ═ DjThe luminance information matrix V of step S1M×NInputting a global feedback network composed of N integral discharge neuron models, generating an action potential matrix (Index) at the moment when the membrane potential of each neuron reaches a threshold valuei)M×NStoring information of whether the ith neuron discharges in the coding process as output;
s3, obtaining (Index) in step S2i)M×NDecoding of the matrix information into a corresponding binary luminance information matrix PixM×N(ii) a Further integrating the luminance decoding information of all the neurons to generate an enhanced luminance information matrix PixM×NThen, the tone information matrix H generated in step S1 is usedM×NSaturation information matrix SM×NAnd an enhanced luminance information matrix PixM×NFused enhanced HSV space image ImgHSV；
S4, enhancing the HSV space image Img obtained in the step S3HSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBCalculating and outputting PQM (Img) of the imageRGB) Obtaining an enhanced image as an evaluation index of the enhanced image;
s5, increasing the Gaussian white noise intensity D in the step S2jAnd (4) repeating the steps S2-S4, and completing the visual perception task if obtaining the enhanced image closest to the optimal value 10 of the perception metric index.
Specifically, step S1, a low-illumination color image I with the size of 3 XMxNRGBThe conversion from the RGB color space to the HSV color space is specifically:
where h is hue, s is saturation, v is brightness, (r, g, b) are the red, green and blue coordinates of a color, respectively, max is the maximum of r, g, b, and min is the minimum of r, g, b.
Specifically, in step S2, the membrane potential of the neuron model is calculated as follows:
s201, inputting a brightness information matrix HM×NSetting the membrane potential evolution time to [0,1]And the iteration step size is delta t is 0.01, and the membrane potential is initialized
And a feedback term fm,n(0) 0, and setting the upper 10% quantile of H (S) matrix element distribution histogram as threshold value Vth；
S202, calculating the membrane potential at the next moment according to the initial membrane potential and the threshold value set in the step S201, if so, calculating the membrane potential at the next moment
Recording the discharge time at this time
Resetting the membrane potential
Memo
Indicates that the ith neuron is [0,1 ]]Discharge occurs within a time period;
s203, utilizing the screening property of the Dirac delta function, and combining the global feedback term and the discharge moment vector of the neuron network
Computing a global feedback term fm,n(t) updating the feedback term;
s204, and loop calculation steps S202 to S203, until the termination condition t is reached to 1, discharge information of all neurons is obtained.
Further, in step S202, the film potential at the next time point
The calculation is as follows:
wherein i is more than or equal to 1 and less than or equal to N, delta t is time step length, fm,nAnd (t) is a feedback item, P (m, n) is an image brightness information matrix, D is noise intensity, and randn is a Gaussian random number.
Further, in step S203, the global feedback term fm,nThe (t) is specifically:
wherein G is the global feedback strength,
the k discharge time when the image brightness matrix passes through the ith neuron, t is the current time, and tauSAnd τDAre parameters of the convolution kernel.
Specifically, in step S3, the enhanced luminance information matrix PixM×NThe method specifically comprises the following steps:
wherein, PixiAnd (m, N) is an element of an m-th row and N columns in the binary brightness information matrix, and N is the number of image columns.
Further, a binary luminance information matrix Pixi(m, n) are specifically:
therein, Indexi(m, n) is discharge index information.
Specifically, in step S4, the enhanced HSV spatial image Img is processed by the following equationHSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBThe method specifically comprises the following steps:
where V is the image luminance information matrix, hiQ, t, p are intermediate variables, respectively.
Specifically, in step S5, the process is repeated j times, and the corresponding enhanced image variance PQM is recordedjWhen the k-th test satisfies | PQMk-2-10|>|PQMk-1-10|>|PQMkWhen the image is 10|, stopping the circulation, and outputting the k-2 th experimental enhanced image as an optimal output image;
when the preset maximum noise intensity is reached, the loop is stopped, and the enhanced image with the PQM value closest to 10 is output as the optimal output image.
Another technical solution of the present invention is a visual perception system based on a biological neuron network and stochastic resonance, comprising:
the conversion module converts the low-illumination color image from the RGB color space to the HSV color space through nonlinear conversion and converts the hue information matrix and saturation corresponding to three channels of the HSV color spaceThe sum degree information matrix and the brightness information matrix are respectively marked as HM×N,SM×NAnd VM×NM and N respectively represent the number of rows and columns of the image, and an output brightness information matrix VM×NTone information matrix HM×NAnd saturation information matrix SM×N；
A feedback module for setting the Gaussian white noise intensity D ═ DjThe luminance information matrix V of step S1M×NInputting a global feedback network composed of N integral discharge neuron models, generating an action potential matrix (Index) at the moment when the membrane potential of each neuron reaches a threshold valuei)M×NStoring information of whether the ith neuron discharges in the coding process as output;
a fusion module for obtaining (Index) from the feedback modulei)M×NDecoding of the matrix information into a corresponding binary luminance information matrix PixM×N(ii) a Further integrating the luminance decoding information of all neurons to generate enhanced luminance information PixM×NThen, the tone information matrix H generated in step S1 is usedM×NSaturation information matrix SM×NAnd enhanced luminance information PixM×NFused enhanced HSV space image ImgHSV；
An index module for enhancing the HSV space image Img after the fusion moduleHSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBCalculating and outputting PQM (Img) of the imageRGB) Obtaining an enhanced image as an evaluation index of the enhanced image;
a sensing module for increasing the Gaussian white noise intensity D in the feedback modulejAnd repeating the feedback module, the fusion module and the index module, and finishing the visual perception task if obtaining the enhanced image closest to the optimal value 10 of the perception metric index.
Compared with the prior art, the invention has at least the following beneficial effects:
a vision perception method based on biological neuron network and stochastic resonance, combine the biophysical process formed with vision of the stochastic resonance principle based on integral discharge neuron model of the synaptic conductance, utilize and employ the perception process that the integral discharge neuron network mainly simulates the cone cell, utilize the noise but eliminate the noise based on the stochastic resonance principle, it has interpretability of the biological perception process, and clear quantization index, easy to screen the optimum enhancement image automatically, have broad application prospects in the fields such as military night vision, road traffic, monitoring video, brain-computer interface, etc.; the steps are corresponding to the visual perception process of human eyes, and the method has better biological interpretability.
Further, the stochastic resonance principle is mainly used for enhancing weak signals, and the illumination of low-illumination images is the weak signals which need to be enhanced. If the RGB space is not converted into the HSV space, R, G, B three channels need to be enhanced respectively, and the calculation amount is greatly increased.
Further, whether the neuron discharges or not is recorded through membrane potential, so that the image brightness information matrix is enhanced according to the discharge information.
Further, the global feedback represents the influence of other neurons on the ith neuron, and the visual perception process of the human eye is simulated more truly.
Further, PixM×NThe matrix is an indirect manifestation of the image enhancement effect, PixM×NEach pixel Pix of the matrixi(m, n) is assigned as 1 or 0 depending on the firing of the ith neuron.
Further, the enhanced HSV space image ImgHSVAnd converting the image into a color image in an RGB space, thereby obtaining a final enhanced image.
Further, using PQM as an image quality evaluation index, the closer the PQM value is to 10, the better the image enhancement effect. In repeated j experiments, the perception metric index PQM corresponding to the enhanced image is recordedjWhen the k-th test satisfies Vark-2>Vark-1>VarkAnd stopping the circulation, and outputting the k-2 th test enhanced image as an optimal output image.
In conclusion, the invention uses the biological visual perception process for reference, has the advantages of biological interpretability, simple operation flow and easy realization.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
FIG. 1 is a flow chart of the inventive method architecture;
FIG. 2 is an experimental test chart of the present invention, (a) is an original low-illumination color image, (b) is an image enhanced by the method of the present invention, (c) is a second original low-illumination color image, and (d) is an image enhanced by the method of the present invention;
FIG. 3 is an experimental comparison diagram of the present invention, (a) is an original low-illumination color image, (b) is an image enhanced by the algorithm of the present invention, (c) is an image enhanced by Retinex, and (d) is an image enhanced by HE;
fig. 4 shows another set of experimental contrast charts of the present invention, in which (a) is the original low-luminance image, (b) is the enhanced image corresponding to the present invention, (c) is the image enhanced by Retinex algorithm, and (d) is the image enhanced by HE algorithm.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be understood that the terms "comprises" and/or "comprising" indicate the presence of the stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Various structural schematics according to the disclosed embodiments of the invention are shown in the drawings. The figures are not drawn to scale, wherein certain details are exaggerated and possibly omitted for clarity of presentation. The shapes of various regions, layers and their relative sizes and positional relationships shown in the drawings are merely exemplary, and deviations may occur in practice due to manufacturing tolerances or technical limitations, and a person skilled in the art may additionally design regions/layers having different shapes, sizes, relative positions, according to actual needs.
The invention provides a visual perception method based on a biological neuron network and a stochastic resonance principle, which mainly simulates the basic physiological process of visual perception of cone cells in a dark environment by utilizing an integral discharge neuron network, combines the stochastic resonance phenomenon of an integral discharge neuron model based on synaptic conductance and the basic biophysical process of visual enhancement by utilizing noise instead of eliminating the noise through the stochastic resonance principle, and gives a biological explanation of the visual perception to a certain extent. The method has clear quantitative indexes, is convenient for automatically screening the optimal enhanced image, and has wide application prospects in the fields of military night vision, road traffic, surveillance video, brain-computer interfaces and the like.
Referring to fig. 1, the present invention provides a visual perception method based on a biological neuron network and stochastic resonance, including the following steps:
s1, image input
Inputting a low-illumination color image, converting the image from an RGB (red, green and blue) color space to an HSV (hue, saturation and brightness) color space through nonlinear transformation, and respectively recording a hue information matrix, a saturation information matrix and a brightness information matrix corresponding to three channels of the HSV as HM×N,SM×NAnd VM×NWhere M and N represent the number of rows and columns of the image, respectively, and an output luminance information matrix VM×NIn step S2, the hue information matrix and the saturation information matrix H are outputM×N,SM×NGo to step S3;
firstly, a low-illumination color image I with the size of 3 XMXN is inputRGBThen, the image is converted from RGB color space to HSV color space by formula (1), so as to obtain new image I whose HSV color space size is 3 XMXNHSVThree of which represent in turn a matrix H of tone information of the imageM×NSaturation information matrix SM×NAnd a luminance information matrix VM×N。
Wherein (r, g, b) are red, green and blue coordinates of a color, respectively, their values are real numbers between 0 and 1, max is the maximum value of r, g, b, and min is the minimum value of r, g, b.
S2, image coding process
When the image signal enters the eye, the retina first converts the image light signal of the HSV space into electrical pulses that can be transmitted through the interneurons. There are two photoreceptors in the retina: rod cells and cone cells, while the reception and processing of color images at low illumination levels is primarily responsible for cone cells. When the optical signal enters the retina through human eyes, the cone cells respectively process the saturation information and the brightness information. Considering that the cause of the low-contrast photo is low brightness, the invention processes the brightness information, and the general process of the integrated discharge neuron network analog cone cell perception process is given below.
Considering a global feedback network consisting of N integral discharge neuron models, the evolution equation of the membrane potential of each neuron model is:
wherein the subscript i corresponds to the ith neuron, and the subscript m, n corresponds to the elements of the mth row n column of the luminance information matrix of the image;
represents the membrane potential of the neuron when
Reach the threshold value V from belowthWhen the nerve is in operation, the ith neuron immediately sends out nerve pulse, and the action potential sending time is recorded as
The corresponding neuron then discharges a string of
Represents white Gaussian noise and satisfies
To describe the noise input to the retinal neural network by peripheral neurons, D being the adjustable noise intensity;
the global feedback term for the neural network is defined as the convolution of the alpha delay function with the neural network firing string:
where G is the global feedback strength, τSAnd τDParameters of a convolution kernel; p (m, n) ∈ (0,1) represents input image information, taken as the luminance component value V (m, n) in step S1; order matrix (Index)i)M×NThe firing information of the ith neuron in this encoding process is stored as the output of step S2.
Setting Gaussian white noise intensity D ═ DjThe luminance information matrix H (m, n) (saturation information matrix S (m, n)) of the image is input to equation (2) and the membrane potential is calculated by the Euler-Maruyama numerical scheme:
the membrane potential of the neuron was calculated as follows:
s201, inputting brightness information matrix H of imageM×N(saturation information matrix SM×N) Instead of P (m, n) in equation (4), the membrane potential evolution time is set to [0,1 ]]The iteration step size is Δ t 0.01. Initialising a Membrane potential
And a feedback term fm,n(0) 0, and setting the upper 10% quantile of H (S) matrix element distribution histogram as threshold value Vth；
S202, calculating the membrane potential at the next moment through a formula (4), wherein randn represents a standard normal random number. If it is
The discharge time at that time is recorded
Resetting the membrane potential
Memo
Indicates that the ith neuron is [0,1 ]]Discharge occurs within a time period;
s203, advantageScreening property of Dirac delta function is used, and formula (3) and discharge moment vector are combined
Computing global feedback terms
S204, and loop calculation steps S202 to S203 until the termination condition t is reached to 1, and finally, discharge information of all neurons is obtained.
S3, image decoding and integrating process
Visual cortex cells first resolve a matrix of luminance information transmitted via the retina into a binary image: because the carrier of the neural information transmission is an electric pulse, the encoded information adopts a spike sequence instead of the membrane potential information of the neuron.
Accordingly, (Index) in step S2i)M×NDecoding of the matrix information into a corresponding binary luminance information matrix Pixi(m,n)
The visual cortex as the command center further integrates the luminance decoding information of all the neurons, and the integration mode is set as linear superposition, namely:
the integration process embodies the cooperative effect of a large number of neurons in accomplishing the same perceptual function: if each binary image is considered as the output of a weak learning machine, the weak learning machines are integrated to form a strong learning machine, enhanced image information is generated, and the color information (H) of the image is further processedM×N,SM×N) And a processed luminance information matrix (Pix)M×N) Are fused togetherThe enhanced image Img of the HSV space can be obtainedHSVAnd will enhance the image ImgHSVAs an output of step S3;
s4 image space conversion and evaluation
The enhanced HSV space image ImgHSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBCalculating and outputting an evaluation index of the enhanced image, namely a perception metric index PQM (Img) of the imageRGB)。
The RGB color space is:
wherein the content of the first and second substances,
q＝V×(1-f×S)，
p ═ V × (1-S), t ═ V × (1- (1-f) × S), and the perceptual metric index (PQM) of the enhanced image is calculated, denoted as PQMj。
S5 selection of optimal output image
Increasing the noise intensity D in step S2jTaking values, repeating the steps S2-S4, j represents the j-th repetition, recording the perception metric index (PQM) of the corresponding enhanced image, and when the k-th experiment meets the condition of | PQMk-2-10|>|PQMk-1-10|>|PQMkWhen the image is minus 10|, the enhancement effect is shown to generate a descending trend, the circulation is stopped, and the k-2 th experimental enhancement image is output as an optimal output image; or when the maximum noise intensity reaches the preset maximum noise intensity, the loop is stopped, and the enhanced image with the PQM value closest to 10 is output as the optimal output image.
In another embodiment of the present invention, a visual perception system based on a biological neuron network and stochastic resonance is provided, where the system can be used to implement the above visual perception method based on a biological neuron network and stochastic resonance, and specifically, the visual perception system based on a biological neuron network and stochastic resonance includes a conversion module, a feedback module, a fusion module, an index module, and a perception module.
The conversion module converts the low-illumination color image from an RGB color space to an HSV color space through nonlinear conversion, and marks a hue information matrix, a saturation information matrix and a brightness information matrix corresponding to three channels of the HSV color space as H respectivelyM×N,SM×NAnd VM×NM and N respectively represent the number of rows and columns of the image, and an output brightness information matrix VM×NTone information matrix HM×NAnd saturation information matrix SM×N；
A feedback module for setting the Gaussian white noise intensity D ═ DjThe luminance information matrix V of step S1M×NInputting a global feedback network composed of N integral discharge neuron models, generating an action potential matrix (Index) at the moment when the membrane potential of each neuron reaches a threshold valuei)M×NStoring information of whether the ith neuron discharges in the coding process as output;
a fusion module for obtaining (Index) from the feedback modulei)M×NDecoding of the matrix information into a corresponding binary luminance information matrix PixM×N(ii) a Further integrating the luminance decoding information of all neurons to generate enhanced luminance information PixM×NThen, the tone information matrix H generated in step S1 is usedM×NSaturation information matrix SM×NAnd enhanced luminance information PixM×NFused enhanced HSV space image ImgHSV；
An index module for enhancing the HSV space image Img after the fusion moduleHSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBCalculating and outputting PQM (Img) of the imageRGB) Obtaining an enhanced image as an evaluation index of the enhanced image;
a sensing module for increasing the Gaussian white noise intensity D in the feedback modulejRepeating the feedback module, the fusion module and the index module, and finishing the process if obtaining the enhanced image closest to the optimal value 10 of the perception metric indexBecomes a visual perception task.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention is further illustrated by the following examples and figures.
1) The experimental conditions are as follows:
the environment used for the experiment is Intel Core i7-6700U @3.40GHz dual-Core CPU, the internal memory is 8GB, and the programming environment platform is Matlab R2016 a. The test images used for the experiment were four test images 1,2,3,4 in size.
2) The experimental contents are as follows:
the method provided by the invention is used for processing the image and comparing the processed image with the conventional single-scale Retinex algorithm and Histogram Equalization (HE) method.
(1) PQM, an index for evaluating image quality based on the human visual system. The closer the PQM is to 10, the better the image quality.
(2) Information entropy, which is used to measure the richness of image information. The larger the information entropy, the more abundant the image information.
(3) CEF, which is a measure of the contrast enhancement performance of the algorithm. The larger the CEF, the better the performance of image enhancement.
TABLE 1 comparison of the indices of the process of the invention with those of the other two processes
A comparison of the individual properties of the three methods is given in table 1. Experimental results show that all indexes of the stochastic resonance-based image enhancement algorithm are superior to those of other three algorithms, the image contrast, brightness, details and the like are considered, and the image visual effect is good.
Referring to FIG. 2, a comparison of the results of three image enhancement methods is given; as can be seen from fig. 2, the method of the present invention is superior to the other three methods in terms of contrast, chromaticity, and the like.
Referring to fig. 3 and 4, a comparison of the results of three image enhancement methods is given; experimental results show that when the single-scale Retinex method is used for processing images, the images are successfully compressed, the color sense is good, details of a partial dark area are not clear enough, the HE method can cause partial distortion of the images, results of the random resonance-based image enhancement algorithm disclosed by the text show that the brightness of the images is greatly improved compared with other two methods, and the details of the dark area are also obviously enhanced. The edge information is obvious, the color impression is better, and the visual effect is greatly improved.
In conclusion, the visual perception method and system based on the biological neuron network and the stochastic resonance can remarkably improve the contrast and brightness of an image, remarkably enhance the details of a dark area, remarkably improve edge information, achieve a better visual effect, and have remarkable advantages compared with a classical single-scale Retinex method and an HE algorithm.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (10)
1. The visual perception method based on the biological neuron network and the stochastic resonance is characterized by comprising the following steps of:
s1, converting the low-illumination color image from RGB color space to HSV color space through nonlinear transformation, and recording hue information matrix, saturation information matrix and brightness information matrix corresponding to three channels of the HSV color space as HM×N,SM×NAnd VM×NM and N respectively represent the number of rows and columns of the image, and an output brightness information matrix VM×NTone information matrix HM×NAnd saturation information matrix SM×N；
S2, setting Gaussian white noise intensity D ═ DjThe luminance information matrix V of step S1M×NInputting a global feedback network composed of N integral discharge neuron models, generating an action potential matrix (Index) at the moment when the membrane potential of each neuron reaches a threshold valuei)M×NStoring information of whether the ith neuron discharges in the coding process as output;
s3, obtaining (Index) in step S2i)M×NDecoding of the matrix information into a corresponding binary luminance information matrix PixM×N(ii) a Further integrating the luminance decoding information of all the neurons to generate an enhanced luminance information matrix PixM×NThen, the tone information matrix H generated in step S1 is usedM×NSaturation information matrix SM×NAnd an enhanced luminance information matrix PixM×NFused enhanced HSV space image ImgHSV；
S4, enhancing the HSV space image Img obtained in the step S3HSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBCalculating and outputting PQM (Img) of the imageRGB) Obtaining an enhanced image as an evaluation index of the enhanced image;
s5, increasing the Gaussian white noise intensity D in the step S2jAnd (4) repeating the steps S2-S4, and completing the visual perception task if obtaining the enhanced image closest to the optimal value 10 of the perception metric index.
2. The visual perception method based on biological neuron networks and stochastic resonance according to claim 1, wherein the step S1, a low-illumination color image I with the size of 3 XMxNRGBThe conversion from the RGB color space to the HSV color space is specifically:
where h is hue, s is saturation, v is brightness, (r, g, b) are the red, green and blue coordinates of a color, respectively, max is the maximum of r, g, b, and min is the minimum of r, g, b.
3. The visual perception method based on biological neuron networks and stochastic resonance according to claim 1, wherein in step S2, the membrane potential of the neuron model is calculated as follows:
s201, inputting a brightness information matrix HM×NSetting the membrane potential evolution time to [0,1]The iteration step is delta t is 0.01, and the membrane potential V is initializedi m,n(0) 0 and the feedback term fm,n(0) 0, and setting the upper 10% quantile of H (S) matrix element distribution histogram as threshold value Vth；
S202, calculating the membrane potential at the next moment according to the initial membrane potential and the threshold value set in the step S201, and if V is greater than Vi m,n(t+Δt)>VthRecording the discharge time T at this timei m,n＝[Ti m,n,t+Δt]Resetting the membrane potential Vi m,n(t+Δt)＝V0To and from
Indicates that the ith neuron is [0,1 ]]Discharge occurs within a time period;
s203, utilizing the screening property of the Dirac delta function, and combining the global feedback term and the discharge time vector T of the neuron networki m,nComputing a global feedback term fm,n(t) updating the feedback term;
s204, and loop calculation steps S202 to S203, until the termination condition t is reached to 1, discharge information of all neurons is obtained.
4. The visual perception method based on biological neuron networks and stochastic resonance as claimed in claim 3, wherein in step S202, the membrane potential V at the next momenti m,n(t + Δ t) is calculated as follows:
wherein i is more than or equal to 1 and less than or equal to N, delta t is time step length, fm,nAnd (t) is a feedback item, P (m, n) is an image brightness information matrix, D is noise intensity, and randn is a Gaussian random number.
5. The visual perception method based on biological neuron networks and stochastic resonance according to claim 3, wherein in step S203, the global feedback term fm,nThe (t) is specifically:
wherein G is the global feedback strength,
the k discharge time when the image brightness matrix passes through the ith neuron, t is the current time, and tauSAnd τDAre parameters of the convolution kernel.
6. The biological nerve-based according to claim 1The visual perception method of meta-network and stochastic resonance is characterized in that in step S3, the enhanced luminance information matrix PixM×NThe method specifically comprises the following steps:
wherein, PixiAnd (m, N) is an element of an m-th row and N columns in the binary brightness information matrix, and N is the number of image columns.
7. The visual perception method based on biological neuron networks and stochastic resonance as claimed in claim 6, wherein the binary luminance information matrix Pixi(m, n) are specifically:
therein, Indexi(m, n) is discharge index information.
8. The visual perception method based on biological neuron networks and stochastic resonance according to claim 1, wherein in step S4, the enhanced HSV spatial image Img is processed according to the following formulaHSVEnhanced color image Img obtained by nonlinear transformation from HSV space to RGB spaceRGBThe method specifically comprises the following steps:
where V is the image luminance information matrix, hiQ, t, p are intermediate variables, respectively.
9. The visual perception method based on biological neuron networks and stochastic resonance as claimed in claim 1, wherein in step S5, repeating j times, and recording corresponding enhanced image variance PQMjWhen the k-th test satisfies | PQMk-2-10|>|PQMk-1-10|>|PQMkWhen the image is 10|, stopping the circulation, and outputting the k-2 th experimental enhanced image as an optimal output image;
when the preset maximum noise intensity is reached, the loop is stopped, and the enhanced image with the PQM value closest to 10 is output as the optimal output image.
10. A visual perception system based on a biological neuronal network and stochastic resonance, comprising:
the conversion module converts the low-illumination color image from an RGB color space to an HSV color space through nonlinear conversion, and marks a hue information matrix, a saturation information matrix and a brightness information matrix corresponding to three channels of the HSV color space as H respectivelyM×N,SM×NAnd VM×NM and N respectively represent the number of rows and columns of the image, and an output brightness information matrix VM×NTone information matrix HM×NAnd saturation information matrix SM×N；
A feedback module for setting the Gaussian white noise intensity D ═ DjThe luminance information matrix V of step S1M×NInputting a global feedback network composed of N integral discharge neuron models, generating an action potential matrix (Index) at the moment when the membrane potential of each neuron reaches a threshold valuei)M×NStoring information of whether the ith neuron discharges in the coding process as output;
a fusion module for obtaining (Index) from the feedback modulei)M×NDecoding of the matrix information into a corresponding binary luminance information matrix PixM×N(ii) a Further integrating the luminance decoding information of all neurons to generate enhanced luminance information PixM×NThen, the tone information matrix H generated in step S1 is usedM×NSaturation information matrix SM×NAnd enhanced luminance information PixM×NFused enhanced HSV space image ImgHSV；
An index module for enhancing the HSV space image Img after the fusion moduleHSVEnhanced color image obtained by nonlinear transformation from HSV space to RGB spaceImgRGBCalculating and outputting PQM (Img) of the imageRGB) Obtaining an enhanced image as an evaluation index of the enhanced image;
a sensing module for increasing the Gaussian white noise intensity D in the feedback modulejAnd repeating the feedback module, the fusion module and the index module, and finishing the visual perception task if obtaining the enhanced image closest to the optimal value 10 of the perception metric index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111605009.1A CN114240802B (en) | 2021-12-24 | 2021-12-24 | Visual perception method and system based on biological neuron network and stochastic resonance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111605009.1A CN114240802B (en) | 2021-12-24 | 2021-12-24 | Visual perception method and system based on biological neuron network and stochastic resonance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114240802A true CN114240802A (en) | 2022-03-25 |
| CN114240802B CN114240802B (en) | 2023-08-01 |
Family
ID=80762902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111605009.1A Active CN114240802B (en) | 2021-12-24 | 2021-12-24 | Visual perception method and system based on biological neuron network and stochastic resonance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114240802B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117709419A (en) * | 2022-10-09 | 2024-03-15 | 航天科工集团智能科技研究院有限公司 | Pulse neural network training method, recognition system building method and recognition system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013139067A1 (en) * | 2012-03-22 | 2013-09-26 | Hou Kejie | Method and system for carrying out visual stereo perception enhancement on color digital image |
| CN105046663A (en) * | 2015-07-10 | 2015-11-11 | 西南科技大学 | Human visual perception simulation-based self-adaptive low-illumination image enhancement method |
| CN106504212A (en) * | 2016-11-07 | 2017-03-15 | 湖南源信光电科技有限公司 | A kind of improved HSI spatial informations low-luminance color algorithm for image enhancement |
| CN113158746A (en) * | 2021-02-02 | 2021-07-23 | 杭州电子科技大学 | Weak signal sensing method based on neuron small-world network stochastic resonance |
-
2021
- 2021-12-24 CN CN202111605009.1A patent/CN114240802B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013139067A1 (en) * | 2012-03-22 | 2013-09-26 | Hou Kejie | Method and system for carrying out visual stereo perception enhancement on color digital image |
| CN105046663A (en) * | 2015-07-10 | 2015-11-11 | 西南科技大学 | Human visual perception simulation-based self-adaptive low-illumination image enhancement method |
| CN106504212A (en) * | 2016-11-07 | 2017-03-15 | 湖南源信光电科技有限公司 | A kind of improved HSI spatial informations low-luminance color algorithm for image enhancement |
| CN113158746A (en) * | 2021-02-02 | 2021-07-23 | 杭州电子科技大学 | Weak signal sensing method based on neuron small-world network stochastic resonance |
Non-Patent Citations (1)
| Title |
|---|
| 毛伟民;赵勋杰;: "基于神经网络的低照度彩色图像增强算法", 光学技术, no. 02 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117709419A (en) * | 2022-10-09 | 2024-03-15 | 航天科工集团智能科技研究院有限公司 | Pulse neural network training method, recognition system building method and recognition system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114240802B (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111709902B (en) | Infrared and visible light image fusion method based on self-attention mechanism | |
| Atoum et al. | Color-wise attention network for low-light image enhancement | |
| Varga et al. | Fully automatic image colorization based on Convolutional Neural Network | |
| Shi et al. | Low-light image enhancement algorithm based on retinex and generative adversarial network | |
| CN110738160A (en) | human face quality evaluation method combining with human face detection | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN111986132A (en) | Infrared and visible light image fusion method based on DLatLRR and VGG & Net | |
| Zhang et al. | Dual-channel multi-task CNN for no-reference screen content image quality assessment | |
| CN110059593A (en) | A kind of human facial expression recognition method based on feedback convolutional neural networks | |
| CN111882516B (en) | Image quality evaluation method based on visual saliency and deep neural network | |
| Yan et al. | Attention-guided dynamic multi-branch neural network for underwater image enhancement | |
| CN114240802B (en) | Visual perception method and system based on biological neuron network and stochastic resonance | |
| Li et al. | Adaptive weighted multiscale retinex for underwater image enhancement | |
| Goel et al. | Gray level enhancement to emphasize less dynamic region within image using genetic algorithm | |
| Liu et al. | WSDS-GAN: A weak-strong dual supervised learning method for underwater image enhancement | |
| CN112541566B (en) | Image translation method based on reconstruction loss | |
| Tu et al. | DRPAN: a novel adversarial network approach for retinal vessel segmentation | |
| CN113810683A (en) | No-reference evaluation method for objectively evaluating underwater video quality | |
| CN110610508B (en) | Static video analysis method and system | |
| Agrawal et al. | Exploring convolutional neural networks for automatic image colorization | |
| CN116309213A (en) | High-real-time multi-source image fusion method based on generation countermeasure network | |
| JP7362924B2 (en) | Data augmentation-based spatial analysis model learning device and method | |
| CN112270220B (en) | Sewing gesture recognition method based on deep learning | |
| Park et al. | Underwater image enhancement using adaptive standardization and normalization networks | |
| Li et al. | Multi-scale fusion framework via retinex and transmittance optimization for underwater image enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |