CN114240802B - Visual perception method and system based on biological neuron network and stochastic resonance - Google Patents
Visual perception method and system based on biological neuron network and stochastic resonance Download PDFInfo
- Publication number
- CN114240802B CN114240802B CN202111605009.1A CN202111605009A CN114240802B CN 114240802 B CN114240802 B CN 114240802B CN 202111605009 A CN202111605009 A CN 202111605009A CN 114240802 B CN114240802 B CN 114240802B
- Authority
- CN
- China
- Prior art keywords
- image
- information matrix
- enhanced
- hsv
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 210000002569 neuron Anatomy 0.000 title claims abstract description 75
- 230000016776 visual perception Effects 0.000 title claims abstract description 37
- 230000008569 process Effects 0.000 claims abstract description 21
- 239000011159 matrix material Substances 0.000 claims description 103
- 239000012528 membrane Substances 0.000 claims description 25
- 230000004927 fusion Effects 0.000 claims description 16
- 230000008447 perception Effects 0.000 claims description 15
- 230000009466 transformation Effects 0.000 claims description 14
- 238000005286 illumination Methods 0.000 claims description 11
- 238000011156 evaluation Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 230000001965 increasing effect Effects 0.000 claims description 7
- 238000005259 measurement Methods 0.000 claims description 7
- 230000036982 action potential Effects 0.000 claims description 5
- 238000007599 discharging Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 3
- 230000001537 neural effect Effects 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 abstract description 19
- 210000004027 cell Anatomy 0.000 abstract description 13
- 230000000007 visual effect Effects 0.000 abstract description 7
- 210000001525 retina Anatomy 0.000 abstract description 5
- 230000000946 synaptic effect Effects 0.000 abstract description 4
- 230000002079 cooperative effect Effects 0.000 abstract description 2
- 238000013529 biological neural network Methods 0.000 abstract 1
- 230000015572 biosynthetic process Effects 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000010304 firing Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000004297 night vision Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 210000000857 visual cortex Anatomy 0.000 description 2
- 229910009207 xMxN Inorganic materials 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 210000001153 interneuron Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002856 peripheral neuron Anatomy 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Neurology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a visual perception method and a visual perception system based on a biological neuron network and stochastic resonance, which combine the stochastic resonance principle of an integral discharge neuron network of synaptic conductivity and a basic biophysical process of visual formation. Two types of nerve cells are known to be predominantly distributed on the retina: rod cells and cone cells; the rod cells are mainly responsible for distinguishing the general outline of the object but not the color, while the cone cells are more sensitive to light and have higher color distinguishing capability. In order to enhance the contrast of the color image, we use a biological neural network to simulate the cooperative action of the video rod cell clusters in the visual perception process, and thus develop a new color image enhancement method. The invention can obviously improve the contrast and brightness of the image, obviously enhance the details of the dark area, obviously improve the edge information and achieve better visual effect; the method also shows obvious superiority compared with the classical single-scale Retinex method and HE algorithm.
Description
Technical Field
The invention belongs to the technical field of image enhancement, and particularly relates to a visual perception method and system based on a biological neuron network and stochastic resonance.
Background
The visual perception technology or the image enhancement technology is a hot research subject in the current image processing field, and is widely applied to engineering fields related to weak signal detection, such as military night vision, road traffic, monitoring video, brain-computer interfaces and the like.
The traditional image enhancement method mainly uses denoising, but under the condition of strong background noise, some useful information of a weak image is weakened in the noise reduction process, so that an image enhancement algorithm aiming at noise reduction often has unavoidable defects. The stochastic resonance principle provides a new idea of using noise to highlight weak image information characteristics, and breaks through the inherent idea that image enhancement can only be performed by eliminating noise.
However, the existing stochastic resonance algorithm for image enhancement mainly considers enhancement of gray scale images, and the existing algorithm mostly lacks details of algorithm implementation and has insufficient biological interpretability. For example, the visual perception algorithm based on a simple threshold model or an over-damped bistable model does not give out the selection details of key parameters such as a threshold value and the evaluation indexes of an optimal target image; the image enhancement algorithm based on the combination of singular value decomposition and an over-damped bistable model also lacks key details, and the stochastic resonance type algorithm based on the total variation regularization and the over-damped bistable model is inconvenient to use and understand because the algorithm structure is too complex.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a color image sensing method and a color image sensing system based on a biological neuron network and stochastic resonance, which combine the stochastic resonance principle in an integral discharge neuron network of synaptic conductivity with the physiological background of human eye imaging, and realize the enhancement of a low-illumination color image by adjusting the noise intensity, thereby providing a visual sensing algorithm of the color image with better biological interpretability.
The invention adopts the following technical scheme:
a visual perception method based on a biological neuron network and stochastic resonance, comprising the steps of:
s1, converting a low-illumination color image from an RGB color space to an HSV color space through nonlinear transformation, and respectively marking a tone information matrix, a saturation information matrix and a brightness information matrix corresponding to three channels of the HSV color space as H M×N ,S M×N And V M×N M and N represent the number of rows and columns of the image, respectively, and output a brightness information matrix V M×N Tone information matrix H M×N And a saturation information matrix S M×N ；
S2, setting Gaussian white noise intensity D=D j The brightness information matrix V of the step S1 M×N Inputting a global feedback network composed of N integral discharging neuron models, and generating a moment when the membrane potential of each neuron reaches a threshold valueAction potentials, matrix (Index) i ) M×N Storing as output information whether the ith neuron is discharged during this encoding process;
s3, the step S2 (Index i ) M×N Decoding the matrix information into corresponding binary luminance information matrix Pix M×N The method comprises the steps of carrying out a first treatment on the surface of the Further integrating the luminance decoding information of all neurons to generate an enhanced luminance information matrix Pix M×N And then the tone information matrix H generated in the step S1 M×N Saturation information matrix S M×N And an enhanced luminance information matrix Pix M×N HSV space image Img after fusion enhancement HSV ；
S4, enhancing the HSV space image Img obtained in the step S3 HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB PQM (Img) of the image is calculated and outputted RGB ) As an evaluation index of the enhanced image, obtaining the enhanced image;
s5, increasing the Gaussian white noise intensity D in the step S2 j Repeating the steps S2-S4, and if the enhanced image closest to the optimal value 10 of the perception measurement index is obtained, completing the visual perception task.
Specifically, in step S1, a low-illuminance color image I with a size of 3 XMXN is obtained RGB The conversion from RGB color space to HSV color space is specifically:
where h is hue, s is saturation, v is brightness, (r, g, b) are red, green and blue coordinates of one color, respectively, max is the maximum of r, g, b, and min is the minimum of r, g, b.
Specifically, in step S2, the membrane potential of the neuron model is calculated as follows:
s201, input luminance information matrix H M×N Setting the film potential evolution time as [0,1 ]]Iteration step size is Δt=0.01, initializing membrane potentialAnd feedback term f m,n (0) =0, and set the upper 10% quantile of the H (S) matrix element distribution histogram as the threshold V th ；
S202, calculating the membrane potential at the next moment according to the initial membrane potential and the threshold value set in the step S201, ifRecord the discharge time +.>Reset membrane potential->And remember->Indicating that the ith neuron is at [0,1 ]]The discharge exists in the time period;
s203, combining the global feedback item and the discharge moment vector of the neuron network by utilizing the screening property of the Dirac delta functionCalculating a global feedback term f m,n (t) performing an update of the feedback item;
and S204, circularly calculating the steps S202 to S203 until the termination condition t=1 is reached, and obtaining the discharge information of all the neurons.
Further, in step S202, the membrane potential is set at the next timeCalculation ofThe following are provided:
wherein i is more than or equal to 1 and less than or equal to N, deltat is the time step length, and f m,n (t) is a feedback term, P (m, n) is an image brightness information matrix, D is noise intensity, and randn is a Gaussian random number.
Further, in step S203, the global feedback term f m,n The (t) is specifically as follows:
wherein G is the global feedback intensity,the kth discharge time when the image brightness matrix passes through the ith neuron is t the current time, and τ S And τ D Is a parameter of the convolution kernel.
Specifically, in step S3, the enhanced luminance information matrix Pix M×N The method comprises the following steps:
wherein, pix i (m, N) is the element of the m-th row and N-th column in the binary brightness information matrix, and N is the number of image columns.
Further, a binary luminance information matrix Pix i (m, n) is specifically:
wherein Index is i (m, n) is discharge index information.
Specifically, in step S4, the enhanced HSV spatial image Img is obtained by using the following formula HSV Nonlinear transformation from HSV space to RGB spaceEnhanced color image Img RGB The method comprises the following steps:
wherein V is an image brightness information matrix, h i Q, t, p are intermediate variables, respectively.
Specifically, in step S5, the method is repeated j times, and the corresponding enhanced image variance PQM is recorded j When the kth test satisfies |PQM k-2 -10|>|PQM k-1 -10|>|PQM k -at 10|, stopping the loop and outputting the kth-2 th trial enhancement image as the optimal output image;
when the preset maximum noise intensity is reached, the loop is stopped, and the enhanced image with the PQM value closest to 10 is output as the optimal output image.
Another technical solution of the present invention is a visual perception system based on a biological neuron network and stochastic resonance, comprising:
the conversion module converts the low-illumination color image from the RGB color space to the HSV color space through nonlinear transformation, and marks the tone information matrix, the saturation information matrix and the brightness information matrix corresponding to three channels of the HSV color space as H respectively M×N ,S M×N And V M×N M and N represent the number of rows and columns of the image, respectively, and output a brightness information matrix V M×N Tone information matrix H M×N And a saturation information matrix S M×N ；
Feedback module, set Gaussian white noise intensity D=D j The brightness information matrix V of the step S1 M×N The global feedback network composed of N integral discharging neuron models is input, and an action potential is generated at the moment when the membrane potential of each neuron reaches a threshold value, and a matrix (Index i ) M×N Storing as output information whether the ith neuron is discharged during this encoding process;
the fusion module is used for obtaining (Index) from the feedback module i ) M×N Decoding the matrix information into corresponding binary luminance information matrix Pix M×N The method comprises the steps of carrying out a first treatment on the surface of the Further integrating the luminance decoding information of all neurons to generate enhanced luminance information Pix M×N And then the tone information matrix H generated in the step S1 M×N Saturation information matrix S M×N And enhanced luminance information Pix M×N HSV space image Img after fusion enhancement HSV ；
The index module is used for enhancing the HSV space image Img after the fusion module is enhanced HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB PQM (Img) of the image is calculated and outputted RGB ) As an evaluation index of the enhanced image, obtaining the enhanced image;
perception module for increasing Gaussian white noise intensity D in feedback module j And repeating the feedback module, the fusion module and the index module, and if the enhanced image closest to the optimal value 10 of the perception measurement index is obtained, completing the visual perception task.
Compared with the prior art, the invention has at least the following beneficial effects:
the visual perception method based on the biological neuron network and stochastic resonance combines the stochastic resonance principle of the integral discharge neuron model based on synaptic conductivity with the biophysical process formed by vision, utilizes the integral discharge neuron network to mainly simulate the perception process of the cone cells, utilizes noise instead of eliminating noise based on the stochastic resonance principle, has the interpretability of the biological perception process, has clear quantization indexes, is convenient for automatically screening the optimal enhanced image, and has wide application prospect in the fields of military night vision, road traffic, monitoring video, brain-computer interfaces and the like; the steps correspond to the human eye visual perception process, and have better biological interpretability.
Further, the stochastic resonance principle is mainly to enhance the weak signal, so that the illumination of the low-illumination image is the weak signal which needs to be enhanced. If the RGB space is not converted into HSV space, the R, G, B three channels are respectively enhanced, so that the calculated amount is greatly increased.
Further, whether the neuron generates discharge is recorded through the membrane potential, so that the image brightness information matrix is enhanced according to discharge information.
Further, the global feedback represents the influence of other neurons on the ith neuron, and the human eye visual perception process is simulated more truly.
Further, pix M×N The matrix is an indirect representation of the image enhancement effect, pix M×N Each pixel point Pix of the matrix i (m, n) is recorded as 1 or 0 according to the firing of the ith neuron.
Further, the enhanced HSV space image Img HSV Is converted into a color image in RGB space, thereby obtaining a final enhanced image.
Further, using PQM as an image quality evaluation index, a PQM value as close as 10 indicates that the image enhancement effect is better. In j repeated experiments, a perception measurement index PQM corresponding to the enhanced image is recorded j When the kth test meets Var k-2 >Var k-1 >Var k And stopping the circulation, and outputting the k-2 test enhancement image as an optimal output image.
In conclusion, the invention refers to the biological visual perception process, has biological interpretability, and has simple operation flow and easy realization.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
FIG. 1 is a flow chart of the structure of the method of the invention;
FIG. 2 is an experimental test chart of the present invention, (a) is an original low-intensity color image, (b) is an image enhanced by the method of the present invention for (a), (c) is a second original low-intensity color image, and (d) is an image enhanced by the method of the present invention for (c);
FIG. 3 is a diagram of an experimental comparison of the present invention, (a) an original low-intensity color image, (b) an image enhanced by the algorithm of the present invention, (c) a Retinex enhanced image, and (d) a HE enhanced image;
fig. 4 is another set of experimental contrast plots of the present invention, wherein (a) is an original low-light image, (b) is a corresponding enhanced image of the present invention, (c) is a Retinex algorithm enhanced image, and (d) is a HE algorithm enhanced image.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it will be understood that the terms "comprises" and "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Various structural schematic diagrams according to the disclosed embodiments of the present invention are shown in the accompanying drawings. The figures are not drawn to scale, wherein certain details are exaggerated for clarity of presentation and may have been omitted. The shapes of the various regions, layers and their relative sizes, positional relationships shown in the drawings are merely exemplary, may in practice deviate due to manufacturing tolerances or technical limitations, and one skilled in the art may additionally design regions/layers having different shapes, sizes, relative positions as actually required.
The invention provides a visual perception method based on a biological neuron network and a stochastic resonance principle, which utilizes the integral discharge neuron network to mainly simulate a basic physiological process of visual perception of a cone cell in a dark environment, combines the stochastic resonance phenomenon of the integral discharge neuron model based on synaptic conductivity with the basic biophysical process of visual enhancement by utilizing noise instead of eliminating noise through the stochastic resonance principle, and gives biological explanation of visual perception to a certain extent. The method has clear quantization indexes, is convenient for automatically screening the optimal enhanced image, and has wide application prospect in the fields of military night vision, road traffic, monitoring video, brain-computer interfaces and the like.
Referring to fig. 1, the visual perception method based on biological neuron network and stochastic resonance of the present invention comprises the following steps:
s1, image input
Inputting a low-illumination color image, converting the image from RGB (red, green and blue) color space to HSV (color, saturation and brightness) color space by nonlinear transformation, and respectively marking hue information matrix, saturation information matrix and brightness information matrix corresponding to three channels of HSV as H M×N ,S M×N And V M×N Wherein M and N respectively represent the number of rows and columns of the image, and outputs a brightness information matrix V M×N Step S2, outputting a tone information matrix and a saturation information matrix H M×N ,S M×N Go to step S3;
first, a low-illumination color image I with the size of 3 xMxN is input RGB Converting the image from RGB color space to HSV color space by the formula (1) to obtain a new image I with the size of 3 xMxN in the HSV color space HSV Wherein three matrices in turn represent the hue information matrix H of the image M×N Saturation information matrix S M×N And a luminance information matrix V M×N 。
Wherein (r, g, b) are red, green and blue coordinates of one color, respectively, their values are real numbers between 0 and 1, max is the maximum value of r, g, b, and min is the minimum value of r, g, b.
S2, image coding process
When the image signal enters the eye, the retina first converts the image light signal of the HSV space into electrical pulses that can be transmitted via the interneurons. There are two photoreceptors in the retina: rod cells and cone cells, whereas the reception and processing of color images in low light conditions is mainly responsible for cone cells. When the optical signal enters the retina through the human eye, the cone cells process the saturation information and the brightness information respectively. Considering that the low contrast photo causes are low brightness, the invention processes brightness information, and the general process of simulating cone cell perception processing by the integral discharge neural network is given below.
Considering a global feedback network consisting of one N integral discharge neuron model, the evolution equation of the membrane potential of each neuron model is:
wherein the subscript i corresponds to the ith neuron and the superscripts m, n correspond to the elements of the mth row and n column of the luminance information matrix of the image;represents the neuronal membrane potential when +.>From the bottomThe square reaches the threshold V th When the ith neuron immediately gives out nerve pulse, the action potential giving time is +.>The corresponding neuron firing string is +.> Represents Gaussian white noise and satisfies +.>To describe the noise input to the retinal neural network by the peripheral neurons, D is the adjustable noise intensity;
the global feedback term of the neural network is defined as the convolution of the alpha delay function with the firing string of the neural network:
wherein G is global feedback intensity, τ S And τ D Parameters that are convolution kernel functions; p (m, n) E (0, 1) represents the input image information, and is taken as a brightness component value V (m, n) in the step S1; letting matrix (Index) i ) M×N The discharge information of the ith neuron in this encoding process is stored as an output of step S2.
Setting the gaussian white noise intensity d=d j The luminance information matrix H (m, n) of the image (saturation information matrix S (m, n)) is input to equation (2) and the film potential is calculated by Euler-Maruyama numerical scheme:
the membrane potential of neurons was calculated as follows:
s201, input image brightness information matrix H M×N (saturation information matrix S) M×N ) Instead of P (m, n) in the formula (4), the film potential evolution time is set to be [0,1]The iteration step is Δt=0.01. Initializing membrane potentialAnd feedback term f m,n (0) =0, and set the upper 10% quantile of the H (S) matrix element distribution histogram as the threshold V th ；
S202, calculating the membrane potential at the next moment through a formula (4), wherein randn represents a standard normal random number. If it isThen record the discharge time at this time +.>Reset membrane potential->And remember->Indicating that the ith neuron is at [0,1 ]]The discharge exists in the time period;
s203, combining the formula (3) and the discharge moment vector by utilizing the screening property of the Dirac delta functionComputing global feedback terms
And S204, circularly calculating the steps S202 to S203 until the termination condition t=1 is reached, and finally obtaining the discharge information of all the neurons.
S3, image decoding and integrating process
The visual cortex cells first resolve the luminance information matrix transmitted via the retina into a binary image: because the carrier for nerve information transmission is an electric pulse, the coded information in the invention adopts peak sequences instead of membrane potential information of neurons.
Accordingly, the (Index) in step S2 is set i ) M×N Decoding the matrix information into corresponding binary luminance information matrix Pix i (m,n)
The visual cortex as a command center further integrates the brightness decoding information of all neurons, and the integration mode is set as linear superposition, namely:
the integration process embodies the cooperative effect of a large number of neurons in completing the same sensing function: if each binary image is considered as the output of a weak learner, then integrating the weak learners forms a strong learner, generates enhanced image information, and then color information (H M×N ,S M×N ) And a processed luminance information matrix (Pix M×N ) The enhanced HSV space image Img can be obtained by fusion HSV And will enhance the image Img HSV As an output of step S3;
s4, image space conversion and evaluation
Enhanced HSV space image Img HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB An evaluation index of the enhanced image, namely, a perception metric index PQM (Img) of the image is calculated and outputted RGB )。
The RGB color space is:
wherein, the liquid crystal display device comprises a liquid crystal display device,q＝V×(1-f×S)，/>p=v× (1-S), t=v× (1- (1-f) ×s), and then a perceptual metric (PQM) of the enhanced image is calculated and denoted as PQM j 。
S5, optimal output image selection
Increasing the noise intensity D in step S2 j Taking the value, repeating the steps S2-S4, j representing the j-th repetition, and recording the corresponding perception measurement index (PQM) of the enhanced image, when the k-th test meets the |PQM k-2 -10|>|PQM k-1 -10|>|PQM k -10|, indicating that the enhancement effect has produced a downward trend, stopping the loop, outputting the k-2 th trial enhancement image as the optimal output image; or stopping the circulation when the preset maximum noise intensity is reached, and outputting the enhanced image with the PQM value closest to 10 as an optimal output image.
In still another embodiment of the present invention, a visual perception system based on a biological neuron network and stochastic resonance is provided, which can be used to implement the visual perception method based on the biological neuron network and stochastic resonance, and specifically, the visual perception system based on the biological neuron network and stochastic resonance includes a conversion module, a feedback module, a fusion module, an index module, and a perception module.
The conversion module converts the low-illumination color image from the RGB color space to the HSV color space through nonlinear transformation, and marks a tone information matrix, a saturation information matrix and a brightness information matrix corresponding to three channels of the HSV color space as H respectively M×N ,S M×N And V M×N M and N represent the number of rows and columns of the image, respectively, and output a brightness information matrix V M×N Tone information matrix H M×N And a saturation information matrix S M×N ；
Feedback module, set Gaussian white noise intensity D=D j The brightness information matrix V of the step S1 M×N Inputting a global composed of N integral discharging neuron modelsA feedback network, which generates an action potential at the moment when the membrane potential of each neuron reaches a threshold value, a matrix (Index) i ) M×N Storing as output information whether the ith neuron is discharged during this encoding process;
the fusion module is used for obtaining (Index) from the feedback module i ) M×N Decoding the matrix information into corresponding binary luminance information matrix Pix M×N The method comprises the steps of carrying out a first treatment on the surface of the Further integrating the luminance decoding information of all neurons to generate enhanced luminance information Pix M×N And then the tone information matrix H generated in the step S1 M×N Saturation information matrix S M×N And enhanced luminance information Pix M×N HSV space image Img after fusion enhancement HSV ；
The index module is used for enhancing the HSV space image Img after the fusion module is enhanced HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB PQM (Img) of the image is calculated and outputted RGB ) As an evaluation index of the enhanced image, obtaining the enhanced image;
perception module for increasing Gaussian white noise intensity D in feedback module j And repeating the feedback module, the fusion module and the index module, and if the enhanced image closest to the optimal value 10 of the perception measurement index is obtained, completing the visual perception task.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention is further illustrated by the following examples and the accompanying drawings.
1) Experimental conditions:
the environment used in the experiment is Intel Core i7-6700U@3.40GHz dual-Core CPU, the memory is 8GB, and the programming environment platform is Matlab R2016a. The test images used for the experiment were four test images 1,2,3,4 of size.
2) The experimental contents are as follows:
the method provided by the invention is used for processing the image and comparing the image with the existing single-scale Retinex algorithm and histogram equalization method (Histogram Equalization HE).
(1) PQM is an index proposed based on the human eye vision system to evaluate image quality. The closer the PQM is to 10, the better the image quality.
(2) Information entropy, which is used to measure the richness of image information. The greater the information entropy, the higher the richness of the image information.
(3) CEF, the index is used mainly to measure the contrast enhancement performance of the algorithm. The larger the CEF, the better the image enhancement performance.
Table 1 comparison of the various indices of the method of the invention with the other two methods
A comparison of the respective properties of the three methods is given in table 1. Experimental results show that various indexes of the image enhancement algorithm based on stochastic resonance provided by the method are better than those of other three algorithms, the contrast, brightness, details and the like of the image are considered, and the visual effect of the image is good.
Referring to fig. 2, a comparison of the results of three image enhancement methods is given; as can be seen from fig. 2, the method of the present aspect is superior to the other three methods in contrast, chromaticity, etc.
Referring to fig. 3 and 4, a comparison of the results of three image enhancement methods is given; experimental results show that the image is successfully compressed when the image is processed by the single-scale Retinex method, the color sense is good, but the details of a part of dark areas are not clear enough, the HE method can cause the partial distortion of the image, and the result of the image enhancement algorithm based on stochastic resonance, which is proposed in the invention, shows that the brightness of the image is greatly improved compared with the other two methods, and the details of the dark areas are also obviously enhanced. The edge information is obvious, the color sense is better, and the visual effect is greatly improved.
In summary, the visual perception method and system based on the biological neuron network and stochastic resonance can remarkably improve the contrast and brightness of images, remarkably enhance details of dark areas, remarkably improve edge information, achieve a better visual effect, and show remarkable advantages compared with a classical single-scale Retinex method and an HE algorithm.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above is only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited by this, and any modification made on the basis of the technical scheme according to the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (10)
1. The visual perception method based on the biological neuron network and stochastic resonance is characterized by comprising the following steps of:
s1, converting a low-illumination color image from an RGB color space to an HSV color space through nonlinear transformation, and respectively marking a tone information matrix, a saturation information matrix and a brightness information matrix corresponding to three channels of the HSV color space as H M×N ,S M×N And V M×N M and N represent the number of rows and columns of the image, respectively, and output a brightness information matrix V M×N Tone information matrix H M×N And a saturation information matrix S M×N ；
S2, setting Gaussian white noise intensity D=D j The brightness information matrix V of the step S1 M×N The global feedback network composed of N integral discharging neuron models is input, and an action potential is generated at the moment when the membrane potential of each neuron reaches a threshold value, and a matrix (Index i ) M×N Storing the ith neuron encoded thereinInformation of whether discharge is generated in the process is taken as output;
s3, the step S2 (Index i ) M×N Decoding the matrix information into corresponding binary luminance information matrix Pix M×N The method comprises the steps of carrying out a first treatment on the surface of the Further integrating the luminance decoding information of all neurons to generate an enhanced luminance information matrix Pix M×N And then the tone information matrix H generated in the step S1 M×N Saturation information matrix S M×N And an enhanced luminance information matrix Pix M×N HSV space image Img after fusion enhancement HSV ；
S4, enhancing the HSV space image Img obtained in the step S3 HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB PQM (Img) of the image is calculated and outputted RGB ) As an evaluation index of the enhanced image, obtaining the enhanced image;
s5, increasing the Gaussian white noise intensity D in the step S2 j Repeating the steps S2-S4, and if the enhanced image closest to the optimal value 10 of the perception measurement index is obtained, completing the visual perception task.
2. The visual perception method based on biological neuron network and stochastic resonance according to claim 1, wherein in step S1, a low-illumination color image I with a size of 3×m×n is obtained RGB The conversion from RGB color space to HSV color space is specifically:
where h is hue, s is saturation, v is brightness, (r, g, b) are red, green and blue coordinates of one color, respectively, max is the maximum of r, g, b, and min is the minimum of r, g, b.
3. The visual perception method based on biological neuron network and stochastic resonance according to claim 1, wherein in step S2, the membrane potential of the neuron model is calculated as follows:
s201, input luminance information matrix H M×N Setting the film potential evolution time as [0,1 ]]Iteration step size is Δt=0.01, initializing membrane potential V i m,n (0) =0 and feedback term f m,n (0) =0, and set the upper 10% quantile of the H (S) matrix element distribution histogram as the threshold V th ；
S202, calculating the membrane potential at the next moment according to the initial membrane potential and the threshold value set in the step S201, and if V i m,n (t+Δt)>V th Record the discharge time T at this time i m,n ＝[T i m,n ,t+Δt]Reset membrane potential V i m,n (t+Δt)＝V 0 And recordIndicating that the ith neuron is at [0,1 ]]The discharge exists in the time period;
s203, combining the global feedback item and the discharge moment vector T of the neuron network by utilizing the screening property of the Dirac delta function i m,n Calculating a global feedback term f m,n (t) performing an update of the feedback item;
and S204, circularly calculating the steps S202 to S203 until the termination condition t=1 is reached, and obtaining the discharge information of all the neurons.
4. A visual perception method based on biological neuron network and stochastic resonance according to claim 3, characterized in that in step S202, the membrane potential V at the next moment i m,n (t+Δt) is calculated as follows:
wherein i is more than or equal to 1 and less than or equal to N, deltat is the time step length, and f m,n (t) is a feedback term, P (m, n) is an image brightness information matrix, D is noise intensity, and randn is a Gaussian random number.
5. A visual perception method based on biological neuronal network and stochastic resonance according to claim 3, characterized in that in step S203, the global feedback term f m,n The (t) is specifically as follows:
wherein G is the global feedback intensity,the kth discharge time when the image brightness matrix passes through the ith neuron is t the current time, and τ S And τ D Is a parameter of the convolution kernel.
6. The visual perception method based on biological neuron network and stochastic resonance according to claim 1, wherein in step S3, the enhanced luminance information matrix Pix M×N The method comprises the following steps:
wherein, pix i (m, N) is the element of the m-th row and N-th column in the binary brightness information matrix, and N is the number of image columns.
7. The visual perception method based on biological neuron network and stochastic resonance according to claim 6, wherein the binary luminance information matrix Pix i (m, n) is specifically:
wherein Index is i (m, n) is discharge index information.
8. The visual perception method based on biological neuron network and stochastic resonance according to claim 1, wherein in step S4, the enhanced HSV space image Img is obtained by using the following formula HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB The method comprises the following steps:
wherein V is an image brightness information matrix, h i Q, t, p are intermediate variables, respectively.
9. The visual perception method based on biological neuron network and stochastic resonance according to claim 1, wherein in step S5, the corresponding enhanced image variance PQM is recorded by repeating j times j When the kth test satisfies |PQM k-2 -10|>|PQM k-1 -10|>|PQM k -at 10|, stopping the loop and outputting the kth-2 th trial enhancement image as the optimal output image;
when the preset maximum noise intensity is reached, the loop is stopped, and the enhanced image with the PQM value closest to 10 is output as the optimal output image.
10. A visual perception system based on a biological neuronal network and stochastic resonance, comprising:
the conversion module converts the low-illumination color image from the RGB color space to the HSV color space through nonlinear transformation, and marks the tone information matrix, the saturation information matrix and the brightness information matrix corresponding to three channels of the HSV color space as H respectively M×N ,S M×N And V M×N M and N represent the number of rows and columns of the image, respectively, and output a brightness information matrix V M×N Tone information matrix H M×N And a saturation information matrix S M×N ；
Feedback module, set Gaussian white noise intensity D=D j The brightness information matrix V of the step S1 M×N The global feedback network composed of N integral discharging neuron models is input, and an action potential is generated at the moment when the membrane potential of each neuron reaches a threshold value, and a matrix (Index i ) M×N Storing as output information whether the ith neuron is discharged during this encoding process;
the fusion module is used for obtaining (Index) from the feedback module i ) M×N Decoding the matrix information into corresponding binary luminance information matrix Pix M×N The method comprises the steps of carrying out a first treatment on the surface of the Further integrating the luminance decoding information of all neurons to generate enhanced luminance information Pix M×N And then the tone information matrix H generated in the step S1 M×N Saturation information matrix S M×N And enhanced luminance information Pix M×N HSV space image Img after fusion enhancement HSV ；
The index module is used for enhancing the HSV space image Img after the fusion module is enhanced HSV Enhanced color image Img from non-linear transformation of HSV space to RGB space RGB PQM (Img) of the image is calculated and outputted RGB ) As an evaluation index of the enhanced image, obtaining the enhanced image;
perception module for increasing Gaussian white noise intensity D in feedback module j And repeating the feedback module, the fusion module and the index module, and if the enhanced image closest to the optimal value 10 of the perception measurement index is obtained, completing the visual perception task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111605009.1A CN114240802B (en) | 2021-12-24 | 2021-12-24 | Visual perception method and system based on biological neuron network and stochastic resonance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111605009.1A CN114240802B (en) | 2021-12-24 | 2021-12-24 | Visual perception method and system based on biological neuron network and stochastic resonance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114240802A CN114240802A (en) | 2022-03-25 |
| CN114240802B true CN114240802B (en) | 2023-08-01 |
Family
ID=80762902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111605009.1A Active CN114240802B (en) | 2021-12-24 | 2021-12-24 | Visual perception method and system based on biological neuron network and stochastic resonance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114240802B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117709419A (en) * | 2022-10-09 | 2024-03-15 | 航天科工集团智能科技研究院有限公司 | Pulse neural network training method, recognition system building method and recognition system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013139067A1 (en) * | 2012-03-22 | 2013-09-26 | Hou Kejie | Method and system for carrying out visual stereo perception enhancement on color digital image |
| CN105046663A (en) * | 2015-07-10 | 2015-11-11 | 西南科技大学 | Human visual perception simulation-based self-adaptive low-illumination image enhancement method |
| CN106504212A (en) * | 2016-11-07 | 2017-03-15 | 湖南源信光电科技有限公司 | A kind of improved HSI spatial informations low-luminance color algorithm for image enhancement |
| CN113158746A (en) * | 2021-02-02 | 2021-07-23 | 杭州电子科技大学 | Weak signal sensing method based on neuron small-world network stochastic resonance |
-
2021
- 2021-12-24 CN CN202111605009.1A patent/CN114240802B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013139067A1 (en) * | 2012-03-22 | 2013-09-26 | Hou Kejie | Method and system for carrying out visual stereo perception enhancement on color digital image |
| CN105046663A (en) * | 2015-07-10 | 2015-11-11 | 西南科技大学 | Human visual perception simulation-based self-adaptive low-illumination image enhancement method |
| CN106504212A (en) * | 2016-11-07 | 2017-03-15 | 湖南源信光电科技有限公司 | A kind of improved HSI spatial informations low-luminance color algorithm for image enhancement |
| CN113158746A (en) * | 2021-02-02 | 2021-07-23 | 杭州电子科技大学 | Weak signal sensing method based on neuron small-world network stochastic resonance |
Non-Patent Citations (1)
| Title |
|---|
| 基于神经网络的低照度彩色图像增强算法;毛伟民;赵勋杰;;光学技术(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114240802A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111709902B (en) | Infrared and visible light image fusion method based on self-attention mechanism | |
| Sihotang | Implementation of Gray Level Transformation Method for Sharping 2D Images | |
| CN107851193B (en) | Hybrid machine learning system | |
| CN109416758A (en) | The method of neural network and neural metwork training | |
| CN110796595A (en) | Tone mapping method and device and electronic equipment | |
| CN110717868A (en) | Video high dynamic range inverse tone mapping model construction and mapping method and device | |
| CN114240802B (en) | Visual perception method and system based on biological neuron network and stochastic resonance | |
| Zhang et al. | Dual-channel multi-task CNN for no-reference screen content image quality assessment | |
| CN115526803A (en) | Non-uniform illumination image enhancement method, system, storage medium and device | |
| Goel et al. | Gray level enhancement to emphasize less dynamic region within image using genetic algorithm | |
| CN108537852B (en) | Self-adaptive color constancy method based on image local contrast | |
| CN113810683A (en) | No-reference evaluation method for objectively evaluating underwater video quality | |
| He et al. | A night low‐illumination image enhancement model based on small probability area filtering and lossless mapping enhancement | |
| Han et al. | Representation of white-and black-box adversarial examples in deep neural networks and humans: A functional magnetic resonance imaging study | |
| CN112818774A (en) | Living body detection method and device | |
| CN110610508B (en) | Static video analysis method and system | |
| Zhang et al. | A novel DenseNet Generative Adversarial network for Heterogenous low-Light image enhancement | |
| Agrawal et al. | Exploring convolutional neural networks for automatic image colorization | |
| JP7362924B2 (en) | Data augmentation-based spatial analysis model learning device and method | |
| CN112270220B (en) | Sewing gesture recognition method based on deep learning | |
| CN114549386A (en) | Multi-exposure image fusion method based on self-adaptive illumination consistency | |
| Li et al. | Multi-scale fusion framework via retinex and transmittance optimization for underwater image enhancement | |
| Park et al. | Underwater image enhancement using adaptive standardization and normalization networks | |
| Wu et al. | Fish Target Detection in Underwater Blurred Scenes Based on Improved YOLOv5 | |
| Lian et al. | A pulse-number-adjustable MSPCNN and its image enhancement application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |