CN100481123C - Implementation method of retina encoder using space time filter - Google Patents

Abstract

This invention relates to one visual film coding method in computer application field, which comprises the following steps: inputting specimen image into time and space filter to get the BP human neural network input, train BP human neural net and to determine human neural net weight value; then inputting specimen any image and taking one random set of filter parameters by use of particle set or strategy method in filter parameter range for optimization; through multiple overlap, outputting image converged to input image and the parameters of the filter are determined with output impulse excitation is relative to input image film codes.

Description

Adopt the retina encoder implementation method of space time filter

Technical field

The present invention relates to the method in a kind of Computer Applied Technology field, specifically is a kind of retina encoder implementation method that adopts space time filter.

Background technology

Retina encoder is an important component part of retina artificial vision prosthese, is from known physiological knowledge and experimental data, and between input picture and the boost pulse string corresponding relation proposes in order to solve.Vision is an important channel of human acquired information.Yet a lot of people's existence dysopia is in various degree arranged in the world, can not pass through the vision acquired information.According to World Health Organization (WHO) statistics, the whole world has nearly 4,000 ten thousand people blind approximately, and other 100,000,000 people that have an appointment have in various degree vision impairment or weakening.For the patient who causes because of retinal disease losing one's sight, still some retina cell and optic cell function are intact for they.So can attempt on retina, designing vision prosthesis, stimulate the next vision of partly rebuilding of int part on the retina by visual information being converted to electro photoluminescence.Visual performance how partially or completely to recover the blind person has become at present the heat subject of research both at home and abroad.

Though present research to the modeling of retina vision prosthesis has obtained noticeable achievement even has tested on the animal and human, yet these models still rest on the simulation to the retinal tissue structure, people do not solve this key problem of coding of retina signal Processing, so also can't help the blind person partly to recover vision effectively so far.

At present both at home and abroad amphiblestroid modeling there are artificial nerve network model based on method, formula simulation of the statistical model of experimental data and mathematical method etc. and hardware based model of simulating retinal structure by the CMOS chip.

Find through literature search prior art, ECKMILLER etc. deliver Tunableretina encoders for retina implants:why and how (adjustable retina vision prosthesis scrambler) on " JOURNAL OF NEURALENGINEERING " (neuroengineering magazine) (phase in February, 2,005 91 to 104 pages), adjustable retina coding method is proposed in this article, concrete grammar is: be divided into two modules of retina and central vision system, with space time filter image is encoded, with the state parameter of the circle that moves as two modules of sample training.Its deficiency is: the used space time filter of ECKMILLER retina coding method also is a kind of reconstruct to image, directly image and impulse stimulation is not connected.

Summary of the invention

The object of the invention is at the deficiencies in the prior art, solve the encoding relation between input picture and the boost pulse, a kind of retina encoder implementation method that adopts space time filter is provided, make it simulate the retinal ganglial cells signal processing with space time filter around the center, simulate brain processing visual signal with the BP artificial neural network nerve impulse is converted to image section, regulate the space time filter parameter to reach optimum output effect with improving population and evolution strategy parameter optimization method.

The present invention is achieved by the following technical solutions, and the present invention at first imports space time filter with sample image, and the output that obtains is as the input of BP artificial neural network, and BP ANN is determined the weights of BP artificial neural network.Import any one image in the sample image then, get one group of space time filter parameter at random, in the space time filter parameter area, carry out parameter optimization,, finally make output image converge on input picture by iteration repeatedly with population or evolution strategy method.After the parameter of space time filter was determined, the impulse stimulation of its output then was the retina coding corresponding to input picture.

Described space time filter is meant a kind of center space time filter of pattern on every side.This space time filter has spatial and temporal resolution preferably.Inseparable and the periphery of its space-time has been simulated the retina signal processing well than character such as center delays.The most important thing is, be different from the model that other research groups such as ECKMILLER propose, this space time filter directly interrelates the output nerve impulsion of input picture and retinal ganglial cells, has truly realized amphiblestroid coding.This space time filter has 7 parameters, comprising 3 space time filter time parameter λ _c, λ _s, d, represent that respectively the output of space time filter center and periphery reaches the delay to the center of time of peak value and space time filter periphery, 2 space time filter spatial parameter σ _c, σ _s, represent the field range of space time filter receptive field center and periphery, 2 weighting parameter α respectively _c, α _s, represent receptive field center and periphery weight respectively.Its mathematic(al) representation is:

$\mathrm{CS}(x,t)={\mathrm{\α}}_{c}K(t,{\mathrm{\λ}}_c)\mathrm{\Σ}\left\{G(x,{\mathrm{\σ}}_c)\mathrm{pix}\left(x\right)\right\}-{\mathrm{\α}}_{s}K(t-d,{\mathrm{\λ}}_s)\mathrm{\Σ}\left\{G(x,{\mathrm{\σ}}_s)\mathrm{pix}\left(x\right)\right\}$

$K(t,\mathrm{\&lambda;})=\left\{\begin{array}{ccc}\mathrm{\&lambda;texp}(-\mathrm{\&lambda;t})& \mathrm{if}& t\&GreaterEqual;0\\ 0& \mathrm{if}& t<0\end{array}\right.$

G(x，σ)＝(2πσ ²) ^-1exp(-x ²/2σ ²)

This space time filter peripheral space scope is bigger than central space scope, i.e. σ _c＜σ _sPeriphery postpones to some extent than the center time response time response, i.e. λ _s＜λ _c, d〉and 0.The retina encoder that the present invention realized adopts 9 space time filters, retinal ganglial cells of each space time filter simulation.The receptive field of 9 space time filters can be overlapping, and the image of 729 picture elements is handled, and the input picture of 729 picture elements is converted to pulse output.

The present invention handles visual signal with BP artificial Neural Network Simulation brain.Before doing human body experimental conditions maturation, replace brain that nerve impulse is converted to image, can be described as the inverse mapping of space time filter.BP artificial neural network have three layers composition, i.e. input layer, hidden layer and output layer.The input layer of used BP network has 279 neurons in the retina encoder that the present invention realized, corresponding to the train of impulses of space time filter output, hidden layer has 35 neurons, and 729 neurons of output layer are corresponding to the output image of 729 picture elements.With of the inverse mapping of BP artificial neural network as space time filter, can freelyr must select sample space, can train the sample image diversified, that quantity is big, and be not limited to the used mobile circle of ECKMILLER.

Describedly in the space time filter parameter area, carry out parameter optimization, be specially: find the optimized parameter vector by error function, error function F (z with the evolution strategy method _i) be Euler's distance of output image and input picture, parameter vector comprises 63 parameters of 9 described space time filters.Parameter area is: 0＜σ _c+ 0.1＜σ _s≤ 3, σ _cAnd σ _sCorrespond respectively to center and picture element receptive field scope on every side, 0.5≤α _c＜1, α _c+ α _s=1, α _cAnd α _sBe center and weights on every side, 17≤λ _s＜λ _c≤ 25, λ c, λ s is relevant with periphery output pulse arrival time to peak with the space time filter center, 0.04≤d≤0.08, d is relevant time delay to the center with periphery.The initial former generation of picked at random vector z in parameter area at first _i, i=1 ..., P.Produce the filial generation vector x by on each element of former generation's vector, adding a zero-mean Gaussian random variable _i=z _i+ N (0, σ _i), i=1 ..., P, σ _i=F (z _i)/300.The variances sigma of gaussian variable is relevant with error function, can accelerate parameter convergence speed.Relative error function F (z _i) and F (x _i), i=1 ..., P, the less vector of Select Error is as the former generation of next iteration.Iteration is till the iteration stopping condition that satisfies method.The advantage of evolution strategy method is to be convenient to realize that speed is very fast, and to the parameter area less-restrictive, but randomness is bigger, lacks the convergent directivity.Have for simple sample image and comparatively fast to restrain effect preferably.

Describedly in the space time filter parameter area, carry out parameter optimization, be specially: realize that by fitness function fitness function is Euler's distance of output image and input picture with the population method.Population method used in the retina encoder that the present invention realized is formed population by 6 particles, and each particle comprises 63 parameters of 9 space time filters.Parameter area is: 0＜σ _c+ 0.4＜σ _s≤ 3, σ _cAnd σ _sCorrespond respectively to center and picture element covering domain on every side, 0.5≤α _c＜0.8, α _c+ α _s=1, α _cAnd α _sBe center and weights on every side, 17≤λ _c≤ 25,14≤λ _s＜λ _c, λ c, λ s is relevant with periphery output pulse arrival time to peak with the space time filter center, 0.04≤d≤0.08, d is relevant time delay to the center with periphery.At first all particle's velocity and position in the initialization particle population in parameter area.With fitness function all particles are estimated, according to the fitness function more individual extreme value p of each particle and whole extreme value 1 in the new population.Individual extreme value be single particle from beginning to search the optimal vector of current iteration, whole extreme value is that the particle population is from beginning to search the optimal vector of current iteration.According to the improvement population method formula that combines by traditional population method and evolution strategy particle rapidity and position are carried out iteration: v then _i=error * randn * (p _i-x _i)+error * randn * (1 _i-x _i)+error * randn, x _i(t+1)=x _i(t)+v _i, error is the output image that draws according to fitness function and the error of original image, randn is a Gaussian random variable.Iteration is till the iteration stopping condition that satisfies method.The advantage of improved population method is that Memorability is arranged, and the result of each search is in store, determines search speed according to individual extreme value and whole extreme value, and directivity is preferably arranged.Be different from traditional population method, the retina encoder of being realized in order more to be adapted to the present invention, the particle rapidity of improved population method is also relevant with Gaussian random variable with fitness function, like this can the relative requirement that must reduce to parameter area, and accelerate speed of convergence.

The present invention combines center periphery space time filter, BP artificial neural network and parameter optimization method, interlocks layer by layer, and contact closely can be simulated the retina signal processing effectively.It is different from existing retina encoder, can encode to image and directly produce corresponding boost pulse.Being used in combination of BP artificial neural network among the present invention and parameter optimization method, make this retina encoder to train to diversified large sample image space, and can get the exptended sample image space flexibly, increase space time filter parameter space scope.The dirigibility and the better variation that must satisfy individual difference and input picture of tunable performance of the retina encoder that the present invention realizes, and can embed use as a pith of retina vision prosthesis.

Description of drawings

Fig. 1 is for to be divided into 9 with the pixel zone, the synoptic diagram of picked at random receptive field center.

The space time filter spatial filtering output waveform synoptic diagram that Fig. 2 realizes for the present invention based on the retina encoder of space time filter; Horizontal ordinate is the distance from the receptive field center, and ordinate is a pulsed frequency.

The space time filter time filtering output waveform synoptic diagram that Fig. 3 realizes for the present invention based on the retina encoder of space time filter.Horizontal ordinate is the time, and ordinate is a pulsed frequency.

Fig. 4 is the part of selected sample image; Selected sample image is the different square or rectangular in big or small position.

The result that Fig. 5 obtains with the evolution strategy parameter optimization method for the embodiment of the invention.

The result that Fig. 6 obtains with improved population parameter optimization method for the embodiment of the invention.

Fig. 7 (a) is a sample image.Fig. 7 (b-d) is the corresponding space time filter output of Fig. 7 (a).

Embodiment

Below embodiments of the invention are elaborated: present embodiment has provided detailed embodiment and process being to implement under the prerequisite with the technical solution of the present invention, but protection scope of the present invention is not limited to following embodiment.

Present embodiment is finished by two parts, i.e. BP artificial neural network training process and parameter optimization method parameters optimization process.Space time filter as simulation retinal ganglial cells signal processing part through wherein.

1. as shown in Figure 1,729 picture element zones are divided into 9, receptive field center of picked at random in each piece zone.

2. select one group of space time filter initial parameter vector a.Vector a contains 63 param elements of 9 space time filters.7 parameters of each wave filter wherein, be respectively 2 weighting parameters, the weight of receptive field center and periphery is described, 2 spatial parameters, the scope of receptive field center and periphery is described, 3 time parameters are described the space time filter output pulse frequency and are reached the delay to the center of time of peak value and receptive field periphery.Fig. 2 and Fig. 3 have provided the time of space time filter and the output pulse in space respectively.The horizontal ordinate of Fig. 2 is the distance of pixel from the receptive field center, and ordinate is corresponding pulsed frequency.The horizontal ordinate of Fig. 2 is the time, and ordinate is a pulsed frequency.

3. be the space time filter of a with sample image by parameter vector, obtain exporting 279 * 77 matrix.As shown in Figure 4, sample image is size, the different square or rectangular in position.

4. the output of space time filter is trained as the input of BP artificial neural network.The input layer of the BP network that the present invention is used has 279 neurons, and hidden layer has 35 neurons, 729 neurons of output layer.This BP network has added biasing and momentum, and it is faster to make it convergence.

5. in sample image, choose an image x wantonly.

6. utilization evolution strategy or improved population parameter optimization method are come parameters optimization.

When using the evolution strategy method, choose the parameter vector that contains 63 elements, wherein each space time filter comprises 7 parameters, σ c, the field range of σ s corresponding space time filter receptive field center of difference and periphery, α c, α s is corresponding receptive field center and periphery weight respectively, λ c, λ s corresponding respectively space time filter center and periphery output reach the time of peak value, and the corresponding space time filter periphery of d is to the delay at center.Parameter area is: 0＜σ _c+ 0.1＜σ _s≤ 3,0.5≤α _c＜1, α _c+ α _s=1,17≤λ _s＜λ _c≤ 25,0.04≤d≤0.08.In parameter area, choose one group of parameter initialization former generation parameter vector.During each iteration, on each element of former generation's parameter vector b, add a zero-mean Gaussian random variable and produce filial generation parameter vector b '.With image x is respectively the space time filter of b and b ' by parameter vector, and the BP network that its output is finished by training again obtains output image y and y '.With output image y, y ' and input picture x compare, and choose the former generation parameter vector of the pairing parameter vector of the less image of error function as next iteration.Iteration is till the iteration stopping condition that satisfies method.The variance of the gaussian variable of being got is relevant with error function.Error function is Euler's distance of input picture and output image.As shown in Figure 5, Fig. 5 (a) (c) is two input pictures, (b), (d) output image result for obtaining with the evolution strategy parameter optimization method: this as a result in the corresponding simulating process selected initial parameter be: σ _c=1, σ _s=3, α _c=0.5, α _s=0.5, λ c=22, λ s=19, d=0.06, the used parameter of initialization former generation parameter vector is: σ _c=0.5, σ _s=2.5, α _c=0.6, α _s=0.4, λ c=24, λ s=20, d=0.07.

When using improved population method, choose 6 particles as population, each particle position vector is the parameter vector of space time filter, each position vector contains 63 param elements of 9 space time filters, wherein each space time filter has 7 parameters, σ c, the field range of σ s corresponding space time filter receptive field center of difference and periphery, the corresponding receptive field of α center is to the periphery weight, λ c, λ s corresponding respectively space time filter center and periphery output reach the time of peak value, and the corresponding space time filter periphery of d is to the delay at center.Parameter area is: 0＜σ _c+ 0.4＜σ _s≤ 3,0.5≤α _c＜0.8, α _c+ α _s=1,17≤λ _c≤ 25,14≤λ _s＜λ _c, 0.04≤d≤0.08.At first all particle's velocity and position in the initialization particle population in parameter area.The particle position vector is the space time filter parameter vector.Is the space time filter of 6 particle positions with image x by parameter vector, and the BP network that its output is finished by training again obtains 6 output images.With fitness function all particles are estimated, according to the fitness function more individual extreme value p of each particle and whole extreme value 1 in the new population.Fitness function is Euler's distance of output image and input picture, individual extreme value be single particle from beginning to search the optimal vector of current iteration, whole extreme value is that the particle population is from beginning to search the optimal vector of current iteration.Upgrade particle's velocity and position: v according to formula then _i=error * randn * (p _i-x _i)+error * randn * (1 _i-x _i)+error * randn, x _i(t+1)=x _i(t)+v _i, error is a fitness function, randn is a Gaussian random variable.Iteration is till the iteration stopping condition that satisfies method.As shown in Figure 6, Fig. 6 (a) (c) is two input pictures, (b), and (d) output image result for obtaining with improved population parameter optimization method; This as a result in the corresponding simulating process selected initial parameter be: σ _c=1, σ _s=3, α _c=0.5, α _s=0.5, λ c=22, λ s=19, d=0.06, the used parameter of initialization particle is: σ _c=0.5, σ _s=2.5, α _c=0.6, α _s=0.4, λ c=24, λ s=20, d=0.07.

7. after finding satisfied space time filter parameter through the parameter optimization method, by space time filter, the boost pulse string of its output then is the retina coding of the output image of correspondence with image x.As shown in Figure 7, Fig. 7 (b-d) is for after finding optimized parameter by the parameter optimization method, with the pulse output of the corresponding sample image of 7 (a) by the pairing space time filter of optimized parameter.The designed retina encoder of present embodiment comprises 9 space time filters, corresponding to this image, has 3 space time filters that response is arranged, and all the other 6 space time filters are output not.Fig. 7 (b-d) is the output of 3 space time filters of response.Horizontal ordinate is the time, and ordinate is a pulsed frequency.

As seen from the above-described embodiment, the present invention has used space time filter to simulate the process of retina signal Processing, used evolution strategy and population method to find optimized parameter realizing the enantiomorphic relationship of image and boost pulse string, for the realization of artificial vision prosthesis provides the scrambler basis.

Claims (5)

1, a kind of retina encoder implementation method that adopts space time filter, it is characterized in that, at first sample image is imported space time filter, the output that obtains is as the input of BP artificial neural network, BP ANN is determined the weights of BP artificial neural network; Import any one image in the sample image then, get one group of space time filter parameter at random, in the space time filter parameter area, carry out parameter optimization with population or evolution strategy method, by iteration repeatedly, finally make output image converge on input picture, after the parameter of space time filter was determined, the impulse stimulation of its output then was the retina coding corresponding to input picture;

Described space time filter, directly the output nerve impulsion with input picture and retinal ganglial cells interrelates, and this space time filter has 7 parameters, comprising 3 space time filter time parameter λ _c, λ _s, d, represent that respectively the output of space time filter center and periphery reaches the delay to the center of time of peak value and space time filter periphery, 2 space time filter spatial parameter σ _c, σ _s, represent the field range of space time filter receptive field center and periphery, 2 weighting parameter α respectively _c, α _s, represent receptive field center and periphery weight respectively, its mathematic(al) representation is:

CS(x，t)＝α _cK(t，λ _c)∑{G(x，σ _c)pix(x)}-α _sK(t-d，λ _s)∑{G(x，σ _s)pix(x)}

$K(t,\mathrm{\&lambda;})=\left\{\begin{array}{cc}\mathrm{\&lambda;texp}(-\mathrm{\&lambda;t})& \mathrm{if\; t}\&GreaterEqual;0\\ 0& \mathrm{if\; t}<0\end{array}\right.$

G(x，σ)＝(2πσ ²) ^-1exp(-x ²/2σ ²)

This space time filter peripheral space scope is bigger than central space scope, i.e. σ _c＜σ _sPeriphery postpones to some extent than the center time response time response, i.e. λ _s＜λ _c, d〉and 0;

Describedly in the space time filter parameter area, carry out parameter optimization, be specially: find the optimized parameter vector by error function, error function F (z with the evolution strategy method _i) be Euler's distance of output image and input picture, parameter vector comprises 63 parameters of 9 described space time filters, at first the initial former generation's vector of picked at random z in parameter area _i, i=1 ..., P produces the filial generation vector x by add a zero-mean Gaussian random variable on each element of former generation's vector _i=z _i+ N (0, σ _i), i=1 ..., P, σ _i=F (z _i)/300, the variances sigma of gaussian variable is relevant with error function, can accelerate parameter convergence speed; Relative error function F (z _i), F (x _i), i=1 ..., P, the less vector of Select Error is as the former generation of next iteration, and iteration is till the iteration stopping condition that satisfies method;

Describedly in the space time filter parameter area, carry out parameter optimization with the population method, be specially: realize by fitness function, fitness function is Euler's distance of output image and input picture, the population method is formed population by 6 particles, each particle comprises 63 parameters of 9 space time filters, at first all particle's velocity and position in the initialization particle population in parameter area, with fitness function all particles are estimated, according to the fitness function more individual extreme value p of each particle and whole extreme value 1 in the new population, individual extreme value is that single particle is from beginning to search the optimal vector of current iteration, whole extreme value be the particle population from beginning to search the optimal vector of current iteration, according to the improvement population method formula that combines by traditional population method and evolution strategy particle rapidity and position are carried out iteration: v then _i=error * randn * (p _i-x _i)+error * randn * (l _i-x _i)+error * randn, x _i(t+1)=x _i(t)+v _i, error is the output image that draws according to fitness function and the error of original image, and randn is a Gaussian random variable, and iteration is till the iteration stopping condition that satisfies method.

2, the retina encoder implementation method of employing space time filter according to claim 1, it is characterized in that, described space time filter, adopt 9 altogether, retinal ganglial cells of each space time filter simulation, the receptive field of 9 described space time filters can be overlapping, and the image of 729 picture elements is handled, and the input picture of 729 picture elements is converted to pulse output.

3, the retina encoder implementation method of employing space time filter according to claim 1, it is characterized in that, described BP artificial neural network is used to simulate brain and handles visual signal, the BP artificial neural network composition that haves three layers, be input layer, hidden layer and output layer, input layer has 279 neurons, and corresponding to the train of impulses of space time filter output, hidden layer has 35 neurons, 729 neurons of output layer are corresponding to the output image of 729 picture elements.

4, the retina encoder implementation method of employing space time filter according to claim 1 is characterized in that, described parameter area when carrying out parameter optimization with the evolution strategy method in the space time filter parameter area is: 0＜σ _c+ 0.1＜σ _s≤ 3,0.5≤α _c＜1, α _c+ α _s=1,17≤λ _s＜λ _c≤ 25,0.04≤d≤0.08.

5, the retina encoder implementation method of employing space time filter according to claim 1 is characterized in that, described parameter area when carrying out parameter optimization with the population method in the space time filter parameter area is: 0＜σ _c+ 0.4＜σ _s≤ 3,0.5≤α _c＜0.8, α _c+ α _s=1,17≤λ _c≤ 25,14≤λ _s＜λ _c, 0.04≤d≤0.08.

Images