US5033103A - Model of the lateral inhibition, energy normalization, and noise suppression processes in the retina - Google Patents
Model of the lateral inhibition, energy normalization, and noise suppression processes in the retina Download PDFInfo
- Publication number
- US5033103A US5033103A US07/283,114 US28311488A US5033103A US 5033103 A US5033103 A US 5033103A US 28311488 A US28311488 A US 28311488A US 5033103 A US5033103 A US 5033103A
- Authority
- US
- United States
- Prior art keywords
- video image
- inhibition
- retina
- cell
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 210000001525 retina Anatomy 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000008569 process Effects 0.000 title claims abstract description 34
- 238000010606 normalization Methods 0.000 title claims abstract description 18
- 230000001629 suppression Effects 0.000 title claims abstract description 13
- 230000023886 lateral inhibition Effects 0.000 title claims description 7
- 230000005764 inhibitory process Effects 0.000 claims abstract description 75
- 238000012545 processing Methods 0.000 claims abstract description 31
- 230000002207 retinal effect Effects 0.000 claims abstract description 31
- 210000004027 cell Anatomy 0.000 claims description 80
- 230000005284 excitation Effects 0.000 claims description 30
- 230000000694 effects Effects 0.000 claims description 21
- 210000003370 receptor cell Anatomy 0.000 claims description 19
- 208000003098 Ganglion Cysts Diseases 0.000 claims description 5
- 208000005400 Synovial Cyst Diseases 0.000 claims description 5
- 210000000170 cell membrane Anatomy 0.000 claims description 4
- 239000012528 membrane Substances 0.000 claims description 3
- 238000004590 computer program Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 24
- 241001529572 Chaceon affinis Species 0.000 description 19
- 230000004044 response Effects 0.000 description 19
- 241000251539 Vertebrata <Metazoa> Species 0.000 description 15
- 230000002401 inhibitory effect Effects 0.000 description 15
- 238000005286 illumination Methods 0.000 description 11
- 238000003909 pattern recognition Methods 0.000 description 11
- 241000239218 Limulus Species 0.000 description 9
- 238000012360 testing method Methods 0.000 description 9
- 238000012935 Averaging Methods 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 7
- 210000002287 horizontal cell Anatomy 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 3
- 230000036755 cellular response Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000001965 increasing effect Effects 0.000 description 3
- 210000004126 nerve fiber Anatomy 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 241001465754 Metazoa Species 0.000 description 2
- 241000146339 Necturus maculosus Species 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000002964 excitative effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 238000002310 reflectometry Methods 0.000 description 2
- 241000269333 Caudata Species 0.000 description 1
- 101001022148 Homo sapiens Furin Proteins 0.000 description 1
- 101000701936 Homo sapiens Signal peptidase complex subunit 1 Proteins 0.000 description 1
- 241000146341 Necturus Species 0.000 description 1
- XOJVVFBFDXDTEG-UHFFFAOYSA-N Norphytane Natural products CC(C)CCCC(C)CCCC(C)CCCC(C)C XOJVVFBFDXDTEG-UHFFFAOYSA-N 0.000 description 1
- 241001290864 Schoenoplectus Species 0.000 description 1
- 102100030313 Signal peptidase complex subunit 1 Human genes 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000004382 visual function Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
Definitions
- the present invention relates generally to image processing systems, and more specifically to a system which models the lateral inhibition, energy normalization, and noise suppression processes in a generic vertebrate retina.
- This retinal model enhances edges, eliminates brightness variations in a scene (image) for a given object and background, and suppresses noise to the extent that objects (signals) are extracted from noise in an image processing system.
- correlator as a model of the human visual system may be important because the human brain may be correlating visual data as part of the pattern recognition process. If correlators actually exist in the brain, other elements must be present to eliminate some of the shortcomings that correlators exhibit.
- the correlator problems considered here are uncontrolled illumination, object reflectivity and noise.
- Tamches patent discloses an analog electronic system for preprocessing an optical pattern in a spatially modulated scene.
- the other patents disclose image and signal processing systems and optical pattern recognition systems.
- the present invention is greatly indebted to the above-cited Werblin reference which discloses retinal electrical characteristics including receptor cell response ranges and response amplitudes. While Werblin is excellent in its documentation of the electrical responses of retinal cells and retinal sensitivity, it does not apply this knowledge to image processing systems.
- Carver A. Mead et al describe a set of analog VLSI retina chips which are used in photoreception and processing. While it is encouraging to find such artificial vision systems being modeled on biological vision processes, the task remains to provide a digital image processing system which models the lateral inhibition, energy normalization, and noise suppression processes in a generic vertibrate retina. The present invention is intended to satisfy that need.
- the present invention is a system which models the lateral inhibition, energy normalization, and noise suppression processes in a generic vertebrate retina.
- the vertebrate retinal model enhances edges, eliminates brightness variations in a scene (image) for a given object (in the form of a signal) and background, and suppresses noise to the extent that objects (in the form of signals) are extracted from noise.
- One particular embodiment of the present invention accomplishes these functions in an image processing system which uses: a video camera, a monitor, a computer, a Micro-Vax Intech board, and a Tektronics hardcopy unit.
- the video camera is electrically connected to the computer by the Micro-Vax Intech board, which acts as a computer interface and digitizes the signals from the camera.
- the computer performs the image processing functions discussed above using an original program to allow the monitor and hardcopy unit to display improved images.
- FIG. 1 is a block diagram of one embodiment of the present invention
- FIG. 2 is an illustration of the electrically responsive area of retinal cells
- FIG. 3 is a block diagram of the steps of the process of the present invention.
- FIGS. 4 and 5 are illustrations that represent the inhibition region and excitation region of a retinal cell.
- the present invention includes an image processing system which models characteristics of a vertibrate retina in order to: enhance image edges, eliminate brightness variations in a scene (image) for a given object and background, and suppress noise in an image.
- a correlator compares patterns using a well defined mathematical process to decide whether the patterns are similar.
- the correlator problems experienced by systems prior to the present invention include uncontrolled illumination, object reflectivity and noise. Based mostly on the work of Werblin, the present invention models the inhibition, energy normalization, and noise suppression processes in the retina to eliminate these correlator problems.
- FIG. 1 is a block diagram of an embodiment of the present invention.
- the system of FIG. 1 is an image processing system which uses: a video camera 100, a Micro-Vax interface board 110, a Micro-Vax II computer 120, a monitor 130 and a hardcopy printer unit 150 in order to process and display enhanced images using some of the image processing features of a biological retina as information from an article by Frank S. Werblin entitled "The Control of Sensitivity in the Retina" published in the Scientific American, Vol 228 p 70-79.
- the system of FIG. 1 is just one example of the application of the present invention, and uses equipment which is commercially-available.
- this version of the system included a Micro-Vax Intech board, Dage 650 camera, Deanza video monitor, and a Tektronics hardcopy unit.
- the video camera 100 is electrically connected to the computer 120 by the Micro-Vax Intech board 110, which acts as a computer interface and digitizes the signals from the camera.
- the computer 120 performs the image processing functions discussed above using an original program to allow the monitor and hardcopy unit to display improved images.
- the retina enhances edges (done through the inhibition process), minimizes brightness variations in a scene (energy normalization) and suppresses noise to help ensure that patterns are successfully identified.
- a device emulating these retinal processes can be used as a preprocessor to a correlator or other pattern recognition device to improve recognition performance.
- a retinal-type processessor can also be used to extract signals (for example communication signals) from noise.
- a computer model of the horseshoe crab eye (presented below as Table 2) was first developed to better understand the general nature of inhibition processes in distributed image processors. A computer program was then developed to model the vertebrate retina so that later a silicon chip circuit or optical device could be developed and implemented as a preprocessor to a pattern recognition system.
- a subroutine was first developed to allow the user to create sub-retinal regions or blocks (a block represents a cell in the vertebrate retina or an omatidium (12 cells that function as a processing unit) in the horseshoe crab eye) of different sizes so that different sized retinas could be tested while using camera images or stored images of 512 ⁇ 480 pixels.
- Computer code to do lateral inhibition as performed by the horseshoe crab was developed next. Code was also written to model inhibition, energy normalization and noise suppression in the vertebrate retina. Additionally an energy normalization subroutine, written to work independently of the inhibition process, is given in Table 3. Each of these subroutines was tested separately. Up to three cell layers can be used depending on the retina of interest. For the vertebrate retina three cell layers were modeled and in the horseshoe crab eye, one layer was used.
- the human visual system can recognize objects over a wide range of illumination levels.
- the retina of the eye helps this function by a complex preprocessing operation.
- the pupil of the eye unlike the aperture of a camera, plays a small role in this process.
- the retina sends data over one million nerve fibers to the brain, and each nerve fiber handles a different part of the visual field.
- the retina compresses the range of intensities it receives before sending the information over these nerve fibers.
- Werblin studied the retina of the mudpuppy (salamander, Necturus maculosas) because it was easy to probe and because it represents a generic vertebrate retina.
- Werblin flashed a spot of light, as shown in FIG. 2, on the retina of a mudpuppy to provide an input stimuli to enable measurements of the behavior of each type of retinal cell.
- Werblin then stimulated the retina with a spot of light with a ring of light around it as shown in FIG. 2. He repeated his steps for each cell type.
- Werblin discovered that each neuron type responded differently. When Werblin illuminated a receptor cell, the voltage across the membrane of the bipolar cell connected to the receptor cell became more negative with respect to a reference electrode attached to the animal (hyperpolarized).
- Ratliff found that the closer an inhibitory omatidium was the more of an inhibitory effect it had on the test cell.
- Ratliff in his experiments on the Limulus eye derived an equation that closely predicts the effects of inhibition on an omatidium by its neighbors. Sillart implementation of Ratliff's equation is shown below:
- R Response of the inhibiting cell.
- Ratliff determined that the inhibitory coefficient decreases in value the farther away an inhibiting cell is. Ratliff also determined that the threshold of an inhibitory cell increases in value the farther it is from the cell to be inhibited. In general, Ratliff found:
- Ratliff was unable to determine an equation for the coefficient of inhibitory action and the threshold frequency. In determining the above equation Ratliff assumed that neighboring cells were separated far enough apart to ignore the inhibitory effects on each other yet close enough to the test cell to influence it significantly.
- Tables 1-3 are three programs which may be run by the computer 120 of FIG. 1 as models of retinal processes.
- Table 1 lists a software model for a vertibrate retina;
- Table 2 is a software model for the retina of a horseshoe crab (Limulus);
- Table 3 is the listing of an energy normalization software program. The rationale behind these programs is presented below, beginning with a discussion of the model of inhibition in the horseshoe and inhibition and excitation by the retina, and a comparison of it with the features of Table 1. ##SPC1##
- the user selects images to process, an omatidium size (block size) (16 to match the parameter of the Limulus eye, and an inhibition region (a square region for the human eye and 31 ⁇ 9 for Limulus).
- the user can display either a camera image or a stored image on a monitor. If a stored image is selected, the user can display a square on a pristine background, squares added to noise, vertical bars of increasing intensity or a real image.
- the subroutine CREATE divides the image into blocks of size [bksize ⁇ bksize] (for the horseshoe crab: [16 ⁇ 16]) where each block represents an omatidium in a crab.
- One option in the invention is as follows. The average intensity for each block is calculated and all pixels in a block are assigned this intensity value. The intensity value Of a block represents the average light intensity shining on that omatidium.
- the user is then asked to input the maximum inhibition region in both the x and y directions ([31 ⁇ 9] for the horseshoe crab) and the inhibition region is calculated for each omatidium on the monitor screen.
- the inhibition region for each omatidium will vary as a function of the omatidium's location on the monitor screen. For example, the inhibition region for the top left omatidium has no omatidia above it or to the left of its center.
- Prevout previous omatidium output
- Nframes The Number Of Frames Processed (present and past)
- K in equation 3 combines the threshold, which increases as a function of distance, and the inhibitory coefficient, which decreases as a function of distance.
- the two dimensional Gaussian function shown in equation 4 was used to model the distance effects of K.
- a Gaussian function was used because the weighting function is dome-like in appearance.
- the energy normalization phenomenon is not observed in the horseshoe crab because the horseshoe crab has no excitatory summation function, only latent inhibition; therefore, the energy normalization effect was not included in K.
- the variance used in the Gaussian function is dependent on the size of the inhibition region input by the user. The variance was calculated in this manner because it was assumed that lateral connection between omatidia are all the same.
- the inhibited intensity value is stored in the array RESP before repeating the inhibition process for the remaining omatidia. After processing the scene through inhibit, the inhibited intensity value (one point of the transient response) for each block is displayed. If inhibit is run over a series of input frames and the image does not change over several of the frames, a steady-state intensity value will be seen.
- TRY A computer program called TRY was developed as part of this invention to simulate the energy normalization, inhibition, and noise suppression processes in the vertebrate retina.
- the user selects images to process, a cell size (blocksize), an excitation region and an inhibition region.
- the user can display either a camera image or one of several stored images as described in the previous discussion of the horseshoe crab.
- the subroutine CREATE as described above for the horseshoe crab is used to create and display receptor cell blocks on a monitor if bksize was not set equal to one.
- the original image or the image resulting from the subroutine CREATE is then processed by the subroutine INHIBIT.
- the intensity values displayed on the monitor using the INHIBIT subroutine simulates the changes in membrane potential across every receptor cell, bipolar cell, or ganglion cell in the retina (depending on user choice).
- the equation used to calculate the potential across a retinal cell membrane is given discussed below (Werblin; 1974:67):
- the inhibition region is an area which surrounds a receptor cell and dampens its electrooptic effect on adjacent cells when it is stimulated by the reception of an illuminating signal thereby allowing contrasts of light and dark images to be detected.
- the excitation region is an area which surrounds a receptor cell and increases its electrooptic effect on adjacent cells.
- FIGS. 4 and 5 are illustrations of a cell which is a pixel or a block surrounded by an inhibition region and an excitation region.
- "I” is the average intensity on the excitation region, of the cell
- "K” the average intensity on the inhibition region of the cell.
- V a measure of intensity displayed on the screen and measured in volts.
- Rmax Vmax which equals the maximum intensity displayable on the screen.
- I the average intensity on the excitation region (excitation center) of the cell
- K average intensity on the inhibition region (inhibition surround) of a cell
- n measure of the steepness of a retinal cell inhibition curve
- n is between 0.7-1.0 for the receptor cells
- n is between 1.4-3.0 for the bipolar cells
- n is between 3.0-4.0 for the ganglion cells
- n is may be selected to be any number in other applications.
- the right most square has an intensity value of three.
- the center square has an intensity value of five.
- the left most square has an intensity value of seven.
- Hardwire connections can be used to connect silicon chips used to implement equation 5.
- connections between silicon chips can be done optically.
- the extraction of intensity information from the excitation and inhibition region for each pixel in an image can be done in parallel or in a serial fashion.
- alternative 2 would be the easiest to implement, especially if the extraction of intensity information from the excitation and inhibition regions of each cell block was extracted serially.
- a mirror of some lens system could perform the task of extracting intensity information serially from an image.
- equation 5 is implemented in hardware it can then be used as a preprocessor to a correlator or other pattern recognition system. As noted above, the present invention digitally implements equation 5.
- equation 5 should be equated to a modified version of equation 2 (horseshoe crab equation) and solved for K (horseshoe crab equation) for different combinations of the parameters in equation 5.
- K horseshoe crab equation
- a retinal processor can be used on other signals (i.e., communication signals).
- the program in Table 1 can be modified to process only one line of data.
- the average intensity for each inhibition; excitation, and averaging region was calculated using this one line of data. It was determined during the course of this invention that averaging a scene is not needed if the parameters in equation 5 are chosen properly.
- the parameters used to show the effects of averaging are as follows:
- the inhibition region is 105 times wider than the excitation region
- the averaging region is 3 times wider than the excitation region.
- this program was modified to process all 512 lines of pixels using the inhibition regions as calculated above. Using this technique reduced the processing of images on the Micro-Vax II took about 30 hours for 512 lines of pixels, and approximately one hour for 512 lines of pixels processed along one dimension.
- FIG. 3 is a block diagram of the steps that the user of the system of FIG. 1 would follow.
- the process begins with the image reception step 301, in which the computer 120 of FIG. 1 receives an image from the camera 100 via the Micro-Vax interface 110.
- the user selects a block size 310 on the computer 120 that replicates the omatidium of the Limuls or the retinal cell of the retina being modeled.
- a suitable Limulus block size would be 16, while a suitable block size to model a human retinal cell would be 1.
- the inhibition region size and in the case of the vertibrate retina is selected 320.
- the inhibition region is an area which surrounds the omatidium which dampens its electrooptic effect on adjacent cells when it is stimulated by the reception of an illuminating signal.
- the presence of an inhibition region allows sharp light and dark contrasts to be detected by the retina, and its size should reflect the type of retina being modeled.
- the excitation region is an area which surrounds a receptor cell and increases its electrooptic effect on adjacent cells.
- a suitable inhibition region size for a Limulus retina model would be 31 ⁇ 9.
- the inhibition region should be a square area, and occupy about 90% of the number of pixels in the image.
- the received image is divided in step 330 into blocks of a size where each block represents an omatidium of the limulus, or retinal cell of the area selected in step 310.
- one option of the invention is then to compute the average intensity of each block, and assign all pixels of this block to average intensity value 340.
- the advantages and disadvantages of this averaging step have been discussed above, and the user may experiment with this step to see if it improves the image processing for his particular application.
- the advantage of this digital image processing system is its extreme flexibility in allowing users to adjust all parameters.
- the computer 120 will execute the inhibit function on the values of each block as defined above in equation 3.
- the result is an output of the inhibited response of each omatidium 350.
- the computer 120 will store the resulting inhibited intensity values to produce a combined processed output 360, and direct the monitor 130 of FIG. 1 to display the combined processed output 370.
- the inhibit subroutine might use equation 5 to simulate the change in potential across every receptor cell, instead of using equation 3.
- the processed image might also be passed through the inhibit subroutine three times.
- the inhibit function generally tends to increase the contrast between light and dark in the processed image. Note that the inhibit function used in the vertebrate computer program performs more than the inhibit function.
- the retina model uses the Weber equation (Equation 5) to determine the response of a cell (represented by a pixel or a block of pixels).
- the retina model described above does not take in account the effects of distance as was done with the Limulus (horseshoe crab) model. This resulted in an average value being used in the Weber equation that might have been lower or higher than the "correct" value. Some cell blocks were assigned a value higher than desired. This is believed to be the main cause of blurring in test images used during this effort. The new retinal model will take this into account. Since the system is using a constant density eye it will be assumed that the weighting between any two cells is equal to alpha where 0 ⁇ alpha ⁇ 1.
- the model described above also assumes an inhibition region that varies only as a function of a pixel or block of pixels position on the screen.
- the size of the inhibition region near the edge should be larger than that used if it is assumed that the attenuation effects are a function of only the distance.
- the above-described model assumed that the full field illumination region was the same no matter what illumination level was used. It is now believed that the region of full field illumination should increase as the intensity level of the inhibition region is increased (taken care of by weighting the intensity of pixels in the inhibition region of a cell block which varies as a function of distance).
- WHERE I Average intensity of a cell block or the average intensity of the cell block and that of cells in adjacent regions (Excitation region).
- PIXINT Average intensity of a cell block in the inhibition region.
- the only way to use the Weber equation is to equate a response with an average intensity value in an inhibition region.
- the only way to use the Weber equation is to equate a response with an average intensity value in an inhibition region.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Biodiversity & Conservation Biology (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
A digital image processing system is disclosed that uses a camera, a monitor and a computer which models the inhibition, energy normalization, and noise suppression processes in a generic retina. The retinal computer program enhances edges, eliminates brightness variations in a scene for a given object and background, and suppresses noise to the extent that objects were extracted from noise. The model uses many parameters which can be extensively adjusted in an attempt to achieve the optimum image.
Description
The invention described herein may be manufactured and used by or for the Government for governmental purposes without the payment of any royalty thereon.
The present invention relates generally to image processing systems, and more specifically to a system which models the lateral inhibition, energy normalization, and noise suppression processes in a generic vertebrate retina. This retinal model enhances edges, eliminates brightness variations in a scene (image) for a given object and background, and suppresses noise to the extent that objects (signals) are extracted from noise in an image processing system.
Mankind has tried for centuries to understand data acquisition and perception processes in animal visual systems. Just as the optical elements of the earliest cameras were modeled on their counterparts in the human eye (lens, iris, and retina etc), modern image processing systems can still be improved on the subtler aspects of the vertibrate retina. Accordingly, an effort has been to construct machines to perform some visual function with specific regard to its realization by living systems. For example, pattern recognition has been investigated for many decades. One reasonably successful method to do pattern recognition is based on correlation. A correlator compares patterns using a well defined mathematical process to decide whether the patterns are similar.
Studying the correlator as a model of the human visual system may be important because the human brain may be correlating visual data as part of the pattern recognition process. If correlators actually exist in the brain, other elements must be present to eliminate some of the shortcomings that correlators exhibit. The correlator problems considered here are uncontrolled illumination, object reflectivity and noise.
The task of enhancing pattern recognition systems with an image processing system based on a model of the human retina is alleviated, to some extent, by the systems described in the following U.S. Patents, the disclosures of which are incorporated herein by reference:
______________________________________ U.S. Pat. No. 3,964,021 issued to Tamches; U.S. Pat. No. 4,318,083 issued to Argyle; U.S. Pat. No. 4,716,312 issued to Mead et al. U.S. Pat. No. 3,016,518 issued to Taylor; U.S. Pat. No. 3,088,096 issued to Steinbuch; U.S. Pat. No. 3,187,304 issued to Taylor; and U.S. Pat. No. 3,701,095 issued to Yamaguchi et al. ______________________________________
Perhaps the most significant of the above-cited patents is the Tamches patent, which discloses an analog electronic system for preprocessing an optical pattern in a spatially modulated scene. The other patents disclose image and signal processing systems and optical pattern recognition systems.
The approach of making an electronic model of a vertibrate retina is more fully explored in the following four technical articles, the disclosures of which are specifically incorporated herein by reference:
an article by Frank S. Werblin entitled, "The Control of Sensitivity in the Retina," published in Scientific American, 228: p. 70-79 on January 1973; an article by K. Fukushima entitled "An Electronic Model of the Retina" published in Proc. of the I.E.E.E. on December 1987, p. 1950-1951; an article by Carver A. Mead et al. entitled "Real-Time Visual Computations Using Analog CMOS Processing Arrays" dated November 1987; and an article by K. Fukushima entitled "Visual Feature Extraction by a Multilayered Network of Analog threshold Elements" published in I.E.E.E. Trans. Syst. Sci. Cybernetics, Vol SSC-5 pp 322-333 on October 1969.
The present invention is greatly indebted to the above-cited Werblin reference which discloses retinal electrical characteristics including receptor cell response ranges and response amplitudes. While Werblin is excellent in its documentation of the electrical responses of retinal cells and retinal sensitivity, it does not apply this knowledge to image processing systems.
Both of the Fukushima articles discuss analog electronic models of a vertibrate retina and are exemplary in the art. While these analog systems are state of the art feature extraction and pattern recognition systems, a digital image processing system would have more adjustable flexibility than these systems, and would be an advance in the art.
Carver A. Mead et al describe a set of analog VLSI retina chips which are used in photoreception and processing. While it is encouraging to find such artificial vision systems being modeled on biological vision processes, the task remains to provide a digital image processing system which models the lateral inhibition, energy normalization, and noise suppression processes in a generic vertibrate retina. The present invention is intended to satisfy that need.
The present invention is a system which models the lateral inhibition, energy normalization, and noise suppression processes in a generic vertebrate retina. The vertebrate retinal model enhances edges, eliminates brightness variations in a scene (image) for a given object (in the form of a signal) and background, and suppresses noise to the extent that objects (in the form of signals) are extracted from noise. One particular embodiment of the present invention accomplishes these functions in an image processing system which uses: a video camera, a monitor, a computer, a Micro-Vax Intech board, and a Tektronics hardcopy unit. The video camera is electrically connected to the computer by the Micro-Vax Intech board, which acts as a computer interface and digitizes the signals from the camera. The computer performs the image processing functions discussed above using an original program to allow the monitor and hardcopy unit to display improved images.
It is a principal object of the present invention to provide a method to enhance edges, minimize brightness variations for a given object and background in a scene and suppress noise to ensure that patterns can be successfully identified by a correlator or other pattern recognition device.
It is another object of the invention to provide an image preprocessing system that digitally enhances detected images using some of the functions found in a vertibrate retina.
These objects together with other objects, features and advantages of the invention will become more readily apparent from the following detailed description when taken in conjunction with the accompanying drawings wherein like elements are given like reference numerals throughout.
FIG. 1 is a block diagram of one embodiment of the present invention;
FIG. 2 is an illustration of the electrically responsive area of retinal cells;
FIG. 3 is a block diagram of the steps of the process of the present invention; and
FIGS. 4 and 5 are illustrations that represent the inhibition region and excitation region of a retinal cell.
The present invention includes an image processing system which models characteristics of a vertibrate retina in order to: enhance image edges, eliminate brightness variations in a scene (image) for a given object and background, and suppress noise in an image.
As mentioned above, a correlator compares patterns using a well defined mathematical process to decide whether the patterns are similar. The correlator problems experienced by systems prior to the present invention include uncontrolled illumination, object reflectivity and noise. Based mostly on the work of Werblin, the present invention models the inhibition, energy normalization, and noise suppression processes in the retina to eliminate these correlator problems.
The reader's attention is now directed towards FIG. 1 which is a block diagram of an embodiment of the present invention. The system of FIG. 1 is an image processing system which uses: a video camera 100, a Micro-Vax interface board 110, a Micro-Vax II computer 120, a monitor 130 and a hardcopy printer unit 150 in order to process and display enhanced images using some of the image processing features of a biological retina as information from an article by Frank S. Werblin entitled "The Control of Sensitivity in the Retina" published in the Scientific American, Vol 228 p 70-79. The system of FIG. 1 is just one example of the application of the present invention, and uses equipment which is commercially-available. For example, this version of the system included a Micro-Vax Intech board, Dage 650 camera, Deanza video monitor, and a Tektronics hardcopy unit. The video camera 100 is electrically connected to the computer 120 by the Micro-Vax Intech board 110, which acts as a computer interface and digitizes the signals from the camera. The computer 120 performs the image processing functions discussed above using an original program to allow the monitor and hardcopy unit to display improved images.
Before proceeding to a description of the software programs, it is important for the reader to understand the characteristics of the retina. Information from the above-cited Werblin article as well as that of chapter 11 of the text edited by R. A. Rosenblith entitled "Sensory Communication" published on July 19, 1959, are included in the discussion that follows in order to acquaint the reader with these features. Virtually all of the information that follows was the subject of a master's degree thesis by the inventor which has been cataloged and retained by AFIT (the Air Force Institute of Technology) as AFIT/GE/ENG 87D-59, the disclosure of which is incorporated herein by reference. The retina enhances edges (done through the inhibition process), minimizes brightness variations in a scene (energy normalization) and suppresses noise to help ensure that patterns are successfully identified. A device emulating these retinal processes can be used as a preprocessor to a correlator or other pattern recognition device to improve recognition performance. A retinal-type processessor can also be used to extract signals (for example communication signals) from noise. These are only two of the many possible applications for such a device.
As part of this invention software was developed to emulate the inhibition, energy normalization, and noise suppression processes. One computer program modeled the eye of an invertebrate, the horseshoe crab (Limulus polyhemus) and the other modeled the generic vertebrate retina. The processing of color was not included in either model. The horseshoe crab model considered the inhibition process in the horseshoe crab eye. The vertebrate model (presented below as Table 1) considered inhibition, energy normalization, and noise suppression. Additionally, both programs model eye behavior consequent to motion or a change in illumination.
A computer model of the horseshoe crab eye (presented below as Table 2) was first developed to better understand the general nature of inhibition processes in distributed image processors. A computer program was then developed to model the vertebrate retina so that later a silicon chip circuit or optical device could be developed and implemented as a preprocessor to a pattern recognition system.
A subroutine was first developed to allow the user to create sub-retinal regions or blocks (a block represents a cell in the vertebrate retina or an omatidium (12 cells that function as a processing unit) in the horseshoe crab eye) of different sizes so that different sized retinas could be tested while using camera images or stored images of 512×480 pixels. Computer code to do lateral inhibition as performed by the horseshoe crab was developed next. Code was also written to model inhibition, energy normalization and noise suppression in the vertebrate retina. Additionally an energy normalization subroutine, written to work independently of the inhibition process, is given in Table 3. Each of these subroutines was tested separately. Up to three cell layers can be used depending on the retina of interest. For the vertebrate retina three cell layers were modeled and in the horseshoe crab eye, one layer was used.
The human visual system can recognize objects over a wide range of illumination levels. The retina of the eye helps this function by a complex preprocessing operation. The pupil of the eye, unlike the aperture of a camera, plays a small role in this process. The retina sends data over one million nerve fibers to the brain, and each nerve fiber handles a different part of the visual field. The retina compresses the range of intensities it receives before sending the information over these nerve fibers.
Werblin studied the retina of the mudpuppy (salamander, Necturus maculosas) because it was easy to probe and because it represents a generic vertebrate retina. Werblin flashed a spot of light, as shown in FIG. 2, on the retina of a mudpuppy to provide an input stimuli to enable measurements of the behavior of each type of retinal cell. Werblin then stimulated the retina with a spot of light with a ring of light around it as shown in FIG. 2. He repeated his steps for each cell type. Werblin discovered that each neuron type responded differently. When Werblin illuminated a receptor cell, the voltage across the membrane of the bipolar cell connected to the receptor cell became more negative with respect to a reference electrode attached to the animal (hyperpolarized). However, when neighboring receptor cells were illuminated the bipolar cell membrane became more positive (depolarized). Werblin concluded that the horizontal cells inverted the bipolar cell response when they were activated by the ring stimulus. When the receptor cells were illuminated by the spot stimulus, they responded strongly, but when neighboring receptors were illuminated with the spot/ring pattern they responded poorly. Werblin concluded that the horizontal cells influenced the behavior of the receptor cells. To further support his theory of the receptor/horizontal cell connection and bipolar/horizontal cell connection, Werblin showed that the horizontal cells responded strongly to both the spot and spot/ring stimuli. He also showed that when the spot stimulus was shown, the sustained ganglion cell responsed strongly. When the spot/ring stimulus was used, the response of the cell was minimal.
Floyd Ratliff discovered that a test eccentric cell was inhibited (discharged fewer pulses and at a lower rate) when neighboring omatidia were illuminated. Ratliff determined that the amount of inhibition was a function of the following three parameters:
1. The number of neighboring omatidia illuminated.
2. The intensity of illumination on the neighboring omatidium.
3. The distance of illuminated omatidia from the test cell.
Ratliff found that the closer an inhibitory omatidium was the more of an inhibitory effect it had on the test cell.
Ratliff in his experiments on the Limulus eye derived an equation that closely predicts the effects of inhibition on an omatidium by its neighbors. Sillart implementation of Ratliff's equation is shown below:
Inhibited response=E-K(R-threshold of neighbor) (1)
where:
E=the response of the cell being processed;
K=Inhibitory coefficient; and
R=Response of the inhibiting cell.
Ratliff determined that the inhibitory coefficient decreases in value the farther away an inhibiting cell is. Ratliff also determined that the threshold of an inhibitory cell increases in value the farther it is from the cell to be inhibited. In general, Ratliff found:
______________________________________ Inhibited Response = E - Sum of effects from (2) each inhibiting cell where E in Eg. 2 is the response of the cell being processed. ______________________________________
Ratliff was unable to determine an equation for the coefficient of inhibitory action and the threshold frequency. In determining the above equation Ratliff assumed that neighboring cells were separated far enough apart to ignore the inhibitory effects on each other yet close enough to the test cell to influence it significantly.
Early on the INHIBIT subroutine used the inhibition equation shown in Table 3. The variance used in Table 3 is in error; therefore, a variance dependent on the maximum inhibition region selected by the user was developed. Also to speed the inhibition calculation up, a look up table (in Table 2) was developed to hold the Gaussian weighting used to weight the intensities of the inhibiting omatidia. An error exists in the horseshoe crab program; therefore, no results were generated. The error is believed to exist in either calculating the Gaussian weightings or in the INHIBIT subroutine itself because all the other subroutines were extensively tested and are used by the vertebrate retinal software.
Tables 1-3, as presented here, are three programs which may be run by the computer 120 of FIG. 1 as models of retinal processes. Table 1 lists a software model for a vertibrate retina; Table 2 is a software model for the retina of a horseshoe crab (Limulus); and Table 3 is the listing of an energy normalization software program. The rationale behind these programs is presented below, beginning with a discussion of the model of inhibition in the horseshoe and inhibition and excitation by the retina, and a comparison of it with the features of Table 1. ##SPC1##
At the beginning of the program of Table 2, the user selects images to process, an omatidium size (block size) (16 to match the parameter of the Limulus eye, and an inhibition region (a square region for the human eye and 31×9 for Limulus). The user can display either a camera image or a stored image on a monitor. If a stored image is selected, the user can display a square on a pristine background, squares added to noise, vertical bars of increasing intensity or a real image.
The subroutine CREATE divides the image into blocks of size [bksize×bksize] (for the horseshoe crab: [16×16]) where each block represents an omatidium in a crab. One option in the invention is as follows. The average intensity for each block is calculated and all pixels in a block are assigned this intensity value. The intensity value Of a block represents the average light intensity shining on that omatidium.
The user is then asked to input the maximum inhibition region in both the x and y directions ([31×9] for the horseshoe crab) and the inhibition region is calculated for each omatidium on the monitor screen.
The inhibition region for each omatidium will vary as a function of the omatidium's location on the monitor screen. For example, the inhibition region for the top left omatidium has no omatidia above it or to the left of its center. The affect of the inhibitory omatidia within an inhibition region is determined by implementing a modified version of equation 2 as follows: ##EQU1## where I=Intensity on the cell to be inhibited
Sumint=Sum of intensities from each omatidium
Prevout=previous omatidium output
Nframes=The Number Of Frames Processed (present and past)
K=Equation 4 as follows: ##EQU2## where distx=distance in the x direction between the cell being inhibited and inhibiting cell
disty=distance in the y direction between the cell being inhibited and inhibiting cell
var=variance in the y direction for the maximum inhibition region
The value of K in equation 3 combines the threshold, which increases as a function of distance, and the inhibitory coefficient, which decreases as a function of distance. The two dimensional Gaussian function shown in equation 4 was used to model the distance effects of K. A Gaussian function was used because the weighting function is dome-like in appearance. The energy normalization phenomenon is not observed in the horseshoe crab because the horseshoe crab has no excitatory summation function, only latent inhibition; therefore, the energy normalization effect was not included in K. The variance used in the Gaussian function is dependent on the size of the inhibition region input by the user. The variance was calculated in this manner because it was assumed that lateral connection between omatidia are all the same.
After an omatidium block is inhibited, the inhibited intensity value is stored in the array RESP before repeating the inhibition process for the remaining omatidia. After processing the scene through inhibit, the inhibited intensity value (one point of the transient response) for each block is displayed. If inhibit is run over a series of input frames and the image does not change over several of the frames, a steady-state intensity value will be seen.
A computer program called TRY was developed as part of this invention to simulate the energy normalization, inhibition, and noise suppression processes in the vertebrate retina.
At the beginning of TRY, the user selects images to process, a cell size (blocksize), an excitation region and an inhibition region. The user can display either a camera image or one of several stored images as described in the previous discussion of the horseshoe crab.
The subroutine CREATE as described above for the horseshoe crab is used to create and display receptor cell blocks on a monitor if bksize was not set equal to one. The original image or the image resulting from the subroutine CREATE is then processed by the subroutine INHIBIT. The intensity values displayed on the monitor using the INHIBIT subroutine simulates the changes in membrane potential across every receptor cell, bipolar cell, or ganglion cell in the retina (depending on user choice). The equation used to calculate the potential across a retinal cell membrane is given discussed below (Werblin; 1974:67):
The inhibition region is an area which surrounds a receptor cell and dampens its electrooptic effect on adjacent cells when it is stimulated by the reception of an illuminating signal thereby allowing contrasts of light and dark images to be detected.
The excitation region is an area which surrounds a receptor cell and increases its electrooptic effect on adjacent cells.
FIGS. 4 and 5 are illustrations of a cell which is a pixel or a block surrounded by an inhibition region and an excitation region. In FIGS. 4 and 5, "I" is the average intensity on the excitation region, of the cell, and "K" the average intensity on the inhibition region of the cell. In the present invention a digital image processing system is used in which the equation used to calculate the potential across a retinal cell membrane is given as follows: ##EQU3## where: V=a measure of intensity displayed on the screen and measured in volts.
Rmax=Vmax which equals the maximum intensity displayable on the screen.
I=the average intensity on the excitation region (excitation center) of the cell
K=average intensity on the inhibition region (inhibition surround) of a cell
n=measure of the steepness of a retinal cell inhibition curve, where:
n is between 0.7-1.0 for the receptor cells
n is between 1.4-3.0 for the bipolar cells
n is between 3.0-4.0 for the ganglion cells
n is may be selected to be any number in other applications.
The following images were used to test the vertebrate retina computer program found in Table 1. The test results and program for the horseshoe crab are found in Table 2:
1. A noise free square surrounded by a noise free background.
2. Vertical stripes, increasing in intensity from left to right.
3. Three squares added to noise.
The right most square has an intensity value of three. The center square has an intensity value of five. The left most square has an intensity value of seven.
Note: This is before noise is added.
4. A picture of part of the Signal Processing Lab at the Air Force Institute of Technology.
An intensity plot for each image was taken from line 100. The parameters in equation 5 were varied to test their effects on the test patterns given above. The parameters in equation 5 were varied as follows:
1. Blocksize was made smaller;
2. Inhibition region was made smaller;
3. Excitation region was made larger; and
4. Multiplication factor (n) was varied.
It was observed after extensively varying the parameters given above that making the excitation region, inhibition region, and/or blocksize larger, excessively blurred the edges of an image. It was also observed that when the value of the multiplication factor (n) was reduced, less enhancement of the edges occurred. The best results were obtained after extensively manipulating the parameters given above. The best results obtained as part of this invention were obtained by varying the parameters in equation 5.
There are several alternatives to choose from when implementing the vertebrate retinal software into hardware.
Three alternatives are listed below:
1. Hardwire connections can be used to connect silicon chips used to implement equation 5.
2. The connections between silicon chips can be done optically. The extraction of intensity information from the excitation and inhibition region for each pixel in an image can be done in parallel or in a serial fashion.
3. The connections and implementation of equation 5 can be done optically.
Although all the above alternatives should be looked at, alternative 2 would be the easiest to implement, especially if the extraction of intensity information from the excitation and inhibition regions of each cell block was extracted serially. A mirror of some lens system could perform the task of extracting intensity information serially from an image.
Once equation 5 is implemented in hardware it can then be used as a preprocessor to a correlator or other pattern recognition system. As noted above, the present invention digitally implements equation 5.
Once the retinal model extracts signals buried essentially in noise, then equation 5 should be equated to a modified version of equation 2 (horseshoe crab equation) and solved for K (horseshoe crab equation) for different combinations of the parameters in equation 5. Once a formula for K is determined the interaction between individual cells can then be modeled when the inhibition and excitation regions are fully illuminated. This approach can also model cell behavior for other than full field illumination.
Once equation 5 is optimized a retinal processor can be used on other signals (i.e., communication signals).
The program in Table 1 can be modified to process only one line of data. The average intensity for each inhibition; excitation, and averaging region was calculated using this one line of data. It was determined during the course of this invention that averaging a scene is not needed if the parameters in equation 5 are chosen properly. The parameters used to show the effects of averaging are as follows:
1. Blocksize=1;
2. the number of excitatory cells to the left and right of the pixel to be processed=21;
3. the inhibition region is 105 times wider than the excitation region;
4. Multfactor=15.0 for each time the image is processed through INHIBIT; and
5. the averaging region is 3 times wider than the excitation region.
The effects of averaging a scene and then using equation 5 indicates that averaging a scene has a negligible effect in reducing noise in a scene.
To reduce blurring at the edges, images were passed through the subroutine INHIBIT up to three times. The effect of processing the data in this way improved resolution (extremely sharp peaks at the edges) and eliminated essentially all the noise in the scene.
Finally, this program was modified to process all 512 lines of pixels using the inhibition regions as calculated above. Using this technique reduced the processing of images on the Micro-Vax II took about 30 hours for 512 lines of pixels, and approximately one hour for 512 lines of pixels processed along one dimension.
If the present invention were expressed as a process, FIG. 3 is a block diagram of the steps that the user of the system of FIG. 1 would follow.
The process begins with the image reception step 301, in which the computer 120 of FIG. 1 receives an image from the camera 100 via the Micro-Vax interface 110. Next, the user selects a block size 310 on the computer 120 that replicates the omatidium of the Limuls or the retinal cell of the retina being modeled. For example a suitable Limulus block size would be 16, while a suitable block size to model a human retinal cell would be 1.
In the next step, the inhibition region size and in the case of the vertibrate retina, the excitation region is selected 320. As in the biological retina, the inhibition region is an area which surrounds the omatidium which dampens its electrooptic effect on adjacent cells when it is stimulated by the reception of an illuminating signal. The presence of an inhibition region allows sharp light and dark contrasts to be detected by the retina, and its size should reflect the type of retina being modeled. The excitation region is an area which surrounds a receptor cell and increases its electrooptic effect on adjacent cells. For example, a suitable inhibition region size for a Limulus retina model would be 31×9. For a human eye, the inhibition region should be a square area, and occupy about 90% of the number of pixels in the image. Now that the inhibition and excitation region size has been selected 320 the received image is divided in step 330 into blocks of a size where each block represents an omatidium of the limulus, or retinal cell of the area selected in step 310. As mentioned above, one option of the invention is then to compute the average intensity of each block, and assign all pixels of this block to average intensity value 340. The advantages and disadvantages of this averaging step have been discussed above, and the user may experiment with this step to see if it improves the image processing for his particular application. The advantage of this digital image processing system is its extreme flexibility in allowing users to adjust all parameters.
Now that all block regions of the image are defined, the computer 120 will execute the inhibit function on the values of each block as defined above in equation 3. The result is an output of the inhibited response of each omatidium 350.
Next, the computer 120 will store the resulting inhibited intensity values to produce a combined processed output 360, and direct the monitor 130 of FIG. 1 to display the combined processed output 370.
As mentioned above, variations on the parameters are easily made by inputting any desired changes into the computer 120 of FIG. 1, for example, the inhibit subroutine might use equation 5 to simulate the change in potential across every receptor cell, instead of using equation 3. To reduce blurring on the edges of the processed image, the processed image might also be passed through the inhibit subroutine three times. As mentioned above, the inhibit function generally tends to increase the contrast between light and dark in the processed image. Note that the inhibit function used in the vertebrate computer program performs more than the inhibit function.
Specifically, it performs energy normalization, inhibition and noise suppression.
The retina model uses the Weber equation (Equation 5) to determine the response of a cell (represented by a pixel or a block of pixels). The retina model described above does not take in account the effects of distance as was done with the Limulus (horseshoe crab) model. This resulted in an average value being used in the Weber equation that might have been lower or higher than the "correct" value. Some cell blocks were assigned a value higher than desired. This is believed to be the main cause of blurring in test images used during this effort. The new retinal model will take this into account. Since the system is using a constant density eye it will be assumed that the weighting between any two cells is equal to alpha where 0≦alpha≦1. Another weighting might work better but with inadequate data this choice seems as good as any. The model described above also assumes an inhibition region that varies only as a function of a pixel or block of pixels position on the screen. The size of the inhibition region near the edge should be larger than that used if it is assumed that the attenuation effects are a function of only the distance. Additionally, the above-described model assumed that the full field illumination region was the same no matter what illumination level was used. It is now believed that the region of full field illumination should increase as the intensity level of the inhibition region is increased (taken care of by weighting the intensity of pixels in the inhibition region of a cell block which varies as a function of distance).
Specifically to calculate the inhibited response INHIBRESP:
INHIBRESP=I-SUM (WT*PIXINT)
WHERE I=Average intensity of a cell block or the average intensity of the cell block and that of cells in adjacent regions (Excitation region).
WT=antenuation factor which varies as a function of distance. In this implementation WT=to some function of ALPHA. Another weight could be used if desired.
PIXINT=Average intensity of a cell block in the inhibition region.
SUM=Add the weighted cell block intensity values in the inhibition region.
Lambert in his pattern recognition system equated the response SUM (WT*PIXINT) to the average intensity level of a fixed inhibition region. If the attenuation weighting between any two cell blocks is set equal to ALPHA the response SUM (WT*PIXINT) may not be equal to the average intensity value of the inhibition region. If you assume that the response SUM (WT*PIXINT) =average intensity level in the inhibition region scale the response SUM (WT*PIXINT) by K (K not equal to one) or equivalently scale all weights ALPHA by K to equate the response to the correct average value. If Capt. Lambert's assumption is correct I expect better results then that demonstrated by the Lambert algorithm because of the refinements made to selecting the "correct" inhibition region.
If the response SUM (WT*PIXINT) is not equal to the average intensity value in the inhibition region then the only way to use the Weber equation is to equate a response with an average intensity value in an inhibition region. By illuminating rings of pixels between full field illumination regions one might be able to derive an equation for use in the Weber equation to predict cell responses over all illumination conditions and not be limited to full field conditions. For best results (Least amount of blurring) use the intensity value of the pixel block being processed as the value of I in the Weber equation (equation 5).
While the invention has been described in its presently preferred embodiment it is understood that the words which have been used are words of description rather than words of limitation and that changes within the purview of the appended claims may be made without departing from the scope and spirit of the invention in its broader aspects.
Claims (5)
1. A digital image processing system which receives a video image and which models lateral inhibition, energy normalization and noise suppression processes in a generic retina in order to display an improved video image with enhanced edges and suppressed noise, said digital image processing system comprising:
a camera which receives said video image and converts it into analog electrical video image signals to produce an output;
a means for digitizing the output of said camera, said digitizing means being electrically connected with said camera and converting said analog electrical video image signals into digital electronic video image signals;
a means for digitally modeling said generic retina which receives and processes said digital electronic video image signals from said digitizing means to produce said improved video image by selecting a block size that replicates a receptor cell of said generic retina, selecting an inhibition region size and an excitation region size, said inhibition region being an area which surrounds said receptor cell and which dampens its electrooptic effect on adjacent cells when it is stimulated by reception of a illuminating signal thereby allowing contrasts of light and dark images to be detected, and wherein said excitation region increases its electrooptic effect on adjacent cells said digitally modeling means dividing said digital electronic image signals into blocks which have a size such that each block represents said receptor cell of said generic retina and executing an inhibit function about each block to determine effects said inhibition region would have, and said inhibit function also performing energy normalization when said generic return represents a vertibrate retina, said inhibit function producing thereby said improved video image; and
a means for displaying said improved video image, said displaying means being electrically connected to said means for digitally modeling said generic retina and receiving signals therefrom.
2. A digital image processing system, as defined in claim 1, wherein said means for digitally modeling said generic retina comprises a computer which is programmed to model a human retina by selecting 1 pixel as said block size which simulates said retinal cell, and selecting a square area as said inhibition region size such that said square area occupies over ninety percent of said digital electronic video image signals, said excitation region also being a square area of at least one pixel in area and as small as possible.
3. A digital image processing system, as defined in claim 2, wherein the equation used to calculate the potential across a retinal cell membrane is given as follows: ##EQU4## where: Rmax=Vmax which equals the maximum intensity displayable on a screen
V=a measure of intensity displayed on the screen and measured in volts
I=the average intensity on the excitation region (excitation center) of the cell
K=average intensity on the inhibition region (inhibition surround) of a cell
n=measure of the steepness of a retinal cell inhibition curve, where:
n is between 0.7-1.0 for the receptor cells
n is between 1.4-3.0 for the bipolar cells
n is between 3.0-4.0 for the ganglion cells.
4. A digital image process which models lateral inhibition and energy normalization characteristics of a generic retina while processing a video image to produce an improved video image with noise suppression and improved contrast distinction, said digital image process comprising the steps of:
receiving said video image from a scene with a camera which receives and converts said video image into an analog electrical video image signal, said camera producing an output thereby;
converting said analog electrical video image signal into a digital electric video image signal;
selecting a block size that replicates a retinal cell of said generic retina;
selecting in inhibition region size, said inhibition region being an area which surrounds said retinal cell and which dampens its electrooptic effect on adjacent cells when it is stimulated by reception of a strong illuminating signal thereby allowing contrasts of light and dark images to be detected;
selecting an excitation region size, said excitation region being an area which surrounds said retinal cell and which increases the electrooptic effect when it is stimulated by said illuminating signal;
dividing said video image into blocks which have a size such that each block represents said retinal cell of said generic retina;
executing an inhibit function about each block to determine effects said inhibition region and said excitation region would have, and to produce thereby said improved video image; and
displaying said improved video image.
5. A digital image process, as defined in claim 4 wherein said execution step comprises performing an equation on said digital video image signal, wherein the equation used to calculate the potential across a retinal cells membrane is given as follows: ##EQU5## where: Rmax=Vmax which equals the maximum intensity displayable on a screen
V=a measure of intensity displayed on the screen and measured in volts
I=the average intensity on the excitation region (excitation center) of the cell
K=average intensity on the inhibition region (inhibition surround) of a cell
n=measure of the steepness of a retinal cell inhibition curve, where:
n is between 0.7-1.0 for the receptor cells
n is between 1.4-3.0 for the bipolar cells
n is between 3.0-4.0 for the ganglion cells.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US07/283,114 US5033103A (en) | 1988-12-09 | 1988-12-09 | Model of the lateral inhibition, energy normalization, and noise suppression processes in the retina |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US07/283,114 US5033103A (en) | 1988-12-09 | 1988-12-09 | Model of the lateral inhibition, energy normalization, and noise suppression processes in the retina |
Publications (1)
Publication Number | Publication Date |
---|---|
US5033103A true US5033103A (en) | 1991-07-16 |
Family
ID=23084585
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US07/283,114 Expired - Fee Related US5033103A (en) | 1988-12-09 | 1988-12-09 | Model of the lateral inhibition, energy normalization, and noise suppression processes in the retina |
Country Status (1)
Country | Link |
---|---|
US (1) | US5033103A (en) |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5276550A (en) * | 1990-04-16 | 1994-01-04 | Olympus Optical Co., Ltd. | Optical microscope with variable magnification |
EP0577085A2 (en) * | 1992-06-30 | 1994-01-05 | Eastman Kodak Company | Method and apparatus for determining visually perceptible differences between images |
US5581660A (en) * | 1993-03-16 | 1996-12-03 | Hitachi, Ltd. | Neural network structure having lateral interconnections |
US6039447A (en) * | 1998-03-06 | 2000-03-21 | Hoya Corporation | Artificial vision system |
US20020034337A1 (en) * | 2000-05-23 | 2002-03-21 | Shekter Jonathan Martin | System for manipulating noise in digital images |
US20120274814A1 (en) * | 2005-10-12 | 2012-11-01 | Active Optics Pty Limited | Method of forming an image based on a plurality of image frames, image processing system and digital camera |
WO2013074232A1 (en) * | 2011-11-18 | 2013-05-23 | X6D Limited | Active glasses for optic nerve stimulation |
CN104517271A (en) * | 2014-12-29 | 2015-04-15 | 小米科技有限责任公司 | Image processing method and device |
US9111182B1 (en) | 2012-09-06 | 2015-08-18 | Hrl Laboratories, Llc | System, method, and computer program product for multispectral image processing with spiking dynamics |
CN110006088A (en) * | 2018-06-13 | 2019-07-12 | 葛高丽 | Safety-type heater based on environmental analysis |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3016518A (en) * | 1955-02-14 | 1962-01-09 | Nat Res Dev | System for analysing the spatial distribution of a function |
US3088096A (en) * | 1957-04-17 | 1963-04-30 | Int Standard Electric Corp | Method for the automatical recognition of characters |
US3187304A (en) * | 1957-08-29 | 1965-06-01 | Ibm | System for analysing the spatial distribution of a function |
US3701095A (en) * | 1970-05-25 | 1972-10-24 | Japan Broadcasting Corp | Visual feature extraction system for characters and patterns |
US3964021A (en) * | 1973-07-27 | 1976-06-15 | Visionetics Limited Partnership | Preprocessing system and method for pattern enhancement |
US4318083A (en) * | 1979-06-29 | 1982-03-02 | Canadian Patents And Development Limited | Apparatus for pattern recognition |
US4521773A (en) * | 1981-08-28 | 1985-06-04 | Xerox Corporation | Imaging array |
US4716312A (en) * | 1985-05-07 | 1987-12-29 | California Institute Of Technology | CMOS logic circuit |
-
1988
- 1988-12-09 US US07/283,114 patent/US5033103A/en not_active Expired - Fee Related
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3016518A (en) * | 1955-02-14 | 1962-01-09 | Nat Res Dev | System for analysing the spatial distribution of a function |
US3088096A (en) * | 1957-04-17 | 1963-04-30 | Int Standard Electric Corp | Method for the automatical recognition of characters |
US3187304A (en) * | 1957-08-29 | 1965-06-01 | Ibm | System for analysing the spatial distribution of a function |
US3701095A (en) * | 1970-05-25 | 1972-10-24 | Japan Broadcasting Corp | Visual feature extraction system for characters and patterns |
US3964021A (en) * | 1973-07-27 | 1976-06-15 | Visionetics Limited Partnership | Preprocessing system and method for pattern enhancement |
US4318083A (en) * | 1979-06-29 | 1982-03-02 | Canadian Patents And Development Limited | Apparatus for pattern recognition |
US4521773A (en) * | 1981-08-28 | 1985-06-04 | Xerox Corporation | Imaging array |
US4716312A (en) * | 1985-05-07 | 1987-12-29 | California Institute Of Technology | CMOS logic circuit |
Non-Patent Citations (13)
Title |
---|
Fukushima, K. "Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements", published in IEEE Trans. Syst. Science and Cybernetics, vol. SSC-5, No. 4, Oct. 1969, pp. 322-333. |
Fukushima, K. Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements , published in IEEE Trans. Syst. Science and Cybernetics, vol. SSC 5, No. 4, Oct. 1969, pp. 322 333. * |
Fukushima, K., "An Electronic Model of the Retina", published in Proceedings of the IEEE, on Dec. 1987, pp. 1950-1951. |
Fukushima, K., An Electronic Model of the Retina , published in Proceedings of the IEEE, on Dec. 1987, pp. 1950 1951. * |
Mead et al., "Real-Time Visual Computations Using Analog CMOS Processing Arrays", dated Nov. 1987. |
Mead et al., Real Time Visual Computations Using Analog CMOS Processing Arrays , dated Nov. 1987. * |
Rosenblith, W. A. Chapter 11 of text "Sensory Communication" pp. 183-203, Jul. 19, 1959. |
Rosenblith, W. A. Chapter 11 of text Sensory Communication pp. 183 203, Jul. 19, 1959. * |
Stillart, J. E. A Computer Model of Inhibition, Energy Normalization, and Noise Suppression Dec. 17, 1987. * |
Werblin W. S., "The Control of Sensivity in the Retina", Jan., 1983 Scientific American, pp. 70-79. |
Werblin W. S., The Control of Sensivity in the Retina , Jan., 1983 Scientific American, pp. 70 79. * |
Werblin, F. S., "Synaptic Interactions Mediating Bipolar Response in the Retina of the Tiger Salamander" pp. 205-228. |
Werblin, F. S., Synaptic Interactions Mediating Bipolar Response in the Retina of the Tiger Salamander pp. 205 228. * |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5276550A (en) * | 1990-04-16 | 1994-01-04 | Olympus Optical Co., Ltd. | Optical microscope with variable magnification |
EP0577085A2 (en) * | 1992-06-30 | 1994-01-05 | Eastman Kodak Company | Method and apparatus for determining visually perceptible differences between images |
EP0577085A3 (en) * | 1992-06-30 | 1994-05-11 | Eastman Kodak Co | Method and apparatus for determining visually perceptible differences between images |
US5581660A (en) * | 1993-03-16 | 1996-12-03 | Hitachi, Ltd. | Neural network structure having lateral interconnections |
US6039447A (en) * | 1998-03-06 | 2000-03-21 | Hoya Corporation | Artificial vision system |
US7599572B2 (en) | 2000-05-23 | 2009-10-06 | Adobe Systems, Incorporated | System for manipulating noise in digital images |
US20050276515A1 (en) * | 2000-05-23 | 2005-12-15 | Jonathan Martin Shekter | System for manipulating noise in digital images |
US6990252B2 (en) * | 2000-05-23 | 2006-01-24 | Adobe Systems, Inc. | System for manipulating noise in digital images |
US20020034337A1 (en) * | 2000-05-23 | 2002-03-21 | Shekter Jonathan Martin | System for manipulating noise in digital images |
US20120274814A1 (en) * | 2005-10-12 | 2012-11-01 | Active Optics Pty Limited | Method of forming an image based on a plurality of image frames, image processing system and digital camera |
US8624923B2 (en) * | 2005-10-12 | 2014-01-07 | Silvercrest Investment Holdings Limited | Method of forming an image based on a plurality of image frames, image processing system and digital camera |
WO2013074232A1 (en) * | 2011-11-18 | 2013-05-23 | X6D Limited | Active glasses for optic nerve stimulation |
US9111182B1 (en) | 2012-09-06 | 2015-08-18 | Hrl Laboratories, Llc | System, method, and computer program product for multispectral image processing with spiking dynamics |
CN104517271A (en) * | 2014-12-29 | 2015-04-15 | 小米科技有限责任公司 | Image processing method and device |
CN104517271B (en) * | 2014-12-29 | 2018-05-18 | 小米科技有限责任公司 | Image processing method and device |
CN110006088A (en) * | 2018-06-13 | 2019-07-12 | 葛高丽 | Safety-type heater based on environmental analysis |
CN110006088B (en) * | 2018-06-13 | 2021-04-16 | 安徽新大陆特种涂料有限责任公司 | Safe type room heater based on environmental analysis |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Itti et al. | A saliency-based search mechanism for overt and covert shifts of visual attention | |
Olshausen et al. | Vision and the coding of natural images: The human brain may hold the secrets to the best image-compression algorithms | |
Francis et al. | Cortical dynamics of feature binding and reset: Control of visual persistence | |
CA2067217C (en) | Categorization automata employing neuronal group selection with reentry | |
US5033103A (en) | Model of the lateral inhibition, energy normalization, and noise suppression processes in the retina | |
Bednar et al. | Tilt aftereffects in a self-organizing model of the primary visual cortex | |
Koch et al. | Neuromorphic vision chips | |
Werblin et al. | The computational eye | |
Francis et al. | Cortical dynamics of boundary segmentation and reset: Persistence, afterimages, and residual traces | |
Engelmann et al. | Electric imaging through active electrolocation: implication for the analysis of complex scenes | |
Yue et al. | Modeling direction selective visual neural network with on and off pathways for extracting motion cues from cluttered background | |
Róka et al. | Edge detection model based on involuntary eye movements of the eye-retina system | |
Chauvin et al. | Natural scene perception: visual attractors and images processing | |
Nowlan et al. | Filter selection model for generating visual motion signals | |
Caputi et al. | Identifying self-and nonself-generated signals: lessons from electrosensory systems | |
Silverman | Segmentation of ultrasonic images with neural networks | |
Cohen Duwek et al. | Perceptual colorization of the peripheral retinotopic visual field using adversarially-optimized neural networks | |
Schiff | Optical and neural pooling in visual processing in crustacea | |
Viola et al. | Recurrent eye tracking network using a distributed representation of image motion | |
Grogan et al. | Image quality measurements with a neural brightness perception model | |
Souihel | Generic and specific computational principles for visual anticipation of motion trajectories | |
Tanaka et al. | Autonomous foveating system and integration of the foveated images | |
Wilson et al. | A two-dimensional, object-based analog VLSI visual attention system | |
Skrzypek et al. | Lightness constancy from luminance contrast | |
Rivera-Alvidrez et al. | A neuronally based model of contrast gain adaptation in fly motion vision |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: UNITED STATES OF AMERICA, THE, AS REPRESENTED BY T Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNOR:SILLART, JEFFREY E.;REEL/FRAME:005805/0633 Effective date: 19910617 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 19990716 |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |