WO2004010378A1 - Circuit d'adaptation de luminance base sur des interactions locales - Google Patents

Circuit d'adaptation de luminance base sur des interactions locales Download PDF

Info

Publication number
WO2004010378A1
WO2004010378A1 PCT/ES2003/000366 ES0300366W WO2004010378A1 WO 2004010378 A1 WO2004010378 A1 WO 2004010378A1 ES 0300366 W ES0300366 W ES 0300366W WO 2004010378 A1 WO2004010378 A1 WO 2004010378A1
Authority
WO
WIPO (PCT)
Prior art keywords
luminance
values
local
global
network
Prior art date
Application number
PCT/ES2003/000366
Other languages
English (en)
Spanish (es)
Inventor
Matthias Sven Keil
Gabriel CRISTOBAL PÉREZ
Original Assignee
Consejo Superior De Investigaciones Científicas
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Consejo Superior De Investigaciones Científicas filed Critical Consejo Superior De Investigaciones Científicas
Priority to AU2003250253A priority Critical patent/AU2003250253A1/en
Publication of WO2004010378A1 publication Critical patent/WO2004010378A1/fr

Links

Classifications

    • G06T5/94—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration by the use of local operators
    • G06T5/60—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/36—Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given point; Non-linear local filtering operations, e.g. median filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20004—Adaptive image processing
    • G06T2207/20012—Locally adaptive
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present invention relates to software integration in intelligent pixel sensors (cellular neural networks). Specifically in this invention a method is described that allows the sensor to adjust the overall luminance incident through local processing operations. Specifically, the scope within which this patent may have the greatest impact is in companies that manufacture both analog and digital video cameras, monitors, videoconferencing, mobile telephony, etc.
  • Other patents http://www.de1phion.com related to this proposal are: (1) Chua, L. and Yang, L., "Cellular neural network", Pat. US5140670 (2) Werblin, F., Roska, T. and Chua, L.O. "CNN programmable topographic sensory device ', Pat. US5717834.
  • normalization is a global process.
  • the usual calculation procedure is the search for a global maximum and minimum.
  • the current values of the maximum and minimum are stored in a temporary memory. These values are compared with the current (local) value and are substituted by that value if it is less than the minimum stored or greater than the maximum stored.
  • the temporary memory will contain the maximum and minimum global values.
  • the original values ie the inputs
  • the overall nature of the algorithmic approach clearly limits its use in electronic applications, and therefore other approaches have been suggested in order to obtain an approximate solution to the problem. These include (data collected from http://www.eleceng.adelaide.edu.au/Groups/GAAS/ Bugeye / visionchips / vision_chips /)
  • This method finds the average value of the values and feeds that value in order to control the gain. It is easily implementable in VLSI using the current sum mode.
  • the methods based on the average error can provide information about the deviation of the variables at the hierarchy level. This can be interpreted as an indication of the level of activity. Therefore, system activity can be controlled by this method.
  • This method provides a decision based on the higher value of the variables.
  • the hardware realization of this method is called the winner-take-all circuit (winner-take-all (WTA)).
  • the proposed circuit can carry out a win-take-all operation if the processing regions are not interconnected. Therefore, in each of these regions, an independent normalization will be carried out, that is, the maximum or minimum regions will be determined.
  • a threshold we get the WTA operation.
  • the network does not eliminate any value at the expense of others, so it is different from WTA architectures. Instead, what the system does is to re-scale the values in each region independently.
  • Other methods for mapping arbitrary values in a given dynamic range include the use of compression or sigmoidal functions. Without the presence of an adaptive mechanism, these functions provide a worse outcome than in the case of incorporating adaptability.
  • the system proposed here can also be used for the adaptation of non-linear functions, through the use of the local value available in the parameterization of the maximum and minimum global values.
  • the mechanism proposed here in addition to its simple implementation both from the algorithmic point of view and through electronic circuit, provides the global maximum and minimum in a local way, that is, each cell has access to the maximum and minimum value without making any reference to global variable values.
  • the proposed system can be implemented as a local WTA network or as a new retinal circuit (eg through cellular neural networks).
  • There is currently a large set of artificial retinal circuits i.e. electronic implementations of models of the vertebrate retina
  • the circuit of adaptation to light changes is carried out locally, extracting only the information from local contrast of the luminance distribution.
  • VLSI silicon retina
  • the local average is subtracted from the signal in each cell.
  • two local averages with different spatial distribution are subtracted from each other.
  • This method has two disadvantages. First, the signal will "center" around zero, and therefore the variation of the signal will depend on the local average. For example, if the average current is 1 nA, the variation of the signal will be around that value. Second, this method is not able to reproduce the response to intensity dependence. Due to its simplicity, this method has been used in many VLSI implementations of artificial retinas, such as:
  • Linear lateral inhibition is a simple case of lateral inhibition, where the signal in a cell is subtracted from fractions of neighboring cells.
  • This model can explain the characteristics of edge enhancement and dynamic range. However, it cannot reproduce the dependence phenomenon with intensity.
  • This method has been used in the implementation of some chips based on lateral inhibition. They include: • S. Wolpert & E. Micheli-Tzanakou, "Silicon models of lateral inhibition,” IEEE Trans. Neural Networks, Vol. 4, No. 6, pp. 955-961, Nov. 1993.
  • the method proposed here is completely different from other existing models. It is also different from other algorithms that fill in surfaces with some color. In this case, the maximum value of the neighbors is assigned to the central value of each pixel. A similar strategy can be followed for the search of the global minimum, but nevertheless the VLSI implementations of the surface filling programs require a higher computational cost than the method proposed here.
  • the method proposed here provides a definitive solution to the problem of finding the global maximum and minimum of a set of numbers based on local information and non-algorithm interactions.
  • non-algorithmic refers to the fact that we do not use explicit comparisons between values (eg if ... then).
  • local refers to the fact that a global memory or "buffer” is not used, but only local interactions between neighboring cells.
  • the method consists of three differential equations. Each equation can be interpreted as an independent processing layer of an ordered array of cells. The topology is the same for all three layers. The first layer performs a nonlinear diffusion converging to the global maximum. This means that the global maximum is locally available, despite only local interactions between the cells in each layer.
  • the second layer will converge to the local minimum.
  • the third layer connects both the maximum and the minimum in order to obtain a normalized representation of the input.
  • the values can be used for the adaptation (normalization) of arbitrary signals, as mentioned in the section Detailed description of the invention Definitions
  • x, j be the input data that we wish to process, with it
  • Nki is a set of indexes that specify a von-Neumann type neighborhood, that is - H ' ⁇ M' + ) . , j - 0-t '+') r. Note that the diffusion equation is linear.
  • each cell a obtains the global maximum A at the end of the process, and cell b, j the global minimum B.
  • the reason for this behavior is due to the fact that the HWR operator only allows a cell to increase its activity If one of your closest neighbors has a greater activity. If there is no more gradient between the cells, there will be no further increase in activity in them. This corresponds to the state of convergence of the network. Propagation activity It is therefore non-conservative, since all cells will increase their activity to the maximum global value.
  • the dynamics of b, j cells take place in a similar way. Each b, j is only allowed to decrease its activity if one of its closest neighbors has a lower activity. This diffusion ceases if there are no more gradients, and say when all cells have reached the global minimum.
  • each island W can therefore be defined by a set of matrix coordinates (row / column indices) together with their corresponding non-zero values (pesos). It is not possible for two islands to be spatially exceeded (in our analogy that means that two connected islands can merge and create a single island).
  • the simple dynamics (that is, for small values of the integration steps) of the network equations (3,4 and 8) reveals responses that are similar to those produced in retinal ganglion cells that have an excitatory center and an inhibitory environment. .
  • Such cells are called ON cells, and are characterized by responding to the brightest part of a luminance ramp.
  • An OFF biological cell has an excitatory environment and an inhibitory center. Responses that are similar to retinal OFF cells can be calculated by modifying equation (7) as 'a -fc;
  • leakage conductance (leakage) g ⁇ ea k > 0 and the resting potential V res t must be chosen with a small zero value without loss of generality.
  • ON cells can be modeled as:
  • g ⁇ eak and V res t can be taken as zero, or with very small values.
  • the temporary variable t has been omitted for simplicity of each variable.
  • the input x ⁇ comes from the photoreceptors.
  • Laplacian can be easily obtained by calculating '" ⁇ / - y , Y (responses of OFF cells of the Laplacian type).
  • Figure 1 (a) Original image; (b) Response of the ON channel of a model based on the use of filters based on Gaussian differences; (c) Expanded detail of the previous response displayed with the inverted contrast. As you can see this model is not able to correctly reproduce the contrast at the intersections of the lines
  • FIG. 2 The retinal circuit model proposed in this patent is able to provide a more efficient response by facilitating a better contrast compared to the classical methods based on the Gaussian difference.
  • the proposed model does not present the problem described above.
  • Figure 3 Original image; (bd) Dynamic response of the network for various values (multiples of integration time, indicated in the legends of the figures).
  • Figure 4 (ad) Dynamic network response for various values (multiples of integration time, indicated in the legends of the figures). Note that high luminance values accelerate the network normalization process, and as the network evolves the response obtained is almost identical to the original image.
  • Figure 5 (ac) Dynamic response of the network for various values (multiples of integration time, indicated in the legends of the figures). Note that high luminance values accelerate the network normalization process, and as the network evolves the response obtained is almost identical to the original image.
  • Figure 6 (ac) Network response for images with different range of contrasts (low values).
  • Figure 10 The figure represents a scheme of the proposed artificial retina. This model allows simultaneous transmission of contrast and luminance information. It facilitates a mechanism of global luminance adaptation with a wide dynamic range, providing a very compact code of visual information.
  • Figure 11 This figure represents a schematic of a cellular neural network where the processing elements are interconnected according to a von-Neumann type neighborhood scheme (4 closest neighbors).

Abstract

L'invention concerne un système basé sur un automate cellulaire. Son fonctionnement comprend essentiellement un ensemble d'équations différentielles. Les équations sont couplées exclusivement par l'intermédiaire d'interactions entre les éléments voisins, c'est-à-dire que chaque cellule échange exclusivement des informations avec les quatre éléments voisins les plus proches. En raison de sa nature locale, ce système est susceptible d'être mis en oeuvre en VLSI, et utilisé par une gamme variée de circuits dans lesquels est nécessaire une normalisation du signal d'entrée dans une étape de traitement ultérieure, présentant une gamme dynamique de niveaux de luminosité restreinte.
PCT/ES2003/000366 2002-07-19 2003-07-15 Circuit d'adaptation de luminance base sur des interactions locales WO2004010378A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2003250253A AU2003250253A1 (en) 2002-07-19 2003-07-15 Luminance-matching circuit based on local interactions

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ESP200201703 2002-07-19
ES200201703A ES2199076B1 (es) 2002-07-19 2002-07-19 Circuito de adaptacion de luminancia basado en interacciones locales.

Publications (1)

Publication Number Publication Date
WO2004010378A1 true WO2004010378A1 (fr) 2004-01-29

Family

ID=30470581

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/ES2003/000366 WO2004010378A1 (fr) 2002-07-19 2003-07-15 Circuit d'adaptation de luminance base sur des interactions locales

Country Status (3)

Country Link
AU (1) AU2003250253A1 (fr)
ES (1) ES2199076B1 (fr)
WO (1) WO2004010378A1 (fr)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5140670A (en) * 1989-10-05 1992-08-18 Regents Of The University Of California Cellular neural network
EP0702482A2 (fr) * 1994-09-15 1996-03-20 Xerox Corporation Obtention de demi-teintes par diffusion d'erreurs avec réponse homogène dans des régions d'images de forte/faible intensité
US5530559A (en) * 1989-04-27 1996-06-25 Casio Computer Co., Ltd. Image processing apparatus including binary data producing unit
US5717834A (en) * 1993-08-26 1998-02-10 Werblin; Frank S. CNN programamble topographic sensory device
EP0866608A2 (fr) * 1997-03-18 1998-09-23 Matsushita Electric Industrial Co., Ltd. Méthode pour corriger la gradation de luminance dans un appareil de prise d'images
JPH10293841A (ja) * 1997-04-18 1998-11-04 Agency Of Ind Science & Technol カラー画像の画質改善方法及びその装置
US5936684A (en) * 1996-10-29 1999-08-10 Seiko Epson Corporation Image processing method and image processing apparatus
WO2000063838A1 (fr) * 1999-04-16 2000-10-26 Izahi Corporation Compensation automatique du niveau du noir, de luminosite et de couleur pour images fixes numeriques et video numerique
WO2001026054A2 (fr) * 1999-10-01 2001-04-12 Microsoft Corporation Egalisation d'histogramme a adaptation locale

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5530559A (en) * 1989-04-27 1996-06-25 Casio Computer Co., Ltd. Image processing apparatus including binary data producing unit
US5140670A (en) * 1989-10-05 1992-08-18 Regents Of The University Of California Cellular neural network
US5717834A (en) * 1993-08-26 1998-02-10 Werblin; Frank S. CNN programamble topographic sensory device
EP0702482A2 (fr) * 1994-09-15 1996-03-20 Xerox Corporation Obtention de demi-teintes par diffusion d'erreurs avec réponse homogène dans des régions d'images de forte/faible intensité
US5936684A (en) * 1996-10-29 1999-08-10 Seiko Epson Corporation Image processing method and image processing apparatus
EP0866608A2 (fr) * 1997-03-18 1998-09-23 Matsushita Electric Industrial Co., Ltd. Méthode pour corriger la gradation de luminance dans un appareil de prise d'images
JPH10293841A (ja) * 1997-04-18 1998-11-04 Agency Of Ind Science & Technol カラー画像の画質改善方法及びその装置
WO2000063838A1 (fr) * 1999-04-16 2000-10-26 Izahi Corporation Compensation automatique du niveau du noir, de luminosite et de couleur pour images fixes numeriques et video numerique
WO2001026054A2 (fr) * 1999-10-01 2001-04-12 Microsoft Corporation Egalisation d'histogramme a adaptation locale

Also Published As

Publication number Publication date
ES2199076A1 (es) 2004-02-01
AU2003250253A1 (en) 2004-02-09
ES2199076B1 (es) 2005-06-16

Similar Documents

Publication Publication Date Title
JP7208156B2 (ja) ピクセルワイズイメージングの方法及びシステム
JP3887420B2 (ja) 多重解像度読出しを備えた能動ピクセルセンサアレイ
Healey et al. Radiometric CCD camera calibration and noise estimation
CN113412611A (zh) 具有多个光电二极管的像素传感器
Steffen et al. Neuromorphic stereo vision: A survey of bio-inspired sensors and algorithms
JP2020526104A (ja) 拡張ダイナミックレンジを有するデジタルピクセル
CN104301636B (zh) 低复杂度高效高动态数字图像的合成方法
JP7136080B2 (ja) 撮像装置、および撮像方法、並びに画像処理装置、および画像処理方法
EP2652678A1 (fr) Systèmes et procédés pour synthétiser des images de haute résolution à l'aide de processus de super-résolution
Lenero-Bardallo et al. A bio-inspired vision sensor with dual operation and readout modes
CN211296854U (zh) 像素阵列及仿生视觉传感器
CN103533267A (zh) 基于列级adc的像素分裂与合并图像传感器及数据传输方法
KR20220052359A (ko) 듀얼 카메라 및 듀얼 픽셀을 통한 조인트 깊이 예측
Shimonomura et al. Wide-dynamic-range APS-based silicon retina with brightness constancy
WO2004010378A1 (fr) Circuit d'adaptation de luminance base sur des interactions locales
US6546148B1 (en) Circuitry for determining median of image portions
Nam et al. Implementation of real-time image edge detector based on a bump circuit and active pixels in a CMOS image sensor
CN114531555B (zh) 一种端侧轮廓增强的仿生视觉传感器
CN207926767U (zh) 像素单元及成像装置
CA2239117C (fr) Systeme d'imagerie
US20220116556A1 (en) Method and system for pixel-wise imaging
JP2022104795A (ja) イメージセンシング装置及びその動作方法
Kim et al. A cmos image sensor-based stereo matching accelerator with focal-plane sparse rectification and analog census transform
JP4411313B2 (ja) イメージングシステム
EP1239666A2 (fr) Système de prise d'image

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP