WO2000052639A2 - Two architectures for integrated realization of sensing and processing in a single device - Google Patents
Two architectures for integrated realization of sensing and processing in a single device Download PDFInfo
- Publication number
- WO2000052639A2 WO2000052639A2 PCT/US2000/005785 US0005785W WO0052639A2 WO 2000052639 A2 WO2000052639 A2 WO 2000052639A2 US 0005785 W US0005785 W US 0005785W WO 0052639 A2 WO0052639 A2 WO 0052639A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- array
- cell
- output
- cells
- sensor
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Definitions
- the present invention relates to an integrated sensing and processing device. More specifically, the present invention relates to an architecture for the integrated realization of sensing and processing within a single device.
- CNN Cellular Neural Networks
- Equations 1 -3 canonical differential equations
- There are several degrees of freedom in setting up these interactions in the form of feedback and feedforward weights (namely the values A and B in Equation 1) and the initial conditions of the states (x(t 0) in Equation 1) and the bias currents (I in Equation 3).
- CNN is in fact programmed by way of manipulating these values.
- the nonlinearity comes from the nonlinear nature of the function ( in Equation 2).
- the function "f ' can take on different shapes and characteristics.
- CNN is a hybrid of Cellular Automata and Neural Networks (hence the name Cellular Neural Networks) and it incorporates the best features of both concepts. Like Neural Networks, its continuous time feature allows real-time signal processing, and like Cellular Automata its local interconnection feature makes physical realization in VLSI possible. Its grid-like structure is suitable for the solution of a high-order system of first order nonlinear differential equations on-line and in real-time.
- CNN can be viewed as an analog nonlinear dynamic processor array.
- the basic unit of CNN is called a cell. Each cell receives input from its immediate neighbors (and itself via feedback), and also from external sources (e.g., the sensor array points and/or previous layers).
- ⁇ ⁇ x, l (t) -x ⁇ ( ⁇ ) + ⁇ A, ,, ( u (xyford (0)+ ⁇ B, lM u kl Ontended ( + 1, klcN r l) lr_N r (,j)
- u represents the input
- x represents the state
- y represents a nonlinear function of the state associated with a cell (or neuron)
- a and B represent the cloning templates.
- Equations (1-3) are not suitable for direct VLSI implementation.
- the summation equation is a current based computation, as the circuit model in Figure 1 suggests.
- the intrinsic resistance values ( R ) are very large and capacitance values are very small, little charge is required to maintain a particular voltage. This also means that the current required to alter the voltage value of the state is relatively very small.
- the values of the noise currents are of sufficient magnitude to make significant difference.
- An integrated sensing device comprising an array of sensor processor cells capable of being arranged into a detection array.
- Each sensor processor cell comprises a sensing medium; at least one transconductance amplifier configured for feedforward template multiplication; at least one transconductance amplifier configured for feedback template weights; a plurality of local dynamic memory cells; a data bus for data transfer; and a local logic unit.
- the array of sensor processor cells by responding to data control signals, is capable of transforming, reshaping, and modulating the original sensed image into varied represenations which include (and extend) traditional spatial and temporal processing transformations.
- Figure 1 (Prior Art) illustrates an existing embodiment of an architecture of a physical realization of the cellular neural network canonical equations.
- Figure 2 illustrates an embodiment of an adaptation of a dynamic cellular computation model to the VLSI domain in accordance with the teachings of one embodiment of the present invention.
- Figure 3 illustrates an alternate embodiment of an adaptation of a dynamic cellular computation model to the VLSI domain in accordance with the teachings of one embodiment of the present invention.
- Figure 4 illustrates an alternate embodiment of an adaptation of a dynamic cellular computation model to the VLSI domain employing analog multipliers in accordance with the teachings of one embodiment of the present invention.
- Figure 5 illustrates an embodiment of an operational process in flow chart format for a dynamic cellular architecture in accordance with the teachings of one embodiment of the present invention.
- Figure 6 illustrates an embodiment of a cellular architecture element or cell in accordance with the teachings of one embodiment of the present invention.
- Figure 7 illustrates an embodiment of the layout of a CMOS chip comprising several types of cellular units in accordance with the teachings of one embodiment of the present invention.
- Figure 8 illustrates an embodiment of a one input - one output cell with initialization in accordance with the teachings of one embodiment of the present invention.
- Figure 9 illustrates an embodiment of a programmable type feedback cell in accordance with the teachings of one embodiment of the present invention.
- Figure 10 illustrates an embodiment of a positive and negative feedback hardwired and state initializable cells in accordance with the teachings of one embodiment of the present invention.
- Figure 1 1(a) illustrates an embodiment of two (+/-) hardwired feedback and floating state cells in accordance with the teachings of one embodiment of the present invention.
- Figure 1 1(b) illustrates one embodiment of a one feedforward only cell in accordance with the teachings of one embodiment of the present invention.
- Figure 12 illustrates an embodiment of digital input accommodation via parallel transistor cascading in accordance with the teachings of one embodiment of the present invention.
- Figure 13 illustrates an embodiment of a feedforward cell in accordance with the teachings of one embodiment of the present invention.
- Figure 14 illustrates an embodiment of the results from a three input one output cell with no feedback connections in accordance with the teachings of one embodiment of the present invention.
- Figure 15 illustrates an embodiment of a dual distributed cellular architecture in accordance with the teachings of one embodiment of the present invention.
- Figure 16 illustrates an embodiment of the layout of a signed sensor in accordance with the teachings of one embodiment of the present invention.
- Figure 17 illustrates an embodiment of a dual output light sensing pixel in accordance with the teachings of one embodiment of the present invention.
- Figure 18 illustrates an embodiment of a schematic diagram of a cell of a programable convolutiuon array (PCA) in accordance with the teachings of one embodiment of the present invention.
- PCA programable convolutiuon array
- Figure 19 illustrates an embodiment of the layout of a single cell of the programable convolutiuon array (PCA) in accordance with the teachings of one embodiment of the present invention.
- PCA programable convolutiuon array
- Figure 20 illustrates an embodiment of the outputs of the programable convolutiuon array (PCA) elements in accordance with the teachings of one embodiment of the present invention.
- Figure 21 illustrates an embodiment of a 5 x 5 programable convolutiuon array (PCA) with I/O pads in accordance with the teachings of one embodiment of the present invention.
- Figure 22 illustrates an embodiment of an operational process in flow chart format for a programmable convolution array (PCA) of the dual distributed architecture in accordance with the teachings of one embodiment of the present invention.
- PCA programmable convolution array
- Figure 23 illustrates an embodiment of an element of the programmable cellular logic array (PCLA) in accordance with the teachings of one embodiment of the present invention.
- Figure 24 illustrates an embodiment of an operational process in flow chart format for a programmable cellular logic array (PCLA) of the dual distributed architecture in accordance with the teachings of one embodiment of the present invention.
- Figure 25 illustrates an embodiment of an operational process in flow chart format for using the programmable convolution array (PCA) and programmable cellular logic array (PCLA) of the dual distributed architecture cooperatively in accordance with the teachings of one embodiment of the present invention.
- PCA programmable convolution array
- PCLA programmable cellular logic array
- Figure 26 illustrates an embodiment of a computing environment in which the invention may be implemented in accordance with the teachings of one embodiment of the present invention.
- Figure 27 illustrates an embodiment of a network environment in which the invention may be implemented in accordance with the teachings of one embodiment of the present invention.
- the steps or process of the present invention are embodied in machine-executable instructions, such as computer instructions.
- the instructions can be used to cause a general-purpose or special-purpose processor that is programmed with the instructions to perform the steps of the present invention.
- the steps of the present invention might be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.
- the present invention is generally directed to a new cellular network architecture that can be implemented successfully using electronic circuits and integrated microelectronic chips, and a new distributed architecture comprising two separate arrays, namely a programmable convolution array PCA and a programmable logic array PLA for processing one. two or three dimensional array (physical) sensor data.
- One embodiment of the present invention introduces a self-normalizing feedback structure for the computation of the inputs and feedforward template. Similarly, the present invention also introduces the same structure to the feedback portion of the architecture. These modifications create a new set of equations, and correspondingly a new set of dynamics, and thus represent a new structure, form, and architecture.
- the feedback in the feedforward input weight multiplication process and the feedback in the feedback weight multiplication are employed to keep the state node ( x ) from quickly accumulating or losing charge to saturate to a power rail.
- Figure 2 illustrates one embodiment of the present invention implemented in circuitry.
- Figure 2(a) illustrates one embodiment of a self regulating mechanism, wherein rather than multiplying the input by B (as in the prior art), a function of the input minus the state is defined per node and that quantity is multiplied by that B.
- the stabilizing and self scaling feedback elements around an amplifier keep the output of the amplifier from saturating.
- the gain of the amplifier is manipulated to obtain a factor which is a scaled (normalized) version of B.
- the output is consistently the input times a normalized vesrion of "B" and the plural (collective) inputs from the cell neighborhood will not cause the cell's aggregate state or output to saturate or be overdriven with slight fluctuations of the input.
- Figure 2(b) illustrates one embodiment of an adaptation of the dynamic cellular computation model to the VLSI domain using the concept shown in Figure 2(a).
- both u (unsigned) and x (signed) are represented as analog voltages u and x;
- template weights can be digitally stored and are programmable;
- the transfer function f ( ), such as that illustrated in Equation 2 can in fact become a programmable tri-level saturation function, f '( ) that produces zero output in a local region of the input (rather than only at a point).
- the capacitor C and the resistor R may be due to the parasitics of the micro-electronic implementation process.
- capacitances in the order of pico-farads may be implemented on chip. Any larger capacitance may be implemented off- chip with the likely consequent cost of limiting the size of the cellular array.
- the input (feedforward) weights (B) are implemented in a negative feedback structure.
- the feedback structure stabilizes the state (x) node, which otherwise has a tendency to charge all the way up or down to a rail limited by the power supply, and the structure is far more resilient to manufacturing related issues such as transistor mismatches.
- the modified dynamic cellular architecture model is shown, wherein some assumptions have been made about the polarity of inputs and weights, such as B is nonnegative and A is nonpositive.
- Equation 9 Equation 9 where the hyperbolic tangent (tanh) represents a basic (or wide-range) transconductance amplifier circuit.
- the indices kl runs over a neighborhood of the cell location ij for the feedback term A, while kl run over a neighborhood of the location ij in the feedforward term associated with the B parameters.
- the input (feedforward) weights (B) are implemented in a negative state- feedback structure. Accordingly, in this implementation, the feedback structure stabilizes the state (x) node, which otherwise has a tendency to charge all the way up or down to a rail limited by the power supply, and the present structure is far more resilient to manufacturing related issues such as transistor mismatches.
- Figure 3 illustrates this modified dynamic cellular architecture model is shown, wherein some assumptions have been made about the polarity of inputs and weights, such as each elements of B and A is nonnegative.
- Figure 4 illustrates yet another embodiment that employs the usage of analog multipliers (rather than amplifiers with gain).
- analog multipliers rather than amplifiers with gain
- the feedback connected amplifiers of the embodiment of Figure 3 may be replaced by analog multipliers to implement weights of positive or negative sign, as illustrated in Figure 4(b).
- digital gains can also be built by way of modifying the analog multipliers gain with the digital input accommodation via parallel transistor cascading method shown in Figure 9. By adding a multiplier that is capable of sign representation, the restrcitions on template weights are eliminated.
- FIG. 5 illustrates one embodiment of the operational process of the
- Dynamic Cellular Architecture illustrated above.
- the necessary A, B, x(0), and bias values for all of the operations to be performed is determined.
- a program is made of operations needed to complete the task, such that each step of the program specifies an operation, where each operation is defined by the pattern of A, B, x(0), and bias values to be applied to the cellular architecture.
- the program may be stored externally or on the same chip where the architecture is implemented.
- each cell of the architecture may contain its own program memory.
- These inputs may come from continuous or discretely sampled data from sensors built into the architecture, or alternately, they can be scanned into the input nodes (u) from an external data source by employing an appropriate scanning mechanism.
- Figure 6 illustrates one embodiment of a pragmatic implementation of the Dynamic Cellular Architecture, as a simple 5x5 array.
- Figure 6 illustrates an integrated light sensing and processing architecture implemented within cellular chip based on the dynamic cellular architecture of this invention. It is understood that the inventive concepts of the present invention may be applied to a variety of different technologies and implemented in a variety of different physical architectures, and as such, the present invention is not meant to be limited to integrated light sensing and processing architectures.
- the integrated light sensing and processing architecture based on the dynamic cellular architecture of this invention is comprised of an array of identical sensor processor cells, each of which contain: (1) a photodiode and circuits for active light sensing; (2) transconductance amplifiers for feedforward template weight multiplication; (3) wide range transconductance amplifiers for feedback template weights; (4) analog/ single bit digital local dynamic memory cells; (5) a data bus for transfers; (6) local programmable logic; and (7) read/write data controls.
- a variety of operations of the cellular paradigm may be performed using, for instance, a pair of 3 x 3 cloning templates, defined by the A and B of Equations (4) or (7).
- each operation can be completed in a short time while it is carried out in each of the elements over the entire image. This implies unprecedented processing rate improvements of sequential processors.
- the operations include linear transformations of the input (convolution operations), plus connected component detection and other types of data manipulation allowed for by the feedback weights.
- some example operations are edge location, morphology operators such as dilation, thinning, and erosion, light adaptation, scratch removal, texture, color and shape analysis.
- temporal operations such as motion analysis, e.g., local velocity detection, motion direction detection.
- the overall cellular chip is illustrated as a system.
- an external (or international core) microcontroller generates the necessary command signals to the integrated sensor processor, such as sensor timing, row select, and program code.
- the microcontroller can be replaced by a processor or a digital signal processor depending on the computing needs of the particular application at hand.
- the program memory can be internal, where a more compact address gets horizontally coded into bit level microinstructions and determine template values and data transfers within the architecture.
- the contents of a cell are illustrated in greater detail.
- the feedforward and feedback weights which also integrate the transfer function are shown in outline format.
- the local logic and memory functions are likewise illustrated in a similar fashion.
- two way connections can be made between the data bus and the two analog and four digital memory units, the state (x) and input (u).
- connection can be made from the logic output and the reference voltage to the data bus.
- the logic can be implemented by programming, similar to that of programmable logic arrays.
- the emerging reconfigurable logic arrays represent another option or embodiment.
- the inventive concepts of the present invention may be applied to a variety of different technologies and implemented in a variety of different physical architectures, and as such, the present invention is not meant to be limited to integrated light sensing and processing architectures.
- the same architectures can be applied to any system, such as an acoustic or sound based system, where there are multiple sensors arranged in a one. two, or three dimensional grid. Each sensor element on the grid may then respond to the same single physical quantity (e.g., frequencies of the audible sound spectrum). As such, each sensor may sense a different attribute of the same signal (e.g., each sensor tuned to a different frequency of the audible sound spectrum). This will measure the frequency pattern of the sound incident on the array. Further, one can also imagine hybrids of these two, such as measuring a color pattern.
- Figure 7 illustrates one embodiment of the layout of a CMOS chip comprising several types of cellular units (e.g., VLSI adapted cellular dynamic architecture cell prototypes), in accordance with the concepts of the present invention.
- the CMOS chip design may be manufactured to the MOSIS 2 micron ORBIT ANALOG process.
- the die size is implemented through the "TINYCHIP" package that is a 2.3 mm x 2.3 mm area bonded to a 40 pin DIP.
- the CMOS chip comprise eleven types of cells: Four One Feedforward - One Feedback Cells with Initialization; Two Programmable Type (Positive or Negative) Feedback Cells; Two Hardwired Feedback and State Initializable Cells; Two (+/-) Flardwired Feedback and Floating State Cells; and One Feedforward only Cell.
- all outputs are made available through wide range followers at the outputs and all programmable cell parameters, i.e., feedforward and feedback weights, and bias are at 3 bit resolution (plus sign bit). External inputs are available via pins to set these weights through the pins.
- Figures 8-1 1 illustrate these respective cells schematically.
- Figure 8 illustrates one embodiment of the schematics of the four cells in a compact manner: Four One Feedforward - One Feedback Cells with Initialization.
- Both the feedforward and the feedback amplifiers have three bit programmable gain in this specific implementation.
- Figure 9 illustrates one embodiment of the schematics of the Two Programmable Type (Positive or Negative) Feedback Cells. Two three-input one-output selectable kind (+ or -) of feedback cells may be implemented. One of these also allowed for initialization of the state in a manner identical to the one illustrated in Figure 8.
- Figure 9 shows this type of cell without initialization.
- the two signals (the state node x and Vref) are directed to the terminals of the feedback weight amplifier based on the feedback sign select bit. In one embodiment, all amplifiers have three bit programmable gain.
- Figure 10 illustrates one embodiment of the schematics of the Two Hardwired Feedback and State Initializable Cells.
- Figure 1 1(a) illustrates one embodiment of the schematics of the Two (+/-) Hardwired Feedback and Floating State Cells. In one embodiment, these cells are identical to the ones described in Figure 10 but do not contain state initialization circuits. Initial value of the state is thus undefined.
- Figure 1 1(b) illustrates one embodiment of the schematics of the One Feedforward only Cell. This is a three input one output cell with programmable bias. In one embodiment, the feedback component is eliminated. Thus the cell diverges from the cellular neural network paradigm and instead is useful for feedforward operations only.
- Figure 13 illustrates one embodiment of a Feedforward cell.
- the input u is always nonnegative but has a signed representation and positive entries of the B template select u+ and negative ones select u ⁇ .
- Convolution is a very common image processing step that precedes many vision tasks. It is often used as a technique to generate a second (different) image from the original where desired features are enhanced and/or undesired characteristics (e.g., noise.) are suppressed.
- convolution can be described as a continuous spatial or temporal operation, but its application to sampled images is discrete and involves a convolution kernel with discrete values. This discrete convolution operation denoted by ⁇ S>, between an image I and kernel k can be described as :
- Equation 10 where I is the input image, x and y are the two dimensional image coordinates, and k is an (2N+1 x 2N+1) square kernel.
- Equation (1) B kl (u kl (t), U ⁇ (t),) in Equation (1) and the sum in Equation (10). kl ⁇ N r ( ⁇ )
- Normalized convolution denoted by ® n is very similar to the conventional convolution operation, except that the output is normalized by the sum of kernel entries. Because kernels are the same across the whole image, the result is essentially division by a normalization factor common to the entire image. The advantage is that the dynamic range of the resulting image is essentially the same as that of the input image.
- input voltage values are loaded with input voltages corresponding to the image, and gains from the entries of the kernel. Since the conductances for gain values) are necessarily positive, to implement negative kernel values with this approach, one needs to be able to define negative input voltages and select negative polarity for the input to the transconductance amplifier for which a negative kernel value is desired. This cuts the dynamic range of images in half, although the range of kernel entries remain unaffected. One can view the cell arrangement in Figure 1 1 (b) in this light, provided that the bias amplifier is ignored.
- a four bit plus sign representation shown in Figure 12 (digital input accommodation via parallel transistor cascading), may used as entries of the B template in Equations (1) and (4) and take on values equivalent to the range of -3.75 and 3.75 in increments of 0.25.
- the cell is equivalent to the feedforward configuration shown in Figure 1 1(a) provided that the gain of the amplifiers are sent by the B template. With more area and more bits higher resolution and dynamic range would be possible.
- other parameters may be as follows:
- V ref may be the representation of zero in this architecture.
- One suggested value for Vref are the midpoint between the power supply ranges ground and the power supply of the microelectronic implementation.
- the state node (x) is referenced to V ref and does not need a signed representation.
- Positive entry values for the feedback template "A" can be used to steer x to the + terminal and negative entries steer it to the - terminal of the wide range differential amplifier.
- a values may likewise take on values in the range -3.75 and 3.75 in increments of 0.25. With more area and more bits higher resolution and dynamic range would be possible.
- Other parameters are as follows: -3.75 ⁇ aij ⁇ 3.75 and 0 ⁇ x ⁇ Vdd.
- the feedback configuration of the feedforward weights provides for some safety against the tendency of active current computation units to be attracted to a power supply rail.
- Dual Distributed Architecture illustrated as a 5x5 array.
- the Dual Distributed Architecture contains two distinct structures: (i) programmable convolution array PCA, a sensing grid with convolution capability, which may also include some short term memory, and (ii) PLA, a programmable logic array and memory area on which transformations or alternate representations of the sensory data can be recreated.
- both areas compute in parallel and communicate with each other as needed in a serial but random access fashion.
- PCLA programmable cellular logic array
- PCLA programmable cellular logic array
- the dual distributed architecture generally provides three significant advantages as compared to currently implemented or existing architectures.
- the computational concept of the architecture is simpler and far more conventional than the concept of the cellular neural network of prior art, which employ nonlinear circuit dynamics. Moreover, practicing engineers already will need far less training to implement this architecture.
- the way operations in the dual distributed architecture are programmed is far more straightforward than the way in which one needs to go about determining or finding the correct A, B, x(0), and I parameters, as compared to currently implemented or existing architectures.
- the architecture of the dual distributed architecture comprises two parts, and each of those parts serves a separate function. In other words, each part is functional on its own and can be implemented without the other.
- the first part of the dual distributed architecture is the programmable convolution array PCA, which in one embodiment comprises a sensing grid with convolution capability, which could also include some short- term memory.
- the programmable convolution array PCA part performs arbitrary linear transformations of the whole sensory data or parts of the sensory data. The desired weights of the individual elements can be directly programmed.
- the second part dual distributed architecture is the Programmable
- PCLA Cellular Logic Array
- the PCLA is cellular, which means that each of the elements of the array connected to a set of its neighbors.
- the PCLA is not a neural network and in one embodiment relies on conventional logic to perform its operations.
- the actual logic can be implemented with conventional digital logic or non-conventional analog logic gates, such as but not limited to AND OR XOR.
- there can be a state-machine i.e., a mini computing engine at each of the elements.
- each element of the PCLA could be equipped with a sensor, or data could be scanned in from an external source.
- SOS Signed Output Sensor
- the convolution function e.g., those using non-negative gain amplifier circuits
- sensors that measure non-negative values referenced to zero e.g., light
- Positive (+) output is used for feeding into the amplifier if it contains a positive kernel value.
- the negative (-) pixel output may used for negative kernel value.
- a one dimensional example is given below.
- Figure 16 illustrates one embodiment of an exemplary layout of such a single signed output active sensor.
- the active area is a square of 100 ⁇ x 100 ⁇ and the whole cell covers a 195 ⁇ x 130 ⁇ area.
- the (grounded) METAL2 layer covering the areas of the cell that should not be exposed to light is not shown.
- the corresponding schematic of the signed sensor is illustrated in Figure 17, where the positive and negative pixel representations are marked.
- the additional follower is employed since the photocurrent is very small and the photosensing node should not be perturbed.
- PCA Programmable Convolution Array
- Figure 18 illustrates one embodiment of a design of one element of a light sensing PCA.
- each pixel cell can be addressed by the combination of row and column select signals.
- weight bits can also thus be written to selected pixels.
- Pixel midpoint determines the "0" value - i.e., no light, of the signed pixel value representation. Reset pixel will charge the pixel output to that value.
- a dedicated ground signal need not be routed since a metal layer covering the circuits (with opening at light sensitive areas) can be grounded.
- each cell contains: (1) one signed output sensor (SOS); (2) four D flip flops for storage of three bit weight and sign; (3) four multiplexers; (4) one wide range transconductance amplifier connected in the unity follower configuration; and (5) a three input AND gate.
- SOS signed output sensor
- D flip flops for storage of three bit weight and sign
- multiplexers for multiplexers
- (5) a three input AND gate a three input AND gate.
- the size of the cell can be reduced significantly, since as the layout of the single cell in Figure 19 shows, there are some empty spaces. Smaller feature sizes and added metal layers will no doubt compact the cell further. A significant area savings would result from a dynamic memory cell, since each kernel would be used for a short period of time. Several data and control signals are generally routed to each cell.
- the controls signals common to all cells include: (1) clock weight, (2) reset photodiode (or sensor), and (3) sign. Addressing signals are row and column.
- the data signals common to all cells include: (1) pixel reference, (2) inverter reference, (3) weight high, (4) weight low, (5) weight MSB, (6) weight MID, (7) weight LSB.
- elements of each row have a common row select signal and elements of each column have a common column select signal. Accordingly, the output of each cell may be summed onto a common line. Alternately, outputs can be combined along columns, rows, or block-vor along alternate designations.
- Figure 21 illustrates an example PCA layout of 5 x 5 pixels.
- each of the 25 elements use four weight bits (three magnitude and one sign) which are stored in D-flip-flops. Accordingly, the weight bits are clocked in only when both the row and the column are selected, as well as when the weight clock bit is active. The outputs of the weight flip flops the select between the weighthigh and weightlow.
- the wide range transconductance amplifier at the output stage has three bias transistors with W/L ratios of VA, Vi, and 1, each corresponding to the magnitude bits. If all weights are zero, the pixel does not contribute any current to the output.
- the pixel makes a positive or negative current contribution to the overall reading.
- the data signal, weighthigh also determines the gain on the photosensor.
- the D flip flops could be replaced with dynamic memory to save silicon area.
- the design may be implemented in a 2 micron CMOS technology design with its pads configured to fit into an area of 2.3 mm x 2.3 mm. and is embedded in a 40 pin DIP chip package. All 25 pixel outputs are summed on a common output line in this example implementation. One can select different signs and weights for each of the cells and thus perform arbitrary convolution operations. Arbitrary 4-bit kernels up to 5 x 5 can be programmed in this particular example implementation.
- PCA Process Figure 22 illustrates one embodiment of the operational process of the
- PCA Programmable Convolution Array
- the necessary weight patterns to be applied to the array for all of the operations to be performed is determined. Accordingly, in one nembodiment, a program is made of operations needed to complete the task, such that each step of the program specifies an operation, where each operation is defined by the pattern of weights to be applied to the array. In one embodiment, it is possible to simply obtain the weight pattern from results of previous operations.
- the program may be stored externally or on the same chip where the array is implemented. In an alternate embodiment, each cell of the array can have its own program memory. Referring back to Figure 22, one embodiment of the operational process of the Programmable Convolution Array (PCA) illustrated above, may be implemented through the following process:
- PCLA Programmable Cellular Logic Array
- PCA programmable cellular logic array
- the PCLA is a processing device that processes binary or digital data using a set of fixed or programmable logic elements arranged on a cellular grid. Accordingly, if the PCLA is used with a PCA, the sizes of the two arrays could be the same or different.
- the PCLA grid can function as a "scratch-pad" for a set of iconic operations, such as shape detection, contour following, pattern matching, all performed using the outputs of the PCA or external outputs, potentially at different resolutions than the data received.
- each element of the PCLA can be addressed like a memory array to which the input (e.g., outputs from the PCA) can be applied or stored, and from which results can be retrieved.
- Added functionality would result from the ability to transfer bits between neighbors (analogous to shift operations) or to arbitrary locations of the cellular logic grid (random access transfers).
- the embodiment of Figure 23 illustrates a simple implementation of the elements of the programmable cellular logic array (PCLA).
- PCLA programmable cellular logic array
- all operations are strictly local. This is not a limitation of the invention.
- global operations such as matching with a global pattern, reset, set, and other can also be implemented into the PCLA.
- added functionality could be realized (1) with increased connectivity, where each cell receives more inputs, such as those from additional cells in the neighborhood, or from arbitrary cells on the array via memory-like addressing; and (2) with increased cell memory, where the results of previous operations can be stored in the cell.
- the logic within the cell should be programmable. In fact, the program may change many times during the operation. Sample logic functions could be AND, OR, INVERT, XOR etc. of a set of the available inputs.
- the PCLA would benefit from the use of the emerging reconfigurable field programmable gate arrays (FGPA's).
- FGPA's reconfigurable field programmable gate arrays
- Figure 24 illustrates one embodiment of the operational process of the Programmable Cellular Logic Array (PCLA) illustrated above.
- PCLA Programmable Cellular Logic Array
- a program is made of the sequence of operations needed to complete the task, such that each step of the program specifies one or more functions to be performed by each element of the array.
- all elements may be performing the same function or different functions.
- the program may be stored externally or on the same chip where the array is implemented.
- the program memory may be local, i.e., each cell of the array can have its own program memory.
- These inputs may come from continuous or discretely sampled data from sensors built into the array, or they can be scanned into the input nodes from an external data source, including, but not limited to the PCA, by employing an appropriate scanning mechanism.
- the output can be stored externally or internally. If the PCLA is used in tandem with a PCA, then the output can also be sent to PCA.
- the row select and column select signals of the PCA are reminiscent of memory addressing diagrams. Only when both signals are logic high's, a specific pixel is selected. These signals may be used to load the weights to the cell.
- Figure 25 illustrates one embodiment of the operational process of using the Programmable Convolution Array (PCA) and the Programmable Cellular Logic Array (PCLA) in tandem.
- PCA Programmable Convolution Array
- PCLA Programmable Cellular Logic Array
- the necessary weight patterns to be applied to the PCA for all of the operations to be performed is determined.
- the necessary local and global logic functions that are needed for the PCLA is determined.
- an operation is defined by the pattern of weights for the PCA and the pattern of logic functions carried out by the individual elements of the PCLA.
- a program is made of the sequence of operations needed to complete the task, such that each step of the program specifies one or more operations to be performed by either the PCA or the PCLA, or both.
- the program may be stored externally or on the same chip or multichip module where the two arrays is implemented.
- the program memory may be central or local, i.e., each cell of the arrays can have its own program memory.
- PCA Programmable Convolution Array
- PCLA Programmable Cellular Logic Array
- These inputs may come from continuous or discretely sampled data from sensors built into one or more of the arrays, or they can be scanned into the input nodes from an external data source, including, but not limited to from one array to the other, by employing an appropriate scanning mechanism.
- the operation code that specifies the operation can be applied from either an external source or from internal stored program memory. 4. Allow sufficient time for the operation to complete, i.e., for the electronic circuits to reach a point where the change of the value of the output of the array equals zero or by some other criteria, the operation is deemed to be complete. 5. Read and store the output(s) of the array (s). The output can be stored externally or internally. The output can also be sent from one array to the other.
- Finger print systems currently require a sensor, memory, processor, and loads of software, which in turn limits their practical application.
- a lock on a door or a drawer or power button that has a fingerprint key are examples. The latch remains in position until one of the "authorized" fingerprints are impressed on the lock.
- the image of the fingerprint is binary
- the logic program to manipulate the image and decide whether this is the right fingerprint or not could be built directly into the device.
- the programs could be the combinations of the weights, which may be stored on the device.
- using both the PCLA and the PCA may enhance the features, increase the resolution, or offer more security.
- Figure 26 and the above description are intended to provide a general description of a suitable computing environment in which the invention may be implemented.
- one embodiment of the present invention may be implemented as a set of general context of computer-executable instructions, such as program modules, being executed by a computer, such as a client workstation or a server.
- program modules include routines, programs, objects, components, data structures and the like that perform particular tasks or implement particular abstract data types.
- program modules may be practiced with other computer system configurations, including hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, communication devices, network PCs, minicomputers, mainframe computers and the like.
- the invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both local and remote memory storage devices.
- an exemplary general purpose computing system may include a conventional personal computer 20 or the like, including a processing unit 21 , a system memory 22, and a system bus 23 that couples various system components including the system memory 22 to the processing unit 21.
- the system bus 23 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- the system memory 22 may include read-only memory (ROM) 24 and random access memory (RAM) 25.
- ROM read-only memory
- RAM random access memory
- a basic input/output system 26 (BIOS), containing the basic routines that help to transfer information between elements within the personal computer 20, such as during start-up, may be stored in ROM 24.
- the personal computer 20 may further include a hard disk drive 27 for reading from and writing to a hard disk (not shown), a magnetic disk drive 28 for reading from or writing to a removable magnetic disk 29, and an optical disk drive 30 for reading from or writing to a removable optical disk 31 such as a CD-ROM or other optical media.
- the hard disk drive 27, magnetic disk drive 28, and optical disk drive 30 may be connected to the system bus 23 by a hard disk drive interface 32, a magnetic disk drive interface 33, and an optical drive interface 34, respectively.
- the drives and their associated computer-readable media provide non-volatile storage of computer readable instructions, data structures, program modules and other data for the personal computer 20.
- exemplary embodiment described herein may employ a hard disk, a removable magnetic disk 29, and a removable optical disk 31 , or combination thereof, it should be appreciated by those skilled in the art that other types of computer readable media which can store data that is accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, random access memories (RAMs), read-only memories (ROMs) and the like may also be used in the exemplary operating environment.
- RAMs random access memories
- ROMs read-only memories
- a number of program modules may be stored on the hard disk, magnetic disk 29, optical disk 31 , ROM 24 or RAM 25, including an operating system 35, one or more application programs 36, other program modules 37 and program data 38.
- a user may enter commands and information into the personal computer 20 through input devices such as a keyboard 40 and pointing device 42.
- Other input devices may include a microphone or microphones, joystick, game pad, satellite disk, scanner, or the like.
- These and other input devices are often connected to the processing unit 21 through a serial port interface 46 that is coupled to the system bus 23, but may be connected by other interfaces, such as a parallel port, game port, or universal serial bus (USB).
- a monitor 47 or other type of display device may also be connected to the system bus 23 via an interface, such as a video adapter 48.
- personal computers may typically include other peripheral output devices (not shown), such as speakers and printers.
- the personal computer 20 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 49.
- the remote computer 49 may be another personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the personal computer 20, although only a memory storage device 50 has been illustrated in Figure 26.
- the logical connections depicted in Figure 26 include a local area network (LAN) 51 and a wide area network (WAN) 52.
- LAN local area network
- WAN wide area network
- the personal computer 20 When used in a LAN networking environment, the personal computer 20 is connected to the LAN 51 through a network interface or adapter 53. When used in a WAN networking environment, the personal computer 20 typically includes a modem 54 or other means for establishing communications over the wide area network 52, such as the Internet.
- the modem 54 which may be internal or external, is connected to the system bus 23 via the serial port interface 46.
- program modules depicted relative to the personal computer 20, or portions thereof may be stored in the remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
- FIG. 27 illustrates one such exemplary network environment in which the present invention may be employed.
- a number of servers 10a, 10b, etc. are interconnected via a communications network 160 (which may be a LAN, WAN, Intranet or the Internet) with a number of client computers 20a, 20b, 20c, etc.
- the servers 10 can be Web servers with which the clients 20 communicate via any of a number of known protocols such as, for instance, hypertext transfer protocol (HTTP).
- HTTP hypertext transfer protocol
- Each client computer 20 can be equipped with a browser 180 to gain access to the servers 10, and client application software 185.
- server 10a includes or is coupled to a dynamic database 12.
- the database 12 may include database fields 12a, which contain information about items stored in the database 12.
- the database fields 12a can be structured in the database in a variety of ways.
- the fields 12a could be structured using linked lists, multi-dimensional data arrays, hash tables, or the like. This is generally a design choice based on ease of implementation, amount of free memory, the characteristics of the data to be stored, whether the database is likely to be written to frequently or instead is likely to be mostly read from, and the like.
- a generic field 12a is depicted on the left side.
- a field generally has sub-fields that contain various types of information associated with the field, such as an ID or header sub-field, type of item sub-field, sub-fields containing characteristics, and so on.
- These database fields 12a are shown for illustrative purposes only, and as mentioned, the particular implementation of data storage in a database can vary widely according to preference.
- the present invention can be utilized in a computer network environment having client computers for accessing and interacting with the network and a server computer for interacting with client computers and communicating with a database with stored inventory fields.
- the present invention can be implemented with a variety of network-based architectures, and thus should not be limited to the examples shown.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Neurology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Amplifiers (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Semiconductor Integrated Circuits (AREA)
Abstract
Description
Claims
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU36167/00A AU3616700A (en) | 1999-03-05 | 2000-03-06 | Two architectures for integrated realization of sensing and processing in a single device |
EP00914830A EP1175659A2 (en) | 1999-03-05 | 2000-03-06 | Two architectures for integrated realization of sensing and processing in a single device |
CA002364182A CA2364182A1 (en) | 1999-03-05 | 2000-03-06 | Two architectures for integrated realization of sensing and processing in a single device |
US09/914,030 US6735482B1 (en) | 1999-03-05 | 2000-03-06 | Integrated sensing and processing |
JP2000602988A JP2002538557A (en) | 1999-03-05 | 2000-03-06 | Two architectures for integrated implementation of sensing and processing in a single device |
HK02101234.2A HK1039992A1 (en) | 1999-03-05 | 2002-02-20 | Two architectures for integrated realization of sensing and processing in a single device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12317799P | 1999-03-05 | 1999-03-05 | |
US60/123,177 | 1999-03-05 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2000052639A2 true WO2000052639A2 (en) | 2000-09-08 |
WO2000052639A3 WO2000052639A3 (en) | 2001-02-15 |
Family
ID=22407147
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2000/005785 WO2000052639A2 (en) | 1999-03-05 | 2000-03-06 | Two architectures for integrated realization of sensing and processing in a single device |
Country Status (7)
Country | Link |
---|---|
EP (1) | EP1175659A2 (en) |
JP (1) | JP2002538557A (en) |
CN (1) | CN1457471A (en) |
AU (1) | AU3616700A (en) |
CA (1) | CA2364182A1 (en) |
HK (1) | HK1039992A1 (en) |
WO (1) | WO2000052639A2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6614911B1 (en) | 1999-11-19 | 2003-09-02 | Gentex Corporation | Microphone assembly having a windscreen of high acoustic resistivity and/or hydrophobic material |
WO2004042822A1 (en) * | 2002-11-04 | 2004-05-21 | Innovaciones Microelectronicas, S.L. | Programmable mixed-signal integrated circuit architecture for the production of autonomous vision systems in a single chip and/or pre-processing of images in higher-level systems |
US7120261B1 (en) | 1999-11-19 | 2006-10-10 | Gentex Corporation | Vehicle accessory microphone |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6278377B1 (en) | 1999-08-25 | 2001-08-21 | Donnelly Corporation | Indicator for vehicle accessory |
US10346944B2 (en) * | 2017-04-09 | 2019-07-09 | Intel Corporation | Machine learning sparse computation mechanism |
CN111368253B (en) * | 2018-12-26 | 2023-09-26 | 兆易创新科技集团股份有限公司 | Convolution operation method and device based on nonvolatile memory |
CN111539178B (en) * | 2020-04-26 | 2023-05-05 | 成都市深思创芯科技有限公司 | Chip layout design method and system based on neural network and manufacturing method |
CN111983629B (en) * | 2020-08-14 | 2024-03-26 | 西安应用光学研究所 | Linear array signal target extraction device and extraction method |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5140670A (en) * | 1989-10-05 | 1992-08-18 | Regents Of The University Of California | Cellular neural network |
US5355528A (en) * | 1992-10-13 | 1994-10-11 | The Regents Of The University Of California | Reprogrammable CNN and supercomputer |
-
2000
- 2000-03-06 WO PCT/US2000/005785 patent/WO2000052639A2/en not_active Application Discontinuation
- 2000-03-06 JP JP2000602988A patent/JP2002538557A/en active Pending
- 2000-03-06 AU AU36167/00A patent/AU3616700A/en not_active Abandoned
- 2000-03-06 CN CN00807117A patent/CN1457471A/en active Pending
- 2000-03-06 CA CA002364182A patent/CA2364182A1/en not_active Abandoned
- 2000-03-06 EP EP00914830A patent/EP1175659A2/en not_active Withdrawn
-
2002
- 2002-02-20 HK HK02101234.2A patent/HK1039992A1/en unknown
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5140670A (en) * | 1989-10-05 | 1992-08-18 | Regents Of The University Of California | Cellular neural network |
US5355528A (en) * | 1992-10-13 | 1994-10-11 | The Regents Of The University Of California | Reprogrammable CNN and supercomputer |
Non-Patent Citations (2)
Title |
---|
ERTEN G ET AL: "Modified cellular neural network architecture for integrated image sensing and processing" ISCAS'99. PROCEEDINGS OF THE 1999 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS VLSI (CAT. NO.99CH36349), ISCAS'99. PROCEEDINGS OF THE 1999 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS. VLSI, ORLANDO, FL, USA, 30 MAY-2 JUNE 1999, pages 120-123 vol.5, XP002149482 1999, Piscataway, NJ, USA, IEEE, USA ISBN: 0-7803-5471-0 * |
ERTEN G ET AL: "Two cellular architectures for integrated image sensing and processing on a single chip" JOURNAL OF CIRCUITS, SYSTEMS AND COMPUTERS, OCT.-DEC. 1998, WORLD SCIENTIFIC, SINGAPORE, vol. 8, no. 5-6, pages 637-659, XP000952076 ISSN: 0218-1266 * |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6614911B1 (en) | 1999-11-19 | 2003-09-02 | Gentex Corporation | Microphone assembly having a windscreen of high acoustic resistivity and/or hydrophobic material |
US7120261B1 (en) | 1999-11-19 | 2006-10-10 | Gentex Corporation | Vehicle accessory microphone |
US7130431B2 (en) | 1999-11-19 | 2006-10-31 | Gentex Corporation | Vehicle accessory microphone |
US7136494B2 (en) | 1999-11-19 | 2006-11-14 | Gentex Corporation | Vehicle accessory microphone assembly having a windscreen with hydrophobic properties |
US7443988B2 (en) | 1999-11-19 | 2008-10-28 | Gentex Corporation | Vehicle accessory microphone |
US8224012B2 (en) | 1999-11-19 | 2012-07-17 | Gentex Corporation | Vehicle accessory microphone |
WO2004042822A1 (en) * | 2002-11-04 | 2004-05-21 | Innovaciones Microelectronicas, S.L. | Programmable mixed-signal integrated circuit architecture for the production of autonomous vision systems in a single chip and/or pre-processing of images in higher-level systems |
ES2209642A1 (en) * | 2002-11-04 | 2004-06-16 | Innovaciones Microelectronicas, S.L. | Programmable mixed-signal integrated circuit architecture for the production of autonomous vision systems in a single chip and/or pre-processing of images in higher-level systems |
Also Published As
Publication number | Publication date |
---|---|
EP1175659A2 (en) | 2002-01-30 |
CA2364182A1 (en) | 2000-09-08 |
AU3616700A (en) | 2000-09-21 |
CN1457471A (en) | 2003-11-19 |
HK1039992A1 (en) | 2002-05-17 |
JP2002538557A (en) | 2002-11-12 |
WO2000052639A3 (en) | 2001-02-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Choi et al. | A high-precision VLSI winner-take-all circuit for self-organizing neural networks | |
Moini | Vision chips or seeing silicon | |
Farabet et al. | Cnp: An fpga-based processor for convolutional networks | |
Roska et al. | Toward visual microprocessors | |
Suarez et al. | CMOS-3D smart imager architectures for feature detection | |
US6768515B1 (en) | Two architectures for integrated realization of sensing and processing in a single device | |
US11599782B2 (en) | Self-powered analog computing architecture with energy monitoring to enable machine-learning vision at the edge | |
Hasler | Opportunities in physical computing driven by analog realization | |
WO2000052639A2 (en) | Two architectures for integrated realization of sensing and processing in a single device | |
Higuchi et al. | Evolvable hardware chips for industrial applications | |
US5255362A (en) | Photo stimulated and controlled imaging neural network | |
US6735482B1 (en) | Integrated sensing and processing | |
Dogaru et al. | The simplicial neural cell and its mixed-signal circuit implementation: an efficient neural-network architecture for intelligent signal processing in portable multimedia applications | |
Laiho et al. | MIPA4k: Mixed-mode cellular processor array | |
Tripathi et al. | Analog neuromorphic system based on multi input floating gate mos neuron model | |
Lange et al. | Optical neural chips | |
Lee et al. | VLSI neuroprocessors for video motion detection | |
CN116883807A (en) | Integrated sensing and storing integrated target detection and identification system and method | |
US20040037464A1 (en) | Unconstrained handwriting recognition | |
Ruckert | ULSI architectures for artificial neural networks | |
Lee et al. | VLSI image processor using analog programmable synapses and neurons | |
Athreyas et al. | Analog signal processing solution for machine vision applications | |
Chou et al. | VLSI design of optimization and image processing cellular neural networks | |
Erten et al. | Two cellular architectures for integrated image sensing and processing on a single chip | |
Bridges et al. | A reconfigurable VLSI learning array |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 00807117.9 Country of ref document: CN |
|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
AK | Designated states |
Kind code of ref document: A3 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A3 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
ENP | Entry into the national phase |
Ref document number: 2364182 Country of ref document: CA Ref document number: 2364182 Country of ref document: CA Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 36167/00 Country of ref document: AU |
|
WWE | Wipo information: entry into national phase |
Ref document number: 09914030 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2000 602988 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2000914830 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
WWP | Wipo information: published in national office |
Ref document number: 2000914830 Country of ref document: EP |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 2000914830 Country of ref document: EP |