MXPA99007732A - Adaptive senso-motor encoder for visual or acoustic prosthesis - Google Patents

Abstract

The invention concerns an adaptive senso-motor encoder for a visual or acoustic prosthesis, said encoder having a central checking unit for signal-processing functions, monitoring functions, control functions and external action functions. The encoder further comprises a group of adaptive spatio-temporal filters for converting sensor signals into stimulation pulse sequences, a bi-directional interface being provided for coupling the encoder to an implantable microstructure (2) for stimulating nerve or glia tissue and monitoring brain functions.

Description

.4 * * < x ENGINE ADAPTABLE MOTOR DETECTOR FOR VISUAL PROSTHESIS * > O ACOUSTIC BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing system that includes a detectable motor t encoder for a visual prosthesis or an acoustic prosthesis for bidirectional coupling using microcontacts implanted for both neural or glial tissue stimulation as well as for the purpose Functional momtoreo of brain function. 2. . Description of Related Art * * Numerous attempts have been made to develop - Vision prosthesis for various groups of blind people by implanting microcontacts in the retina exit layer (RET) or within the visual cortex (VCO) and by coupling these implants with an external signal transmitter (the encoder ) in order to induce functional visual perceptions. For example, encoders for implantable vision prostheses are described in the patents of the REF .: 31188 United States 5,498,521, United States Patent 5,351,314 or * document WO95 / 06288; Implantable microcontacts for the retina, visual cortex or auditory system are described in U.S. Patents 5,597,381; US 5,569,307; US 5,549,658, US 5 545,219; US 5,496,369, US 5,411,540; US 2,215,088; US 5,109,844; and US 4,628,933. US 5,277,886, EP 0435559 and US 3,766,311 relate to neural networks and the visual system, and US 5,571,148, US 5,512,906, US 5,501,703, US 5,441,532, US 5,411,540 relate to addressing microcontacts. The target groups of the RET projects suffer from retinal degenerative disease (for example retinitis pigmentosa, macular degeneration) whereby the photoreceptor layer has degenerated but at least a portion of the retinal ganglion cells and part of the optic nerve originate in that place, as well as the central visual system still remains functional. As shown by the publications mentioned above, investigations are underway in the development of various types of implantable microcontact structures (stimulators) that are applied within the eyeball (eyeball) in the ganglion cell layer of the retina and in the development of a wireless signal and energy transfer systems for connection between the external encoder and the implanted simulator, or generally to the interface. * The inventor has been given the task of further developing an adaptive encoder for a visual prosthesis which is coupled to the retina or the visual cortex for conversion of image patterns, or for acoustic prostheses coupled to the appropriate areas of the neural additive system for conversion of sound patterns into stimulation signals through adaptive temporal space filters using characteristics of the receptor field of the respective sensory neurons (RF filters) addressed and their optimal adjustment by neural networks that act as approximators of the adaptive function. The target groups of the VCO projects typically no longer have a retrievable optic nerve function and therefore require implantation of, for example, brush-like microcontact structures in the regions of the visual cortex; that is, the occipital cortex, which is directly adjacent to the skull. There are also several familiar types of acoustic prostheses (for example, the cochlear implant) using implanted microcontacts that make possible the partial recovery of auditory perception in deaf people. The current designs of the implant systems do not provide any information processing of the dynamic image pattern data within the visual system (for example the retina) from the optical input to the neural layer with which contact is made by the implanted microcontacts (for example the ganglion layer of the retina or the neural layer in the visual cortex). Instead of this, simple image patterns (for example lines) are sent directly as stimulation signals in locally distributed microcontacts without information processing individually adapted as a substitute for that part of the visual system that is to be technologically eliminated (for example the retina). The visual system that has been stimulated in such a rudimentary and maladaptive way is confronted with the difficult problem of generating visual perceptions that are sufficiently similar to the image pattern, from trajectories of processed or incorrectly encoded signals in a local and temporal manner. In addition, the physiological adjustment of visual sensitivity (brightness adaptation) for approximately 10 decades and the functional alteration of receptive field characteristics or features related to technical photosensing systems are not taken into consideration. Currently developed implant systems use available technology and do not use visual prostheses for the purpose of warning the implant bearer of the hazards and technically identified patterns reported.

The same applies to the implant systems currently developed for the auditory system. Due to its structure established ontogénicamente and its stabilized structure as well as its function through the years of the visual perceptual experience before the presentation of the blindness, the visual system waits for example, a particular class of information for processing the data of image pattern for each retinal ganglion cell via the optic nerve. This hope is neurophysiologically transcribed by the traits of corresponding receptor (RF) fields of the neuron and is very variable, and is not satisfied to the extent that, for example, the function of the retina electrically stimulated by static preprocessing can not be adjusted individually. for each single stimulation contact produced by implant. The same applies to the auditory system. The necessary collaboration for the normal vision process of the central vision system that receives its inputs from the retina with the respective eye movement system is called the "active vision" detector motor system. Such systems for pattern recognition, tracking of an object, recognition of the position of an object, etc., are taken into consideration in conventional applications. However, the visual system requires eye movements for all vision functions and produces not only recognition functions (what is it?) But very important recognition functions (where is it?) As well as orientation in space, all which has a very high priority for a visually exposed person who is in the process of partially recovering their ability to see. However, the activation of visually induced eye movements, in the case of actual stimulation by the use of an implant system, can not be expected from only a small fraction of the retinal ganglion cells or the cells in the visual cortex Therefore, the visual system, using the implants that are currently in the development stage and which are based on normal cooperation with eye movements, can only perform unsatisfactory visual perception, if any. In addition, in several visually exposed persons, eye movements induced in a non-visual, unwanted manner, with slow and fast phases, can significantly damage the optimal use -and therefore the acceptance- of this type of visual prosthesis. If, for example, the encoder with a photosensor array is fixed on the eyeball, then the fixation point is constantly displaced by the unwanted movements of the eye. On the other hand, if the encoder is constructed within a glass eye frame, the visual system will interpret the image pattern with the eye movements that have not yet been harmonized with the image pattern, such as ambiguous visual perceptions, for example , as an apparent movement, for example, as in the case of vertiginous perceptions. Without the possibility of real eye movements, induced visually and the additional conflict with unwanted spontaneous eye movements, visual orientation in space, identifying the position of various objects in relation to one's own body position -for example for the intentional clamping of a door knob-using a visual prosthesis currently in development, which depends on the movements of the upper part and the head for the change of direction of vision, are scarcely possible. The structures to be implanted have a very limited number of microcontacts. The number of effective usable microcontacts is even smaller since only a fraction of the contacts can be placed by implantation in relation to a nerve cell or fiber, so that with the use of individual contacts, or pairs of contacts, neural action potentials with acceptably low stimulation amplitudes can be activated. In cases of current development, there is hardly an opportunity to increase the number of permanently contacted neurons77 and selectively beyond the amount accidentally established at the time of implantation. This is one reason why you can expect only a minimum quality of visual perception. The same applies to implant systems in the auditory system.

BRIEF DESCRIPTION OF THE INVENTION The microcontacts currently developed and the signal and energy transfer systems for implantation that the function of visual prosthesis unidirectionally from the external encoder to the implanted simulator and therefore does not offer an opportunity for monitoring in the process of the neural impulse activity of neurons stimulated. Therefore, the stimulation pulse frequency can not be adjusted to the spontaneous activity of the neurons. In addition, neurological impulses by stimulation impulse (exciter) can not be directly monitored. In addition, there is no safe opportunity to boost monitoring for possible temporal tuning and synchronization of the pulse sequences of several neurons. The same applies to the auditory system. The task of the present invention is to elite these disadvantages and to provide an adaptive sensor motor encoder. This problem is solved by a device that has the characteristics described in this document.

Because the encoder operates in a bidirectional coupling with microcontacts implanted on the retina or on the visual cortex then - preferably with the help of neural networks to carry out the dialogue with the implant carrier - functional tuning as required individually , several tasks of visual recognition, tracking and identification of position, as well as information of the dangers and technically identified patterns can be performed by a shift of technical image pattern and simulated eye movements, the amount of simulation sites selectively achievable It increases functionally, and you can monitor the neural activity of the neurons that are going to be stimulated. In addition, the functions of brightness adaptation encoder and composition of a visual operating range can be realized from extracts of a large function range described herein. With the implementation of an acoustic prosthesis, the adaptive encoder can provide corresponding services in the auditory area. In the case of the preferred design form of the coder for a visual prosthesis, a digital signal processor (DSP) is integrated into a glass eye frame, for example Texas Instruments Model C80 with a photo-sensor array with optics as the receiver light pattern, a preprocessing module for visual patterns, a pulse signal emitter and receiver for bidirectional communication with the implanted structure, various signal interfaces for communication with the evaluation input unit, the head movement sensor, the eye movement sensor, the perception, warning and recognition system, the pattern and object report system and the external monitoring and control system. The various adaptive information processing functions, particularly for RF filters, dialogue module, pattern recognition and active vision functions are provided in DSP with a central control unit. On the one hand, the user receives the signals as stimulation pulses or receives sensory perceptions of the encoder and transmits, on the one hand, signals regarding the movements of the head and the eye as well as an evaluation input and neural activity to the encoder. Due to a directional wireless signal and energy transfer, the encoder can be installed in a glass eye frame, attached to a contact lens in the eye, placed in the body, or at a location remote from the body. Finally, the temporal space functional range of the RF filter includes, for example, the use of a retinal encoder, the characteristics of the reception field of the ganglion cells of the retina or other kinds of neurons within the retina of the retina. of primate. The same applies to the preferred designs of the encoder in coupling with the neurons of the visual cortex or in the case of an acoustic prosthesis for coupling with neurons of the auditory system. In FIG. 1 and FIG. 2, a suitable form of a method for setting or setting the RF filter of the encoder in the dialogue with the user is illustrated diagrammatically. RF filters are executed as space-time filters whose spatial and temporal functional parameters are modified in a sufficiently large range of function for approximation of the reception field characteristics of visual neurons, specifically by externally accessible parameter exchange points placed in appropriate positions in the filter algorithms. A human being communicates as a person with normal vision or implant carrier in a dialogue based on perception with the coder of the comparisons of perception between the desired patterns and the real ones.; for example, through an evaluation input unit consisting of a line of several immersion switches (see Figure 2) to a technical neural network with non-monitored adaptation rules and the neural network establishes the following parameter vector for the FR filter as well as the next desired pattern with the objective of reducing the difference of pattern received in the next stage of dialogue. In the search for optimal parameter vectors for the RF filter can occur in the dialogue mode of a neural network using the unmonitored adaptation, which results in some visual perception for a given lighting pattern presented and interpreted subjectively in an appropriate manner . Alternatively, in the dialogue module, another parameter adjustment system may produce sequences of parameter vectors for virtual movement in the functional space of the RF filter, for example as continuous trajectories that depend on the type of scan or sweep as sequences lacking in rule. or as sequences of neurophysiological functions especially typical of the filter and the user, during the process of this sequence in an appropriate synchronization scheme occasionally presents "sensitive" perceptions that result from the interaction of a given pattern of light, the series connects the module of preprocessing, the series connects the RF filter and part of the central visual system coupled via the appropriate microcontact. Then a more precise optimization of parameters based on perception is carried out in the interval determined in this way from the filter function space. A suitable form of dialogue-based establishment of the RF filter of the coder of an acoustic prosthesis is performed in a similar manner. In the production of asynchronous pulse sequences, the output signals of the individual RF filters are transformed by appropriate conversion algorithms from quasi-continuous temporal functions of the RF filter into synchronous pulse sequences appropriate to the activity of the visual neurons in the visual system of primates and impulse sequences-temporal courses of the presentation of individual impulses are diverted by elements of time delay variables in the dialogue phase. Normal blind people carrying a perception comparison between the desired pattern of the left screen and a corresponding real pattern on the right screen. A suitable form of the vision system model for the comparison of perception in normal blind people is found in the fact that for a series of RF filters, individually or as a group, a respective appropriate inverse representation is provided and therefore the vectors of parameters are established considering them precise. "By sequential switching with the encoded an actual image is produced on the right screen are considerable similarity to the desired pattern." At the start of the dialog the RF filter parameter vectors are set to random start values so that initially there is a clear difference between the patterns, but in the course of the dialogue with the unmonitored adaptation becomes consistently less.Functional adaptation of the RF filters for the implant carrier in the perception-based dialogue occurs in contrast to the functional adaptation for people with sight normal where the normal perception is not accessible on a monitor but only internally accessible to the implant carrier and where the desired perception is communicated to the implant carrier is, for example, as speech information or as a tactile contact pattern on the When used in acoustic prostheses, there is a similar situation in Onde, instead of an inaccessible additive organ, for example, along with the sense of touch, you can use the sense of vision available to communicate the desired perception. The coupling of the asynchronous pulse sequences produced by various RF filters of the encoder to activate the neural pulses occur at the transmission time points of the individual pulse signals which are varied by controllable time delay elements so that there is a temporal coupling that results in accurate synchronous presentation so as to control the variation of time delay by the implant carrier resulting in dialogue as a perception based on events by means of a neural network or that is externally controlled, where the selection of the impulse groups to be temporarily coupled can be taken into account [berücksichtlgt] both for the pulse coming from the RF filter as well as for the impulse recorded at the interface, and in which of the momentary impulse velocities very different from the various RF filters, suitable criteria are established for the participation of individual impulses in the impulse groups to be coupled. For the purpose of functional improvement of the number of separations (separation powers) of the stimulation sites selectively achievable with a given number of stationary microcontact implants, pulse signals from a given RF filter are guided to several neighboring microcontacts . The characteristic time courses of the electromagnetic field in the area of the neurons to be stimulated - based on the instructions or commands of the encoder and in the exact setting for each microcontact and the decoded stimulation time functions corresponding in the interface with with respect to the amplitude of current, polarity and phase length- have the effect that these stimulation signals that are tuned to each other by superposition, locally and temporarily activate selective neural impulse excitations of field resistance of several microcontacts. Selective stimulation occurs through appropriate variation of superimposed stimulation signals and can be changed rapidly. The corresponding variation of the various parameters of the stimulation signals reciprocally tuned in the perception-based dialogue with the implant carrier occurs via the neural network, or other process of signal variation in the adjacent electrodes, such as, for example , a continuous automatic displacement, similar to scanning the function parameters of the stimulation impulses that are superimposed on the nervous tissue and used in the determination of as many stimulation sites as possible resulting in neural excitation. In addition, through the corporation of the neural impulses recorded with the stimulation signals, the optimization of the stimulation time functions is improved with respect to the proposed selectivity of a single cell and the permanent biocompatibility. For the purpose of stimulating eye movements for the use of the encoder in a vision prosthesis, an image pattern shift is made electronically in the input layer of the encoder by optical variation of the viewing direction; for example, with the help of a mirror in movement or by the movement of photosensors to stimulate the movements of the eye. Head movements and eye movements are detected by microminiaturized motion detectors that operate multidimensionally, and conventional motion or neural guidance or control is provided by utilizing the detected head and eye movement and displacement signals of image pattern. Fast and slow eye movements for pattern recognition tasks, rapid peripheral scanning of eye tracking movements occur and by means of appropriate movement sequences of the eye with fast and slow phases, an optimal adaptation of the eye is obtained. Sensory data flow in the central vision system that responds. With respect to the production of eye tracking movements, there is a suitable design form where a neural prediction element, adaptive, acquires the function of movement time unknown initially on the object that is to be followed from the position and errors of movement of its projection onto the photodetector arrangement, and uses an appropriate non-linear adaptive prediction algorithm with consideration of the analyzed frequency segments of the object movement and a time-tracking function of the object with minimal delay or even with minimal electrode it is generated internally with reliability that increases rapidly and, in the case of temporary disappearance of the object that is followed, for example, similar to the situation in which the eye tracking system in primates produces a movement time function which, depending on the nature of the object, results in a continuation of the This is the result of the reappearance of the object that has moved in relation to the sensor arrangement, with minimal error of position and movement. For the purpose of detecting the movements of the eye and the head and to compensate for unwanted movements of the eye, rapid and slow movements of the eye are produced by the use of detected head and eye movement signals and movements of the eye. simulated eye [created] with the help of a control or guide of neural or conventional movement. With the help of the control circuit, the unwanted movements of the eye are compensated by following an appropriate period of adaptation and thus a suitably satisfactory stimulation of the vestibular / ocular reflex (that is, the automatic stabilization of the reflex of the direction of vision in the eye). the space by eye movements that counteract the head movements that can occur) is produced from the head movement detector, image pattern shift and a neural network in a control loop optimized in the adaptation period, allows the positional stabilization of the image pattern, in the presence of natural movements of the head and the upper body, by means of corresponding ocular compensatory movements. Simulated eye movements and compensatory eye movements are available as separately selected programs. Individual movements and movement modes can be selected as separate or combination programs, assigned for automatic operation or set externally. The implant carrier can select the "active vision" functions, such as search around or tracking an object, by means of a portable instruction input device; for example, a manual device with keyboard. The communication that occurs to the implant carrier of the current position of the objects captured visually or acoustically in space consists of determining the coder of the position of the objects by evaluating image patterns or sound patterns, movements of the eye and of the head using a portable signal transmitter, appropriate, presented to the appropriate sensory organ; for example, for the sense of touch, and wherein the encoder by means of an internal pattern recognition program, in particular together with eye movements that occur automatically, warns the implant bearer of obstacles or hazards and reports the type and position of technically identified patterns or objects. A monitoring system for a partially detector motor that operates autonomously from the implanted structure of the encoder consists of the implanted microcontacts that are used both for stimulation and for the registration of neural impulses, so that the recorded impulses and other physical signals or chemistries for the implanted structures are presented to the encoder by the appropriate preamplifiers and the electromagnetic transmitters and where, once there, the recorded neural signals are further processed for various purposes of the coding functions. For the purpose of technical adaptation of the brightness-operation range from, for example, via a range of photosensor array function extending over 6 to 10 levels of brightness, which includes both large portions of the scotopic range and dark adaptation as well as the phototypic range of adaptation to brightness, an electronic display quickly adjustable over an internal operating range for the encoder, in relation to brightness size and adaptation, is adjusted based on perception, and is selected automatically or by the user. By doing this, the different scenes of the very different brightness within the current operating range are compensated for and therefore a contrast optimization and avoidance of dazzle is obtained. In addition, an advantageous design is inherent here where the functional RF filter; for example, corresponding neurobiologically to family processes that can be adjusted using the adaptation interval.In a suitable form, a processing module is provided which is equipped for recognition of characteristic patterns and / or for data reduction. When this is done, the processing module can recognize brightness and dark image areas of different luminance, separated from each other and integrated into a general image with the respective regional optimum contrast; in the same way the image areas that are separated in a close or remote way from each other, can be optimized with respect to the adjustment of sharpness and then they are reintegrated as a total image and, finally, characteristic patterns can be emphasized, for example , warning signs. This preprocessing can be used to improve the presentation of image, but can also be used for the production of warnings and for data reduction in the communication channel between the camera and the encoder. It is an additional advantage if in the course of a process the adaptation of this accommodation is adjusted selectively, for example, in the first plane, and these intervals are stored, and then the secondary intervals of the visual field are adjusted in a defined manner as images; for example, the background of a visual field. The initial intervals that now become flat (that is, undefined) are removed from the pattern of the second image and replaced by the defined intervals stored first. By doing this, a depth of definition is produced that is not affordable in the scope of geometric optics. With the cyclically repetitive course of this process step and the corresponding image balance, the user remains without perceiving the process so that even with a minimum light intensity and large focal lengths, the appropriate optics of a large image are apparently obtained. definition. The adaptive preprocessing module processes the image pattern or sound pattern by neural networks or in preprocessing algorithms for a visual prosthesis accessible in physical elements (hardware) particularly with respect to color, contrast, edge detection, segmentation and separation of figures with respect to the background or, in the case of an acoustic prosthesis, for example, with respect to the suppression of interference noise, unwanted formations and separation of individual sound sources in such a way that the subsequent RF filter arrangement is simplified considerably and contains image patterns or sound patterns formed in part from the pattern area in a feature or area of properties, which are as well adapted as possible to part of the visual system or additive system contacted. The various preprocessing functions are established directly by the user or are selected by a perception-based dialogue, or are automatically set. The simplified image in this process and communicated to the retinal encoder is advantageous for recognition of figures even in the case of a limited number of visual system neurons contacted. In preprocessing for acoustic prosthesis, the corresponding benefits are obtained if complex sound patterns are prepared in the course of preprocessing for speech recognition.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates a visual prosthesis that uses the example of a retinal implant. Figure 2 illustrates an adaptive encoder with a dialogue system. Figure 3 illustrates a drawing of the eye movement and head movement detectors, the retinal encoder anterior to the eye and a neck collar for tactile responses of object positions. Figure 4 illustrates a schematic illustration of a microcontact structure implanted for stimulation of nerve tissue to which there is no direct contact. Figure 5 illustrates a scene with very different brightness ranges and areas of different focal length. Figure 6 illustrates the course of brightness steps in the camera image for four different image areas from Figure 5.

DETAILED DESCRIPTION OF THE PREFERRED MODALITY Fig. 1 shows schematically a visual prosthesis using the example of a retinal implant with an adaptive encoder motor detector with image displacement mounted on a glass eye frame; an interface implanted in the eye in proximity to the ganglion cell layer for stimulation and registration as a microstructure (2) with accessory electronic parts and a bidirectional wireless signal and energy transfer system between the encoder and the interface. The individual time-space filters (RF filters) of the encoder with each of the approximately round cutouts of the photodetector array (4) in the input (left) layer and the related signal outputs in the output (middle) layer represent typical of the reception field, for example ^, of ganglion cells of the primate retina, or of neurons in the visual cortex and are individually adjustable so that they function by parameter vectors. The interface receives not only stimulator signals from the encoder but also sends neural impulse signals recorded to the recorder. The components that are additionally associated with the encoder such as, for example, the central control unit or the processing module connected upstream of the RF filters are not shown in Figure 1.

Figure 2 shows the distribution of the encoder (1) with the dialogue system with a coupling with the central visual system, either as a model (implementation in a person with normal vision), or as a real system from the level of implanted microcontacts to visual perception ( implementation in the implant carrier).

The angle as an exemplary image pattern on the photodetector array (left) represents at the same time the desired pattern on the monitor, which moves up and to the right in the internal connection pattern pattern that is made electronically in the encoder, the angle in the example moves from another start portion, in another direction, in order to indicate the function of a technical image shift. The screen to the right shows the appropriate real image pattern and represents either a second monitor (person with normal view) or a virtual projection of the area of visual perception (implant carrier). The elliptical disk that moves from the upper left to the lower right represents the corresponding real perceived image pattern. The person (bottom right) articulates his subjective evaluation in the comparison of the desired and actual patterns over a multi-channel evaluation entry. The dialogue module as a neural network with a decision system (below) forms the output signals of the evaluation input in a parameter vector for RF filter adjustment. With the replacement of the image, the patterns with sound patterns and the visual system with the auditory system, figure 1 applies correspondingly to the use of coders for acoustic prostheses. Figure 3 schematically shows a suitable design form of an encoder with respect to the positioning - of a head movement sensor (K) operating multidimensionally above the ear, an eye movement sensor (A) operating multidimensionally and an encoder that has been integrated into the glass eye frame with the indicated moving image pattern. A collar (H) in the throat for the production of local sensations to the touch, for example by vibration, provides the user with information regarding the position of an object in position to the user. If an object is in front of it and you want it to move out of the field of view to the left, then the area of vibration produced in the throat also moves to the left, as indicated in Figure 3. Figure 4 shows an illustrated example of microcontacts (6) that affect the nervous tissue (8). In the present example, three microcontacts (16, 17, 18) are implanted in the nervous tissue (8) and placed more or less randomly close to certain nerve cells. The microstructure (6, 16, 17, 18) of the mocrocontact is essentially uniformly thicker than the matrix of the nerve cells (8). The microcontacts (16, 17, 18) are supplied with signals (SI, S2 and S3) by means of the stimulator (12). In order to create directed neural excitation, a stimulation focus must be reached, for example (F) that can not be directly affected by the microcontact. However, the focus (F) of stimulation can be reached if signals (SI, S2, S3) are passed to the electrodes (16, 17, 18) using different resistances, time courses and above all separation in time. The placement in the upper part or superposition of the signals produced can be established in such a way that the convergence of the signals in the area of the proposed stimulation focus (F) exceeds the excitation threshold of the individual or of some nerve cells, while that the addition of the signal that flows in the rest of the nervous tissue area remains below the excitation threshold. By changing the temporal sequence and the temporal signal flow of the various signals tuned to each other, the focus (F) Stimulation can also be shifted to (F1). For the precompensation of these stimulation functions that reach a stimulation focus that is not in direct connection with an electrode, an adaptable process is required. Since the way in which the stimulation focus (F), a (F1) for a particular neural stimulation must be resolved is not known precisely, the adaptive detector motor control unit may offer only a particular signal pattern that the implant carrier then determines by means of sensory perception or other evaluation of sensor data. A second signal pattern that has changed in comparison to the first one, is then also determined subsequently so that it reaches or does not reach the directed neural excitation. The user only needs to tell if the last signal pattern is better or worse than the previous one. Using this control mechanism of a neural network, an optimal signal time function is determined for the electrodes (16, 17, 18) for stimulation of the stimulation focus (F) in the course of the control process. Figure 5 shows a scene perceived under practical conditions by the photodetectors in which the door (30) of a patio is observed from the interior of a room. The door (30) shows a window (31), a keyhole (32) and a door panel (33). At the front of the door there is a spider (34) and through the window (31) a beach scene (35) is visible. The differences in lighting in this scene are between approximately 10"1 cd / m2 in the area of the door lock, 109 cd / m2 in the area of the chandelier and 101 cd / m2 in the panel area of the door or up to 104-10s cd / m2 in the outer area.Those differences in brightness are not visible simultaneously using conventional cameras and even otherwise with the human eye.The brightness adjustment always occurs only in the area observed at the time Figure 6 shows diagrammatically the manner in which the preprocessing module of the camera (1), due to its pattern recognition functions, delimits the individual areas from each other and converts them using different functions in the layers of brilliance of the formation of the camera image On the x axis the brightness (ie, luminance) is represented in cd / m2 over a total of 9 decades, in the same way as it is produced in the real image in figure 5. E l axis and shows 256 relative units of brightness information, attributed to the image representation by the camera or its preprocessing module, 265 units correspond to 8-bit brightness modulation. An initial brightness curve L32 shows the brightness area of the door latch (32), illustrated in the 256 relative brightness levels of the camera's image formation. The corresponding brightness curves L33 for the door panel (33), L34 for the spider (34) and L35 for the outer area (35), are likewise illustrated.

The preprocessing module recognizes the different detailed image formation and delimits one of the other defined contour areas with the four different brightness areas. These areas are separated in their construction from each other and each one is transported with optimal resolution of the 256 brightness levels of the camera's image formation. In the result, the scene is shown to the observer as an image in figure 5 in which the image areas (32, 33, 34) are illustrated with equal brightness and with the corresponding structuring at the various levels of brightness. Such an illustration may be unusual but it provides a wealth of detail in various regions that can not be illustrated simultaneously with the human eye or with conventional camera systems. The illustration in figure 5 also shows objects at various distances. Thus, for example, the objects (32, 33 and 34) are at a distance of 2 meters from the observer, while the palm (36) in the outer area (35) can be at a distance of 40 m. By using conventional camera lenses, it is generally not possible to simultaneously display both objects (34 and 35) with the same definition. The available definition ranges are not adequate to do this. Using the processing module, the adaptive detector motor encoder can initially place the remote area (35) within definition and recognize and store the delimited regions defined in its outline (the palm) at that location. Then a second interval (ie distance) can be selected in which the spider (34) is clearly defined, so that the area (35) becomes undefined, ie (blurred). The preprocessing module can recognize this condition and instead of the blurred region (35), it will incorporate the captured defined image unit previously determined constructively within the focused image at close range. This sequence can be repeated cyclically in a focus scan class so that it forms defined areas with different focal lengths which are determined continuously, captured and incorporated into the total defined image. The visual definition that is obtained virtually in this way is many times better than that of a normal optical system. With an adequate frequency of reiteration of the process, the produced image can be differentiated by the user only by the particular definition. According to the invention, an encoder is recommended which optimizes the various functions by neural networks in dialogue with the implant carrier, in which various modes of function can be selected and the positions of the objects taken can be used and in which be aware of the obstacles, technical recognition of pattern reports as well as functional increments in the number of selectively addressable stimulation sites and monitors of the neural activity of individual neurons. The implanted structure can operate almost independently of the detector motor when using suitable detector and motor components as well as an adaptive control system. Advantageous designs and variations of the invention are discernible. The adaptive encoder is characterized in comparison with conventional visual prosthesis systems by numerous essential advantages. First of all, an encoder that is pretrained by people with normal vision and then to be individually adapted by the implant carrier to its functional requirements is recommended here. For the first time, an encoder is described here that provides eye movement functions as well as compensation for unwanted movements of the eye. In addition, for the first time, an encoder is described that functionally increases the number of stimulation sites that are selectively available and that can subsequently be adapted to new stimulation conditions. In addition, an encoder that works bi-directionally is described for the first time; therefore, together with the stimulation functions, it also allows the monitoring and evaluation of the neural activity of the neurons that are going to be stimulated. The corresponding advantages will result with the use of the adaptive encoder in front of previously developed auditory prosthesis systems. The adaptability of the individually adjustable temporal space filter image forming functions of the encoder using the receptive field properties (RF filters) over the entire relevant neurophysiological functional range will be secured together with the neural networks or other signaling processes. Establishments of parameters when they are used for visual or acoustic prostheses. The individual image formation functions of the individual RF filters determined in the perception-based dialogue are sufficiently similar to the receptive field properties expected by the visual system; therefore, they adapt to the function of the visual system created by the battery connection of the encoder and the coupled central vision system. This means, on the one hand, that the interval of temporal space function prepared by RF filters incorporates the neurophysiologically relevant fusion interval and, on the other hand, that RF filters allow, with the help of a neural network, a continuous movement in the function interval with adequate adjustment procedures. The same applies with the use of an encoder in acoustic prosthesis.

Already in its use in people with normal sight, a reasonable adjustment by omission of the RF filter has been carried out using corresponding neurophysiological data regarding the function of the visual system or the 'primate auditory systems. In addition, the dialogue process is tested using the associated components under realistic conditions by stimulating the perception process. The same applies with the use of the encoder in acoustic prostheses. The RF filters associated with the individual microcontacts are individually tuned to the optimum visual or auditory perception quality in the dialogue between the encoder and the implant carrier. In contrast to an encoder with static preprocessing; that is, one without the opportunity for individual preprogramming, the individual RF filters are adjusted as separate encoding channels based on the unique relevant criterion; specifically directed visual or auditory perception. The subsequent function changes; for example, as a result of the re-assignment of micro-contacts, or changes in the functional parts of the central visual system, they can be compensated throughout the perception process by the appropriate adaptation of the RF filter functions. One advantage of tuning the function of the RF filters in the dialogue with the implant carrier is the consideration of aspects of function that only the implant carrier can present and then only implicitly by subjective evaluation of their visual perceptions and their implementation in the encoder setting in the optimization process. The same applies to an acoustic prosthesis. The synchronous pulse sequences of the individual RF filter outputs of the current functionally separate encoder channels are tuned to each other as stimulation signals for selective stimulation sites in the dialogue with the implant carrier in consideration of the recorded neural impulses. Due to the temporal coupling or synchronization of the neural impulses of several neurons for coding of neurobiological signal in sensory systems which is used, this is carried out technically (also by evaluation of the recorded neural activity of neurons to be stimulated), Temporary coupling has the advantage of improving the quality of visual perception. The number of selectively achievable stimulation sites as well as their definition (separation definition) in the case of a fixed number of implanted microcontacts is functionally increased. With a given relatively small amount of implanted and functionally permanent microcontacts, whose relative position with respect to neurons can not be modified, it is a considerable advantage, functionally speaking; that is, by producing signals suitable for increasing the number of selectively achievable stimulation sites or neurons and therefore at the same time the number of coding channels accessible separately with sufficient reserve in the RF filters. This produces an improvement in the quality of visual perception. The detection of the eye and the movements of the head have the advantage of determining the current position of visual objects in space. In addition, there is an advantage that the actual unwanted movements of the eye can be compensated by appropriate stimulated movements of the eye and, in addition, visual perception conflicts such as, for example, apparent movements or vertigo are suppressed. The production of individual movement functions as programs that can be selected as separate combined programs and that have the advantage that the implant carrier itself can select the programs based on their uses for the quality of visual perception, instead of being submitted to an automatic function. However, the choice can be made between the automatic operation and the option operation. It is very important that the implant bearer is able to perceive the current position of the perceived visual or auditory object in order to be able to correct its orientation in space accordingly and, if necessary, your activities. In addition, it is of considerable benefit that the implant bearer is automatically warned of obstacles or hazards and that he or she is informed of the technical recognition of patterns or objects that support their orientation in space. With the encoder a direct connection is established to a part of the nervous system that is already spontaneously active. Therefore, neural impulses from individual neurons are generated without technical stimulation. The monitoring of the neural activity of individual neurons to be stimulated is of considerable advantage for optimal adaptation of the impulse stimulation sequences to the respective spontaneous activity, for precise determination of the stimulation parameters to ensure a biologically compatible conversion. of impulses of stimulation in neural impulses as well as for an improved optimization of the temporal tuning and synchronization of the neural activities of the various neurons. With the technical adaptation of the operating range it is possible to adapt, in a brightness range adapted to the bright or adapted to the dark, the function for the image pattern or the sound pattern, and consequently the filter parameters are varied temporary space or to technically compose an operating interval that, for example, consists of partial areas of the largest photodetector function interval that are separated from each other by decades. The preprocessing of incoming image patterns, particularly with respect to the quick selection and change opportunities of the respective preprocessing function becomes possible. With the preprocessing module connected, the function of an encoder consists of only a limited number of RF filters which is facilitated by the essential simplification of the image pattern or sound pattern and consequently the quality of perception is improved. It is noted that in relation to this date, the best method known to the applicant to carry out the aforementioned invention, is the conventional one for the manufacture of the objects or products to which it refers.

Claims (28)

CLAIMS Having described the invention as above, the content of the following claims is claimed as property:

1. An adaptive detector motor encoder for a visual prosthesis or an acoustic prosthesis, with a central control unit for signal processing functions, monitoring functions, control functions and external feedback functions as well as a group of time-space filters adaptable to the purpose of conversion of the sensory signals into stimulation pulse sequences, characterized in that the visual prosthesis is provided with a bidirectional interface for coupling the encoder with an implantable microstructure for the stimulation of nervous tissue and glia, as well as for functional monitoring of brain functions.

The encoder according to claim 1, characterized in that the control unit includes, in a perception-based dialogue process, adjustable temporal space filters with reception field properties, RF filters.

The encoder according to claim 1, characterized in that a simulated eye movement is provided with head and eye movement detectors.

4. The encoder according to claim 2, characterized in that the RF filters are associated with time delay elements for the relative temporal sequence of the pulse sequences produced.

5. The encoder according to claim 2, characterized in that a monitor is provided that exhibits the imaging functions produced by the RF filters at least approximately inverted and optically or acoustically.

6. The encoder according to claim 1, characterized in that, ___on the side of the implant, there is provided a device for application of an active substance controlled by the central control unit and, on the side of the implant, is recorded and transmitted. activity of the neurons to a monitoring system of the central control unit.

7. The encoder according to claim 1, characterized in that the adaptive preprogramming module is provided in the central control unit of the encoder for simplification of image patterns or sound patterns.

8. A process for use with an encoder, according to claim 1, characterized in that: a. for the purpose of individual adjustment of the signal processing functions in a dialogue process, a single, minimal channel evaluation input unit supplied using a subjective evaluation vector, whereby the evaluation vector represents the similarity of the currently perceived image pattern, or a sound pattern, to a desired pattern; b. the evaluation vector in the dialogue module is transmitted to a parameter adjustment system for the purpose of producing suitable sequences of parameter vectors, in particular with a neural network with unmonitored adaptation rules; c. the dialogue module generates a parameter vector in a multiple channel output so that the respective signal processing functions are adjusted.

9. The process according to claim 8, characterized in that the signal processing functions are constituted by RF filter.

The process according to claim 9, characterized in that the dialogue module establishes the appropriate pattern sequences for the adaptive phase in a decision system for single or group optimization of the RF filters.

The process according to claim 10, characterized in that the internally stored sequences of the parameter vectors are generated for the purpose of establishing typical RF filter functions.

The process according to claim 11, characterized in that the RF filters have a temporary space function space that includes the function space of the receptive field properties of visual or auditory neurons at the respective implantation site.

13. The process according to claim 12, characterized in that a signal generated by the RF filters is emitted, for the release of a real perception in the microstructure and simultaneously the associated desired pattern is passed over a second output unit that is accessible to other human sensory organs.

The process according to claim 13, characterized in that, with a given number of stationary implanted microcontacts, the impulse signals are conducted by an RF filter to several adjacent microcontacts locally for the purpose of increasing the amount and definition of the sites of stimulation selectively available.

15. The process according to claim 13, characterized in that the selective stimulation sites can be changed rapidly by suitable variation of the superimposed pulse signals for local displacement of the stimulation source.

16. The process according to claim 13, characterized in that the variation in the stimulation signals for the displacement of the stimulation focus and in dialogue based on the perception with the implant carrier occurs via a neural network of another optimization algorithm. in the central control unit for the determination of the largest possible stimulation sites resulting in selective and defined neural excitation.

The process according to claim 13, characterized in that, in comparison with the neural impulses registered to the stimulation signals, the additional impulses that result from neural impulses that occur spontaneously and those produced by stimulation, the compound neural excitation with respect to selectivity and biocompatibility.

18. The process according to claim 1, characterized in that, for the purposes of stimulating the movements of the eye, the movements of the head and real undesirable eye movements are detected.; the movements of the eye are simulated by means of electron image pattern shifts and / or optical variation of the direction of vision and / or movement of the photosensors through the use of true movement signals of the head and eye detected as well as The stimulation of eye movements using a control or movement guide for fast and slow eye movements that occur for pattern recognition tasks, tracking moving objects and for quick circumspection.

19. The process according to claim 18, characterized in that the movements of the head and the true movements of the eye are detected and where the detected movements of the head and the eye are used with the aid of a movement control or a fast and slow regulator that simulates eye movements that occur in order to compensate for true unwanted eye movements or perceptions of apparent movement.

The process according to claim 18, characterized in that the compensatory movements of the eye with reference > The vestibule / ocular reflex is generated to stabilize the situation of the image pattern in the presence of normal movements of the head and upper body.

The process according to claim 18, characterized in that, for the purpose of technical adaptation of brightness, or illumination, the operating range resulting from the range of function of the photodetector array extends over several decades of brightness, a operating range for the encoder with respect to the magnitude and adaptation brightness can be selected and the selected operating range and operating range can be composed of sections evaluated non-linearly of the function range.

22. The process according to claim 18, characterized in that, by means of visual scenes of image formation system with very different local subsets of brightness can be transported to a real common brightness operating range of the encoder and thus, by For example, visual perception can become non-variable compared to a change in the average brightness of the scene.

23. The process according to claim 18, characterized in that the RF filter operates for perception stimulation in the adapted brightness and dark range, if required, and is shifted using the brightness operation interval of the encoder.

The process according to claim 18, characterized in that the selection of the brightness operating range and the associated RF filter shift, particularly in the perception-based dialogue, can be changed rapidly automatically during the simulated movements of the eye or during pattern recognition.

25. The process according to claim 18, characterized in that in a process, by variation of accommodation, the initial areas of the visual field are defined precisely and the areas are stored; after this, the secondary areas of the visual field are adjusted in a defined manner as an image, so that the first areas become flat and in addition, the first stored areas are removed from the image instead of the first areas that become flat .

26. The process according to claim 18, characterized in that the process repeats itself cyclically.

27. The process according to claim 1, characterized in that the position of an object visually detected in the space determined by the encoder by evaluation of the image pattern, the eye and head movements or where the position of an object in acoustically detected space by interpretation of the sound pattern movements it is retransmitted to an appropriate sensory organ with a suitable portable signal transmitter.

28. The process according to claim 1, characterized in that in the encoder, by means of the internal pattern recognition programs, particularly in relation to the automatic operation, simulated eye movements, warnings of the implant carrier with respect to obstacles or hazards and reports on the type and position of technically identified patterns or objects.