Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
  • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
  • H04N25/445—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by skipping some contiguous pixels within the read portion of the array
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/47—Image sensors with pixel address output; Event-driven image sensors; Selection of pixels to be read out based on image data
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
  • H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/70—SSIS architectures; Circuits associated therewith
  • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
  • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/70—SSIS architectures; Circuits associated therewith
  • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
  • H04N25/78—Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters

Definitions

• Embodiments of the invention relate to a readout arrangement for an image sensor.
• Embodiments according to the invention relate to an architecture for compressing the image sensors to be read.
• Capturing images with the help of image sensors is useful in many applications. In particular, it is desirable in cases to further process the image data obtained by the image sensors to obtain information about an image content.
• Image sensors with pixel-parallel (or pixel-parallel) signal processing in a sensor matrix provide intermediate results and final results, which are then transmitted to a next processing unit or to an output interface. If a lot of data is generated, it may be that the bandwidth is too low and the results can not be retrieved.
• An embodiment according to the invention provides a readout arrangement for an image sensor.
• the readout device is configured to receive in parallel a plurality of image sensor analog signals describing brightness values detected by the image sensor in analog form from a plurality of column lines of the image sensor.
• the readout arrangement is further configured to select which subset of a plurality of analogue values represented by the image sensor analogue signals or based on the image sensor analogue signals in an analogue memory for a further processing, and to cause the storage of the selected analog values in the analog memory, or to store the selected analog values in the analog memory.
• This readout arrangement is based on the recognition that a preselection of analog values represented by image sensor analog signals or derived from the image sensor analog signals (typically by a preferably fast analog signal processing) and an injection of these analog values into an analog memory Reduction in the amount of data that must be processed digitally (for example, following a high-resolution analog-to-digital conversion).
• a reduction of the data volume takes place, so that it is no longer necessary to convert analog values of all picture elements of the image sensor into digital signals in a complex manner. The effort in the analog-to-digital conversion of image sensor analog signals is thus reduced.
• the amount of data to digitally transmitted image sensor information can be significantly reduced, which significantly increases the performance of a system that includes an image sensor and the readout arrangement and significantly reduces a required data rate.
• selected analog values that is to say, for example, not all analog values of all image sensor pixels
• a certain sorting of the analog values can also take place, so that, for example, As analog values that are to be processed together after an analog-to-digital conversion, can be stored in a contiguous area of the analog memory.
• access can be simplified, for example, and an additional effort for resorting digitized values can be reduced or avoided.
• the read-out arrangement described here for processing image sensor output signals recommends the use at high image repetition rates. rations and, moreover, can be implemented without substantial impairment of the properties of an image sensor (eg a fill factor).
• an image sensor eg a fill factor
• the readout array may include a column-parallel processing unit.
• the readout arrangement can be designed, for example, to decide based on an evaluation of the image sensor analog signals which subset of a plurality of analog values represented by the image sensor analog signals or based on the image sensor analog signals is stored in an analog memory for further processing.
• analog values represented by the image sensor analog signals or based on the image sensor analog signals can be stored in the analog memory for further processing, which is read by the read-out device as relevant for further processing (for example for a determination a layer of lines).
• the evaluation device can be decided, for example, by a fast analog preprocessing (such as, for example, a threshold value comparison of an analog value with a threshold value or a difference between two analog values and a subsequent comparison of the difference with a threshold value), which analog values (resp which image sensor analog signals) are to be assessed as relevant.
• a fast analog preprocessing such as, for example, a threshold value comparison of an analog value with a threshold value or a difference between two analog values and a subsequent comparison of the difference with a threshold value
• analog values resp which image sensor analog signals
• the readout arrangement is adapted to detect when a trace of an image sensor analog signal associated with a respective image column has a history over a plurality of image lines satisfying a predetermined condition.
• the readout arrangement may be configured to store analog values describing the history in response to detection that a trace of an image sensor analog signal associated with a respective image column has a history across a plurality of image lines that satisfies a predetermined condition to initiate in the analog memory.
• the read-out arrangement can be used to decide on a column-by-column basis which analog values are to be regarded as relevant.
• This recognition of relevant analog values can take place, for example, by analyzing over a plurality of image lines in the course of an image sensor analog signal belonging to a respective image column. If a characteristic characteristic curve for a light line or laser line is detected, analog values from a sequence of adjacent image lines of the same image column can be recognized as relevant analog values to be stored in the analog memory and treated accordingly.
• the readout arrangement may be configured to perform ana-pre-processing of the desk-sensor analog signals to obtain pre-processed signals.
• the readout array may then be arranged to select analogue signals based on the image sensor analog signals represented by the preprocessed signals for storage in the analogue memory. For example, an amount formation or a difference between several analysis values can take place. This allows particularly relevant information to be stored in the analog memory.
• the preprocessing can also serve or help to identify relevant analog values or to recognize a characteristic profile of the image sensor analog signals (for example over a plurality of adjacent image lines).
• the readout arrangement is configured to be responsive to an image sensor analog signal or signal based on an image sensor analog signal (eg, by comparison to an image sensor analog signal from a spatially adjacent pixel or by comparison to an image sensor Analog signal of the same pixel at an earlier time) passes a predetermined threshold in a first direction and / or in a second direction, to decide that an analog value represented by the respective image sensor analog signal or based on the respective image sensor analog signal in the analog memory is saved for further processing.
• an image sensor analog signal or signal based on an image sensor analog signal passes a predetermined threshold in a first direction and / or in a second direction, to decide that an analog value represented by the respective image sensor analog signal or based on the respective image sensor analog signal in the analog memory is saved for further processing.
• the readout arrangement may be designed to additionally store, when passing the predetermined threshold value (or in response to the passage of the predetermined threshold value), a digital information bearing information on reading out of which image line of the image sensor predetermined threshold was detected.
• the additional digital information can provide information about which image line (and / or which image column) or which image area in the analog memory are assigned analog values. This additional digital information can then be used in an evaluation of the analog values stored in the analog memory.
• the additional detailed information is helpful in particular if not all of the image content but only relevant sections of an image content (or analog values derived therefrom) are stored in the analog memory, since it is thus possible to infer which image lines or image columns or image areas in which Analog memory stored analog values.
• the readout arrangement may be designed to decide separately for each column or for different columns or for different groups of columns, whether an analog value represented by an image sensor analogue signal of a respective column line or an image sensor on the image sensor analog signal.
• Analog value of the respective column line based analog value to be stored in the analog memory for further processing (so that, for example, based on image sensor analog signals that are assigned to different image columns, analog values that are assigned to different line areas are stored in the analog memory).
• the relevant analog values eg the analog values of relevant picture lines
• the available analog memory can be used efficiently.
• analog values of the first row range are then stored in the analog memory for the first image column, and analog values of the second row range, which differ from the first row range, are then stored in the analog memory for the second image column.
• the readout device may be configured to obtain and evaluate configuration information that specifies separately for different columns, analogue values represented by which image lines through the image sensor analog signals, or analog values based on the image sensor analog signals in the analog memory for one further processing should be stored.
• configuration information that specifies separately for different columns, analogue values represented by which image lines through the image sensor analog signals, or analog values based on the image sensor analog signals in the analog memory for one further processing should be stored.
• the readout device may be configured to cause the analog values represented by the image sensor analog signals or based on the image sensor analog signals to be stored without checking whether the analog values are to be obtained for further processing.
• the readout arrangement may be designed to cause (for example, immediately or immediately) overwriting of the analog values stored without prior checking (for example, by the analog values obtained by the image sensor in a next step) (for example, by appropriately selecting a next write address, for example, equal to the current write address) when the read array determines that the analog value stored without prior check should not be stored for further processing (eg, longer term).
• storing an analog value for further processing means storing the analog value for a longer period of time so that the analog value is available for a later read-out process
• Selection of analog values for storage in the analog memory for further processing can also take place in that first all analog values are stored and such analog values that are not selected for storage for further processing, short-term (for example immediately upon the presence of the next analog value from the next pixel).
• the timings can be accelerated or parallelized, and it can be considered that the storage of the analog value takes a relatively long time.
• the read-out arrangement may be designed to successively store successive analog values represented by the image sensor analog signals or based on the image sensor analog signals in a region of the analog memory which is driven or designed as a ring buffer (for example assigned to a respective column of the image sensor) according to save. This can be done, for example, such that a cyclic overwriting of analog values takes place in the region addressed or designed as a ring buffer. Thus, it can be achieved, for example, that a certain number of analog values preceding a current analog value are always stored, which in turn allows the analog values previous to a triggering event to remain stored.
• analog values may be obtained in the analog memory (for read-out) based on one or more are based on the i-th image line (eg, immediately) preceding image lines, and analog values (for later processing) can also be stored in the analog memory which are based on one or more of the i-th picture line following picture lines (so that, for example, analogue signals from an area around the ith picture line are simultaneously stored in the analog memory).
• the readout array may be configured to be maintained in response to detection that analog values stored in the ring buffer are to be stored for further processing (and, for example, beyond an instant at which overwriting would occur), prevent overwriting (for example, by aborting writing of analog values based on the column column concerned or by using a new memory area as the new ring buffer).
• recognition is made that analog values stored in the ring buffer are stored for further processing (eg, beyond an instant at which overwriting would occur) based on detection of a local or absolute maximum of a brightness value on the Base of an image sensor analog signal.
• analog values may be maintained in the analog memory for further processing that are (for example, in the row direction) around a location of maximum brightness.
• a position of an image of a line on the image sensor can be determined with high accuracy, for example those analog values which are not in the vicinity of a brightness maximum, can be discarded to reduce a data amount (for example, by an overwriting in the analog memory).
• the output device may be configured to store a fixed number of analog values for further processing in response to detection that analog values are to be stored for further processing.
• the read-out arrangement may be designed to store a variable number of analog values for further processing in response to detection that analog values are to be stored for further processing, depending on the analog values.
• the storage of a fixed number of analog values for further processing can be described, for example, as Saving a variable number of analog values for further processing can, for example, be understood as "storing with a dynamic interval".
• the readout arrangement may be configured to selectively store analog values in the analog memory that satisfy a predetermined condition.
• analog values may be stored in the analog memory that describe a brightness value that is greater than a threshold.
• a hysteresis can be used.
• z can be used.
• the readout array may be configured to store information (eg, in digital form) describing what subset of the plurality of analog values represented by the image sensor analog signals or based on the image sensor analog signals in the analog memory was stored for further processing.
• the information can describe, for example, to which image lines the analog values stored for further processing are assigned.
• the position of the pixels on which the stored analog values are based can be taken into account in the evaluation.
• the readout device may be configured to (for example, by driving the image sensor to set image lines or image columns to be read out, and / or by driving a multiplexer to specify in which memory cell of the analog memory an analog value is to be stored) Assignment between an image column and columns of the analog memory, in which to the image column belonging analog values are stored, to vary.
• analog values belonging to an image column may be in different Memory columns of the analog memory are stored.
• analog values that belong to a line running diagonally across the image sensor can be stored in a substantially rectangular area of the analog memory (at least logically, with regard to memory lines and memory columns).
• the read-out arrangement may be designed to re-allocate analog values associated with pixels when storing in the analog memory.
• analog values of pixels that lie at midpoints along a curved or angled line can be stored in a linear region of the analog memory, that is, for example, in a continuously addressed region or in a substantially rectangular region of the analog memory.
• a storage space requirement in the analog memory can be minimized, and further processing of the rearranged or shifted analog values can be carried out in a particularly efficient manner.
• the readout arrangement is designed to store the analog values in the analog memory so that signals read from the analog memory and the image sensor analog signals are compatible in terms of signal level. In other words, for example, signal compatibility between outputs of the memories and the pixels can be achieved.
• the output device is designed to perform analog arithmetic operations based on signals read from the analog memory.
• analog arithmetic operations can only be performed on the memories.
• the issuing arrangement is designed to carry out analog computing operations in which image sensor analog signals and signals read from the analog memory are combined.
• analog computing operations with memory and sensor matrix can be performed.
• An embodiment according to the invention provides an image sensor system.
• the image sensor system includes an image sensor, a readout array as described herein, an analog-to-digital converter, and a digital processing device.
• the analog-to-digital converter is designed, for example, to digitize analog values stored in the analog memory or analogue values derived therefrom.
• the digital processing device is designed, for example, to analyze image information based on digital signals supplied by the analog-to-digital converter.
• an efficient and fast analog pre-processing can be used to select which analog values are to be stored in the analog memory for subsequent analog-to-digital conversion and for subsequent digital further processing.
• the amount of data is thereby reduced since, for example, the analog-to-digital converter no longer has to digitize the analog values of all the pixels, but only the analog values of the pixels recognized as relevant and stored in the analog memory.
• the amount of digital data that needs to be transported from the analog-to-digital converter to the processing device also decreases, which digital data is a bottleneck in some conventional image sensor systems.
• the image sensor system described here thus enables a particularly advantageous division of the processing tasks, whereby recognition of relevant analog values takes place very early in the processing chain and, for example, even before the analog-to-digital converter.
• the digital processing device can thus be realized with comparatively low computing power. This results in significant cost advantages and the implementation is simplified.
• the analog-to-digital converter is configured to provide the analog values (1 150b-d, 1 152b-d; 1260b-d, 1262b-d) stored in the analog memory (130; 220; 920; , 1264b-d), or analog values derived therefrom, in a separate readout process, which is downstream of a readout of the image sensor to digitize.
• a downstream digitization from analog memories can take place in a separate read-out process.
• the digital processing device is configured to determine, based on digital signals supplied by the analog-to-digital converter, a position of a line in an image captured by the image sensor (for example with subpixel accuracy or subpixel accuracy). capture.
• the digital processing means may evaluate the preselected analog values stored in the analog memory (after a corresponding analog-to-digital conversion) and, for example, perform one or more calculations based on the values obtained by the analog-to-digital conversion.
• the digital processing device only has to process digitized values that have been previously classified as relevant by the readout arrangement.
• a position of a line can be determined with a high accuracy, while the processing cost is kept comparatively low.
• the digital processing means is arranged to evaluate information (eg, in digital form) describing which subset of the plurality of analog values represented by the image sensor analog signals or based on the image sensor analog signals in the analog memory for one further processing was saved.
• the digital information can describe to which image lines or to which image areas the in the analog memory stored analog values include.
• the information describing which subset of the analog values has been stored in the analog memory for further processing may be utilized by the digital processing device to determine a location of the features detected from the stored analog values.
• the image sensor system may be configured to determine a position of a light line (formed, for example, by a laser light slit of a three-dimensional object) along a respective column of the image sensor for different columns of the desk sensor.
• a position of a line which runs obliquely over an image sensor can be determined with high accuracy, wherein in each of the columns, for example, only analog values of such pixels (or image lines) are stored in the analog memory and subsequently evaluated by the digital processing device, which also belong to the line or lie in an environment of the line.
• different lines areas can be stored and evaluated in different columns.
• the image sensor system may be configured to selectively store in the analog memory values that are indicative of a significant change (eg, sign change) of an image sensor analog signal or signal based on an image sensor analog signal (eg, one on one Compared to an image sensor analog signal from a spatially adjacent pixel, or on a comparison to a Biidsensor- analog signal of the same pixel to an earlier time based analogue signal) are present.
• a significant change eg, sign change
• an image sensor analog signal eg, one on one Compared to an image sensor analog signal from a spatially adjacent pixel, or on a comparison to a Biidsensor- analog signal of the same pixel to an earlier time based analogue signal
• an image sensor analog signal eg, one on one Compared to an image sensor analog signal from a spatially adjacent pixel, or on a comparison to a Biidsensor- analog signal of the same pixel to an earlier time based analogue signal
• the biosensor system may be configured to perform an evaluation of a white light interferometry based on the selectively stored analog values. It has been shown that with white-light interferometry straight changes of an image sensor analog signal or of a signal derived therefrom (ie, for example, changes which are greater than a given given threshold) have special significance. In this respect, the described image sensor system allows a particularly efficient implementation or evaluation of a white light interferometry.
• Another embodiment according to the invention describes a method for reading an image sensor. The method includes receiving in parallel a plurality of image sensor analog signals that describe brightness values acquired by the image sensor in analog form (for example, from a plurality of column lines of the image sensor).
• the method further includes selecting which subset of a plurality of analog values represented by the image sensor analog signals or based on the image sensor analog signals are stored in an analog memory for further processing.
• the method further includes storing the selected analog values in the analog memory.
• the corresponding method is based on the same considerations as for the device described above.
• the method may be supplemented with all features and functionalities of the readout array and the image sensor system as described herein. These features may be used singly or in combination in the method.
• the readout device is configured to receive in parallel a plurality of image sensor analog signals describing brightness values detected by the image sensor in analog form from a plurality of column lines of the image sensor.
• the read-out arrangement is designed, for example, to determine (for example by triggering the image sensor to set lines to be read, and / or by controlling a multiplexer which determines in which memory cell of the analog memory an analog value is stored) an association between an image column and Columns of an analog memory in which to the image column associated analog values are stored, to vary, so that belonging to a picture string analog values are stored in different memory columns of the analog memory.
• the corresponding concept makes it possible, for example, for analog values, which describe a line running diagonally across the image sensor, to be stored in a rectangular memory area of the analog memory and thus to be evaluated and processed in an efficient manner.
• analog values which describe a line running diagonally across the image sensor
• the storing of associated features in a rectangular memory area of the analog memory is typically also very memory-efficient, since, for example, in a comparatively small memory, analog values which belong to different areas of interest, can be stored. For example, even if lines are in different directions in the various regions of interest, easy-to-handle rectangular memory areas of the analog memory can be used, saving memory space and facilitating read-out.
• the read-out arrangement is designed to re-allocate analog values associated with pixels when being stored in the analog memory.
• analog values of pixels whose centers lie along a curved or angled line can be stored in a linear or rectangular region of the analog memory.
• the pixels located along a curved or angled line, or the associated analog values can be stored in a continuously addressed area of the analog memory. This facilitates reading and further processing and is otherwise memory-efficient.
• Fig. 1 is a block diagram of a readout arrangement according to an embodiment of the present invention
• FIG. 2 shows a schematic representation of an architecture of a "vision SoC” (visual system-on-chip) with an analog memory matrix;
• Fig. 3 is a schematic representation of an analog data path, according to a
• 4a is a schematic representation of a memory multiplexer, according to a
• Fig. 4b is a schematic representation of a memory multiplexer, according to an embodiment of the invention.
• Fig. 5 is a schematic representation of a picture cell (Pixelze! Le) with a global
• 6 is a schematic representation of a memory cell according to an embodiment of the present invention
• 7a is a schematic representation of a laser light section arrangement
• 7b is a schematic representation of a course of a gray value along a
• Sensor column; 8a-8c are schematic representations of different variants for determining a coordinate Xo of a gray value along a sensor column;
• FIG. 9 is a schematic representation of a system according to an embodiment of the present invention.
• FIG. 10 shows a schematic illustration of an evaluation in the presence of a horizontally extending light line; a schematic representation of an evaluation in the presence of a substantially horizontally extending light line, which runs slightly inclined to the bottom right;
• FIG. 12 is a schematic representation of an evaluation in the presence of a substantially vertically extending light line, which runs slightly inclined to the right and bottom;
• FIG. 13 is a flowchart of a method according to an embodiment of the present invention.
• 1 read-out arrangement according to FIG. 1 1 shows a schematic representation of a readout arrangement 100 according to an embodiment of the present invention.
• the readout arrangement 100 is designed, for example, to receive in parallel a plurality of image sensor analog signals 10a to 10b, which describe brightness values recorded by an image sensor 120 in analog form, from a plurality of column lines of the image sensor 120.
• the readout array 100 is further configured to select which subset of a plurality of analogue values represented by the image sensor analog signals 0a to 11 Od or based on the image sensor analog signals are stored in an analog memory 130 for further processing.
• the readout arrangement 100 is designed to cause the storage of the selected analog values in the analog memory 130 or to store the selected analog values in the analog memory 130.
• the readout arrangement 100 thus represents, for example, an interface between the image sensor 120 and the analog memory 130.
• the readout arrangement controls the analog memory 130 so that not all of the analog values output by the image sensor 120 are stored in the analog memory 130 for further processing. Rather, the readout arrangement 100 makes a selection which of the analog values are to be stored in the analog memory 130, and which of the analog values are either not even stored in the analog memory 30 or directly overwritten (and thus not in the analog memory for further processing get saved).
• the readout array 100 may use a variety of criteria to decide which of the analogue values are stored in the analogue memory 130 for further processing.
• the analog values may, for example, be based directly on the biosensor analog signals 110a to 110b, or the analog values stored in the analog memory 130 may be generated by an analog preprocessing of the image sensor analog signals.
• a plurality of image sensor analog signals (for example, by summation or by subtraction) can be combined or values of the image sensor analog signals can be combined at different times, for example in the sense of difference or in the sense of determining a magnitude of change over time.
• the readout arrangement 100 may, for example, perform a substantially analogous processing of the sensor analog signals in order finally to decide, for example based on a binary threshold decision, which analog values are to be stored in the analog memory 130. It should be noted that the readout arrangement 100 according to FIG. 1 can be supplemented by all the features described herein. For example, the Features all functionalities, as described below with reference to FIGS. 2 to 12, individually or in combination in the readout arrangement 100 are added. 2. Architecture according to FIGS. 2 to 6
• FIG. 2 shows a schematic representation of an architecture of a "vision SoC" (ie a vision system on chip) with an analog memory matrix Details with regard to a possible analog data path are shown, for example, in FIG With regard to a possible memory Multiplexer are shown in Fig. 4a, Fig. 5 shows details with regard to a possible pixel cell and Fig. 6 shows details with regard to a possible memory cell.
• vision SoC ie a vision system on chip
• the architecture described here is based on the consideration that it is necessary or desirable for certain applications that greyscale data read out from the image cells or pixel cells is buffered before the analog-to-digital conversion.
• a well-known example is image sensors with image acquisition rates that are higher than can be continuously digitized by the integrated or externally connected analog-to-digital converters.
• the pixel values or pixel values must be written very quickly into an immediately connected analog memory, from which they can be slowly read out, processed, digitized and output after completion of the image acquisition. It has been recognized that with this conventional approach, refresh rates up to the megahertz range can be achieved.
• FIG. 2 a possible arrangement is shown schematically by way of example.
• the arrangement according to FIG. 2 is designated in its entirety by 200.
• the arrangement 200 comprises a sensor matrix 210, which may comprise a plurality of pixels, for example.
• An "individual pixel" is exemplified at 212.
• the sensor array 210 which may also be referred to as an image sensor, includes a plurality of image rows 214a-2141, each of the image rows 214a-2141 having a plurality of image columns 216a-216n
• the sensor matrix provides an image sensor analog signal 218a-218d for each sensor column, for example via an associated column line.
• the arrangement 200 further comprises an analog storage matrix 220 and a so-called “SIMD unit” 230.
• the SIMD unit 230 may, for example, be a “single-instruction-multiple-data” unit, that is to say one unit, with a single instruction (“Single Instruction") processes multiple data (“Multiple Data”).
• the arrangement 200 also comprises a frame controller 250, which is coupled to the sensor matrix 210, for example, and which may be designed, for example, to enable readout of rows of the sensor matrix 210.
• the row controller 250 may also be coupled to the analog storage matrix 220 to enable readout of rows of the analog storage matrix 220.
• the arrangement 200 includes a "SIMD controller" 260 configured to receive control instructions over a bus, for example, and to control the "SIMD unit” 230 accordingly.
• the SIMD unit 230 comprises, for example, a multiplexer 232 which, for example, is coupled on the one hand to column lines of the sensor matrix 210 in order to receive analog column line signals 218a-218d from the sensor matrix 210.
• the multiplexer 232 is also coupled to the analog storage matrix 220, for example bidirectionally (but alternatively also unidirectionally).
• the multiplexer 232 may be configured to connect a group of column lines of the sensor array 210 to a group of column lines of the analog storage matrix 220, wherein the association between the column lines of the sensor array 210 and the column lines of the analog storage matrix 220 is variably adjustable, for example ,
• the multiplexer 232 may connect (temporarily) a given set of column lines of the sensor array 210 to a first group of column lines of the analog storage matrix 220 and temporarily to a second group of column lines of the analog storage matrix 220 of column lines is different from the first group of column lines.
• each column line of the sensor matrix 210 (or at least a subset of the column lines of the sensor matrix 210) may each be assigned a processor element PE.
• a first column line (which here for example belongs to a picture column 216a) is assigned a first processor element 234a.
• a second column line which belongs, for example, to the second image column 216b (which, for example, is coupled to the picture elements of the second image column 216b), is assigned a second processor element 234b, for example.
• An nth column line which belongs, for example, to the nth image column 216n, may for example be associated with an nth processor element 234n.
• the processor elements 234a, 234b, 234c may be substantially similar. Therefore, in the following, for example, only the processor element 234a described.
• the processor element 234a includes, for example, an analog processing 236, an (optional) analog-to-digital conversion 238, and an (optional) digital processing 240.
• the analog processing may include, for example, a read circuit ("READ"), a differentiation circuit.
• the analog processing 236 may also comprise a substantially analog detection, which can determine, for example, by an analog combination of different signal values and by a subsequent threshold decision, whether a particularly characteristic one .DIFF "and a sign determination circuit or" SIGN "
• the analog signal processing 236 may be configured to detect when a trace of an image sensor output signal associated with a respective image column extends across a plurality of image lines away has a course that meets a predetermined condition.
• the analog processing may also be designed to operate e.g. For example, when a pixel in a column satisfies a certain condition, or when a difference between two spatially adjacent pixels satisfies a certain condition, or when a difference between analog values of a pixel at different times satisfies a predetermined condition.
• a corresponding evaluation can be carried out here column-individually, so that for each image column 216a to 216n is evaluated separately, at which time the corresponding predetermined condition is met.
• the (optional) digital processing 240 may support or configure the analog processing 236.
• the digital processing 240 may comprise, for example, an arithmetic and logic unit "ALU” which may access flags ("FLAGS") or which may also change the flags.
• the arithmetic and logic unit ALU can, for example, read and / or write to register REG.
• the arithmetic logic unit ALU may also access a random access memory "RAM” and may be coupled to a bus 242 via a bus interface "BUS".
• the digital processing 240 for example, the function of a microprocessor, or at least a sub-function of a microprocessor meet.
• the digital processing 240 may be coupled to the analog processing 236 via the analog-to-digital converter 238 to obtain, for example, information preprocessed by the analog processing 236.
• the analog-to-digital conversion 238 can operate, for example, with comparatively low accuracy, which is, for example, lower than an accuracy used when reading out the rows of the sensor matrix 210.
• the SIMD controller 260 may be configured to configure the analog processing 236 and / or the analog-to-digital conversion 238 and / or the digital processing 240.
• the SIMD controller 260 may set arbitration thresholds for the analog processing 236 or load a program into the digital processing 240.
• the SIMD controller 260 can also specify externally which region of the sensor matrix 210 is to be taken over into the analog memory 220. This can be set, for example, column-individually.
• the SIMD unit determines, for example, in cooperation with the SIMD controller 260, based on which pixels or pixels of the sensor matrix analog values are stored in the analog memory matrix 220.
• a column-individual evaluation of column signals from the sensor matrix can at least influence the decision.
• Specifications can alternatively or additionally come from the SIMD controller.
• the analog values stored in the analog memory matrix 220 may be identical to the analog values provided on the column lines of the sensor array 210, or may be pre-processed by the analog processing 230, for example.
• the analog processing 236 can thereby carry out a subtraction and / or a scaling and / or an amount formation and / or another analogue preprocessing.
• the multiplexer 230 can also control where (in which column) of the analog memory matrix an analog value from a given image column of the sensor matrix 210 is stored.
• each pixel column (or pixel column) of the sensor array includes a processor element column (eg, one of the processor element columns 234a, 234b, 234n).
• a processor element column eg, one of the processor element columns 234a, 234b, 234n.
• several pixels or pixels can also be used a process element column 234a, 234b, 234n or a memory column 222a, 222b, 222n be assigned.
• a minimum functionality of the processor element PE is to select the memory cells in the analog memory array 220.
• the information as to whether and to which memory cell is written can be obtained, for example, either from a memory of the processor element or obtained by evaluating the pixel data (or pixel data) of the sensor matrix 210.
• FIG. 3 shows an example of an analog data path in the form of a schematic representation.
• FIG. 3 shows by way of example a principal function of the analog part of a processor element (green background block in FIG. 2) for processing pixel data or pixel data.
• the analog data path 300 according to FIG. 3 can, for example, take on the function of the multiplexer 232 and of the analog processing 236 (and optionally also of the analog-to-digital converter 238).
• the analog data path comprises, for example, a pixel column or pixel column.
• a pixel column typically includes a plurality of pixels, such as one pixel per image column.
• the pixel column further includes a pixel row selection, such as allowing a column line 310 to output an analog value of a selected pixel of the pixel column, the analog value representing, for example, a brightness value (or average brightness value). as it was in the selected pixel in a certain period of time.
• Analog signals or signal values from the pixel column (which are represented, for example, by voltages) are designated here, for example, by Po and Pi.
• the analog data path further comprises a column read circuit 320 adapted to enable readout of an analog value from a pixel (or a pixel circuit) and to further enable buffering of an analog value read from a selected bi-point.
• the column read circuit 320 may have two capacitances Mo, 322 and M i, 324, wherein a first capacitance Mo, 322 via a first switch SPO with the column line 310 can be coupled, and wherein the second capacitance Mi, 324 Koppeibar via a second switch SPI with the column line 310 kop.
• capacitances M 0 , 322 and Mi, 324 can be coupled to the column line 310 at different times and thus be charged, for example, based on analog values of different pixels. Alternatively, the capacitances may also be charged based on analog values of the same pixel at different times.
• the analog data path 300 further comprises a differential circuit or difference-forming stage 330.
• the difference-forming stage can be designed, for example, to determine a value difference in an analogous manner. For example, a voltage applied across the first capacitor Mo, 322 can be buffered by a first buffer amplifier ADO whose input is coupled to the first capacitor Mo, 322.
• a voltage applied to the second capacitor Mi, 324 can be buffered by a second buffer amplifier ADI whose input is coupled to the second capacitor Mi, 324.
• An output of the first buffer amplifier ADO is, for example, coupled via a first switch SDO to a first terminal of a further capacitor MD, 332.
• An output of the second buffer amplifier ADI is, for example, coupled via a switch SDI to a second terminal of the further capacitor MD, 332. If, for example, the switches SDO, SDI are simultaneously closed, then the further capacitor MD, 322 is charged, for example, to a voltage which is equal to a difference between the voltage applied to the first capacitor Mo, 322 and to the second capacitor Mi, 322.
• a sign of the voltage across the capacitor MD, 332 depends on whether the voltage across the capacitor Mo, 322 is greater or less than the voltage across the capacitor Mi, 324.
• the difference-forming stage 330 can provide a voltage total that is the difference represents two analog values provided by column line 310 from the image sensor.
• the voltage applied to the capacitor Mo may represent a difference in the analog values provided by the pixels of the same image column in different picture cells.
• MD, M and Mo here denote the voltages at the respective capacitors 332, 324, 322
• the voltage across the capacitance MD, Mo denotes the voltage across the capacitance Mo, 322
• Mi denotes the voltage across the capacitance Mi, 324.
• a preprocessed signal is denoted here, for example, by V sgn .
• the voltages applied to the capacitors Mo and Mi can also be supplied to an analog-to-digital conversion (for example, by the analog-to-digital converter 238), so that the capacitances Mo, 322 and Mi, 324 applied voltages can also be processed by the digital processing 240 of the processor element 234a.
• the analog data path 300 further includes a sign value determination stage 340 configured to determine the sign and / or the value of the voltage applied to the capacitor 332 (or, more generally, the voltage provided by the differencing stage 330).
• the sign and value determination stage 340 provides a signal describing the magnitude of the voltage provided by the stage 330 and another signal describing a sign of the voltage provided by the stage 330.
• the sign of the voltage provided by stage 330 (eg, the voltage across capacitor 332) may be determined by a simple threshold comparison.
• the voltage applied across the capacitance 332 as a differential signal may be converted to a ground referenced signal by, for example, connecting the more negative terminal of capacitance 332 to the ground and connecting the more positive terminal of capacitance 332 to the output.
• the switch SG U ⁇ O and the switch Ssi can be closed. This is at an output of the stage 340 based on the bossspotenziai (or the mass) positive voltage.
• the second terminal (upper terminal) can be connected to the reference potential (ground) via the switch Sax » and the The first (lower terminal) of the capacitor 332 can be connected to the output of the stage 340 via the switch Sso.
• a corresponding sign signal which is also provided by stage 340, may indicate the capacitance of the capacitor voltage, for example, and may be stored separately (for example, in a digital memory).
• the analog data path 300 further includes a column write circuit 350 arranged to decide in which column of the analog memory array the output of the stage 340 should be written.
• the column write circuit 350 may include a multiplexer 352 and be configured to find a suitable route through the multiplexer 352.
• a further buffer amplifier As may be connected between the output of the stage 340 and an input of the multiplexer 352.
• the buffer amplifier can be coupled to one of the terminals of the capacitor 332 via the switches Sso and Ssi, for example, so that the input of the buffer amplifier As is connected to the more positive one of the terminals of the capacitor 332, for example.
• the multiplexer 352 can then select, for example by means of a corresponding control signal, to which of the memory columns the signal applied to the output of the buffer amplifier As is to be applied or stored. Outputs of multiplexer 352 are thus connected to different ones of the memory columns, one of the memory columns being designated 360.
• the storage column has a row selection, so that in each of the storage columns, a row or row accessed by a write access or a read access, respectively, can be selected.
• the memory strings also have control terminals for column reading and column writing, respectively. In this regard, it should be noted that, for example, at a given time in different memory columns different memory lines can be selected for reading or writing.
• the processor elements of the individual columns can individually determine in which line of the analog memory matrix a current value should be written.
• a value read out of the memory column can optionally also be fed back, for example, to a column line and thus, for example, stored in one of the capacitors 322, 324.
• Such a feedback can be useful, for example, if differences between successively read values are to be formed.
• control of the stages 320, 330, 340, 350, 360 can be performed, for example, by the digital processing 240 and / or by the SIMD controller 260.
• one or more threshold value judges can also be coupled to nodes of the analog data path 300, which can evaluate whether certain conditions are being met by the analog signals. be filled. Based on the presence of a specific condition, it can be decided, for example, whether an analog value should be stored in a memory column or in which row of a memory column the analog value should be stored or in which column of the analog memory matrix an analog value should be stored.
• the result obtained in this analysis or a value taken from the memory of the processor element PE can be used to determine whether and with what sign the analog value VD after activation of SDO and SDI by driving on Mo and Mi by means of ADO and ADI Memory value Mo is to be stored.
• the driver As ensures that for negative values V Sgn the output value V s is set to zero. If both values are stored in two memory cells or determined by means of a non-illustrated additional circuit a sum and stored this, it is z. For example, it is possible to analogously determine and store the absolute value of a difference. It should be understood, however, that the processing described in this section is optional, and that only a few of the processing steps may be present.
• the assignment of the target column of the memory matrix takes place, for example, in a multiplexer (Mux), which can be used for example as a multiplexer.
• Mux multiplexer
• B. allows to swap columns.
• the assignment can be made by the multiplexer 232 or by the multiplexer 352.
• the multiplexer is optional and can be used, for example, to vary an association between an image column and columns of analog memory in which analog values associated with the image column are stored or from which they are read in the sensor array.
• the multiplexer can optionally also be changed.
• One possible implementation for the multiplexer is shown by way of example in FIG. 4a.
• FIG. 4 a shows a schematic representation of a multiplexer 400, which can take over the task of the multiplexer 232 or the task of the multiplexer 352, for example.
• the multiplexer has, for example, a plurality of multiplexer input lines 410a to 410h.
• the multiplexer further includes a plurality of output lines 414a-414h.
• the various input lines may for example be assigned to different processor elements 234a to 234n.
• the output lines 414a to 414h may, for example, be assigned to different memory columns (or column lines) of the analog memory matrix 220.
• the multiplexer further comprises a plurality of connection lines 420a, 420b, 420c, 420d, which are each connectable to a plurality of columns, for example, and which are offset from each other so that different ones of the connection structures 420a to 420d are connected to different sets of input lines and
• the first interconnect structure 420a is connectable to the input lines 410b to 41 Oe and the output lines 414b to 414e.
• the second connection structure 420b is connectable to the input lines 410c to 41Of and to the output lines 414C to 414F.
• the third connection line 420c is connectable to the input lines 41Od to 410g and to the output lines 414d to 414g.
• the fourth connection line 420d is connectable to the input lines 41 Oe to 410h and to the output lines 414e to 414h.
• one of the input lines 410b to 41Oe can be connected to one of the output lines 414b to 414e, for example by connecting the connection line 420 to one of the said input lines and to one of the said output lines.
• the connection line 420a is thus usable, for example, to connect the connection line 410b to the output line 414e.
• the connection line 420a may also be used to connect the input line 41 Oe to the output line 414b.
• connection lines are thus usable, for example, to connect an input line to an output line having a smaller index than the input line or to connect the input line to one of the output lines having a larger index than the input line (in the picture spoken) on the right is the input line).
• the trunk can also be used to connect an input line to an output line having the same index.
• a connection line eg, connection line 420a
• a connection line (eg the connection line 420a) can also be connected to the associated output lines (for example the output lines 414b to 414e) via switches, which are arranged, for example, at the crossing points between the connection lines and the output lines.
• a length of the connecting lines 420a to 420d determines how many column positions an input line can be shifted relative to an output line coupled thereto by a connecting line.
• Input signals from the columns are supplied from below, e.g. B. via blue marked input lines 410a to 410h.
• the output signals are also output, for example upwards, also column by column (eg via the output lines 414a to 414h).
• Lines are routed horizontally (eg, the connection lines 410a-410b) by means of which the input lines and output lines can be connected. This is done for example by switches, which are shown in Fig. 4a as squares on the crossing points.
• the switches can z. B. as shown are set line by line.
• the number of switch control signals required in each link line for linking the horizontal links (eg, lines 420a-420d) to the input lines (Sin, 410a-410h) and output lines S ou t (eg, 414a to 414h) results from the length of the contiguous pieces (eg outlined in gray) or the horizontal spacing between the breaks (or the length of the join). supply lines 420a to 420d). In the present case, the length of such a segment and thus the number of pairs per segment is four in each row.
• Read-back may be necessary or helpful if a memory driver is used in each memory cell whose dispersion ("mismatch") is to be corrected by means of control during storage (so-called closed-loop storage or "closed-loop storage”).
• the multiplexer allows analog values from a given column of the image sensor to be transferred into different columns of the analog memory matrix or else analog values from different columns of the image sensor to be transmitted to a predetermined column of the analog memory matrix.
• filtering for example spatial filtering
• filtering to be achieved by appropriate control of the multiplexer and by appropriate scaling of analog values, for example by weighting analog values of different pixels in a common memory cell of the analog memory matrix be combined. Due to the weighted combination of analog values, filtering according to a "filter matrix" can thus be achieved.
• the column outputs 414a to 414h of the multiplexer 400 are connected to the inputs of the memory array.
• Conceivable for example, programmable shift registers or address decoder, by means of which a line is selected for storage.
• FIG. 6 a memory cell 600 having an address decoder 620 is depicted.
• a write access takes place by setting an address (address) on an address bus 622 and activating a connection between SOut or Sin and the switch transistors (SeM or Sel2) by means of an activation signal 624 (Act).
• the setting of a voltage on the storage capacitor Cint takes place, for example, via the analog input Min and the reading back of the voltage resulting from the source follower SF via the analog output Mout. Also conceivable is a variant in which no internal power source transistor Src is used, but instead the same external power source for all memory source followers. The actual reading of the internal state is achieved by a line-by-line control equal to that for reading the pixel cells or pixel cells.
• the processor element PE belonging to a column of the image sensor can, for example, supply to a selection logic 620 associated with a column of the analog memory matrix an address information 622 which indicates which row of the respective column of the analog memory matrix is to be accessed.
• the corresponding processor element may also provide an enable signal 624 to signal a memory access.
• further control signals 626, 628 may be provided by the processor element, specifying, for example, whether write access to the memory cell selected by the address information 622 is to be made or whether read access to the memory cell selected by the address information 622 is to occur.
• a read-write access may also be performed in which both a write signal (eg, Min) is supplied to the memory cell and a read signal (eg, Mout) is read back from the memory line, for example, a precise one Feed-in using a feedback enable.
• the write signal Min may, for example, be supplied from one of the outputs 414a to 414h of a multiplexer. If no multiplexer is present, but also, for example, the signal Vs can be applied as a write signal Min.
• the select logic 620 then causes the correct row to be activated in the respective column to which the select logic 620 belongs.
• the transistor 642 when writing, the transistor 642 is activated in the row selected by the address information 622 so that, for example, the capacitance Cint in the selected memory cell is connected to the column line of the analog memory matrix. If a specific memory cell is to be read out, this is in turn selected by suitable address information 622, and the associated transistor 648 is put in a conductive state in response to a corresponding control signal, so that a drain terminal (source terminal) of the source follower Transistor 646 is connected to the corresponding spa line (or read column line) of the analog memory matrix.
• each memory column of the analog memory matrix has an associated selection logic which selects a row of the analog memory matrix individually for each column.
• the control of this selection logic is preferably carried out by an associated processor element, wherein the various processor elements, which are assigned to the various columns of the image sensor, can select different lines of the analog memory matrix at a time.
• a first row of a first column of the analog storage matrix may be described, and a second row of a second column, with the second row different from the first row.
• the analog memory matrix can also be read out in a conventional manner, so that, for example, in a read-out step, the same row is read out in all columns.
• This can be done, for example, by a read-out circuit 660, wherein, for example, read-out transistors 662 of an entire memory line of the analog memory matrix can be activated simultaneously (for example, by a common read-out signal).
• all columns or at least one column area comprising a plurality of columns
• can be read out at the same time which enables an efficient transmission of the read-out data to a digital further processing (after a corresponding analog-to-digital conversion).
• lines to be described or rows to be read out can be selected column-individually by means of corresponding selection logics, the selection taking place, for example, by the processor elements operating in parallel.
• the analog memory matrix can also be read out in another way, but preferably also line by line, for example to provide data for further digital processing.
• the line-by-line readout can be controlled by the line controller 250, for example.
• details regarding a time of access are described, these details being considered optional.
• writing to memory For example, this can be done simultaneously or shortly after reading the pixels or pixel cell, but even before the evaluation (for example without prior checking whether the analog values are to be obtained for further processing) in the processor element PE.
• the stored value is adopted by selecting the next memory cell in the corresponding column (which avoids, for example, a timely overwriting). If the stored value is to be discarded, the selected address (represented, for example, by address information 622) remains the same and the memory value is overwritten by the next readout value.
• the memory contents are output as voltages as in the pixel cells or pixel cells.
• PE processor elements
• Post processing in the processor elements (PE) can, depending on their parameterization and their internal state, be performed for all at the same time or only with the participation of specific columns. The latter, for example, allows a very fine setting of a region of interest ("Rol”) and further compression by properly removing data, thus disabling the inactive column outputs.
• the processor elements of the individual columns can decide whether data of the respective column or image sensor column should be stored in the analog memory matrix or not.
• the individual storage information it is possible, for example, to store only information from image sensor columns which is available for later evaluation (for example, by digital processing according to an analog-to-digital process). Conversion) were identified as relevant. For example, this identification of soft data may be column-individual.
• a data output Out of the pixel cell or pixel cell 500 shown by way of example in FIG. 5 may be connected, for example, to an associated column line of the image sensor matrix.
• the image sensor, or rather the image sensor matrix can thus have a matrix of pixel cells 500 according to FIG. 5.
• the data output Out of the pixel cell 500 may be connected to the line 310, for example. Selection Terminals If a plurality of pixel cells of a row of the image sensor or of the image sensor matrix can be interconnected, for example, the output of data by the image sensor matrix simultaneously takes place simultaneously for all columns of a picture line or at least for a plurality of columns of a picture line.
• FIG. 4b A further exemplary embodiment of a multiplexer 450, which can assume the function of the multiplexer 232 or the function of the multiplexer 352, for example, or which can, for example, take the place of the multiplexer 400, is shown in FIG. 4b.
• the multiplexer 450 has connection lines 470a to 470d and 472a to 472d, which extend over 4 column line positions, for example, and offset from each other in the manner shown.
• connection lines 470a to 470d and 472a to 472d are connectable, for example, via switches to input lines (eg, input lines 460a to 460d). Further, the connection lines 470a to 470d and 472a to 472d are connectable, for example via switches, to output lines (eg, output lines 464a to 464d).
• the multiplexer can be used, for example, to produce a variable connection of input lines to output lines, with an adjustable offset between input lines and output lines being achievable. For example, by a corresponding activation of the switches, it can be achieved that a group of input lines is connected in one direction offset with a group of output lines.
• the range is (only) 3 in both directions. From below, not four but only two selection lines (to the right or to the left) are necessary. The other two selection lines are, for example, redundant (and can optionally be saved). Basically, it should be said that two or more switch positions from below make sense.
• the read path (eg, the connection 1 10a-1 1 Od between the image sensor 120 and the readout array 100 or the connection 218a-218d between the sensor array 219 and the SIMD unit 230 or the connection between a pixel column and a slit array).
• a standard DC offset (0" offset, for example, corresponds to a full modulation of the signal) is subtracted, for example, (eg in the read path) a corresponding circuit is included, the so long of both Disconnects the same current from the differential lines until the line with the lower potential reaches the lower limit, then, for example, both current draws are terminated (or kept at a constant level).
• the following section first provides background information with regard to the implementation of a laser light section.
• laser light section (English: “Sheet of a Light", “SoL"), as shown schematically for example in FIG. 7, a laser line 710 (which is generated by a laser 708) on projecting a three-dimensional surface 720 to be measured, which is equipped with a camera 730 and a triangulation. Lung angle «between the laser plane and camera level is considered.
• a point P on the surface is at a height h, which in turn leads along a column in a camera image to a deflection x from a fixed zero point.
• the image processing task to be solved consists, for example, in an exact, possibly sub-pixel-accurate (sub-pixel-accurate) determination of the position of the maximum of the gray value along an image sensor column. This object can be achieved, for example, by the image sensor system described herein (for example, in a following section).
• FIG. 8 shows a schematic representation of different variants for determining a position x 0 of a maximum of the gray value along a sensor column.
• the abscissas 810, 840, 870 each describe a coordinate x along a column of the image sensor or the image sensor matrix.
• Ordinates 812, 842, 872 each describe brightness values in arbitrary units.
• Curves 820, 850, 880 describe the course of the brightness values along pixels of the respective column of the image sensor, which can be represented, for example, by analog values on the column line. In the following, various methods are discussed:
• the maximum of the brightness or a brightness value is sought by calculating the increase of successive brightness values, and by reversing the sign at a zero crossing (positive to negative) Value ma x is registered This method delivers exactly this one pixel position or pixel position.
• a comparison of the gray value with a threshold N M and for the exceeding occurs for each pixel or for each pixel or undershooting the corresponding values x a and x are registered (for example, indices of the image lines that precede or follow an exceeding of the threshold).
• the third standard method is based on the assumption that the laser line has a Gaussian brightness distribution and the sensor provides a linear transfer function of mapped brightness and digital gray value (or analog gray value). Under these circumstances, that is
• the position x c of the center of gravity can be determined with high accuracy, and used as a measure of a maximum of the location of the brightness.
• a correctness of the assumption that there is a Gaussian projected line and that there is linearity is not required from 3.
• they are based on an evaluation of the
• a corresponding number of gray values to be analyzed is the greater, the more accurately the position has to be determined or the larger the examined interval along a column.
• the region of interest primarily determines the speed of image acquisition and image processing.
• a programmable image sensor (“Vision System-on-Chip”) was introduced, which can perform one-dimensional convolutions (1-D convolutions) when reading out the sensor matrix, thus enabling very good column-wise curve smoothing It is possible to achieve very good reliability and a very high profile rate, but a sub-pixel resolution is better than (1) / (2) and can only be achieved by means of different threshold values Speed limited.
• each column it is determined in each column separately, for example by evaluating the pixel data read from the sensor matrix, whether the gray-scale data are relevant, that is to say lie approximately within the interval marked red in FIG. 8 or not. All external data is neither recorded nor output (that is, for example, not transferred to the analog memory or immediately overwritten).
• the processor element may be configured in a column to detect a maximum, as described with reference to FIG. 8a, and, for example, a predetermined (or variable) number of analog values (from the image sensor column lines) "around the maximum" (FIG.
• the processor element may be processed by analogous processing (eg, by subtraction of analog values from two adjacent image sensor lines and subsequent image lines adjacent to the detected line position of the maximum) Sign determination) determine whether there is a maximum (wherein, for example, it is still possible to check whether an absolute value of the intensity is sufficient.) If the presence of a maximum, as shown, for example, in FIG Furthermore, for example, the analog memory matrix so drive that analog values from lines that are around a line position of a detected maximum, in which analog memory is stored for further processing.
• the processor element associated with an image column of the image sensor array may transition from an operating state in which analog values in the analog memory matrix are cyclically overwritten to an operating state in response to detection of a maximum, as shown in FIG. 8a, in which analog values to a line position around the detected line position of the maximum, stored for further processing (and no longer be overwritten cyclically).
• the processor element may detect, for example, by an analog preprocessing with a threshold value comparison, if analog values from a column line of the image sensor are greater than a predefined threshold value, for example greater than the threshold value N n shown in FIG. 8b.
• a predefined threshold value for example greater than the threshold value N n shown in FIG. 8b.
• the analog values selected for storage by the respective process element may be digitized (analog-to-digital converted) and, for example, at a later time (for example, when a region of interest of the image sensor is completely processed by the processor elements) for further digital processing then digitally processed.
• a determination of an interval can take place in various ways.
• a selection of the suitable method occurs, for example, on the basis of, among other things, the expected scattering of the laser line width imaged on the sensor, taking into account the "noise carpet" and the available analog memory in relation to the expected number of laser lines stored analog values per column (that is, for example, the number of analog values stored in a column when the presence of a laser line is detected) or adjusted by a corresponding configuration of the processor element or by digital processing, or by a combination of analog processing and digital processing, column-individually decide in which lines of the image sensor analog values in the analog storage matrix for further processing saved.
• the processor element can change an address information (for example the address information 622) before a next memory access, so that the address information no longer exists a memory cell referenced whose content is to be stored for further processing.
• the processor element may leave the address information unchanged, for example, to cause immediate overwriting on the next write access if it is found that the just-stored analog value should not be stored for further processing.
• the length of the memory interval For storage with a fixed memory content, for example, at the beginning of a certain number of cells in each column, so the length of the memory interval, set, which are controlled according to a ring buffer.
• the address encoder in each processor element or at least in some processor elements) count from a start address to an end address and then reset again.
• the length of the memory interval defines how many values are stored and the start address plus length offset, which memory area is currently active.
• a time for a correct storage can be determined in various ways, depending on the desired complexity and requirement by the subsequent evaluation of the recorded analog values.
• a first option is to trigger the storage process on the basis of the pixel position of the maximum value x ma x (corresponding to variant 1 described above according to FIG. 8 a). Since the maximum value determines the midpoint of the envelope, it must (or should) complete the save operation for that profile approximately halfway along the length of the interval.
• For sub-pixel (sub-pixel) accurate determination of the position of the maximum for example, the value of x ma x and, for example, the associated address value must be stored for later evaluation in addition to the content of the ring buffer. In this case, either each gray value or every n-th gray value can be stored, which can be realized, for example, by incrementing the address counter of the memory only after every n-th read-out image line (pixel line).
• the processor element associated with the image column of the sensor matrix initially receives analog values from successive image lines of the image sensor and cyclically stores them in the form of a ring buffer in a memory area of the analog memory matrix, for example in a predetermined range of lines a given column of the analog memory matrix. For example, the processor element increments (or decrements) an address counter that selects the corresponding line of the analog memory array after each write (or alternatively after each nth write). If the address counter reaches a limit of the predetermined memory area, ie an upper limit (or lower limit) of the predetermined memory area, the address counter is reset in order to re-reference the lower limit (or upper limit) of the predetermined memory area.
• the processor element If the processor element now recognizes the presence of a maximum, as has been described with reference to FIG. 8a, the processor element stores a current state of the address counter and optionally also (or alternatively) information about an upper limit of the current memory area and / or information over a lower bound of the current memory area in a digital memory.
• the process element can, for example, continue to control the address counter in such a way that, after the maximum has been detected, a certain number of analog values are stored in the predetermined memory area, for example to store analog values which follow the maximum.
• the processor element can, for example, select a new memory area by the address counter an initial value of a new memory area is set. In this new memory area, anaiog values of further lines of the image sensor matrix can then be stored again, and said method can be repeated.
• the number of elements (lines) to be digitized in the analog memory for each profile results in relation to a number of lines of the region of interest (Rol) on the pixel field and / or Pixels field. It is greater the larger the region of interest (Rol) considered. If, for example, nine analog values are required for 1000 sensor lines, the result is a compression of approx. 11 1: 1, which also represents a maximum acceleration of the output.
• dynamic interval storage may be optional and, for example, alternative to “fixed interval storage” may be used.
• This variant has the advantage in comparison to the previous one (storing with a fixed interval) that gray values which can be evaluated very well are stored for laser lines of very different widths.
• the disadvantage is the unpredictability of the number of gray values to be stored and thus the number of storable laser lines.
• all analog values that are greater than a certain threshold can be stored in the analog memory for further processing.
• information is stored on which lines of the image sensor the stored analog values belong. The number of analog values stored for a line depends on how many analog values belonging to the line are greater than a corresponding threshold value.
• the white-light interferometry the image processing task consists in a stack of, for example, 10,000 images, pixel by pixel or gray-scale differences or differences of successive gray values in interference modulations, ie if they change significantly, issue.
• zero crossings are to be determined as accurately as possible, then their occurrence can be used as a trigger or trigger, for example, to record one or more associated analog values in the memory for the pixel concerned (pixel). If the zero crossing has occurred, for example, the memory address in the relevant column is incremented and the value stored simultaneously with the zero crossing determination is maintained. The zero crossing determination is made in the processor element (PE), for example, by comparing the current sign with the digitally stored previous one. The image number and the analog value are output for the pixel (pixel).
• PE processor element
• Achievable compression can be up to 1000, depending on the coherence length and stack size. It is important to ensure that the memory is read out at sufficiently short intervals, depending on the surface being considered.
• the processor element may determine that a Analog value in which analog memory is to be stored for further processing.
• the analog memory for example, only those analog values are stored which are regarded as relevant, ie which belong to a zero crossing of a difference value. This significantly reduces the amount of data stored compared to storing all the analog values.
• the filter operator can be formed by the readout arrangement 100, for example.
• the SIMD unit 230 or the analog data path 300 can be used.
• the filtering operation can be carried out by analog signal processing, wherein, for example, analog values from a plurality of sensor cells can be stored in an analogous manner and combined in an analogous manner (for example, weighted).
• Output values of such a filter operation ie, for example, a weighted combination of analog values of several sensor cells
• the output values of the filter operations may also represent the analog values to be stored in the analog memory for further processing.
• an expense in digital image processing can be reduced, for example, by the analog implementation of a filter operator.
• the application of a filter operator or a filter operation can make the decision as to which analog values to store in the analog memory for later processing more reliable.
• Another embodiment includes the use or implementation of "tracking".
• the tracking can be used, for example, to track lines or the movement of lines in an image field, for example, in order to efficiently determine which analog values are to be stored in the analog memory for further processing.
• Another application example is the use or implementation of a flexible region-of-interest (Rol).
• Rol flexible region-of-interest
• the corresponding control can be effected, for example, by the SIMD unit or by a processor element PE.
• the multiplexer described above can also help define a flexible region-of-interest.
• shift in terms of association
• the multiplexer it may be individually decided what "shift" (in terms of association) between column lines of the image sensor array and column lines of the analog memory is be flexibly adjusted by the multiplexer, so that, for example, analog signals originating from a parallelogram-shaped area of the image sensor matrix are stored in a "rectangular" area of the analog memory (with regard to the organization of the analog memory in rows and columns).
• the corresponding control can again take place, for example, via the SIMD unit or via the processor elements.
• Another possible application example is a pattern projection.
• FIG. 9 shows a schematic representation of a system according to an exemplary embodiment of the present invention.
• the system according to FIG. 9 is designated 900 in its entirety.
• the system 900 includes a readout array 910 configured to receive image sensor column signals 912a-912n, and configured to cause a storage of selected analog values in an analog memory 920 based thereon. sen.
• the readout arrangement 910 is connected to memory write lines 922a to 922n.
• the readout arrangement is designed to determine which row of the respective analog memory matrix columns is to be written.
• the readout array may provide a corresponding tick selection signal 924a through 924n for each column of the analogue memory array.
• the number of columns of the analog memory matrix may differ from the number of columns of the image sensor matrix.
• the numbers can be the same.
• the read-out arrangement comprises, for example, column-individual column evaluations and / or column-signal processing for all columns (or at least for a plurality of columns).
• the column evaluations may receive the image sensor column signals 312a-312n and provide write signals 916a-916n provided to a multiplexer 918, for example.
• the multiplexer may set an association between the write signals 916a-916n and the memory write lines 922a-922n, for example, assigning a range of the write signals 916a-916n to a portion of the memory write line 922a-922n in a variable manner.
• a contiguous range of write signals may be associated with a contiguous area of memory write lines, where the range of write signals may be column-offset from the area of memory write lines (eg, assigning an ith write signal to a jth memory write line) and, for example, assigning an i + 1th write signal to a j + 1 th memory write line, etc., where i is different from j.
• the column evaluations 914a to 914n for example, respectively For example, the function of the analog data path 300 shown in FIG.
• 3 may be wholly or partially adopted, and may additionally have the function of the SIMD unit 230 in whole or in part, for example, to decide from which columns analog values for further processing in the Analog memory matrix to be stored It is a device 930 for selecting memory lines to be written (which may, for example, operate on a column-by-column basis) Thus, a selection is possible in which columns of the analog memory matrix analog values from which pixels are stored for further processing, or which memory lines of the analog memory matrix are overwritten or remain unchanged.
• the read-out arrangement can decide in a very fine-grained manner, by means of the interaction of the column evaluation / column signal processing 914a to 914n, of the multiplexer 918 and the selection 930 of memory lines to be written, which input values into the analog memory matrix 920 are stored for further processing and where the analog values are stored in the analog memory matrix. Additional information regarding which analog values have been stored in the analog memory matrix for further processing may be stored, for example, in a digital memory 914 and then available for further processing.
• the column evaluations 914a to 914n can assume the task of the SIMD unit 230 and / or of the analog data path 300, for example.
• the multiplexer 918 may correspond to the memory multiplexer 400, where the write signals 916a-916n may correspond to the signals 410a-410h, for example, and the signals 922a-922n may correspond to the signals 414a-414h, respectively.
• FIG. 10 shows a schematic representation of an evaluation in the presence of a horizontally extending light line.
• FIG. 10 shows a section 1010 of an image sensor matrix, assuming, for example, that pixels in an area 1020 are illuminated with an intensity that is greater than a predetermined threshold (or, alternatively, satisfy another condition involving the pixels as relevant for further processing).
• region 1020 is substantially rectangular.
• a center image row 1030 of the area 1020 may be irradiated with a maximum intensity while outer rows 1032, 1034 of the area 1020 be irradiated with less intensity.
• the remaining image lines in the vicinity of region 1020 are only irradiated at an intensity that is below a threshold (or may alternatively be classified as irrelevant to further processing by some other means).
• analog signals from the pixels in the area 1020 are to be stored in the analog memory for further processing.
• analog signals can be selected from the various pixels, and it can be decided by the evaluation unit assigned to the first image column that, for example, (only) analog values which originate from image sensor cells (pixels or image sensor matrix cells) in rows 1032, 1030, 1034 a first column of the analog memory for further processing are stored. Corresponding decisions can also be made for analog values of image sensor matrix cells in the remaining columns.
• analog signals from the image sensor cells in region 1020 are stored in the analog memory for further processing, whereas analog values from image sensor cells located outside of region 1020 are not stored for further processing. As a result, it can be avoided that irrelevant information is stored in the analog memory for the later digital evaluation.
• a somewhat more complex example is described with reference to FIG.
• a section of the image sensor matrix is shown at reference numeral 1 1 10.
• a light line extending substantially horizontally across the image sensor matrix runs slightly from top left to bottom right.
• a range of image sensor array cells in which a light intensity is greater than a predetermined threshold value (or which fulfill another condition) is denoted by 1 120, for example.
• the region 1 120 is no longer rectangular due to the slightly oblique course of the light line, but has "steps.”
• the region 1 120 extends, for example, in a first considered column 1 140a from a second line 1 130b up to a fourth row 1 130d.
• the area 1 120 extends from the fourth row 1 130d to a sixth row 1 130f (the corresponding widths being understood here purely by way of example).
• analog values of pixels of range 1 120 should be stored in the analog memory, whereas analog values of pixels outside range 120 need not be stored ( and should not be saved) because they do not carry any essential information.
• the readout arrangement may recognize, for example, that only in the image lines 1 130b to 1 130d are analog signal values indicating a relevant brightness.
• the read-out arrangement can for example control the analog memory in such a way that from the column 1 140a only analog values which originate from the picture lines 1 130b to 1 130d are stored in the analog memory for further processing.
• Corresponding analog values which originate from the pixels 1 130b to 1 130d are stored, for example, in rows 1 150b to 1 150d of the analog memory matrix, which is shown schematically at reference numeral 1 148.
• Analog values that originate from the first image column 1 140a are designated by 1 160a to 1 160c, for example.
• the associated processing range of the read-out arrangement recognizes that analog values of image lines 1 130b and 1 130c are not relevant. Therefore, analog values that originate from these image lines 1 130b, 1 130c for the image column 1 140n are not stored in the analog memory matrix for further processing (but at best buffered there and immediately overwritten again).
• the read-out arrangement belonging to the picture chip 1 140n detects that intensities relevant to the later evaluation are present in the picture lines 1 130d, then the read-out arrangement initiates the storage of corresponding analog values, preferably in the same lines of the analog memory matrix 1 148 in which the analog values of the sensor matrix cells (pixels) of the first column 1 40a were also stored.
• the corresponding analog values belonging to the column 1 140n are designated 152b to 1 152d.
• the read array of each column identifies individually (using predetermined or programmable criteria and optionally using signal pre-filtering) signals from which rows 1 130b to 1 130f of the image sensor array belong to a line.
• analog values belonging to a line are then stored in the same lines of the analog memory (for example in the memory lines 1 150b to 1 150d), even if the line runs obliquely over the image sensor matrix.
• analog values belonging to a line are stored in a "rectangular" area of the analog memory (for example, lines 1 150b to 1 150d), even if the line runs obliquely across the image sensor matrix. that analog values are only stored in an area of the analog memory provided for the storage of a line if they fulfill a specific condition in column-individual evaluation, for example above a threshold value or lie around a maximum value within a certain range.
• FIG. 12 shows another example.
• An image sensor array is shown at reference numeral 1210.
• a line here yields significant radii in a region 1220 that is not rectangular, since the line here is slightly oblique from top left to bottom right.
• the line passing over the image sensor matrix produces an intensity profile such that in each image line about three adjacent image sensor cells are illuminated with a "significant" light intensity (which, for example, results in sensor signals above a threshold).
• the line is such that, for example, in a first line under consideration 1230a there is significant light intensity in a second image sensor column 1240b, in a third image sensor column 1240c, and in a fourth image sensor column 1240d.
• a first line under consideration 1230a there is significant light intensity in a second image sensor column 1240b, in a third image sensor column 1240c, and in a fourth image sensor column 1240d.
• a last viewed line 1230h there is a significant light intensity in the fourth column 1240d, in the fifth column 1240e, and in the sixth column 1240f.
• Column-parallel processing of image sensor analog signals of line 1230a by the readout array thus results in significant light intensity values (represented by "significant" image sensor analog signals) being present in columns 1240b, 1240c, 1240d
• analogous values of pixels 1244b e.g.
• 1244c, 1244d are stored in the analogue memory, for example in memory cells 1260b, 1260c, 1260d of memory columns 1250b, 1250c, 1250d. This is accomplished, for example, by adjusting the multiplexer for immediate loop-through so that analog signals are looped through columns 1240b, 1240c, 1240d of the image sensor matrix to the memory columns 1250b, 1250c, 1250d of the analog memory.
• the read array may configure the multiplexer to provide analog signals to the image sensor columns 1240c, 1240d, 1240e are looped through to columns 1250b, 1250c, 1250d of the analog memory, which is associated with a column offset between the image sensor columns and the analog memory columns.
• analog signals from the image sensor cells 1246c, 1246d, 1246e may be stored in storage locations 1262b, 1262c, 1262d.
• analog signals from the pixels 1246c, 1246d, 1246e are thus stored in the same columns of the analog memory, we store the analog signals from the pixels 1244b, 1244c, 1244d, although the analog signal to be stored for further processing is in the image sensor row 1230c a "column offset" set of column lines (as compared to analogue image line 1230a signals to be stored for further processing).
• the read-out arrangement can also detect when the column areas in which significant analog values or analog signals (or light intensities) are present shift from line to line (of the image sensor). If such a shift of the column areas is detected with significant (to be stored) analog values, then the read-out arrangement correspondingly control the multiplexer in order to ensure that the analog values to be stored of different picture lines are stored in the same column area of the analog memory. Thus, all analogous values associated with a line are stored in a rectangular area, even if the line is oblique over the image sensor. This is achieved by the interaction of the components of the readout arrangement, in particular also with the multiplexer.
• FIG. 13 shows a flowchart of a method 1300 according to an embodiment of the present invention.
• the method 1300 includes receiving in parallel 1310 a plurality of image sensor analog signals describing brightness values detected by the image sensor in analog form from a plurality of column lines of the image sensor.
• the method further comprises selecting 1320 which subset of a plurality of analog values represented by the image sensor analog signals or based on the image sensor analogue signals are stored in an analog memory for further processing.
• the method includes storing 1330 the selected analog values in the analog memory.
• the method can be supplemented by all the features and functionalities of the read-out arrangement and of the image sensor system-individually or in combination.
• a memory array (optional) (e.g., the analog memory) (e.g., the analog memory)
• the output that can be addressed line by line to the column is addressable
• a mixed-signal processing unit (optional) a multiplexer (optional)
• ALU digital processing unit
• aspects have been described in the context of a device, it will be understood that these aspects also constitute a description of the corresponding method, so that a block or a component of a device is also to be understood as a corresponding method step or as a feature of a method step , Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device.
• Some or all of the method steps may be performed by a hardware device (or using a hardware device). Apparatus), such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some or more of the most important method steps may be performed by such an apparatus.
• the encoded audio signal of the present invention may be stored on a digital storage medium, or may be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
• embodiments of the invention may be implemented in hardware or in software.
• the implementation may be using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or FLASH memory, a hard disk, or other magnetic or optical storage on which electronically readable stored signals that can cooperate with a programmable computer system or cooperate such that the respective method is performed. Therefore, the digital storage medium can be computer readable.
• some embodiments according to the invention include a data carrier having electronically readable control signals capable of interacting with a programmable computer system such that one of the methods described herein is performed.
• embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is operable to perform one of the methods when the computer program product runs on a computer.
• the program code can also be stored, for example, on a machine-readable carrier.
• inventions include the computer program for performing any of the methods described herein, wherein the computer program is stored on a machine-readable medium.
• an exemplary embodiment of the method according to the invention is thus a computer program which has program code for carrying out one of the methods described here when the computer program runs on a computer.
• a further embodiment of the method according to the invention is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program is recorded for performing one of the methods described herein.
• the data carrier, the digital storage medium or the computer-readable medium are typically representational and / or non-transitory.
• a further exemplary embodiment of the method according to the invention is thus a data stream or a sequence of signals which represents or represents the computer program for performing one of the methods described herein.
• the data stream or the sequence of signals may, for example, go there be configured to be transferred via a data communication connection, for example via the Internet.
• Another embodiment includes a processing device, such as a computer or a programmable logic device, that is configured or adapted to perform one of the methods described herein.
• a processing device such as a computer or a programmable logic device, that is configured or adapted to perform one of the methods described herein.
• Another embodiment includes a computer on which the computer program is installed to perform one of the methods described herein.
• a further embodiment according to the invention comprises a device or a system designed to transmit a computer program for carrying out at least one of the methods described herein to a receiver.
• the transmission can be done for example electronically or optically.
• the receiver may be, for example, a computer, a mobile device, a storage device or a similar device.
• the device or system may include a file server for transmitting the computer program to the recipient.
• a programmable logic device eg, a field programmable gate array, an FPGA
• a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein.
• the methods are performed by any hardware device. This may be a universal hardware such as a computer processor (CPU) or hardware specific to the process, such as an ASIC.
• the devices described herein may be implemented, for example, using a hardware device, or using a computer, or using a combination of a hardware device and a computer.
• the devices described herein, or any components of the devices described herein may be implemented at least in part in hardware and / or software (computer program).
• the methods described herein may be implemented using a hardware device, or using a computer, or using a combination of a hardware device and a computer.
• the methods described herein, or any components of the methods described herein may be performed at least in part by hardware and / or by software.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Transforming Light Signals Into Electric Signals (AREA)
• Image Input (AREA)
• Facsimile Heads (AREA)

Priority Applications (6)

Applications Claiming Priority (4)

Related Child Applications (1)

Publications (1)

Family

ID=60972241

Family Applications (1)

Country Status (6)

Cited By (2)

Families Citing this family (9)

Citations (2)

Family Cites Families (9)

• 2018
  • 2018-01-18 EP EP18700438.7A patent/EP3571836B1/de active Active
  • 2018-01-18 WO PCT/EP2018/051231 patent/WO2018134317A1/de not_active Ceased
  • 2018-01-18 KR KR1020197024226A patent/KR102310054B1/ko active Active
  • 2018-01-18 JP JP2019538626A patent/JP7113830B2/ja active Active
  • 2018-01-18 CN CN201880019211.6A patent/CN110431839B/zh active Active
  • 2018-01-18 EP EP23187782.0A patent/EP4283983A3/de active Pending
• 2019
  • 2019-07-18 US US16/515,418 patent/US11115613B2/en active Active

Patent Citations (2)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events