RU2423016C1 - Method of electronic processing of photodetector signals in image formation and device for its realisation - Google Patents

Abstract

FIELD: information technologies. ^ SUBSTANCE: signals from a photodetector are transformed from an analogue form into a digital one and then processed. The treatment process is subdivided into an initial, an intermediate and a final stage. The first two are executed simultaneously. At the intermediate stage a processor accumulates full-format image arrays, and processing functions are executed over them. At the initial stage - EPLD is used to execute internal functions of image processing. Results are recorded in an intermediate memory with common address and data buses relative to the processor and EPLD, and into a television memory with individual address and data buses connected to EPLD. After processing with the processor, data arrays are sent into the main memory with individual address and data buses connected to EPLD, which store coefficients of uneven sensitivity correction, coefficients of dark background correction and data of defect elements. Using the processor, any specified main memory is addressed at any specified moment of time. At the final stage, using EPLD, the data with completed correction of uneven sensitivity and dark background correction is read from the television memory, and defect elements are replaced. Then the signal is converted into an analogue form. The device comprises ADC, EPLD, a processor, DAC. Units of initial and final processing are based on EPLD. A unit of intermediate processing is based on the processor. The latter is connected to the unit of initial processing by means of the intermediate memory with common address and data buses. The unit of initial processing is equipped with the main memory, connected to EPLD, with individual address and data buses, with quantity equal to quantity of variables used in processing logics. The processor is arranged with the possibility to address any main memory. The initial processing unit is connected to the final processing unit by means of the television memory connected to EPLD. ^ EFFECT: expansion of functional capabilities and increased indices of throughput capacity in processing of signals. ^ 16 cl, 1 dwg

Description

The invention relates to techniques for imaging, in particular, in thermal imaging systems for visualizing images of objects recorded by multi-element photodetectors in the infrared ranges of the spectrum, and are intended for processing signals of matrix or line photodetectors.

A known method of electronic processing of photodetector signals during image formation (RF patent No. 2066057 for the invention, IPC 6 H04N 5/33), which consists in the fact that the formation of the compensation code of the background signal and signal drift for each photosensitive element of the photodetector; generating a sensitivity correction code for each photosensitive element of the photodetector; reception of the radiation flux to be recorded. The first reference radiation stream is supplied to the photodetector when there is no reception of the radiation stream to be registered with the formation and storage of the electrical signal generated by responding to the first reference radiation stream, and the background component of the photodetector is compensated for when the radiation stream to be registered is received by summing the electric signal from the detected stream radiation with an electrical signal in respect of which the memorization is made. At the same time, during the reception of the first reference radiation flux, an electric signal for compensating the background signal and signal drift for each photosensitive element of the photodetector is formed and stored. Additionally, in the absence of reception of the radiation flux to be registered, a second reference radiation flux is supplied with the formation and storage of an electric sensitivity correction signal for each photosensitive element of the photodetector. When receiving the radiation flux to be registered simultaneously with the compensation of the background signal and signal drift for each photosensitive element of the photodetector by summing the pre-processed signal of the corresponding element with the compensation signal of its background signal, in relation to which the memorization was carried out, and the signal drift, sensitivity correction is performed for each photosensitive element photodetector by changing the amplitude of the summed signal for each photosensor an element in accordance with the correction signal of its sensitivity, in relation to which memorization has been made.

In the method, the magnitude of the first reference radiation flux is set in proportion to the magnitude of the external background radiation flux.

In the method, upon receiving the first reference radiation stream, during which the background signal compensation signal and signal drift signal are generated in the form of a digital code for each photosensitive element, the signal is read from the elements synchronously with a sequential sampling of the cell addresses of the first digital random access memory, the number of cells of which is chosen equal to the number of photosensitive elements of the photodetector, a digital code is read at a given address, it is stored and converted to an analog voltage, summed from Tel'nykh processed signal corresponding photosensitive member summed signal is compared with a zero voltage from the comparison result reduce or increase the code against which produced memorizing unit LSB and written in the memory location from which it was read. When a second reference radiation stream is received, during which a sensitivity correction signal is generated in the form of a digital code for each photosensitive element, the signal is read from the photodetector elements in synchronization with the sequential selection of the cell addresses of the first digital random access memory and the organized second digital random access memory, the number of cells of which is chosen equal to the number of photosensitive elements, the digital code for compensating the background signal and signal drift from the first memory is read at a given address and Having formed into an analog voltage, they are summed with a pre-processed signal of the corresponding element of the photodetector, at the same time a digital code is read from the second memory at the same address, stored and with its help the summed signal is reduced or increased in amplitude, which is compared with the reference voltage, and the result of the comparison is reduced or increase the code, in relation to which the memorization is made, by a unit of the least significant digit and write it in that cell of the second memory from which it was read. When receiving the radiation flux to be registered, the signal is read from the photosensitive elements in synchronization with the successive sampling of the addresses of the cells of the first and second memory, the digital code for compensating the background signal and the signal drift from the first memory is read at the given address, and converted to the analog voltage, they are summed with the pre-processed signal of the corresponding photosensitive element, at the same time, the digital sensitivity correction code is read at the same address from the second memory and with its help shayut or increase in amplitude summed signal.

The disadvantages of this method include the narrowness of functionality and low bandwidth when processing signals. These disadvantages are due to the following reasons. The computational actions when generating a compensation code for the background signal and signal drift for each photosensitive photodetector element and generating a sensitivity correction code for each photosensitive photodetector element are based on the use of analog elements, which, due to the limited bandwidth of the digital-to-analog converter (DAC) and analog-to-digital converter (ADC) ) significantly reduces the possible functions. In addition, the method does not allow to realize the possibility of artificial restoration of signals from defective elements, which is one of the most important features of signal processing of the photodetector.

A device for electronic processing of photodetector signals during image formation (RF patent No. 2066057 for an invention, IPC: 6 H04N 5/33), made in the following composition. Optical-mechanical unit with sources of the first and second reference radiation, optically coupled to a photodetector. The signal preprocessing unit, the input of which is electrically connected to the photodetector. A node for sampling the address and synchronizing the reading of signals from the elements of the photodetector containing a clock pulse generator electrically connected to the photodetector, and an address counter with an input connected to a sync generator. A node for generating a background signal compensation and signal drift code for each photosensitive photodetector element, made as part of a random access memory block for storing a background signal and signal drift compensation code, a digital-to-analog converter (DAC), a reversible counter, an adder, a comparator, and a bus with zero voltage. In this case, the multi-bit input of the RAM block for storing the compensation code for the background signal and signal drift is electrically connected to the node for selecting the address and synchronizing the reading of signals from the photodetector elements, and its multi-bit output is connected to the multi-bit input of the reverse counter and DAC. The multi-bit output of the reversible counter is connected to the multi-bit input of the DAC and the multi-bit output of the RAM block to store the code for compensating the background signal and signal drift. The DAC output is connected to one of the inputs of the adder, the second input of which is connected to the output of the signal preprocessing unit. The output of the adder is connected to one of the inputs of the comparator, the second input of which is connected to the bus with zero voltage, and the output of the comparator is connected to the second input of the reversible counter. The device also has a node for generating a sensitivity correction code for each element of the photodetector as part of a random access memory block for storing the sensitivity correction code, second DAC, second reversible counter, amplifier, second comparator, and reference voltage source. In this case, the multi-bit input of the random access memory block for storing the sensitivity correction code is electrically connected to the node for selecting the address and synchronizing the reading of signals from the photodetector elements, and its multi-bit output is connected to the multi-bit input of the second reverse counter and the second DAC. The multi-bit output of the second reversible counter is connected to the multi-bit input of the second DAC and the multi-bit output of the RAM block for storing the sensitivity correction code. The output of the second DAC is connected to one of the inputs of the amplifier, the second input of which is connected to the output of the adder of the formation unit of the compensation code for the background signal and signal drift. The output of the amplifier is connected to one of the inputs of the second comparator, the second input of which is connected to a reference voltage source, and the output of the second comparator is connected to the second input of the second reversible counter. The output of the amplifier is designed to remove the processed signals of the photodetector. The signal preprocessing unit is configured to amplify, integrate, filter, and bias samples. The amplifier is made with the function of regulating the transmission coefficient.

The disadvantages of this device include the narrowness of functionality and low bandwidth of the device when processing signals. These disadvantages are due to the following reasons. Computational actions in the known device when generating a compensation code for the background signal and signal drift for each photosensitive photodetector element and generating a sensitivity correction code for each photosensitive photodetector element are carried out based on the use of analog elements, which, due to the limited bandwidth of the DAC and ADC, significantly reduces the possible functions. In addition, the device does not allow to realize the possibility of artificial restoration of signals from defective elements, which is one of the most important features of signal processing of the photodetector.

As the closest analogue of the proposed method, the method of electronic processing of the photodetector signals during image formation was selected (Tarasov V.V., Yakushenkov Yu.G. Infrared systems of the "looking" type. - M .: Logos, 2004 - 444 p., P. p. 244-247), which consists in the fact that the signal from the photodetector is converted from analog to digital form and then processed by a signal processing processor (PIC). During processing, accurate correction of the heterogeneous sensitivity of the elements, correction of defective pixels (replacing defective elements), automatic adjustment of brightness and contrast, electronic scaling are carried out. Next, a digital video signal is generated. With periodically repeated calibration procedures using PIC calculate the values of the correction coefficients, which are stored in the random access memory (RAM). PIC modes, as well as video signal parameters such as brightness, contrast, electronic scaling factor, are controlled from the control panel. Using a video processor, the digital video signal is converted to a standard analog PAL or NTSC signal.

Prior to converting the signal from the photodetector from an analog form to a digital one, in order to reduce geometric noise, compensation is made for the heterogeneity of the parameters of the sensitive elements in an analog form.

The disadvantages of the closest analogue of the proposed method include the narrowness of functionality and low bandwidth when processing signals. These disadvantages are due to the following reasons. Using PIC to perform basic processing functions imposes a limitation on the performance of electronic processing of photodetector signals, since the input / output channels available on modern processors do not allow the transfer of large amounts of image data in real time. Also, all the above processing functions in the processor are performed sequentially. Such an implementation requires very high performance, which is not yet characteristic of modern digital signal processing processors.

As the closest analogue of the claimed device, a device for electronic processing of photodetector signals during image formation was selected (I.I. Kremis, Yu.F. Only. Unified signal processing system of a multi-element infrared photodetector based on a FPGA type programmable logic chip // Applied Physics, 2007, No. 4, pp. 133-140), containing an analog-to-digital converter (ADC), random access memory (RAM), read-only memory (ROM), programmable logical integral with a chip (FPGA), a signal processing processor (PIC), a digital-to-analog converter (DAC), a microcontroller, a USB 1.1 interface driver (USB is a serial data transfer interface for medium-speed and low-speed peripherals in computer technology). Based on the FPGA, which is the core of the control and data processing unit, an arithmetic logic device (ALU), a control circuit, a driver of the photodetector array control signal, a television signal shaper are implemented. The ADC is connected to the FPGA, the converted signal from the analog form to the digital is fed to the ALU, in which it is exposed to the processing algorithms. FPGA via ALU is connected to memory banks - RAM and ROM.

The disadvantages of the closest analogue of the claimed device include the narrowness of functionality and low bandwidth when processing signals. These disadvantages are due to the following reasons. The lack of intermediate memory between the PIC and the FPGA does not allow the processor to accumulate full-format image arrays and perform processing functions on them and at the same time execute them simultaneously with the FPGA performing its own image processing functions. Further, RAM based on CMOS SRAM chips has small amounts of memory on the chip and a relatively low operating frequency. In addition, the device uses a microcontroller, which fundamentally limits the functionality. Also, the replacement of the signal of the defective element in the known device occurs with the signal of the neighboring element, while the pixel is stretched, which creates the effect of stretching pixels in the image.

The technical result of the group of inventions is:

- expansion of functionality;

- increase in throughput when processing signals.

The technical result is achieved in that in the method of electronic processing of the signals of the photodetector during image formation, which consists in the fact that the signals from the photodetector are converted from analog form to digital and then processed, and the processing process is divided into three stages - initial, intermediate and final, and the original and the intermediate stages are carried out jointly, at the intermediate stage, during processing, the processor accumulates full-format image arrays and performs functions on them and processing, while these steps of the intermediate stage are performed in conjunction with the implementation of the programmable logic integrated circuit of the own image processing functions, comprising the contents of the initial stage, the results of which are recorded in the intermediate memory with shared address and data buses in relation to the processor and the programmable logic integrated circuit, as well into a television memory with individual address and data buses connected to a programmable logic integrated circuit, in its essay d, after processing by the processor, the data arrays are sent to random-access memory with individual address and data buses connected to a programmable logic integrated circuit, storing the data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and data of defective elements; , at any given point in time, access to any of the mentioned random access memory device, connecting data to the programmable logic integrated circuit, at the final stage, carried out by means of the programmable logic integrated circuit, the data with the correction of the inhomogeneous sensitivity and the correction of the dark background are read from the television memory and the defective elements are replaced with the defective elements, then the signal is converted to analog form.

In the method at the initial stage, when the programmable logic integrated circuit implements its own image processing functions, dark background correction and / or inhomogeneous sensitivity correction algorithms are performed, and the corresponding data array is read in synchronously with the address bus value of random access memory devices storing the heterogeneous sensitivity correction coefficients, coefficients correction of the dark background and defective elements, the result of the execution of the algorithms is recorded in opera memory device, which is an intermediate memory, and into one of the two online memory devices, which is a television memory, the address and data bus of which is switched with the frame rate of the television monitor, allowing the alternate use of one of the online memory devices of the television memory for recording originally processed programmable logical integrated circuit data, and the other - to read data and conduct at the final stage of replacement of defective e lements; the number of random-access memory devices storing data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and defective elements is selected in accordance with the number of variables used in the processing algorithms, thereby setting each memory address with the corresponding pixel value in the data array from the photodetector.

In the method at an intermediate stage, when the processor accumulates full-format image arrays and performs the processing function on them, the data array obtained as a result of the initial processing and recorded in the intermediate memory is accumulated the required number of times and processed by the processor, after the processor completes the processing cycle, the resulting data array is sent to online storage devices storing data of correction coefficients of heterogeneous sensitivity, correction coefficients new background and defective elements.

In the method at an intermediate stage, when the processing function is performed by recording the intermediate data by a programmable logic integrated circuit in the intermediate memory in the initial frames of the full processing cycle, the values of the dark background and the values of the corrected image, and in the last frames of the recording from the intermediate memory to random-access memory devices data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and data of defective elements, - updated table If the defective elements, the updated array of correction coefficients of inhomogeneous sensitivity and averaged in accordance with the specified number of times the dark background, for a time equal to the time minus the total processing cycle of the mentioned frames, the processor accumulates the data array of the dark background a specified number of times, form the adjusted table defective elements and update the correction coefficients of heterogeneous sensitivity.

In the method at the final stage, when reading data with the correction of inhomogeneous sensitivity and correcting the dark background from the television memory and replacing the defective elements, data that passed the previous joint processing stages is read from one of the television memory devices, the average value from the neighboring ones is set to the defective element and to the left of full-fledged elements, after which they bind to the television synchronization signals, then adjust the brightness l and contrast, then the service information symbols are inserted into the image array and the defective element is directly replaced by the average value from the surrounding full elements and the median image filtering.

In the method, the separation of the actions of the initial and final stages of processing is carried out by means of a television memory from two random-access memory devices with individual address and data buses, which are switched at a frame rate of the television monitor, allowing the alternate use of one of the random-access memory devices of the television memory to record processed programmed logical an integrated circuit and a data processor, and the other for reading data and conducting Flax treatment stage.

The technical result is achieved in that in the electronic signal processing device of the photodetector when forming an image containing an analog-to-digital converter, a programmable logic integrated circuit, a processor, a digital-to-analog converter, an initial processing unit that implements its own image processing functions is implemented on the basis of a programmable logic integrated circuit, and final processing unit, performing the replacement of defective elements, on the basis of the processor, an industrial unit processing, accumulating full-format image arrays and performing processing functions on them, which is connected to the source processing unit via intermediate memory with shared address and data buses, with the ability to access the processor’s intermediate memory and programmable logic integrated circuit for writing and reading data, analog the digital converter is connected to the initial processing unit, the initial processing unit is equipped with online storage devices connected to the programmable an integrated circuit, with individual address and data buses, in an amount equal to the number of variables used in the processing algorithms, the processor being configured to access any random access memory device connected to the programmable logic integrated circuit, the source processing unit is connected to the final processing unit through a television memory connected to a programmable logic integrated circuit, and the final processing unit is connected to a digital-to-analog conversion ovatelem.

In the device, the initial processing unit that performs its own image processing functions, implemented on the basis of a programmable logic integrated circuit, is made as part of an arithmetic-logic device of a programmable logic integrated circuit; the initial processing unit is equipped with random-access memory connected to a programmable logic integrated circuit with individual address and data buses in an amount equal to the number of variables used in the processing algorithms, namely, a synchronous dynamic random-access memory device for storing defective elements, a synchronous dynamic random-access memory device storing the correction coefficients of the heterogeneous sensitivity of the elements, synchronous dynamic operational storage inaudible device for storing the correction coefficients of the dark background; the arithmetic-logical device is connected to the specified operational storage devices through a memory controller.

In the device, the television memory connected to the programmable logic integrated circuit is made up of the first and second synchronous dynamic random-access memory devices for storing image data, which are connected to the programmable logic integrated circuit via a memory controller, the operative memory devices of the television memory are made with individual address and data buses , with the ability to switch them with a frame rate of a television monitor, providing use Maintenance of one of them to write the processed node of the original data, and the other - to read the data and final processing.

In the device, the intermediate processing unit, implemented on the basis of the processor, accumulating full-format image arrays and performing processing functions on them, is made up of the processor itself, the USB controller, with which the processor is connected to access a personal computer, and flash memory associated with the processor that stores arrays of correction coefficients of inhomogeneous sensitivity and a dark background for periodic, if necessary, their rewriting by the processor in the intermediate memory, electrically resync rammable read-only memory associated with the processor for accounting for operating time.

In the device, an intermediate memory with shared address and data buses with the possibility of accessing the processor and the programmable logic integrated circuit for writing and reading data is made as part of a synchronous dynamic random-access memory, to achieve the possibility of accessing the programmable logic integrated circuit, the processor conclusions are implemented with the possibility switching to the third state.

In the device, the connection of address buses and processor data to the programmable logic integrated circuit is configured to access any random access memory device connected to the programmable logic integrated circuit by using a memory controller.

In the device, the final processing unit, which is based on a programmable logic integrated circuit, replacing defective elements, is made up of sequentially connected defective element replacement unit, television generator, brightness and contrast adjustment unit, service symbol formation unit, digital filtering unit, and is connected via the latter to television memory, which provides the ability to read processed by the nodes of the initial and intermediate data processing, and the node Digital filtering is connected to a digital-to-analog converter.

In the device, the analog-to-digital converter is connected to the initial processing unit, implemented on the basis of a programmable logic integrated circuit, by means of an arithmetic-logic device that implements processing algorithms with respect to the signals received from the analog-to-digital converter.

In the device, the nodes of the initial, intermediate and final processing by the memory controller are synchronized with the processed data stream.

The essence of the group of inventions is illustrated by the following description and the accompanying drawing.

The drawing shows a structural diagram of a device for electronic processing of photodetector signals during image formation, where 1 is an analog-to-digital converter (ADC), 2 is a programmable logic integrated circuit (FPGA), 3 is a synchronous dynamic random access memory storage device for defective elements (SDRAM), 4 - USB controller (USB is a serial data transfer interface for medium-speed and low-speed peripherals in computer technology), 5 - synchronous dynamic random-access memory your storage of correction coefficients of inhomogeneous element sensitivity (SDRAM); 6 - arithmetic logic unit (ALU); 7 - synchronous dynamic random access memory storage device for dark background correction coefficients (SDRAM), 8 - signal processing processor (PIC), 9 - synchronous dynamic random access memory storage device for intermediate, PIC-FPGA, data (SDRAM), 10 - defective element replacement unit 11 - a memory controller, 12 - a television generator, 13 - the first synchronous dynamic random-access memory (SDRAM), 14 - brightness and contrast adjustment unit, 15 - flash memory, 16 - electrically reprogrammed iruemoe read only memory (EEPROM), 17 - second synchronous dynamic random access memory image data storage device (SDRAM), 18 - forming unit characters 19 - digital filter unit, 20 - analog converter (DAC).

The achievement of the technical result in the group of inventions is as follows.

In the proposed method, in addition to signal processing by the processor (PIC), programmable logic integrated circuit (FPGA) processing is used, and the FPGA is selected as the basis for implementing the processing functions. The entire processing process is divided into three stages: initial, intermediate and final. Two parts of the processing functions at the initial and final stages are performed by the FPGA; one third of the processing functions at the intermediate stage are performed by the FPGA. In this case, the initial and intermediate processing stages are carried out jointly. The role of FPGAs is to perform their own image processing functions, and the PIC is allocated to the accumulation of full-format image arrays and the implementation of the processing functions on them. The specified division of the process at the stage with the distribution of functions between the PIC and FPGA allows you to move away from the sequential execution of processing functions; to increase the performance of electronic processing of photodetector signals, providing the ability to transmit large amounts of image data in real time.

To carry out the initial processing stage, FPGAs use random-access memory devices that store the data necessary to execute processing algorithms, namely, arrays of defective elements, inhomogeneous sensitivity coefficients, and dark background correction coefficients. Data reading is carried out synchronously, which allows, in particular, to triple the throughput in signal processing. The number of random access memory devices is chosen equal to the number of variables used in the processing algorithms. Each memory address corresponds to a specific pixel (sensitive element) value in the data array from the photodetector, which is read from random-access memory devices synchronously with the clock cycles of the polling frequency of the photodetector pixels. The data obtained as a result of the initial stage of signal processing is recorded in a television memory - as part of two random-access memory devices connected to the FPGA, storing image data. Each of all the aforementioned storage devices has an independent address bus and a data bus.

In addition, to implement the intermediate processing stage, carried out in conjunction with the initial stage, which processes the intermediate data arrays, use a similar random access memory device - intermediate memory, which records the data obtained as a result of the initial FPGA processing stage, to which the PIC is accessed . The specified storage device - intermediate memory has a common address and data bus for sharing in relation to the PIC and FPGA. When the FPGA is accessed to the intermediate memory, the PIC conclusions are transferred to the third state. In the implementation of the intermediate stage ^{- the} process of processing by means of PIC, a periodic, at any given point in time, call to any random-access memory device connected to the FPGA is performed.

When performing the specified function of processing the intermediate data, periodically, at a given point in time of the full processing cycle, corresponding to the initial frames of the cycle, the resulting array of image data (dark background values and corrected image values) is written via FPGAs to the intermediate memory in addition to the television memory, thereby providing PIC for processing the remaining time from the full cycle corresponding to the frames following the recording in the initial frames and free from Apis. In turn, by means of PIC, when performing the specified function of processing the intermediate data, the processed image data array (updated table of defective elements, updated array of correction coefficients of non-uniform sensitivity and averaged in accordance with a specified number of times) is also recorded at a given point in time corresponding to the last frames of the processing cycle dark background) from the intermediate memory to any random access memory, thereby providing a PIC for processing the remaining the entire cycle time corresponding to previous frames that are free from recording.

To separate the actions of the initial and final stages of FPGA processing in television memory, two separate random-access memory devices that store image data are used. The address and data buses of these devices are switched with the frame rate of the television monitor, which allows the use of one random access memory device for recording data obtained as a result of the initial processing, and the second for reading data and processing them in the final stage.

At the final stage of processing at the previous joint stages, the processed data recorded in the television memory is read from one of the operative memory devices of the television memory and the defective elements are replaced, in which the latter are replaced by the average value from the full-fledged elements adjacent to the right and left. In this case, the data is read with the correction of the inhomogeneous sensitivity performed on them and the correction of the dark background from the television memory, the defective element is set according to the updated table the average value from the full-fledged elements adjacent to the right and left, then they are linked to the television synchronization signals, then the brightness and contrast are adjusted, Then, service information symbols are inserted into the image array and the defective element is directly replaced by the average value from the environment complete elements and median image filtering.

Thus, the achievement of the specified technical result in the method is based on the following. Firstly, the possibility of ensuring the PIC accumulation of full-format image arrays and processing functions performed on them, which is performed in conjunction with the implementation of FPGAs of their own image processing functions and recording the results of FPGA processing in intermediate memory with common address and data buses with respect to PIC and FPGA, and also in television memory with individual address and data buses connected to the FPGA. Secondly, the PIC accumulation of full-format image arrays and the processing functions performed on them accompany the transfer of data to random-access memory with individual address and data buses connected to the FPGA, storing data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and defective elements. Thirdly, in the process of processing by means of PIC, a periodic, at any given moment in time, call to any random-access memory device connected to the FPGA is carried out. Fourthly, at the end of the processing process, the data with the correction of the inhomogeneous sensitivity and the correction of the dark background are read from the television memory and defective elements are replaced. Moreover, the entire process of processing the signals of the photodetector is divided in such a way that the signals are first processed with the correction of sensitivity inhomogeneity and dark background — the initial and intermediate stages are performed together, and the signal processing is completed by replacing defective elements at the final stage, which follows after the initial and intermediate .

In the proposed device, the main features to achieve a technical result are the following features of its implementation. On the basis of FPGA (2), an initial processing unit that implements its own image processing functions and a final processing unit that replaces defective elements is implemented. On the basis of PIC (8), an intermediate processing unit has been implemented that accumulates full-format image arrays and performs processing functions on them. The nodes of the initial and intermediate processing function together. The distribution of functions between the PIC (8) and the FPGA (2) allows you to move away from the sequential execution of processing functions and increase the performance of electronic processing of photodetector signals, making it possible to transmit large amounts of image data in real time. Between the PIC processor (8) and the programmable logic integrated circuit - FPGA (2) (see drawing), an intermediate memory is made - a synchronous dynamic random-access memory device for storing intermediate data SDRAM (9), which allows the processor to accumulate full-format image arrays and perform functions over them processing and at the same time to execute them in conjunction with the performance of FPGAs (2) of their own functions, which is the basis for the implementation of processing functions. In this case, the SDRAM intermediate memory (9) has common address and data buses, which provide access to the intermediate memory of both the PIC (8) and the FPGA (2). The result of the initial processing for the formation of intermediate data arrays is recorded in the intermediate data storage device PIC-PLIS-SDRAM (9) to perform intermediate processing on the generated data array. This array can be accumulated the required number of times and processed by PIC (8). The initial processing unit is also equipped with random-access memory devices connected to the FPGA (2), with individual address and data buses storing data on defective elements, correction coefficients of the inhomogeneous sensitivity of the elements and the dark background. The number of devices mentioned is equal to the number of variables used in the processing algorithms. This feature allows you to get the maximum throughput of the device, limited only by the interface through which the video stream from the photodetector arrives for processing. After processing, the PIC (8) sends an array of data to random-access memory devices connected to the FPGA (2), storing data on defective elements, correction coefficients of the inhomogeneous sensitivity of the elements and the dark background. PIC (8) is configured to access any random access memory device connected to the FPGA (2). The initial processing unit is connected to the final processing unit by means of a television memory connected to the FPGA (2). This separates the actions of the nodes of the initial and final processing. The presence of three processing nodes with a strict distribution in relation to their signal processing functions also allows you to expand the functionality of the electronic signal processing device of the photodetector.

The proposed device for electronic processing of photodetector signals during image formation contains the following elements (see drawing): ADC (1), FPGA (2), synchronous dynamic random access memory storage device for defective elements - SDRAM (3), USB controller (4), synchronous dynamic online storage device for storing the coefficients of correction of the heterogeneous sensitivity of the elements - SDRAM (5), ALU (6), synchronous dynamic online storage device for storing the coefficients of correction of the dark background - SDRAM (7), about signal processing processor - PIC (8), synchronous dynamic random access memory storage device for intermediate, PIC-FPGA, data - SDRAM (9), defective element replacement unit (10), memory controller (11), television generator (12), the first synchronous dynamic random access memory device for storing image data - SDRAM (13), brightness and contrast adjustment unit (14), flash memory (15), electrically reprogrammable read-only memory device - EEPROM (16), second synchronous dynamic random access memory device XP Image data analysis - SDRAM (17), symbol generation unit (18), digital filtering unit (19), DAC (20).

In the general case, the device for electronic processing of photodetector signals during image formation is made up of ADCs (1), FPGAs (2), PICs (8), DACs (20), intermediate memory - SDRAM (9), television memory, random access memory in the amount of equal to the number of variables used in the processing algorithms. Moreover, on the basis of FPGA (2), the initial processing unit that implements its own image processing functions and the final processing unit that performs the replacement of defective elements is implemented. On the basis of PIC (8), an intermediate processing unit has been implemented that accumulates full-format image arrays and performs processing functions on them. The intermediate processing unit is connected with the initial processing unit by means of an intermediate SDRAM (9) intended for storing intermediate data. SDRAM (9) has common address and data buses in relation to PIC (8) and FPGA (2), which provides the ability to access the intermediate memory SDRAM (9) PIC (8) and FPGA (2) for writing and reading data. ADC (1) is connected to the source processing unit. The initial processing unit is equipped with random-access memory devices connected to a programmable logic integrated circuit with individual address and data buses in the indicated quantity. PIC (8) is configured to access any random access memory device connected to the FPGA (2). The initial processing unit is connected to the final processing unit by means of a television memory connected to the FPGA (2). The final processing unit is connected to the DAC (20).

The initial processing unit that performs its own image processing functions implemented on the basis of FPGA (2), in particular, is made as a part of ALU (6) FPGA (2). In a specific implementation case, the initial processing unit is equipped with a synchronous dynamic random access memory device for storing defective SDRAM elements (3), a synchronous dynamic random access memory device for storing the coefficients of non-uniform sensitivity of SDRAM elements (5) and a synchronous dynamic random access memory device for storing SDRAM dark background correction coefficients ( 7). ALU (6) is connected with the indicated SDRAM (3), SDRAM (5) and SDRAM (7) through the memory controller (11).

The television memory connected to the FPGA (2) is made up of the first and second synchronous dynamic random-access memory devices SDRAM (13) and SDRAM (17), respectively, which are connected to the FPGA (2) via a memory controller (11). SDRAM (13) and SDRAM (17) are made with individual address and data buses with the ability to switch them with a frame rate of a television monitor. The specified execution ensures the use of one of them for recording the data processed by the node of the initial processing, and the other for reading the data and final processing.

The intermediate processing unit implemented on the basis of PIC (8), which accumulates full-format image arrays and performs processing functions above them, is made as a part of the PIC (8) directly, USB controller (4), flash memory (15) and EEPROM (16). PIC (8) is connected to a USB controller (4) to access a personal computer. Flash memory (15) associated with PIC (8) is used to store arrays of correction coefficients of inhomogeneous sensitivity and a dark background with the aim of periodically rewriting them, if necessary, of PIC (8) in the intermediate SDRAM (9). The electrically reprogrammable read-only memory EEPROM (16) is connected to the PIC (8) and is used to implement the PIC (8) to record operating time.

To achieve the possibility of access FPGA (2) to the intermediate memory SDRAM (9) with common address and data buses and the ability to access it PIC (8) and FPGA (2) for writing and reading data, performed as part of a synchronous dynamic random-access memory, PIC conclusions (8) are implemented with the possibility of switching them to the third state.

PIC (8) is configured to access any synchronous dynamic random access memory device connected to the FPGA (2): SDRAM (3), SDRAM (5) and SDRAM (7), as well as SDKAM (9), by using a memory controller (11 ) Connecting the address bus and PIC data (8) to the FPGA (2) via the memory controller (11) provides the ability to access any random-access memory device connected to the FPGA (2).

The final processing unit implemented on the basis of FPGA (2), which replaces defective elements, is made up of sequentially connected replacement nodes for defective elements (10), a television generator (12), a brightness and contrast adjustment unit (14), and a service symbol generation unit (18) ), a digital filtering unit (19). Elements (10) - (19) are composite FPGAs (2). The node for replacing defective elements (10) is associated with a television memory that provides the ability to read processed by the nodes of the initial and intermediate data processing. The digital filtering unit (19) is connected to the DAC (20).

The ADC (1) is connected to the initial processing unit implemented on the basis of the FPGA (2) by means of the ALU (6), which implements the processing algorithms with respect to the signals received from the ADC (1).

In the device, the nodes of the initial, intermediate and final processing by means of the memory controller (11) are synchronized with the processed data stream.

For the practical implementation of the device it is possible, for example, to use the following electronic components: as FPGAs (2) - FPGA FP3C120F78017 FPGA chip; as PIC (8) - chip processor ADSP-BF533SBBC500; as a USB controller (4) - a FT2232HQ USB controller chip; as random access memory devices SDRAM (3), SDRAM (5), SDRAM (7), SDRAM (9), SDRAM (13), SDRAM (17) - SDRAM chip MT48LC16M16A2BG-75 IT; as flash memory (15) - chip FLASH memory M25P64VME6G; as an electrically reprogrammable read-only memory EEPROM (16) chip EEPROM AT25256AN-10SI-2.7; as an ADC (1) - an ADS850 ADC chip; as the DAC (20), the AD9762ARU DAC chip.

The device operates as follows (see drawing).

The video signal from the photodetector arrives before the initial processing at the ADC (1), where it is converted from digital form to digital. In digital form, the signal is fed to the initial processing unit, implemented on the basis of FPGA (2), namely, it is transmitted to ALU (6), which implements algorithms for correcting the dark background or correcting the inhomogeneous sensitivity of elements. In each of these cases, in parallel with the value of each address bus of random-access memory devices connected to the FPGA (2), - SDRAM (3), (5) and (7), the corresponding data array is read. Each memory address corresponds to a pixel value in the data array from the photodetector. Arrays of defective elements, correction coefficients of inhomogeneous sensitivity and correction of dark background are stored in the SDRAM microcircuits (3), (5) and (7), respectively. Data is read from the indicated random access memory devices synchronously with the polling frequencies of the photodetector, which allows for one clock cycle of the polling frequency to obtain for ALU (6) the values of three variables for arithmetic operations. It also allows you to triple the correspondingly increase the system bandwidth compared to using sequential access to the same data arrays if they are located in the same random access memory. The result of arithmetic operations is recorded in the television memory - in one of the operational storage devices of the television output, SDRAM (13) or SDRAM (17). The address and data buses of the SDRAM (13) and SDRAM (17) random access memory devices are switched at the frame rate of the television monitor, which allows one of them to be used alternately for recording processed ALU (6) data, and the other for reading data and its final processing.

Together with the source processing unit, the intermediate processing unit performs its functions.

After processing in ALU (6), the result of which, in addition to the television memory (SDRAM (13) and SDRAM (17)), is written to the intermediate SDRAM (9), where this array can be accumulated the required number of times and then processed by the PIC processor (8) . After processing, the PIC processor (8) sends the data array to the synchronous dynamic random access memory devices SDRAM (3), (5) and (7). If we put 50 frames for the full cycle of processing intermediate data, then recording in the intermediate SDRAM (9), in the first frame, the value of the dark background, in the second frame the value of the corrected image, and in 48 frames, writing from the intermediate memory SDRAM (9) to the RAM SDRAM device (3) an updated table of defective elements, in frame 49 writing from the intermediate memory SDRAM (9) to the random access memory SDRAM (5) updated array of correction factors for the heterogeneous sensitivity of the elements, in frame 50 recording If the dark background of the array of photodetector elements averaged over the SDRAM (9) from the intermediate memory SDRAM (7) averaged over a specified number of times is dark, then the remaining time, including from 3 frames to 47 frames each, the PIC processor (8) accumulates a dark background for a given amount times, forms an adjusted table of defective elements and updates the correction coefficients of the inhomogeneous sensitivity depending on the temperature of the observed object. In addition, arrays of correction coefficients for the inhomogeneous sensitivity of the photodetector elements stored in flash memory (15) are periodically rewritten by the POS processor (8) into the SDRAM intermediate memory (9) depending on external conditions. The PIC processor (8) also records the operating time of the system and writes this information to the electrically reprogrammable read-only memory EEPROM (16). Downloading primary data to the specified memory chips and processing control is carried out using USB. The access of the PIC processor (8) to each random access memory device is carried out thanks to the memory controller (11) and allows you to use the PIC processor (8) to read and write data arrays through the channel USB controller (4) personal computer (PC).

The final processing unit performs its functions as follows.

The data processed during the joint action of the nodes of the initial and intermediate processing are read from the operational memory SDRAM (13) or SDRAM (17) of the television memory and fed to the replacement node for defective elements (10). The defective element is set to the average value from the full-fledged elements adjacent to the right and left. The signal is fed to a television generator (12) and linked to the television synchronization signals. Next, the signal enters the brightness and contrast adjustment unit (14), the image array is analyzed to form the values of the automatic brightness and contrast adjustments necessary during operation. After that, the signal enters the symbol forming unit (18), where service information symbols are inserted into the image array. At the end of the final processing, the signal enters the digital filtering unit (19), where the defective elements are directly replaced by the average value from the surrounding high-grade elements and the median image filtering.

After passing through all stages of processing, the digital signal is fed to the DAC (20) and converted into analog form.

Thus, the proposed group of inventions allows you to perform the following functions: background compensation, correction of the inhomogeneous sensitivity of the photodetector elements, replacing defective elements with neighboring elements on the right and left or the average value from surrounding elements, scaling, inserting symbolic information into the image, automatic and manual brightness and contrast adjustment image, median and low-pass filter (background low-frequency) image filtering, filtering unsharp masking, frame com sensations of the background, accumulation of frames of the current image and the output of their average frame, the formation and correction of the table of defective elements in real time, the correction of the correction coefficients of the heterogeneous sensitivity of the photodetector elements in real time, the loading of an array of correction coefficients depending on the temperature of the observed objects in real time , accounting of operating time and the number of device starts, data exchange with a PC via USB.

These functions are absent, in aggregate, for all known technical solutions of a similar purpose and determine the advantages of the proposed technical solution, if the proposed one is compared with the selected prototype, as well as other analogues, under equal conditions, including, for example, energy consumption indicators and a set of electronic components for it practical implementation. The use of memory chips such as SDRAM significantly reduces the power consumption of the system.

Claims (16)

1. The method of electronic processing of the signals of the photodetector when forming an image, which consists in the fact that the signals from the photodetector are converted from analog form to digital and then processed, characterized in that the processing process is divided into three stages - initial, intermediate and final, with the initial and intermediate the stages are carried out jointly, at the intermediate stage during processing, the processor accumulates full-format image arrays and performs processing functions on them, while These actions of the intermediate stage are carried out in conjunction with the implementation of the programmable logic integrated circuit of its own image processing functions, which comprise the contents of the initial stage, the results of which are recorded in the intermediate memory with common address and data buses in relation to the processor and the programmable logic integrated circuit, as well as in the television memory with individual address and data buses connected to a programmable logic integrated circuit, in turn, after processing the processor rum data arrays are sent to random-access memory with individual address and data buses connected to a programmable logic integrated circuit, storing data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and data of defective elements, when the processing process is implemented by the processor, they are carried out periodically, at any given point in time, access to any mentioned random access memory device connected to a programmable log At the final stage, carried out by means of a programmable logic integrated circuit, the data with the correction of the inhomogeneous sensitivity and the correction of the dark background are read from the television memory and the defective elements are updated with the defective elements, then the signal is converted to the analog form. 2. The method according to claim 1, characterized in that at the initial stage, when the programmable logic integrated circuit implements its own image processing functions, dark background correction and / or inhomogeneous sensitivity correction algorithms are performed, while the corresponding data array is read in synchronization with the value of the address memory bus devices storing data of correction coefficients of heterogeneous sensitivity, correction coefficients of the dark background and defective elements, the result of performing al the rhythms are recorded in a random access memory device, which is an intermediate memory, and in one of two random access memory devices, which are a television memory, the address and data buses of which are switched at a frame rate of a television monitor, allowing the alternate use of one of the random access memory devices of a television memory for recording originally processed data by a programmable logic integrated circuit, and the other for reading data and conducting at the final stage adia substitution of defective elements; the number of random-access memory devices storing data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and defective elements is selected in accordance with the number of variables used in the processing algorithms, thereby setting each memory address with the corresponding pixel value in the data array from the photodetector. 3. The method according to claim 1, characterized in that at an intermediate stage, when the processor accumulates the full-format image arrays and performs the processing function on them, the data array obtained as a result of the initial processing and recorded in the intermediate memory is accumulated the required number of times and processed by the processor, after completion of the processing cycle by the processor, the resulting data array is sent to random-access memory devices storing data of correction coefficients of heterogeneous sensitivity, to correction factors for dark background and defective elements. 4. The method according to claim 1 or 3, characterized in that at an intermediate stage, when the processing function is performed by recording the intermediate data by a programmable logic integrated circuit in the intermediate memory in the initial frames of the complete processing cycle, the values of the dark background and the value of the corrected image, and the last frames of the recording from the intermediate memory into random access memory devices storing data of correction coefficients of inhomogeneous sensitivity, correction coefficients of the dark background and the data are defective elements, - an updated table of defective elements, an updated array of correction coefficients of inhomogeneous sensitivity and averaged in accordance with a given number of times the dark background, for a time equal to time minus the above frames from the full processing cycle, the processor accumulates the data array of the dark background a specified number of times form an adjusted table of defective elements and update the correction coefficients of the inhomogeneous sensitivity. 5. The method according to claim 1, characterized in that at the final stage, when reading data with the correction of the inhomogeneous sensitivity and correcting the dark background from the television memory and replacing the defective elements, data that passed the previous joint processing stages is read from one of the television storage devices memory, defective element set the average value from the neighboring on the right and left of the full elements, and then carry out the binding to television synchronization signals and then adjust the brightness and contrast, the array further inserted image and overhead information symbols is carried out directly replacing a defective element on the average value of the surrounding elements of full and median filtering the image. 6. The method according to claim 2, characterized in that at the final stage, when reading data with the correction of inhomogeneous sensitivity and correcting the dark background from the television memory and replacing the defective elements, the data that passed the previous joint processing stages is read from one of the television storage devices memory, defective element set the average value from the neighboring on the right and left of the full elements, and then carry out the binding to television synchronization signals and then adjust the brightness and contrast, the array further inserted image and overhead information symbols is carried out directly replacing a defective element on the average value of the surrounding elements of full and median filtering the image. 7. The method according to claim 1, or 2, or 5, or 6, characterized in that the separation of the actions of the initial and final stages of the processing is carried out by means of television memory from two random-access memory devices with individual address and data buses, which switch with a frame rate a television monitor, providing the possibility of alternately using one of the operative storage devices of the television memory for recording data processed by a programmable logic integrated circuit and processor, and another go - for reading data and conducting the final stage of processing. 8. A device for electronic processing of signals of a photodetector during image formation, comprising an analog-to-digital converter, a programmable logic integrated circuit, a processor, a digital-to-analog converter, characterized in that, on the basis of a programmable logic integrated circuit, an initial processing unit that implements its own image processing functions and a node final processing, performing the replacement of defective elements, an intermediate processing unit is implemented on the basis of the processor and, accumulating full-format image arrays and performing processing functions over them, which is connected to the source processing unit via intermediate memory with shared address and data buses, with the ability to access the intermediate processor memory and programmable logic integrated circuit for writing and reading data, analog-digital the converter is connected to the initial processing unit, the initial processing unit is equipped with online storage devices connected to a programmable logic integra an individual circuit with individual address and data buses in an amount equal to the number of variables used in the processing algorithms, the processor being configured to access any random access memory connected to the programmable logic integrated circuit, the initial processing unit is connected to the final processing unit by television memory connected to a programmable logic integrated circuit, and the final processing unit is connected to a digital-to-analog converter. 9. The device according to claim 8, characterized in that the initial processing unit that performs its own image processing functions, implemented on the basis of a programmable logic integrated circuit, is made as part of an arithmetic-logic device of a programmable logic integrated circuit; the initial processing unit is equipped with random-access memory connected to a programmable logic integrated circuit with individual address and data buses in an amount equal to the number of variables used in the processing algorithms, namely, a synchronous dynamic random-access memory device for storing defective elements, a synchronous dynamic random-access memory device correction factors for the heterogeneous sensitivity of the elements, synchronous dynamic operative storage huge capacity storage device the correction coefficients dark background; the arithmetic-logical device is connected to the specified operational storage devices through a memory controller. 10. The device according to claim 8, characterized in that the television memory connected to the programmable logic integrated circuit is made up of the first and second synchronous dynamic random-access memory devices for storing image data that are connected to the programmable logic integrated circuit via a memory controller television memory devices are made with individual address and data buses, with the ability to switch them with a frame rate of a television monitor ora, ensuring the use of one of them for recording the data processed by the node of the initial processing, and the other for reading the data and final processing. 11. The device according to claim 8, characterized in that the intermediate processing unit, implemented on the basis of the processor, accumulating full-format image arrays and performing processing functions on them, is made as a part of the processor itself, the USB controller, with which the processor is connected to access a personal computer flash memory associated with the processor, storing arrays of correction coefficients of inhomogeneous sensitivity and a dark background for periodic, if necessary, their rewriting by the processor in the intermediate kneading, electrically reprogrammable read-only memory associated with the processor for work time tracking. 12. The device according to claim 8, characterized in that the intermediate memory with shared address and data buses, with the possibility of access to it by a processor and a programmable logic integrated circuit for writing and reading data, is made as part of a synchronous dynamic random-access memory device, to achieve the possibility of access to it a programmable logic integrated circuit, the processor pins are implemented with the ability to switch to the third state. 13. The device according to claim 8, characterized in that the connection of the address bus and processor data to the programmable logic integrated circuit is configured to access any random access memory connected to the programmable logic integrated circuit by using a memory controller. 14. The device according to claim 8, characterized in that the final processing unit implemented on the basis of the programmable logic integrated circuit, which replaces the defective elements, is made up of sequentially connected defective element replacement unit, a television generator, a brightness and contrast adjustment unit, a service forming unit characters, the digital filtering node and is connected through the latter with a television memory that provides the ability to read processed by the nodes of the source and intermediate data processing, and the digital filtering node is connected to a digital-to-analog converter. 15. The device according to claim 8, characterized in that the analog-to-digital converter is connected to the initial processing unit, implemented on the basis of a programmable logic integrated circuit, by means of an arithmetic-logical device that implements processing algorithms in relation to signals received from the analog-to-digital converter . 16. The device according to claim 8, characterized in that the nodes of the initial, intermediate and final processing by the memory controller are synchronized with the processed data stream.