RU2663537C1 - Optoelectronic device - Google Patents

Abstract

FIELD: optics.

SUBSTANCE: invention can be used in surveillance systems made on matrix photodetectors (MFDs). Optoelectronic device (OED) contains an optical system in which the MFD is located in the focal plane, the outputs of which are connected to a multichannel analog-to-digital converter (ADC) via the multichannel analog processing unit and then through the multiplexer to the video input of the video processing and control unit (VCU), as well as a control unit, whose input is connected to the first output of the VCU, and the corresponding outputs are connected to the control inputs of the multichannel ADC and MFD, and the device of the interface, the video input of which is connected to the video output of the VCU, and the video output is the video output of the OED. Second output of the VCU is connected to the control input of the multiplexer. VCU is made in the form of a multiprocessor device with the possibility of an incremental and dynamic redistribution of computing resources. Third output of the VCU is connected to the input of the optical system control, the first port of the interface device is connected to the control port of the VCU, and the second port of the interface device is the control port of the OED.

EFFECT: increased speed with expanded functionality without significant increase in power consumption.

5 cl, 1 dwg

Description

The invention relates to optical instrumentation and can be used in surveillance systems made on matrix photodetector devices (MFPs).

A known optical-electronic device (patent RU 2558351, IPC H04N 5/33, published 08/10/2015), containing a lens in the focal plane of which is a matrix photodetector, outputs connected to the inputs of a multi-channel pre-amplifier, analog-to-digital converter, multiplexer , a control unit, the output of which is connected to a control input of a photodetector, a video processor, a control output connected to an input of a control unit and a control input of a multiplexer, a video signal output unit, the output of which is the output of a thermal imaging channel, a hardware signal processing unit connected between the multiplexer output and the input of the video output unit and connected to the data input / output port to the corresponding port of the video processor, and a control input to the control output of the video processor, while the hardware signal processing unit is implemented with the possibility simultaneously performing at least two different computational operations of digital data processing.

An optical-electronic device operating in the infrared region of the spectrum is also known (patent RU 2387092, IPC H04N 5/33, published January 20, 2010) containing a lens in the focal plane of which an MFP is located, the outputs of which are connected to the inputs of the corresponding preamplifiers, in series connected multichannel analog-to-digital converter (ADC), the inputs of which are connected to the outputs of the corresponding preamplifiers, a multiplexer, a video processor and a video output unit, as well as a control unit whose output is connected to the control unit MFP input.

A disadvantage of the known analog devices is the low reliability of the detection of thermal objects, due to the limited matching of the dynamic range of the electrical signal with the MFP with a valid input range of a multi-channel ADC.

The prototype is an optical-electronic device operating in the infrared range of the spectrum (patent RU 2425463, IPC H04N 5/33, published July 27, 2011), containing an optical system in the focal plane of which an MFP is located, the outputs of which are through a multi-channel analog processing device ( UAO) are connected to the multi-channel ADC and then through the multiplexer to the video input of the video processing and control device (UVU), a control unit whose input is connected to the first output of the UVU, the first output of the control unit is connected to the control input nogokanalnogo ALM, and the second output - to the control input of the MFP, interface device, which is connected to the video input UBV video output and the video output video output is opto-electronic device, the second output UBV connected to the control input of the multiplexer.

Such a device with the help of a multi-channel UAO controlled through the control unit of the UVU device ensures automatic optimal matching of the dynamic range of the output signal with the MFP with the allowable input range of the multi-channel ADC and thereby increases the reliability of detection of thermal objects.

The disadvantage of this optoelectronic device is a significant decrease in performance due to the increased workload of the UVU associated with the need to process a large amount of information to control multichannel UAO.

This disadvantage is enhanced by the possible use of additional devices and information processing modes in such an optical-electronic device, which allow expanding its functional capabilities, namely:

- modes of search, decryption and tracking of the object of observation;

- MFPs operating in several spectral ranges of radiation (both structurally independent and combined in one design), with an increased matrix format (at least 640 × 512 pixels), which significantly increases the reliability of detection of thermal objects (see V.V. Tarasov et al. "Looking" type infrared systems, Moscow, Logos, 2004, pp. 419-421);

- high-precision mechanisms for moving optical components that allow autofocusing, i.e. to compensate for the temperature defocusing of the optical system, due, in particular, to the significant temperature coefficients of the refractive indices of optical materials, especially optical materials that are transparent in the IR range, which can significantly expand the operating temperature range of optoelectronic devices. This is especially true for complex optical systems, for example, systems that make it possible to obtain an image in two or more spectral ranges (see VV Tarasov et al. “Watching” type infrared systems, Moscow, Logos, 2004, p. 97) ;

- high-speed noise-protected input-output devices.

An increase in the speed of an optical-electronic device when using additional devices and information processing modes is possible due to parallel processing of information, i.e. the creation of multiprocessor systems in which to solve each specific range of tasks a separate processor is used that works in parallel with others, for example, as in an astrovizing device (patent RU 2540136, IPC G01C 21/02, published 02.10.2015).

For the timely solution of tasks from each of the processors at certain intervals of time, significant computing power is required. The use of a number of powerful processors in the optoelectronic device leads to increased energy consumption and internal overheating of the equipment, which is often unacceptable, especially for portable compact devices, unmanned aerial vehicles, etc. Moreover, the processors are loaded unevenly in time, i.e. at certain points in time (in general, different for different processors), their computing power is used to the maximum, and at some points it is not completely, or practically not used at all (power consumption decreases slightly).

The objective of the invention is to develop an optical-electronic device, which eliminated the disadvantages of analogues and prototype.

The technical result of the invention is to improve performance while expanding the functionality of the optoelectronic device without a significant increase in power consumption.

The technical result is achieved by the fact that in the optical-electronic device containing the optical system, in the focal plane of which is located a photodetector array (MFP), the outputs of which are connected through a multi-channel analog processing device (UAO) to a multi-channel analog-to-digital converter and then through the multiplexer to the video input of the video processing and control device (CCA), as well as the control unit whose input is connected to the first output of the CCA, and the corresponding outputs are many to the control inputs channel UAO and MFP, and an interface device, the video input of which is connected to the video output of the UVU, and the video output is the video output of the optoelectronic device, while the second output of the UVU is connected to the control input of the multiplexer, according to the present invention, the UVU is made in the form of a multiprocessor device with the mode and dynamic redistribution of computing resources, while the third output of the UVU is connected to the control input of the optical system, the first port of the interface device is connected to the control port Lenia UBV and the second interface device port is a management port opto-electronic device.

The control device contains a video processor, the video input and video output of which are the video input and video output of the control device, the first processor control device (CCU), the first and second outputs of which are the first and second outputs of the control device, and the transmit-receive port is connected to the first port of the video processor, the second control device, the first port of which is the control port of the UVU, the second port is connected to the second port of the video processor, and the output is the third output of the UVU.

The optical system comprises a lens and a device for moving the optical components of the lens, the input of which is the control input of the optical system.

The interface device comprises a video information output unit, the video input and video output of which are respectively the video input and video output of the interface device, and a transceiver, the first and second ports of which are the first and second ports of the interface device, respectively.

The MFP includes N matrix photodetectors, where N≥1, each of which is located in the corresponding focal plane of the optical system, while the control unit is multi-channel.

The invention is illustrated in the drawing (Fig. 1), which shows a functional diagram of the proposed optoelectronic device.

In the drawing, the blocks and nodes of the optoelectronic device are indicated by the following positions:

1 - optical system

2 - matrix photodetector,

3 - multi-channel analog processing device,

4 - multi-channel analog-to-digital Converter,

5 - multiplexer,

6 - device video processing and control,

7 - control unit,

8 - interface device,

9 - video processor,

10 is a first processor control device,

11 is a second processor control device,

12 - lens

13 is a device for moving optical components,

14 - block output video information,

15 - transceiver.

The optical-electronic device contains an optical system 1, in the focal plane of which is located a photodetector array device 2 (MFP), the outputs of which are connected to a multi-channel analog-to-digital converter 4 (ADC) through a multichannel analog-to-digital converter (A / D) 4 and then through a multiplexer 5 to video input device 6 video processing and control (UVU).

The optoelectronic device also comprises a control unit 7 and an interface device 8.

The input of the control unit 7 is connected to the first output of the UVU 6. The second output of the UVU 6 is connected to the control input of the multiplexer 5. The first output of the control unit 7 is connected to the control input of the multi-channel UAO 3, and the second output is connected to the control input of the MFP 2.

The video input of the interface device 8 is connected to the video output of the UVU 6. The video output of the interface device 8 is the video output of the optoelectronic device.

The difference of the proposed optical-electronic device is that the UVU 6 is made in the form of a multiprocessor device with the possibility of intermittent and dynamic redistribution of computing resources, while the third output of the UVU 6 is connected to the control input of the optical system 1, the first port of the interface device 8 is connected to the control port of the UVU 6, and the second port of the interface device 8 is a control port of the optoelectronic device.

UVU 6 contains a video processor 9, the first 10 and second 11 processor control devices (PUU). In the General case, the multiprocessor UVU 6 may contain K processor control devices, where K> 1. Their number depends on the complexity of the tasks performed by the optoelectronic device and the required speed of their solution. The video input and video output of the video processor 9 are respectively the video input and video output of the UVU 6. The first and second outputs of the first processor control device 10 are respectively the first and second outputs of the UVU 6, and the receive-transmit port of the first PUU 10 is connected to the first port of the video processor 9. The first port of the second processor control device 11 is the control port of the control unit 6. The second port of the second control unit 11 is connected to the second port of the video processor 9. The output of the second control unit 11 is the third output of the control unit 6. Optical system 1 rzhit lens 12 and the optical device 13 moving the lens components. The lens 12 for obtaining uniform illumination and uniform image quality on all sensitive elements of the MFP 2 includes, as a rule, several optical components — mirrors, lenses, including defocusing lenses used in calibration mode (not shown in FIG. 1). Compensation of the temperature deviation of the focal plane of the lens 12 from the plane of placement of the sensor array of MFP sensors (not shown in Fig. 1) is ensured by the corresponding mechanical displacement of the optical components of the lens 12 (see VV Tarasov et al. “Looking” type infrared systems, Moscow, Logos, 2004, p. 97) using a device 13 for moving optical components, consisting, for example, of a number of precision mechanical drives (not shown in FIG. 1). The device 13 for moving the optical components also includes a defocusing lens drive, for which, to accelerate the movement, a relay mechanism can be used (not shown in Fig. 1). The input of the optical component transfer device 13 is the control input of the optical system 1. The interface device 8 comprises a video information output unit 14 and a transceiver 15. The video input and video output of the video information output unit 14 are respectively the video input and video output of the interface device 8. Ports I and II of the transceiver 15 are respectively ports I and II of the interface device 8. The MFP 2 includes N matrix photodetectors, where N≥1, each of which is located in the corresponding focal plane of the optical system 1. The control unit 7 is multi-channel.

Optoelectronic device operates as follows.

Let us consider in more detail the main modes of operation of an optoelectronic device.

1. The installation mode of the optimal characteristics of the optoelectronic device

After turning on the optoelectronic device, the radiation from the observed scene using the lens 12 is focused on the sensitive elements of the MFP 2.

Electrical signals from the output of the MFP 2 through the multi-channel UAO 3, multi-channel ADC 4, are fed to the multiplexer 5, from the output of which the video processor 9 of the multi-processor UVU 6 receives a sequence of digital signals of a two-dimensional data array corresponding to the frame, processes them, forms a digital video frame, processes digital video frame, etc.

From the video processor 9, this two-dimensional data array also enters the first PUU 10 and the second PUU 11 of the multiprocessor UVU 6.

The first PUU 10 conducts array processing - determines the dynamic range and the average value of the signals from the sensitive elements of the MFP 2 and, taking into account the results obtained, calculates and sets using the multi-channel control unit 7 the necessary characteristics of the multi-channel UAO 3, which ensure optimal coordination of the dynamic range of the output signal of the MFP 2 photodetector with a valid input range of multi-channel ADC 4, the optimal accumulation time of the MFP 2.

In addition, the first PUU 10 generates constantly generated in all operating modes of the optoelectronic device, the control signals of the multiplexer 5 necessary for its functioning.

The second PUU 11, in addition to receiving and transmitting information (for example, via the CAN channel), polls a number of temperature sensors installed inside the optoelectronic device (not shown in Fig. 1) and, in accordance with the current temperature distribution inside the optoelectronic device, the previously incorporated dependencies, obtained, as a rule, empirically, calculates the position of the optical components of the lens 12 required for the current moment of time, ensuring that the focal position matches the required accuracy plane of the lens 12 with the plane of the sensitive elements of the matrix arrangement of the MFP 2 and installs them in this position by the moving device 13 of optical components (performs autofocus).

Then, the video processor 9 through the second PUU 11 and the transceiver 15 sends an external control device (not shown in Fig. 1) a calibration request.

Further, when a calibration command is received from the external control device through port II of the transceiver 15 of the interface device 8 and port I of the second PUU 11 of the multiprocessor UVU 6, the optoelectronic device goes into calibration mode.

2. Calibration Mode

Calibration is carried out each time after turning on the optoelectronic device or during operation in the event of a significant change in the intensity of the received MFP 2 radiation.

During the calibration, the standard operation of the optoelectronic device is interrupted, so the calibration time should be minimal (about 0.1 ... 0.2 s). In this regard, the calibration process is carried out, as a rule, by the video processor 9, the first PUU 10 and the second PUU 11 of the multiprocessor UVU 6, i.e. the process uses the maximum computing resources.

At the same time, to free up computing resources during this period:

- the video processor 9 does not generate a video image (does not issue a video signal to the interface device 8);

- the first PUU 10 does not analyze the video signal in order to optimize the characteristics of multi-channel UAO 3 and MFP 2 (their previously established characteristics are saved), but only generates the signals necessary for the operation of photodetector devices of the MFP 2 and multiplexer 5;

- the second PUU 11 does not issue and does not receive signals from the transceiver 15, does not analyze the signals from the temperature sensors, does not control the device 13 for moving the optical components (with the exception of the defocusing lens - not shown in Fig. 1).

When calibrating the second PUF 11, using the optical component moving device 13, the lens 12 is fully defocused, for example, by quickly introducing a defocusing lens using the relay mechanism into the optical channel, which ensures that the same optical signals arrive at all the sensitive elements of the MFP 2 photodetector devices.

The first PUC 10 sets the minimum value of the integration time equal to Int 1.

Arrays of data obtained during the integration time Int 1, enter the video processor 9, the first PUU 10 and the second PUU 11, each of these devices working with its own part of the frame, the sizes of which depend on the free computing power of the corresponding device, then sets a different time value integration equal to Int 2 (Int 2> Int 1), which changes the magnitude of the signals at the output of the sensitive elements of the MFP 2.

The data of this array are used together with the data of Int 1 to determine the inhomogeneity in a certain section of the characteristic for each of the sensitive elements of the corresponding photodetector devices of the MFP 2 (calculation of correction factors and correction biases).

The operation is repeated for other values of Int (with values of Int 2 and Int 3, Int 3 and Int 4, etc.), i.e. multi-point correction is carried out. Upon completion of calibration, all calculated correction values are sent to video processor 9 for use in processing data arrays, taking into account the heterogeneity of the sensitive elements of the corresponding photodetector devices of the MFP 2. Next, the optoelectronic device switches to the search mode of the object to be observed.

3. The search mode of the object of observation

In this mode:

- the video processor 9 processes the data arrays arriving at it in real time taking into account defective sensitive elements and heterogeneity of sensitive elements, with overlay of service information on the frame, etc. and provides video information through the video output unit 14;

- the first PUU 10 processes the data arrays received through the video processor 9 and, using the control unit 7, constantly optimizes the characteristics of the multi-channel UAO 3, controls the operation of the MFP 2 and multiplexer 5, with a significant change in the input data, for example, a significant change in the intensity of the received MFP 2 radiation issues a calibration request through video processor 9, or conducts it automatically;

- the second PUU 11 provides reception, processing and transmission of service information through the transceiver 15, processes data from temperature sensors of the optoelectronic device, and controls the device 13 for moving optical components to perform temperature autofocus.

In addition, the first PUK 10 and the second PUU 11 constantly analyze the incoming data arrays for the presence of monitoring objects with certain signs (sizes, contrast), and each device analyzes its own group of signs.

When such objects of observation appear, the second launcher 11 passes through the transceiver 15 the corresponding information to the external control device and switches (automatically or by command from the external control device) to the decoding mode of the image of the observation object, while in the channel there is a wide and narrow field of view established by moving the optical components of the lens 12 of the optical system 1 a narrow field of view.

4. The mode of decoding the image of the object of observation

Decryption of the image of the object of observation is carried out according to its decryption features, for example:

- by the nature of the distribution of radiation temperature over the area of the object of observation, i.e. by indicators of texture (“roughness”, elongation, contrast, regularity, “roughness”, etc.);

- the shape of the object of observation (compactness, straightforwardness, sharpness, collinearity, symmetry, etc.).

The calculation of these indicators is carried out according to certain algorithms.

When the optoelectronic device performs this task, the decryption time is a critical value and, as a rule, should not exceed 0.2 ... 0.5 s (depending on the purpose of the optoelectronic device).

The calculation of indicators is performed by the video processor 9, the first PUU 10 and the second PUU 11 (each calculates its own specific signs). At the time of calculations, their computing resources are directed, mainly, to the solution of these problems, therefore during these calculations:

- the first PUU 10 does not correct the operation of the MFP 2 and multi-channel UAO 3 (previously established modes are saved);

- the second PUU 11 does not control the movement of the optical components and does not work with the transceiver 15;

- the video processor 9 operates in a mode with a lower frame rate.

The values calculated by the first ПУУ 10 and the second ПУУ 11 are transferred to the video processor 9, where the totality of all the indicators of the observation object (object metrics) calculated by them is compared with the metrics of the known (of interest) observation objects stored in the memory of the video processor 9.

When coinciding with a certain accuracy with any of the metrics, the video processor 9 detects the presence of the object of observation corresponding to this metric in a known field of view of the optoelectronic device, provides information about it through the second PUC 11 and the transceiver 15 to the external control device and, when upon receipt of the corresponding command, it goes into the tracking mode of the monitoring object.

5. The mode of tracking the object of observation

In this mode:

- the video processor 9 receives and processes data arrays corresponding to the frame (or, as a rule, part of the frame) with an increased frequency, for which, to accelerate the operation of the MFP 2, it switches to the "window" mode when data is output only from a certain selected area ("window" ) the photodetector MFP 2 and determines the position of the object of observation on the matrix of the MFP 2;

- the second PUU 11 receives data at a higher frequency through the transceiver 15 from an external control device, including data on the position (movement during tracking) of the optical axis of the optoelectronic device, issues via the transceiver 15 of the interface device 8 to the external the control device received from the video processor 9 current data on the position of the object of observation on the matrices of the MFP 2, calculates the estimated path of movement of the object of observation;

- the first launcher 10 through the control unit 7 controls the "windows" of the photodetector MFP 2 taking into account the current position of the object of observation and data on the expected trajectory of its movement, coming through the video processor 9 from the second launcher 11 to the first launcher 10. In addition, depending on parameters (sizes) of the "window" the first PUU 10 adjusts the control of the multiplexer 5.

To free computing resources in the tracking mode of the monitoring object:

- the first PUU 10 does not analyze video signals to optimize the characteristics of multi-channel UAO 3 and MFP 2, preserving their previously established characteristics;

- the second PUU 11 does not analyze the signals from the temperature sensors, does not provide control signals to the device 13 for moving the optical components.

In the main modes discussed above, all operations should be performed as quickly as possible (with full use of computing resources).

The above distribution of computing resources in each of the main modes (mode distribution) is indicative and in the real case is not always optimal, because time to complete operations may differ from expected.

For example, the processing time of the data array corresponding to the frame will be different for MFPs with a large (several thousand) and a small (unit) number of defective sensitive elements, the decryption time of the image will depend on the observation conditions, image quality, type of observed object, etc.

In this regard, the multiprocessor UVU 6 uses the dynamic redistribution of computing resources of its components, which is carried out automatically according to certain algorithms.

For example, it is possible to redistribute computing resources between the video processor 9, the first PUU 10 and the second PUU 11 when processing data arrays in the calibration mode, redistribute the analyzed characteristics between the first PUU 10 and the second PUU 11 in the search mode of the object to be observed, redistribute the calculated indicators between the video processor 9, the first PUU 10 and the second PUU 11 in the decryption mode of the image, etc. Such a redistribution of computing resources allows for maximum load, and hence speed, at any time of all the components of multiprocessor UVU 6.

Thus, the use of the present invention will improve performance while expanding the functionality of the optoelectronic device without a significant increase in power consumption. The specified technical result is achieved due to the fact that the video processing and control device is made in the form of a multiprocessor UVU 6 on economical processors of low power with the possibility of intermittent and dynamic redistribution of computing resources, which ensures maximum load of all processors and high total performance under different operating modes of the optoelectronic device without a significant increase in power consumption, which is important when using optoelectronic th unit in the composition, such as portable compact devices or unmanned aerial vehicles.

Claims (5)

1. An optical-electronic device containing an optical system in the focal plane of which is located a photodetector array (MFP), the outputs of which are connected to a multi-channel analog-to-digital converter through a multichannel analog processing device (UAO) and then to the video input of the video processing and control device through a multiplexer (UVU), as well as a control unit, the input of which is connected to the first output of the UVU, and the corresponding outputs are connected to the control inputs of a multi-channel UAO and MFP, and devices interface, the video input of which is connected to the video output of the UVU, and the video output is the video output of the optoelectronic device, while the second output of the UVU is connected to the control input of the multiplexer, characterized in that the UVU is designed as a multiprocessor device with the possibility of intermittent and dynamic redistribution of computing resources, while the third output of the UVU is connected to the control input of the optical system, the first port of the interface device is connected to the control port of the UVU, and the second port of the device and a control port opto-electronic device. 2. The optoelectronic device according to claim 1, characterized in that the UVA comprises a video processor, the video input and video output of which are respectively the video input and video output of the UVU, the first processor control device (PUU), the first and second outputs of which are the first and second outputs of the UVU, respectively and the transmit-receive port is connected to the first port of the video processor, the second launcher, the first port of which is the control port of the ILC, the second port is connected to the second port of the video processor, and the output is the third output of the IED. 3. The optical-electronic device according to claim 1, characterized in that the optical system comprises a lens and a device for moving the optical components of the lens, the input of which is the control input of the optical system. 4. The optical-electronic device according to claim 1, characterized in that the interface device comprises a video information output unit, the video input and video output of which are respectively the video input and video output of the interface device, and a transceiver, the first and second ports of which are the first and second ports, respectively interface devices. 5. The optical-electronic device according to claim 1, characterized in that the MFP includes N matrix photodetector devices, where N≥1, each of which is located in the corresponding focal plane of the optical system, while the control unit is multi-channel.

Images