RU2707714C1 - Device for automatic acquisition and processing of images - Google Patents

Abstract

FIELD: optoelectronic instrument making.

SUBSTANCE: invention relates to the field of optoelectronic instrument making and can be used as a hardware-software system for automatic acquisition and processing of images with a submatrix photodetector to increase quality of formed image from RAW image into digital form, fully adapted for further image processing, incl. in formats of static images with losses, as well as any moving images. Hardware-software system for automatic acquisition and processing of images is based on a three-level microprocessor architecture and comprises an image output module, a signal processing module and a photodetector. Quality of the digital image is determined by three factors: the action of the lens, the number of pixels and the power of the CCD matrix, as well as the power of the data processing device.

EFFECT: technical result consists in expansion of functional capabilities due to use of hardware-software system for automatic acquisition and processing of images.

1 cl, 3 dwg

Description

The invention relates to the field of optical-electronic instrumentation and can be used as a hardware-software complex (AIC) for automatic acquisition and processing of images with a submatric photodetector (FPU) to improve the quality of image formation from RAW image ¹ ( ¹ RAW (in English) raw ") is a file format that captures all the image parameters recorded by the camera’s matrix. RAW format does not compress information as it does in JPEG, image information is compressed and partially lost If necessary, you can get a high-quality image, which allows you to eliminate image problems that are not fixed on JPEG images.) in a digital form that is fully adapted for further image processing, including in formats of static images with losses, as well as any moving images.

A device for imaging with correction of signals is known [French patent No. 2720175, IPC. G06F 17/10, G06T 5/00, publ. 04/20/1994], containing an optical circuit, a scanning device and a submatricial FPU with signal accumulation and reading circuits, the outputs of which are connected to the corresponding inputs of the signal processing unit, one of the outputs of which is connected to adjustable sources of reference radiation, optically coupled through a scanning device with FPU, and a system synchronization circuit, electrically connected to a scanning device and a signal processing unit. The optical design may include input and projection lenses. The signal processing unit for performing signal correction includes a series-connected electrically connected analog-to-digital converter (ADC), a filter, a multiplier, an adder, as well as a computing device, selection tools, and random access memory (RAM). The optical circuit and scanning system forms a thermal image on the sensitive elements of the FPU. The analog video signal from the FPU is fed to the signal processing unit, where the ADC is converted to digital form. Next, the digital signal passes through the filter, multiplier and adder in order to correct the error of signals from the FPU caused by the influence of the previous analog sample of order n-1 on the subsequent sample of order n and the error of the output signals caused by the spread in the characteristics of the sensitive elements of the FPU. The multiplying device and the adder performs a mathematical linear transformation of the video signal from the FPU, using one or more sets of correction factors previously calculated by the computing device for each sensitive element of the FPU. The coefficients are calculated based on the response of each sensitive element of the FPU to exposure by two different fluxes from the source of reference radiation, the values of which are selected in the dynamic range of signals from the scene observed by the device. The selection tools available in the signal processing unit select a specific set of coefficients for each sensitive element of the FPU depending on the signal value at the ADC output.

The disadvantage of this device is its dependence on working conditions and consists in the need to complicate the circuit in the case of complicating the task at hand, which limits the universality of the device, limiting the achievable accuracy of the calculations to the achievable accuracy of reading the source data, and leads to the appearance of unacceptably large calculation errors, underdevelopment of the human-device interface , poor quality of automatic acquisition and processing of images and the speed of reading pixels RAW images.

Closest to the alleged invention (prototype) in technical essence and the achieved effect is an image forming apparatus [Image forming apparatus No. 87854 IPC H04N 5/33, Bull. No. 29 Publ. 20.10.2009], which contains a serially mounted optically conjugated input lens, a scanning device, a projection lens and a photodetector, as well as a signal processing unit, the first output of which is the output of the image forming device, the second output of the signal processing unit is connected to at least one an adjustable reference radiation source optically coupled through a scanning device to a photodetector, and a synchronization circuit electrically connected to the scanning device the triple and the synchronizing input of the signal processing unit, as well as the interface unit connected between the output of the photodetector device and the input of the signal processing unit, and a control circuit for the operation mode of the photodetector device, the input of which is connected to the third output of the signal processing unit, and the output to the control input of the photodetector.

The disadvantage of this device is its dependence on operating conditions and consists in the need to complicate the circuit in the case of complicating the task at hand, which limits the versatility of the device, limiting the achievable accuracy of the calculations to the achievable accuracy of reading the source data, and leads to the appearance of unacceptably large calculation errors, underdeveloped human-device interface , poor quality of automatic acquisition and processing of images and the speed of reading pixels RAW images.

The objective of the device for automatic acquisition and processing of images is to expand the functionality, due to:

- the use of the agro-industrial complex for automatic acquisition and processing of images, both in stationary conditions and during its operation according to the statistical characteristics of the measured parameters, i.e. universality of the complex;

- a developed human-device interface, which allows the operator to transmit an image that is comfortable for perception under various illumination of the observed scene, as well as in conditions of interference and local flare;

- possessing sufficient computation speed to ensure processing of the resulting image with a resolution of at least 576 × 768 pixels (typical size of a television frame) and its output to a display or external devices in real time;

- improving the quality of automatic image acquisition and processing and the speed of reading RAW pixels of images obtained by the video matrix, which are damaged by noise and interference of various origins (video matrix noise, grain noise and transmission channel errors). To reduce visual noise and to remove high-frequency components from the image, filters formed on the basis of computational mathematics are used to analyze the measured parameters at the output of the pixels.

The technical result of the proposal is the organization of human-machine communication to obtain information on three-level processing of RAW images obtained by the video matrix and damaged by noise and interference of various origins by the operating personnel in the agro-industrial complex, which allows to improve the speed and quality of the required format obtained after digital processing of the image, fully adapted for further image processing, including in the formats of static images with losses, as well as any mobile images.

The solution to the problem is achieved by using an automatic image acquisition and processing device that contains a series-connected photodetector and a signal processing module, and the photodetector contains an optically paired input lens of the lens, a scanning device, and an image output module is additionally introduced into the device, the photodetector containing an auto-adjustable aperture with an aperture control actuator at the control input, the lens of the lens is made as tr nsfokator, the control input of which is connected through the device for controlling the focal length to the first control input of the photodetector, as well as a sequentially matched filter for RGB decomposition of the light flux from the object in the Bayer arrangement, a video matrix and its control circuit, the control input and information inputs of which are through the interface controller, respectively connected to the information inputs and to the second control input of the photodetector, and the third control input of the photodetector and connected to the control input of the aperture control drive, the RGB outputs in the Bayer layout of the video matrix control circuit are connected through the outputs of the photodetector to the inputs of the signal processing module, which contains an analog-to-digital converter, a synchronization unit, a data register block, a digital signal processor with a memory unit on the input and input / output register, an address multiplexer and a control unit, and inputs of an analog-to-digital converter and inputs are connected to the inputs of the signal processing module synchronization unit, the information outputs of the analog-to-digital converter are connected to the first information inputs of the data register block, the second information inputs of which are connected to the first information inputs of the signal processing module, and the outputs of the data register block are connected through the data inputs of the input blocks of the storage devices to the digital signal processor, and through the address inputs of the blocks of storage devices the outputs of the address multiplexer are connected, the information inputs of which are connected to the second inform to the input inputs of the signal processing module, the outputs of the synchronization unit are connected to the inputs of the control unit, the first control output of which is connected to the first control input, the second control with the second input, and the third with the third control input of the photodetector, the information outputs of the control unit are connected to the information outputs of the processing module signals that are connected to the information inputs of the photodetector, the fourth output of the control unit is connected to the gating input of an analog-digital pre of the educator, the fifth control unit control outputs are connected to the control inputs of the data register unit, the sixth and seventh to the first and second control unit inputs of the storage device, the eighth control output of the control unit is connected to the control input of the input / output register of the digital signal processor, the bi-directional connector of which is connected with the output of the signal processing module, which is connected to the bi-directional connector of the image output module and through the parallel exchange unit and the internal bus with IBM PC / AT m After outputting the image, the first control output of the parallel exchange unit of the image output module is connected to the output of the image output module, which is connected to the control input of the signal processing module connected to the control input of the control unit, the Ready output of which is connected to the first output of the signal processing module, connected with the first control input of the image output module, which is connected to the first input of the parallel exchange unit, and the control output of the address multiplexer is connected to the second output the signal processing module connected to the second control input of the image output unit, which is connected to the second control input of the parallel exchange unit of the image output unit, the reset outputs of the "Reset STB" columns and the "Reset СTP" columns of the control unit are connected to the corresponding inputs of the address multiplexer, to to which the outputs of setting the switching frequencies of columns F _STB and rows F _STR of the control unit are connected.

The difference between the proposed architecture and existing analogues is the use of a three-level system with a control unit (BU) in the form of a control computer (UVM) in conjunction with a digital signal processor (DSP) based on, for example, FPGA ² ( ² Programmable Logic Integrated Circuit (FPGA) Programmable logic device, PLD) - an electronic component (integrated circuit) used to create configurable digital electronic circuits.Unlike conventional digital circuits, the FPGA logic is not determined during manufacture, but is set by programming (design) The programmer and IDE (debugging environment) are used for programming, allowing you to set the desired structure of a digital device in the form of a circuit diagram or a program in special hardware description languages: Verilog, VHDL, AHDL, etc.) in two orthogonal image coordinates . This allows you to build a flexible solution software and hardware at the same time.

In FIG. 1 shows a functional diagram of the AIC of automatic acquisition and processing of images, FIG. 2 shows an example of the initial setup of the auto iris by a video camera according to a test table consisting of strips of different brightness, and in FIG. Figure 3 shows an example of the implementation of phase autofocus in accordance with US pat 5589909 fig 2.png, the numbering in which corresponds to the numbering of the original.

It should be noted that the quality of a digital image is determined by three factors: the action of the lens, the number of pixels and the power of the CCD, as well as the power of the data processing device.

The AIC of automatic image acquisition and processing contains a computing and measuring unit, which is made according to a three-level microprocessor architecture, and contains a lower level 1 - photodetector, middle level 2 - signal processing module and upper level 3 - image output module, and photodetector 1 includes aperture 4, aperture control drive (PUD) 5, lens (zoom) (O (TF)) 6, a device for controlling the focal length (URFR) 7, a light filter (SF) 8, a video sensor I (Vm) 9, control scheme of the video matrix SUVm (10), video matrix interface controller (KIVm) 11, the signal processing module 2 includes an analog-to-digital converter (ADC) 12, a synchronization unit (BS) 13, a data register block ( BRD) 14, a digital signal processor (DSP) (DSP) 15, which, in turn, contains a buffer memory (BZU) 16 and the input / output register P BB / B 17, and then the signal processing module (middle level) 2 also contains an address multiplexer (MPA) (MUXA) 18 and a control unit (CU) 19, and the image output module 3 contains a parallel exchange unit (BPO interface) 20 and is made in the form of a general purpose computer 21.

The photodetector (lower level) 1 is made in the form of a RAW imaging device, read by a video matrix with a Bayer layout, has one color component in each pixel, extracting a value for each color component should result in pixel layouts with its subsequent scanning and converts the incoming to the lens Instrument luminous flux into electrical signals. The camera’s image processing device converts information from an analog image recorded on a CCD-matrix ³ , ( ³ CCD-matrix (Charge-coupled device) or CCD-matrix (CCD, “charge-coupled device”) - a specialized analog integrated circuit consisting of photosensitive photodiodes using charge-coupled CCD technology.) into digital data and performs various types of data processing to create an image. Functional diagram of the lower level 1 includes:

- перегородка) (Д) 4 [Гонта А. Системы охранного телевидения. Методическое пособие. - М.: НИЦ "Охрана". - 2008. - 222 с.; Гонта А. Практическое пособие по CCTV. - М.: Спецкнига. - 2006. - 80 с.; Козлов А.А., Козлов С.A. CCTV. Охранное видеонаблюдение: метод. указания к лаб. работам. - Владимир: Изд-во Владим. гос. ун-та. - 2008. - 36 с.], позволяющую в оптических приборах регулировать относительное отверстие объектива изменением диаметра проходящих через него пучков света. Диафрагма объектива представляет непрозрачную перегородку с круглым отверстием переменного диаметра, центр которого совпадает с оптической осью. Диафрагма поддерживает оптимальную яркость на элементе видеоматрицы и обеспечивает его резкость, четкость и правильность экспонирования для достижения хорошей контрастности и разрешения. Диафрагма применяется также для контроля глубины резкости. Регулировка диаметра отверстия выполняется тремя основными способами:- diaphragm (from Greek

- partition) (D) 4 [Gonta A. CCTV systems. Toolkit. - M.: Research Center "Protection". - 2008. - 222 p .; Gonta A. CCTV Practical Guide. - M .: Special book. - 2006. - 80 s .; Kozlov A.A., Kozlov S.A. CCTV. Security video surveillance: method. directions to the lab. work. - Vladimir: Publishing house Vladim. state un-that. - 2008. - 36 p.], Which allows optical instruments to adjust the relative aperture of the lens by changing the diameter of the light beams passing through it. The lens aperture is an opaque baffle with a round hole of variable diameter, the center of which coincides with the optical axis. The diaphragm maintains optimal brightness on the element of the video matrix and ensures its sharpness, sharpness and correct exposure to achieve good contrast and resolution. Aperture is also used to control the depth of field. The hole diameter is adjusted in three main ways:

револьверная диафрагма представляет собой поворотный диск с набором отверстий разного диаметра.

revolving diaphragm is a rotary disk with a set of holes of different diameters.

вставная диафрагма представляет собой набор пластин с разными отверстиями, вставляющихся в прорезь оправы объектива между линзами.

insertable diaphragm is a set of plates with different holes inserted into the slot of the lens barrel between the lenses.

ирисовая диафрагма позволяет бесступенчато регулировать относительное отверстие и имеет самую компактную конструкцию.

The iris diaphragm allows stepless adjustment of the relative aperture and has the most compact design.

- diaphragm control drive (PUD) 5 [Gonta A. CCTV systems. Toolkit. - M.: Research Center "Protection". - 2008. - 222 p .; Gonta A. CCTV Practical Guide. - M .: Special book. - 2006. - 80 s .; Kozlov A.A., Kozlov S.A. CCTV. Security video surveillance: method. directions to the lab. work. - Vladimir: Publishing house Vladim. state un-that. - 2008. - 36 p.] May be fixed or adjustable. Auto-adjusting aperture ⁴ ( ⁴ Aperture is an opaque obstruction with a hole located in the path of the light flux.) (ARD) is an automatic adjustment of the aperture to control the amount of light entering the matrix. The diaphragm controls the amount of light entering the CCD - camera matrix, and affects the final image. If the aperture is small, then a large amount of light enters the matrix, which is especially important for obtaining high-quality images in low light. If the aperture is large, then the amount of light entering the matrix decreases, which prevents the illuminated frame. The aperture value adjusts the brightness of the image and changes depending on the brightness.

Manual aperture adjustment is not suitable for video surveillance systems, so Auto-Iris (auto iris technology) is installed in these cameras. There are three options for automatic iris adjustment:

контроль по постоянному току (Direct Drive) (размещается в камере);

direct current control (Direct Drive) (located in the camera);

контроль по видеосигналу (Video Drive) (размещается в объективе с использованием видеолинз). Регулировка Video Drive позволяет точнее настроить диафрагму и срабатывает даже при попадании прямых солнечных лучей. Технология Direct Drive дороже и применяется не всегда;

control by video signal (Video Drive) (placed in the lens using video lenses). The Video Drive adjustment allows you to fine-tune the aperture and works even in direct sunlight. Direct Drive technology is more expensive and not always applied;

автоматический контроль и управление диафрагмой (P-Iris) с высокой точностью, причем в комплект входят объектив P-Iris и специальное программное обеспечение для оптимизации качества изображения.

automatic control and iris control (P-Iris) with high accuracy, and the kit includes a P-Iris lens and special software to optimize image quality.

P-Iris lenses are equipped with a stepper motor that controls the iris in digital mode. This allows you to optimize the opening of the diaphragm under all lighting conditions, and as a result, more contrast and sharp images with better resolution and depth of field are obtained. In bright light, a diaphragm with P-Iris control technology restricts hole closure to avoid blurring (diffraction) caused by severe narrowing of the iris hole.

The diaphragm consists of petals, the number of which can be from 3 to 20. The more the petals in the diaphragm, the more a uniformly illuminated light spot is created on the CCD. The diaphragm affects:

аберрацию - чем меньше отверстие диафрагмы, тем ниже уровень аберраций и выше разрешение, но до определенного предела, далее разрешение падает из-за влияния дифракции. Это вызывает размытость изображения, получаемого от видеокамеры мегапиксельных систем с малым размером ячейки матрицы;

aberration - the smaller the aperture, the lower the level of aberration and the higher the resolution, but to a certain limit, then the resolution decreases due to the influence of diffraction. This causes blurring of the image received from the video camera of megapixel systems with a small matrix cell size;

глубину резкости - чем меньше отверстие, тем больше глубина резкости.

depth of field - the smaller the hole, the greater the depth of field.

Auto iris lenses control the luminous flux due to the signals coming from the camcorder. With Direct Drive, the diagram for deciding on the position of the diaphragm is in the camcorder, and in the lens there is only a motor as an actuator.

With Video Drive, the analysis of the video signal and the control of the aperture motor are carried out by a special device located in the lens. To control the iris of the video signal, there are two control elements "Level" and "ALC". The “Level” adjustment is used to adjust the operating mode of the electronic circuitry of the lens in real light. Turning the “Level” knob changes the aperture value. On the monitor, a change in the position of the Level control is perceived as a change in image brightness. The “ALC” control is used to eliminate backlight in high-contrast scenes and has two control areas: average values (indicated by “A”) and peak areas (indicated by “P”).

- lens (zoom ⁵ ( ⁵ Zoom (zoom lens or zoom lens from English. zoom) - a variable focal length lens that can change in steps or smoothly.)) (O (TF)) 6 [Gonta A. CCTV systems. Toolkit. - M.: Research Center "Protection". - 2008. - 222 p .; Gonta A. CCTV Practical Guide. - M .: Special book. - 2006. - 80 s .; Kozlov A.A., Kozlov S.A. CCTV. Security video surveillance: method. directions to the lab. work. - Vladimir: Publishing house Vladim. state un-that. - 2008. - 36 p.], Focusing the image of the object on the matrix, to which the requirements of high resolution are imposed due to the small size of the decomposition element of the charge-coupled device (CCD);

- a device for adjusting the focal length (URFR) 7 [Gonta A. CCTV systems. Toolkit. - M.: Research Center "Protection". - 2008. - 222 p .; Gonta A. CCTV Practical Guide. - M .: Special book. - 2006. - 80 s .; Kozlov A.A., Kozlov S.A. CCTV. Security video surveillance: method. directions to the lab. work. - Vladimir: Publishing house Vladim. state un-that. - 2008. - 36 p.] Has a multi-lens design (zoom lens, zoom), and the number of optical elements in them can exceed 20. Autofocus drives use stepper motors located in the camera body.

Autofocus (AF) is an adaptive system for automatically focusing the camera lens on one or more objects and consists of a sensor, a control system and a drive that moves the lens barrel or its individual lenses. With autofocus, the exact distance from the focal plane to the subject is determined; by the method for determining this parameter, autofocus systems are divided into active and passive. In active systems, the distance the lens focuses is calculated using echolocation or triangulation. Active systems are independent of lighting conditions and can be induced in complete darkness to objects without contrasting details, but precise focusing is impossible in them. Passive autofocus is based on the analysis of light beams entering the camera and does not radiate anything into the surrounding space. These technologies are called phase detection autofocus. In this case, individual lenses move inside the frame and relative to each other, changing the focal length of the entire system at a constant position of the focal plane. The focal length adjusting device and the iris control drive can be implemented on the IEEE 1394 ⁶ controller ( ⁶ IEEE 1394 (FireWire, i-Link) is a high-speed serial bus designed to exchange digital information between a computer and other electronic devices with a serial interface. IEEE-1394 provides users with high-speed access to various input devices, including video cameras.) an interface for interconnecting the camera with external devices;

- the light filter (SF) 8 divides the light flux into spectral components - red (R), green (G) and blue (B) - according to the number of image converters on the video matrix;

- the video matrix of the converter (Vm) 9 is a device that converts a light signal into electrical pulses with image sampling. According to the methods of reading and analysis stored in the charge sensors, there are linear, interlaced reading, full-frame CCD matrices.

Currently, digital cameras use charge-coupled CCD sensors [Nosov Yu.R. Charge coupled devices. - M.: Knowledge, Radio electronics and communications. 1989 No. 12. - 64 p.] And active pixel sensors [Ilyin A.A., Ovchinnikov A.M., Ovchinnikov M.Yu. The principle of operation and the device active-pixel sensors / Preprints IPM them. M.V. Keldysh. - 2003. - 25 pp.] CMOS APS (CMOS-Complementary Metal Oxyde Semiconductor, APS-Active Pixel Sensor), based on sensors created on a complementary ⁷ ( ⁷ Complementary structure is a chip built according to a scheme in which p-channel and n-channel field effect transistors are used in pairs and are mutually complementary.) metal-oxide-semiconductor (CMOS) structures.

Each pixel of an active-pixel image sensor, in contrast to CCD arrays, contains not only a photodetector element (photodiode or photo-gate (photogate)), but also an active transistor circuit (“strapping”) for reading the signal from the pixel. CMOS technology allows you to integrate the entire imaging system, including an array of photosensitive elements and control circuits, on a single silicon crystal.

- The control scheme video matrix SUVm 10. The video matrix is a medium-integration chip consisting of four main sections - accumulation, memory, lower single-line register and output device.

The CCD output device is designed to convert charge packets into video signal voltage.

- Video Matrix Interface Controller (KIVm) 11.

The signal processing module (intermediate level) 2 AIC of automatic acquisition and processing of images implements the function of input-output images and converts signals received from the image, suitable for the operator. It includes

1. An analog-to-digital converter (ADC) 12 is required to obtain a digital image signal for subsequent processing.

2. Synchronization block (BS) 13 - is intended to generate pulse sequences of time-coordinated operation of nodes and devices in a subsystem or system in hardware.

3. The data register block (DBD) 14 - are used to store intermediate calculations.

4. The digital signal processor (DSP) (DSP) 15 is implemented on a specialized computer and has the functionality and design features that can be used to effectively solve the problems of digital signal processing as part of a data processing complex.

Like the central processor, the digital signal processor is the “brain” of the agro-industrial complex for automatic acquisition and processing of images. The digital signal processor 15 contains a buffer memory (BZU) 16, which is a dynamic memory of 256 KB. The main task of the digital signal processor is to process the signals of three image colors coming from the ADC 13, which provides high resolution, wide dynamic range, color fidelity and high reliability of the camera displayed on the monitor screen, i.e. solving the inverse problem, relieving the central processor of the top-level agro-industrial complex of automatic acquisition and image processing from this duty. For input / output of information at the output of the DSP 15 there is an input / output register P BB / B 17.

Tasks of the DSP 15 (DSP): control of the video matrix 9, control of the AIC of automatic acquisition and processing of images in general, the formation of the service frame, and image processing can be performed by FPGA.

5. The address multiplexer (MPA) (MUXA) 18 is a combined digital device that provides sequential transmission of several input signals to one output. It allows you to transmit (switch) the signal from the desired input to output, in this case, the choice of the required input is implemented by a certain combination of control signals.

6. The control unit (CU) 19 is used to coordinate all camera units and the formation of control pulses. BU 19 is implemented on the basis of a host computer and is characterized by a set of real-time operation capabilities of both the input-output subsystem and the properties of the operating system. These are specially designed PCs that are compatible with standard architectural and software, but differ in their design. So to control the SUVm 10 photodetector device of the lower level 1 of the agro-industrial complex of automatic image acquisition and processing, KIVm 11 is used, and the PUD 5 and URFR 7 servo motor are also controlled by the corresponding controller. The structure of the control unit includes (in Fig. 1, these elements are not allocated from the structure of the control unit):

Оперативное запоминающее устройство.

Random access memory

Перепрограммируемое запоминающее устройство. В нем может храниться следующая информация:

Reprogrammable memory device. It may store the following information:

- camera operating system, ensuring the operation of the device;

- system settings for image parameters and network settings;

- image analysis software.

Image Output Module (upper level) 3

- contains a parallel exchange unit (BPO interface) 20, which implements the OST 11.305.903-80 and GOST 26765.51-86 standards, which determine the exchange rate of up to 5.6 MB / s with a data bit length of 8 or 16 bits and an address bit capacity of 16 up to 24 bits, as well as the procedures for exchanging the processor and peripheral modules inside the computer using the multiplex address and data bus, and is focused on use in small and medium-capacity systems. Communication between two devices connected to the interface is carried out according to the principle of "control" - "controlled" (active - passive). Only one device is active at a time. The active (control) device manages the circulation cycles, serves interrupts if necessary and carries out arbitration. With a parallel interface - each bit of the transmitted group uses its own signal line (with binary representation) and all bits of the group are transmitted in one time slot, providing fast data transfer.

- made in the form of a general-purpose computer 21, which allows you to solve most of the emerging technical problems, including the tasks of pairing with a computer a wide range of external devices and sensors, i.e. image transmission in the form of electrical signals to external devices, for example, to a display. All video devices have a flat screen for a two-dimensional (flat) image. General-purpose computers include a standard set of components:

- The central processor and arithmetic coprocessor, performing information conversion to generate a control action, organizing the interaction of all computer elements, and consisting of arithmetic, logic, control and random access memory (RAM).

- High-speed storage device for collecting and converting information about the current state of the generated and processed image and the current disturbances.

- External storage devices of various nature, storing image information and current tasks received in the external memory from the control panel.

- Multimedia (graphic and sound) devices are PC devices that are used to work with sound, graphic and video information.

- A user terminal (display, keyboard, mouse, etc.), necessary for organizing a person’s connection with the machine, designed for operational personnel to receive information about the process of image formation and processing, correction of the control algorithm, and information input-output means, including various recording, indicating and storage devices (recording on tapes and floppy disks), on-screen remotes - displays, printers, plotters.

- Network support tools give the OS the ability to provide local resources (disk space, printers, etc.) and access the resources of other computers over the network.

- Ability to connect additional interface devices for direct connections with the image forming and processing processor, transmitting control actions in the form of controllers connected to a computer bus.

- Ability to install a variety of software.

The last two properties determine the openness of computer architecture. The openness of architecture is an indispensable condition for universality with widespread use. The universality of computers as a means of solving a wide variety of problems, the huge amounts of information processed and stored in computers, powerful algorithmic capabilities have made computers an effective tool for solving modern control problems.

An automatic image acquisition and processing device may include modules that perform image processing algorithms adapted to low light, and modules that use image processing algorithms in high brightness conditions, for which the same computing resources can be used.

In conclusion of the analysis of the AIC of automatic acquisition and processing of images, it should be noted that the RAW image obtained by Vm 9 is damaged by noise and interference of various origins (video sensor noise, granular noise of photo materials and errors in the transmission channel). Their influence can be minimized using classical methods of statistical filtering. The noise of the video matrix or errors in the transmission channel usually appear on the image as scattered changes of isolated elements that do not have spatial correlation. Distorted elements noticeably differ from neighboring elements, which served as the basis for many algorithms that provide noise reduction. Since noise is spatially decorrelated, its spectrum contains higher spatial frequencies than the spectrum of a conventional image. Simple low-pass spatial filtering serves as an effective means of smoothing out noise.

Such filters are used both to reduce visual noise and to remove high-frequency components from the image. The frequency range of low-pass filters is determined by the size and coefficients of the filter mask. As the low-pass filter mask passes (scans) through the image area, the new value of the elements to be converted is calculated as the sum of the mask coefficients multiplied by the values of the image elements in the adjoining area. Low-pass filtering is used to improve image sharpness.

The devices, the analysis of which is given earlier, form a 3-level structure of the agro-industrial complex of automatic acquisition and processing of images ⁸ , ( ⁸ Image processing - any form of information processing for which the input data is represented by an image, for example, photographs or video frames. Image processing can be performed as for receiving images at the output, as well as for obtaining other information.In addition to static two-dimensional images, it is also necessary to process images that change with time.) shown in FIG. 1, and works as follows.

The light emitted or reflected by the object, the optical system of the APC module 1 of automatic acquisition and processing of images is projected onto the image section of the object in the VM 9 plane. A flat image is a function of the dependence of color on coordinates taking discrete values. Two main approaches to image formation and storage are known: raster graphics ⁹ , ( ⁹ Raster image - an image representing a grid of pixels - colored dots (usually rectangular) on the monitor and other display devices.) Vector graphics ¹⁰ ( ¹⁰ Vector graphics - a way of representing objects and images (description format) in computer graphics, based on a mathematical description of elementary geometric objects, usually called primitives.) and their combinations. Today, image editing is carried out on a computer by digital raster editors.

At the initial stage of the work of the agro-industrial complex, automatic acquisition and processing of images by command from the 3rd level through the 2nd level control unit, “ALC” and “Level” are used to configure the initial state D 4 for normal operation of O (TF) 6 in the entire range of illumination and “Back focus” of the lens of the 1st level of the agro-industrial complex of automatic acquisition and processing of images, which determines the degree of detail and depth of field of the image.

I. For the initial setup of the auto iris, a test table consisting of strips of different brightness is set in front of the camcorder. A monitor is connected to the camcorder and the test chart is displayed on the screen in the form of six gradations of brightness. An oscilloscope is connected to the additional output of the monitor and set it to display one line. On the screen of the oscilloscope displays six evenly spaced steps (Fig. 2). The lower step corresponds to the black bar on the test table, and the upper to the white bar. The steps between them convey intermediate gradations of brightness.

An increase in brightness at the input of the camera is also present in its output signal in the form of an increased amplitude of the white strip relative to the “white level” and the PUD 5 reduces the hole D 4, reducing the amplitude of the white strip, returning it to the “white level” of the video signal.

But with a decrease in the amplitude of the white strip, the levels of all other gradations of brightness are proportionally reduced. As a result, instead of six gradations with a uniform change in brightness, three gradations are obtained on the screen, and most of the screen turns black. This case is typical when the camera is in high-contrast scenes, when the object of observation in the foreground represents a dark spot, and the background is a brightly lit background. By manipulating the “Level” and “ALC” controls, it is possible to preserve most of the initial gradations of brightness. The illumination obtained during testing is remembered.

During operation of the agro-industrial complex, automatic acquisition and processing of images, a change in the illumination of the object is possible. In this case, by manipulating the Level and ALC controls, the aperture of the pixel matrix is controlled in real time by aperture, providing brightness gradations stored in the test results.

So in high light P-Iris technology does not allow a strong narrowing of the hole D 4 of the photodetector 1 due to the joint operation of the PUD 5 electric motor for fine-tuning the hole D 4 and special software. If D 4 has narrowed to a possible non-distorting limit, and the amount of light incident on the matrix is large, then the sensitivity of the matrix is programmed. Through the interface, the preferred limits for automatically adjusting the diameter of D 4 are set. P-Iris increases the depth of field of the image, that is, the distance between near and far objects in the field of view of the photodetector 1, while they are still in focus. P-Iris allows high-quality video surveillance in a wide range of lighting and distances.

II. After automatically adjusting the initial state of the aperture D 4 in the lens, the initial adjustment of the “reverse focus” of the lens is performed. For O (TF) 6, the back focus setting provides a sharp image in the entire range of the focal length. To do this, determine the distance from the photodetector 1 to the focusing object, according to which the presence of sharpness or its absence is checked, and at a certain distance from the photodetector 1, find the object through which, for example, using the phase autofocus method, focus is made. Phase autofocus requires a separate optical path in which the focus detector is located. It is installed in the lower part of the photodetector 1, and the light enters it using an auxiliary mirror mounted on a hinge under a translucent main. The length of the optical path of light from O (TF) 6 to the detector during sighting and focusing must exactly match the path length to the matrix during exposure [Parshev A. Autofocus technology. Lens and autofocus: the tragedy of misunderstanding // Foto & video: magazine. - 2008. - No. 2. - S. 82-87]. This condition is achieved by adjusting O (TF) 6, on the accuracy of which the accuracy of autofocus operation depends [Volkov Yu., Kapustina N., Korotkov V. Automatic focus systems / Soviet photo. - 1986. - No. 11. - S. 42].

In FIG. Figure 3 shows an example of the implementation of phase autofocus (the numbering corresponds to the numbering of the original in accordance with US pat 5589909 fig 2.png), where 30 is the exit pupil of the lens; 72 - collective lens; 70 - window bounding box; 75 - mask; 76, 77 - micro lenses; 8 - sensor; 80, 81 - CCD line.

The detector consists of a condenser 72 located in the focal plane of the lens, and a sensor 8 with CCD arrays 80 and 81. During focusing, opposite areas 31 and 32 of the exit pupil 30 of the lens are compared. To do this, in the mask 75, on which the condenser builds a real image of the exit pupil of the lens, two micro-lenses 76 and 77 are installed, forming on the rulers 80 and 81 images of the subject, visible through different halves of the exit pupil. The size of the images is limited to the window 70 of the condenser frame 71. If the lens is focused precisely, the images of the object are located in the centers of neighboring CCD arrays and the signals received by the processor from different arrays coincide (they are “in phase”). With inaccurate focusing, these images due to parallax shift inward or outward of the rulers depending on the direction of the error, and the signals cease to coincide. Based on the phase difference of the signals, a command is generated for the focusing drive.

Phase autofocus provides maximum performance, because unlike contrast autofocus, it does not require sharpness comparison for different lens positions, and the magnitude and direction of its movement are immediately known. The focusing accuracy is directly dependent on the distance (basis) between the compared areas of the exit pupil 31 and 32.

The O (TF) 6 lens with an automatic diaphragm controls the luminous flux due to the feedback of signals coming from the 2nd and 3rd levels of the image forming and processing device.

Each of the array of pixels has conclusions on a grid of metal conductors, which allows temporary, readout and output signals to pass through all cells. The conclusions of this grid are connected to reading and decoding elements that are located outside the array of pixels.

This allows the accumulated charges to be read from the pulse generator from the entire pixel array, from some parts of this array and from individual pixels using simple XY addressing, and then, since the received signal is too small for independent use, it is amplified by the amplifier. The generation of voltage applied to the pixels is a function of the column driver, which is a reference voltage switch included in the ZCD panel.

When starting the process of automatic acquisition and processing of images by a command from a computer 21 at the upper level in module 3 at the lower level in module 1, each cycle begins by setting the coordinate code of the polled pixel and the appearance of a clock signal at the input of KIVm 11 from the control unit of module 2. A sequential poll is performed SUVm 10 pixels VM 9. The standard signal from the output SUVm 10 through the connector is fed to the inputs of the BS 12 and the ADC 13 of the average level in module 2.

In module 2, at the average level, the received analog signal with Vm 9 at the ADC 13 is converted to digital form, i.e. in the form of samples in both coordinates and in amplitude. Temporal sampling ¹¹ ( ¹¹ Discretization - representing the coordinates in the form of a finite set of samples.) And quantization ¹² ( ¹² Quantization - representing the amplitude with values from a finite set.) By level is performed in the ADC 13. The output signal of the ADC 13 is a sequence of numbers received in the DSP 15 performing the required processing. So, the ADC 13, which converts the input signal into 8-bit binary code combinations with a quantization frequency of 10 MHz, provides the transmission of 14 512 bytes of information to the BRD during one scan line. In BS 12, line and frame clock pulses are extracted from the signal, which, entering the control unit 19, start the write-read cycle of the BZU 16, form the ADC start-up pulses 13 and other control signals.

BRD 14 receives 8-bit code combinations from the ADC 13 and groups them into 16-bit code words, sending them to the BZU 16 DSP 15. The same register BRD 14 is used to store data from the computer, for their subsequent placement in the buffer memory BZU 16.

BZU 16 DSP 15 provides storage of one image frame in the format of 512 × 512 8-bit elements. MPA 18 performs a temporary separation of the address of the row of columns and transmits to the address bus BZU 16 combinations of A _p and A _stb at the corresponding moments of the cycle of accessing the memory.

The memory regeneration occurs during the flyback of the horizontal scan of the image by sequentially sorting the address of the memory lines. Filling of the BZU 16 is carried out for two half-frames of the image (1/25 sec).

When the last memory cell is full, writing to the BZU 16 stops, and the device enters the "Read" mode.

The amount of memory for storing a digital image depends on the resolution and color depth, and can be approximately determined by multiplying the number of pixels by the number of bits corresponding to one pixel.

BU 19 contains a crystal oscillator (ƒ = 20 MHz), shapers of service signals, a synchronization circuit. In the block, pulses CAS ¹³ are generated ( ¹³ CAS is the signal for selecting the column of the memory matrix.), RAS ¹⁴ ( ¹⁴ RAS is the signal for selecting the microcircuit - line selection of the memory matrix.), WR / RD ¹⁵ ( ¹⁵ WR / RD is the write / read control signal information.), providing dynamic memory operation, the cycle of accessing the memory is synchronized with the signal, control signals for all blocks are generated.

BPO 20 exchanges data between the 2nd level of the device and the IBM PC / AT 21 computer. BPO 20 writes / reads 16-bit words with the formation of the corresponding control signals using the instruction and status register (RKS), data register (RD), and Address Register (RA).

At the same time, at the upper level, module 3 real-time interfaces with the computer 21 external devices and sensors, i.e. image transmission in the form of electrical signals to external devices on a personal computer and using special software, the received data is converted to a format that the user works with and displayed on the monitor screen.

Computer 21 collects and converts information about the current state of the process of obtaining and processing images and current disturbances, followed by storing information about the receipt and processing of images and current tasks received in external memory.

In module 3, the specified algorithms for converting information to generate a control action are automatically executed, organizing the interaction of all elements of the computer 21, and consisting of arithmetic, logic, control, and RAM; a block of direct connections with the control object, transmitting control actions.

According to the situation in real time, the computer 21 receives information about the possibility of connecting additional interface devices, including in the form of controllers connected to the computer bus and their shutdown.

At the request of the operator, a computer can be connected to the computer screen 21, intended for operational personnel to obtain information on the progress of the technology for the process of obtaining and processing images, to correct the control algorithm and to take the necessary measures for input-output of information, including various recording and storage devices, screen remotes - displays, printers, plotters.

Using a general-purpose computer 21 simplifies and speeds up the process of automatic acquisition and processing of images. The versatility of digital computers as a means of solving a wide variety of problems, the huge amounts of information processed and stored in the computer 21, powerful algorithmic capabilities made the computer 21 an effective tool for solving modern problems of obtaining and processing images.

When playing back dynamic images, the memory is divided into pages that are alternately displayed on each regeneration (while one page is displayed on the screen, the second is filled with the next frame).

This device allows you to effectively register, process and visualize low-contrast images for a wide range of tasks. Those. we are talking about a basic device for creating an end-to-end format for receiving and processing digital images in any signal range, depending on the existing scanning tools. The resulting images are structurally prepared for the immediate effective implementation of procedures for all processing classes: storage and conversion (lossy compression, group compression, editing); analysis (recognition, filtering); synthesis (approximation, vectorization); transmission via communication channels (with or without confirmation, with or without protection).

Claims (1)

A device for automatically receiving and processing images containing a series-connected photodetector and a signal processing module, and the photodetector contains optically paired an input lens of the lens, a scanning device, characterized in that the image output module is additionally inserted into the device, the photodetector including an auto-adjustable diaphragm with a drive aperture control at the control input, the lens of the lens is designed as a zoom, the control input of which о is connected through a device for controlling the focal length to the first control input of the photodetector, as well as a sequentially matched filter for RGB decomposition of the light flux from the object in the Bayer arrangement, a video matrix and its control circuit, the control input and information inputs of which are connected to the information inputs through the interface controller, respectively and to the second control input of the photodetector, and the third control input of the photodetector is connected to the control input yes aperture control, RGB outputs in the Bayer layout of the video matrix control circuit are connected through the outputs of the photodetector to the inputs of the signal processing module, which contains an analog-to-digital converter, a synchronization unit, a data register unit, a digital signal processor with an input storage unit and an input / input register output, an address multiplexer and a control unit, the inputs of the analog-to-digital converter and the inputs of the synchronization unit, information the outputs of the analog-to-digital converter are connected to the first information inputs of the data register block, the second information inputs of which are connected to the first information inputs of the signal processing module, and the outputs of the data register block are connected through the data inputs of the input blocks of the storage devices to the digital signal processor, and through the address inputs of the blocks the storage devices connected outputs of the address multiplexer, the information inputs of which are connected to the second information inputs of the processing module s the outputs of the synchronization unit are connected to the inputs of the control unit, the first control output of which is connected to the first control input, the second control output with the second input, and the third with the third control input of the photodetector, information outputs of the control unit are connected to the information outputs of the signal processing module, which connected to the information inputs of the photodetector, the fourth output of the control unit is connected to the gate input of the analog-to-digital converter, the fifth outputs of the control The control unit is connected to the control inputs of the data register block, the sixth and seventh to the first and second control inputs of the storage unit, the eighth control output of the control unit is connected to the control input of the input / output register of the digital signal processor, the bi-directional connector of which is connected to the output of the processing module signals, which is connected to the bidirectional connector of the image output module and through the parallel exchange unit and internal bus with the IBM PC / AT image output module, the first the control output of the parallel exchange unit of the image output module is connected to the output of the image output module, which is connected to the control input of the signal processing module connected to the control input of the control unit, the Ready signal of which is connected to the first output of the signal processing module connected to the first control input image output module, which is connected to the first input of the parallel exchange unit, and the control output of the address multiplexer is connected to the second control output of the signal processing module Of the signals connected to the second control input of the image output unit, which is connected to the second control input of the parallel exchange unit of the image output unit, the outputs of the reset columns "Reset STB" and the lines "Reset STR" of the control unit are connected to the corresponding inputs of the address multiplexer, to which are also connected the outputs of the job switching frequencies of columns F _STB and rows F _PTP control unit.

Images