RU2389153C1 - Image formation device - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: device has an optical image reception unit (OIRU) fitted with photoelectric converters (PEC), a controller, first and second image memory units (MU), a multiple-channel correlation analyser (CA), a computer processor and a background frame storage unit. The optical image reception unit is based on a CMOS photoelectric converter with running electronic shutter with possibility of synchronous formation of frames with oppositely directed reference and base image scans. The computer processor is made with possibility of sorting pixels background images and moving objects under control of the correlation analyser, as well as conversion of pixel coordinates of moving objects. The principle of operation of the device lies in formation and analysis of reference and correction images with subsequent calculation of coordinates of image elements, in which there are no geometric distortions of moving objects, wherein the technical result lies.

EFFECT: increased accuracy of formation of images of moving objects.

3 dwg

Description

The invention relates to television technology. It is most effective to use it as part of television surveillance systems, as well as monitoring moving objects in various technological processes.

Known imaging device containing a source of light radiation, a scanning device configured to deflect light radiation and read an optical signal for alternately shooting portions of a test object, and an indication unit. To increase the scanning area, the node deflecting light radiation and reading the optical signal is made multipath (US 20050094234, G02B 26/08, 2005; US 20050018268, G02B 26/08, 2005).

However, this device does not allow external monitoring of objects without their illumination, which limits its scope.

Also known is an image shaper comprising an optical image pickup unit equipped with a CMOS photoelectric converter (PEC) with a traveling shutter connected to an image processing and imaging module including an image memory unit (PSU), an image unit PS coordinate, and a computing unit configured to convert the coordinates of the image sections depending on the mismatch of the coordinates of the sections of the current and previously captured basic images (JP 2006058945, G06T 3/00, H04N 1/04, H04N 1/40, 2006). To increase the accuracy of imaging of a moving object, the memory unit is capable of storing a large array of portions of sequentially captured frames, and the computing unit is capable of calculating and averaging the motion vectors of image regions (JP 2007180734, H04N 5/232, H04N 101/00, 2007).

The disadvantage of such devices is the low accuracy of image formation due to the need for long-term retention of the object in the field of view to capture a large array of memorized frames. Therefore, these devices cannot work in real time, especially at a high speed of movement of the observed object.

Closest to the claimed one is an image shaper containing an optical image pickup unit based on an optical microscope and equipped with a photomultiplier integrated in a digital camera, a controller, an image PSU, the first output of which is connected to the first input of a correlation analyzer (KA). The direct scan information output of the optical image pickup unit is connected to the first input of the image BP. The controller output is connected to the control input of the image BP used for storing image areas, as well as with the actuator for moving the optical image receiving unit (RU 2304807, G06K 9/36, 2007).

However, this device does not allow to correctly form images in which there are separate moving objects, especially when moving at different speeds. In addition, the implementation of the block optical image reception based on a scanning microscope limits the scope of the device. For this reason, it is fundamentally unacceptable in surveillance systems.

The technical task of the invention is to increase the accuracy of imaging of moving objects.

A new principle of operation of the proposed technical solutions consists in the synchronous formation of the reference and corrective images of the same frame with oppositely directed sweeps with their subsequent analysis using a correlation analyzer to detect changes in the coordinates of moving objects with subsequent calculation and formation of a transmitted image without geometric distortions of moving objects. Hereinafter, the reference image refers to the image of a moving object formed from the calculation of the coincidence of the vertical direction with the direction of the vertical component of the velocity vector of the observed object. A corrective image is understood as an image of a moving object formed from the calculation of the vertical direction of rotation opposite to the direction of the vertical component of the velocity vector of the observed object.

The indicated principle of action implements the observation established by the author: if the direction of the vertical component of the velocity vector of the moving object coincides with the direction of the vertical scan, the image of the moving object will be more vertically stretched compared to the image when they are in the opposite direction. (Pilipko N.E., Rychazhnikov AE. Features of the CMOS photodetector in the running shutter mode. News of St. Petersburg Electrotechnical University "LETI", Publishing House of St. Petersburg Electrotechnical University "LETI", 2008, No. 1, pp. 40-54). The algorithm for determining which of the channels contains the reference and which is the corrective image consists in comparing the maximum vertical sizes of moving objects selected by the spacecraft. Moreover, the channel in which the maximum size of the moving object vertically will be the largest is the reference for this object.

The solution to this technical problem lies in the fact that the design of the image former containing the optical image pickup unit equipped with a photoelectric converter, a controller, a first image memory unit, the first output of which is connected to the first input of the correlation analyzer, while the information output is a direct scan of the optical reception unit image is connected to the first input of the first image memory block, and the first output of the controller is connected to the control input of the first memory block and siderations, is amended as follows:

1) the imager further comprises a computing processor, a second image BP, and a background frame storage unit;

2) the optical image receiving unit is based on a CMOS FEP with a traveling electronic shutter with the possibility of synchronous formation of frames with oppositely directed sweeps;

3) The SC is multichannel with the possibility of synchronous comparison of signals at its first and third inputs with a signal applied to the second input of the correlation analyzer, as well as comparing signals received at the first and third inputs of the correlation analyzer;

4) the second output of the controller is connected to the control input of the optical image receiving unit directly or through the address picker of the read pixels of the photocell;

5) the second output of the first image BP is connected to the first information input of the computing processor;

6) the second input of the first image BP is connected to the first output of the computing processor;

7) the third output of the first image BP is connected to the information input of the background frame storage unit;

8) the second input of the spacecraft is connected with the first output of the background frame storage unit;

9) the third input of the spacecraft is connected with the first output of the second PSU image;

10) the first output of the spacecraft is connected with the third input of the first PSU image;

11) the second output of the spacecraft is connected with the second input of the second PSU image;

12) the counter-scan output of the optical image receiving unit is connected to the first input of the second image BP;

13) the second output of the background frame storage unit is connected to the third input of the second image BP;

14) the control input of the background frame storage unit is connected to the third controller output;

15) the control input of the second image BP is connected to the fourth controller output;

16) the fifth controller output is connected to the control input of the computing processor;

17) the second information input of the computing processor is connected to the second output of the second image BP;

18) the second output of the computing processor is connected to the fourth input of the second image BP;

19) the computing processor is configured to sort the pixels of images of the background and moving objects under the control of a correlation analyzer, as well as transforming the coordinates of the pixels of moving objects according to the formulas:

(one)

where x and y are the coordinates of the point of the formed image horizontally and vertically, respectively, a pixel;

x ₁ and y ₁ are the corresponding coordinates of the same point in the image at which the vertical component of the velocity vector of a moving object coincides with the vertical direction, pixel;

x ₂ and y ₂ are the corresponding coordinates of the same point in the image at which the vertical component of the velocity vector of a moving object is opposite to the vertical direction, pixel;

L 'is the maximum vertical size of a moving object in the image, the vertical direction of which coincides with the vertical component of the velocity vector of the moving object, pixel;

L ”is the maximum vertical size of a moving object in the image, the vertical direction of which is opposite to the vertical component of the velocity vector of the moving object, pixel;

h - height of the object in the adjusted image, pixel, determined by the formula

,

,

where H is the height of the active photosensitive region of the photoelectric transducer, pixel.

The multichannel correlation analyzer for the proposed device can be performed on the basis of two GP1020 microcircuits (Zarlink). However, the technical implementation of the correlation analyzer, computing processor, and controller based on one programmable logic integrated circuit, for example ЕР3С10АТС144 (Altera Corp.), is most appropriate.

Figure 1 shows a block diagram of the proposed imaging device; figure 2 and 3 shows a block diagram of possible embodiments of the block optical image reception.

The imager (Fig. 1) contains an optical image pickup unit 1 equipped with a photoelectric converter, a controller 2, a first image BP 3, the first output of which is connected to the first input of the multi-channel spacecraft 4. The direct scan information output of the optical image pickup unit 1 is connected to the first input first PSU 3 images. The first output of the controller 2 is connected to the control input of the first PSU 3 image. The imager further comprises a computing processor 5, a second image BP 6 and a background frame storage unit 7. In this case, the optical image receiving unit 1 is made on the basis of a CMOS FEP with a traveling electronic shutter with the possibility of synchronous formation of frames with oppositely directed sweeps; the second output of the controller 2 is connected to the control input of the optical image receiving unit 1; the second output of the first image BP 3 is connected to the first information input of the computing processor 5; the second input of the first image BP 3 is connected to the first output of the computing processor 5; the third output of the first image BP 3 is connected to the information input of the background frame storage unit 7; the second input of the multi-channel spacecraft 4 is connected to the first output of the background frame storage unit 7, the third input of the multi-channel spacecraft 4 is connected to the first output of the second image BP 6; the first output of the multi-channel spacecraft 4 is connected with the third input of the first PSU 3 image; the second output of the multi-channel spacecraft 4 is connected to the second input of the second PSU 6 image; the counter-scan output of the optical image receiving unit 1 is connected to the first input of the second image BP 6; the second output of the background frame storage unit 7 is connected to the third input of the second image BP 6; the control input of the background frame storage unit 7 is connected to the third output of the controller 2; the control input of the second PSU 6 image is connected to the fourth output of the controller 2; the fifth output of the controller 2 is connected to the control input of the computing processor 5; the second information input of the computing processor 5 is connected to the second output of the second PSU 6 image; the second output of the computing processor 5 is connected with the fourth input of the second PSU 6 image. Multichannel KA 4 is made with the possibility of synchronous comparison of signals at its first and third inputs with a signal supplied to the second input of KA 4 (on the first and second channels, respectively), as well as comparing between themselves the signals received at the first and third inputs (third channel) KA 4. Computing processor 5 is configured to sort the pixels of images of the background and moving objects under the control of KA 4, as well as transform the coordinates of the pixels of moving objects according to formulas (1).

The second controller output can be connected to the control input of the optical image receiving unit directly (Fig. 2) or through the address picker of readable pixels of the photomultiplier (Fig. 3).

In the embodiment of FIG. 2, the optical image receiving unit 1 includes a lens 8 connected in series, a beam splitter 9, and two parallel channels of photoelectric conversion of the light flux into images with oppositely directed sweeps connected to its outputs. In each of these channels, a CMOS FEP (pos. 10 and 10 '), a FIFO stack (pos. 11 and 11') and an amplifier-converter (pos. 12 and 12 ') are connected in series.

The information luminous flux of the image, passing through the lens 8 and the beam splitter 9, falling on the photomultiplier tubes 10 and 10 ', is converted by them into an electrical signal in proportion to the intensity of this light flux. In this case, the analog-to-digital converters built into the photovoltaic cells 10 and 10 'transmit for storing into the FIFO stacks 11 and 11' a sequence of digital data corresponding to the samples of signals from the unit cells of the photosensitive regions of the photovoltaic cells 10 and 10 '. The organization of stacks 11 and 11 'in the form of FIFO allows you to store in them the received image line by line with popping the earliest of the received data to update the information and match the speed of its receipt with the speed of further processing. In this case, the opposite directionality of the sweeps in the channels is ensured by the corresponding relative position of the CMOS FEP 10 and 10 ', and the synchronization of the frame formation is provided by the synchronizing pulse supplied to the control inputs of the said FEP from the output 2 of the controller 2. The amplifiers-converters 12 and 12' filter and normalize the output signals unit 1 of the optical image reception. When forming CMOS FEP 10 and 10 'together with the possibility of their illumination from one source, the need to turn on the beam splitter 9 disappears.

In the embodiment of FIG. 3, the forward and onward scan channels are made using one CMOS FEP, the address input of which is connected to a pixel address generator 13 controlled by clock pulses from the output 2 of controller 2. Moreover, the opposite direction of the scan in the channels is ensured by setting the corresponding synchronously read pixel addresses FEP 10.

The imaging device (figure 1) works as follows. Controller 2 sets the sequence diagram of the device. According to the signal from the second output (“α”) of the controller 2, the optical image receiving unit 1 generates two images with oppositely directed sweeps. The data of these images synchronously arrive in the first BP 3 images and the second BP 6 images, respectively. In this case, by switching the addresses (pixels) of the first BP 3 of the image and the second BP 6 of the image, the controller 2 controls its first and fourth outputs, respectively, with each image element being allocated memory for storing information about brightness and color, as well as for the following service information used in the process of forming the final image with eliminated geometric distortions of moving objects. After saving the original images in blocks 3 and 6, they are compared using the multi-channel KA 4 with the corresponding elements of the background image captured earlier and stored in block 7. The addresses of block 7 are switched by the signal from the third output of controller 2. Image pixels that do not match the corresponding pixels background frame are marked as belonging to a moving object. At the same time, there is a correlation comparison of the image data stored in blocks 3 and 6 with each other to determine a new background frame.

After the correlation analysis of the image data is completed, the controller 2 in the fifth output initiates the operation of the computing processor 5, which sequentially for each current pixel analyzes all its neighboring pixels. Computing processor 5 performs operations of selection of different moving objects, determining points lying inside the object or on its border, and determining the maximum vertical sizes of detected moving objects. The rules for interpreting the connectivity of neighboring pixels, for example, vertically, horizontally and / or diagonally, are set by controller 2 and can be changed by changing the corresponding settings.

Pixels marked by spacecraft 4 as belonging to a moving object and located directly next to each other are marked by computing processor 5 as belonging to the same object.

Pixels belonging to a moving object and bordering only pixels belonging to the same object are marked by them as internal pixels of this object.

Pixels belonging to a moving object and bordering background pixels are marked as belonging to the boundary of the object.

Computing processor 5 also determines the maximum vertical dimensions of moving objects in images formed with opposite scans, and compares them to determine which image stored in the first image BP 3 is reference or corrective for each of the detected moving objects.

Then, the controller 2 performs the operation of transferring the background frame from the background frame storage unit 7 to the second image BP 6, replacing the image stored there with the background. If the computational processor 5, when analyzing the frame, does not find differences between images formed with opposite scans, it issues at its first output permission to transfer the image from block 3 to block 7. This transfer occurs at the command of controller 2 synchronously with the transfer of the previous background frame from the block 7 to block 6.

Then, the computing processor 5 by formulas (1) converts the coordinates of the pixels belonging to moving objects in the image stored in block 3, and generates in block 6 the corrected images of moving objects in the background image located at that moment. In this case, the final image can be read from the output of block 6, which is the output of the entire device.

Further cycles of the device are repeated.

The use of the invention allows to increase the accuracy of imaging of moving objects by transmitting their shape without distortion, which is confirmed by mathematical modeling of the device under the conditions of moving moving objects with speeds not exceeding the speeds of horizontal and vertical scan of photomultiplier.

Claims (1)

An image forming apparatus comprising an optical image pickup unit equipped with a photoelectric converter, a controller, a first image memory unit, the first output of which is connected to a first input of a correlation analyzer, wherein a direct scan information output of an optical image pickup unit is connected to a first input of a first image memory unit, and the first output of the controller is connected to the control input of the first block of image memory, characterized in that it further comprises a computing a processor, a second image memory unit and a background frame storage unit, while the optical image receiving unit is made on the basis of a CMOS photoelectric converter with a running electronic shutter with the possibility of synchronous formation of frames with oppositely directed sweeps, the correlation analyzer is multi-channel with the possibility of synchronous comparison of signals from its first and third inputs with a signal applied to the second input of the correlation analyzer, as well as comparing the signals to each other, p stumbled onto the first and third inputs of the correlation analyzer, the second controller output connected to the control input of the optical image pickup unit directly or via the address picker of the read pixels of the photoelectric converter, the second output of the first image memory block connected to the first information input of the computing processor, the second input of the first memory block image is connected to the first output of the computing processor, the third output of the first block of image memory is connected is accessed to the information input of the background frame storage unit, the second input of the correlation analyzer is connected to the first output of the background frame storage unit, the third input of the correlation analyzer is connected to the first output of the second image memory unit, the first output of the correlation analyzer is connected to the third input of the first image memory unit, second output the correlation analyzer is connected to the second input of the second image memory unit, the on-off output of the optical image receiving unit is connected to the first one of the second image memory unit, the second output of the background frame storage unit is connected to the third input of the second image memory unit, the control input of the background frame storage unit is connected to the third controller output, the control input of the second image memory unit is connected to the fourth controller output, the fifth controller output is connected to the control input of the computing processor, the second information input of the computing processor is connected to the second output of the second image memory unit, and the second output is islitelnogo processor connected to a fourth input of the second memory block of the image, wherein the computer processor is operative to sort pixels background image and moving objects under control of the correlation analyzer and the coordinate conversion of pixels of moving objects according to the formulas

where x and y are the coordinates of the point of the formed image horizontally and vertically, respectively, a pixel;
x ₁ and y ₁ are the corresponding coordinates of the same point in the image at which the vertical component of the velocity vector of a moving object coincides with the vertical direction, pixel;
x ₂ and y ₂ are the corresponding coordinates of the same point in the image at which the vertical component of the velocity vector of a moving object is opposite to the vertical direction, pixel;
L 'is the maximum vertical size of a moving object in the image, the vertical direction of which coincides with the vertical component of the velocity vector of the moving object, pixel;
L ”is the maximum vertical size of a moving object in the image, the vertical direction of which is opposite to the vertical component of the velocity vector of the moving object, pixel;
h - height of the object in the adjusted image, pixel, determined by the formula

where H is the height of the active photosensitive region of the photoelectric transducer, pixel.

Images