RU2812612C1 - Device of computer system for panoramic television surveillance for unmanned aerial vehicle - Google Patents

Abstract

FIELD: security systems.

SUBSTANCE: panoramic television surveillance, which is performed by a computer system using a television camera that provides all-round visibility simultaneously or sequentially in four spherical layers of the surrounding spherical region of space. For each of these spherical layers, television surveillance in real time is carried out in a spatial angle of 360 degrees in azimuth and tens of degrees in elevation. The sensor block of a television camera consists of one single photodetector, manufactured using the technology of complementary metal-oxide-semiconductor structures, which has the shape of a target in the form of a circular ring.

EFFECT: elimination of losses of photosensitive elements in a television camera, providing the ability to monitor video information of a panoramic plot in four layers of the surrounding space in non-stop mode.

2 cl, 6 dwg

Description

The proposed invention relates to panoramic television surveillance, which is performed by a computer system using a television camera that provides a sequential all-round view in four spherical layers of the surrounding spherical region of space. Moreover, for each of these spherical layers, television monitoring of the situation in real time is carried out in a spatial angle of 360 degrees in azimuth and tens of degrees in elevation. The sensor block of this television camera consists of one (single) photodetector manufactured using complementary metal-oxide-semiconductor (CMOS) technology, which has the shape of a target in the form of a circular ring.

The closest in technical essence to the claimed invention should be considered a device for a computer system for panoramic television surveillance for an unmanned aerial vehicle [1], containing a serially connected television camera and a server, which is a local area network node to which two or more personal computers are connected, while the television camera includes a first panoramic lens, a second panoramic lens, a third panoramic lens, a fourth panoramic lens; and also a sensor block containing a first photodetector and a second photodetector, which have the shape of a circular ring, with the first panoramic lens being optically connected to the target of the first photodetector, the second panoramic lens to the target of the second photodetector, and each of the two photodetectors of the sensor block is made on a crystal made using CMOS technology, and its target consists of photodiode active pixels, each of which has an amplifier with a gain K _m , as well as a built-in analog-to-digital converter (ADC), which ensures the transmission of the video signal of the active pixel to its “radial” video bus, while all of them together combine the active pixels of the target into “radial” columns, and the ADC control for pixels located along each “ring” line of the photodetector is carried out using a separate “ring” line bus, the total number of which determines the number of lines in the photodetector, and the number of “radial” video buses is the number of pixels in each row of the photodetector, while on the common crystal of the photodetector there are also blocks that perform scanning and formation of the output voltage of the digital video signal, namely: a “ring” frame scanning register that selects the “ring” line; A "ring" video switcher containing video switches for each "radial" column, which are controlled from the corresponding output of the "ring" horizontal multiplexer and provide transmission of the video signal at the output of each "radial" video bus to the "ring" video bus, the output of which is the output “video” photodetector, and the gain K _m of the active pixel for each current “ring” line of the photodetector changes according to the ratio:

where Δ ₁ and Δ _m are, respectively, the photosensitive area of the active pixel for the first and current “ring” reading line in the “ring” sensor, and K ₁ is the gain coefficient of the active pixel of the first “ring” line with an indicator value equal to unity, ensuring the same reading value apertures within the entire “ring” raster of the image,

Moreover, on the crystals of both photodetectors of the sensor unit there is a cut (through slot) made in the radial direction from the imaginary center of the ring to its outer periphery, in a location between the first and last pixels of the first “ring row”, while the first photodetector is installed in a cantilever connection with the second photodetector into a slot at an angle of 90°, while the output of the first photodetector of the sensor unit is connected to the first information input of the multiplexer-modulator, and the output of the second photodetector is connected to the second information input of the multiplexer-modulator, the output of which is the “video” output of the television camera; in this case, the television camera includes a ball bearing installed in the space, which consists of a pixel-free area of the first and second photodetectors, and a demodulator of operator commands, while in the non-stop mode of the aircraft according to commands received by the television camera from the computer of the system operator, sequential spatial movement of these photodetectors is ensured using the first and second rotation blocks of the electromagnet armature, installed on the outer periphery of the first and second photodetectors, respectively, wherein the first rotation block implements a spatial rotation of the first photodetector through an angle of 180°, and the second rotation block implements a spatial rotation through an angle of 180 ° the second photodetector, so that the target of the first photodetector is installed symmetrically in a position opposite the third panoramic lens, and the target of the second photodetector is symmetrically in a position opposite the fourth panoramic lens, while the video card located on the server motherboard demodulates the new input digital television signal and its demultiplexing into two additional channels with subsequent recording of each of the “ring” video signals into the corresponding RAM of the server and sequential conversion of each of the “ring” recording frames into “rectangular” frames, with the number of “rectangular” frames n corresponding to one current “ring” frame, satisfies the relation:

Where is the horizontal angle of the field of view in degrees of the image observed by the operator, and this transformation itself is performed programmatically.

As noted in the description of the invention [1], the prototype device can be installed in an unmanned aerial vehicle, such as a hexacopter. The term “hexacopter” is commonly used in the technical literature to refer to a radio-controlled model of an unmanned aerial vehicle with six wings, designed to perform aerial video shooting of an area.

Obviously, the multiplexer-modulator of the prototype television camera [1] should be considered as two units connected in series, namely: a multiplexer and a modulator.

The disadvantage of the prototype is the inevitable loss of photosensitive elements of both photodetectors of the television camera, which occurs due to their cantilever connection to each other by cutting using the through slot method.

The objective of the invention is to fundamentally eliminate the inevitable losses of photosensitive elements in a television camera by using one (single) photodetector, eliminating the need to cut it using the through slot method, while maintaining the ability for an unmanned aerial vehicle to monitor video information of a panoramic plot in four layers of the surrounding space in a non-stop mode.

The stated problem in the proposed device for a panoramic television surveillance system for an unmanned aerial vehicle is solved by the fact that, as in the prototype device [1], containing a series-connected television camera and a server, which is a local area network node to which two or more personal computers are connected, wherein the television camera includes a first panoramic lens, a second panoramic lens, a third panoramic lens and a fourth panoramic lens; as well as a sensor unit containing a photodetector, which has the shape of a circular ring, and the first panoramic lens is optically connected to the target of the photodetector, and it itself is made on a crystal made using CMOS technology, and its target consists of photodiode active pixels, each of which has an amplifier with a gain K _m , as well as a built-in ADC that ensures the transmission of the active pixel’s video signal to its “radial” video bus, while all of them collectively combine the active pixels of the target into “radial” columns, with ADC control for pixels located along each “ the ring" line of the photodetector is carried out using a separate "ring" line bus, the total number of which determines the number of lines in the photodetector, and the number of "radial" video buses is the number of pixels in each line of the photodetector, while blocks are placed on the common photodetector crystal performing scanning and generating the output voltage of a digital video signal, namely: a “ring” frame scan register that selects a “ring” line; A "ring" video switcher containing video switches for each "radial" column, which are controlled from the corresponding output of the "ring" horizontal multiplexer and provide transmission of the video signal at the output of each "radial" video bus to the "ring" video bus, the output of which is the output “video” photodetector, and the gain K _m of the active pixel for each current “ring” line of the photodetector changes according to relation (1), providing the same value of the reading aperture within the entire “ring” raster of the image, while the output of the photodetector of the sensor unit is connected to the information input of the digital television signal modulator, the output of which is the “video” output of the television camera, which also includes a rotation unit and a demodulator of operator commands that performs electromechanical control of the rotation unit from the server computer, while a video card is installed on the server motherboard that performs demodulation of the input digital television signal with subsequent recording of the “ring” video signal into the corresponding cell of the server’s RAM and sequential conversion of the “ring” recording frame into “rectangular” frames, and the number of “rectangular” frames n, corresponding to one current “ring” frame, satisfies the relation (2 ); but at the same time, unlike the prototype [1], a frame is introduced into the television camera, which houses a ball bearing and a photodetector, and the rotation unit is made on the basis of a stepper motor, the shaft of which is directly or through a gearbox mechanically connected to the frame, providing at the operator’s command from the computer, spatial rotation of the photodetector frame at an angle of 90° in a step-by-step mode four times per cycle, so that its target, set by default symmetrically in a position opposite the first panoramic lens, is sequentially installed first symmetrically in a position opposite the second panoramic lens, then symmetrically in a position opposite third panoramic lens, and finally - symmetrically to a position opposite the fourth panoramic lens, and if the computer operator sends alternative (opposite commands) to the television camera, then the frame with the photodetector will sequentially and step by step return back to the initial (original) position, and the value the gain factor of the active pixel K ₁ for the first line of the photodetector of the sensor unit of a television camera can be equal to unity or be greater than it, providing the same value of the reading aperture within the entire “ring” raster of the image in the mode of organizing manual or automatic adjustment of the sensitivity of the photodetector depending on the illumination of objects panoramic plot, while a new digital video signal generated in the television camera after completion of each of the four rotations of the photodetector is alternately recorded in the corresponding RAM cell of the server, which is used as the system unit of the system operator’s computer.

The set of known and new features for the proposed device is not known from the prior art, therefore, the proposed technical solution meets the criterion of novelty.

It is important to note the following. The photosensitive area of the pixels of the “ring” target of the photodetector of the sensor unit, as for the prototype [1], is different from line to line. This is caused by the need for a “ring” photodetector, which has the same number of pixels in each line, to equalize the resolution within the frame by ensuring the same size of the technological gap between the photosensitive elements along each “ring” line.

But at the same time, in the proposed solution, as in the prototype [1], there is no interline violation of the sensitivity of the sensor due to the following circumstances.

The reading aperture parameter for all pixels of each current row of the “ring” frame is determined by the product of the gain coefficient K _m of the pixel by the value of its photosensitive area Δ _m .

As follows from relation (1), this indicator remains constant (unchanged) for all photosensitive pixels of the photodetector.

The size of the noise “track” for each active pixel of the photodetector does not change either, which is a prerequisite for achieving its high sensitivity and signal-to-noise ratio.

Therefore, the proposed technical solution meets the criterion of an inventive step.

In fig. Figure 1 shows a block diagram of the proposed computer system for panoramic television surveillance and in the same drawing - a block diagram of the television camera in its composition, when the photodetector occupies a position that ensures monitoring of the surrounding space in its layer on the left; in fig. Figure 2 shows all four possible positions of the photodetector of a television camera, which implement monitoring of the surrounding space sequentially in four layers: left, top, right and bottom; in fig. Figure 3 shows the circuitry of a “ring” photodetector; in fig. 4 - details of this organization in relation to a single “radial” column; in fig. Figure 5, according to [2], shows a photograph of an image obtained using a panoramic lens; in fig. 6 - an illustration of the operating principle of a stepper motor, which is used as part of a television camera.

Design of a computer system for panoramic television surveillance, see Fig. 1, contains a serially connected television camera 1 and a server 2 (with a video card installed in it), which is a local area network node to which two or more personal computers are connected in position 3, while the television camera 1 includes the first panoramic lens 1 -1, a second panoramic lens 1-2, a third panoramic lens 1-3 and a fourth panoramic lens 1-4, as well as a sensor unit 1-5 containing a photodetector in position 5, a ball bearing 1-6, a photodetector frame in position 1-7 , as well as a rotation unit 1-8, which, upon command coming from server 2 through the demodulator 1-9, performs a smooth rotation of the frame 1-7 clockwise at an angle of 90° in one discrete movement (step) of the stepper motor shaft, while The “video” output of the photodetector of the sensor unit 1-5 is connected to the television signal modulator 1-10, the output of which is the “video” output of television camera 1.

The panoramic lenses 1-1, 1-2, 1-3 and 1-4 are designed to form all-round optical images sequentially for four spherical layers of the controlled space on the left, top, right and bottom, corresponding to the viewing direction of the television operator, as shown in FIG. 2.

On the other hand, the rotation unit 1-8 can perform its functional duties by providing the operator with sequential monitoring of the all-round view of the spherical layers of the observed space, respectively, from the left, rear, right and front.

As a technical solution for panoramic lenses 1-1, 1-2, 1-3 and 1-4, coinciding with a similar solution for the prototype [1], a panoramic mirror-lens lens can be proposed, the design of which is patented in Russia by domestic specialists from Moscow State University of Geodesy and Cartography [2].

A photograph of the annular image formed by a panoramic lens is shown in Fig. 5. The angular field in object space for this lens is 360 degrees in azimuth and can reach (75-80) degrees in elevation.

The “ring” photodetector 5 (see Fig. 3) is made using CMOS technology and contains on a common crystal a “ring” photodetector area (target) 5-1, a “ring” frame scanning register 5-2, a “ring” switch 5-3 video signals and a “ring” multiplexer 5-4.

As shown in FIG. 3, the active pixels on the photodetector target are combined into columns, which are located along radial directions from the imaginary center of the circular ring.

Each active pixel of the target (see Fig. 4) includes a photosensitive region (area) 5-1-1, an amplifier 5-1-2 with a gain K _m for each current “ring” line and an ADC 5-1- 3.

The “ring” video switch 5-3 consists of individual video switches 5-3-1, the number of which corresponds to the number of active pixels in a row, connected by a “ring” video bus 5-3-2.

Note that shown in FIG. 3 the shape of the photosensitive pixel area is in the form of a rectangle, and in FIG. 4 - the Latin letter L - are conditional. In practice, the charge accumulation electrodes of the active pixels of the sensor target, which coincide with the area of their photosensitive area, can be made completely differently, for example, with a geometric shape in the form of part of a circular ring.

The video signal from the output of each ADC 5-1-3 for each active pixel of a single “radial” column is transmitted to the “radial” video bus 5-1-5. Next, using “its own” key MOS transistor of the switch 5-3-1, controlled from one of the outputs of the multiplexer 5-4, the digital video signal of the current pixel is transmitted to the “ring” video bus 5-3-2, and then transmitted through it to photodetector output.

Note that in FIG. 3 dotted arrows show the control of the “ring” line buses 5-1-4 of the photodetector from the side of the “ring” register 5-2 frame scanning. What is here, as in Fig. 4, only four line buses are shown; this is a convention of the drawing. As noted above, the number of buses 5-1-4 corresponds to the actual number of “ring” lines in the proposed photodetector.

Let us further explain in Fig. 3 and more. The solid line arrows mark the transfer of the image signal in the sensor along the 5-1-5 video "radial" buses towards the 5-3 video "ring" switch.

As a result, in a “ring” raster, sequentially one after the other for each pixel of an individual “ring” line and sequentially line by line for the target as a whole, the voltage of the output video signal of the photodetector is generated in digital form.

Thanks to the CMOS technology adopted for the manufacture of the proposed video signal sensor, it is possible to integrate onto one common crystal not only a photodetector with an ADC for each active pixel, but also digital scanning units of a television camera. The implementation of such a solution ensures a significant reduction in the overall energy consumption of the television camera.

It must be recognized that the concept of a matrix (rectangular) photodetector with an active pixel, an ADC built into it and a digital video signal at the output, which was supposed to be implemented using CMOS technology by implementing the “coordinate addressing” method, was developed by American specialists in the “zero” two thousand years. This was also reported in the domestic monograph [3, p. 67, fig. 1.21]. However, the circuitry of a “ring” photodetector on a CMOS chip with similar capabilities was not mentioned there.

The “ring” shape of the target of the CMOS photodetector and scanning units proposed here makes it possible to more effectively use the useful area of the crystal used for television and computer observation of panoramic scenes.

The demodulator of operator commands 1-9 is designed to convert on-air digital signals coming from server 2 to television camera 1 into logical signals “1” or “O” transmitted over wires.

The television signal modulator 1-10 is used to convert the output video signal of the television camera into the form necessary for transmitting it over the air channel (wireless network).

Obviously, thanks to CMOS technology, the modulator 1-10 can be made as part of the photodetector of the sensor unit 1-5.

The ball bearing 1-6 is designed to ensure smooth running of the frame 1-7 with the photodetector, which is mechanically connected to the rotation unit 1-8.

The turning unit 1-8 is made on the basis of a stepper motor, which is an electrical machine designed to convert electrical energy into mechanical energy, see, for example, the monograph [4, p. 95] and publication on the Internet [5].

Structurally, a stepper motor consists of stator windings and a magnetically soft or magnetically hard rotor.

A distinctive feature of a stepper motor is discrete rotation, in which a given number of pulses corresponds to a certain number of steps taken. Obviously, if necessary, the stepper motor can be equipped with a gearbox.

The operating principle of the stepper motor used in the rotation unit 1-8 of the proposed device is shown in Fig.6. This drawing shows 4 windings that belong to the motor stator, and their arrangement is arranged so that they are at an angle of 90° relative to each other. From which it follows that such a machine is characterized by a step size of 90°.

At the moment a high level of pulse voltage U1 is supplied to the first winding, the rotor moves by these 90°. If a high level of pulse voltage U2, U3, U4 is alternately supplied to the corresponding windings, the shaft will continue to rotate until the full circle is completed. After which the cycle repeats again. To change the direction of rotation, it is enough to change the sequence of pulses supplied to the corresponding stator windings of the stepper motor.

To apply potential to the windings of the stepper motor, a mechanism is required that can issue a control pulse or group of pulses in a specific sequence. The role of such a mechanism is the stepper motor control unit, which contains a microprocessor-based driver, as well as a controller that provides direct connection of digital signals to this motor.

It is obvious that in the inventive computer system device, the rotation unit 1-8 of the television camera contains a successor-connected stepper motor, providing four fixed shaft positions at an angle of 90°, and a stepper motor control unit, the input of which can be connected via a USB interface to an external unit digital control (from computer side 4).

The device of a computer system for panoramic television surveillance for an unmanned aerial vehicle (see Fig. 1) works as follows.

Let television camera 1 be mounted on a hexacopter and located at a certain height relative to the Earth.

We will assume that the communication lines of the television camera 1 with the computer 4 of the system operator for video signal and control are implemented in the IEEE 802/11 standard for wireless networks [6, p. 380-381]. This means that data transmission occurs at one of two speeds (1 or 2 Mbit/s) in the 2.4 GHz frequency band.

Let the design solution of the sensor unit 1-5 as part of the television camera 1, see Fig. 1 and fig. 2 is implemented in such a way that in the initial position 1 the viewing axis of the first panoramic lens 1-1, and, consequently, the optical axis of the photodetector 5 is directed horizontally to the left. Note that the target of the photodetector 5 occupies a position in this situation, which is shown by lines of square dots.

The “ring” target of the photodetector 5 is exposed continuously. Therefore, at the output of the sensor unit 1-5, a digital video signal of the “ring” frame of the photodetector 5 is generated.

The received digital television signal (DTS), containing video information about the spherical layer on the left for the surrounding space, is then subjected to high-frequency modulation in block 1-10, becoming available for broadcast, and is sent to the output of television camera 1.

Then this DTS is transmitted wirelessly to server 2, where (on the video board) it is demodulated and subsequently recorded in the first RAM block per frame.

To obtain the PZT in the television camera 1 in relation to the other three spherical layers of the surrounding space without landing the hexacopter on Earth, it is enough to submit from the computer 4 necessary commands (in manual or automatic modes) that ensure sequential rotation of the photodetector 5 (clockwise) at an angle of 90° using stepper motor block 1-8.

As a result, the target of the photodetector 5 first appears in position 2 opposite the second panoramic lens 1-2, then in position 3 - opposite the third panoramic lens 1-3, and at the end of the cycle - opposite the fourth panoramic lens 1-4.

Like the first DZT, each of the three new DZTs will sequentially be subject to high-frequency modulation (detection) in block 1-10 of the television camera, and then in server 2 - demodulation of the video signal and its subsequent recording, respectively, into the second, third and fourth block of RAM on frame.

Sequential conversion according to relation (2) in server 2 of each of the four “ring” recording frames into “rectangular” frames successfully completes the task of sequential monitoring of video information for four spherical layers of the surrounding space (left, top, right and bottom).

Currently, all elements of the block diagram of a computer system for panoramic television surveillance for an unmanned aerial vehicle, see FIG. 1 have been mastered or can be mastered by domestic industry.

Therefore, the proposed invention should be considered to meet the requirement of industrial applicability.

INFORMATION SOURCES

1. RF patent No. 2780801. IPC H04N 5/225. SPK H04N 5/2251; H04N 5/2258; H04N 5/2259; H04N 5/23203. Design of a computer system for panoramic television surveillance for an unmanned aerial vehicle. / V.M. Smelkov // B.I. - 2022. - No. 28.

2. RF patent No. 2185645. IPC G02B 13/06, G02B 17/08. Panoramic reflex lens. / A.V. Kurtov, V.A. Solomatin // B.I. - 2002. - No. 20.

3. Berezin V.V., Umbitaliev A.A., Fakhmi Sh.S., Tsytsulin A.K. and Shipilov N.N. Solid-state revolution in television: Television systems based on charge-coupled devices, systems on a chip, and video systems on a chip. Ed. A.A. Umbitaliev and A.K. Tsytsulina. - M.: “Radio and Communication”, 2006.

4. Ginzburg S.A., Lekhtman I.Ya., Malov V.S. Fundamentals of automation and telemechanics. Under general ed. S.A. Ginsburg. Ed. 4th revised. Per. from English - “Energy”, 1968.

5. https//www.asutpp.ru/shagovyy-dvigatel.html

6. Vlado Damjanovski. CCTV. The CCTV Bible. Digital and network technologies. 2nd edition. Per. from English ISS Press LLC, 2006.

Claims (6)

1. A computer system device for panoramic television surveillance for an unmanned aerial vehicle, containing a serially connected television camera and a server, which is a local area network node to which two or more personal computers are connected, wherein the television camera includes a first panoramic lens, a second panoramic lens , a third panoramic lens and a fourth panoramic lens; as well as a sensor unit containing a photodetector, which has the shape of a circular ring, and the first panoramic lens is optically connected to the target of the photodetector, and it itself is made on a crystal made using complementary metal-oxide-semiconductor (CMOS) technology, and its target consists of photodiode active pixels, each of which has an amplifier with a gain K _m , as well as a built-in analog-to-digital converter (ADC), which ensures the transmission of the active pixel video signal to its “radial” video bus, while all of them together combine the active pixels targets into “radial” columns, and the ADC control for pixels located along each “ring” line of the photodetector is carried out using a separate “ring” line bus, the total number of which determines the number of lines in the photodetector, and the number of “radial” video buses the number of pixels in each line of the photodetector, while on the common crystal of the photodetector there are also blocks that perform scanning and formation of the output voltage of the digital video signal, namely: a “ring” frame scanning register that selects the “ring” line; a “ring” video switch containing video switches for each “radial” column, which are controlled from the corresponding output of the “ring horizontal multiplexer and provide transmission of the video signal at the output of each “radial” video bus to the “ring” video bus, the output of which is the output of “ video" photodetector, and the gain K _m of the active pixel for each current "ring" line of the photodetector changes according to the ratio: where Δ ₁ and Δ _m are, respectively, the photosensitive area of the active pixel for the first and current “ring” reading line in the “ring” sensor, and K ₁ is the gain coefficient of the active pixel of the first “ring” line with an indicator value equal to unity, ensuring the same reading value apertures within the entire “ring” raster of the image, while the output of the photodetector of the sensor unit is connected to the information input of the digital television signal modulator, the output of which is the “video” output of the television camera, which also includes a rotation unit and a demodulator of operator commands that performs electromechanical control rotation unit from the server computer, while a video card is installed on the server motherboard, which demodulates the input digital television signal with subsequent recording of the “ring” video signal into the corresponding cell of the server’s RAM and sequential conversion of the “ring” recording frame into “rectangular” frames, and the number of “rectangular” frames n, corresponding to one current “ring” frame, satisfies the relation: Where - the horizontal angle of the field of view in degrees of the image observed by the operator, and this transformation itself is performed by software, characterized in that a frame is introduced into the television camera in which a ball bearing and the specified photodetector are placed, and the rotation unit of the sensor block of the television camera contains series-connected a stepper motor and a stepper motor control unit, the input of which is connected to the output of the operator command demodulator, wherein the stepper motor shaft is directly or through a gearbox mechanically connected to the photodetector frame, providing, upon the operator’s command from the computer, a spatial rotation of the photodetector frame at an angle of 90° in step-by-step mode four times per cycle, so that its target, set by default symmetrically in front of the first panoramic lens, is successively set first symmetrically in front of the second panoramic lens, then symmetrically in front of the third panoramic lens, and finally symmetrically in front of the fourth panoramic lens , while the new digital video signal generated in the television camera after completion of each of the four rotations of the photodetector is alternately recorded in the corresponding RAM cell of the server, which is used as the system unit of the system operator’s computer. 2. The device of a computer system for panoramic television surveillance according to claim 1, characterized in that in the photodetector of the sensor unit of the television camera, the charge accumulation electrodes of the active pixels of the target, coinciding with the area of their photosensitive area, are made with a geometric shape in the form of part of a circular ring.