RU2734070C9 - Method of measuring spatial distance between small objects - Google Patents

Abstract

FIELD: measurement.

SUBSTANCE: invention relates to optical stereoscopic methods of determining relative location of objects in surrounding space. Method involves forming an optical image, determining the location of each small object, measuring distance between selected small objects, changing the viewing angle. Measurement of distances between small objects and determination of location of each small-sized object is carried out based on all full-format optical images of the area of the observed space with algorithmically calculated different viewing angles and the focusing distance of the optoelectronic system consisting of an optical part which converts the direction of the rays from the object to the plane of the electronic device for detecting energy rays, and the electronic part which detects the energy of the beams.

EFFECT: technical result is high accuracy of determining distance between small objects and resolution for objects when their images are placed in plane of photodetector.

1 cl, 7 dwg

Description

The invention relates to photo and video surveillance methods, and more specifically, to optical stereoscopic methods for determining the location of objects in the surrounding space, including determining the distances to the objects of interest and their angular coordinates. The invention is applicable, for example, in automatic systems for determining the location of objects in the surrounding space, and in particular in means for monitoring the location of remote space objects.

Prior Art

1. A known method for determining the temperature deformation of the sample and a device for its implementation [1]. The method consists in that the registration and modulation of the image of light marks is carried out in an optoelectronic system by moving the streaked grating perpendicular to the direction of the beams of light rays. This method allows to determine with high accuracy the distance between the sources of beams of rays, i.e. light marks (sources) with a previously unknown brightness in the conditions of background illumination.

The disadvantages of the method are: a narrow range of distances of the mutual arrangement of light marks (sources), the need to adjust the measuring system in the range of spatial arrangement of light marks (sources), the impossibility of directly determining the distances between the marks in the direction of the viewing angle of the optoelectronic system, relatively low speed and reliability of the system due to the presence of a mechanical modulation node of the streaked grating.

2. There is a known method for determining the distances between small-sized light marks (sources) by an optoelectronic system based on phase-raster measurements, which make it possible to determine the coordinates of light sources from laser designators [2], which consists in the fact that light sources are projected by the optical system onto a matrix photodetector ( matrix of photodiodes), based on the algorithm for summing signals from the photodiodes of the receiver, a sequence of signal samples is obtained in the area of each small-sized object, which is a light mark, based on a comparison of the received signal samples, the distance between the light marks is determined by the degree of their correlation. In order to expand the brightness dynamic range of light marks, reduce the saturation level of the receiver photodiodes, and also to increase the accuracy of measuring their coordinates, an additional optical system is used in this method, which forms a series of duplicate shifted images of light marks on the matrix. The measurement results are refined by averaging the measurements in the images of the corresponding main and backup light marks.

The disadvantages of this method are a narrow range of relative position of the light marks, a significant error associated with the discreteness and smoothness of the array of photodiodes of the receiver, and errors associated with optical aberrations of the optical system when forming an image of the area of the observed space, the need to adjust the measuring system in the range of location of the light marks, as well as limited possibilities for determining distances along the viewing angle of the optical system.

3. There is a method for determining the location of an object in the surrounding space [3]. The method consists in the formation of two panoramic images on the plane of a multi-element photodetector, which is part of the optoelectronic system, by an optoelectronic system, fixing on each image the image of the area of interest of the surrounding space observed from a given angle with an object located in it, registering the generated images, determining the location an object with an estimate of the distance to it and determining the distance to the object by the relative position of its two images on a simultaneously registered pair of spaced panoramic images.

The disadvantages of the method are the impracticality of observing small-sized light objects, due to the low resolution of the multi-element photodetector, since two panoramic images are simultaneously formed in the plane of one photodetector, which leads to an insufficient number of pixels per unit area of the observed space. At the same time, if you use a photodetector with a large number of pixels, then the level of performance decreases due to an increase in the amount of recorded and transmitted information, which does not allow for observation of moving objects (images of objects in the images are blurry or absent due to going beyond the boundaries of the optical system view area) . In addition, the selection of an object in the image is performed by contour analysis methods (for example, the discrete difference method using the Sobel operator) without additional background suppression operations, which leads to a decrease in the reliability of object selection under conditions of a non-uniform background.

As a prototype, a method was chosen for identifying and determining the relative position of light marks, including when they are partially superimposed, by an optoelectronic control system [4], which consists in the fact that light objects are projected by an optical system onto a matrix photodetector (matrix of photodiodes), and onto based on the summation algorithm of the product of signals received from the photodetector with a scalable shifted along the selected direction, and on the basis of a given function, a signal sample is formed containing information about all light objects in the observation area, and the distance between the observed light objects (marks) is determined by the difference in the coordinates of the extrema of the generated signal sample .

The disadvantages of the prototype are limitations on the accuracy of determining the coordinates of small-sized objects (light sources), due to the low resolution of the photodetector in the area of small-sized objects, since several images are simultaneously formed on the plane of the photodetector, which leads to an insufficient number of pixels per unit of each small-sized object. At the same time, if you use a photodetector with a large number of pixels or several photodetectors, then the level of performance decreases due to an increase in the amount of processing of the information received, which does not allow monitoring moving objects in real time. In addition, in the prototype, the selection of the object is performed by analyzing binary clusters formed from their images, without performing additional operations to improve the image quality, which leads to a decrease in the reliability of the selection of objects under conditions of a heterogeneous background.

The task to be solved by the present invention is to determine the distance between small-sized objects, which are light sources, in the surrounding space, located in the field of view of the optoelectronic system, in a wide range of their relative positions, with higher accuracy and reliability.

The essence of the invention lies in the fact that the area of interest of the surrounding space is observed by a composite optical system (part of an optical-electronic system) [11], which forms an array of sub-aperture images in the photodetector plane. In the process of forming an array of sub-aperture images in the plane of the photodetector, sets of beam paths along the directions of their propagation into the optoelectronic system and their spatial coordinates are recorded, which ensures the receipt of volumetric parameters of the observed objects (X _O , Y _O , Z _O ) (coordinates in space) [ 8], characterized by the distribution of brightness in the plane perpendicular to the axis of sight of the optoelectronic system (X _O , Y _O ) [2], as well as the distance of this plane from the optoelectronic system (along the axis Z _O ) [3]. This action is achieved through the use of a composite optical system, including the main projecting lens and an additional matrix optical system, consisting of an array of microlenses located after the main projecting lens in front of the matrix photodetector [9]. Panoramic full-size images of objects are formed algorithmically from an array of sub-aperture images with an algorithmically changed viewing angle of the optical system and its focusing distance to the object, while changing the viewing angle is carried out algorithmically based on the rule of summing signals from the photodiodes of the photodetector corresponding to the direction of the rays. The summation of images in a set with algorithmically variable viewing angles provides an increase in the number of significant pixels in the area of images of small-sized objects [8], which improves the accuracy of measuring distances between small-sized objects using the maxima of continuous wavelet transform curves.

The technical result of the invention is to increase the accuracy of determining the distance between small-sized objects and the resolution for objects when their images are superimposed in the plane of the photodetector, which makes it possible to determine with greater accuracy the distance between objects in the control zone and the movement of objects against a non-uniform background. The formation of a set of stereoscopic images for a series of viewing angles with high resolution leads to an increase in the accuracy of determining the distance between the observed small-sized objects and an increase in the measurement reliability due to selective suppression of local heterogeneity of the image of a small-sized object by varying the pixel structure with a change in the viewing angle, compensating for the scaling error by possible calibration of the optical system [5], re-pixelization in the region of the contrast gradient of the borders of a small-sized object, the possibility of partial selection and removal of an inhomogeneous background in the recorded image.

To achieve a technical result, a method for highly accurate determination of the distances between small-sized objects in the surrounding space is proposed. The method includes forming a set of small-sized panoramic images on the plane of the multi-element photodetector, containing in each image images of the area of interest of the surrounding space with small-sized objects located in it, observed from given angles. Synthesis of a series of full-length images corresponding to different viewing angles of the optical system makes it possible to determine the location of small-sized objects in each panoramic image with an estimate of the distance to them. In this case, the determination of the distance to small-sized objects is carried out by their location in the layers of space by a resolvable optical system.

, координаты отображаемых точек (х, у) определяются формулой (1):According to the invention, the area of interest of the surrounding space is observed by an optical system for image transfer into the plane of the matrix photodetector, consisting of a projecting lens and an array of microlenses (NxM) located in front of the matrix photodetector (Fig. 1). This provides an array of sub-aperture images on the photodiode array.

, the coordinates of the displayed points (x, y) are determined by formula (1):

where I (distance of the microlens array to the photodiode array) and L (distance from the lens to the microlens array) are the design parameters of the optical scheme, (X Y) are the coordinates in the coordinate system in the lens plane, (x, y) are the image coordinates of the points of the object (light marks) in the coordinate system in the plane of the photodetector, Z is the distance from the plane of the formed image (the plane of the photodetector) to the optoelectronic system.

синтезируют полноформатные изображения Im_k(х,у) наблюдаемого пространства (рис. 2) известным алгоритмом [6], которые фотоприемником представляются двухмерной выборкой сигналов

. Для обеспечения большей точности определения относительных координат световых меток расширяют границу градиента яркости световых меток путем суммирования ряда изображений Im_k(х,у) с различными алгоритмически изменяемыми углами визирования Im(x,y)=Σ Imk(x+q⋅Δx,y+p⋅Δy) (рис. 3, 4, 5), что позволяет обеспечить более высокую точность определения координат малоразмерного объекта в пространстве алгоритмом непрерывного вейвлет преобразования (НВП), который сканирует полученные полноформатные изображения наблюдаемого пространства (2), вдоль заданной координаты С(а,ξ) в каждом масштабе (а) и смещении в заданном направлении (ОХ - ξ) выбранной функции преобразования ψ(х):From an array of sub-aperture images

synthesize full-size images Im _k (x, y) of the observed space (Fig. 2) using a known algorithm [6], which are represented by a photodetector as a two-dimensional sample of signals

. To ensure greater accuracy in determining the relative coordinates of the light marks, the boundary of the brightness gradient of the light marks is expanded by summing a series of images Im _k (x, y) with different algorithmically variable viewing angles Im(x, y)=Σ Imk(x+q⋅Δx,y+ p⋅Δy) (Fig. 3, 4, 5), which allows for a higher accuracy in determining the coordinates of a small-sized object in space by the continuous wavelet transform (CWT) algorithm, which scans the received full-format images of the observed space (2), along a given coordinate С( а,ξ) in each scale ( а ) and shift in the given direction (ОХ - ξ) of the chosen transformation function ψ(х):

where ψ(x) is a function that has the following properties: the average value is zero, it has the properties of scalability and coordinate shift x.

The maxima of the CWP coefficients D(ξ) (3) [6] determine the relative coordinates of each small-sized source (i) along each selected direction (j) (4.5) [7, 8], the difference of which determines the distance d _ik between small-sized sources (i) and (k):

In the general case, for curves of the CWP coefficients [7] for a given function ψ(x) that have several maxima or minima d ⁿ , the linear size d _ik can be determined by different maxima or with greater accuracy by the totality of the corresponding extrema (maxima or minima) coefficient curves of CWP D(x) in several ways (6) [8]:

here n is the number of maxima (determined by the type of function ψ(x), distance), and _i is the amplitude of the maxima of the CWP coefficients, which is used as a weighting factor in formula (6) (the smaller the amplitude, the higher the discretization error).

Confirmation of the results was carried out on the developed virtual device [10] for modeling the distribution of illumination in shifted light marks (Fig. 6, 7).

In the available sources of information, no technical solutions were found containing in aggregate signs similar to the distinguishing features of the proposed method. Therefore, the invention meets the criterion of inventive step.

The presence of new essential features, together with the known and common with the prototype, made it possible to create a new technical solution - a method for measuring the spatial distance between small-sized objects. The method comprehensively solves the problem of increasing the speed and reliability of signal processing while maintaining high visibility and providing an increase in resolution.

Information sources

1) Patent No. 1392354 dated 04/30/1988, class. G01B 11/16. A method for determining the temperature deformation of a sample and a device for its implementation, authors: V.E. Makhov, A.I. Potapov, (USSR). - No. 1392354; Appl. 03/24/1986; Published 04/30/88, Bull. No. 16.

2) Makhov V.E. Influence of the characteristics of laser designators on the accuracy of measuring their coordinates // Proceedings of the Twelfth All-Russian Scientific and Practical Conference of the RARAN: Actual problems of protection and security, (April 1-3, 2009), volume 3. - St. Petersburg: NPO Special Materials, 2009. - C 281-288.

3) Patent of the Russian Federation No. 2420774 dated February 4, 2009, class. G03B 37/04, G02B 13/06. A method for determining the location of an object in the surrounding space and panoramic equipment for implementing the method, authors: Podgornov V.A., Podgornov S.V., Shcherbina A.N.

4) A method for identifying and determining the position of light marks when they are superimposed in optical control systems Control. Diagnostics. 2019. No. 7 (253). P. 4-13, authors Makhov V.E., Shirobokov V.V., Potapov A.I.

5) Bok Y., Jeon H.-G., Kweon I. S. Geometric Calibration of Micro-Lens-Based Light-Field Cameras using Line Features // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017 Vol. 39/Issue 2.pp. 17-61.

6) Ng R. Digital light field photography. A dissertation submitted to the department of computer science and the committee on graduate studies of Stanford university in partial fulfillment of the requirements for the degree of doctor of philosophy, 2006.

7) Makhov V.E. Investigation of wavelet-transformation algorithms for determining the coordinates of light marks // Problems of radio electronics. TV Technique series, no. 2. - St. Petersburg: FSUE NIIT, 2012. - S. 78-89.

8) Gruzman I.S. Digital Image Processing in Information Systems: Textbook / I.S. Gruzman, B.C. Kirichuk, V.P. Kosykh, [and others] - Novosibirsk: Publishing house of NGTU, 2000. - 168 p.

9) Makhov V., Potapov A., Zakutaev A. Principles of operation of light field digital cameras with an array of microlenses // Components and technologies. 2018. No. 1 (198). pp. 14-20.

10) Makhov V.E. Control of the linear dimensions of products based on the technologies of the company "NATIONAL INSTRUMENTS)) / V.E. Makhov // News of higher educational institutions. Instrumentation. 2010. V. 53, No. 7. pp. 54-60.

11) Makhov E.M. Applied Optics: Proc. allowance / E.M. Makhov, A.I. Potapov, V.E. Makhov - St. Petersburg: SZTU, 2004. - 348 p.

Claims (1)

A method for measuring the spatial distance between small-sized objects in the surrounding space, including the formation of an optical image, determining the location of each small-sized objects, measuring the distance between selected small-sized objects, changing the viewing angle, characterized in that measuring the distances between small-sized objects and determining the location of each small-sized object is carried out on based on all full-length optical images of the area of the observed space with algorithmically calculated different viewing angles and focusing distances of the optoelectronic system, consisting of the optical part, which converts the direction of the rays from the object into the plane of the electronic device that registers the energy of the rays (matrix photodetector), and the electronic part, which registers the energy of the rays (matrix photodetector), while the formation of optical images is carried out algorithmically based on the a rowed group of sub-aperture small-sized images obtained by an additional matrix optical system that complements the main projecting optical system that forms an image of small-sized objects of the observed space and consists of an array of microlenses located after the lens in front of the matrix photodetector, by registering the totality of the path of rays in the directions of their propagation into the optical-electronic a system that allows fixing the volumetric parameters of spatially separated small-sized objects and combining sub-aperture small-sized images with different viewing angles to increase the number of significant pixels in each image area of the observed small-sized objects, while changing the viewing angle is carried out algorithmically based on the rule of summing signals from the photodiodes of the photodetector corresponding to the direction of the rays.

Images