RU2740435C2 - Method of determining position of region of searching for matches on distortion-degraded images - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to the field of information-measuring systems, in particular, systems of technical vision, and is intended for solving problems of automating search of correspondences on two or more digital images. Disclosed is a method of determining the position of the search region of matches on distortion-degraded images. To implement the disclosed method, at least two spaced apart (in any plane) can be used and randomly oriented digital j^th video cameras, which enable to capture stereoscopic images of objects of interest and together with a processing system, generating computer vision system. Digital cameras used can be both visible and infrared ranges. First of all, the invention is aimed at use in low-power mobile systems which require real-time operation with distortion-degraded high-resolution images without preliminary processing thereof.

EFFECT: technical result consists in improvement of speed of automatic matching on distortion-degraded images with dynamically changing their spatial position and orientation of high-resolution digital cameras.

1 cl, 6 dwg

Description

The invention relates to the field of information-measuring systems, in particular, technical vision systems and is intended to solve the problems of automating the determination of the position and boundaries of search areas for matching on two or more digital images. The invention is primarily focused on the use in low-power mobile systems, involving real-time operation with distortion-distorted high-resolution images without their preliminary processing.

When developing technical vision systems (STS) for spatial positioning (measurement of motion parameters, coordinates, dimensions, etc.) of both the system itself and objects observed through its digital cameras, one of the main requirements is the implementation of measurements in real time, high the accuracy of the measurement results and the computational feasibility of the applied automatic determination of the position and boundaries of the search areas for matching on two or more digital images. The latter requirement is especially relevant when implementing mobile STZs based on computers with limited computing power. Fulfillment of this requirement largely determines the error of the measuring system and the possibility of conducting accurate and high-speed comparison of the corresponding images of objects of interest on a set of digital image frames.

Currently, a distinction is made between global and local matching.

Global search is based on minimizing the energy functional, when the discrepancy is found at once for a set of points (objects) of the processed images. As a rule, such tasks are associated with a three-dimensional reconstruction of the observed scene from a variety of different-angle images. A common disadvantage of global search is high computational complexity.

The peculiarity of local search is that the search for matches in it is organized by sequential scanning between local areas of images, as a rule, this is a certain area of interest in one image and a search area in another image.

In turn, the sizes of the search areas and the order of their finding will determine the search accuracy and the requirements for the computing power of the computer.

The simplest way to implement the method for determining the position of the search area (in local search) can be applied in STZ, which involves the reception of digital images from at least two identical videos or cameras located in space strictly parallel to each other and on the same baseline [Zubar A. V., Sidorenko A.A., Tishin S.A., Shcherbo A.N. Analysis of ways to search for matches in images for low-power vision systems // National priorities of Russia. Ser. 1. Science and military security. 2017. No. 4 (11). S. 5-12.]. In this case, if on image 1 of the left camera area 4 with the center at point 3 is highlighted, then on image 2 of the right camera, area 6 of the image of this object centered at point 5 will have the same vertical coordinates as on image 1 (Fig. 1). Then the search area 7 on image 2 can be limited by a line from point 8 to point 9. Point 8 characterizes the nearest, and point 9 - the infinitely distant boundaries of the search area 7. In this case, the pixel coordinates n _{j + 1} and m _{j + 1 of the} remote boundary 9 in image 2 will be equal to the pixel coordinates n _j and m _{j of the} center 3 of region 4 characterizing the position of the object of interest in image 1. The nearest border 8 of the search area 7 with this arrangement of cameras will be on the left edge of image 2 (n _{j + 1} = 0) with the same vertical coordinate m _{j + 1} as at the boundary 9.

The disadvantage of this method for determining the position of the match search area is the limited use of it due to the need to use only exactly the same cameras, and the high requirements for ensuring their exact parallel placement in space. The latter is especially critical for the use of STZ on moving mobile platforms, where shock and vibration loads can lead to vibrations and deformations of the body, including at the points of camera attachment, which makes the condition of ensuring absolute parallelism of the cameras, if not practically, then difficult to implement.

Leveling the requirement to ensure parallel placement of cameras (and only in their vertical and horizontal mutual orientation) can be achieved by expanding the search area 7, while the maximum search area 7 can cover almost the entire image 2 (Fig. 2). However, in this case, the number of computational operations increases many times, and, accordingly, the requirements for the computing power of the computer grow. Also, the probability of a false search result is quite high due to the possibility of finding objects identical in shape, angle and color in the frame.

All this leads to the fact that today these methods are practically not applicable, especially in those cases when it is necessary to carry out measurements in real time from randomly located and different in their technical parameters digital video or cameras based on a low-power computer.

An analogue to the claimed invention is a method for determining the position of the matching search area on digital images based on their rectification [Zubar A. V., Sidorenko A. A., Tishin S. A., Shcherbo A. N. Analysis of ways to search for matches in images for low-power vision systems // National priorities of Russia. Ser. 1. Science and military security. 2017. No. 4 (11). S. 5-12, Pyankov, D.I. Spatial processing of unsynchronized video sequences based on rectification of frames // Software products and systems. - No. 1 (101) - Tver: Research Institute "Tsentrprogrammsystem", 2013, p. 61-66]. The essence of rectification lies in the reprojection of the planes of images 1 and 2 so that they are in the same plane. As a result, the search area 7 is converted to a horizontal line (Fig. 3). In this case, if images 1 were obtained from the left camera, and image 2 from the right, then the near border 8 of the search area 7 will be located on the left edge of image 8, and the far border 9 - on its right edge. The vertical coordinate m _{j + 1} , which specifies the position of the search area 7 in image 2, is determined by the corresponding coordinate m _{j of the} center 3 of the area of interest 4 in image 1.

There are several types of rectification, for example, planar, polar. Together with this, the general disadvantages of organizing a search for rectified images are:

- the need to establish with high accuracy all internal, including random, camera parameters;

- the complexity of real-time calculations for high-resolution images due to the large number of calculations associated with full pixel-by-pixel processing of a pair of images;

- distortions of images associated with the fact that when eliminating projective distortions using rectification in one form or another, interpolation of readings is always carried out, since in fact, after this stage, the search for matches is carried out “on other” images that differ from the original ones;

- the limited possibility of using rectification, in particular, when processing distortion-distorted and "multi-angle" images after rectification, the images of objects of interest can be so deformed that the calculation of the similarity measure by itself becomes impossible.

As a prototype to the claimed invention, the selected method for determining the position of the search area for matches on digital images along the epipolar lines [A. Zubar, A. Sidorenko, S. A. Tishin, A. Shcherbo. Analysis of ways to search for matches in images for low-power vision systems // National priorities of Russia. Ser. 1. Science and military security. 2017. No. 4 (11). S. 5-12, Goshin, E.V. Model of reconstruction of 3D-scenes taking into account epipolar constraints // Young scientist. - 2014. - No. 12. - S. 71-73. - URL https://moluch.ru/archive/71/12174/ (date of access: 10.01.2020)]. Finding the position of epipolar lines is based on the model of epipolar geometry, according to which all epipolar straight lines pass through the epipole, and the set of epipolar planes is a one-parameter family of planes. And if an object is indicated in image 1 (Fig. 4), then for a given relative position of the cameras in image 2 there is only one line 7 on which the image of this object can be found.

An obvious advantage of this approach to determining the position of the search area in comparison with rectification is the absence of the need for pixel-by-pixel image processing. And from this follows a decrease in the number of computational operations and an increase in the reliability of the search result, since the search is carried out according to the original images.

On the other hand, when finding epipolar lines, as well as during rectification, the fundamental matrix must be calculated first, for the construction of which it is necessary to know exactly all the internal parameters of the chambers and their mutual orientation. Consequently, the productivity of this method in the case when the internal parameters of the cameras are either unknown at all, or are known only to a limited extent, and the cameras themselves constantly change their orientation, and the search must be carried out using the video sequence in real time, will be reduced.

Another important drawback is related to the fact that the epipolar geometry model does not take into account distortion distortions of images. So, figure 4 shows that if images 1 and 2 are distorted by radial and tangential distortion, the search area 7 should be a curve (Fig. 5). The model of epipolar geometry does not imply the construction of curves, and, therefore, to find the corresponding epipolar line, it is necessary either to use high-quality and expensive optics or to perform image correction, again associated with their complete pixel-by-pixel processing. This greatly increases the number of computational operations and the search time, and also requires a high-performance STZ with a powerful graphics processor to perform measurements in real time. And for low-power mobile STZs, taking into account the provision of measurements on distorted high-resolution images in real time, this method of determining the position of the search area may not be applicable.

To determine the position of the search area 7 in image 2 along the epipolar line, the epipolar plane is assumed to be dimensionless. In this connection, the search area 7 runs through the entire image 2. And for most shooting cases this is quite true. But at the same time, in some situations, the relative orientation of the cameras may turn out to be such that the search area 7 should not pass through the entire image (Fig. 5). In this case, the near boundary 8 of the search area 7 can be set from the condition of the minimum distance to the object of interest, and the far boundary 9 - by the maximum distance at which the measurement errors do not exceed a certain set value, or the range beyond which all objects will be considered as equidistant. The introduction of such additional restrictions will further help to reduce the size of the search area, and, as a consequence, to reduce the number of computational operations during the direct search.

Thus, the problem to be solved by the claimed invention is to develop a method for automatically determining the position of the search area bounded on both sides and adaptive in the shape of the distortion of the search area, provided that it can be quickly reconstructed in cases of dynamic change in the relative position in the space of cameras with unknown internal parameters.

The solution to this problem is provided by:

и максимальной

дальностей измерений, а также отсутствия необходимости предварительной попиксельной переработки изображений, вычисления фундаментальной матрицы и решения сложных систем уравнений.first, the high speed of rebuilding the position and shape of the search area when changing the relative position of cameras in space due to the introduction of additional restrictions on the possible position in space of the object of interest in the form of a minimum

and maximum

measurement ranges, as well as the absence of the need for preliminary pixel-by-pixel processing of images, calculation of the fundamental matrix and solution of complex systems of equations.

secondly, by adapting the shape of the search area to distortion distortions of lenses, errors in the production, assembly and operation of cameras, as well as the ability to conduct a search in conditions when only the resolution of the processed images is known, through the use of expressions that allow (Fig. 6):

в системе координат (СК)

j-ой камеры на основании выражения"Direct" transition from pixel coordinates n _j and m _{j of the} image of the object of interest to its metric three-dimensional coordinates

in coordinate system (SC)

j-th camera based on the expression

центра изображения интересующего объекта на плоскости изображения j-ой камеры, условно отстоящего от оптического центра

ее объектива на расстоянии

where A _j is a vector of three-dimensional coordinates

the center of the image of the object of interest on the image plane of the j-th camera, conditionally distant from the optical center

her lens in the distance

n ' _j = n _j -0.5N _j , m' _j = m _j -0.5N _j are the given pixel coordinates of the object image on the image of the j-th camera;

объектива j-ой камеры, вычисляемые согласно выражений:α _j and β _j - angles, respectively, in the horizontal and vertical planes to the object of interest relative to the optical axis

lens of the j-th camera, calculated according to the expressions:

N _j and M _j - horizontal and vertical image resolution of the j-th camera;

c ₀ , c ₁ , c ₂ , ... c _q and d ₀ , d ₁ , d ₂ , ..., d _q are the polynomial coefficients of the "direct" transformation of the functions ƒ (n ' _j ) and ƒ (m' _j ), respectively,

к плоским пиксельным координатам n_j и m_j на основании выраженияand "reverse" transition from metric three-dimensional coordinates

to flat pixel coordinates n _j and m _j based on the expression

Where

и

- полиномиальные коэффициенты «обратного» преобразования функций ƒ(α_j) и ƒ(β_j) соответственно,Where

and

are the polynomial coefficients of the "inverse" transformation of the functions ƒ (α _j ) and ƒ (β _j ), respectively,

в СК

камеры при работе с дисторсионно-искаженными цифровыми изображениями не соответствующими своим разрешением физическому разрешению фотоприемных устройств и применение камер с неизвестными техническими параметрами, в том числе с учетом возможных погрешностей, допущенных при их изготовлении или возникших по вине эксплуатации. При чем вычисляться коэффициенты с₀, с₁, с₂, …c_q, d₀, d₁, d₂, …, d_q

и

полиномов (2), (3) и (5), (6) должны для каждой j-ой камеры индивидуально при ее производстве и сборке, или уточняться непосредственно перед применением измерительной системы путем описания полиномами q-го порядка взаимосвязей между вертикальными и горизонтальным углами на интересующий объект относительно j-ой камеры и соответствующими значениями его пиксельных координат на принимаемых с j-ой камеры цифровых изображениях.In this case, the use of polynomials "direct" (2), (3) and "reverse" (5), (6) transformation provides the establishment of a mathematical relationship between the pixel coordinates n _j , m _{j of the} object image and its three-dimensional coordinates

in the UK

cameras when working with distortion-distorted digital images that do not correspond to their resolution to the physical resolution of photodetectors and the use of cameras with unknown technical parameters, including taking into account possible errors made during their manufacture or arising from the fault of operation. Moreover, the coefficients are calculated with ₀ , with ₁ , with ₂ ,… c _q , d ₀ , d ₁ , d ₂ ,…, d _q

and

polynomials (2), (3) and (5), (6) must for each j-th chamber individually during its production and assembly, or be refined immediately before using the measuring system by describing the relationships between the vertical and horizontal angles by polynomials of the q-th order to the object of interest relative to the j-th camera and the corresponding values of its pixel coordinates on the digital images received from the j-th camera.

The main technical result provided by the given set of features are:

- reducing the requirements for computing resources of the measuring computer system (PC);

- increasing the speed of automatic search for matches on distortion-distorted images from high-resolution digital cameras dynamically changing their spatial position and orientation;

In addition, the technical result can be attributed to a reduction in cost and greater ease of implementation of the measuring system due to the possibility of using cheap household "non-metric" cameras, including those with unknown technical parameters.

To implement the claimed method, at least two spaced apart in space (and in any plane) and arbitrarily oriented digital j-th video cameras 10 and 11 (Fig. 6) can be used, providing the ability to capture stereo images of objects of interest and together with the processing system (on Fig. 6 is not shown) forming the STZ. Moreover, the digital cameras used can be both in the visible and infrared ranges.

Each of the j-th cameras used, for example, camera 10 and 11 (Fig. 6), can be mounted on its gimbal, which provides the ability to change the angular orientation of the camera (depending on the need) in the horizontal, vertical and transverse planes. Each of the gimbals, in turn, may additionally contain angle sensors that obtain the values of the camera orientation angles (depending on the gimbal design) in the horizontal, vertical and transverse planes. In this case, the camera 10 and the camera 11, the sensors of the angle of rotation of the suspension frames should be made with the possibility of transmitting video data and data on the spatial orientation of the cameras to the processing system through cables (for example, cables of the universal serial bus USB) or wirelessly (for example, Wi-Fi ).

При этом начало

каждой из СК

располагается в оптическом центре объектива j-ой камеры. Ось

направляется строго по оптической оси объектива, ось

- влево относительно направления оси

вдоль строк цифровых изображений, ось

- вверх вдоль столбцов изображений. Так на фиг. 6 в качестве примера показано размещение двух СК 12

и 13

для камер 10 и 11.The current position and orientation of each of the j-th cameras are determined by their coordinate systems (SC)

At the same time, the beginning

each of the UK

is located in the optical center of the lens of the j-th camera. Axis

is directed strictly along the optical axis of the lens, the axis

- to the left relative to the direction of the axis

along lines of digital images, axis

- up along the columns of images. Thus, in FIG. 6 shows as an example the placement of two SK 12

and 13

for chambers 10 and 11.

In turn, to describe the spatial position and orientation of SC 12 and 13 relative to the initial SC 17 O _w X _w Y _w Z _w use matrices

Where

- углы в горизонтальной, вертикальной и поперечной плоскостях ориентации осей СК 12 и 13 относительно соответствующих осей СК 17;

- angles in the horizontal, vertical and transverse planes of orientation of the axes SK 12 and 13 relative to the corresponding axes SK 17;

- координаты начала СК 12 и 13 заданные в СК 17.

- coordinates of the beginning of SC 12 and 13 given in SC 17.

Digital images with a resolution of N _j × M _j received from each of the j-th cameras (Fig. 6) are composed of pixels. Each pixel is characterized by a value, which consists of a grayscale value or a color value. In grayscale images, a pixel value is a single value that characterizes the brightness of a pixel. The most common format for describing a pixel is an image byte, in which the pixel value is represented by an eight-bit integer that ranges from 0 to 255. Typically, a pixel value of zero is used to denote a black pixel, and a value of 255 is used to denote white. pixel. Intermediate values describe different shades of midtones. In color images, to describe each pixel (located in an RGB color space - red, green, blue), the red, green and blue components must be separately defined. In other words, the pixel value is actually a vector described by three numbers. The three different components can be saved as three separate halftones, known as color planes (one each for red, green, and blue), which can be reconnected when displayed or processed.

СК 12 и 13 на расстояниях

(Фиг. 6). При этом строки изображений 1 и 2 параллельны осям

а столбцы - осям

Digital images 1 and 2 with their centers 15 and 16 are taken placed on the optical axes of their lenses in the positive directions of the axes

SK 12 and 13 at distances

(Fig. 6). In this case, rows of images 1 and 2 are parallel to the axes

and columns to axes

If, for example, an object 14 is in the field of view of the camera 10 (Fig. 6), then on the digital image 1 the image 3 of this object will correspond to a pixel whose position in the pixel SC of image 1 will be characterized by the number of the column n _j and the row m _j . Similarly for the camera 11 on its image 2 the coordinates of the image 5 of the object 14 will correspond to n _{j + 1} and m _{j + 1} .

The processing system, which is, for example, a remote computer, such as a laptop or a personal computer (workstation), must provide a user selection of images and / or input processing commands and contain, in turn, executable modules or commands capable of being executed by at least one processor, a user interface containing a display, such as a liquid crystal monitor, for viewing video data, and a control and data input device such as a keyboard or pointing device (for example, a mouse, trackball, stylus, touch pad, or other device) to provide interaction user with video data.

The essence of the invention is illustrated by drawings, which do not cover and even more so do not limit the entire scope of the claims of this invention, but are only illustrative materials of a particular case of implementation, in which:

in fig. 1 illustrates a method for determining the position of a search area using identical and parallel cameras;

in fig. 2 illustrates a method for determining the position of the search area using the same cameras, but randomly located in the vertical and horizontal planes;

in fig. 3 illustrates a method for determining the position of the matching search area during preliminary rectification of images of both cameras;

in fig. 4 illustrates a method for determining the position of a matching region along epipolar lines;

in fig. 5 illustrates the developed method for determining the position of the search area;

in fig. 6 shows a diagram of the relative position of CC cameras and their images, and also reflects the possible position of the image of an object of interest on the image of one camera and the search area for matches on the image of the second camera.

The claimed method is carried out as follows (Fig. 5-6).

трехмерных координат изображения 3 в СК 12 камеры.On the image 1 of the camera 10, in manual or automatic modes, the pixel coordinates n _j and m _{j of the} image 3 of the object of interest 14 are set, on the basis of which, according to expressions (1-3), the vector is calculated

three-dimensional coordinates of the image 3 in the SK 12 camera.

и максимальной

дальностей измерений относительно СК камеры 12.Sets the values of the minimum

and maximum

measurement ranges relative to the SC camera 12.

до

на i интервалов и вычисляют шаг h_i, одного интервала согласно выражения

Break up the distance from

before

into i intervals and calculate the step h _i , one interval according to the expression

Calculate the matrix M _{j of the} positions of the object 14 in the SC 12 of the camera 10

и

в соответствии с выражением (7).Based on the data from the gimbal sensors on which the cameras are installed, or according to their current external calibration, position matrices are calculated

and

in accordance with expression (7).

Calculate the matrix M _{j + 1} positions of the object 14 in the SC 13 of the camera 11

The matrix M _{j + 1 is} projected onto the image plane 2 of the camera 11, for which, taking into account the values of the matrix M _{j + 1, the} matrix is calculated

определяет положение ближней границы 8, а вектор

- положение дальней границы 9 области поиска 7 на изображении 2.where vector

defines the position of the near border 8, and the vector

- the position of the far boundary 9 of the search area 7 in image 2.

то любым из известных способов аппроксимируют значения первого и второго столбцов матрицы

полиномом

g-го порядкаIf

then any of the known methods approximate the values of the first and second columns of the matrix

polynomial

gth order

where a ₀ , a ₁ , a ₂ ,… a _g are the polynomial coefficients of the function

то аппроксимируют значения первого и второго столбцов матрицы

полиномом

If

then the values of the first and second columns of the matrix are approximated

polynomial

(12)

(12)

where b ₀ , b ₁ , b ₂ ,… b _g are the polynomial coefficients of the function

Using expression (4), the pixel coordinates of the near 8 (vector P _{j + 1,0} ) and far 9 (vector P _{j + 1, i} ) boundaries of the search area 7

то генерируют ряд

номеров столбцов от n_j+1,0 до n_j+1,i шагом в один пиксель, при этом если n_j+1,0≤0, то n_j+1,0 принимают, равным нулю, если n_j+1,i≥N_j+1-1, то n_j+1,i принимают равным N_j+1-1.If a polynomial was obtained

then generate a series

of column numbers from n _{j + 1.0} to n _{j + 1, i} in one pixel increments, while if n _{j + 1.0} ≤0, then n _{j + 1.0 is} taken equal to zero, if n _{j + 1 , i} ≥ N _{j + 1} -1, then n _{j + 1, i is} taken equal to N _{j + 1} -1.

Далее на оснований значений L_α вычисляют ряд

According to expression (2), for the values of the series L _{n, the} series

Further, based on the values of L _{α, the} series

записывают ряд

The values of the series L _{x are} substituted into expression (11), while the calculated values

record a number

Convert the values of the series L _y to the series L _m in accordance with the expression

то генерируют ряд

номеров строк от m_j+1,0 до m_j+1,i, при этом если m_j+1,0≤0, то m_j+1,0 принимают, равным нулю, если m_j+1,i≥M_j+1-1, то m_j+1,i принимают равным M_j+1-1.If a polynomial was obtained

then generate a series

line numbers from m _{j + 1.0} to m _{j + 1, i} , and if m _{j + 1.0} ≤0, then m _{j + 1.0 is} taken equal to zero, if m _{j + 1, i} ≥M _{j + 1} -1, then m _{j + 1, i is} taken equal to M _{j + 1} -1.

Далее на оснований значений L_β вычисляют ряд

According to expression (3), for the values of the series L _{m, the} series

Further, based on the values of L _{β, the} series

записывают ряд

The values of the series L _{y are} substituted into expression (12), while the calculated values

record a number

Convert the values of the series L _x to the series L _n according to the expression

At the final stage, after calculating the series L _n and L _m , a search array is formed from their values

with rounded to the nearest integer values of pixel coordinates n _{j + 1, i} and m _{j + 1, i} , in their totality, determining the position additionally limited on both sides and adapted in shape to distortion distortions of the lens of the search area 7 in the image 2 j + 1- th camera 11.

Claims (60)

A method for determining the position of the search area for matches on distortion-distorted images, which consists in receiving digital images from at least two j-th and j + 1-th video cameras and their further processing in order to find the search area of the object of interest in the j + 1-th image camera by its known pixel n _j , m _j coordinates on the image of the j-th camera, where n _j is the column number, m _j is the row number, characterized in that when determining the position and boundaries of the search area of the object of interest on the image of the j + 1-th camera according to its known pixel coordinates n _j and m _j on the image of the j-th camera, the vector A _{j of the} three-dimensional coordinates of the image of this object in the coordinate system ( CK) of the j-th chamber according to the expression:

where A _j is a vector of three-dimensional coordinates

center of the image of the object of interest on the image plane of the j-th camera, conventionally spaced from the optical center

her lens in the distance

- the given pixel coordinates of the object image on the image of the j-th camera; α _j and β _j - angles, respectively, in the horizontal and vertical planes to the object of interest relative to the optical axis

lens of the j-th camera, calculated according to the expressions:

N _j and M _j - horizontal and vertical image resolution of the j-th camera; s ₀ , s ₁ , s ₂ , ... c _q and d ₀ , d ₁ , d ₂ , ... d _q - polynomial coefficients of the "direct" transformation of functions

and

respectively; set the values of the minimum

and maximum

ranges of measurements of the relative SC of the j-th camera; split the distance from

before

into i intervals and calculate the step h _{i of} one interval

calculate the matrix M _j of object positions in the SC of the j-th camera

position matrices are calculated based on the data received from the gimbal sensors on which the cameras are installed, or according to the data of their current external calibration

and

according to expression

Where

and

- angles in the horizontal, vertical and transverse planes of the orientation of the axes of the spacecraft, respectively, of the j-th and j + 1-th cameras relative to the corresponding axes of the initial (world) spacecraft;

and

- coordinates of the beginning of the CS of the j-th and j + 1-th cameras, specified in the initial external CS; calculate the matrix M _{j + 1 of the} positions of the object of interest in the SC j + 1-th camera

the matrix M _{j + 1 is} projected onto the image plane of the j + 1-th camera, for which, taking into account the values of the matrix M _{j + 1, the} matrix is calculated

where vector

defines the position of the near border, and the vector

- the position of the far boundary of the search area on the image of the j + 1st camera; while if

then any of the known methods approximate the values of the first and second columns of the matrix

polynomial

gth order

where a ₀ , a ₁ , a ₂ ,… a _g are the polynomial coefficients of the function

if

then the values of the first and second columns of the matrix are approximated

polynomial

where b ₀ , b ₁ , b ₂ ,… b _g are the polynomial coefficients of the function

calculate pixel coordinates (n _{j + 1.0} , m _{j + 1.0} and n _{j + 1, i} , m _{j + 1, i} ) near (vector P _{j + 1.0} ) and far (vector P _{j + 1 , i} ) the boundaries of the search area on the image of the j + 1st camera

where ƒ (α _{j + 1} ) and ƒ (β _{j + 1} ) are the “inverse” transformation functions calculated according to the expressions

q - order (degree) of polynomials (13) and (14);

and

- polynomial coefficients of the "inverse" transformation of the functions ƒ (α _{j + 1} ) and ƒ (β _{j + 1} ), respectively; if a polynomial was obtained earlier

then generate a series

of column numbers from n _{j + 1.0} to n _{j + 1, i} in one pixel increments, while if n _{j + 1.0} ≤0, then n _{j + 1.0 is} taken equal to zero, if n _{j + 1 , i} ≥N _{j + 1} -1, then n _{j + 1, i is} taken equal to N _{j + 1} -1; according to expression (2) for the values of the series L _{n, the} series

then, based on the values of L _{α, the} series

substitute the values of the series L _x into expression (9), while the calculated values

record a number

convert the values of the series L _y to the series L _m in accordance with the expression

if the polynomial was obtained

then generate a series

line numbers from m _{j + 1.0} to m _{j + 1, i} , and if m _{j + 1.0} ≤0, then m _{j + 1.0 is} taken equal to zero, if m _{j + 1, i} ≥M _{j + 1} -1, then m _{j + 1, i is} taken equal to M _{j + 1} -1; according to expression (3) for the values of the series L _m calculate the series

Next, based on the values of L _{β, the} series

substitute the values of the series L _y into expression (10), while the calculated values

record a number

convert the values of the series L _x to the series L _n according to the expression

at the final stage, after calculating the series L _n and L _m , a search array is formed from their values

with rounded to the nearest integer values of pixel coordinates n _{j + 1, i} and m _{j + 1, i} , in their totality, determining the position of the search area additionally bounded on both sides and adapted in shape to distortion distortions of the lens in the image of the j + 1-th camera ...

Images