RU2420774C2 - Method of determining position of object in space and panoramic apparatus for implementing said method - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: method involves two-aspect reflection from a convex light-reflecting surface which directs light to an optical system for transferring an image to the plane of a multi-element photodetector, recording the image and determining coordinates of the object. Two spaced-apart panoramic images with the object, from which the distance to the object is determined, are obtained on the plane of the photodetector. The panoramic apparatus has a convex mirror with a light-reflecting surface which captures an all-round view of the monitored space in a given solid angle and formed by two bodies of rotation running into one another, an optical transfer system, a multi-element photodetector and a device for recording and processing the obtained image. The bodies of rotation have a common axis of rotation which coincides with the principal optical axis of the objective lens of the optical image transfer system. Geometric parametres of the bodies of rotation facilitate display of two different-aspect panoramic images on the plane of the photodetector.

EFFECT: high accuracy of determining distance to an object while maintaining the possibility of viewing the object in a wide solid angle.

3 cl, 6 dwg

Description

The invention relates to panoramic optical means for detecting objects in the surrounding space in a given solid angle.

Known methods for providing image formation with a large field of view so that you can get as much information as possible about the changes occurring around. In particular, this concerns the need for recognition of emerging objects with simultaneous determination of the distance to them.

For this purpose, you can use a camcorder with a wide-angle lens, for example FISH EYE (fisheye lens), but it also has an insufficient field of view.

A known method of forming a panoramic image, proposed by Apple Computer, using a conventional 35 mm camera and QuickTime.RTM.VR (QTVR). In accordance with this method, a camera equipped with a wide-angle lens (focal length is 15-18 mm) is mounted on a tripod rotated around the vertical axis and 12 or more pictures are sequentially recorded every 30 degrees, thus covering a circular view. Then the pictures are “stitched” using a special program, forming a panoramic image. However, this process does not provide a one-time observation mode for the object, because Each shot is taken at a different time. In addition, the presence of mechanical camera movement is also a definite disadvantage.

There is also known a method of forming a panoramic image, in accordance with which they use a lot of flat reflective surfaces in conjunction with many cameras facing them, and each camera considers a little more than 90 degrees of the 50-degree solid angle of the visualized space (see article VS Nalwa "A True Omni -Directional Viewer ", AT&T Bell Laboratories Technical Memorandum, BL0115500-960115-01, Jan. 1996, as well as European Patent Applications No. 09982946, 1056292, Applicant: LUCENT TECHNOLOGIES INC., Inventor VS Nalwa, IPC G02B 27/14, G03B 37/00).

The disadvantage of this method is the need to have many sensing sensors in order to simultaneously capture a 360-degree image of the surrounding space.

There are also known methods of forming a panoramic image of the surrounding space on the surface of a single photodetector, which receives light reflected from a mirror surface in the form of a body of revolution through an optical image transfer system. In this case, the body of revolution can have various shapes. In particular, it can be a cone or a sphere (see, for example, international application WO 2001/044870, priority date 12/14/2000, applicant UNIVERSITE DE MONTREAL, IPC G03B 37/00, 37/04).

A similar method described in US patent No. 6542184, priority from 08.03.2000, IPC G02B 13/06, G03B 37/00, Driscoll Jr. et al., can be, inter alia, used to determine the location of an object in the surrounding space in a recorded panoramic image. This method is selected and the prototype of the proposed method.

The essence of the known method consists in forming on the photosensitive surface of a flat multi-element photodetector a panoramic image of the observed portion of the surrounding space using a convex reflective surface shaped like a body of revolution, directing light reflected from it from the surrounding space to the optical system for transferring the observed portion of the surrounding space to the photosensitive surface of the photodetector, in registration of the generated image and in the definition of dinat of the object of interest in the surrounding space.

The disadvantage of this method is the low accuracy of determining the distance to the object, if its dimensions are not known in advance, since in this case it is impossible to accurately compare the remoteness of the object with its dimensions in the image.

The task is to create a method that eliminates this drawback while maintaining the ability to observe the object in the surrounding space in a wide solid angle.

The problem is solved as follows.

In the known method, which consists in forming on the plane of a multi-element photodetector a panoramic image of the observed area of space using a convex reflective surface having the shape of a body of revolution directing the scattered light reflected from it to the optical system for transferring the image of the observed area of space to the photodetector plane, in registering the formed image and in determining the coordinates of the object of interest, according to the invention form a two-way reflection from the a reflective surface, while on the plane of a multi-element photodetector, two spaced panoramic images are simultaneously obtained, each of which contains an image of a portion of the surrounding space observed with a given angle with the object of interest in it, and the distance to the specified object is determined by the relative position of its two images on the simultaneously recorded a couple of spaced panoramic images.

The technical result consists in obtaining on the plane of a multi-element photodetector a unique, simultaneously recorded, panoramic stereo pair due to the formation of a two-way reflection, which allows us to estimate with greater accuracy the distance to the object of interest with previously unknown sizes, analyzing the relative position of the images of the object of interest on the registered panoramic stereo pair, using famous mathematical apparatus.

As a prototype of panoramic equipment for determining the location of an object, equipment was selected that implements the method described above, containing a convex mirror with a reflective surface formed by a body of revolution and covering a circular view of the controlled space in a given solid angle, a multi-element photodetector, an optical system for transferring a panoramic image reflected from a mirror onto the plane of the multi-element photodetector, the axis of the body of rotation being aligned with the main optical axis of the lens optical image transfer system as well as a recording device and image processing obtained (see. the above-mentioned US patent №6542184). This device is designed to monitor the environment in any direction of the review, covering 360 degrees, in real time with the ability to simultaneously register what is happening around. In this case, a single video camera is used, which is stationary and oriented relatively to the convex mirror, in the plane of the multi-element photodetector of which a circular panoramic image is formed.

However, this device does not provide the ability to accurately determine the coordinates of an unfamiliar object that appears in the field of view of the observed space.

The task is to create a device that provides increased accuracy in determining the coordinates of the object of interest. The problem is solved in that in a panoramic equipment containing a convex mirror with a reflective surface formed by a body of revolution and covering a circular view of the controlled space in a given solid angle, a multi-element photodetector, an optical system for transferring a panoramic image reflected from the mirror onto the plane of a multi-element photodetector, and the body axis rotation is combined with the main optical axis of the lens of the optical image transfer system, as well as a recording device and rabotki obtained image according to the invention a convex surface reflective mirror is formed by two bodies of revolution with a common axis of rotation, which become one another, which provides the geometric parameters of the mapping on a plane photodetector two unified point panoramic image of the observation portion of the space.

Thanks to this embodiment of the device, it becomes possible to implement the above method due to the appearance of a stereo pair of images of the surrounding space, including with the object of interest, which reveals the relative position of two images of the same object, taken from different angles, used in mathematical expressions to determine the coordinates of the body in space relative to a given observation point.

Embodiments of the reflective surface may be different.

The reflective surface of a convex mirror can be formed by two conical surfaces with different angles at the vertices, while both conical surfaces face their vertices to the input aperture of the optical image transfer system, and the vertex of the cone with a smaller angle at the vertex is shifted closer to the input aperture of the optical image transfer system, than the second.

In another embodiment, the reflective surface of the convex mirror can be formed by two spherical surfaces with different radii, while a spherical surface with a smaller radius is shifted closer than the other with respect to the input aperture of the optical image transfer system.

In the third embodiment, the reflective surface of the convex mirror can be formed by two elliptical surfaces with different curvatures, while a spherical surface with less curvature is shifted closer than the other with respect to the input aperture of the optical image transfer system.

In the fourth embodiment, the reflective surface of the mirror is formed by two parabolic surfaces with different curvatures, while both parabolic surfaces face their vertexes to the input aperture of the optical image transfer system, and the parabolic surface with less curvature is shifted closer than the second to the input aperture of the optical image transfer system.

The choice of one or another option depends on the required solid angle of view of the surrounding space, while the greater the curvature of the convex mirror, the larger the viewing angle.

In addition, a convex mirror with a reflective surface can be made up of two separate parts, each of which contains a reflective surface in the form of a body of revolution, while the parts are fixed relative to each other with the condition that their rotation axes are aligned with the main optical axis of the lens of the optical image transfer system .

In some cases, this may turn out to be more technological.

In addition, the apparatus includes a directional illumination unit, consisting of at least one light source and at least one slotted collimator with a slit plane perpendicular to the main optical axis of the lens of the optical image transfer system.

This ensures the determination of the distance to a large uniformly colored object, such as a wall, due to the formation of a light reference spot or strip on the object of interest.

Figure 1 schematically shows the principle of forming a panoramic stereo pair of the claimed method, and figure 2 the corresponding image formed on the plane of a multi-element photodetector. Figures 3, 4, 5 schematically depict embodiments of panoramic equipment for determining the location of an object in the surrounding space, implementing a method in which cones are used to form convex reflective surfaces (Fig. 3), spheres (Fig. 4), ellipses (Fig. .5). Figure 6 shows one embodiment of a convex mirror consisting of two separate parts mounted on one mandrel.

The method is implemented as follows.

First, a panoramic stereo pair is formed on the plane of the multi-element photodetector. Light (rays OA ₁ and OB ₁ ) from the surrounding space enters a convex reflective surface having the shape of a body of revolution, and is reflected from its two sections H ₁ and H ₂ under two different angles (figure 1). The light (rays A ₁ O ₁ and B ₁ O ₁ ) reflected from H ₁ and H ₂ , passing through the image transfer system of the observed part of space, forms two images (points A ₂ and B ₂ ) of one and one on the plane H _{3 of the} multi-element photodetector the same piece of space (point O), observed from different angles. After that, the panoramic stereo pair, thus formed on the plane of the photodetector, is recorded. As shown in figure 2, the first panoramic image from the composition of the stereo pair corresponds to the region of the registered image, bounded by a circle of radius R ₁ centered at point O ₂ , which is the intersection point of the Y axis - the main optical axis (g.o.) of the image transfer system - with a plane H ₃ (figure 1); and the second panoramic image is an annular region centered at the point O ₂ limited by the inner radius R ₁ and the outer radius R ₂ .

The configuration and location of the surfaces H ₁ and H ₂ are chosen so that the rays (OA ₁ and OB ₁ ) emanating from one point of the object (point O) and incident on a convex mirror are reflected from it and passed through one point, for example, the center ( point O ₁ ) of the input lens of the optical system for transferring the image of the observed portion of space onto the plane of the multi-element photodetector. The shape of the reflective surfaces H ₁ and H ₂ depending on the desired viewing angle of the surrounding space Θ can be, for example, conical, spherical, elliptical or parabolic. Moreover, the greater the curvature of the reflective surfaces, the greater the viewing angle Θ. The maximum range of measuring the distance to the object is also determined by the configuration and location of the surfaces H ₁ and H ₂ , and in particular by a stereoscopic basis (segment a). The larger the basis, the greater the maximum range for measuring the distance to the object.

Further, in order to determine the coordinates of the object of interest (points O), proceed as follows.

On the registered image (figure 2) determine the coordinates of the points (A ₂ (φ, r ₁ ) and B ₂ (φ, r ₂ )) corresponding to two images of the object of interest, taken from different angles.

The angles α and β are calculated between the X ₁ axis parallel to the H ₃ plane and directed toward the object of interest (point O) and the rays reflected from the surfaces H ₁ and H _2, respectively (FIG. 1). These angles are calculated based on the found coordinates (φ, r ₁ , r ₂ ) and the known characteristics of the optical image transfer system (focal length f).

The coordinates of the points (A ₁ and B ₁ ) of the reflection (incidence) of the rays from the surfaces H ₁ and H ₂ are determined, forming on the plane H _{3 the} corresponding different-angle images of the object of interest. These coordinates of the points (A ₁ (X _A1 , Y _A1 ) and B ₁ (X _B1 , Y _B1 )) of the reflection (incidence) of the rays in the O ₁ X ₁ Y system are determined as the solution of the following two systems of equations:

where F ₁ (x, y) is a known equation in an implicit form of the curve, which is a section of the reflecting surface H ₁ plane O ₁ X ₁ Y; F ₂ (x, y) is a well-known equation in an implicit curve that is a section of the reflecting surface H _{2 by the} plane O ₁ X ₁ Y. For reflecting surfaces of conical shape, the corresponding equations are F ₁ (x, y) = 0, F ₂ (x, y) = 0 are the equations of the lines in an implicit form, for the spherical form - the equations of arcs of circles, for the elliptic - the arcs of ellipses, for the parabolic - the arcs of parabolas.

The angle λ ₁ , the angle between the beam A ₁ O and the vector O ₁ X ₁ , is calculated, and the angle λ ₂ , the angle between the beam B ₁ O and the vector O ₁ X ₁ , according to the following formulas obtained on the basis of well-known representations of geometric optics.

,

- значения частных производных по X и по Y функции F₁(x,y) в точке A₁(X_A1,Y_A1) соответственно;

,

- значения частных производных по X и по Y функции F₂(x,y) в точке B₁(X_B1,Y_B1) соответственно.Where

,

- the values of the partial derivatives with respect to X and Y of the function F ₁ (x, y) at the point A ₁ (X _A1 , Y _A1 ), respectively;

,

- the values of the partial derivatives with respect to X and Y of the function F ₂ (x, y) at the point B ₁ (X _B1 , Y _B1 ), respectively.

Find the coordinates of the object of interest (i.e. O (X _O , Y _O )) in the system O ₁ X ₁ Y from the solution of the equation:

Thus, the position of the object (point O) in space completely determines the found angle φ and coordinates (i.e. O (X _O , Y _O )) in the system O ₁ X ₁ Y.

Panoramic equipment for determining the location of an object in the surrounding space, which implements the method, generally comprises a convex mirror 1, an optical image transfer system 2, a multi-element photodetector 3, a mathematical image processing unit 4 of the registered image, see Figs. 3, 4, 5.

In the device shown in FIG. 3, the convex mirror 1 is formed by a first conical reflective surface (H ₁ ) with an angle at the apex δ and a second conical reflective surface (H ₂ ) with an angle at the apex ε. The central axes of the conical surfaces coincide with the main optical axis (g.o.) of the system 2. The first conical surface passes into the second conical surface, while in the central section the generatrix of the surface of the convex mirror is a broken line.

переходит во вторую сферическую поверхность (Н₂) с большим радиусом

. Геометрические центры сфер находятся на главной оптической оси системы 2.In the device shown in figure 4, the convex mirror 1 is formed by two intersecting reflective spherical surfaces. First spherical surface (H ₁ ) with radius

passes into the second spherical surface (H ₂ ) with a large radius

. The geometric centers of the spheres are located on the main optical axis of system 2.

In the device shown in Fig. 5, a convex mirror 1 is shown formed by two intersecting reflective elliptical surfaces. The first smaller elliptical surface (H ₁ ) passes into the second larger elliptical surface (H ₂ ). In this case, one of the two semi-axes of each elliptical surface coincides with the main optical axis of system 2.

To ensure greater adaptability, the convex mirror 1 can be made composite, see Fig.6. In this case, the mirror 1 consists of two separate parts, each of which contains a reflective surface in the form of a body of revolution, while the parts are fixed relative to each other on the mandrel 7 with the condition of combining their axis of rotation with the main optical axis of the lens of the optical system 2.

The optical system 2 for transferring the image of the observed portion of space onto the plane of the photodetector 3 can be made in the form of a lens objective. As the photodetector 3, a multi-element charge-coupled device (CCD array) or a multi-element complementary metal oxide semiconductor (CMOS matrix) can be used.

The configuration of the convex mirror 1 and the optical system 2 is selected so that all the incident light (in a given solid angle Θ with a circular coverage of 360 degrees) on the convex mirror 1, the light from the surrounding space is reflected from the convex mirror 1 and passes through one point, for example, the center of the input lens the lens of the optical system 2. In this case, the panoramic stereoscopic image formed by the convex mirror 1 and the optical system 2 is fully displayed in the plane of the multi-element photodetector 3.

Block 4 of the mathematical processing of the registered image can be made in the form of a programmable logic integrated circuit (FPGA). The signal inputs of block 4 are connected by electrical communication with the signal outputs of the photodetector 3.

Panoramic equipment for determining the location of an object in the surrounding space may include a directional illumination unit 5, see Fig. 5 (shown in section), consisting of at least one light source (indicated by the letter S) and at least one slotted collimator with a slit plane perpendicular to the main optical axis of the lens of the optical system 2 image transfer. In this case, the slit collimator can be made in the form of an opaque shell with a slit 6 made in it. Node 5 provides the creation of a light frame on a large uniformly colored object, such as a wall, which simplifies the selection of two images of the same fragment (light spot) on a registered stereo pair object taken from different angles, the analysis of the relative position of which is carried out in the future to determine the distance to the object.

Power supply of panoramic equipment for determining the location of an object in the surrounding space can be carried out via wired electrical communication from a separate power supply system.

The work of panoramic equipment for determining the location of an object in the surrounding space is as follows.

First, using a convex mirror 1 and an optical system 2, a panoramic (circular) stereoscopic image of the surrounding space in solid angle Θ is formed on the plane (H ₃ ) of the multi-element photodetector 3.

This image is recorded by the photodetector 3, converted to digital and transmitted by electrical communication to the signal input of the mathematical processing unit 4.

In block 4 of mathematical processing, the registered image is analyzed according to the algorithm incorporated in the block, based on the described method. An object of interest is localized on the stereo pair. The localization process consists in selecting the contours of two images of an object of interest using existing algorithms (for example, the Gauss or Sobel algorithm) and two areas limited by these contours. Then on the registered image are the coordinates of the points ((A ₂ (φ, r ₁ ) and B ₂ (φ, r ₂ )), Figs. 1, 2) corresponding to the centers of two images of the object of interest (point O is the center of the object). Further, using the coordinates found and the above system of equations and relationships, find the coordinates (i.e. O (X _O , Y _O )) in the system O ₁ X ₁ Y.

In block 4, a signal is generated containing the found angle φ and coordinates (i.e., O (X _O , Y _O )) in the O ₁ X ₁ Y system, which is transmitted to the signal output of block 4.

The technical implementation of panoramic equipment for determining the location of an object in the surrounding space is carried out on commercially mastered structural units and elements. Creating a program for processing registered images is not difficult when using well-known algorithms. The claimed method and panoramic equipment can be used in technological equipment, for example, on loaders to detect their dangerous proximity to an obstacle (wall), which will help to prevent collisions.

Claims (3)

1. A method for determining the location of an object in the surrounding space, which consists in forming on the plane of a multi-element photodetector a panoramic image of the observed space using a convex reflective surface directing the scattered light reflected from it to the optical system for transferring the image of the observed space on the plane of the photodetector, in registering the generated image and in determining the coordinates of the object of interest, characterized in that they form two directional reflection from a reflective surface, while on the plane of a multi-element photodetector, two spaced panoramic images are simultaneously received, each of which contains an image of a portion of the surrounding space with the object of interest located in it at a given angle, and the distance to the specified object is determined by the relative position of its two images on a simultaneously recorded pair of spaced panoramic images. 2. Panoramic equipment for determining the location of an object in the surrounding space, containing a convex mirror with a reflective surface, covering a circular view of the controlled space at a given solid angle, a multi-element photodetector, an optical system for transferring a panoramic image reflected from the mirror onto the plane of a multi-element photodetector, as well as a recording device and processing the obtained image, characterized in that the reflective surface of the convex mirror is formed two bodies of rotation, which become one another, with a common axis of rotation, combined with the main optical axis of the objective optical image transfer system, which provides the geometric parameters of the mapping on a plane photodetector two unified point panoramic image of the observation portion of the space. 3. Panoramic equipment according to claim 2, characterized in that the reflective surface of the convex mirror is formed by two conical surfaces with different angles at the vertices, while both conical surfaces face their apices to the input aperture of the optical image transfer system, and the vertex of the cone with a smaller angle at the vertex is shifted closer to the input aperture of the optical image transfer system than the second.

Images