FI97085C - Method and imaging device for determining distance and use thereof - Google Patents

Description

97085

Method and imaging equipment for determining distance and its use

The invention relates to a method for determining the distance, in which imaging apparatus is used to determine the distance of an object, by means of which optical means are used to image an image area contained in said object to obtain at least two separate two-dimensional images, preferably in the same image area 10. converted to digital form by a data processing unit, such as a microprocessor or the like, for determining the distance of said object, such as one or more corresponding pixels in said image area.

Previous methods for determining distances have been based on e.g. the application of stereo-vision algorithms, the application of triangulation methods, active sounding by laser-20 or other electromagnetic radiation or ultrasound. However, the present invention is mainly concerned with the "autofocus" methods used in many commercial systems, as well as the "depth from focus" methods developed in particular by research institutes. For example, in the "autofocus" technique used in cameras, one way to sharpen an image is to adjust the focus of a motorized lens until one point in the image plane, most commonly the center, is at its most accurate. point distance. To determine the sharpest focus, e.g. high-pass filtering, which at its simplest means calculating the difference between each digitized pixel and its 35 neighbors in both the x and y directions.

However, in current applications, it is not possible to perform the above-mentioned distance determination simultaneously for the entire image area.

2 97085

On the other hand, in a method developed by APPentland (ref: APPentland, T.Darell, M.Turk and W.Huang, A Simple, Real-Time Range Camera, IEEE Computer Society Conf. On Computer Vision and Pattern Regognition, 5 1989, p. 256 -261) the distance determination is based on taking two images of the subject using two separate cameras with apertures of different sizes in each. The small-aperture image of the second camera has a large sharpness range, while the sharp-aperture range of the second camera taken with the large aperture of the 10 cameras is small around the correct focusing distance. By comparing the two images, the distance of all pixels with sufficient contrast in the image area can be determined.

15

The above method has certain problems and has therefore not been generally accepted for practical applications. This is due, firstly, to the fact that the method has to use two separate cameras, to which the images are divided, for example, by a beam splitter before their lenses. On the other hand, in this method, the second image is taken using a small aperture, which significantly reduces the brightness of the system and thus also the effective operating range and conditions of the method.

In addition, in said method, there is a risk that in the image corresponding to the large aperture, the blurring discs caused by adjacent objects partially overlap, whereby erroneously large amplitude values are obtained at the boundary points after high-pass filtering 30.

The object of the method according to the present invention is to obtain a decisive improvement over the above-mentioned drawbacks and thus to substantially increase the state of the art in the art. To achieve this object, the method according to the invention is mainly characterized in that 3 97085 - substantially the same image area contained in said object is imaged by a single objective arrangement or the like belonging to optical elements by at least two, preferably three separate, two-dimensionally and equally illuminating two-dimensional to produce an image, each of which uses a fixed focal length substantially different from the other images, at least before the shooting situation, at which the subject at distance 10 is substantially sharpest, wherein - the distance of said subject, such as each corresponding pixel in the image area, is determined by comparing For 15 pixels, each image, preferably converted to digital format, which is focused to a different focusing distance from the other images, by means of a data processing unit 1a computationally, such as by interpolating 20 or extrapolating from image information on images.

The most important advantages of the method according to the invention are the simplicity and reliability of its use and the constructions suitable for its implementation. In particular, compared to the commercial "autofocus" methods, it is also advantageous that the method according to the invention makes it possible to perform the measurement quickly and also to calculate the distances to all such points in the image with a sufficiently large contrast difference from nearby points.

The speed of the method is based in particular on the fact that it is not necessary to use any moving parts in the camera applying the method according to the invention. Furthermore, compared to the above-mentioned Pentland method, the method according to the invention enables a clearly better luminous intensity, which is due to the fact that when applying the method, a suitable, sufficiently large aperture can always be used in the lens. In addition, with respect to the Fentland method mentioned above 4 97085, the invention has the further advantage that the distance determination is preferably based on data from three images, so that errors in a single image can be eliminated if they contradict the results of the other two images.

Preferred embodiments of the method 10 according to the invention are set out in the dependent claims of the method.

The invention also relates to an imaging apparatus for applying the method. The imaging equipment is further defined in the preamble of the independent patent claim 15 for imaging equipment. The main characteristic features of the imaging apparatus are set out in the characterizing part of the corresponding claim.

Firstly, the imaging apparatus according to the invention enables significantly lower manufacturing costs compared to current arrangements, which is a consequence of the development of microelectronics and can be seen, for example, in the fact that in semiconductor camera constructions optics or optomechanics components are more expensive than microelectronics components. Thus, in the simplest possible construction of the imaging device according to the invention, there is only one single-lens fixed focal length lens and two beam splitters and three image detector elements, which clearly forms a more advantageous whole than the corresponding solutions of the present invention, e.g.

. * stereo camera solutions require two lenses, laser rangefinders and triangulation devices typically have transmit and receive optics, and possibly also scan mechanics. A further advantage of the imaging apparatus according to the invention is the simplicity of the calculation algorithm applied therein, which includes only a few simple operations per pixel, whereby there is no need to use expensive special processors.

A further advantage of the method and imaging device 5 according to the invention is in particular robustness. Since the distance measurement is always based on a comparison of three pixels obtained simultaneously from different focusing distances, the result is not affected by, for example, changes in lighting conditions. In addition, the 10 objects to be measured are not required to have any special characteristics other than that they are not completely uniform (and even then active lighting can give good results if the object is not completely black), so there is no need to use a comparison of two identified objects like stereo cameras. active illumination of laser scanners. In addition, the imaging apparatus according to the invention can also be used in the same way as a conventional camera; if one of the pixels of the three images is always selected to correspond to the sharpest image and thus one image assembly is assembled, a camera is obtained whose sharpness range of the image produced is considerably larger than that of a conventional camera.

The dependent claims on the imaging apparatus set out preferred embodiments of the imaging apparatus according to the invention.

The invention is also defined by one of the independent use requirements for the use of the above-mentioned method.

In the following description, the invention will be illustrated in detail with reference to the accompanying drawings, in which Figure 1 shows a longitudinal section of a preferred imaging device used in the method according to the invention, Figures 2a-2c show a preferred principle of the method according to the invention, and Figure 3 shows a block diagram of the invention. in accordance with that method.

A method for determining a distance using imaging apparatus 10 for determining the distance of an object, the optical means 1 of which imposes an image area contained in said object to produce at least two separate two-dimensional images, which are compared with substantially the same image area, preferably digitally converted to a data processing unit 2 , such as a microprocessor or the like, for determining the distance of said object, such as one or more corresponding pixels in said image area. When applying the method: 20 - substantially the same image area contained in said object is imaged by one objective arrangement 1a or the like belonging to the optical members 1 to obtain at least two, preferably three separate, two-dimensional x, y images substantially equal in image area and light intensity, each using a fixed focusing distance A or 30B, preferably A, B or C, set substantially different from the other images, at least before the shooting situation, at which distance the subject is substantially sharpest, wherein - the distance of said subject, such as each corresponding pixel in the captured image area, is determined; by comparing, for each pixel of said subject, such as for each pixel, three images with a focusing distance A, B, C different from the other images, preferably digitized, 7 97085 images Olxy, 02xy, 03xj, where χ-1 ... x, ^ and y = 1 .-. y * ,,, information by the processing unit 2 computationally, such as by interpolating or extrapolating the image information Hplxy, Hp2xy, Hp3xy on the images.

5

As a preferred embodiment of the method according to the invention, substantially the same image area contained in said object obtained by means of optical means 1 is transmitted simultaneously to at least two analog imaging elements 3, such as a semiconductor matrix camera element or the like, at least before 10 shooting situations, whereby the images stored therein are converted. converting means 4, such as an A / D converter or the like to digital, 15 after which the data processing unit 2 determines the distance of said object, such as each corresponding pixel in the described image area, preferably comparing high-pass filtering techniques or at least two corresponding images of the object .

The image area obtained by the lens arrangement 1a belonging to the optical members 1 is preferably transmitted to three imaging members 3a, 3b, 3c fixed to the same imaging device x as the lens arrangement 1a 25 according to Fig. 1, such as a semiconductor matrix camera element or the like arranged in different to provide focusing distances A, B, C in a direction perpendicular to the image plane along the optical propagation path 30 measured at substantially different distances from the optical center of the objective arrangement 1a.

A further preferred embodiment of the method is to divide the light beam transmitted by the objective arrangement 1a by dividers 5 in the device X, such as one or more beam splitters 5a, 5b or the like, * to obtain two-dimensional x xy images 3, at least substantially equal in intensity. 3c.

As a further preferred embodiment of the method, said object is compared, such as at least three images of each pixel in the field of view, most preferably digitized 01, γ, 02xy, 03xy and further high-pass filtered Hplxy, Hp2xy, Hp3xy, are compared by means of a data processing unit 2, the distance is determined by interpolating the image information Hplxy, Hp2xy describing the subject from within the focus area AC, as in the situation of Fig. 2b, and extrapolating the image information Hplxy describing the subject from outside the focus area AC 15,

Hp2xy, Hp3xy forming a concave curve, as is the case in Figure 2c.

It should be noted that in the method according to the invention it is possible to determine the distance only for pixels whose corresponding high-pass filtering values exceed a certain threshold value (i.e. for an image containing only one tonal value or where tonal values change only slowly from one pixel to another, it is not possible to determine distances). When the subject is at a distance B, the highest intensity value after high-pass filtering is obtained from the three camera elements • · for the high-pass filtered pixel Hp2xy at the focusing distance B. This is because when the image 30 is "sharp", it contains high spatial frequencies that are emphasized in the high-pass filtering operation. The amplitudes of the respective high-pass filtered pixels of the camera elements corresponding to the focusing distances A and C, respectively, are then lower as shown in Fig. 2a. In this case, by comparing the amplitudes of the corresponding three pixels, the distance of that pixel can be determined: B.

9 97085

If the distance does not correspond exactly to any of the three possible focusing distances, but is between A-C, the corresponding amplitudes at distances A, B and C are obtained according to Fig. 2b such that the elements (A, B) closest to the correct focusing distance give a larger amplitude than the third element. In this case, the approximate distance is determined by fitting the interpolation curve to the amplitudes and retrieving the the maximum point of the curve. In practice, this operation can be performed, for example, by plotting a sufficient number of distance values corresponding to three amplitude values, whereby interpolation can be performed with one table search. If the distance of the object is outside the range A-C, the distance can be determined by extrapolating la within certain limits 15 according to Fig. 2c.

Thus, when applying the method according to the invention, e.g. a comparison is performed for all 20 pixels exceeding the threshold value, whereby a table defining e.g. a 3d surface can be determined. Similarly, you can always select the pixel tone value closest to the correct focusing distance from the original three images.

The imaging apparatus according to the invention is thus intended for determining distance, comprising in its simplicity optical means 1, in which the imaged image area comprising a distance-detectable object, such as one or more pixels in said image area, is arranged simultaneously for at least two, preferably -min to three analog imaging elements 3, such as a semiconductor matrix camera element 3a, 3b, 3c or the like, for transmitting at least three separate two-dimensional images of said image area. The imaging apparatus further comprises for determining the actual distance of the object: a data processing unit 2, such as a microprocessor or the like, for comparing images containing substantially the same image area, preferably in digital form. The semiconductor matrix camera elements 5 in the imaging device X belonging to the imaging apparatus have a common lens arrangement 1a, by means of which three separate two-dimensional images with substantially equal image area and light intensity are obtained for each semiconductor matrix camera element 3a, 3b, 10 3c.

Referring to Fig. 3, the imaging apparatus further includes conversion means 4, such as an A / D converter or the like, for digitizing the image transmitted to the imaging means 3a, 3b, 3c. In addition, the data processing unit 2 is preferably arranged by applying a high-pass filtering technique for comparing corresponding images describing the object. In the block diagram shown in Fig. 3, reference numeral 7 denotes a multiplexer (MUX) with which the output of each of the three camera elements 20 can be selected.

Referring to Fig. 1, the objective arrangement 1a belonging to the optical members 1 is preferably detachably arranged in the same imaging device X together with the semiconductor matrix camera elements 3a, 3b, 3c, most preferably fixedly fixed to it, at different fixed target distances A, B, C. In order to obtain different focusing distances A, B, C, each imaging member 3a, 3b, 3c in the imaging device X30 is arranged in a direction perpendicular to the image plane along the path of light viewed at substantially different distances from the optical center 1a of the objective arrangement.

35

The imaging device X is further provided with dividing means 5 for distributing the light beam • transmitted by the objective arrangement 1a to the imaging means 3a, 3b, 3c, for them to obtain two-dimensional images with substantially equal light intensities.

A further preferred embodiment for the imaging device 5, in particular for adjusting the focusing area AC, comprises adjusting means 6, which in the solution according to Fig. 1 are preferably arranged by screwing the objective arrangement 1a to the imaging device X. This can of course also be moved by moving each camera element to the imaging device X or using e.g. the so-called variable focal length (zoom) lens.

Referring to Figure 1, the imaging device X comprises two perpendicular beamsplitters 5a, 5b which divide the light beam transmitted by the objective arrangement 1a into three perpendicular elements at the top, rear and bottom elements of the imaging device Xa. 3b, -3c, of which the image transmitted to the first is focused closer to the others A, to the third further away from the others C and to the second between the above B.

25

It is clear that the invention is not limited to the applications presented or described above, but can be considerably modified within the framework of the basic idea. In this case, for example, four or 30 more camera elements can be used. Of course, it is also possible to fit the camera elements in the same horizontal plane by dividing the light beam coming through the lens in three places in the corresponding horizontal plane.

Claims (10)

12,97085 Claims; A method for determining the distance, wherein imaging apparatus is used to determine the distance of 5 objects, the optical means (1) of which are used to image an image area contained in said object to obtain at least two separate two-dimensional images, preferably 10 images containing substantially the same image area converted to digital form by a data processing unit (2) such as a microprocessor or the like to determine the distance of said object, such as one or more corresponding pixels in said image area, characterized in that - substantially the same image area contained in said object is imaged on optical means (1) by one lens arrangement (1a) or the like to produce at least two, preferably three separate, two-dimensional (x, y) images of substantially equal image area and light intensity, each of which uses a substantially a fixed focusing distance (A or B, preferably A, B or C) set at least before the 25 shooting situations, whereby - the distance of said subject, such as each corresponding pixel in the described image area, is determined by comparing said subject, such as each with respect to the pixel, each of the preferably three images (01xy, 02xy, 03xy) focused at a focusing distance (A, B, C) different from the other images, preferably digitally converted by means of a data processing unit (2), such as image information (Hplxy, Hp2xy, Hp3xy) ) by interpolation or extrapolation. «13 97085 A method according to claim 1, wherein the substantially same image area contained in said object obtained by the optical means (1) is transmitted simultaneously to at least two analog imaging elements (3), such as a semiconductor matrix camera element or the like, focused at a fixed distance before at least 5 shooting situations, wherein the images stored therein are converted by conversion means (4), such as an A / D converter or the like into digital 10, after which the distance of said object, such as each corresponding pixel in the described image area, is determined by comparing preferably a high-pass filtering method. technique by applying or correspondingly at least two corresponding images contained in 15 objects to each other, characterized in that the image area obtained by means of the lens arrangement (1a) belonging to the optical members (1) is transmitted to at least two, preferably three, preferably lens ) to a imaging device (3a, 3b, 3c) fixed to the same imaging device (X), such as a semiconductor matrix camera element or the like, arranged to provide different focusing distances (A, B, C) in the direction perpendicular to the image plane from the optical 25 along the propagation path for substantially different distances from the optical center of the objective arrangement (1a). Method according to Claim 2, characterized in that the light beam passing through the objective arrangement (1a) is divided by dividing elements (5) in the imaging device (X), such as one or more beam splitters (5a, 5b) or the like, at least 35 dimensionally substantially identical two-dimensional images. to the imaging means (3a, 3b, 3c) in the imaging device (X). 14 97085 Method according to one of the preceding claims 1 to 3, characterized in that said object, such as at least three representing each pixel of the image field, preferably converted to digital format 5 (01xy, 02xy, 03xy) and further high-pass filtered (Hplxy, Hp2xy, Hp3xy ) the image is compared with each other by means of a data processing unit (2), whereby the distance of the subject is determined by interpolating the image information (Hplxy, Hp2xy, Hp3 ^ y) from within the focus area (AC) to form a convex curve and extrapolating the subject from outside the focus area (AC) descriptive image information (Hplxy, Hp2xy, Hp3xy) to form a concave curve. An imaging apparatus for determining a distance, comprising at least optical means (1), wherein the imaged image area comprising an object to be determined at a distance, such as one or more pixels in said image area, is arranged simultaneously on at least two analog imaging means (3). ), such as a semiconductor matrix camera element or the like, for providing at least two separate two-dimensional (x, y) images of said image area, and a data processing unit (2) for determining the distance 25 of said object, such as a microprocessor or the like, for comparing images containing an image area, preferably converted to digital form, characterized in that the imaging apparatus 30 comprises an imaging device (X) having at least two, preferably three imaging elements (3), such as semiconductor * matrix camera elements or the like, having common optical bodies (1 ), such as an objective arrangement (1a) or the like, for producing at least two, preferably three, three, two-dimensional images of substantially equal image area and light intensity, preferably simultaneously for imaging means (3a, 3b, 3c). 15 97085 Imaging apparatus according to claim 5, comprising conversion means (4), such as an A / D converter or the like, for digitizing the image 5 transmitted to the imaging means (3) by means of the optical means (1), said object as in the described image to determine the distance of each corresponding pixel in the region by means of the data processing unit (2), preferably comparing the high-pass filtering technique by applying or correspondingly at least two corresponding images of the 10 objects, characterized in that the objective arrangement (1a) of the optical members (1) is arranged, preferably detachably attached to the same imaging device (X), together with at least two, preferably three, at least two fixed target distances (3a, 3b, 3c) fixed to it at least differently before the imaging situation, fixedly fixed thereto, such as a semiconductor matrix camera element or the like. 20 Imaging apparatus according to Claim 5 or 6, characterized in that at least three imaging elements (3a, 3b, 3c) in the imaging device (X) are arranged to provide their different focusing distances (A, B, C) 25 in a direction perpendicular to the image plane in the light path. viewed along substantially different distances from the optical center of the objective arrangement (Ia). Imaging apparatus according to one of the preceding claims 5 to 7, characterized in that: the imaging device (X) comprises dividers (5), such as one or more beam splitters (5a, 5b) or the like, for dividing the light beam 35 transmitted by the objective arrangement (1a) to the imaging means (3a); , 3b, 3c) to provide them with two-dimensional images at least substantially identical in light intensity. 16 97085 Method according to one of the preceding claims 5 to 8, characterized in that for adjusting the focusing area (AC) of the imaging device (X), it comprises adjusting elements (6) which are preferably arranged by attaching optical elements (1), such as an objective arrangement. (1a) or the like and / or imaging means (3a, 3b, 3c), such as semiconductor matrix camera elements or the like movably, such as on a spiral, bellows, telescopic principle or correspondingly to the imaging device (X). 10 Imaging apparatus according to one of the preceding claims 5 to 9, characterized in that the imaging device (X) preferably comprises two beam splitters (5a, 5b) or the like which are perpendicular to each other and which divide the objective arrangement (1a) or the like through a light beam preferably in three perpendicular directions to each other, such as a semiconductor-20 matrix camera element at the top, rear and bottom of the imaging device (X) or the like (3a, 3b, 3c), the first image being closer (A), the third farther (C) and the other between the above (B). 11. A method for determining the distance of an object using an imaging apparatus having optical means (1) for imaging an image area contained in said object and a data processing unit (2) such as a microprocessor or the like for comparing images captured by optical means (1). , wherein substantially the same image area contained in said object is imaged by one objective arrangement (1a) or the like belonging to the optical members (1) of at least two, preferably three separate, two-dimensional (x, y) images with substantially equal image area and light intensity each of which is used to describe substantially others. . different from the images, at least before the 9. Ht-fr 1,1 H II) 4. . 17 97085 lumbar focusing distance (A or B, preferably Af B or C), at which the subject is substantially blade to 11, and wherein the distance of said subject, such as each of the 5 corresponding pixels in the described image area, is determined by comparing said subject, such as, for each pixel, each of the 10 images (01xy, 02xy, 03xy) focused on the focusing distance (A, B, C), preferably converted to a digital format differently from the other images, preferably computed into a digital format (2) by computing, such as image image information ( Hpxy, Hp2xy, Hp3xy) by interpolation or extrapolation, use in an imaging device such as a camera or the like to form a unitary image set, such as a photograph or the like, which is assembled using the image formulation correspondingly most sharply for each pixel. 18 97085