WO2008034942A1

WO2008034942A1 - Method and apparatus for stereopanoramic imaging

Info

Publication number: WO2008034942A1
Application number: PCT/FI2007/050423
Authority: WO
Inventors: Jussi Heikkinen; Henrik HAGGRÉN; Petteri PÖNTINEN; Petri RÖNNHOLM; Hannu HYYPPÄ
Original assignee: Teknillinen Korkeakoulu
Priority date: 2006-09-22
Filing date: 2007-07-10
Publication date: 2008-03-27
Also published as: FI20060846A0

Abstract

The present invention discloses a new type of method, apparatus and computer program for reconstructing a stereoscopic panoramic image. In the invention, the stereopanoramic image is generated by combining an image sequence and a separate panoramic image. In a preferred embodiment, the image sequence covers 360 degrees, as does the panoramic image. In the method, the time-lapse camera which stores the image sequence is moveable about a navel point at a fixed distance. The panoramic camera, on the other hand, is positioned at the navel point. The first image sequence is stored by moving the camera about the navel point, the camera being oriented, at each moment of exposure, to a direction that is tangential to the trajectory of the camera. The second panoramic image sequence can be stored by also moving the camera about the navel point, the camera being, this time, oriented to a direction that is tangential to the trajectory of the camera and opposite to the previous direction. Finally, an individual stereo pair of the stereopanoramic image is generated by combining part of the panoramic image and an individual image from the image sequence. The stereopanoramic image can be formed of a set of adjacent stereo pairs. In one embodiment, rays of light can be deflected using reflecting and/or refracting elements. The surrounding scene can also be examined by measuring it with a laser scanner, and the measurement data can be further modified based on the viewing properties of the stereopanoramic image.

Description

METHOD AND APPARATUS FOR STEREOPANORAMIC IMAGING FIELD OF THE INVENTION

The invention relates to imaging, and more specifically to combining stereo images and panoramic image to enable three-dimensional observation, reconstruction and visualization of the surrounding scene.

BACKGROUND OF THE INVENTION

Imaging can be regarded as a process of gen- erating images, animation or three-dimensional graphics from an object or a surrounding scene. This type of imaging is necessary for various applications, examples of which include navigation systems, computer game applications and different animation applications displayed for example in the Internet. The objective is thus to generate, by means of different observations and measurements and by taking photographs , an imitation of the surrounding scene that is as realistic as possible. A stereoscopic image (i.e. stereo image) refers to an image that is combined of two images in order to provide a stereoscopic effect. In known technology, a stereo image can be generated by taking two different photographs with cameras, such that the cam- era perspectives differ slightly from each other. Moreover, the image points must in some manner be coordinated with each other, i.e. for each three- dimensional point of the first image, a point corresponding thereto must be known in the second image. A stereo image relates to epipolar geometry, which comprises the concept of an epipolar line. It is assumed that two adjacent cameras have both their proper image plane on which the photographed objects are projected. Between the point of the photographed object and the focus, there is the projection point of the object on the image plane, i.e. in this simple example "image of the object". Both cameras have their proper focuses, and the intersections of the segment which joins these focuses and the image plane are so-called epipoles . The epipolar line is defined as the line that passes through the projection point and the epipole. With the traditional, so-called perspective camera, this epipolar line is straight.

Specification Peleg et al . (Peleg I; US 6,665,003) describes a method and a system for forming and displaying motion pictures (movies) and images in panoramic form. The forming principle comprises combining two image sequences that are not concentric, i.e. that do not have the same projection center. Specification Shum et al . (US 6,639,596) describes generation of a stereopanoramic image based on panoramic images from multiple different perspectives. The images by Shum are thus similarly eccentric as those by Peleg I. Publication 'Jussi Heikkinen: "The Circular

Imaging Block in Close-Range Photogrammetry" , Dissertation at Helsinki University of Technology, 2005' (Heikkinen I) describes generation of two non- concentric image sequences and their registration in a common coordinate system. Moreover, three-dimensional measurements can be conducted in the method to enable reconstruction of the objects.

Relating to imaging in general, there are a number of publications of the prior art, represented herein by the following ones. Some of them are referred to below.

1) Chapman, D., Deacon, A. and Brown, J. -L., 2004. An omnidirectional imaging system for the reverse engineering of industrial facilities, in H. -G. Maas and D. Schneider (eds) , Panoramic Photogrammetry Workshop, Vol. XXXIV, Part 5/W16 of International Ar- chives of Photogrammetry and Remote Sensing, ISPRS, Dresden, Germany, pp. 1-8.

2) El-Hakim, S., Beraldin, J. -A. and Picard, M., 2002. Detailed 3d reconstruction of monuments us- ing multiple techniques, in W. Boehler and P. Patias (eds) , Int. Workshop on Scanning for Cultural Heritage Recording (CD) , CIPA WG6 and ISPRS Comm. V, Corfu, Greece, pp. 58-64.

3) Hartley, R., 1993. Photogrammetric tech- niques for panoramic cameras, Integrating Photogrammetric Techniques with Scene Analysis and Machine Vision, Vol. 1944, Orlando, U.S.A., 13 p.

4) Heikkinen, J., 2005. (Heikkinen I) The Circular Imaging Blocks in Close-Range Photogrammetry, Doctoral dissertation. Publications, TKK, Institute of Photogrammetry and Remote Sensing, Vol. 1/2005, ISSN1796-O711,ISBN 951-22-7965-7 (printed) and ISBN 951-22-7966-5 (pdf) , http://lib.tkk.fi/Diss/2005/isbn9512279665/, Espoo, Finland, 2005, 142 p.

5) Heikkinen, J., 2004. (Heikkinen II) Accuracy Analysis of Circular Image Block Adjustment, International Archives of Photogrammetry and Remote Sensing, Vol. XXXV, Part B5, ISPRS, Commission V, WG V/l, XX Congress Proceedings, July 12-23, 2004, Istanbul, Turkey 2004, pp. 30-35.

6) Heikkinen, J., 2000. (Heikkinen III) Circular Image Block Measurements , International Archives of Photogrammetry and Remote Sensing, Vol. XXXIII, Part 5A, ISPRS Commission V XIX Congress Proceedings, July 16-23, 2000, Amsterdam, the Netherlands, pp. 358- 365.

7) Kukko, A., 2004. A new method for perspective centre alignment for spherical panoramic imaging, The Photogrammetric Journal of Finland 19(1): 37-46. 8) Luhmann, T. and Tecklenburg, W., 2002. Bundle orientation and 3-d object reconstruction from multiple-station panoramic imagery, Close-Range Imaging, Long-Range Vision, Vol. XXXIV, Part 5 of Interna- tional Archives of Photogrammetry and Remote Sensing, ISPRS Symposium Comm. V, Corfu, Greece, pp. 181-186.

9) Parian, J. A. and Gruen, A., 2004. A refined sensor model for panoramic cameras, in D. S. H-G Maas (ed. ) , Panoramic Photogrammetry Workshop, Vol. XXXIV, Part 5/W16, ISPRS, Dresden, Germany, 12 p.

10) Peleg, S. and Ben-Ezra, M., 1999. (Peleg II) Stereo panorama with a single camera, Computer Vision and Pattern Recognition (CVPR) , IEEE, Fort Collins, Colorado, USA, pp. 1395-1402. 11) Pontinen, P., 1999. On the creation of panoramic images from image sequences, Photogrammetric Journal of Finland 16(2): 43-67.

12) Rόnnholm, P., Hyyppa, H., Pontinen, P., Haggren, H. and Hyyppa, J., 2003. A Method for Inter- active Orientation of Digital Images Using Backprojec- tion of 3D Data, Photogrammetric Journal of Finland 18(2): 58-69.

13) Schneider et al . , 2005: 'An omnidirectional imaging system for the reverse engineering of industrial facilities, Panoramic Photogrammetry Workshop, Vol. XXXIV, Part 5/W16 of International Archives of Photogrammetry and Remote Sensing, ISPRS, Dresden, Germany, pp. 1-8'.

14) Seitz, S. M., Kalai, A. and Shum, H. -Y., 2002. Omnivergent stereo, International Journal of

Computer Vision 48(3): 159-172.

15) Sequeira, V, Ng, K., Wolfart, E., Gon- calves, J. and Hogg, D. 1999. Automated reconstruction of 3d models from real environments, ISPRS Journal of Photogrammetry & Remote Sensing 54(1): 1-22. 16) Wester-Ebbinghaus, W., 1982. Single station self calibration mathematical formulation and first experiences. Precision and Speed in Close Range Photogrammetry, Vol. XXIV, Part V/2 of International Archives of Photogrammetry and Remote Sensing, ISPRS Symposium Comm. V, Yorks, UK, pp. 533-550.

17) Wendt, A., 2004. On the automation of the registration of point clouds using the metropolis algorithm. Vol. XXXV, Part B/3 of International Archives of Photogrammetry and Remote Sensing, ISPRS Congress, Istanbul, Turkey, 6 p.

Also the following publications have been disclosed in the prior art.

Specification GB 2417570 describes generation of a cylindrical 360 degree image. This is accomplished using a projector from which the light is guided via a dome-shaped mirror on a cylindrical surface.

Specification US 6,023,588 describes genera- tion of a 360 degree panoramic image. In the camera assembly, the 360 degree images are taken from positions on the same vertical axis.

Specification US 2005/0128196 describes an apparatus for 3 dimensional modeling. The apparatus comprises a video camera and a computer. The 3D model is generated by analyzing image points for example as for colors and redundancy.

Specification US 2002/0159032 describes a multilayer image generation system. It comprises mainly a liquid-crystal-type display device for stereo images. In accordance with Fig. 2, panel-type units are positioned adjacently, and the display units are illuminated by light sources from a specific direction. Specification US 6,009,190 describes generation of a panoramic image by means of a stationary camera.

Specification US 2001/0040671 describes an apparatus required for displaying a panoramic image for example in a movie theater suitable for this objective. Using the apparatus, the image is projected on a surface that covers 360 degrees, such that the projectors are radially oriented as seen from the cen- ter of the screen. The projection can be directed outwards from the center or towards the center from outside.

The following section comprises description of background information for various known imaging methods.

In general terms, panoramic image combinations can be displayed by any mathematically defined map projection, such as, for example, a planar, cylindrical, conical or spherical projection. As to imaging of interior spaces, panoramic images can be considered an ideal way of visualizing such spaces. The idea of panoramic images per se has already been known for over a century, but only recently has it become necessary to apply the principle of panoramic images to close-range photogrammetry. One panoramic image can thus be provided by joining together multiple images provided by central projection. An alternative is to use a specific camera designed for taking wide-angle images . In order to form a panoramic image from multiple central projection images, it is essential that all individual images have the same projection center. Positions of the projection centers of the images can be mathematically estimated in order to discover the possible eccentricity of the projection centers, which can be taken into account in the generation of the panoramic image (for example as in publications Wester-Ebbinghaus, Hartley and Luhmann et al . ) . One alternative is to place the camera on a rotatable support such that (as, for example, in the example of Fig. 6) there will not be any difference between the perspective centers of the images (publications Pόnti- nen and Kukko) .

In the case of specific panoramic cameras, the exposure is based on a so-called light-sensitive line sensor. The panoramic image is provided for example such that the vertically positioned line sensor passes in a horizontal direction so that the light intensity information can be stored for the entire area of the rotation angle. The resolution of the image de- pends therefore, in the horizontal direction, on the resolution resulting from the rate of the rotational motion and from the properties of the camera, and, in the vertical direction, on the number of elements in the line sensor. In this example, the line sensor is assumed to be parallel to the rotation axis. However, this assumption has been proven incorrect in publication Schneider. On the other hand, in publication Parian, rotation of the sensor has been detected to be uneven and the so-called planarity constant to change. By applying the correction terms based on calibration, it will be possible to reduce the systematic error from ten pixels to 0.2 pixels (Schneider).

However, reconstruction of an object based on intersections of the image rays cannot be provided based on one panoramic image. Image observations from a panoramic image can be compared for example with observations on vertical and horizontal angles from theodolite. Consequently, at least two panoramic images having different perspective centers must be gen- erated in order to measure the position of a 3D point with a method which utilizes the intersections of im- age rays. Acquisition of redundancy, i.e. additional information, to the observations requires at least three panoramic image sequences . Before reconstructing the object, the position of the images must be known either in the desired coordinate system or relative to the position of some other image. In photogrammetry, a so-called bundle adjustment method can be used to optimize the measurement accuracy by reducing the random error in the case of redundant observations. In this situation, there will be more observations available than what is necessary. In publication Luhmann et al . it has been observed that generally 5 to 7 tie points are sufficient for determining the relative position of the panoramic images in a room. Corresponding image point observations must be made for the objects of the image, similarly as with central projection images. However, unlike in the case of individual images, the epipolar line is a sinusoidal curve instead of a straight line. This is be- cause the panoramic image is projected on the surface of a cylinder. In interior imaging, the advantage provided by panoramic images is that they can cover full 360 degrees of the interior space, exploring the space outwards from the inside. This means that it is possi- ble to see the desired point of an object in all panoramic images. However, scaling of the images may vary greatly from one image to another. This is why it has become necessary to generate an image management and image browsing system for many research projects in order to find all possible images in which the measured point of the object can be seen (Luhmann et al . and Chapman et al . ) . In complex environments, the image management system has a very important role in the performance of the imaging task (Chapman et al . ) . 3D measurements that are based solely on images typically require that at least two images having different projection points (i.e. projection centers) be available. An image sequence of a traditional panoramic image does not fulfill this condition, because the projection center remains at the same position in all images in the sequence. Publication Heikkinen I describes the method of circular imaging sequences which provides an imaging system that is comparable to panoramic images. In the system in accordance with Heikkinen I, the camera is rotated about a stationary navel point along a circular trajectory. This type of imaging fulfills the condition of multiple perspective views referred to above and also covers horizontally full 360 degrees of the view.

If two concentric image sequences are gener- ated such that only the viewing direction of the camera is different relative to the trajectory of the camera in different image sequences, an adequate imaging geometry can be achieved for three-dimensional measurements. If the image sequences are taken such that the viewing directions of the camera are opposite in different image sequences (such as in the example of Fig. 2) , it has been observed that the accuracy of the object measurements will be approximately 1:2000, which is the ratio between accuracy of the object measurement and size of the object. In this example, the radius of the rotational movement was r = 0.45m, and the distances between the object point and the measurement center varied from 2 to 15m (Heikkinen I). In the situation of the example, the position of the camera at the moment of exposure is presented in polar coordinates and the viewing direction of the camera depends on the plane rotation relative to the orientation of the first camera. These quantities are described by equations (1) and (2) :

x = r^• cosα y = constant ( 1 ) z = r ^■ sinα

⁾

In equation (2), the plane rotation is applied to the so-called rotation matrix R (marked with subindexes '0') of the first image to determine the rotation matrices of other images at the moment of ex- posure. The estimation model is based on tie point measurements obtained from overlapping images . The so- called LSQ-estimation is applied in order to solve the problem referred to above for determining the unknown rotation angles. If two concentric image sequences are used, estimation can be applied to both sequences simultaneously. This ensures that both image sequences will be positioned in the same coordinate system. Only image observations are used in the estimation, which is why the coordinate system is, so to speak, local. A scale can also be used in the object space in order to convert the measurement results for example into the metric system. A more detailed description of the model has been disclosed in publications Heikkinen I

(2005) and Heikkinen II (2004) . The configuration presented herein provides, in principle, an adequate imaging geometry for object measurements. Although a stereo view can basically be provided by means of the two image sequences referred to above, a problem arises from the fact that the ste- reo view will not be natural, because the two views are mutually too wide apart from each other.

Disclosed in prior art are also results from some research projects in which a stereo imaging system with panorama-type viewing capabilities have been examined (Peleg II; Seitz et al . ) . Stereopanoramic images can, thus, be generated by combining two image sequences. Fig. 7 represents the differences of a stereo panorama (left image) and a stereoscopic panorama provided with a true panoramic image (right image) from the viewer's perspective. The objective is to generate a panoramic image which consists of image strips that have been extracted from both image sequences. The image strip is extracted from each image, the viewing angle of the image strip being tangential to the trajectory of the camera. In order to enable stereo viewing, the image is projected on a planar surface as a stereo pair for which the viewing direction can be selected. Neither of the image sequences fulfills the condition of concentricity when the panoramic image is being generated. In the research projects, the objective has been to reduce redundant , i.e. repeating image information and, on the other hand, store from the surrounding view only those parts that are necessary for generating a panoramic image. Information concerning the image rays can be stored in a matrix form, such that image rays having the same horizontal angle can be stored in the same vertical image column and, respectively, image rays having the same vertical angle are positioned on the same horizontal line. This way, epipolar lines will coincide with scan-lines and the correlation between the image points can be observed from the same line. This provides an advantage when using traditional stereo matching algorithms for reconstructing the object. In prior art, the main objec- tive has been only to generate a stereoscopic panoramic viewing possibility, which is why reconstruction of the image has not been geometrically adequate in order to provide precise measurements. In prior art, it has also been attempted to reconstruct the object by the so-called stereo image matching (Seitz et al . ) . The objective of the methods referred to above has been to generate a visualization means for storing a sufficient amount of data for a three-dimensional view. In the method of concentric image sequences, the redundant information is retained, and the geometry of perspectively projected images together with known camera calibration information enables production of accurate 3D measurements.

Generation of a stereopanoramic image from two image sequences can be regarded as a kind of com- promise when examining the viewing convenience and the capability of geometrical measurements. If the trajectory of the image sequences is too distant from the so-called navel point, stereoscopic viewing will basically not be practical. On the other hand, if the ra- dius of the trajectory of the camera is too short, the imaging geometry becomes worse and 3D measurements can be considered reliable only within short distances .

A problem with the prior art has been the generation of an adequately natural combination of a stereo image and a panoramic image. Combining three- dimensional measurements with the characteristics of the captured images has also been problematic.

OBJECTIVE OF THE INVENTION The objective of the invention is to disclose a new type of way to generate stereopanoramic images. One specific objective of the invention is to alleviate the problems referred to above.

SUMMARY OF THE INVENTION

The present invention describes a novel method and apparatus for generating a stereoscopic panoramic image. In the method of the invention, a panoramic image is taken of the surrounding scene from a desired projection point, at least one image sequence is taken of the surrounding scene such that the projection point of the image sequence moves at a desired distance from the projection point of the pano- ramie image when the image sequence is being taken, and the stereopanoramic image is generated by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the desired image sequence. In one embodiment of the present invention, said part of the panoramic image is projected on a plane, and the image from the image sequence to constitute part of the stereo pair is selected such that the viewing direction of the image is closest to the direc- tion of the normal of said plane.

In one embodiment of the invention, the image sequence is taken using a time-lapse camera, such that the time-lapse camera is rotatable about a substantially stationary navel point along a circular trajec- tory, and that the projection point of the panoramic image and the projection points of the individual images from the image sequence are positioned substantially on the same plane.

In one embodiment of the present invention, the time-lapse camera is fixed at a constant distance outside the navel point, the first image sequence is stored by rotating the time-lapse camera about the navel point between exposures, the time-lapse camera being oriented at each moment of exposure substantially in a direction that is tangential to the trajectory of the camera, and the second image sequence is stored by rotating the time-lapse camera about the navel point between exposures, the time-lapse camera being oriented at each moment of exposure substantially in a direction that is tangential to the trajectory of the camera, and opposite as compared to the above-mentioned directions . The first stereopanoramic image is then generated by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the first image sequence. The second stereopanoramic image is then generated by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the second image sequence.

In one embodiment of the present invention, the panoramic image is projectable on a cylindrical surface, and the image sequence is taken using horizontal rotation and viewing directions of the time-lapse camera .

In one embodiment of the present invention, the panoramic image to be taken is projectable on a spherical surface.

In one embodiment of the present invention, the light coming into the time-lapse camera is deflected using at least one light-reflecting and/or light-refracting element during exposure of the image sequences.

In one embodiment of the present invention, the time-lapse camera is rotated in a substantially vertical direction, the exposures are repeated until the desired viewing angle has been entirely captured in the vertical direction, the time-lapse camera is rotated substantially horizontally to the extent of the desired viewing angle, and said exposures in conjunction with the vertical rotation and the horizontal ro- tations are repeated until the desired image sequence that is projectable on a cylindrical surface has been taken .

In one embodiment of the present invention, the panoramic image and the image sequences are regis- tered by adapting their coordinate systems to each other. In one embodiment of the present invention, three-dimensional measurements are conducted for at least one image from the image sequence and for the panoramic image, the coordinate system of the measure- ment results is adapted to the coordinate system of the exposed images, and the measurement data derived from the object is modified based on the stereopanoramic image.

In one embodiment of the present invention, a laser scanner is used for conducting the three- dimensional measurements .

In one embodiment of the present invention, the discrepancy between the actual trajectory and the desired trajectory of the time-lapse camera is compen- sated through correction using a mathematical model.

The inventive idea of the present invention also comprises an apparatus for generating a stereopanoramic image. The apparatus comprises at least one camera, rotation means for each camera, and control means for controlling the apparatus. The apparatus of the invention is characterized in that at least one camera of the apparatus is a panoramic camera arranged to take a panoramic image of the surrounding scene from a desired projection point. In the apparatus, at least one time-lapse camera is also arranged to take an image sequence of the surrounding scene, such that the projection point of the image sequence moves at a desired distance from the projection point of the panoramic image when the image sequence is being taken, the appara- tus further comprising said control means, arranged to generate the stereopanoramic image by combining stereo pairs such that one stereo pair is generated by combining part of the panoramic image and one image from the desired image sequence. In one embodiment of the present invention, the apparatus further comprises said control means for projecting said part of the panoramic image on a plane, and said control means for selecting the image from the image sequence to constitute part^' of the stereo pair such that the viewing direction of the image is closest to the direction of the normal of said plane.

In one embodiment of the present invention, the apparatus further comprises a time-lapse camera for taking the image sequence, such that the time-lapse camera is rotatable about a substantially stationary navel point along a circular trajectory, and that the projection point of the panoramic image and the projection points of the individual images from the image sequence are positioned substantially on the same plane. In one embodiment of the present invention, the apparatus further comprises the time-lapse camera fixed at a constant distance outside the navel point, said control means for storing the first image sequence by rotating the time-lapse camera about the navel point between exposures, the time-lapse camera being oriented at each moment of exposure substantially in a direction that is tangential to the trajectory of the camera, and said control means for storing the second image sequence by rotating the time-lapse camera about the navel point between exposures, the time-lapse camera be- ing oriented at each moment of exposure substantially in a direction that is tangential to the trajectory of the camera and opposite as compared to the above- mentioned directions . The control means are also arranged to generate the first stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the first image sequence. The control means are also arranged to generate the second stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image- from the second image sequence .

In the apparatus in accordance with one embodiment of the present invention, the image of the panoramic camera is projectable on a cylindrical surface, and the image sequence is taken using horizontal rotation and viewing directions of the time-lapse camera.

In the apparatus in accordance with one em- bodiment of the present invention, the image taken by the panoramic camera is projectable on a spherical surface.

In one embodiment of the present invention, the apparatus further comprises at least one light- reflecting and/or light-refracting element for deflecting the light coming into the time-lapse camera during exposure of the image sequences .

In one embodiment of the present invention, the apparatus further comprises rotation means for the camera, the means being arranged to rotate the time- lapse camera in a substantially vertical direction. The control means are also arranged to repeat the exposures until the desired viewing angle has entirely been captured in the vertical direction. The rotation means for the camera are arranged to then rotate the time-lapse camera in a substantially horizontal direction to the extent of the desired rotation angle, and said control means are arranged to repeat said exposures in conjunction of the vertical rotation, and the horizontal rota- tions until the desired image sequence that is projectable on a cylindrical surface has been captured.

In one embodiment of the present invention, the control means of the apparatus further comprise registration means for registering the panoramic image and the image sequences by adapting their coordinate systems to each other. In one embodiment of the present invention, the apparatus further comprises measuring means for conducting three-dimensional measurements for at least one image from the image sequence and for the panoramic image. Said registration means are further arranged to adapt the coordinate system of the measurement results to the coordinate system of the captured images. Modifying means are also arranged to modify the measurement data derived from the object based on the stereopano- ramie image.

In one embodiment of the present invention, the apparatus further comprises a laser scanner for conducting the three-dimensional measurements.

In one embodiment of the present invention, the apparatus further comprises said control means for compensating the discrepancy between the actual trajectory and the desired trajectory of the time-lapse camera through correction using a mathematical model.

The inventive idea of the present invention further comprises a computer program for generating the stereopanoramic image, the computer program comprising program code. The computer program is characterized in that it is, when run on a data-processing device, arranged to take a panoramic image of the surrounding scene from a desired projection point. Furthermore, by means of the computer program, at least one image sequence is taken of the surrounding scene, such that the projection point of the image sequence moves at a desired distance from the projection point of the pano- ramie image when the image sequence is being taken, and the stereopanoramic image is further generated by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the desired image sequence. In a preferred embodiment, the computer program is arranged to perform different steps of the method referred to above either entirely or to the appropriate extent .

The combination of a stereo image and a panoramic image in accordance with the invention differs from the known prior art (Peleg II and Seitz et al . ) in that in the invention, the true panoramic image constitutes part of the stereo image. In the method by Peleg II and the method in which the stereo image is obtained from different view points (Seitz) , the stereoscopic image is provided from two different image sequences .

The advantage of the present invention, as compared for example with specification Peleg I, is that in prior art, the image sequences comprise camera movement only horizontally. The present invention enables also the vertical tilting movement of the camera when the image sequences are being taken. Moreover, the imaging accuracy is improved, because, when capturing the image sequences, observations of each ob- ject point can be obtained from several images in the image sequence.

The invention also provides the advantage of making the registration easier, because the view point of the panoramic image is the same as the rotational center of the trajectories of the image sequences. Furthermore, the use of a laser scanner provides the advantage of being able to directly produce three- dimensional data of the object in the form of a laser point cloud. Thus, the laser scanner can be positioned at the same view point referred to above, so that the intensity images obtained can be used in combining the different data sets.

LIST OF FIGURES Fig. 1 represents an imaging arrangement for a three-dimensional panoramic presentation, Fig. 2 represents a camera assembly for storing the image sequences,

Fig. 3 represents an arrangement in which a component that deflects rays of light is associated with the exposure geometry,

Fig. 4 represents projection of a cylindrical image on a plane, such that the horizontal line of the planar image is parallel to the so-called epipolar line, Fig. 5 represents an imaging arrangement in which a stereopanoramic image is generated such that it is spherically projected, utilizing also the tilting movement in the exposure geometry,

Fig. 6 represents an example of a camera sup- port on which the camera is rotatable,

Fig. 7 represents the principle of a stere- opanorama and a stereopanorama provided with a true panoramic image,

Fig. 8 shows comparison of two principles of forming a stereopanorama, and

Fig. 9 represents projective rectification of concentric images .

DETAILED DESCRIPTION OF THE INVENTION For the features characteristic of the invention, reference is made to the claims.

The present invention describes a method and an apparatus to enable generation of a stereopanoramic image. To the appropriate extent, it is also possible to implement the method by means of a computer program. In the invention, the apparatus that generates the stereopanoramic image is a combination of an apparatus that takes a true panoramic image (for example a panoramic camera) and an apparatus that takes at least one image sequence (in this conjunction a so-called time-lapse camera) . The stereo view is generated by combining true panoramic image with images from the image sequence. In one embodiment of the invention, the image sequence is stored by rotating the camera between exposures about a fixed navel point, such that the camera is positioned at all times at a specific fixed distance from said rotational center. The trajectory of the time-lapse camera that stores the image sequence is, in a preferred embodiment of the invention, a planar circle. The direction of the sensor (i.e. the direction of the optical axis of the camera) is substantially tangential to this circle defined by the trajectory, as shown in Fig. 2. As also shown in Fig. 2, the distance of the camera from the rotational center is r, and, between taking two different image sequences, the camera is rotated 180 degrees (image sequence 1 and image sequence 2) . Angle oti represents the horizontal rotation of the camera. The positioning of the camera of Fig. 2 produces thus two different image sequences. With both orientations of the camera, the camera rotates at a distance of radius r from the rotational center, which is the reason for referring to two concentric circular image sequences, in which the path approximates the trajectory of a circle.

It is useful to store the panoramic image such that the projection point of the rotating camera is practically positioned at the navel point, i.e. at the rotational center. With the positioning of the camera is this manner, the image that is composed of the image sequence can be called a true panoramic image. Ideally, the projection center of the panoramic image (the position of the camera) and the position of the rotational center of the projection centers of an image sequence is one and the same point in three- dimensional space, as shown in Fig. 1. In one embodi- ment of the invention, the apparatus can be provided with an auxiliary means which reflects or refracts rays of light, for example a prism or other element that deflects rays of light. This type of embodiment is shown in Fig. 3 and 5. By deflecting the rays of light, it will be practically possible to provide a similar type of measurement geometry as that shown in Fig. 2.

A stereoscopic image is provided such that a panoramic image and a symmetrical image sequence are combined in a selected viewing direction. The stereo image thus refers to two generated images, one of which has been projected from a panoramic image and the other one from an individual image of the image sequence on a common planar surface, such that the viewing direction of an individual image from the im- age sequence is closest to the direction of the normal vector of the projection plane. Generation of a stereo image results thus in the production of two images, in which the epipolar line is parallel to the direction of the x-axis of the image, as shown in Fig. 4. In other words, an individual stereo pair of the stereopanoramic image is generated by combining part of the panoramic image and an individual image from the image sequence. A full stereopanoramic image can be formed of a set of adjacent stereo pairs. In an individual stereo pair, the horizontal viewing angle covered by the part of the panoramic image (i.e. the relative width of the image) is selected such that it is approximately equal to the horizontal viewing angle covered by the respective individual image from the image sequence as seen from the projection point.

The panoramic image can be produced by means of digital image sensors with one or more sensor rows placed one upon the other. The sensors receive electromagnetic energy (light) while they are rotated 360 degrees about a fixed axis in a manner that is similar to the traditional panoramic cameras . This can be re- alized for example such that the sensor rows are placed vertically, so that the sensor array thus rotates horizontally. For example a normal image sensor of a digital camera can be used in this situation in order to form a panorama by combining multiple traditional photographs . The principal of this type of image formation has been disclosed in publication Pόnti- nen.

One example of projecting adjacent images of a panoramic image composed of multiple individual images on a common plane is shown in Fig. 9. The procedure is referred to as so-called projective rectification. The images are concentric, and the plane of the middle image serves as the rectification plane in the case of Fig. 9.

The image sequence can be stored following the principal in accordance with Fig. 2 with two different camera positionings . The quality of the captured images depends on the number of the images in the image sequence, i.e. on the so-called exposure rate. This can also be referred to herein as the digitizing rate of the rotational movement. Depending on which direction the camera is oriented (direction of the optical axis of the image sensor) , the panoramic image that is taken will function in the stereo image as the left or the right image. If the image sequence is taken for example using both positions of the image sensor referred to above, it will be possible to provide two slightly different viewing angles for the panoramic image taken from one spot.

The two methods referred to above are not mutually exclusive when storing image data or reconstructing images. The benefits of both techniques can be taken advantage of in combining panoramic imaging with the image sequences. The advantage of the panoramic image is the view that covers 360 degrees as seen from one spot, providing a realistic view of the observed surrounding scene. On the other hand, by means of the image sequence or sequence in which the camera is moved along a predetermined trajectory such that, in this case as well, the camera angle is rotated full 360 degrees, another kind of a comprehensive representation of the surrounding scene can be provided. If the trajectory of the camera is a circle positioned on an arbitrary planar surface or, in the simplest scenario, horizontally, such trajectory can be mathematically modeled. If the center of the captured concentric image sequences is the same as the perspective center of the panoramic image, it will be possible to generate a stereoscopic panorama relative to this center.

In one embodiment of the present invention, the panoramic camera functions also as the time-lapse camera. In this case, the panoramic image and the image sequence can be stored successively with the same apparatus under control of the control means. In another embodiment of the invention, the panoramic camera and the time-lapse camera are separate cameras and may comprise different optical and other properties.

In one embodiment of the invention, a proces- sor, a microcontroller or such functions as the control means .

In one embodiment of the present invention, the traditional optics used in the art can be applied in order to provide a situation that equals to the natural viewing experience. In this case, the panoramic image is assumed to be a cylindrical projection taken of a panoramic view, the cylinder axis being same as the rotation axis of the camera (as in Fig. 1) . The positions of the projection point of the cy- lindrical panoramic image and of the projection cen- ters of the image sequence are assumed to be at least approximately on the same plane (as in Fig. 4) .

In another embodiment of the present invention, use can be made of a wide angle lens in order to provide a viewing angle of 180 degrees. Such lens is also referred to as a fish-eye lens. The wide angle lens is positioned in front of the sensor, i.e. for example a light-sensitive cell, so that a spherical panoramic image that captures comprehensively the en- tire surrounding scene can be taken.

In one embodiment of the invention, a planar cell can be used, so that multiple photographs can be combined to form a spherical image by suitably altering the orientation of the cell if the projection cen- ter of the cell is kept stationary. In each horizontal rotational angle of the image sequence, the viewing direction of the time-lapse camera can also be rotated in the vertical direction (i.e. tilted), such that one photograph is taken in each orientation, operating in accordance with the principle of Fig. 1. In this manner, a tilted image can be taken, which means that the optical axis of the cell is moved between exposures only in the vertical direction, and the projection center is kept stationary. When a tilted image (the desired number of individual images) has been fully provided for one horizontal angle, the camera will be rotated horizontally into a new orientation, and the tilting movement of the camera and the exposures will then be repeated. The exposure geometry can also be provided by using a prism or a light-reflecting element (for example a mirror) attached to a rod-like element. This type of generation of a stereopanoramic image enables the formation of such stereo image which fully surrounds the viewer, i.e. models the surround- ing scene spherically and not cylindrically. The only requirement is that the so-called base vector of the image should be on the same plane. The spherical image in accordance with the example can also be provided by using multiple image sensors, so that a tilted image can be taken in one exposure. The sensor array can be rotated horizontally to the extent of the desired rotation angle after the exposures, and the exposures repeated as described above.

Registering the images (adapting together the coordinate systems of the panoramic image and the im- ages from the image sequence) in the coordinate system used is based only on observations from the actual images. The viewing direction of the cell (the direction of the normal of the cell surface) is calculated based on points in said image and/or on features included in the image. When the image sequence is being taken, also mutual relations between adjacent images, with special regard to directions, as well as the assumed trajectory of the cell are taken into account in the calculation of the viewing direction of the cell. In a preferred embodiment, all rotation angles of the cell are determined in one calculation step. The dependencies between the image sequence and the panoramic image are also resolved by means of image observations, i.e. points in the images or features of the images that correspond to each other. However, in order to resolve the length of the base vector, some known distance must be introduced into the computation as a reference value.

Management of the geometrical properties of an image in arbitrary conditions, as well as generation of a mathematical model that models adequately the situation is demanding. In reconstructing objects, accurate registration and attributes associated with the positional information of the image are decisive. Modeling and resolving the positional reference and the direction of the image are one of the most sub- stantial challenges in photogrammetry. The direction can be determined in the desired coordinate system or relative to other data, such as for example data obtained by laser scanning. Laser scanning will be de- scribed in more detail below. After resolving the positional information, the images can be used for updating the data that relates to the actual surrounding scene. In this manner, different imaging sources can be combined in order to provide means for collecting and visualizing three-dimensional data for different applications. Such applications include for example navigation systems, mobile telephone applications, WWW applications, service applications relating to real estate business, game applications, telecommunications applications, virtual museums and animated applications in general. For example in game industry, a three-dimensional image that imitates well reality has become a standard for the graphics used in generating game worlds. In real estate business, on the other hand, it is advantageous to generate a three- dimensional presentation of the property for sale which imitates well reality, for example to be displayed on the WWW pages of the real estate agent. A stereopanoramic image presentation can also be used as an instrument in a situation in which three- dimensional data on the surrounding scene is modified or combined. In this case, it should be taken into account that in combining the data, the different data sets should be defined in a common coordinate system. In one embodiment of the invention, in order to provide accurate 3D measurements, two cameras and two image sequences to be captured can be used in conjunction with the method of the concentric image sequences. The mutual distance of the positions of the camera between two image sequences is then equal to the diameter of the circular trajectory of the camera in the situation where the viewing directions are the same. If a stereopanoramic image is generated by combining a panoramic image and two image sequences, it will be possible to generate two separate stereoscopic panoramas with different view points. In this case, one stereopanoramic image corresponds to the positioning of the left eye and the other stereopanoramic image corresponds to the positioning of the right eye in the navel point (the upper image of Fig. 8) . By way of comparison, the images on the left in Fig. 7 and below in Fig. 8 represent the principle of the stereopanoramic image in accordance with the prior art, in which neither of the view points is positioned in the navel point, but are instead positioned at a same specific distance from the navel point.

In one embodiment of the invention, the camera is fixed on a support such that the view points are positioned on the same planar surface. However, this is not absolutely necessary for the operability of the method. Discrepancy of the view points from the same planar surface can be compensated when necessary using a mathematical model, and the image can be then resampled.

A more comprehensive task in this situation is to ensure that the projection point of the panoramic image (with the viewing direction oriented directly outwards) is as close as possible to the center of the image sequence. Eccentricity can be detected in practice as the varying length of the base vector. When this variation exceeds a certain limit, it causes disturbance if viewing for example a stereoscopic image sequence transmitted by stream technique, i.e. in other words a video clip. On the other hand, the human eye accommodates quite well to small changes in the base vector. Since the stereo image system is mainly used for visual interpretation and the actual measure- merits are conducted based on the image sequences, a small degree of eccentricity can be ignored.

Generation of a stereo pair thus comprises conversion of the cylindrical projection of a pano- ramie image into an image that is projected on a plane. This planar surface should be the same as the planar surface of the image selected from the image sequence. Thus, the task is to generate a projection of a part of the panoramic image with the same view as in the individual image. The apparatus can determine mutually corresponding parts or properties from the images, so that these two images can be accommodated in the same coordinate system. The procedure should be equivalent to the registration performed on the images in the image sequence, with the difference that the image geometry is different in these two cases.

In this context, one of the images (either the right or the left image) is taken from the image sequence and is therefore a traditional frame image. This image can be used as the reference image for generating a stereo pair for a specific viewing angle. However, transformation from one planar surface to another can be necessary in some situations. The rate of the stereo pairs obtained depends much on the number of images in the image sequence. In order to display the stereoscopic panorama smoothly and well, the number of the required images will become quite large. The increase in the number of images is actually a great challenge for the development of automatic image registration.

One embodiment of the present invention relates to an apparatus in which three-dimensional imaging is combined with additional data associated with the three-dimensional image. In this conjunction, the apparatus is referred to as an augmented 3D imaging system. In this embodiment, two different imaging methods are combined for being used in a laser scanning system. The purpose is to combine the principle of the panoramic image with two concentric image sequences, and, also, by using the laser scanning sys- tern, to provide an augmented 3D imaging. The objective of the system is to adapt together image data accommodated in a spherical form and image data in a traditional planar form. The concentric image sequence and the 360 degree view of the panoramic image can be com- bined such that a stereoscopic image can be generated of the surrounding scene. Furthermore, the image set of the image sequence can be used for conducting accurate three-dimensional measurements. Thanks to the embodiment, it will be possible to visualize the sur- rounding scene very realistically. Additional three- dimensional data obtainable for example by means of laser scanning can be stored together with the image data. The measurement data must also be registered in a common coordinate system with the image data. By means of laser scanning, the system can be converted into a mapping system in which the advantages associated with methods based on laser technique and image can be utilized in reconstructing the object. On the other hand, when applying laser scanning and panoramic imaging to the same object, modification of the set of 3D points based on visual observations will be considerably improved. A view from the same view point is preferable in the interpretation of the sets of points obtained from the laser scanner, and the efficiency of modification of the object data will clearly be improved thanks to the possibility of viewing the 3D data. The possibility to project three- dimensional parts accurately onto the stereo view, combined with storing the images in an adequate reso- lution, enables modeling of the surrounding scene re- alistically for example for the needs of the applications referred to above.

In one embodiment of the present invention, a device that performs laser scanning is thus utilized in order to detect, by means of the laser beam, the so-called point cloud associated with the image data, i.e. a large set of unstructured data associated with for example ground geometry. An accurate projection of a three-dimensional point cloud measured by the laser scanner on the images requires thus also registration of the point could data. The measurement results obtained from the laser scanner are based on angular observations or on both angular and distance observations, depending on whether the device uses triangula- tion or ranging technique. Regardless of the laser used, the point clouds can be projected and observed from one perspective point. Consequently, it is logical to combine stereopanoramic imaging and laser measurement results obtained from the same point. The measurement results from the laser scanner are, for example, three-dimensional coordinates of reflection points on the ground. The task will then be to extract and match the desired 3D data between images and measured coordinates. When the viewing direc- tioήs of the laser point cloud and the part of the panoramic image are close to each other, the use of the so-called intensity images obtained from the laser scanner becomes easier in the registration of the data sets. Similarities can be more efficiently adapted to each other because the intensity and range images from the laser are from the same view point as the panoramic image.

With laser scanning, the problem with the prior art has often been the accurate registration of data from separate measurements (for example Sequeira et al . and Rδnnholm et al . ) . If the laser beam hits the surface obliquely, the inaccuracy increases. Such critical surfaces are difficult to detect directly, because the data obtained from the laser scanner is formed solely of the point coordinates. By means of stereopanoramic images, even errors in measurement results of the laser device can be located from the measurement data. With image data, the accuracy of positioning of the laser data can clearly be improved. Image data is preferably used for example for imaging edge areas of objects or components included in the image, whereas laser measurement is more suitable for reconstruction of smooth surfaces and more detailed structures .

The invention is not limited merely to the examples of its embodiments referred to above; instead many variations are possible within the scope of the inventive idea defined by the claims.

Claims

1. A method for generating a stereopanoramic image, charac teri zed in that it comprises the steps of: taking a panoramic image of the surrounding scene from a desired projection point; taking at least one image sequence of the surrounding scene, such that the projection point of image sequence moves at a desired distance from the pro- jection point of the panoramic image when the image sequence is being taken; and generating a stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the desired image sequence.

2. The method in accordance with claim 1, characteri z ed in that it further comprises the steps of: projecting said part of the panoramic image on a plane; and selecting the image from the image sequence to constitute part of the stereo pair such that the viewing direction of the image will be closest to the direction of the normal of said plane.

3. The method in accordance with claim 1, charac teri z ed in that it further comprises the step of: capturing the image sequence using a time-lapse camera, such that the time-lapse camera is rotatable about a substantially stationary navel point along a circular trajectory, the projection point of the panoramic image and the projection points of the individual images in the image sequence being positioned substantially on the same plane.

4. The method in accordance with claim 3, charac - teri z ed in that it further comprises the steps of: fixing the time-lapse camera outside the navel point at a constant distance; storing the first image sequence by rotating the time-lapse camera about the navel point between expo- sures, the time-lapse camera being oriented, at each moment of exposure, substantially to a direction that is tangential to the trajectory of the camera; storing the second image sequence by rotating the time-lapse camera about the navel point between expo- sures, the time-lapse camera being oriented, at each moment of exposure, substantially to a direction that is tangential to the trajectory of the camera and opposite as compared to the above-mentioned directions; generating the first stereopanoramic image by com- bining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the first image sequence; and generating the second stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the second image sequence.

5. The method in accordance with claim 1, charac ter i z ed in that the panoramic image is pro- jectable on a cylindrical surface, and that the image sequence is captured using horizontal rotation and viewing directions of the time-lapse camera.

6. The method in accordance with claim 1, charac t eri zed in that the panoramic image to be captured is projectable on a spherical surface.

7. The method in accordance with claim 1, charac ter i z ed in that it further comprises the step of: deflecting the light coming into the time-lapse camera using at least one light-reflecting and/or light-refracting element when the image sequences are being taken.

8. The method in accordance with claim 1, charac teri z ed in that it further comprises the steps of: rotating the time-lapse camera substantially in a vertical direction; repeating the exposures until the desired viewing angle has been fully captured in the vertical direction; rotating the time-lapse camera substantially in a horizontal direction to the extent of the desired rota- tion angle; and repeating said exposures in conjunction with the vertical rotation, and the horizontal rotation, until the desired image sequence, which is projectable on a cylindrical plane, has been captured.

9. The method in accordance with claim 1, charac teri z ed in that it further comprises the step of: registering the panoramic image and the image sequences by adapting their coordinate systems to each other .

10. The method in accordance with claim 9, charact er i z ed in that it further comprises the steps ^■ of: conducting three-dimensional measurements for at least one image from the image sequence and for the panoramic image; adapting the coordinate system of the measurement results to the coordinate system used by the captured images ; and modifying the measurement data obtained from the object based on the stereopanoramic image.

11. The method in accordance with claim 10, charact eri z ed in that it further comprises the step of: using a laser scanner for conducting the three- dimensional measurements.

12. The method in accordance with claim 1, char ac teri z ed in that it further comprises the step of: compensating the discrepancy between the actual trajectory and the desired trajectory of the time-lapse camera through correction by means of a mathematical model .

13. An apparatus for generating a stereopanoramic image, comprising: at least one camera; rotation means for each camera; control means for controlling the apparatus; charac teri z ed in that at least one camera in the apparatus is : a panoramic camera, arranged to take a panoramic image of the surrounding scene from the desired projection point, the apparatus also comprising: at least one time-lapse camera, arranged to take an image sequence of the surrounding scene, such that the projection point of the image sequence moves at a desired distance from the projection point of the panoramic image when the image sequence is being taken, and that the apparatus further comprises: said control means, arranged to generate the ste- reopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the desired image sequence.

14. The apparatus in accordance with claim 13, c h a - rac teri z ed in that it further comprises: said control means for projecting said part of the panoramic image on a plane; and said control means for selecting an image from the image sequence to constitute part of the stereo pair such that the viewing direction of the image will be closest to the direction of the normal of said plane.

15. The apparatus in accordance with claim 13, characteri z ed in that it further comprises : a time-lapse camera for taking the image sequence, such that the time-lapse camera is rotatable about a substantially stationary navel point along a circular trajectory, the projection point of the panoramic image and the projection points of the individual images in the image sequence being positioned substantially on the same plane.

16. The apparatus in accordance with claim 13, charac teri z ed in that it further comprises: a time-lapse camera fixed outside the navel point at a constant distance; said control means for storing the first image se- quence by rotating the time-lapse camera about the navel point between exposures, the time-lapse camera being oriented, at each moment of exposure, substantially to a direction that is tangential to the trajectory of the camera; said control means for storing the second image sequence by rotating the time-lapse camera about the navel point between exposures, the time-lapse camera being oriented, at. each moment of exposure, substantially to a direction that is tangential to the tra- jectory of the camera and opposite as compared to the above-mentioned directions; said control means, arranged to generate the first stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the first image sequence; and said control means, arranged to generate the second stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the second image sequence.

17. The apparatus in accordance with claim 13, charac teri z ed in that the image of the panoramic camera is projectable on a cylindrical surface, and that the image sequence is captured with horizontal rotation and viewing directions of the time-lapse camera.

18. The apparatus in accordance with claim 13, charac teri z ed in that the image captured by the panoramic camera is projectable on a spherical sur- face.

19. The apparatus in accordance with claim 13, charac teri zed in that it further comprises: at least one light-reflecting and/or light- refracting element for deflecting the light coming into the time-lapse camera when the image sequences are being taken.

20. The apparatus in accordance with claim 13, charac teri zed in that it further comprises: rotation means for the camera, arranged to rotate the time-lapse camera in a substantially vertical direction; said control means , arranged to repeat the exposures until the desired viewing angle has been fully captured in the vertical direction; rotation means for the camera, arranged to rotate the time-lapse camera substantially in a horizontal direction to the extent of the desired rotation angle; and said control means, arranged to repeat said expo- sures in conjunction with the vertical rotation, and the horizontal rotation, until the desired image sequence, which is projectable on a cylindrical surface, has been taken.

21. The apparatus in accordance with claim 13, characteri zed in that the control means of the apparatus further comprises: registration means for registering the panoramic image and the image sequences by adapting their coordinate systems to each other.

22. The apparatus in accordance with claim 21, charac teri zed in that it further comprises : measuring means for conducting three-dimensional measurements for at least one image in the image sequence and for the panoramic image; said registration means for adapting the coordinate system of the measurement results to the coordinate system of the captured images; and modifying means for modifying the measurement data obtained from the object based on the stereopanoramic image .

23. The apparatus in accordance with claim 22, charac teri z ed in that it further comprises: a laser scanner for conducting the three- dimensional measurements .

24. The apparatus in accordance with claim 13, characteri zed in that it further comprises: said control means for compensating the discrepancy between the actual trajectory and the desired trajectory of the time-lapse camera through correction by means of a mathematical model .

25. A computer program for generating a stereopanoramic image, the computer program comprising program code, charac teri z ed in that it has been arranged, when run on a data-processing device, to perform the steps of: taking a panoramic image of the surrounding scene from the desired projection point; taking at least one image sequence of the surrounding scene, such that the projection point of the image sequence moves at a desired distance from the projection point of the panoramic image when the image sequence is being taken; and generating the stereopanoramic image by combining stereo pairs, such that one stereo pair is generated by combining part of the panoramic image and one image from the desired image sequence.