CA2399713A1 - Design, function, and utilisation of an equipment for capturing of three-dimensional images - Google Patents

Abstract

The invention considers construction and usage of three-dimensional image capture equipment. The invention is based on capturing of two images different perspective with a predefined displacement, which is dependent on a number of parameters. The construction is based on the lens located in a displaceable cradle/sledge. On each side of the cradle/sledge there is a so-called system of cushions. These systems of cushions can influence the displacement of the lens through filling and emptying of liquid/gas/any other substance. The captured object is captured by a camera, for example a CCD camera which in turn transmits the image as a video signal to a computer. The computer fuses the two images through image processing to one 3D image. The 3D image can later be seen by the help of HMDs (Head Mounted Displays), on a monitor or through 3D glasses. The invention can with benefit be used in applications where the size or diameter of the instrument is decisive for the result, for example in the medical field, endoscopy, i.e. arthroscopy, laparoscopy, cystoscopy aso. Additionally it is important for the surgeon to get a spatial perception of the field of the operation. It will then become easier to estimate distances, sizes etc.

Description

Design, function, and utilisation of an equipment for capturing of three-dimensional images.
DEFINITION
Design, function, and utilisation of an equipment for capturing of three-dimensional images of different objects.
TECHNICAL AREA
The present invention refers to composition, utilisation and function of an equipment for capturing of three-dimensional images for medical purposes, for example for usage in micro-invasive surgery, for capturing of three-dimensional images of objects.
Not long time ago humanity wasn't aware of an existing separate binocular 'depth-sense'. Through ages individuals such as Euclid and Leonardo have understood that we see different images of reality with each eye. It was Wheatstone, who, with his stereoscope and his drawings, explained to the world 1838 that there is a unique 'depth-sense', stereopsis, produced by differences between retinal projections. Euclid, Kepler, and others wondered why we don't see a phantom image of reality. V~lheatstone explained that the problem was the actual solution, by demonstrating that the brain merges the two planar retinal images into one with stereopsis ("solid seeing") .1 Stereoscopic images A stereoscopic image presents to the observer's left and right eyes an image with pixels from different perspectives, exactly as the observer sees the visual reality. _ These two images, with slightly different perspectives, are synthesised by the cortex of the brain 1 Wheatstone, Charles. On some remarkable, and hitherto unobserved, phenomena of binocular vision (Part the first). Philosophical Transactions of the Royal Society of London, 1838, 371-94.

- 2 - PCT/SE00/00282 into one with stereoscopic depth. The synthesised image is bigger than the sum of the images.
The reality can also be captured by a camera. Irrespective of the image is calculated or captured, the generation or copying of images from two perspectives, one for the left and one for the right eye, must take place. If a perspective can be produced, it is conceptually possible to create another perspective.
The monocular, or extra-stereoscopic, depth-variables are the conditions for the perception of depth on visual displays. They are as important as stereopsis to create images, which are percepted as three-dimensional. These variables include light and shadow, relative size, interposition, text gradient, spatial perspective, motion parallax, and the most important is the perspective. Images rich on monocular depth-variables will be even easier to visualise when binocular stereoscopy is added.
Stereoscopic displays A stereoscopic display is an optical system, where the human brain is the end-unit. It functions by presenting left and right images of reality to the brain.
Those stereoscopic images are more realistic than their planar equivalence is accepted by most. Though, the addition of stereopsis to a display means a depth-sense, it can be discussed if this addition makes the display more realistic. Stereopsis adds information in a format, which is both sensoric, comfortable and useful.
A stereoscopic image, which has a one-to-one conformity, isomorphic, with reality, can be uncomfortable to look at, and be of debatable value for the scientist, engineer, technician, doctor and the artist.
There are many different ways a stereoscopic image can diverge from being isomorphic with reality, psycho-physics, the psychology of depth-perception, and geometric or

3~ optical changes. They can have the following states:
breaking down of accommodation/convergence, interacting variables around the monitor, and ortho-stereoscopic states.

There are many different means of producing time-shared multiplex images for electronic stereo displays. Further, the formatting of an image is separated from the selection technique, or the way to supply each eye with its necessary S image (and to erase unnecessary images). Formatting and image selection must be simultaneous, and by design of such system, the selection technique decides the format.
Stereoscopic image capturing equipment With stereoscopic image capturing equipment means equipment, which capture a three-dimensional image of reality.
First, there can be two cameras with parallel lenses. The object is captured simultaneously with the two lenses and thereby creates two images with different perspectives captured at the same moment.
Second, there can be cameras with one lens, which moves mechanically in telescopic, vertical, horizontal or rotational direction. Here is also two images captured from the periphery of the movement. The images are captured with a specified time difference and are therefore never in real time. This has only importance if the object and/or the lens are moving, and must therefore be corrected for.
The patent concerns format and capture equipment which function together with visual presentation equipment and uses time multiplex methods. Such visual presentation equipment can be active by using liquid crystal (LC) shutter, or it can be passively polarising.
The invention is defined as a way to use equipment for 3D
seeing (three dimensions) to capture objects close to or in real time.
More specifically the invention can be defined as a way to use such equipment to capture objects better and create a depth-sense to see the relation of the object to its environment.

- 4 - PCT/SE00/00282 THE PRESENT STATE OF THE ART
The camera model The geometry of the basic algorithm to produce computer generated electro-stereoscopic distortion-free images is illustrated in Figure 1, with left and right camera positioned in the data area. Their inter-axial separation is given by t~. The axes of the camera lenses are parallel in the z-direction. The distance between the cameras to the object is do.
Imagine that the cameras are two still images or video cameras whose lens axes are parallel, mounted on a planar bed. t~ is the inter-axial separation or distance between the cameras, or more exactly the centre of the lenses. The cameras are mounted such that the lens axes', the lines l~ through the centres of each lens with right angle to the image plane, all the time are parallel.
Both the cameras produce two different perspectives of the same object, because they are horizontally displaced by the distance t~ . Both the cameras use lenses with the same focal distance. Lenses with short focal distance produce a wide-angle image, and lenses with long focal distance produce a narrow-angle image. For example wide-angle lenses for 35-millimeter photography are usually under 50 millimetres, and long focal distance or lenses of tele 2~ optics are usually over 50 millimetres.
Let us assume two video cameras with lenses directed straight ahead. Each camera captures a perspective of an object. The change between the two video camera signals occurs with a certain rate.2 If we look at the images on a TV-monitor, two non-identical images can be seen, because the cameras are t~ apart from each other. Because the cameras are directed straight ahead with parallel lens axes, they will get different parts of the object. Let us displace these two images horizontally in relation to each other, such that part of the images fuse. Whenever the image is fused the parallax is zero, and that part of the Z Lipton, Lenny, and Meyer, Lhary. A time-multiplexed two-times vertical frequency stereoscopic video system. International Symposiasm Digest, SID 1984, VoI.XV.

- 5 - PCT/SE00/00282 object (or the scene) is in the monitor plane. It has been called 'convergence', but the term is easily exchanged with the human physiology term 'eye rotation' which is necessary for fusion. Therefore we use the term HIT (horizontal. image translation), which is used to get ZPS (zero parallax setting).
If the image is horizontally displaced such that any part of the image lies perfectly on top of the other, that part of the image has ZPS. The object doesn't exist only on x-and y-axes, but also on the .--axis. Because the object is three-dimensional, we can reach ZPS for only one point (or set of points oriented in one plane) of the object. If ZPS
is used for the mid part of the object, the parts behind the object have ZPS positive parallax, and the parts in front have ZPS have negative parallax.
The algorithm for parallel lens axes for stereoscopic computer generated images uses two camera perspectives, with parallel lens axes with a distance t~ between them.
Both perspectives and the camera lenses, have the same angle. The degree of horizontal translation (HIT) of the images is more a question about taste. Previously, the advantages of creating images with low parallax were put forward. One has to consider the usage of ZPS to produce the best compromise of parallax for the whole image.
2~ This approach doesn't mean any rotation and results in a geometric distortion producing vertical parallax, but the two necessary perspectives for a stereoscopic image have been achieved.
If HIT is used as described, with other conditions (lens angle, distance to the object, t~) constant, there won't be any change of the depth contents. It will take some time for the eyes to reconverge for the different values of parallax, but the total depth contents of the image will be the same. After adjustment of the eyes, the image will be as deep as before. One can produce, by HIT, images shown completely in front of or behind the monitor. As previously has been discussed, is it best to position ZPS at or near the centre of the object to reduce separation of the accommodation and the convergence, and to reduce artefacts.

- 6 - PCT/SE00/00282 Parallax It has been suggested to use small t~ values and a big angle, at least 40 degrees horizontally. The equation for the degree of depth, y Pn: - Mfctc do _d~
,which is used by stereographers, describes how the maximal value for parallax (Pm ) changes when we change the camera set-up. Imagine usage of HIT, the parallax value for an object on distance do will be zero. Then, one says that ZPS
is reached on the distance d°. An object on a maximal distance dm now has a parallax value of Pm.
The aim is to produce the strongest stereoscopic effect without exceeding a maximum of 1.5°parallax. The form of the equation is helpful at understanding of this. The value of the magnification (M) changes the value of Pm. For example: An image on a big screen has more parallax than the image on a smaller screen. A 24-inch monitor has twice as much parallax as a 12-inch monitor, with other parameters unchanged. To reduce t~ will reduce the value of monitor parallax. To reduce the focal distance of the lens f~ (use a wide-angle lens) also reduces Pm.
The most important factor controlling the stereoscopic effect is the distance t~ between the two camera lenses.
The bigger t~ the bigger parallax values and bigger stereoscopic depth-sense. The contrary is also true. If we look at the obj ect very close to the camera, for example a coin, insects or small objects, t~ becomes small and still produce a strong stereoscopic effect. On the other hand, if we look at distant hills maybe t~ must be hundreds of metres to produce any kind of stereoscopic effect. To change t~ is a way of controlling the depth of a stereoscopic image.
Small values on t~ can be reached by using wide-angle optics (low f~ ). Considering the perspective, we can see that the relative position of the object's surrounding

- 7 - PCT/SE00/00282 parts and the distant part of the object, for wide-angle, will be exaggerated. This exaggeration of the perspective decides the degree of the stereoscopic depth effect. With usage of wide-angle optics, t~ can be reduced.' The approach with parallel lens axes is particular important with usage of wide-angle for close objects. If rotation is used in this case, the geometric distortion will exaggerate the generation of falsified vertical parallax.
The stereoscopic effect's strength is controlled by the inter-axial distance t~, which in turn controls parallax values for the object. G~lhenever the inter-axial values are established, HIT will be used to control the ZPS of the images, and will often be received by software, through 1~ moving of two parts of the buffer relative to each other.
Horizontal movement of left and right images in the upper or lower part of the buffer produces HIT, which is used to control ZPS. For an image with an object, the mid part of the object is a good position for ZPS. If this recommendation is followed the user will get difficulties because of broken down because of accommodation, convergence and concomitant 'cross talk'. It is important to implement ZPS conditions of the image processing software as default values. In addition, when the size of the image is changed it is wished to keep ZPS, and this can be implemented in the software. In the camera model for parallel lens axes, when t~ changes, ZPS will also change;
and therefore, it is wished that the software implements a constant ZPS even when t~ varies. It gives the user a better quality and it is easier to look at the stereoscopic Image.
There are two important requirements to get stereoscopic three-dimensional images on a monitor:
1. The software must perform a perspective with a given offset-for each eye, thereby simulating what each eye would see if it had been in a three-dimensional world upon which the software rendering is based, and 3 MacAdam, D.L. Stereoscopic perceptions of size, shape, distance and direction.
SMPTEJoz~rnal, 1954, 62:271-93.

WO 01/60076 - $ - PCT/SE00/00282 2. The rendering of left and right eye must be formatted for the computer monitor in a way, which allows the stereoscopic hardware to separate information for the left and right eye.
Parallel Lens Axes Calculation of a stereoscopic image includes two monocular perspectives from two different positions. In the old days, computer generated stereoscopic pairs were created through rotation of an object a few degrees, but it is not recommended (see Figure 2). The use of rotation to generate stereoscopic pairs results in vertical non-adjustment of corresponding left and right image points. Such a vertical non-adjustment causes the eye muscles to work in an extraordinary way, which most persons experience as uncomfortable. If the eyes try to fuse vertical parallax without any depth information, the result can be painful.
The rectangle ABCD, on the left side has the vertical axis marked with a dotted line. Rotated around the axis as in the figure, the corner points A', B', C', and D are higher or lower than corresponding points A, B, C, and D. Despite facts, the example contains a simple figure, which is typical for what is happening when rotation is used to produce stereoscopic pairs. Instead it is recommended to produce two perspective with a horizontal translation along the x-axis, with a horizontal displacement for the resulting images (HIT 'horizontal image translation') to establish ZPS ('zero parallax setting').
The invention The invention is based on mechanical displacement of the lens in telescopic, horizontal, vertical or rotational direction. The displacement is adjustable to the wished distance between the two parallel images of the captured 3~ object, two different perspective of the same object not in full real time. The image rate is dependent on, among others, the shutter speed of the camera.

The advantage is an image capturing equipment with a much smaller size (diameter) than two parallel cameras, and is an important factor where the size of the capture equipment is decisive.
It is referred to the following patent:
US4956705 A A set-up to electronically capture images of an object to be used for three-dimensional representation of the object.
It is referred to the following publications:
Wheatstone, Charles. On some remarkable, and hitherto unobserved, phenomena of binocular vision (Part the first).
Philosophical Transactions of the Royal Society of London, 371-94, 1838;
Lipton, Lenny, and Meyer, Lhary. A time-multiplexed two-times vertical frequency stereoscopic video system.
International Symposium Digest, Vol.XV, SID 1984;
MacAdam, D.L. Stereoscopic perceptions of size, shape, distance and direction. SMPTE Journal, 62:271-93, 1954.
DESCRIPTION OF THE INVENTION
The purpose of the invention is to solve such problems, which have been described above, involving bad overview and spatial orientation of the captured object in relation to its surrounding, by adding the third dimension i.e. 3D.
The invention The invention consists of a tube, called the optics, with an optical and/or a fibre optic system transmitting reflected light from the object. Through fibre optics, light is transmitted from an external light source through the tube. In the end of the tube is a lens in a sledge or a cradle. On both sides of the lens' cradle there is a cushion filled with gas/liquid. This cushion can be filled and emptied alternately with micrometer precision, where the lens is displaced when the cradle is displaced sideways. The gas/liquid is regulated by an external pump connected to a controller card of a computer.
In the other end of the tube is a so called 'beam splitter' to split the incoming light between different cameras considered for different tasks. A 'beam splitter' physically means that part of the light will be lost when it is split between two external units/cameras.
A camera, for example a CCD-camera, is screwed on tight at the end, and is concerned to capture the incoming light in a Bitter. Thereby all reflected light points from the object get a dot. Every dot has a value registered in an image. The shutter of the camera decides immediately how long the time of exposure is needed. It is wished that the choice of camera, results in as short time of exposure as possible, because blur contributes strongly to lose the 3D-information of the image. Additionally, it is optimal that a camera takes at least 30 images/second, i.e. twice as many for 3D-images.
Also an IR-camera can be used at the same time. The purpose of the IR-camera is to detect heat changes and gradients in and around the object, and it can for example be used to measure circulation, separate between vital and dead tissue and see tissue reactions aso.
Peripheral equipment The computer processes later the two images by using stereographic mathematical algorithms. The images are delivered to the computer from a controller card as a video signal. The video signal from two images with different perspective of the same object are processed and later fused into one image, then presented in the display for the observer.
The computer controls the pump with a controller card, after the interpreted information has been received, which can be extracted after the image processing. The accuracy is on micrometer level.
Two controller cards: one for the video signal, which is a normal video card with high resolution; and one for the control of the displacement of the lens, after filling and emptying of the side-cushions.
Head mounted displays (HNm) are used to visualise the 3D-surrounding to the observer. For example surgeons can use Hl~s to get a spatial perception of a body cavity or hollow viscus. A HNB7 is connected to the computer, which transmits a video signal of one/two perspective of the object. The image shifts between the eyes take place after the computer has commanded it.
3D-glasses are cheaper alternatives with considerably less precision and resolution, but serves it purpose to visualise the captured object and its surrounding in 3D.
Different variants with polarising and LCD-technique exist.
3D-glasses are connected to the computer, which transmits a video signal of one/two perspective (-s) of the object.
Images shifts between the eyes take place after the computer has commanded it.
Instead of two cameras, i.e. double systems, use one lens system is an advantage, 1. when the lens can be displaced a certain distance in telescopic, vertical, horizontal or circular direction, 2. because the size or the diameter of the instrument become smaller, 3. because the equipment can be used in narrow localities, and where other factors of size determine about the usage of the instrument, 4. where, the invention is of specific value to applications when the size (diameter) of the capture equipment is decisive upon if the production of three-dimensional images will take place.
Example of products with such requirement are medical rigid and flexible endoscopes, for example arthroscopes, laparoscopes, cystoscopes, bronchoscopes aso.

Other examples are fibre (opto)scopes, rigid as well as flexible, and other camera applications where the size of the instrument is the decisive factor for the usage of the instrument.
FIGURE TEXT
One presently suggestive variant of the invention will be described below with reference to drawings included where Figure 1: Distortion free images. The camera axes (the axes in the centre of the perspective) must be parallel.
Figure 2: Rotation produces distortion.
Figure 3: The test bench (from above). To the left is the endoscope mounted in a test bench, and to the right is a concomitant controller unit with belonging controller card.
The figure shows a camera controller unit (7), a light source (8), a linear step activator with screw (9), a 9-way D-type for the Stepper motor controller unit (10), a programmable motor controller unit with a simple axis and a chip on the card (11), a back plate (12), to the computer ( 13 ) , a PSU ( 14 ) , a KM6 II 3U rack in a rigid box ( 15 ) , a linear sledge with the endpoint connected (16), an endoscope (17), a test plate (18) and a camera (19).
Figure 4: Test bench (from side). To the left is the endoscope mounted on a test plate, and to the right is concomitant front panel of the controller unit. The image shows power on button (20), fuse button (21), on/off button (22) and the rack(23);
Figure 5: Test bench (close-up).
Figure 6: Endoscope (from the side of the endoscope and frontal of the edge of the optics).
Figure 7:-The lens of the optics (frontal). The image shows the so called side cushions (24).
Figure 8: Displacement of the lens of the optics (frontal).
Figure 9: The lens (details).

Figure 10: Linear step activator (details).
DETAILED MANUFACTURING DESCRIPTION
Below is a couple of examples of suitable displacement for the invention.
Example Figure 1 shows a horizontal or vertical displacement of the lens. Horizontal displacement is preferred, when vertical parallax is created at vertical displacement. Horizontal displacement gives a less loss of the of clearness the periphery, but only a small side way displacement of the lens must take place. In every side way an image is captured by the CCD-camera, which later are fused into one 3D-iamge. The figure shows the outer fitting of the endoscope (1), the inner fitting (2), cradle/sledge (4), lens (5), lens holder (6) and displacement distance (FA) .
Example Figure 2 shows circulatory displacement of the lens. The lens rotates around its own axis. Rotational movement can create problems with the accuracy of the end points. In two decided positions are images captured, which later are fused into one 3D-image.
Example Figure 3 shows telescopic displacement of the lens.
The distance the lens is displaced decides what ordinate light reflected by the captured object will achieve. The closer the object the lens lies, the more peripheral the 2~ dot, and thereby increased number details, i.e. the farther the lens is from the object the more central is the dot, and thereby decreased number of details. In the inner and outer position is one image captured, which later are fused to a 3D-image. The figure shows the protection glass (3).
JO
In connection with the following figures some examples are shown of usage of the invention in different contexts.
In surgery are all operations associated with more or less tissue damage. It is considered important to try to 35 minimise the tissue damage, which could occur and occurs during surgery. The tissue damage occurring during surgery delays wound healing, rehabilitation and full recovery to normal function.
Surgery can be carried out either open, closed,. or a combination of the both techniques so called-micro-invasive surgery.
Open surgery is an operation where the surgeon releases anatomical structures and organs to get to the organ (-s) which the operation is focused on. The tissue damage is more when the release is more.
Closed surgery is an operation where the surgeon manipulates the organ without using the scalpel on the patient, i.e. no external wound is brought about. Example on closed surgery is fixation of fractures, where the fixature itself constitutes a wound, and closed correction of fractures, where inner wounds might occur. The method exposes the body for essentially less tissue damage in comparison with open surgery.
The combination of open and closed could be characterised as micro-invasive or minimal invasive surgery. A small incision, usually 1-2 cm, is made for an optical instrument, which later is conveyed into the body. One strives to create a hollow viscus if doesn't already naturally exist like a joint. The hollow viscus is created by pumping a fluid or air/gas mixture into it.
The invention considers among others a way to use an endoscopic instrument for 3D-seeing (three-dimensional) in the body at the procedure:
More specifically the invention can be defined as a way to use such equipment to visualise the field of the operation to create a better spatial perception.
In Figure- 3 is shown an example of one of the test benches we have been using to test the invention. The purpose was to get the least necessary side displacement and to compare different side displacements and their 3D-effects. The test results have then been used to build the prototype of the invention. The test bench consists of a CCD-camera connected to the invention and an object possible to displace sideways. A computer co-ordinates the capturing and displacement of the object.
The camera has a connection to the computer, which receives the incoming video signal. The camera is controlled by a controller card of the computer. Parameters such as exposure time, frames per second and light sensitivity can be controlled.
The invention consists of a stiff optics (described later) connected to the camera. It is anchored in the bottom plate.
The object has been positioned in a linear sledge with right angle to the lens. The sledge can be pulled and pushed back and forth, such that the object is displaced in horizontal direction with micrometer precision. The sledge is pulled by a linear step activator, and it is in turn controlled by a controller card of the computer.
Figure 4 shows the test bench from the side. The controller box' s front side has two buttons, one power on/off and one to fuse captured images. Additionally there is a power lamp.
Figure 5 shows the test bench in more detail.
Figure 6 shows the invention together with a connected CCD
camera. The end of the optics illustrates that the optics consists of a tube with different insulating envelopes.
Figure 7Fe1! Hittar ante ref erenskalla. shows an image in detail how the lens of the invention lies between two horizontal cushions. These cushions can be filled with air/gas or any fluid. By filling one and emptying one cushion, is a side displacement of the lens in horizontal direction attained. Additionally, other vibration mechanisms can be used to achieve side displacement, for example oscillating fibres. The distance the lens shall be displaced is decided by a number of parameters and is controlled by the connected computer.
Figure 8 shows only the lens displacement in detail.

Figure 9 shows the optics, which consists of an outer envelope protecting from thrusts and bending. Inside there is an isolating layer for the optical components. The lens system rests in a sledge supported on both sides horizontally by a system of cushions. In front of the lens there is a cover glass to protect from external damage of the lens. Along the sides, parallel with the tube, lies the mechanism for side displacement as an open channel, because the pump itself must lie outside the optics.
Additionally, one can see frontal figures of the lens and its side movement/displacement.
Figure 10 shows the linear step activator in all projections.
The invention is not limited to the above given designs l~ described above which can be varied within the framework of the following patent description. Thus, the mechanical displacement can be in both or several of the following directions; telescopic, horizontal, vertical and rotational. The invention is neither limited to medical applications, endoscopy, but can also be used for other kinds of three-dimensional captures, especially where the size of the instrument is a limiting factor.

Claims (9)

~17~

l.Construction of a three-dimensional _image capture equipment with the purpose of producing or generating 3D-images, still images or video images, as close to real time as possible, which is character-ised by 1.1. an optical tube, called the optics with independent length, flexible or stiff, or independent form or combination of these, and 1.2. having a lens which can be covered by a glass, and 1.3. the location of this lens in a cradle/sledge enveloped by one, two or more system(-s) of cushions in independent directions, and 1.4. displacement of the cradle/sledge and thereby the lens in telescopic, horizontal, vertical and/or rotational direction, and

1.5. image captured, through its side way displacement, different projections of the same object and its surroundings.

2.Construction according to patent claim 1 characterised by a system of cushions which can alternately be filled and emptied with gas/liquid/any other substance.

3.Construction according to patent claim 1 characterised by of a so called cradle/sledge displaced in a specified direction,

4.Construction according to patent claim 1 characterised by a system of cushions displacing the lens an exact distance from the original position.

5.Method according to patent claim 1 characterised by a camera for example a CCD-camera, connected to the optics capturing images of the reflected light from the object in the position where the lens is located at different occasions.

6.Optics displaced as patent claim. 4 characterised by capturing two images with different perspectives with a connected camera.

7.Construction according to patent claim 6 characterised by the equipment capturing the object and its surroundings and is used as image capture equipment where a 3D image is generated.

8.Construction according to patent claim 5 characterised by usage of the image capture equipment with benefit in applications where visualisation is dependent on the size of the equipment.

9.Construction according to patent claim 6 characterised by the computer processing the two captured images and fused into one 3D image, which is presented for the observer with a head mounted display or any other visualisation equipment.