EP1275258A1 - Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles - Google Patents

Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles

Info

Publication number
EP1275258A1
EP1275258A1 EP00911528A EP00911528A EP1275258A1 EP 1275258 A1 EP1275258 A1 EP 1275258A1 EP 00911528 A EP00911528 A EP 00911528A EP 00911528 A EP00911528 A EP 00911528A EP 1275258 A1 EP1275258 A1 EP 1275258A1
Authority
EP
European Patent Office
Prior art keywords
image
images
lens
camera
equipment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP00911528A
Other languages
German (de)
English (en)
Inventor
Kent Olsson
Torsten Wredmark
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Forte Visio Medica AB
Original Assignee
Forte Visio Medica AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Forte Visio Medica AB filed Critical Forte Visio Medica AB
Priority claimed from PCT/SE2000/000282 external-priority patent/WO2001060076A1/fr
Publication of EP1275258A1 publication Critical patent/EP1275258A1/fr
Withdrawn legal-status Critical Current

Links

Definitions

  • the present invention refers to composition, utilisation and function of an equipment for capturing of three- dimensional images for medical purposes, for example for usage in micro-invasive surgery, for capturing of three- dimensional images of objects.
  • a stereoscopic image presents to the observer's left and right eyes an image with pixels from different perspectives, exactly as the observer sees the visual reality.
  • __ These two images, with slightly different perspectives, are synthesised by the cortex of the brain
  • the reality can also be captured by a camera. Irrespective of the image is calculated or captured, the generation or copying of images from two perspectives, one for the left and one for the right eye, must take place. If a perspective can be produced, it is conceptually possible to create another perspective.
  • the monocular, or extra-stereoscopic, depth-variables are the conditions for the perception of depth on visual displays. They are as important as stereopsis to create images, which are percepted as three-dimensional. These variables include light and shadow, relative size, interposition, text gradient, spatial perspective, motion parallax, and the most important is the perspective. Images rich on monocular depth-variables will be even easier to visualise when binocular ⁇ tereoscopy is added.
  • a stereoscopic display is an optical system, where the human brain is the end-unit. It functions by presenting left and right images of reality to the brain.
  • stereopsis adds information in a format, which is both sensoric, comfortable and usef l.
  • a stereoscopic image which has a one-to-one conformity, ⁇ somorphic, with reality, can be uncomfortable to look at, and be of debatable value for the scientist, engineer, technician, doctor and the artist.
  • a stereoscopic image can diverge from being isomorphic with reality, psycho-physics, the psychology of depth-perception, and geometric or optical changes. They can have the following states: breaking down of accommodation/convergence, interacting variables around the monitor, and ortho-stereoscopic states .
  • the formatting of an image is separated from the selection technique, or the way to supply each eye with its necessary image (and to erase unnecessary images) . Formatting and image selection must be simultaneous, and by design of such system, the selection technique decides the format.
  • stereoscopic image capturing equipment means equipment, which capture a three-dimensional image of reality.
  • the object is captured simultaneously with the two lenses and thereby creates two images with different perspectives captured at the same moment.
  • the patent concerns format and capture equipment which function together with visual presentation equipment and uses time multiplex methods.
  • visual presentation equipment can be active by using liquid crystal (LC) shutter, or it can be passively polarising.
  • LC liquid crystal
  • the invention is defined as a way to use equipment for 3D- seeing (three dimensions) to capture objects close to or in real time .
  • the invention can be defined as a way to use such equipment to capture objects better and create a depth-sense to see the relation of the object to its environment .
  • FIG. 1 The geometry of the basic algorithm to produce computer generated electro-stereoscopic distortion-free images is illustrated in Figure 1, with left and right camera positioned in the data area. Their inter-axial separation is given by t c .
  • the axes of the camera lenses are parallel in the z-direction.
  • the distance between the cameras to the object is d 0 .
  • the cameras are two still images or video cameras whose lens axes are parallel, mounted on a planar bed.
  • t c is the inter-axial separation or distance between the cameras, or more exactly the centre of the lenses.
  • the cameras are mounted such that the lens axes', the lines through the centres of each lens with right angle to the image plane, all the time are parallel.
  • Both the cameras produce two different perspectives of the same object, because they are horizontally displaced by the distance t c . Both the cameras use lenses with the same focal distance. Lenses with short focal distance produce a wide-angle image, and lenses with long focal distance produce a narrow-angle image. For example wide-angle lenses for 35-millimeter photography are usually under 50 millimetres, and long focal distance or lenses of tele- optics are usually over 50 millimetres.
  • the image is horizontally displaced such that any part of the image lies perfectly on top of the other, that part of the image has ZPS.
  • the object doesn't exist only on x- and -axes, but also on the z-axis. Because the object is three-dimensional, we can reach ZPS for only one point (or set of points oriented in one plane) of the object. If ZPS is used for the mid part of the object, the parts behind the object have ZPS positive parallax, and the parts in front have ZPS have negative parallax.
  • the algorithm for parallel lens axes for stereoscopic computer generated images uses two camera perspectives, with parallel lens axes with a distance t c between them. Both perspectives and the camera lenses, have the same angle.
  • the degree of horizontal translation (HIT) of the images is more a question about taste.
  • HIT horizontal translation
  • the aim is to produce the strongest stereoscopic effect without exceeding a maximum of 1.5° parallax.
  • the form of the equation is helpful at understanding of this.
  • the value of the magnification (M) changes the value of P m .
  • An image on a big screen has more parallax than the image on a smaller screen.
  • a 24-inch monitor has twice as much parallax as a 12-inch monitor, with other parameters unchanged.
  • To reduce t c will reduce the value of monitor parallax.
  • To reduce the focal distance of the lens f c (use a wide-angle lens) also reduces P m .
  • t c The most important factor controlling the stereoscopic effect is the distance t c between the two camera lenses.
  • the bigger t c the bigger parallax values and bigger stereoscopic depth-sense.
  • t c becomes small and still produce a strong stereoscopic effect.
  • t c must be hundreds of metres to produce any kind of stereoscopic effect.
  • To change ⁇ is a way of controlling the depth of a stereoscopic image.
  • HIT will be used to control the ZPS of the images, and will often be received by software, through moving of two parts of the buffer relative to each other. Horizontal movement of left and right images in the upper or lower part of the buffer produces HIT, which is used to control ZPS.
  • HIT is used to control ZPS.
  • the mid part of the object is a good position for ZPS. If this recommendation is followed the user will get difficulties because of broken down because of accommodation, convergence and concomitant 'cross talk'. It is important to implement ZPS conditions of the image processing software as default values.
  • the size of the image is changed it is wished to keep ZPS, and this can be implemented in the software.
  • the software must perform a perspective with a given offset - for each eye, thereby simulating what each eye would see if it had been in a three-dimensional world upon which the software rendering is based, and
  • Calculation of a stereoscopic image includes two monocular perspectives from two different positions.
  • computer generated stereoscopic pairs were created through rotation of an object a few degrees, but it is not recommended (see Figure 2) .
  • the use of rotation to generate stereoscopic pairs results in vertical non-adjustment of corresponding left and right image points .
  • Such a vertical non-adjustment causes the eye muscles to work in an extraordinary way, which most persons experience as uncomfortable. If the eyes try to fuse vertical parallax without any depth information, the result can be painful.
  • the rectangle ABCD on the left side has the vertical axis marked with a dotted line.
  • the corner points A ', B', C, and D ' are higher or lower than corresponding points A , B, C, and D.
  • the example contains a simple figure, which is typical for what is happening when rotation is used to produce stereoscopic pairs. Instead it is recommended to produce two perspective with a horizontal translation along the x-axis, with a horizontal displacement for the resulting images (HIT 'horizontal image translation') to establish ZPS ('zero parallax setting').
  • the invention is based on mechanical displacement of the lens in telescopic, horizontal, vertical or rotational direction.
  • the displacement is adjustable to the wished distance between the two parallel images of the captured object, two different perspective of the same object not in full real time.
  • the image rate is dependent on, among others, the shutter speed of the camera.
  • the advantage is an image capturing equipment with a much smaller size (diameter) than two parallel cameras, and is an important factor where the size of the capture equipment is decisive.
  • the purpose of the invention is to solve such problems, which have been described above, involving bad overview and spatial orientation of the captured object in relation to its surrounding, by adding the third dimension i.e. 3D.
  • the invention is to solve such problems, which have been described above, involving bad overview and spatial orientation of the captured object in relation to its surrounding, by adding the third dimension i.e. 3D.
  • the invention consists of a tube, called the optics, with an optical and/or a fibre optic system transmitting reflected light from the object.
  • a tube called the optics
  • the invention consists of a tube, called the optics, with an optical and/or a fibre optic system transmitting reflected light from the object.
  • a fibre optic system transmitting reflected light from the object.
  • light is transmitted from an external light source through the tube.
  • On both sides of the lens' cradle there is a cushion filled with gas/liquid. This cushion can be filled and emptied alternately with micrometer precision, where the lens is displaced when the cradle is displaced sideways.
  • the gas/liquid is regulated by an external pump connected to a controller card of a computer.
  • a so called 'beam spli tter ' to split the incoming light between different cameras considered for different tasks.
  • a 'beam splitter' physically means that part of the light will be lost when it is split between two external units/cameras .
  • a camera for example a CCD-camera, is screwed on tight at the end, and is concerned to capture the incoming light in a gitter. Thereby all reflected light points from the object get a dot. Every dot has a value registered in an image.
  • the shutter of the camera decides immediately how long the time of exposure is needed. It is wished that the choice of camera, results in as short time of exposure as possible, because blur contributes strongly to lose the 3D- information of the image. Additionally, it is optimal that a camera takes at least 30 images/second, i.e. twice as many for 3D-images.
  • an IR-camera can be used at the same time.
  • the purpose of the IR-camera is to detect heat changes and gradients in and around the object, and it can for example be used to measure circulation, separate between vital and dead tissue and see tissue reactions aso .
  • the computer processes later the two images by using stereographic mathematical algorithms.
  • the images are delivered to the computer from a controller card as a video signal .
  • the video signal from two images with different perspective of the same object are processed and later fused into one image, then presented in the display for the observer.
  • the computer controls the pump with a controller card, after the ⁇ interpreted information has been received, which can be extracted after the image processing.
  • the accuracy is on micrometer level.
  • Two controller cards one for the video signal, which is a normal video card with high resolution; and one for the control of the displacement of the lens, after filling and emptying of the side-cushions.
  • Head mounted displays are used to visualise the 3D- surrounding to the observer. For example surgeons can use HMDs to get a spatial perception of a body cavity or hollow viscus.
  • a HMD is connected to the computer, which transmits a video signal of one/two perspective of the object. The image shifts between the eyes take place after the computer has commanded it.
  • 3D-glasses are cheaper alternatives with considerably less precision and resolution, but serves it purpose to visualise the captured object and its surrounding in 3D. Different variants with polarising and LCD-technique exist.
  • 3D-glasses are connected to the computer, which transmits a video signal of one/two perspective (-s) of the object. Images shifts between the eyes take place after the computer has commanded it.
  • the invention is of specific value to applications when the size (diameter) of the capture equipment is decisive upon if the production of three- dimensional images will take place.
  • Example of products with such requirement are medical rigid and flexible endoscopes, for example arthroscopes, laparoscopes, cystoscopes, bronchoscopes aso.
  • Other examples are fibre (opto) scopes, rigid as well as flexible, and other camera applications where the size of the instrument is the decisive factor for the usage of the instrument .
  • Figure 1 Distortion free images.
  • the camera axes (the axes in the centre of the perspective) must be parallel.
  • Figure 2 Rotation produces distortion.
  • FIG. 3 The test bench (from above) . To the left is the endoscope mounted in a test bench, and to the right is a concomitant controller unit with belonging controller card. The figure shows a camera controller unit (7) , a light source (8), a linear step activator with screw (9), a 9-way
  • Stepper motor controller unit (10) a programmable motor controller unit with a simple axis and a chip on the card (11), a back plate (12), to the computer
  • Figure 4 Test bench (from side) . To the left is the endoscope mounted on a test plate, and to the right is concomitant front panel of the controller unit. The image shows power on button (20), fuse button (21), on/off button (22) and the rack(23);
  • Figure 5 Test bench (close-up).
  • Figure 6 Endoscope (from the side of the endoscope and frontal of the edge of the optics) .
  • Figure 7 The lens of the optics (frontal) .
  • the image shows the so called side cushions (24) .
  • Figure 8 Displacement of the lens of the optics (frontal).
  • Figure 9 The lens (details).
  • Figure 10 Linear step activator (details).
  • Example Figure 1 shows a horizontal or vertical displacement of the lens. Horizontal displacement is preferred, when vertical parallax is created at vertical displacement. Horizontal displacement gives a less loss of the of clearness the periphery, but only a small side way displacement of the lens must take place. In every side way an image is captured by the CCD-camera, which later are fused into one 3D-iamge.
  • the figure shows the outer fitting of the endoscope (1), the inner fitting (2), cradle/sledge (4), lens (5), lens holder (6) and displacement distance (FA) .
  • Example Figure 2 shows circulatory displacement of the lens.
  • the lens rotates around its own axis. Rotational movement can create problems with the accuracy of the end points. In two decided positions are images captured, which later are fused into one 3D-image.
  • Example Figure 3 shows telescopic displacement of the lens.
  • the distance the lens is displaced decides what ordinate light reflected by the captured object will achieve.
  • In the inner and outer position is one image captured, which later are fused to a 3D-image.
  • the figure shows the protection glass (3).
  • tissue damage In surgery are all operations associated with more or less tissue damage. It is considered important to try to minimise the tissue damage, which could occur and occurs during surgery. The tissue damage occurring during surgery delays wound healing, rehabilitation and full recovery to normal function.
  • Surgery can be carried out either open, closed, or a combination of the both techniques so called " micro-invasive surgery .
  • Open surgery is an operation where the surgeon releases anatomical structures and organs to get to the organ (-s) which the operation is focused on. The tissue damage is more when the release is more.
  • Closed surgery is an operation where the surgeon manipulates the organ without using the scalpel on the patient, i.e. no external wound is brought about.
  • Example on closed surgery is fixation of fractures, where the fixature itself constitutes a wound, and closed correction of fractures, where inner wounds might occur. The method exposes the body for essentially less tissue damage in comparison with open surgery.
  • the combination of open and closed could be characterised as micro-invasive or minimal invasive surgery.
  • a small incision usually 1-2 cm, is made for an optical instrument, which later is conveyed into the body.
  • the hollow viscus is created by pumping a fluid or air/gas mixture into it.
  • the invention considers among others a way to use an endoscopic instrument for 3D-seeing (three-dimensional) in the body at the procedure .
  • the invention can be defined as a way to use such equipment to visualise the field of the operation to create a better spatial perception.
  • FIG. ⁇ 3 is shown an example of one of the test benches we have been using to test the invention. The purpose was to get the least necessary side displacement and to compare different side displacements and their 3D-effects. The test results have then been used to build the prototype of the invention.
  • the test bench consists of a CCD-camera connected to the invention and an object possible to displace sideways. A computer co-ordinates the capturing and displacement of the object.
  • the camera has a connection to the computer, which receives the incoming video signal.
  • the camera is controlled by a controller card of the computer. Parameters such as exposure time, frames per second and light sensitivity can be controlled.
  • the invention consists of a stiff optics (described later) connected to the camera. It is anchored in the bottom plate.
  • the object has been positioned in a linear sledge with right angle to the lens.
  • the sledge can be pulled and pushed back and forth, such that the object is displaced in horizontal direction with micrometer precision.
  • the sledge is pulled by a linear step activator, and it is in turn controlled by a controller card of the computer.
  • Figure 4 shows the test bench from the side.
  • the controller box's front side has two buttons, one power on/off and one to fuse captured images. Additionally there is a power lamp.
  • Figure 5 shows the test bench in more detail.
  • Figure 6 shows the invention together with a connected CCD- camera.
  • the end of the optics illustrates that the optics consists of a tube with different insulating envelopes.
  • FIG. 7 shows an image in detail how the lens of the invention lies between two horizontal cushions. These cushions can be filled with air/gas or any fluid. By filling one and emptying one cushion, is a side displacement of the lens in horizontal direction attained. Additionally, other vibration mechanisms can be used to achieve side displacement, for example oscillating fibres. The distance the lens shall be displaced is decided by a number of parameters and is controlled by the connected computer.
  • Figure 8 shows only the lens displacement in detail.
  • Figure 9 shows the optics, which consists of an outer envelope protecting from thrusts and bending. Inside there is an isolating layer for the optical components. The lens system rests in a sledge supported on both ⁇ sides horizontally by a system of cushions. In front of the lens there is a cover glass to protect from external damage of the lens. Along the sides, parallel with the tube, lies the mechanism for side displacement as an open channel, because the pump itself must lie outside the optics . Additionally, one can see frontal figures of the lens and its side movement/displacement .
  • Figure 10 shows the linear step activator in all projections .
  • the invention is not limited to the above given designs described above which can be varied within the framework of the following patent description.
  • the mechanical displacement can be in both or several of the following directions; telescopic, horizontal, vertical and rotational.
  • the invention is neither limited to medical applications, endoscopy, but can also be used for other kinds of three-dimensional captures, especially where the size of the instrument is a limiting factor.

Landscapes

  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Endoscopes (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)

Abstract

L'invention concerne la construction et l'utilisation d'un équipement de capture d'images tridimensionnelles. L'invention est fondée sur la capture de deux images présentant une perspective différente avec un déplacement préétabli, sur la base de plusieurs paramètres. La construction est fondée sur la lentille qui est placée dans un berceau/traîneau. Sur chaque côté du berceau/traîneau se trouve un système appelé système de coussins. Les systèmes de coussins peuvent influer sur le déplacement de la lentille selon l'état de remplissage ou de vidage du liquide, gaz ou autre substance. L'objet saisi est capturé par une caméra, par exemple une caméra CCD, qui transmet l'image à un ordinateur en tant que signal vidéo. L'ordinateur utilise ensuite un procédé de traitement d'image pour fusionner les deux images en une image tridimensionnelle. L'image tridimensionnelle peut être visualisée ultérieurement au moyen d'un casque HMD (visiocasque), sur un écran ou à travers des lunettes à vision X. Le procédé de l'invention peut être utilisé avantageusement dans des applications où la taille ou le diamètre de l'instrument est un élément décisif pour le résultat, comme, par exemple, dans le domaine médical, pour l'endoscopie, notamment l'arthroscopie, la laparoscopie, la cystoscopie. De plus, il est important pour le chirurgien d'avoir une perception spatiale du champ opératoire, ce qui lui permet d'apprécier plus facilement les distances, les dimensions, etc.
EP00911528A 2000-02-11 2000-02-11 Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles Withdrawn EP1275258A1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SE2000/000282 WO2001060076A1 (fr) 1998-11-04 2000-02-11 Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles

Publications (1)

Publication Number Publication Date
EP1275258A1 true EP1275258A1 (fr) 2003-01-15

Family

ID=20278272

Family Applications (1)

Application Number Title Priority Date Filing Date
EP00911528A Withdrawn EP1275258A1 (fr) 2000-02-11 2000-02-11 Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles

Country Status (5)

Country Link
EP (1) EP1275258A1 (fr)
JP (1) JP2003522341A (fr)
CN (1) CN1451243A (fr)
AU (1) AU2000233394A1 (fr)
CA (1) CA2399713A1 (fr)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005077253A1 (fr) * 2004-02-18 2005-08-25 Osaka University Endoscope
JP4810295B2 (ja) * 2006-05-02 2011-11-09 キヤノン株式会社 情報処理装置及びその制御方法、画像処理装置、プログラム、記憶媒体
JP5196840B2 (ja) * 2007-04-26 2013-05-15 キヤノン株式会社 情報処理装置および方法
KR20110018696A (ko) * 2009-08-18 2011-02-24 주식회사 이턴 입체 영상 처리 장치 및 방법
CN102058380B (zh) * 2011-01-31 2012-09-19 广州宝胆医疗器械科技有限公司 三维立体硬质电子喉镜系统
CN102058382B (zh) * 2011-01-31 2012-07-11 广州宝胆医疗器械科技有限公司 三维立体电子胃镜系统及其使用方法
CN102090878B (zh) * 2011-01-31 2012-11-21 广州宝胆医疗器械科技有限公司 三维立体硬质电子膀胱镜系统
CN102090879B (zh) * 2011-01-31 2012-08-15 广州宝胆医疗器械科技有限公司 三维立体硬质电子胆囊镜系统
CN103150757A (zh) * 2012-02-22 2013-06-12 林善红 数字城市3d被动立体展示系统及其制作方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO0160076A1 *

Also Published As

Publication number Publication date
AU2000233394A1 (en) 2001-08-20
JP2003522341A (ja) 2003-07-22
CA2399713A1 (fr) 2001-08-16
CN1451243A (zh) 2003-10-22

Similar Documents

Publication Publication Date Title
AU2018258089B2 (en) Stereoscopic visualization camera and platform
US7768702B2 (en) Medical stereo observation system
US6414708B1 (en) Video system for three dimensional imaging and photogrammetry
US9192286B2 (en) Stereoscopic visualization system
JP4537916B2 (ja) 医療用立体観察システム
JP2004309930A (ja) 立体観察システム
JP2017509925A (ja) ３ｄビデオ顕微鏡装置
CN215839550U (zh) 一种显示3d图像的手术显微镜系统
US20200030054A1 (en) Observation system for dental and medical treatment
JP7178385B2 (ja) 撮像システムおよび観察方法
WO2020220890A1 (fr) Dispositif d'affichage et robot chirurgical
US20140285632A1 (en) Imaging device and method
US20040070823A1 (en) Head-mount recording of three-dimensional stereo video images
JP2002223458A (ja) 立体映像作成装置
JP3464197B2 (ja) 立体視用表示装置及びこれを利用した内視鏡並びに顕微鏡
JP4383188B2 (ja) 立体観察システム
US20100259820A1 (en) Stereoscopic image display
CN115919239A (zh) 用于3d内窥镜成像系统的成像方法和3d内窥镜成像系统
EP1275258A1 (fr) Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles
WO2016194446A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations, programme, et système d'imagerie in-vivo
WO2001060076A1 (fr) Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles
KR101339667B1 (ko) 의료 수술용 현미경의 3차원 고화질 영상 인터페이스 시스템
JP2021191320A (ja) 手術支援システム、表示制御装置、および表示制御方法
JP4246510B2 (ja) 立体内視鏡システム
JPH10117362A (ja) 立体視映像表示装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20020911

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20050830