JP2005087468A - Image pickup device provided with distance image measurement function and endoscopic apparatus - Google Patents

Image pickup device provided with distance image measurement function and endoscopic apparatus Download PDF

Info

Publication number
JP2005087468A
JP2005087468A JP2003324579A JP2003324579A JP2005087468A JP 2005087468 A JP2005087468 A JP 2005087468A JP 2003324579 A JP2003324579 A JP 2003324579A JP 2003324579 A JP2003324579 A JP 2003324579A JP 2005087468 A JP2005087468 A JP 2005087468A
Authority
JP
Japan
Prior art keywords
distance
image
sensor
lens
imaging surface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2003324579A
Other languages
Japanese (ja)
Inventor
Shoji Kawahito
祥二 川人
Susumu Terakawa
進 寺川
Keiichi Abe
圭一 阿部
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to JP2003324579A priority Critical patent/JP2005087468A/en
Priority to PCT/JP2004/013699 priority patent/WO2005027739A1/en
Publication of JP2005087468A publication Critical patent/JP2005087468A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/2476—Non-optical details, e.g. housings, mountings, supports
    • G02B23/2484—Arrangements in relation to a camera or imaging device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00193—Optical arrangements adapted for stereoscopic vision
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1076—Measuring physical dimensions, e.g. size of the entire body or parts thereof for measuring dimensions inside body cavities, e.g. using catheters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1079—Measuring physical dimensions, e.g. size of the entire body or parts thereof using optical or photographic means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G02B7/30—Systems for automatic generation of focusing signals using parallactic triangle with a base line
    • G02B7/32—Systems for automatic generation of focusing signals using parallactic triangle with a base line using active means, e.g. light emitter

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Multimedia (AREA)
  • Dentistry (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Remote Sensing (AREA)
  • Endoscopes (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Automatic Focus Adjustment (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To display a scale for measuring the actual size of an object on an image since perspective is lost and the absolute size of the object is not recognized because focusing is made in a wide range in the case that the depth of field is large. <P>SOLUTION: The size of the object is measured by obtaining distance images to the individual parts of the object by using a range image sensor and overwriting isometric distribution contour within a plane where the object is projected on a two-dimensional image as a mesh-like scale from a geometrical relation by using the distance in the depth direction. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

本発明は、通常の明暗画像を得るイメージアレイと、対象物までの距離を計測する距離画像センサを一体化することで、取得した明暗画像に対象物の大きさを表すスケールを重ね合わせることで、対象物の大きさを表示する形状計測機能を有するイメージセンサとこれを用いた撮像装置に関するものである。   The present invention integrates an image array that obtains a normal light and dark image and a distance image sensor that measures the distance to the object, thereby superimposing a scale representing the size of the object on the obtained light and dark image. The present invention relates to an image sensor having a shape measurement function for displaying the size of an object, and an imaging apparatus using the image sensor.

内視鏡において、撮像側から光スポットの列を放射状に被写体に照射し、各スポットの位置情報から三角測量の原理を用いて、撮像レンズから対象物までの距離を算出する技術が、特許文献１に示されている。かかる従来技術においては、照射される各スポットの対象物の位置における間隔は、絶対的な大きさを表すものではないので、対象物の大きさを表すスケールとして用いることはできなかった。
特開平５−４１９０１号公報 In an endoscope, a technique for irradiating a subject with a row of light spots radially from the imaging side and calculating the distance from the imaging lens to the object using the principle of triangulation from the position information of each spot is disclosed in Patent Literature 1. In such a conventional technique, since the interval at the position of the object of each spot to be irradiated does not represent an absolute size, it cannot be used as a scale representing the size of the object.
JP-A-5-41901

撮像装置において、レンズのF値(焦点距離／レンズの口径)を大きくすることで、被写界深度が深くなり、被写体に対してピント調節を行うことなく撮像を行うことが可能である。この技術は内視鏡などのように機構上ピント調節を行うのが容易でない場合などに用いられる。しかしこれによって広範囲でピントが合うために遠近感が失われ、被写体の絶対的な大きさがわからなくなる。撮像対象物の絶対的な大きさを知りたいという要求は、例えば内視鏡により病巣の大きさ、その成長または縮小の度合いを知りたい場合など多くあると考えられる。本発明は被写体の実寸を計測できるスケールを画像上に表示することで対象物の絶対的な大きさを知り得るものである。   In the imaging apparatus, by increasing the F value (focal length / lens aperture) of the lens, the depth of field can be deepened, and imaging can be performed without adjusting the focus on the subject. This technique is used when it is not easy to perform focus adjustment mechanically, such as in an endoscope. However, since the focus is adjusted over a wide range, the sense of perspective is lost and the absolute size of the subject cannot be known. There may be many requests for knowing the absolute size of an imaging target, for example, when it is desired to know the size of a lesion and the degree of growth or reduction by an endoscope. In the present invention, the absolute size of an object can be known by displaying a scale capable of measuring the actual size of an object on an image.

本発明は、通常の明暗画像を得るイメージアレイと、対象物までの距離を計測する距離画像センサを一体化することで、取得した明暗画像に対象物の大きさを表すスケールを重ね合わせることで、対象物の大きさを表示する形状計測機能を有するものである。   The present invention integrates an image array that obtains a normal light and dark image and a distance image sensor that measures the distance to the object, thereby superimposing a scale representing the size of the object on the obtained light and dark image. And a shape measuring function for displaying the size of the object.

撮像対象物の絶対的な大きさを知りたいという要求は、例えば内視鏡により病巣の大きさ、その成長または縮小の度合いを知りたい場合など多くあり、本発明はこれらの要求を満たすことができる。   There are many requests for knowing the absolute size of an object to be imaged, for example, when it is desired to know the size of a lesion, the degree of growth or reduction by an endoscope, and the present invention can satisfy these requirements. it can.

本発明は、通常の明暗画像を得るイメージアレイと、対象物までの距離を計測する距離画像センサを一体化し、取得した明暗画像に対象物の大きさを表すスケールを重ね合わせることで、対象物の大きさを表示する形状計測機能を有する撮像装置に関するものである。これは、内視鏡などに有用である。レンズのＦ値(焦点距離／レンズの口径)を大きくすることで、被写界深度が深くなり、被写体に対してピント調節を行うことなく撮像を行うことが可能である。しかし、図１に示すように、これによって、被写体の絶対的な大きさがわからなくなる。レンズのしぼりを絞り、Ｆ値を大きくすると、３つの距離のいずれに対してもピントのあった画像が得られる。   The present invention integrates an image array that obtains a normal light and dark image and a distance image sensor that measures the distance to the object, and superimposes a scale that represents the size of the object on the obtained light and dark image. The present invention relates to an imaging apparatus having a shape measuring function for displaying the size of the image. This is useful for endoscopes and the like. By increasing the F value (focal length / lens aperture) of the lens, the depth of field becomes deep, and it is possible to take an image without adjusting the focus on the subject. However, as shown in FIG. 1, this makes it impossible to know the absolute size of the subject. When the aperture of the lens is reduced and the F value is increased, an image that is in focus at any of the three distances can be obtained.

そこで、明暗画像を取得するイメージセンサと距離画像を取得するイメージセンサを一体化するか、あるいは、明暗画像と距離画像がともに得られる距離イメージセンサを用いて、対象物の各部までの距離画像を得、その奥行き方向の距離を用いて幾何学的関係により、対象物を２次元画像に投影した面内の距離分布を網目状のスケールを重ねがきすることにより、対象物の大きさを測れるようにする。
距離を測るイメージセンサとして、本発明者が特願２００３−１３２９４５号で提案したCMOSイメージセンサが有用である。このCMOSセンサは光飛行時間法(Time of flight)による距離測定を行うものであり、対象物の明暗画像を距離情報と同時に得ることができる。 Therefore, by integrating the image sensor that acquires the light and dark image and the image sensor that acquires the distance image, or by using the distance image sensor that can obtain both the light and dark image and the distance image, the distance image to each part of the object is obtained. By using the distance in the depth direction, the size of the object can be measured by overlaying a mesh-like scale on the in-plane distance distribution obtained by projecting the object on a two-dimensional image. To.
A CMOS image sensor proposed by the present inventor in Japanese Patent Application No. 2003-132945 is useful as an image sensor for measuring the distance. This CMOS sensor measures distance by the time of flight method, and can obtain a bright and dark image of an object simultaneously with distance information.

奥行き距離から、面内距離を求める原理図を図２に示す。対象物のある点とレンズ間の距離Ｌ１が、レンズと撮像面までの距離ｄに比べて十分大きければ、明らかに次式がなりたつ。

Figure 2005087468
撮像面上のある点と原点の間の距離ｘ１が与えられ、対象物の対応する点までの距離Ｌ１を距離画像センサにより測定することで、対象物の面内の距離Ｘ１を求めることができる。 FIG. 2 shows a principle diagram for obtaining the in-plane distance from the depth distance. If the distance L1 between a certain point of the object and the lens is sufficiently larger than the distance d between the lens and the imaging surface, the following equation is clearly obtained.
Figure 2005087468
 A distance x1 between a certain point on the imaging surface and the origin is given, and the distance L1 to the corresponding point of the object is measured by the distance image sensor, whereby the in-plane distance X1 of the object can be obtained. .

このようにして求めた面内の等距離線を網目状に、明暗画像上に重ね書きする。その原理を説明する図を図３と図４に示す。
説明の便宜上、１次元の場合について説明するが、容易に２次元に拡張可能である。
図３に示すように、釣り鐘状の対象物を撮像しているとし、撮像面上に一定間隔で配置された画素と、レンズを結ぶ光線を対象物側に延長し、これが、対象物と交わる点を図３では、黒丸で表している。これより明らかなように、対象物までの距離が短い場合には、対象物は大きく写るため、１画素のピッチに対して、対象物と光線の交点のピッチは、距離が遠い場合に比べて狭くなる。そのため、距離画像センサの画素毎に、対象物までの距離を求めて、その線間の距離が一定になるように網目状の面内等距離分布を求めると、図３の釣り鐘状の対象物の場合には、図４のようになる。 The in-plane equidistant lines thus obtained are overwritten on the light and dark image in a mesh pattern. 3 and 4 are diagrams for explaining the principle.
For convenience of explanation, a one-dimensional case will be described, but it can be easily extended to two dimensions.
As shown in FIG. 3, it is assumed that a bell-shaped object is being imaged, and light beams connecting the pixels arranged at regular intervals on the imaging surface and the lens are extended to the object side, and this intersects the object. The dots are represented by black circles in FIG. As is clear from this, when the distance to the object is short, the object appears large, and therefore, the pitch of the intersection of the object and the light beam is larger than that when the distance is long with respect to the pitch of one pixel. Narrow. Therefore, when the distance to the object is obtained for each pixel of the distance image sensor, and the in-plane equidistant distribution is obtained so that the distance between the lines is constant, the bell-shaped object of FIG. In the case of FIG.

これを求める方法を説明する。図３に示すように、撮像面の画素からの光線と対象物との交点の位置に番号をつける。これをｉとすると、図３の例ではｉは０から１０までをとる。いま撮像面と平行な対象物の基準面を考えると、この基準面と光線との交点は、常に一定間隔になる。そのピッチをｐとする。
距離画像センサにより、各画素における面内の距離が式(１)により求められる。これは、図３では、光線と対象物との交点を、基準面に投影した点を求めていることに相当する。例えば、ｉ＝１の点では１ｐ、ｉ＝２の点では、２.２ｐである。従ってｉ＝２の画素に対する等距離線の位置としては、ｉ＝１側にすこし寄った点を取る必要があることが分かる。 A method for obtaining this will be described. As shown in FIG. 3, a number is assigned to the position of the intersection between the light beam from the pixel on the imaging surface and the object. If this is i, i takes a value from 0 to 10 in the example of FIG. Considering the reference plane of the object parallel to the imaging plane, the intersection between the reference plane and the light beam is always a constant interval. Let the pitch be p.
The distance image sensor determines the in-plane distance at each pixel by equation (1). In FIG. 3, this corresponds to obtaining a point obtained by projecting the intersection of the light beam and the object onto the reference plane. For example, it is 1p at the point i = 1, and 2.2p at the point i = 2. Accordingly, it can be seen that the equidistant line for the pixel of i = 2 needs to take a point slightly closer to the i = 1 side.

これを求めるため、対象物と光線との交点の隣接した２点間で、対象物がほぼ直線的に変化すると考え、近似式を求める。ｉ番目の点の基準面内距離をＸｉとする。ｉ番目の等距離点（２次元の場合は、等距離面）の座標をＹｉとすると、これは、隣接した２点の面内距離の値を用いて次式により求めることができる。

Figure 2005087468
例を考える。Ｘ３＝３.３ｐ，Ｘ２＝２.２ｐであり、式(２)より、Ｙ３を計算すると、
Figure 2005087468
 となる。つまり、ｉ＝３の点は、０.３程度、ｉ＝２の点側にずらした点になる。これを図３の釣り鐘状の対象物に対して実施すると図４のように描くことができる。 In order to obtain this, it is assumed that the object changes almost linearly between two adjacent points of intersection between the object and the light beam, and an approximate expression is obtained. Let Xi be the distance in the reference plane of the i-th point. If the coordinate of the i-th equidistant point (equal distance plane in the case of two dimensions) is Yi, this can be obtained by the following equation using the value of the in-plane distance between two adjacent points.
Figure 2005087468
 Consider an example. X3 = 3.3p, X2 = 2.2p, and Y3 is calculated from equation (2),
Figure 2005087468
 It becomes. That is, the point where i = 3 is shifted to the point side where i = 2 by about 0.3. When this is performed on the bell-shaped object of FIG. 3, it can be drawn as shown in FIG.

次に、このような機能を実現するためのイメージセンサについて説明する。これには、繰り返しパルス光を対象物に照射し、光源から放たれた光が対象物で反射して戻ってくるまでの時間を計測することで、距離分布を求めることができる光飛行時間法(Time of flight)による距離画像センサが有用である。
この距離画像センサは、CMOSイメージセンサの技術により、CMOS集積回路として構成することができる。CMOSイメージセンサの技術を用いた距離画像センサは、１つのセンサアレイで、距離情報と明暗情報をともに得ることができるため、その場合には、求めた距離情報と、これにより計算された面内距離分布を、明暗画像上に重ね書きするのは容易である。
なお、これまで面内距離のスケールを網目すなわち変形した格子状のものとして説明したが、これに代えて網目の交点のみをドット表示するもの、面状ではなく直線として１次元のスケールとするものなどの変形が考えられる。また、網目状の変形として正６角形や正３角形を結合した網目状のスケールも考えられる。
１次元のスケールとした場合には、直線の起点・終点を任意に設定できるようにすることが好ましい。 Next, an image sensor for realizing such a function will be described. This is an optical time-of-flight method in which distance distribution can be obtained by repeatedly irradiating an object with pulsed light and measuring the time until the light emitted from the light source is reflected by the object and returns. A distance image sensor based on (Time of flight) is useful.
This distance image sensor can be configured as a CMOS integrated circuit by the technology of the CMOS image sensor. The distance image sensor using the CMOS image sensor technology can obtain both distance information and light / dark information with one sensor array. In this case, the distance information obtained and the in-plane calculated by the distance information are obtained. It is easy to overwrite the distance distribution on the light and dark image.
In the above, the scale of the in-plane distance has been described as a mesh, that is, a deformed grid, but instead, only the intersection of the mesh is displayed as a dot, or a one-dimensional scale as a straight line instead of a plane Variations such as are possible. Further, as a mesh-like deformation, a mesh-like scale in which regular hexagons and regular triangles are combined can be considered.
In the case of a one-dimensional scale, it is preferable that the start and end points of the straight line can be arbitrarily set.

しかし、この場合の明暗画像は、距離画像センサと同じ解像度になるため、距離画像センサの解像度が十分でない場合には、問題となる。
そこで、CMOSイメージセンサの技術を用いて図５に示すように、１つのシリコンチップ上に、明暗画像のイメージセンサと距離画像用のイメージセンサを形成する。それぞれにレンズを設け、同じ対象物の画像を、明暗画像と距離画像の両方に投影する。
シリコンチップ上に一体化することに代えて、セラミックなど他の基盤素材上に明暗画像センサと距離センサを近接して設ける形態での一体化であってもよい。
このとき、明暗画像用イメージセンサと距離画像用イメージセンサの撮像面の大きさを等しくすることもできるが、用途によっては、距離画像用センサの解像度はそれほど高くなくてよく、距離画像用センサの面積を小さくして、イメージセンサチップ全体の面積を小さく実現したい場合がある。 However, since the bright and dark image in this case has the same resolution as the distance image sensor, it becomes a problem when the resolution of the distance image sensor is not sufficient.
Therefore, as shown in FIG. 5, using a CMOS image sensor technique, an image sensor for bright and dark images and an image sensor for distance images are formed on one silicon chip. Each is provided with a lens, and an image of the same object is projected on both the light and dark image and the distance image.
Instead of integration on a silicon chip, integration in a form in which a light and dark image sensor and a distance sensor are provided close to each other on a substrate material such as ceramic may be used.
At this time, the size of the imaging surface of the image sensor for the light and dark image and the image sensor for the distance image can be made equal, but depending on the application, the resolution of the sensor for the distance image may not be so high. There are cases where it is desired to reduce the area of the image sensor chip to reduce the overall area.

このような場合には、距離画像センサと明暗画像センサに対し、図６のように異なる焦点距離のレンズを用いることで、サイズの異なるセンサに同じ対象物の画像を投影することができる。図６に示すように、明暗画像センサと距離画像センサの大きさが異なり、異なる焦点距離のレンズを用いる場合、対象物の位置によって、両者の倍率の比が変動する。
両者の倍率の比は、次式で与えられる。

Figure 2005087468
ｄ１，ｄ２は既知であり、距離画像センサによりＤ２を求めることができるので、距離画像センサで求めた、面内距離分布を明暗画像上に重ね書きするときには、上式により、倍率の補正を行えばよい。なお、この倍率は、対象物が遠ざかればｄ１/ｄ２の一定値に近づく。 In such a case, an image of the same object can be projected onto sensors having different sizes by using lenses having different focal lengths as shown in FIG. 6 for the distance image sensor and the light / dark image sensor. As shown in FIG. 6, when the size of the light / dark image sensor and the distance image sensor are different and lenses having different focal lengths are used, the ratio of the magnifications of both varies depending on the position of the object.
The ratio between the two magnifications is given by the following equation.
Figure 2005087468
 Since d1 and d2 are known and D2 can be obtained by the distance image sensor, when the in-plane distance distribution obtained by the distance image sensor is overwritten on the bright and dark image, the magnification is corrected by the above formula. Just do it. Note that this magnification approaches a constant value of d1 / d2 as the object moves away.

次に、図５のように距離画像センサと、明暗画像センサの２つのセンサを持つ場合の両者の画像の位置ずれについて考える。説明を簡単にするために、図７に示すように、距離画像センサと、明暗画像センサは、同じ大きさの撮像面を持ち、レンズも同じ焦点距離のものを用いるものとする。
対象物の位置により、明暗画像センサ上の対象物の位置と、距離画像センサ上の対象物の位置にずれが生じる。両者に位置ずれが生じない距離が存在する。これを、Ｄ０とすると、対象物の基準面がこれからずれたときの撮像面上での位置ずれ量は、次式で与えられる。

Figure 2005087468
ここで、ｓは、２つのセンサ用、レンズの光軸間の長さ、Ｄは、レンズから対象物の基準面までの距離、ｄは、レンズから撮像面までの距離である。この位置ずれ量を補正することで、明暗画像センサで得た明暗画像上に、正確に距離画像センサにより得た面内距離情報を重ねがきすることができる。 Next, as shown in FIG. 5, the positional deviation between the images when the distance image sensor and the light and dark image sensor are provided will be considered. In order to simplify the explanation, as shown in FIG. 7, it is assumed that the distance image sensor and the light / dark image sensor have imaging surfaces of the same size and lenses having the same focal length.
Due to the position of the object, there is a difference between the position of the object on the light and dark image sensor and the position of the object on the distance image sensor. There is a distance where no positional deviation occurs between the two. When this is D0, the positional deviation amount on the imaging surface when the reference plane of the object deviates from this is given by the following equation.
Figure 2005087468
 Here, s is for two sensors, the length between the optical axes of the lenses, D is the distance from the lens to the reference plane of the object, and d is the distance from the lens to the imaging plane. By correcting this positional deviation amount, the in-plane distance information accurately obtained by the distance image sensor can be overlaid on the light and dark image obtained by the light and dark image sensor.

なお、距離画像センサと、明暗画像センサの撮像面の大きさが異なり、レンズの焦点距離が異なる場合についても、計算はやや複雑となるが同様な計算により、位置ずれ量の計算を行うことができる。
距離画像センサでは、対象物までの距離分布が分かるので、図４の面内距離分布を表す網目を重ねがきする機能に加えて、図８に示すように、対象物までの距離を等高線表示し、これを、明暗画像上に重ねがきすることもできる。この等高線表示と、面内等距離線をともに重ねて書くこともできる。 It should be noted that even when the distance image sensor and the light and dark image sensor have different imaging surfaces and the focal lengths of the lenses are different, the calculation is slightly complicated, but the amount of misalignment can be calculated by the same calculation. it can.
Since the distance image sensor knows the distance distribution to the object, in addition to the function of overlaying the mesh representing the in-plane distance distribution of FIG. 4, the distance to the object is displayed as a contour line as shown in FIG. This can also be overlaid on the light and dark image. The contour line display and the in-plane equidistant line can be written together.

撮像対象物の絶対的な大きさを知りたいという要求は、例えば内視鏡により病巣の大きさ、その成長または縮小の度合いを知りたい場合など多くあり、本発明はこれらの要求を満たすことができる。   There are many requests for knowing the absolute size of an object to be imaged, for example, when it is desired to know the size of a lesion, the degree of growth or reduction by an endoscope, and the present invention can satisfy these requirements. it can.

レンズのＦ値を大きくした場合の被写界深度Depth of field when F-number of lens is increased 対象物に対する奥行き距離からの面内距離を求める原理図Principle diagram for determining the in-plane distance from the depth distance to the object 網目スケール表示の原理Principle of mesh scale display 面内等距離線による網目スケールの実例Example of mesh scale with in-plane equidistant lines 明暗画像センサと距離画像センサの集積化チップIntegrated chip for light and dark image sensors and distance image sensors 明暗画像センサと距離画像センサによる同じ対象物の撮像Imaging the same object with a light and dark image sensor and a distance image sensor 明暗画像センサと距離画像センサにおける対象物の位置ずれを説明する図The figure explaining the positional offset of the target object in a light-dark image sensor and a distance image sensor 距離分布の等高線表示Contour display of distance distribution

符号の説明Explanation of symbols

Ｄ：レンズ−被写体間距離
ｄ：レンズ−撮像面間距離
Ｘ：被写体の位置における距離スケール
ｘ：撮像面の位置における距離スケール
Ｌ１：レンズ−対象物間距離
Ｘ１：対象物における基準線からの距離
ｘ１：撮像面における基準線からの距離
ｐ：格子のピッチ
ｈ：対象物の撮像面における大きさ
ｓ：レンズの中心線間距離 D: Lens-subject distance d: Lens-imaging distance X: Distance scale at the subject position x: Distance scale at the imaging surface position L1: Lens-object distance X1: Distance from the reference line on the object x1: Distance from the reference line on the imaging surface p: Pitch of the grating h: Size of the object on the imaging surface s: Distance between the center lines of the lenses

Claims (7)

撮像装置において、対象物までの距離の２次元分布を取得するセンサと、対象物をセンサの撮像面上に投影するレンズと、計算手段とからなり、前記レンズと対象物までの距離、前記センサの撮像面上の距離及び撮像面と前記レンズまでの距離から、対象物の面内等距離分布を求めることを特徴とする距離画像計測機能を有する撮像装置。 In the imaging apparatus, the sensor includes a sensor that acquires a two-dimensional distribution of the distance to the object, a lens that projects the object on the imaging surface of the sensor, and a calculation unit. The distance between the lens and the object, the sensor An imaging apparatus having a distance image measurement function, wherein an in-plane equidistance distribution of an object is obtained from a distance on the imaging surface and a distance from the imaging surface to the lens. 前記センサは、繰り返しパルス光を対象物に投じ、その反射光を撮像面で受信するまでの光の飛行時間を、その画素毎に測定するために、その飛行時間により生じた遅れ時間に依存する信号を取り出すものである請求項１記載の距離画像計測機能を有する撮像装置。 The sensor relies on the delay time caused by the flight time in order to repeatedly measure the flight time of light until the reflected light is repeatedly projected on the object and received by the imaging surface. The imaging apparatus having a distance image measurement function according to claim 1, which extracts a signal. 撮像装置において、対象物の明暗を表す２次元画像と対象物までの距離を表す２次元分布を取得するセンサと、対象物をセンサの撮像面上に投影するレンズと、計算手段とを備え、前記計算手段により前記レンズと対象物までの距離、前記センサの撮像面上の距離及び撮像面と前記レンズまでの距離から、対象物の面内等距離分布を求め、得られた面内等距離画像分布を、明暗画像上に重ねがきすることを特徴とする距離画像計測機能を有する撮像装置。 The imaging apparatus includes a sensor that acquires a two-dimensional image representing the brightness and darkness of the object and a two-dimensional distribution representing the distance to the object, a lens that projects the object on the imaging surface of the sensor, and a calculation unit. The in-plane equidistant distribution of the object is obtained by calculating the in-plane equidistant distribution of the object from the distance between the lens and the object, the distance on the imaging surface of the sensor and the distance between the imaging surface and the lens by the calculating means. An imaging apparatus having a distance image measurement function, wherein an image distribution is overlaid on a light-dark image. 撮像装置において、対象物の明暗を表す２次元画像と対象物までの距離を表す２次元分布を取得するセンサと、対象物をセンサの撮像面上に投影するレンズと、計算手段とを備え、前記計算手段により前記レンズと対象物までの距離、前記センサの撮像面上の距離及び撮像面と前記レンズまでの距離から、対象物の面内等距離分布を求め、得られた面内距離分布を用いて、明暗画像上の指定した位置に対象物の大きさを測ることができる目盛りを表示することを特徴とする距離画像計測機能を有する撮像装置。 The imaging apparatus includes a sensor that acquires a two-dimensional image representing the brightness and darkness of the object and a two-dimensional distribution representing the distance to the object, a lens that projects the object on the imaging surface of the sensor, and a calculation unit. The in-plane equidistant distribution of the object is obtained from the distance between the lens and the object, the distance on the imaging surface of the sensor and the distance between the imaging surface and the lens by the calculating means, and the obtained in-plane distance distribution is obtained. An image pickup apparatus having a distance image measurement function, characterized in that a scale capable of measuring the size of an object is displayed at a specified position on a light and dark image. 前記センサは、１つのシリコンチップ上に、明暗画像を取得する２次元アレイと距離画像を取得する２次元アレイが一定距離を隔てて配置され、それぞれ毎にレンズを設けて同じ対象画像を投影するものである請求項３または請求項４記載の距離画像計測機能を有する撮像装置。 In the sensor, a two-dimensional array for acquiring a light and dark image and a two-dimensional array for acquiring a distance image are arranged on a single silicon chip at a predetermined distance, and a lens is provided for each to project the same target image. 5. An imaging apparatus having a distance image measurement function according to claim 3 or 4. 前記計算手段は奥行き方向を表す等高線を求め、得られた等高線を明暗画像上にかさねて描く機能を有するものである請求項３または請求項４記載の距離画像計測機能を有する撮像装置。 5. The imaging apparatus having a distance image measurement function according to claim 3, wherein the calculating means has a function of obtaining contour lines representing a depth direction and drawing the obtained contour lines over a bright and dark image. 請求項１乃至６記載の距離画像計測機能を有する撮像装置を組み込んでなる内視鏡装置。 An endoscope apparatus incorporating the imaging apparatus having the distance image measurement function according to claim 1.
JP2003324579A 2003-09-17 2003-09-17 Image pickup device provided with distance image measurement function and endoscopic apparatus Pending JP2005087468A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2003324579A JP2005087468A (en) 2003-09-17 2003-09-17 Image pickup device provided with distance image measurement function and endoscopic apparatus
PCT/JP2004/013699 WO2005027739A1 (en) 2003-09-17 2004-09-13 Imaging device and endoscope having distance image measuring function

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2003324579A JP2005087468A (en) 2003-09-17 2003-09-17 Image pickup device provided with distance image measurement function and endoscopic apparatus

Publications (1)

Publication Number Publication Date
JP2005087468A true JP2005087468A (en) 2005-04-07

Family

ID=34372749

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2003324579A Pending JP2005087468A (en) 2003-09-17 2003-09-17 Image pickup device provided with distance image measurement function and endoscopic apparatus

Country Status (2)

Country Link
JP (1) JP2005087468A (en)
WO (1) WO2005027739A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008042580A (en) * 2006-08-07 2008-02-21 Chugoku Electric Power Co Inc:The Imaging apparatus
JP2009025225A (en) * 2007-07-23 2009-02-05 Fujifilm Corp Three-dimensional imaging apparatus and control method for the same, and program
EP2160974A1 (en) 2008-09-04 2010-03-10 Olympus Medical Systems Corporation Imaging system
JP2010102113A (en) * 2008-10-23 2010-05-06 Olympus Corp Image processor, endoscope apparatus, endoscope system, and program
US7889916B2 (en) 2006-06-30 2011-02-15 Brother Kogyo Kabushiki Kaisha Image processor
JP2011069965A (en) * 2009-09-25 2011-04-07 Japan Atomic Energy Agency Image capturing apparatus, image display method, and recording medium with image display program recorded thereon
JP2011151087A (en) * 2010-01-19 2011-08-04 Panasonic Corp Component mounting device, component detecting apparatus, and component mounting method
JP2017510409A (en) * 2014-03-28 2017-04-13 インテュイティブ サージカル オペレーションズ， インコーポレイテッド Surgical system using haptic feedback based on quantitative three-dimensional imaging
JPWO2016181453A1 (en) * 2015-05-11 2017-10-26 オリンパス株式会社 Endoscope device
JPWO2016181454A1 (en) * 2015-05-11 2017-10-26 オリンパス株式会社 Endoscope device
JPWO2016181452A1 (en) * 2015-05-11 2017-11-30 オリンパス株式会社 Endoscope device
US10334227B2 (en) 2014-03-28 2019-06-25 Intuitive Surgical Operations, Inc. Quantitative three-dimensional imaging of surgical scenes from multiport perspectives
US10350009B2 (en) 2014-03-28 2019-07-16 Intuitive Surgical Operations, Inc. Quantitative three-dimensional imaging and printing of surgical implants
US10368054B2 (en) 2014-03-28 2019-07-30 Intuitive Surgical Operations, Inc. Quantitative three-dimensional imaging of surgical scenes
JP2019521778A (en) * 2016-07-14 2019-08-08 アエスキュラップ アーゲー Endoscope apparatus and endoscopy method
US11266465B2 (en) 2014-03-28 2022-03-08 Intuitive Surgical Operations, Inc. Quantitative three-dimensional visualization of instruments in a field of view

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9545220B2 (en) 2014-03-02 2017-01-17 V.T.M (Virtual Tape Measure) Technologies Ltd. Endoscopic measurement system and method

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0541901A (en) * 1991-08-09 1993-02-23 Olympus Optical Co Ltd Endoscope for three-dimensional measurement
JP3041420B1 (en) * 1999-01-27 2000-05-15 工業技術院長 Endoscope system and recording medium storing control program for detecting depth information of endoscope image
JP2002065581A (en) * 2000-08-25 2002-03-05 Fuji Photo Film Co Ltd Endoscope device
SE0003666D0 (en) * 2000-10-11 2000-10-11 Astrazeneca Ab A delivery device

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7889916B2 (en) 2006-06-30 2011-02-15 Brother Kogyo Kabushiki Kaisha Image processor
JP2008042580A (en) * 2006-08-07 2008-02-21 Chugoku Electric Power Co Inc:The Imaging apparatus
JP2009025225A (en) * 2007-07-23 2009-02-05 Fujifilm Corp Three-dimensional imaging apparatus and control method for the same, and program
US8278615B2 (en) 2008-09-04 2012-10-02 Olympus Medical Systems Corp. Imaging system having a variable optical transmission timing
EP2160974A1 (en) 2008-09-04 2010-03-10 Olympus Medical Systems Corporation Imaging system
JP2010057740A (en) * 2008-09-04 2010-03-18 Olympus Medical Systems Corp Imaging system
JP2010102113A (en) * 2008-10-23 2010-05-06 Olympus Corp Image processor, endoscope apparatus, endoscope system, and program
DE102010040518A1 (en) 2009-09-25 2011-04-28 Advanced Technology Co., Ltd. Image recording device, image display method and recording medium, and image recording program recorded thereon
JP2011069965A (en) * 2009-09-25 2011-04-07 Japan Atomic Energy Agency Image capturing apparatus, image display method, and recording medium with image display program recorded thereon
US8792000B2 (en) 2009-09-25 2014-07-29 Japan Atomic Energy Agency Image capturing apparatus, image displaying method and recording medium, image displaying program being recorded thereon
JP2011151087A (en) * 2010-01-19 2011-08-04 Panasonic Corp Component mounting device, component detecting apparatus, and component mounting method
JP2017510409A (en) * 2014-03-28 2017-04-13 インテュイティブ サージカル オペレーションズ， インコーポレイテッド Surgical system using haptic feedback based on quantitative three-dimensional imaging
US10555788B2 (en) 2014-03-28 2020-02-11 Intuitive Surgical Operations, Inc. Surgical system with haptic feedback based upon quantitative three-dimensional imaging
US11304771B2 (en) 2014-03-28 2022-04-19 Intuitive Surgical Operations, Inc. Surgical system with haptic feedback based upon quantitative three-dimensional imaging
US11266465B2 (en) 2014-03-28 2022-03-08 Intuitive Surgical Operations, Inc. Quantitative three-dimensional visualization of instruments in a field of view
US10334227B2 (en) 2014-03-28 2019-06-25 Intuitive Surgical Operations, Inc. Quantitative three-dimensional imaging of surgical scenes from multiport perspectives
US10350009B2 (en) 2014-03-28 2019-07-16 Intuitive Surgical Operations, Inc. Quantitative three-dimensional imaging and printing of surgical implants
US10368054B2 (en) 2014-03-28 2019-07-30 Intuitive Surgical Operations, Inc. Quantitative three-dimensional imaging of surgical scenes
JPWO2016181453A1 (en) * 2015-05-11 2017-10-26 オリンパス株式会社 Endoscope device
JPWO2016181452A1 (en) * 2015-05-11 2017-11-30 オリンパス株式会社 Endoscope device
JPWO2016181454A1 (en) * 2015-05-11 2017-10-26 オリンパス株式会社 Endoscope device
JP2019521778A (en) * 2016-07-14 2019-08-08 アエスキュラップ アーゲー Endoscope apparatus and endoscopy method
US11213189B2 (en) 2016-07-14 2022-01-04 Aesculap Ag Endoscopic device and method for endoscopic examination
JP2022062231A (en) * 2016-07-14 2022-04-19 アエスキュラップ アーゲー Endoscope device and method of endoscopic examination

Also Published As

Publication number Publication date
WO2005027739A1 (en) 2005-03-31

Similar Documents

Publication Publication Date Title
JP2005087468A (en) Image pickup device provided with distance image measurement function and endoscopic apparatus
US8900126B2 (en) Optical scanning device
US9330324B2 (en) Error compensation in three-dimensional mapping
EP1555507B1 (en) Three-dimensional visual sensor
JP4885584B2 (en) Rangefinder calibration method and apparatus
JP5140761B2 (en) Method for calibrating a measurement system, computer program, electronic control unit, and measurement system
CN109938837B (en) Optical tracking system and optical tracking method
US6377298B1 (en) Method and device for geometric calibration of CCD cameras
TW200919271A (en) System and method for performing optical navigation using scattered light
KR20130032368A (en) Three-dimensional measurement apparatus, three-dimensional measurement method, and storage medium
JP5633058B1 (en) 3D measuring apparatus and 3D measuring method
JP2011069965A (en) Image capturing apparatus, image display method, and recording medium with image display program recorded thereon
JP2008241643A (en) Three-dimensional shape measuring device
JPH11160021A (en) Wide area three-dimensional position measuring method and equipment
JP2008249431A (en) Three-dimensional image correction method and its device
JP2007093412A (en) Three-dimensional shape measuring device
JP5487946B2 (en) Camera image correction method, camera apparatus, and coordinate transformation parameter determination apparatus
JP2010048553A (en) Inspecting method of compound-eye distance measuring device and chart used for same
JP2012013592A (en) Calibration method for three-dimensional shape measuring machine, and three-dimensional shape measuring machine
JP6108383B2 (en) Piping position measuring system and piping position measuring method
JPH0412724A (en) Measuring endoscope
CN114383817B (en) High-precision synchronous scanning optical system adjustment precision evaluation method
JP2006271503A (en) Endoscopic apparatus with three-dimensional measuring function
JP3369235B2 (en) Calibration method for measuring distortion in three-dimensional measurement
US20240078694A1 (en) Image processing apparatus, image processing method, and recording medium

Legal Events

Date Code Title Description
A711 Notification of change in applicant

Free format text: JAPANESE INTERMEDIATE CODE: A711

Effective date: 20050315

A711 Notification of change in applicant

Free format text: JAPANESE INTERMEDIATE CODE: A711

Effective date: 20050303

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A821

Effective date: 20050316

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20060116

RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20070223

RD05 Notification of revocation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7425

Effective date: 20070223

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20070724

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20080205