JP2008061659A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more accurately display the size and shape of a region to be observed in an imaging device. <P>SOLUTION: The imaging device 1 comprises an imaging part 11 with a lens 12 for capturing the image of the region to be observed via the lens 12; a light source 20 and optical fiber 15 constituting a laser irradiating means for irradiating a plurality of points in the region to be observed with laser light; a correction part 32 for correcting the distortion of the captured image caused by the lens 12 based on the information indicating the distortion characteristics of the lens 12; a detection part 33 for detecting coordinates of the irradiation points of the laser light; a three-dimensional coordinate extraction part 34 for extracting the three-dimensional coordinates of the irradiation points of the laser light from the detected coordinates and conditions for laser irradiation; a prescribed point extraction part 35 for extracting coordinates of scale marks on the image between the irradiation points from the extracted three-dimensional coordinates; and an output part 36 for outputting the information on the image and the information indicating the coordinates of the scale marks on the image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus applied to an endoscope or the like.

  In a medical endoscope, obtaining an actual size or shape of a lesion from an image of a portion to be observed is very important as a judgment material for determining medical means. However, normally, because of the distortion of the lens, the captured image is distorted due to the configuration of the endoscope, and accurate distance information from the lens to the observed part cannot be obtained. did not understand. For this reason, endoscope operators, doctors, and the like have estimated their sizes and shapes from images displayed based on experience.

In order to accurately display the size and shape of the observed portion, for example, in Patent Document 1 below, focusing is performed according to the shape of the unevenness or the like of the observed portion using the field curvature characteristics of the lens. It is carried out. Thereby, the height information of the uneven shape can be obtained. Further, Patent Document 2 described below describes that scale display (concentric, orthogonal scale, etc.) from a predetermined point in an image can be performed.
JP 2001-269306 A JP 2002-153421 A

  However, in the technique described in Patent Document 1, since the acquired image is distorted, an accurate shape is not displayed. Moreover, in the technique described in Patent Document 2, since the scale is displayed based on the focal length, it cannot always be said that the scale is accurate.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an imaging apparatus capable of displaying the size and shape of the observed portion more accurately.

  In order to achieve the above object, an imaging apparatus according to the present invention has a lens and irradiates laser light to a plurality of points on the observed portion with imaging means for capturing an image of the observed portion via the lens. Laser light irradiation means, correction means for correcting distortion caused by the lens of the image captured by the imaging means based on information indicating the distortion characteristics of the lens stored in advance, and laser light irradiation means in the image corrected by the correction means From the detection means for detecting the coordinates of each irradiation point of the laser light emitted by the laser beam, the coordinates detected by the detection means, and the irradiation conditions of the laser light, each irradiation point of the laser light emitted by the laser light irradiation means 3D coordinate deriving means for deriving 3D coordinates, and 3D coordinates of points having a predetermined positional relationship between the irradiation points are derived from the 3D coordinates derived by the 3D coordinate deriving means. A predetermined point deriving unit for deriving the coordinates of the predetermined positional relationship point on the image corrected by the correcting unit from the three-dimensional coordinates, and information on the image corrected by the correcting unit, and the predetermined point deriving unit Output means for outputting information indicating the coordinates of the derived point of the predetermined positional relationship on the image.

  In the imaging apparatus according to the present invention, distortion caused by the lens is removed from the captured image, so that a more accurate shape can be displayed. In the imaging apparatus according to the present invention, each irradiation point of the laser light is detected from an image in which lens distortion is corrected. From the detected irradiation points of the laser light, three-dimensional coordinates of the irradiation points are derived, and further, coordinates on the corrected image of points having a predetermined positional relationship between the irradiation points are derived. The coordinates of this point can be used as information indicating the size of the observed part based on the three-dimensional coordinates of the laser light irradiation point, mainly as the coordinates of the scale. Thereby, according to the imaging device concerning the present invention, the size of a part to be observed can be displayed more correctly.

  It is desirable that the output means superimposes a scale on the coordinate position of a point having a predetermined positional relationship on the image derived by the predetermined point deriving means, and outputs an image in which the scale is superimposed. According to this configuration, since the image on which the scale is superimposed is displayed, it is possible to reliably perform image display so that the size of the observed portion can be accurately understood.

  It is desirable that the detection unit detects the coordinates by searching a predetermined range in the image corrected by the correction unit. According to this configuration, the irradiation point can be detected efficiently, and as a result, the processing in the imaging apparatus according to the present invention can be performed efficiently.

  Desirably, the predetermined point deriving means receives an input of information indicating that a point having a predetermined positional relationship is to be moved, and moves the point having a predetermined positional relationship in accordance with the input. According to this configuration, for example, the scale can be moved to a desired location, and the size of the observed portion can be recognized more easily.

  The imaging means is preferably configured to be included in the endoscope. According to this structure, the application to the medical field of this invention can be performed more easily.

  According to the present invention, distortion caused by a lens is removed from a captured image, so that a more accurate shape can be displayed. Further, in the present invention, the corrected coordinates on the image of the points having a predetermined positional relationship between the irradiation points of the plurality of laser beams are derived. As a result, according to the present invention, information indicating the size of the observed portion based on the three-dimensional coordinates of the laser light irradiation point, mainly used as the coordinates of the scale, can be used more accurately. Can be displayed.

  Hereinafter, preferred embodiments of an imaging apparatus according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. Further, the dimensional ratios in the drawings do not necessarily match those described.

  FIG. 1 shows an imaging apparatus 1 according to this embodiment. The imaging device 1 is a device that captures an image of a portion to be observed inside a living body such as a human body and displays the captured image. That is, the imaging apparatus 1 according to the present embodiment is an application of the present invention to an endoscope. As shown in FIG. 1, an imaging apparatus 1 includes an endoscope 10 that is inserted into a living body and performs imaging, a light source 20 that supplies light beams emitted from the distal end of the endoscope 10, and an endoscope 10 is configured to include an information processing device 30 that processes an image picked up by the camera 10 and a monitor 40 that displays the image.

  The endoscope 10 has an elongated shape so that it can be inserted into a living body, and includes an imaging unit 11 that is an imaging unit for imaging an observed portion inside the living body at a distal end portion 10a thereof. Specifically, the imaging unit 11 includes a lens 12 that is positioned at the distal end portion 10 a of the endoscope 10 so as to face the observed portion, and a CCD (Charge Coupled) that is provided at a resolution position of the lens 12. Device Image Sensor) includes an image sensor 13 such as an image sensor. Information on the image captured by the imaging unit 11 is output to the information processing apparatus 30 connected to the endoscope 10 by a cable at the connector 14 provided at the end opposite to the distal end portion 10a of the endoscope 10. .

  Since the imaging unit 11 captures an image of the observed portion via the lens 12, distortion due to the lens 12 occurs. In the present embodiment, the image to be captured is circular. For example, when a test pattern in which dots are arranged so as to have a lattice shape as illustrated in FIG. ). This distortion is caused by the fact that the ratio of the actual distance corresponding to the distance to the distance from the center of the image to each point is not uniform and is different between the points of the image.

  In addition, the endoscope 10 irradiates a plurality of points on the observed part imaged by the imaging part 11 with laser light. In addition to the information processing apparatus described above, the light source 20 is optically connected to the connector 14 of the endoscope 10. Laser light is input from the light source 20 to the optical fiber 15 disposed inside the endoscope 10. The laser light is emitted from a plurality of emission points 15 a at the distal end portion 10 a of the endoscope 10 through the optical fiber 15. FIG. 3 shows the front of the distal end portion 10 a of the endoscope 10. As shown in FIG. 3, in the present embodiment, the endoscope 10 is provided with four laser light emission points 15a (four optical fibers are arranged in the endoscope 10). The laser light is emitted so that its emission direction (optical path) is parallel to the optical axis direction of the imaging unit 11 (lens 12 thereof).

  As shown in FIG. 3, the endoscope 10 further includes an illumination 16 for enabling imaging in a bright state inside the living body, a hole 17 for passing forceps for performing treatment on the inside of the living body, A hole 18 through which water or air passes is provided in the living body (these are omitted in FIG. 1).

  As described above, the light source 20 is a device that supplies laser light for irradiating the observed portion. That is, the light source 20 constitutes laser light irradiation means for irradiating laser light to a plurality of points of the observed portion together with the endoscope 10 and the optical fiber 15 included in the endoscope 10. Specifically, a laser diode can be used as the light source 20. The light source 20 may be used not only as a light source for laser light emitted from the emission point 15a but also as a light source for the illumination 16.

  The information processing apparatus 30 is an apparatus that processes an image captured by the endoscope 10. The information processing apparatus 30 is connected to the endoscope 10 via a cable and can receive information from the endoscope 10, and information on an image captured by the imaging unit 11 of the endoscope 10 is input. The Specifically, the information processing apparatus 30 is configured by hardware such as a CPU (Central Processing Unit) and a memory, and the following operations of the information processing apparatus 30 are realized by operating these components. As shown in FIG. 1, the information processing apparatus 30 functionally includes a receiving unit 31, a correcting unit 32, a detecting unit 33, a three-dimensional coordinate deriving unit 34, a predetermined point deriving unit 35, and an output unit 36. And is configured.

  The receiving unit 31 is means for receiving information on an image output from the endoscope 10. The reception unit 31 outputs the received image information to the correction unit 32.

  The correction unit 32 is a correction unit that corrects distortion caused by the lens 12 that occurs in an image captured by the endoscope 10. The correction unit 32 stores information indicating the distortion characteristics of the lens 12 and corrects distortion based on the information. The information indicating the distortion characteristics of the lens 12 is information indicating the ratio of the actual distance to the distance between two points in the image, and is, for example, as shown in the graph of FIG. In FIG. 4, the vertical axis indicates the distance from the center of the image on the image (normalized with the distance to the edge of the image being 1), and the horizontal axis is the actual distance corresponding to the distance on the image. (Normalized in the same manner as the distance on the image). As shown in FIGS. 2 and 4, due to the influence of the lens 12, the distance on the image becomes longer than the actual distance depending on the position on the image. Note that the correction unit 32 does not necessarily store the information of the graph shown in FIG. 4 as information indicating the distortion characteristics of the lens 12, and stores information on the actual distance with respect to the distance on the image. Just keep it.

  The correction of the image by the correction unit 32 is performed by, for example, deriving the actual distance from each position on the image using the information indicating the distortion characteristics of the lens 12 from the distance on the image. The pixel value (color) of the pixel at the distance position on the image is given to the pixel at the distance position. The corrected image information is output to the detection unit 33 and the output unit 36.

  Here, an example of a method for acquiring information indicating the distortion characteristics of the lens 12 stored by the correction unit 32 will be described. First, as shown in FIG. 2A, a test pattern in which points (hereinafter referred to as pattern points) are arranged in a grid pattern is imaged by the imaging unit 11 of the imaging device 1. The captured image is distorted in the lens 12 in accordance with the distortion characteristics as shown in FIG. Information indicating the distortion characteristics of the lens 12 is acquired by measuring the distance on the captured image and the corresponding distance on the test pattern.

The measurement of each distance will be described with reference to FIG. As shown in FIG. 5A, first, the center point 50 of the captured image is obtained. The center point 50 is obtained as a point equidistant from the edge of the circular image. Next, a straight line 52 is drawn from the center point 50 so as to pass through the center of any pattern point 51. As shown in FIG. 5B, even in a test pattern (not an imaged image), a pattern point P _m corresponding to a pattern point 51 on the image passes from a center point O corresponding to the center point 50 of the image. Draw a line like so.

  Subsequently, as shown in FIG. 5A, a line segment 54 is drawn between two adjacent pattern points 53 located so as to sandwich the straight line 52 in the captured image. The line segment 54 is drawn between a set of a plurality of pattern points 53 as shown in FIG. An intersection 55 is formed between the straight line 52 and the line segment 54 drawn in this way, and the distance from the center point 50 to each intersection 55 is measured. The measured distance is divided by the radius (the distance from the center point 50 to the circumference). That is, normalization is performed so that the center point is 0 and the end (radius) of the image is 1.

Next, in the test pattern shown in FIG. 5B (actual, not imaged), a point P _n corresponding to each intersection 55 (where P _n is the n + 1th point from the center point 50 (n is non-negative) (Integer) is a point corresponding to the intersection 55). Although this distance may be actually measured, it is not necessary to actually measure all, and can be obtained by calculation based on the actual measurement of some distances. The distance D _n from the center point O to the point P _n in FIG. 5B is obtained as follows. Let _xb be the actual distance between adjacent pattern points. Here, the distance _{x 0} in FIG. 5, knowing the distance _{y m,} can be obtained the value of _{D m.} The distance x ₀ from the center point O, the distance to the line connecting the pattern points according to the point P _0, the distance y _m is such that the line perpendicular connecting the pattern points according to the point P ₀ is the distance from to the pattern point P _m (dashed line in FIG. 5 (b)) a line drawn from the center point O to.

From these values, the similarity of triangles
D _n = (x ₀ + nx _b ) D _m / (x ₀ + mx _b )
Thus, the distance D _n to each point P _n can be obtained. With the above method, for example, information on the correspondence between the distance on the image and the actual distance can be obtained for dozens of points, and this can be used as information indicating the distortion characteristics of the lens 12.

  In addition, as a method for acquiring information indicating the distortion characteristics of the lens 12, for example, a method described in Japanese Patent Laid-Open No. 63-246716 (taking a square square on a plane under the same conditions, It is also possible to apply a correction value for each pixel so that the image is square.

  The detection unit 33 is a detection unit that detects the coordinates of each irradiation point of the laser light emitted from the endoscope 10 to the observed portion in the image corrected by the correction unit 32. The detected coordinates are, for example, coordinates whose origin is the center of the image corresponding to the optical center of the imaging unit 11 (the lens 12 thereof). The detection of the coordinates of each irradiation point will be described in detail later. Information indicating the coordinates of each detected irradiation point is output to the three-dimensional coordinate deriving unit 34.

  The three-dimensional coordinate deriving unit 34 derives the three-dimensional coordinates of the laser light irradiation points from the coordinates of the laser light irradiation points detected by the detection unit 33 and the laser light irradiation conditions. The laser light irradiation conditions used for derivation are the laser light emission position and irradiation direction in the endoscope 10. Since the emission position of the laser beam in the endoscope 10 is previously determined, it is stored in advance in the three-dimensional coordinate deriving unit 34 as the position (three-dimensional coordinates) with respect to the optical center of the imaging unit 11 (the lens 12 thereof). . Further, the irradiation direction of the laser light is similarly determined in advance and is stored in the three-dimensional coordinate deriving unit 34. The derivation of the three-dimensional coordinates will be described in detail later. Information indicating the three-dimensional coordinates of each derived irradiation point of the laser beam is output to the predetermined point deriving unit 35.

  The predetermined point deriving unit 35 derives the three-dimensional coordinates of the points having a predetermined positional relationship between the irradiation points from the three-dimensional coordinates derived by the three-dimensional coordinate deriving unit 34, and the derived three-dimensional coordinates Is a predetermined point deriving unit for deriving the coordinates of the predetermined positional relationship point on the image corrected by the correcting unit 32. The point of the predetermined positional relationship between each irradiation point is a point which becomes a scale which can know the magnitude | size of a to-be-observed part (for example, organ etc.), for example. The predetermined positional relationship between the irradiation points is stored in advance in the predetermined point deriving unit 35. Derivation of the predetermined positional relationship point will be described later in more detail. Information indicating the derived point of the predetermined positional relationship is output to the output unit 36.

  The output unit 36 is an output unit that outputs the information of the image corrected by the correction unit 32 and the information indicating the coordinates of the point of the predetermined positional relationship on the image derived by the predetermined point deriving unit 35 to the monitor 40. is there. When output to the monitor 40, the information indicating the coordinates is incorporated into the information of the image corrected by the correction unit 32 as a scale and output as information of one image.

  The monitor 40 displays the image information output from the information processing apparatus 30 on the screen. The user can see the image captured by the endoscope 10 by referring to the monitor 40. The above is the configuration of the imaging apparatus 1.

  Subsequently, the operation of the imaging apparatus 1 will be described with reference to the flowchart of FIG. This operation is performed when the endoscope 10 of the imaging device 1 is inserted into the living body and the observed portion is imaged. First, the laser beam is irradiated from the endoscope 10 to a plurality of points on the observed portion (S01).

  Subsequently, an image of the observed portion is captured by the imaging unit 11 of the endoscope 10 (S02). Information on the captured image is output from the endoscope 10 to the information processing apparatus 30. In the information processing apparatus 30, information on the image is received by the receiving unit 31. The image information is output from the receiving unit 31 to the correcting unit 32.

  Subsequently, the correction unit 32 corrects the distortion of the image by the lens 12 based on the information indicating the distortion characteristics of the lens 12 of the imaging unit 11 (S03). The image whose distortion has been corrected is output to the detection unit 33 and the output unit 36.

  Subsequently, the detection unit 33 detects (two-dimensional) coordinates of each irradiation point of the laser light in the image whose distortion is corrected (S04). Detection of the coordinates of each irradiation point of the laser light is performed as follows. First, it is not necessary to examine the entire image for detecting the irradiation point. The position where the irradiation point exists is limited by the positional relationship between the laser light emission position and the imaging unit 11 (the lens 12), the irradiation direction of the laser light, and the optical axis of the imaging unit 11 (the lens 12). In the present embodiment, as described above, four laser beams are irradiated so that the emission direction (optical path) is parallel to the optical axis direction of the imaging unit 11 (the lens 12 thereof). Therefore, as shown in FIG. 7, depending on the positional relationship between the observed portion and the endoscope 10, any point on the four line segments 60 in the image picked up by the image pickup portion 11 is the laser light. It becomes an irradiation point (the closer the observed portion is to the distal end portion 10a of the endoscope 10, the closer the irradiation point is to the end on the image). The position on the image where the line segment 60 is located can be detected in advance and set in the detection unit 33. That is, the detection unit 33 detects the coordinates of the irradiation point by searching for a predetermined range such as the line segment 60 in the image corrected by the correction unit 32.

  The detection of the coordinates of each irradiation point uses the density information of the image. For this purpose, the pixel value is converted from RGB (256 levels each) to 256 levels of shading information. In order to detect the coordinates of the irradiation point, the distribution of the gray value in the vicinity of each pixel and the similarity of the gray distribution are used. For the detection, a sample of a gray distribution at an irradiation point such as a Gaussian distribution and a threshold value of the distribution of gray values are set in the detection unit 33 in advance. The setting of the threshold value is due to the occurrence of a large difference in gray value with surrounding pixels at the laser light irradiation point. Among the pixels whose gray value variance is larger than the set threshold value, the pixel having the highest gray level distribution similarity with the sample is set as the irradiation point. Note that the variance and the similarity can be calculated in one dimension along the line segment 60 shown in FIG. 7 instead of in two dimensions. As the similarity here, for example, a correlation coefficient between an input image (an image corrected by the correction unit 32) and a Gaussian distribution which is a sample of a light and shade distribution is used. Information on the coordinates of each detected irradiation point is output to the three-dimensional coordinate deriving unit 34.

  In the imaging device 1 according to the present embodiment, a highlight is generated in the image due to the reflection of the light of the illumination 16. Since this highlight hinders the irradiation point detection process, a pixel having an RGB component G or B value equal to or larger than a set value may be a highlight portion, and a pixel in the highlight portion may not be an irradiation point. it can. Further, when imaging is performed to detect the irradiation point, it may be controlled not to irradiate the light of the illumination 16.

  Moreover, it is good also as smoothing in order to remove the noise of an input image in the case of detection of an irradiation point. As a smoothing method, it is preferable to use a median filter in order to leave the feature of the irradiation point. The median filter is performed in a range of, for example, 5 × 5 pixels in consideration of both smoothing of an image and retention of feature points.

Subsequently, based on the coordinates of each irradiation point of the laser beam on the detected image, the three-dimensional coordinate deriving unit 34 derives the three-dimensional coordinates of each irradiation point (S05). The detection of the three-dimensional coordinates of each irradiation point of the laser light will be described using the perspective projection model of FIG. Here, the three-dimensional coordinate axes are the x-axis, y-axis, and z-axis. The three-dimensional coordinates P (X _p , Y _p , Z _p ) of the irradiation point derived here are coordinates when the optical center of the imaging unit 11 (the lens 12 thereof) is the origin O of the coordinate system. As shown in FIG. 8, the z-axis direction is the optical axis direction of the imaging unit 11 (the lens 12 thereof). Further, the laser beam emission position stored in advance by the three-dimensional coordinate deriving unit 34 is L (X _L , Y _L , Z _L ). Further, the three-dimensional coordinates of the irradiation point on the image are set to p (x _p , y _p , f). Here, f is the focal length of the lens 12. In the actual imaging unit 11, an image surface on the side opposite to the imaging target with the lens 12 interposed therebetween is drawn on the imaging target side in FIG. 8 for simplicity. Further, the three-dimensional coordinates _p irradiation point in the image plane (x _{p, y} p, f) in the _(x p, _{y p)} is detected by the detection unit 33 corresponds to the (two-dimensional) coordinates is there. Note that each coordinate value is associated with a physical distance.

In this case, the straight line connecting the origin O and the irradiation point _P of the laser beam (X _p, Y p, _{Z p)} and passes through the three-dimensional coordinates _p irradiation point on the image (x _{p, y} p, f) the So
x / x _p = y / y _p = z / f (1)
It is represented by The straight line connecting the laser beam emission position L (X _L , Y _L , Z _L ) and the laser beam irradiation point P (X _p , Y _p , Z _p ) is
(X−X _L ) / B _x = (y−Y _L ) / B _y = (z−Z _L ) / B _z (2)
It is represented by Here, (B _x , B _y , B _z ) is a vector indicating the irradiation direction of the laser light, and is stored in advance by the three-dimensional coordinate deriving unit 34. As (B _x , B _y , B _z ), values determined by design may be used, or a screen or the like is previously placed in front of the endoscope 10 of the imaging apparatus 1, and the irradiation point on the screen and the laser beam A value experimentally obtained from the emission position may be used. If the coordinates of the irradiation point of the laser beam (on the screen) are (X _s , Y _s , Z _s ),
_{(B x, B y, B} z) = (X s -X L, Y s -Y L, Z s -Z L) ... (3)
It is obtained as

The laser beam irradiation point P (X _p , Y _p , Z _p ) can be derived by calculating the intersection of the two straight lines (1) and (2) as follows.
_{X p = (B y X L} -B x Y L) x p / (B y x p -B x y p)
_{Y p = (B y X L} -B x Y L) y p / (B y x p -B x y p) ... (4)
_{Z p = (B y X L} -B x Y L) f / (B y x p - B x y p)

In the above derivation process, the three-dimensional coordinates of the irradiation point on the image plane are set to p (x _p , y _p , f), and the position of a specific pixel in the captured image is obtained in units of pixels. Although it is easy, it is generally not easy to obtain the physical distance as a unit. When it is difficult to use (x _p , y _p ) as a physical distance unit, three-dimensional coordinates (of a physical distance) of an arbitrary point are measured in advance and actually photographed. The above derivation may be performed based on these values.

Three-dimensional coordinates of an arbitrary point _{(X t, Y t, Z} t) and, i _t pixels in the x-axis direction from the image center position in the image plane of that point, if it is j _t pixels in the y-axis direction To do. Also, the physical distance per pixel is assumed to be d _y in the x-axis direction d _x, the y-axis direction on the image plane. At this time, the three-dimensional coordinates of the above-mentioned arbitrary point on the image plane can be represented by (i _t d _x , j _t d _x , f),
_{X t / (i t d x} ) = Y t / (j t d x) = Z t / f ... (5)
Is established. Further, _x p, _{y p} likewise representing the xy coordinates of the irradiation point of the image plane in the formula (4),
x _p = i _p d _x , y _p = j _p d _y (6)
It can be. Here, i _p and j _p represent the positions of the irradiation points on the image plane in the x-axis direction and the y-axis direction, respectively, in units of pixels.

If Expressions (5) and (6) are used, Expression (7) can be obtained from Expression (4).
_{X p = (B y X L} -B x Y L) i p j t X t / (B y i p j t X t -B x j p i t Y t)
_{Y p = (B y X L} -B x Y L) j p i t Y t / (B y i p j t X t -B x j p i t Y t)
_{Z p = (B y X L} -B x Y L) i t j t Z t / (B y i p j t X t -B x j p i t Y t)
... (7)
From the right side of Equation (7), x _p , y _p , and f for which it is difficult to obtain a physical distance disappear, and all irradiations are performed using variables that can be actually measured (including those that are measured in advance). The three-dimensional coordinates P (X _p , Y _p , Z _p ) of the point can be derived. The above is a method for deriving the three-dimensional coordinates of each irradiation point of the laser beam. Information indicating the three-dimensional coordinates of each derived irradiation point of the laser beam is output to the predetermined point deriving unit 35.

  Subsequently, the predetermined point deriving unit 35 derives the three-dimensional coordinates of the points having a predetermined positional relationship between the irradiation points from the derived three-dimensional coordinates (S06). Further, the coordinates of the point having the predetermined positional relationship on the image corrected by the correction unit 32 are derived from the derived three-dimensional coordinates. In the present embodiment, the point having a predetermined positional relationship is a point serving as a scale. As shown in FIG. 9, the scale points are points at regular intervals on a line segment from one point to the other of the coordinates of the two irradiation points derived as described above. It is. By displaying this point together with the image, it is possible to know the actual distance between the two irradiation points, that is, the actual size of the observed portion. Information about what scale points to derive, such as scale intervals, is stored in advance in the predetermined point deriving unit 35, and the predetermined point deriving unit 35 performs deriving based on the information.

  Note that if the line segment connecting two irradiation points for deriving a scale point is parallel to the image plane, the equally spaced scales in the real space are also equally spaced on the image plane. What is necessary is just to derive the point which becomes a scale repeatedly at intervals. However, as shown in FIG. 9, the line segment connecting the irradiation points is not always parallel to the image plane. Even if the scales are equally spaced in real space, they are not equally spaced on the image. It is necessary to calculate the position of the scale on the image one by one.

となる。ここで、‖Ｐ_ｅ−Ｐ_ｓ‖は、２つの照射点間の距離である。これらから、Ｍ_ｋ（Ｘ_ｋ，Ｙ_ｋ，Ｚ_ｋ）の画像上での位置ｍ_ｕ（ｘ_ｋ，ｙ_ｋ，ｚ_ｋ）も以下のように導出することができる。
ｘ_ｋ＝Ｘ_ｋｆ／Ｚ_ｋ
ｙ_ｋ＝Ｙ_ｋｆ／Ｚ_ｋ …（９）
ｚ_ｋ＝ｆ
即ち、画像上でのｋ本目の目盛りを画像上における（Ｘ_ｋｆ／Ｚ_ｋ，Ｙ_ｋｆ／Ｚ_ｋ）の位置に表示すればよい。 For example, two irradiation point _{P s (X s, Y s} , Z s) and _{P e (X e, Y e} , Z e) between the, tick marks u intervals as a base point _{P s M} k (k = (0, 1, 2,...) Are displayed. If the position of the kth scale in real space is M _k (X _k , Y _k , Z _k ),

It becomes. Here, ‖P _e -P _s ‖ is the distance between the two irradiation points. From these, the position m _u (x _k , y _k , z _k ) on the image of M _k (X _k , Y _k , Z _k ) can also be derived as follows.
x _k = X _k f / Z _k
y _k = Y _k f / Z _k (9)
z _k = f
That is, the k-th scale on the image may be displayed at the position (X _k f / Z _k , Y _k f / Z _k ) on the image.

Further, instead of using the physical distance as a unit as shown in Equation (9), the position of a point serving as a scale on the image may be determined in units of pixels as follows. As described above, by imaging a point where arbitrary three-dimensional coordinates (X _t , Y _t , Z _t ) are known,
_{f / d x = i t Z} t / X t ... (10)
f / d _y = j _t Z _t / Y _t
Is established. From the three-dimensional coordinates (X _t , Y _t , Z _t ) of the above points and the coordinates (i _t , j _t ) in units of pixels on the image, f / d _x and f / d _y are obtained. Therefore, the coordinates _{(X k f / Z k,} Y k f / Z k) of the k-th scale of an image per pixel each x-axis and y-axis directions physical distance _d x, by using the _{d y} When expressed in pixel units (i _k , j _k ),
i _k = X _k f / (Z _k d _x ) = X _k i _t Z _t / (Z _k X _t ) (11)
_{j k = Y k f / (} Z k d y) = Y k j t Z t / (Z k Y t)
It becomes. That is, the scale of the k-th pixel by pixel on the image _{(X k i t Z t /} (Z k X t), Y k j t Z t / (Z k Y t)) may be displayed at the position of. Information indicating the coordinates of the scale points on the image derived as described above is output from the predetermined point deriving unit 35 to the output unit 36.

  Subsequently, in the output unit 36, the scale is superimposed on the image so that the scale is displayed in the information of the image corrected by the correction unit 32 at the coordinate position of the point derived by the predetermined point deriving unit 35. The information of the image is output to the monitor 40 (incorporated). On the monitor 40, an image related to the information output from the information processing apparatus 30 is displayed on the screen (S07). As described above, if an image on which the scale is superimposed is output, it is possible to surely display an image so that the size of the observed portion can be accurately understood. FIG. 10 shows an example displayed on the screen. As shown in FIG. 10, a line segment is drawn between each of the four irradiation points 71, and a scale 72 is displayed on the line segment. In the example shown in FIG. 10, one scale is 1 mm as an actual distance.

  Further, since the three-dimensional coordinates are obtained for each point, the distance from the imaging unit 11 (the lens 12) of each point can be displayed. In the example illustrated in FIG. 10, the distances from the imaging unit 11 (the lens 12 thereof) of the four irradiation points 71 are displayed.

  Further, the scale display may be performed based on the input when the output unit 36 receives an input from the user who is an operator to the imaging apparatus 1. As the input, for example, it is possible to give instructions such as “display the scale now”, “display the scale here”, and “change the type of the scale”. Further, it is preferable that the input is performed by an input interface such as a foot switch connected to the imaging device 1 so that a user who is an operator can perform manual operations such as operation of an endoscope.

  As described above, in the imaging device 1 according to the present embodiment, since the captured image is removed from the distortion caused by the lens 12, a more accurate shape can be displayed. Further, in the imaging apparatus 1 according to the present embodiment, a scale point indicating the actual size of the observed portion can be displayed between the laser light irradiation points in the image in which the distortion of the lens 12 is corrected. Thereby, according to the imaging device 1 which concerns on this embodiment, the magnitude | size of a to-be-observed part can be displayed more correctly.

  If the imaging unit 11 is configured to be included in the endoscope 10 as in the present embodiment, application to the medical field can be performed more easily. Due to the above-described effect of the present embodiment, for example, it becomes easy to grasp the size of the affected area that has been relied on the experience of a doctor so far, and quantitative observation becomes possible.

  In the above-described embodiment, the scale point is fixedly displayed at a preset position such as 1 mm, but the scale point may be moved. The scale points are moved as follows, for example. An input interface from a user such as a keyboard or a mouse is prepared in the information processing apparatus 30. When the user inputs information indicating that the scale is moved by the input interface, the predetermined point deriving unit 35 receives the input and moves the point that becomes the scale according to the input (the scale is changed according to the movement). The coordinates of the point to be derived again). For example, by preparing two scales between two irradiation points and aligning the two scales with the ends of the target object such as the affected part by the above movement, the size of the target object is adjusted. It is good to be able to measure. According to this configuration, for example, the scale point can be moved to a desired location, and the size of the observed portion can be recognized more easily.

  In the imaging apparatus 1 according to the present embodiment described above, the scale between the irradiation points of the laser light is displayed. However, the scale can be displayed at (one) irradiation point as described below. As shown in FIG. 11, when u is the interval between the scales to be displayed, it is sufficient to know how many pixels u correspond to on the image.

であるので、ｆが不明であっても、例えば、

を使って、
（画像面上での照射点までの距離）／（光軸から照射点までの距離）
の縮尺で目盛りを表示すればよい。 The line segment of the plane (z = Z _p ) that passes through the laser beam irradiation point P (X _p , Y _p , Z _p ) and is parallel to the image plane has a length of f / Z _p on the image plane (z = f). Reduced to double. Therefore, it is sufficient tick marks in uf / _{Z p} intervals. Also,

Therefore, even if f is unknown, for example,

Use
(Distance to irradiation point on image plane) / (Distance from optical axis to irradiation point)
The scale may be displayed at the scale of.

Moreover, even if the physical distance on the image is unknown, the scale can be displayed in units of pixels. At this time, the scale varies depending on the direction in which the scale is to be displayed. If you want to write the scale in the direction of θ in real space, x-axis component ucosθf _/ Z p _{d x} (pixels) may be displayed the scale at intervals of y-axis component usinθf _/ Z p _{d y} (pixels). Even if f / d _x or f / d _y is unknown, both the x-axis component and the y-axis component are
(Number of pixels to the irradiation point on the image plane) / (Number of pixels from the optical axis to the irradiation point)
The scale may be displayed at the scale of.

  FIG. 12 shows an example in which a scale is displayed around the irradiation point of the laser beam on the image. FIG. 12A shows an example displayed in two orthogonal directions, and FIG. 12B shows an example displayed concentrically.

  In the above-described embodiment, the endoscope 10 that images the inside of a living body such as a human body is used. However, the present invention can be applied to other types. For example, the present invention can be used for a defect detection device that images the inside of piping of a plant, a fluid device, or other production equipment. As described above, when the human eye cannot confirm from the outside or the accuracy is not sufficient even though it can be confirmed, the defect or the like can be detected directly and quantitatively with an image. As a result, it is possible to improve maintenance such as prevention and prediction against damage to facilities and the like, accidents, abnormal stops, and the like.

It is a figure which shows the structure of the imaging device which concerns on embodiment of this invention. It is the figure which showed the test pattern and the distortion which generate | occur | produced in the test pattern. It is a figure which shows the front of the front-end | tip part of an endoscope. It is the graph which showed the information which shows the distortion characteristic of a lens. It is the figure which showed the distance on the test pattern imaged when deriving the distortion characteristic of a lens, and a test pattern. It is a flowchart which shows the process in the imaging device in embodiment of this invention. It is the figure which showed the range which can become an irradiation point of the laser beam in the image imaged and the distortion correct | amended. It is the figure which showed the relationship between the optical center of a lens, the irradiation point of a laser beam, the irradiation point on an image, etc. It is the figure which showed the scale between irradiation points. It is a figure which shows the image in which the scale between irradiation points was put. It is the figure which showed the scale around the irradiation point. It is a figure which shows the image in which the scale around the irradiation point was put.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging device, 10 ... Endoscope, 11 ... Imaging part, 12 ... Lens, 13 ... Imaging element, 14 ... Connector, 15 ... Optical fiber, 20 ... Light source, 30 ... Information processing apparatus, 31 ... Reception part, 32 ... Correction unit, 33... Detection unit, 34... Three-dimensional coordinate deriving unit, 35.

Claims (5)

An image pickup means having a lens and picking up an image of the observed portion through the lens;
Laser light irradiation means for irradiating a plurality of points of the observed portion with laser light;
Correction means for correcting distortion caused by the lens of an image captured by the imaging means based on information indicating distortion characteristics of the lens stored in advance;
Detecting means for detecting coordinates of each irradiation point of the laser light irradiated by the laser light irradiation means in the image corrected by the correction means;
3D coordinate deriving means for deriving 3D coordinates of each irradiation point of the laser light irradiated by the laser light irradiation means from the coordinates detected by the detection means and the irradiation condition of the laser light;
Deriving the three-dimensional coordinates of the predetermined positional relationship between the irradiation points from the three-dimensional coordinates derived by the three-dimensional coordinate deriving means, and correcting by the correcting means from the derived three-dimensional coordinates Predetermined point deriving means for deriving the coordinates of the predetermined positional relationship point on the image,
Output means for outputting the information of the image corrected by the correcting means and the information indicating the coordinates of the points of the predetermined positional relationship on the image derived by the predetermined point deriving means;
An imaging apparatus comprising:   The output means superimposes a scale on the coordinates of the predetermined positional relationship on the image derived by the predetermined point deriving means, and outputs the superimposed image. Item 2. The imaging device according to Item 1.   The imaging device according to claim 1, wherein the detection unit detects the coordinates by searching for a predetermined range in the image corrected by the correction unit.   The predetermined point deriving unit receives an input of information indicating that the point having the predetermined positional relationship is to be moved, and moves the point having the predetermined positional relationship according to the input. 4. The imaging device according to any one of 3. The imaging apparatus according to claim 1, wherein the imaging unit is included in an endoscope.

Images