JP2011206435A - Imaging device, imaging method, imaging program and endoscope - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device, an imaging method, an imaging program and an endoscope, intuitively acquiring sense of distance in the direction of depth by simple configuration and processing.SOLUTION: A feature point P is extracted from an image picked up by cameras 41A, 41B by a feature point extract part 26, and a distance D in the direction of depth between an actual feature point position P' corresponding to the feature point P and the camera position Pc is measured by a distance measuring part 27. A wire frame configured by a plurality of circular arcs surrounding the feature point position P' disposed at equal spaces in the direction of depth and straight lines arranged at equal spaces and extends in the direction of depth based on the feature point position P' and the distance D by a wire frame construction part 28. A laser irradiation device 42 is controlled so that the constructed wire frame is drawn within an object range by a laser control part 29.

Description

  The present invention relates to an imaging device, an imaging method, an imaging program, and an endoscope, and in particular, an imaging device, an imaging method, an imaging program, and an endoscope that can grasp a sense of distance in the depth direction of a captured image. Regarding mirrors.

  Conventionally, a corresponding point detection unit that detects a corresponding point of a predetermined pixel from imaging signals obtained by a plurality of imaging units, and a three-dimensional position of the corresponding point based on information on the corresponding point obtained by the corresponding point detection unit. A three-dimensional position calculating means for calculating; a distance calculating means for calculating a distance between predetermined pixels based on the calculated three-dimensional position information; and a scale corresponding to the distance between the pixels based on the calculated distance information. A scale display device having scale display means for displaying has been proposed (see Patent Document 1). In the scale display device described in Patent Document 1, the distance in the direction perpendicular to the imaging plane is compared with a predetermined threshold value for the corresponding points, and the scale display method is changed according to the comparison result. .

  Also, an ultrasonic probe, a transmission / reception unit that drives the ultrasonic probe to scan a three-dimensional region in the subject with ultrasonic waves, and a target organ in the three-dimensional region based on a reception signal obtained by scanning by the transmission / reception unit Generating three-dimensional morphological image data representing the morphology of the object, generating three-dimensional functional image data representing the function of the target organ based on the received signal, and combining the three-dimensional functional image data with the three-dimensional functional image data An ultrasonic diagnostic apparatus including a signal processing unit that displays the image is proposed (see Patent Document 2). In the ultrasonic diagnostic apparatus described in Patent Document 2, three-dimensional morphological image data is displayed as a wire frame model of a target organ.

  In addition, a method and apparatus for creating a wireframe representation of a three-dimensional object at high speed using a general-purpose wireframe that is a typical prototype of a type to which an object belongs and can be modified to meet the specific requirements of that type has been proposed. (See Patent Document 3). In the method and apparatus described in Patent Document 3, a combination of general-purpose wire frames is collected as a tool kit together with software for editing the wire frame, from the original stereo rendered wire frame to the target stereo rendered wire frame. A method for morphing is disclosed, and the clearer camera viewpoint can be changed during morphing.

JP 2002-156212 A JP 2000-139917 A JP 2000-511658 A

  However, in the invention described in Patent Document 1, the display method of the scale is changed depending on whether the distance in the depth direction between the two points selected on the image is smaller than the threshold value. There is a problem that it is difficult to intuitively grasp the sense of distance to the position corresponding to the point.

  Moreover, in the invention described in Patent Document 2, an ultrasonic probe is required to obtain three-dimensional morphological image data, and a three-dimensional morphological image is displayed as a wire frame model. There is a problem that it is difficult to grasp the sense of position and distance.

  Further, in the invention described in Patent Document 3, since a general-purpose wire frame is used, a dedicated database and a device having a function for editing based on the database are required, and the configuration and processing become complicated. is there.

  The present invention has been made to solve the above-described problems, and an imaging apparatus, an imaging method, an imaging program, and an endoscope that can intuitively grasp the sense of distance in the depth direction with a simple configuration and processing. The purpose is to provide.

  In order to achieve the above object, an imaging apparatus according to a first aspect of the invention is extracted by an imaging unit that images a target range, an extraction unit that extracts a feature point from an image captured by the imaging unit, and the extraction unit. Based on the measurement unit that measures the distance in the depth direction between the position corresponding to the feature point and the imaging unit, the position corresponding to the feature point extracted by the extraction unit, and the distance measured by the measurement unit, Construction means for constructing a wire frame corresponding to the distance in the depth direction, rendering means for rendering a wire frame in the target range, and the wire frame constructed by the construction means is rendered in the target range Control means for controlling the drawing means.

  According to the imaging apparatus of the first invention, the imaging unit images the target range, the extraction unit extracts the feature point from the image captured by the imaging unit, and the measurement unit extracts the feature extracted by the extraction unit. The distance in the depth direction between the position corresponding to the point and the imaging means is measured. Then, the construction unit constructs a wire frame corresponding to the distance in the depth direction based on the position corresponding to the feature point extracted by the extraction unit and the distance measured by the measurement unit. Then, the control means controls the drawing means for drawing the wire frame in the target range so that the wire frame constructed by the construction means is drawn in the target range.

  Thus, since the wire frame corresponding to the depth direction is constructed and the wire frame is drawn in the target range, the sense of distance in the depth direction can be intuitively grasped with a simple configuration and processing.

  Further, the imaging apparatus of the first invention is a setting for setting a range for drawing the wire frame by the drawing unit based on the distance measured by the measuring unit and the angle of view of the image captured by the imaging unit. The control means can control the drawing means so that the wire frame is drawn in a range set by the setting means. Thereby, since a wire frame is drawn only in a necessary range, useless processing can be reduced.

  Further, in the first invention, the drawing means can be constituted by a laser irradiation apparatus using a movable laser light source or a laser irradiation apparatus constituted by a fixed laser light source and a movable mirror. As described above, when the laser irradiation apparatus is used as the drawing unit, the configuration of the focus system is unnecessary, and therefore, the wire frame can be appropriately drawn with a simple configuration.

  The imaging device of the second invention corresponds to the imaging means for imaging the target range, the extracting means for extracting feature points from the image captured by the imaging means, and the feature points extracted by the extracting means. Measuring means for measuring the distance between the position and the imaging means; and a wire corresponding to the distance in the depth direction based on the position corresponding to the feature point extracted by the extracting means and the distance measured by the measuring means A construction means for constructing a frame, and a wire frame constructed by the construction means are converted into an image projected on a plane perpendicular to the imaging direction of the imaging means, and the converted image and the imaging means are captured. Synthesizing means for synthesizing the images.

  According to the imaging apparatus of the second invention, the imaging unit images the target range, the extraction unit extracts the feature point from the image captured by the imaging unit, and the measurement unit extracts the feature extracted by the extraction unit. The distance between the position corresponding to the point and the imaging means is measured. Then, the construction unit constructs a wire frame corresponding to the distance in the depth direction based on the position corresponding to the feature point extracted by the extraction unit and the distance measured by the measurement unit. The synthesizing unit converts the wire frame constructed by the constructing unit into an image projected on a plane perpendicular to the imaging direction of the imaging unit, and synthesizes the converted image and the image captured by the imaging unit. To do.

  Thus, since the wire frame corresponding to the depth direction is constructed and the wire frame is combined with the captured image, the sense of distance in the depth direction can be intuitively grasped with a simple configuration and processing.

  Further, the wire frame extends in the depth direction arranged at equal intervals in the horizontal direction with reference to the feature points and a plurality of arc-shaped lines surrounding the feature points arranged at equal intervals in the depth direction. It can be constituted by a straight line. Thereby, it becomes easy to intuitively grasp the sense of distance in the depth direction.

  According to a third aspect of the present invention, there is provided an imaging method of acquiring an image captured by an imaging unit that captures a target range, extracting a feature point from the acquired image, a position corresponding to the extracted feature point, and the imaging Measure the distance to the means, and construct a wire frame corresponding to the distance in the depth direction from the imaging means to the position corresponding to the feature point based on the position corresponding to the extracted feature point and the measured distance Then, the drawing means for drawing the wire frame in the target range is controlled so that the constructed wire frame is drawn in the target range.

  According to a fourth aspect of the present invention, there is provided an imaging method of acquiring an image captured by an imaging unit that captures a target range, extracting a feature point from the acquired image, a position corresponding to the extracted feature point, and the imaging Measure the distance to the means, and construct a wire frame corresponding to the distance in the depth direction from the imaging means to the position corresponding to the feature point based on the position corresponding to the extracted feature point and the measured distance Then, the constructed wire frame is converted into an image projected on a plane perpendicular to the imaging direction of the imaging unit, and the converted image and the image captured by the imaging unit are synthesized.

  According to a fifth aspect of the present invention, there is provided an imaging program for causing a computer to extract a feature point from an image captured by an imaging unit that captures a target range, a position corresponding to the feature point extracted by the extraction unit, and the Measuring means for measuring a distance in the depth direction with the imaging means, a position corresponding to the feature point extracted by the extracting means, and a wire frame corresponding to the distance in the depth direction based on the distance measured by the measuring means And a program for functioning as control means for controlling the drawing means for drawing the wire frame in the target range so that the wire frame constructed by the construction means is drawn in the target range. is there.

  According to a sixth aspect of the present invention, there is provided an imaging program for extracting a feature point from an image captured by an imaging unit that images a target range, a position corresponding to the feature point extracted by the extraction unit, and the position Measuring means for measuring a distance in the depth direction with the imaging means, a position corresponding to the feature point extracted by the extracting means, and a wire frame corresponding to the distance in the depth direction based on the distance measured by the measuring means And a wire frame constructed by the construction unit is converted into an image projected on a plane perpendicular to the imaging direction of the imaging unit, and the converted image and the imaging unit are captured. This is a program for functioning as a combining means for combining with an image.

  An endoscope according to a seventh aspect of the invention includes the imaging device according to the first and second aspects of the invention. The endoscope according to a seventh aspect of the present invention is the endoscope, wherein the imaging unit includes the first imaging unit that captures the target range from a first viewpoint and the target from a second viewpoint different from the first viewpoint. It can comprise with the 2nd imaging means which images a range. In an endoscope that performs stereoscopic viewing using images from two different viewpoints, there is a problem that it is difficult to grasp the sense of distance in the depth direction on the image, and therefore it is highly meaningful to apply the present invention.

  According to the present invention, a wire frame corresponding to the depth direction is constructed, and the wire frame is drawn in the target range or synthesized with an image. Therefore, the sense of distance in the depth direction can be intuitively achieved with a simple configuration and processing. The effect that it can grasp is acquired.

It is the schematic which shows the external appearance of the stereoscopic endoscope of this Embodiment. It is (A) side view and (B) front view which show schematic structure of an imaging part. It is a block diagram showing a schematic structure of a stereoscopic endoscope of a 1st embodiment. It is the schematic for demonstrating the coordinate system set to a target range. It is a flowchart which shows the content of the imaging process routine in 1st Embodiment. It is an image figure for demonstrating extraction of a feature point. It is a flowchart which shows the content of the wire frame construction process routine in 1st Embodiment. It is the schematic for demonstrating formation of the arc-shaped line | wire which comprises a wire frame. It is the schematic for demonstrating formation of the straight line which comprises a wire frame. It is an image figure for demonstrating the description range on an image. It is a figure which shows an example of a distance-drawing range database. It is the schematic for demonstrating the relationship between the drawing range hx, hy on an image, and the drawing range Hx, Hy in a target range. It is the schematic for demonstrating the relationship between drawing range Hx, Hy and the plane S. FIG. It is a block diagram which shows schematic structure of the stereoscopic endoscope of 2nd Embodiment. It is a flowchart which shows the content of the imaging process routine in 2nd Embodiment. It is the schematic for demonstrating Z-axis view conversion of the wire frame on the plane S. FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a case where the imaging device of the present invention is applied to a stereoscopic endoscope will be described.

  As shown in FIG. 1, the stereoscopic endoscope 10 according to the first embodiment includes a long insertion portion 12 that is inserted into a body cavity of a patient, and a proximal end portion of the insertion portion 12. The main body operation unit 14 is continuously provided. A long light guide 16 that is detachably connected to the control unit 20 is coupled to the main body operation unit 14. In addition, the main body operation unit 14 is provided with an operation knob 18 for operating the insertion unit 12. The control unit 20 is connected with a display 50 for performing various displays and a control operation unit 60 for performing various input operations.

  In addition, an imaging unit 40 that images the target range is provided at the distal end of the insertion unit 12. 2A is a schematic side view of the imaging unit 40, and FIG. 2B is a schematic front view. As shown in FIG. 2, the imaging unit 40 includes a plurality of (here, two) cameras 41A and 41B for imaging a common target range, and a movable laser that draws a wire frame, which will be described later, in the target range. A laser irradiation device 42 using a light source, an illumination device 43 that illuminates the target range, camera drive units 44A and 44B that drive the camera 41A and the camera 41B according to the control of the control unit 20, and a control of the control unit 20 And a laser driving unit 45 that drives the laser irradiation device 42. The camera 41A and the camera 41B include an image sensor 46A and an image sensor 46B, an imaging optical system 47A that forms a subject image on the image pickup area of the image sensor 46A, and a subject image formed on the image pickup area of the image sensor 46B. An image optical system 47B. For example, a CCD area sensor or a CMOS image sensor can be used for the imaging elements 46A and 46B. Further, the imaging optical systems 47A and 47B may each be composed of, for example, only a single imaging lens, or in addition to the imaging lens, a focus adjustment mechanism, a zooming mechanism, and the like are provided. A configuration having functions such as focus adjustment and zooming may be employed.

  FIG. 3 is a schematic block diagram showing the configuration of the stereoscopic endoscope 10. As shown in FIG. 3, the control unit 20 of the stereoscopic endoscope 10 includes an imaging control unit 21, an image processing unit 22, an image storage unit 23, an internal memory 24, a display control unit 25, a feature point extraction unit 26, a distance. A measurement unit 27, a wire frame construction unit 28, a laser control unit 29, and a CPU 30 are provided.

  When imaging is instructed by operating the control operation unit 60, the imaging control unit 21 captures the target range with the cameras 41A and 41B and outputs the image data so that the camera driving units 44A and 44B (FIG. 3). , The instruction signal is output.

  The image processing unit 22 performs image processing such as analog / digital conversion processing, white balance adjustment, gradation correction, sharpness correction, and color correction on the image data of the left image and the right image output from the cameras 41A and 41B. Apply. In addition, the image processing unit 22 performs compression processing on the image data representing the left image and the right image subjected to predetermined image processing in a compression format such as JPEG, and generates a stereoscopic image file. To do. Further, in order to stereoscopically display the left image and the right image on the display 50, a stereoscopic image is generated by performing a three-dimensional process on the stereoscopic image file.

  The image storage unit 23 stores image data representing a right image and a left image that have been subjected to predetermined image processing by the image processing unit 22, and image data such as an image file for stereoscopic viewing.

  The internal memory 24 stores various constants set in the stereoscopic endoscope 10 and programs executed by the CPU 30.

  The display control unit 25 performs control so that a stereoscopic image generated from the left image and the right image stored in the image storage unit 23 is displayed on the display 50.

  The feature point extraction unit 26 extracts a feature point P from the left image, the right image, or the stereoscopic image. For example, a pixel that matches a predetermined pattern, a pixel value that satisfies a predetermined condition, or the like is extracted so that an affected part that is a target of diagnosis and treatment is extracted as a feature point. Further, the feature point set by the user confirming the image displayed on the display 50 and operating the control operation unit 60 may be extracted.

  The distance measuring unit 27 performs a stereo measurement using the left image and the right image, and a midpoint between the center of the camera 41A and the center of the camera 41B (hereinafter referred to as a camera position Pc) and a feature point P extracted from the image. The distance to the actual position corresponding to (hereinafter referred to as the feature point position P ′) is measured. In this embodiment, the case of measuring the distance by stereo measurement will be described. However, in the case of a monocular imaging apparatus, the distance to the feature point may be measured using the focal length.

  As shown in FIG. 4, the wire frame construction unit 28 has a depth direction in a coordinate system in which the horizontal direction set as the target range is the X axis, the vertical direction is the Y axis, and the imaging direction passing through the center of the camera is the Z axis. Build a wireframe corresponding to the distance. Specifically, in the plane S determined by the X axis and the feature point position P ′, a plurality of arc-shaped lines surrounding the feature point positions P ′ arranged at equal intervals in the Z axis direction, the same plane S, etc. A wire frame composed of straight lines parallel to the straight lines Pc-P ′ arranged at intervals is constructed.

  In FIG. 4, only one camera is shown in a simplified manner, and the imaging direction of the camera is described as the Z axis. However, in practice, a straight line indicating the imaging direction of the camera 41A passing through the center of the camera 41A and the camera 41B are shown. The straight line passing through the intersection of the camera 41B passing through the center and the straight line passing through the camera position Pc is defined as the Z axis, and the straight lines passing through the center of the camera 41A and the center of the camera 41B are defined as the X axis, the X axis, The direction orthogonal to is the Y axis. When the imaging direction of the camera 41A and the imaging direction of the camera 41B are parallel, a straight line passing through the camera position Pc and parallel to the imaging direction may be set as the Z axis.

  The laser control unit 29 outputs a control signal to the laser driving unit 45 (not shown in FIG. 3) so that the wire frame constructed by the wire frame construction unit 28 is drawn in the target range by the laser irradiation device 42.

  Next, an imaging processing routine executed in the stereoscopic endoscope 10 according to the first embodiment will be described with reference to FIG.

  In step 100, image data indicating the left image and the right image output from the cameras 41A and 41B is acquired, and predetermined image processing and three-dimensional processing are performed to display a stereoscopic image on the display 50.

  Next, in step 102, a pixel that matches a predetermined pattern or a pixel value that satisfies a predetermined condition is extracted as a feature point, or information input from the control operation unit 60 is acquired, and It is determined from the image displayed on the display 50 whether or not a feature point indicating the affected part position has been set. If a feature point is extracted or set, the process proceeds to step 104. If a feature point is not extracted or set, the process stands by until it is extracted or set.

  In step 104, the feature point extracted or set in step 102 is used as the feature point P, and the distance D in the depth direction between the camera position Pc and the feature point position P ′ is measured by stereo measurement.

  Next, in step 106, it is determined whether or not the distance D is greater than a predetermined threshold value Dth. The threshold value Dth is determined in advance so that it is not necessary to draw a wire frame for easily understanding the distance in the depth direction. If D> Dth, the process proceeds to step 108. If D ≦ Dth, the process ends.

  In step 108, two feature points A and B are extracted from the vicinity of the feature point P. The feature points A and B are for specifying the range of the affected area on the image. For example, as shown in FIG. 6, when the apex of the affected part on the hump is extracted or set as the feature point P, the skirt part of the affected part is extracted as the feature points A and B. The feature points A and B can be extracted by, for example, edge extraction processing.

  Next, in step 110, it is determined whether or not the feature points A and B are extracted in step 108. In some cases, the image is unclear, or depending on the shape of the affected part itself, appropriate feature points A and B cannot be extracted. If the feature points A and B cannot be extracted, the process proceeds to step 112. If extracted, the process proceeds to step 114.

  In step 112, information input from the control operation unit 60 is acquired, and it is determined whether or not the feature points A and B are set from the image displayed on the display 50 by the user. When the feature points A and B are set, the process proceeds to step 114, and when the feature points A and B are not set, the standby state is set until they are set.

  In step 114, a wire frame construction process is executed. Here, a wire frame construction processing routine executed in the stereoscopic endoscope 10 according to the first embodiment will be described with reference to FIG.

  In step 150, the plane S determined by the X axis and the feature point position P ′ is set, and the intersection of the plane S with the straight line passing through the position corresponding to the camera position Pc and the feature point A is the feature point position A ′ and the camera position. An intersection of a straight line passing through the positions corresponding to Pc and the feature point B and the plane S is set as the feature point position B ′. Then, as shown in FIG. 8, the radius r of the circular arc passing through the feature point positions P ′, A ′ and B ′ and whose center is on the straight line Pc-P ′ is calculated.

Next, in step 152, the arc C ₀ whose center of the radius r × t passing through the feature point positions A ′ and B ′ is on the straight line Pc-P ′ is surrounded on the plane S so as to surround the feature point position P ′. Set. t is a value larger than 1. Further, it may be changed according to the distance D.

Next, in step 154, it is determined whether or not the number n of drawn arcs is set. If it is not set, it will be in a standby state until it is set. A predetermined value may be set as an initial value. If the number n of arcs to be drawn is set, the process proceeds to step 156, and arcs C _{1 to} C _n−1 having a radius r × t parallel to the arc C ₀ are set on the plane S at intervals of D / n. Set.

Next, in step 158, as shown in FIG. 9, a straight line L ₀ (= straight line Pc−P ′) passing through the feature point position P ′ and the camera position Pc is set on the plane S.

  Next, in step 160, it is determined whether or not the straight line drawing interval Δx is set. If it is not set, it will be in a standby state until it is set. A predetermined value may be set as an initial value. If the straight line drawing interval Δx is set, the process proceeds to step 162.

In step 162, on a plane S, the straight line _{L 0} parallel straight line _L -m ~L _m set at Δx intervals. For m, an arbitrary positive number is set in advance according to the target range for drawing the wire frame and the value of Δx. Thereby, a wire frame composed of a plurality of arcs surrounding the feature point positions P ′ arranged at equal intervals in the depth direction and straight lines extending in the depth direction arranged at equal intervals with respect to the X-axis direction, It is constructed on the plane S.

  Next, returning to step 116 in FIG. 5, a drawing range for drawing the wire frame is set. For example, as shown in FIG. 12, the drawing range refers to the range hx, hy on the image including the feature point P as shown in FIG. 10 with reference to the distance-drawing range database as shown in FIG. Based on the relationship between the focal length of the camera and the angle of view and the distance D, ranges Hx and Hy corresponding to the X-axis direction and the Y-axis direction are set.

  Next, in step 118, the drawing ranges Hx and Hy set in step 116 are made to correspond to the plane S. Specifically, as shown in FIG. 13, the range S ′ of the plane S included in the region surrounded by the XY planes of the ranges Hx and Hy including the camera position Pc and the feature point position P ′ (indicated by diagonal lines in FIG. 13) Is a range on the plane S corresponding to the drawing ranges Hx and Hy. Then, control is performed so that the wire frame constructed in step 114 is drawn by the laser irradiation device 42 in the range S ′ of the plane S. Laser irradiation is performed from a position behind the camera position Pc with respect to the feature point position P 'because the Y coordinate is larger than the feature point position P'. Thereby, as shown in FIG. 10, the target range in a state where the wire frame is drawn in the drawing range is imaged by the camera.

  As shown in FIGS. 8 and 9, since the arcs constituting the wire frame are arranged at equal intervals, the Y-axis view appears to be equal intervals, but when viewed in the Z-axis view (imaging direction), That is, the intervals are unequal on the image. In addition, since the straight lines constituting the wire frame are arranged at equal intervals, they appear to be equally spaced in the Y-axis view, but when viewed in the Z-axis view, that is, on the image, the front side is closer to the depth side in the depth direction. Open arrangement.

  As described above, according to the stereoscopic endoscope of the first embodiment, the plurality of arcs surrounding the feature point positions P ′ arranged at equal intervals in the depth direction and the straight line extending in the depth direction. Since the constructed wire frame is constructed, and the constructed wire frame is drawn by the laser irradiation apparatus, an image in which the wire frame indicating the depth direction interval is drawn is acquired, and the simple configuration and processing are performed. A sense of distance in the depth direction can be intuitively grasped.

  In the first embodiment, the case where the wire frame is drawn by the laser irradiation device using the movable laser light source has been described. However, the laser irradiation device including the fixed laser light source and the movable mirror is used. It may be used. Further, the constructed wire frame may correspond to the liquid crystal panel on the XY plane, and the light transmitted through the backlight applied to the liquid crystal panel may be projected onto the target range. In addition, a wire frame is made to correspond to a DMD (Digital Micromirror Device) in which a large number of individually driven micromirror surfaces (micromirrors) are arranged in a plane, and a light source is irradiated to the DMD. The reflected light may be projected onto the target range. In the case of such a transmission type or reflection type irradiation device, the configuration of the focus system is also used so that the wire frame is appropriately drawn in the target range.

  In the first embodiment, the case where the drawing range is set after the wire frame is constructed and the wire frame is drawn in the set drawing range has been described. However, the drawing range is set and set first. You may make it construct | assemble a wire frame in the set range.

  Next, a second embodiment will be described. In addition, about the structure of the stereoscopic endoscope 210 of 2nd Embodiment, about the structure same as the stereoscopic endoscope 10 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted. .

  FIG. 14 is a schematic block diagram showing the configuration of the stereoscopic endoscope 210. As shown in FIG. As shown in FIG. 14, the control unit 220 of the stereoscopic endoscope 210 includes an imaging control unit 21, an image processing unit 22, an image storage unit 23, an internal memory 24, a display control unit 25, a feature point extraction unit 26, a distance. A measurement unit 27, a wire frame construction unit 28, an image composition unit 31, and a CPU 30 are provided.

  The image composition unit 31 synthesizes an image representing the wire frame constructed by the wire frame construction unit 28 with the images captured by the cameras 41A and 41B.

  Next, an imaging processing routine executed in the stereoscopic endoscope 210 according to the second embodiment will be described with reference to FIG. Note that the same processing as that of the imaging processing routine of the first embodiment is denoted by the same reference numeral, and description thereof is omitted.

Through steps 100 to 114, in step 200, the constructed wire frame is projected onto the XY plane including the feature point position P ′ so that the Z-axis view from the same Y coordinate position as the feature point position P ′ is obtained. As a result, the Z-axis view is converted. For example, as shown in FIG. 16, regarding the intersections c _{1 to} c ₃ between the straight line PC-P ′ and the arcs C _{1 to} C ₃ , c _{1 to} c ₃ are c ₁ ′ to c ₃ on the XY plane, respectively. 'Projected to. By converting to the Z-axis view in this way, arcs arranged at equal intervals on the plane S become unequal intervals.

Next, in step 202, the wire frame projected on the XY plane is combined with the captured image. For example, as shown in FIG. 16, points c ₁ ′ to c ₃ ′ on the XY plane correspond to CI ₁ to CI ₃ on the image, respectively. In this way, a point on the image corresponding to each point on the wire frame is obtained, a wire frame image is generated, alignment is performed based on the feature point P and the like, and the captured image is combined.

  As described above, according to the stereoscopic endoscope of the second embodiment, the plurality of arcs surrounding the feature point positions P ′ arranged at equal intervals in the depth direction and the straight line extending in the depth direction. Since the constructed wire frame is constructed and the constructed wire frame is combined with the captured image, an image on which the wire frame indicating the depth direction interval is drawn is displayed. You can intuitively grasp the sense of distance in the direction.

DESCRIPTION OF SYMBOLS 10,210 Stereoscopic endoscope 20,220 Control part 26 Feature point extraction part 27 Distance measurement part 28 Wire frame construction part 29 Laser control part 31 Image composition part 40 Imaging part 41A Camera 41B Camera 42 Laser irradiation apparatus

Claims (11)

Imaging means for imaging a target range;
Extraction means for extracting feature points from the image captured by the imaging means;
A measuring means for measuring a distance in a depth direction between the position corresponding to the feature point extracted by the extracting means and the imaging means;
Construction means for constructing a wire frame corresponding to the distance in the depth direction based on the position corresponding to the feature point extracted by the extraction means and the distance measured by the measurement means;
Drawing means for drawing a wire frame in the target range;
Control means for controlling the drawing means so that the wire frame constructed by the construction means is drawn in the target range;
An imaging apparatus including: Setting means for setting a range for drawing the wire frame by the drawing means based on the distance measured by the measuring means and the angle of view of the image taken by the imaging means;
The imaging apparatus according to claim 1, wherein the control unit controls the drawing unit such that the wire frame is drawn in a range set by the setting unit.   The imaging apparatus according to claim 1, wherein the drawing unit is configured by a laser irradiation apparatus using a movable laser light source, or a laser irradiation apparatus including a fixed laser light source and a movable mirror. Imaging means for imaging a target range;
Extraction means for extracting feature points from the image captured by the imaging means;
Measuring means for measuring a distance between a position corresponding to the feature point extracted by the extracting means and the imaging means;
Construction means for constructing a wire frame corresponding to the distance in the depth direction based on the position corresponding to the feature point extracted by the extraction means and the distance measured by the measurement means;
Combining means for converting the wire frame constructed by the constructing means into an image projected on a plane perpendicular to the imaging direction of the imaging means and synthesizing the converted image and the image taken by the imaging means When,
An imaging apparatus including:   A plurality of arc-shaped lines surrounding the feature points arranged at equal intervals in the depth direction; and straight lines extending in the depth direction arranged at equal intervals in the horizontal direction with reference to the feature points. The imaging device according to claim 1, which is configured by: Obtain an image captured by an imaging means that captures the target range,
Extract feature points from the acquired image,
Measure the distance between the position corresponding to the extracted feature point and the imaging means,
Based on the position corresponding to the extracted feature point and the measured distance, a wire frame corresponding to the distance in the depth direction from the imaging unit to the position corresponding to the feature point is constructed,
An imaging method for controlling drawing means for drawing a wire frame in the target range so that the constructed wire frame is drawn in the target range. Obtain an image captured by an imaging means that captures the target range,
Extract feature points from the acquired image,
Measure the distance between the position corresponding to the extracted feature point and the imaging means,
Based on the position corresponding to the extracted feature point and the measured distance, a wire frame corresponding to the distance in the depth direction from the imaging unit to the position corresponding to the feature point is constructed,
An imaging method in which the constructed wire frame is converted into an image projected on a plane perpendicular to the imaging direction of the imaging unit, and the converted image and the image captured by the imaging unit are combined. Computer
Extraction means for extracting feature points from the image captured by the imaging means for capturing the target range;
Measuring means for measuring a distance in a depth direction between the position corresponding to the feature point extracted by the extracting means and the imaging means;
Based on the position corresponding to the feature point extracted by the extraction means and the distance measured by the measurement means, the construction means for constructing a wire frame corresponding to the distance in the depth direction, and the construction means An imaging program for functioning as a control unit that controls a drawing unit that draws a wire frame in the target range such that a wire frame is drawn in the target range. Computer
Extraction means for extracting feature points from the image captured by the imaging means for capturing the target range;
Measuring means for measuring a distance in a depth direction between the position corresponding to the feature point extracted by the extracting means and the imaging means;
Based on the position corresponding to the feature point extracted by the extraction means and the distance measured by the measurement means, the construction means for constructing a wire frame corresponding to the distance in the depth direction, and the construction means An imaging program for converting a wire frame into an image projected on a plane perpendicular to the imaging direction of the imaging unit, and causing the converted image and an image captured by the imaging unit to function as a combining unit .   An endoscope comprising the imaging device according to any one of claims 1 to 5.   The imaging unit includes a first imaging unit that images the target range from a first viewpoint, and a second imaging unit that images the target range from a second viewpoint different from the first viewpoint. The endoscope according to claim 10.

Images