JP2005013557A - Endoscope apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an endoscope apparatus capable of improving an identification ratio when identifying a pixel showing a marker among all the pixels of a wide-range image in a case of displaying an area extracted from the wide-range image as a magnified image and making the extracted area follow the position of the marker in the wide-range image. <P>SOLUTION: This endoscope apparatus is so formed that an image separation device 20 of an operation support system is provided with a first image reformation system 23 re-forming the wide-range image by an objective optical system 12 in a rigid scope 10 on an image picked up by a first camera C1, and a second image reformation optical system 25 re-forming a part of the wide-range image by the objective optical system 12 on the image picked by a second camera C2. A marker part 44 including a retroreflection material is disposed near the tip of an insertion part 42 of a treatment device 40. A control device 40 permanently stations the marker part 44 in the center of a magnified image projected on a first monitor M1 based on the position of high-luminance pixel group, which is extracted from all the pixels of first image data from the first camera C1. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an endoscope apparatus for allowing an observer to observe an arbitrary region of an image formed by an objective optical system in an endoscope.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, an endoscope apparatus that displays a wide-angle image formed by an objective optical system in an endoscope on one display device, and enlarges a part of the wide-angle image and displays it on the other display device. Has been proposed. One of the endoscope apparatuses of this type is an endoscope apparatus described in Patent Document 1.
[0003]
The endoscope apparatus described in Patent Document 1 includes a magnifying optical system that enlarges a part of a wide-angle image formed by an objective optical system in the endoscope, and further, a wide-angle image formed by the objective optical system. And a second imaging device that captures an enlarged image formed by the magnifying optical system. Among these, the second imaging device is selected from the wide-angle images displayed on the display device for wide-angle display because it is held so as to be movable in a plane orthogonal to the optical axis of the magnifying optical system. A part is enlarged and displayed on the display device for enlargement display.
[0004]
Furthermore, the endoscope apparatus described in Patent Document 1 specifies the position of the green or yellow pixel from the pixels in the wide-angle image and sets the second imaging device so that the position is located at the center of the enlarged image. A follow-up control mechanism to be moved is provided. For this reason, an operator who performs an operation on a subject using an endoscope uses a forceps provided with a marker that reflects green or yellow light at the tip, so that even if the forceps are moved during the operation, an enlarged display is provided. The tip of the forceps can be made resident at the center of the screen of the display device.
[0005]
[Patent Document 1]
JP 08-332169 (paragraph 0016, paragraph 0035)
[0006]
[Problems to be solved by the invention]
However, in the endoscope apparatus described in Patent Document 1 described above, since the illumination light with high intensity cannot be emitted from the distal end of the endoscope due to prevention of adverse effects on the body or the like, the imaging element is also passed through a large number of optical systems. Since the intensity of the light reaching the light source deteriorates, the brightness of the wide-angle image tends to be low overall. If the brightness of the wide-angle image is low in this way, there arises a problem that pixels that show green or yellow light reflected on the surface of the marker cannot be identified from all the pixels of the wide-angle image.
[0007]
In particular, in the irradiation range of illumination light emitted from the distal end of the endoscope to the subject, the intensity decreases when the distance from the center decreases. Therefore, if the distal end of the forceps is positioned at the peripheral portion of the irradiation range, a wide-angle image is obtained. Pixels indicating markers at the inner peripheral edge are more difficult to be identified from among all the pixels.
[0008]
The present invention has been made in view of the above-described problems of the prior art, and the problem is that a region extracted from a wide-angle image is displayed as an enlarged image. To provide an endoscope apparatus capable of improving the identification rate when identifying a pixel indicating a marker among all pixels of a wide-angle image when following the position of the marker in the wide-angle image. .
[0009]
[Means for Solving the Problems]
In order to solve the above problems, an endoscope apparatus according to the present invention employs the following configuration.
[0010]
That is, the endoscope apparatus according to the present invention can be inserted into the body of a subject and a treatment tool having an elongated insertion portion in which a marker that includes a retroreflective material and exhibits a predetermined characteristic is disposed near the tip thereof An endoscope having an objective optical system in the insertion portion that has an elongated insertion portion having a thickness and forms an image of a subject facing the distal end on the proximal end side, and an image formed by the objective optical system The first image data is generated by imaging the first image data and the first image data is output in the form of a signal, and at least a part of the image formed by the objective optical system is imaged. The second imaging device that generates two image data and outputs the second image data in the form of a signal, and the optical axis of the objective optical system and the second imaging device are relatively shifted. A shift device, a feature point specifying unit for specifying the coordinates of the pixel group having the predetermined feature from all the coordinates of the coordinate system defined for all the pixels in the first image data, and the feature point specifying A control unit that positions the pixel group having the predetermined characteristic at substantially the center of all the pixels in the second image data by controlling the shift device based on the coordinates specified by the unit. It is a feature.
[0011]
With this configuration, when the distal end of the treatment instrument enters the field of view of the objective optical system when the front end of the insertion portion of the endoscope is illuminated, the first imaging device is 1st image data which includes the pixel which has the predetermined characteristic exhibited by the marker near the front-end | tip of the insertion part in all the pixels is output. Then, the coordinates of a pixel having a predetermined feature are extracted from all the coordinates of the coordinate system defined for all the pixels of the first image data, the shift device is controlled based on the extracted coordinates, and the second A pixel group having a predetermined characteristic is located at substantially the center of all the pixels in the image data. For this reason, when the second image data is output to the monitor, the marker and the enlarged image of the periphery thereof are displayed on the monitor with the marker resident at the center. At this time, although all the pixels of the first image data have a low luminance as a whole, the marker includes a retroreflective material, so that the pixels of the first image data are reflected with extremely high luminance without scattering the illumination light. Pixels for displaying the marker being displayed are included. As a result, when identifying pixels with extremely high brightness from pixels with overall low brightness, compared to the conventional identification rate when identifying pixels with similar brightness from low brightness pixels as in the prior art. Since the identification rate is high, the identification rate when identifying the pixel indicating the marker is improved as compared with the conventional one.
[0012]
In the endoscope apparatus according to the present invention, the predetermined feature exhibited by the marker may be high-intensity reflection of white light in the retroreflective material, or only a predetermined wavelength band in the retroreflective material. The reflection may be high-brightness reflection of light, the shape of a marker, or the number of markers.
[0013]
In the endoscope apparatus according to the present invention, the endoscope may be a flexible endoscope whose insertion portion is soft or a rigid endoscope whose insertion portion is rigid. In the case of a soft mirror like the former, the treatment tool may be used by being inserted into a forceps channel provided in the soft mirror. In the case of a rigid endoscope like the latter, the endoscope may be inserted into a hole different from the hole for inserting the insertion portion of the endoscope into the body of the subject.
[0014]
In the endoscope apparatus according to the present invention, the endoscope needs to include an illumination optical system for guiding the illumination light supplied from the light source device to the tip thereof.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an operation support system according to an embodiment of the present invention will be described with reference to the drawings. Note that the operation support system described below is a system used when an operator performs a surgical operation on an internal organ of a subject.
[0016]
Embodiment 1
<Overall configuration of surgery support system>
First, the overall configuration of the surgery support system will be described. FIG. 1 is a block diagram schematically showing this surgery support system. As shown in FIG. 1, the surgery support system includes a first monitor M1, a second monitor M2, a first camera C1, a second camera C2, a rigid endoscope 10, an image separation device 20, a control device 30, and a light source device LS. , A light guide LG, and a treatment instrument 40.
[0017]
The first and second monitors M1 and M2 are display devices that display an image based on the image data when the image data is input in the form of a signal, specifically, a cathode ray tube display or a liquid crystal display.
[0018]
The first and second cameras C1 and C2 are imaging devices that use a general moving image shooting area image sensor. The first and second cameras C1 and C2 generate image data by photoelectrically converting a light beam incident on the imaging surface into an electric signal, and output the image data.
[0019]
Since the first camera C1 is connected to the control device 30 described later, the image data generated by the first camera C1 is input to the control device 30. Further, since the second camera C2 is directly connected to the second monitor M2, the image data generated by the second camera C2 is input to the second monitor M2.
[0020]
The rigid endoscope 10 is an instrument for illuminating a subject's body and forming an image of the illuminated body, and the outer shape thereof is formed in an elongated, substantially rod-like shape so that it can be inserted into the subject's body. . The configuration of the rigid endoscope 10 will be described in detail later. In addition, as FIG. 1 shows, the base end vicinity of the rigid endoscope 10 is attached to the front-end | tip of the arm A extended from the indoor floor surface or wall surface. For this reason, the posture of the rigid endoscope 10 is maintained in the space without being held by the operator.
[0021]
The image separation device 20 is a device for forming an in-vivo image formed by the rigid endoscope 10 on the imaging surface of the first camera C and the imaging surface of the second camera C2. The configuration of the image separation device 20 will also be described in detail later. The image separation device 20 is integrated with the rigid endoscope 10 by being fixed to the proximal end side of the rigid endoscope 10, and its posture is determined by the above-described arm A connected via the rigid endoscope 10. The space is maintained in the space together with the rigid endoscope 10.
[0022]
The control device 30 is a device that acquires image data from the first camera C1 and outputs the image data to the first monitor M1. That is, the control device 30 relays image data between the first camera C1 and the first monitor M1. The configuration of the control device 30 will also be described in detail later.
[0023]
The light source device LS is a device that outputs illumination light for illuminating the front end of the rigid endoscope 10, and although not shown, a lamp that emits illumination light, a condenser, a light source drive circuit for adjusting the amount of light, etc. It has.
[0024]
The light guide LG is an instrument for guiding the illumination light output from the light source device LS to the rigid endoscope 10. The light guide LG includes a number of bundled optical fibers and a flexible tube that covers the entire optical fibers, and further includes connectors at both ends, although not shown. One connector is detachably attached to a connector receiver formed in the light source device LS, and illumination light condensed by a condenser in the light source device LS is attached to one end surface of the light guide LG. Is introduced. The other connector is detachably attached to a base formed near the proximal end of the rigid endoscope 10.
[0025]
The treatment tool 40 is an instrument used when an operator performs a surgical operation on an internal organ of a subject. The treatment tool 40 will also be described in detail later.
[0026]
<Rigid endoscope>
Next, the rigid endoscope 10 will be described in detail. FIG. 2 is a configuration diagram schematically showing the rigid endoscope 10 and the image separation device 20. The rigid mirror 10 includes a large number of optical fibers 11, an objective optical system 12, and two metallic pipes having different diameters as main components.
[0027]
A large number of optical fibers 11 are arranged so as to be in close contact with the inner surface of the large-diameter pipe that functions as an outer frame of the rigid endoscope 10, and thereby are arranged in a cylindrical shape as a whole. Further, a small diameter pipe is inserted into the large diameter pipe so as to be inscribed in the optical fibers 11 arranged in a cylindrical shape. For this reason, one thin tube 10a is formed as a whole by sandwiching the optical fiber 11 between two large and small pipes.
[0028]
In addition, the front-end | tip of each optical fiber 11 is being fixed to the front-end | tip part of the said thin tube 10a in the state in which a mutual front-end | tip surface is contained in the same surface. On the other hand, the base end of each optical fiber 11 is bundled together in the vicinity of the base end of the thin tube 10a, and is fixed in a base B formed near the base end on the side surface of the thin tube 10a. When the connector of the light guide LG is connected to the base B, the end face of the optical fiber of the light guide LG is in close contact with the base end face of the optical fiber 11 of the rigid mirror 10.
[0029]
The objective optical system 12 includes an objective lens group 12a and a large number of relay lenses 12b, and is fitted into a small diameter pipe of the narrow tube 10a. The objective lens group 12a is configured as a retrofocus type objective lens group capable of forming an image of a wide field of view having an angle of view of 120 ° or more, and is disposed at the tip of the capillary tube 10a. Then, an image in front of the tip of the thin tube 10a is formed as a wide-angle image. In the objective lens group 12a, the most object side lens seals the tip opening of the thin tube 10a.
[0030]
The multiple relay lenses 12b all have the same shape, and are arranged at equal intervals in the narrow tube 10a. Each relay lens 12b relays the image formed on the image plane by the objective lens group 12a to the base end side of the thin tube 10a by sequentially forming the image on each image plane. As a result, an image substantially the same as the wide-angle image formed by the objective optical system 12a is formed on the final imaging surface 10i.
[0031]
<Image separation device>
Next, the image separation device 20 will be described in detail. In the following, the optical configuration of the image separation device 20 will be described first, and then the Pechan prism and its peripheral mechanism will be described.
[0032]
(Optical configuration)
As shown in FIG. 2, the image separation device 20 includes a translucent mirror 21, a folding mirror 22, a first re-imaging optical system 23, a Pechan prism 24, and a second re-imaging optical system 25. I have.
[0033]
The translucent mirror 21 is a branching element that reflects a part of the incident light beam and transmits the remaining light beam, and is disposed on the optical path of the light beam from the objective optical system 12.
[0034]
A folding mirror 22 is disposed on the optical path of the light beam reflected by the semitransparent mirror 21. The optical axis Ax1 of the objective optical system 12 that is bent by the semitransparent mirror 21 is further bent by the folding mirror 22, thereby forming a crank-shaped optical axis. Further, the optical axis Ax1 of the objective optical system 12 bent by the folding mirror 22 passes through the first re-imaging optical system 23 in a state of being coaxial with the first re-imaging optical system 23, and the first camera C1. The center of the imaging surface is vertically penetrated.
[0035]
As described above, since the semitransparent mirror 21, the folding mirror 22, the first re-imaging optical system 23, and the first camera C1 are arranged, the light beam reflected by the semitransparent mirror 21 is reflected by the folding mirror 22. Then, the light passes through the first re-imaging optical system 23 and enters the imaging surface of the first camera C1.
[0036]
When light enters the imaging surface in this way, the first re-imaging optical system 23 reconnects the wide-angle image formed on the imaging surface 10i by the objective optical system 12 to the imaging surface of the first camera C1. Let me image. In addition, the 1st camera C1 will output image data, if the wide-angle image on the imaging surface is imaged. Hereinafter, in order to distinguish the image data output from the first camera C1 from that of the second camera C2, it is referred to as first image data for convenience.
[0037]
On the other hand, a Pechan prism 24 is disposed on the optical path of the light beam passing through the semitransparent mirror 21. The Pechan prism 24 will be described later with reference to FIG. 3. Briefly, the Pechan prism 24 is an image inverting optical system that acts to invert a wide-angle image formed by the objective optical system 12.
[0038]
Further, a second re-imaging optical system 25 is disposed on the opposite side of the translucent mirror 21 across the Pechan prism 24. The optical axis Ax2 of the second re-imaging optical system 25 is set on the same axis as the optical axis Ax1 of the objective optical system 12 after passing through the semitransparent mirror 21 and before entering the Pechan prism 24. The center of the imaging surface of the second camera C2 passes vertically.
[0039]
The second re-imaging optical system 25 includes first to fourth lens groups 25a to 25d. Among these, the first lens group 25a is fixed to a lens frame that engages with a cam ring in the lens barrel 251 and is held so as to be movable in parallel along the optical axis Ax2. The first lens group 25a itself functions as a focusing optical system for focusing.
[0040]
The second and third lens groups 25b and 25c are fixed to a lens frame that engages with a cam ring in the lens barrel 252 and are held so as to be movable in parallel along the optical axis Ax2. The second and third lens groups 25b and 25c themselves function as a variable magnification optical system for changing the observation magnification. The fourth lens group 25d is fixed in the image separation device 20.
[0041]
As described above, since the translucent mirror 21, the Pechan prism 24, the second re-imaging optical system 25, and the second camera C2 are arranged, the light beam that has passed through the translucent mirror 21 passes through the Pechan prism 24. Then, the light passes through the second re-imaging optical system 25 and enters the imaging surface of the second camera C2.
[0042]
When light enters the imaging surface in this way, the second re-imaging optical system 25 enlarges a part of the image formed on the imaging surface 10i by the objective optical system 12 to a predetermined magnification, and The image is re-imaged on the imaging surface of the second camera C2, and the Pechan prism 24 inverts the image formed on the imaging surface of the second camera C2. In addition, the 2nd camera C2 will output image data, if the image on the imaging surface is imaged. Hereinafter, the image data output from the second camera C2 is referred to as second image data for convenience.
[0043]
By the way, the actuator which is the drive source of the cam ring in the lens barrel 251 is connected to a seesaw type switch of an operating device (not shown), and when one of the polarity switches is pressed, the actuator according to the polarity. Drive the cam ring in the direction. Thereby, the surgeon can freely change the observation magnification of the image displayed on the second monitor M2 during the operation. The actuator that is the drive source of the cam ring in the lens barrel 252 is controlled by a focus control circuit (not shown).
[0044]
(Pechan Prism and surrounding mechanisms)
Next, the Pechan prism 24 and the surrounding mechanism will be described in detail. FIG. 3 is a perspective view of the Pechan prism 24. As shown in FIG. 3, the Pechan prism 24 includes an auxiliary prism 241 and a roof prism 242.
[0045]
The auxiliary prism 241 is a quadrangular prism, and one side surface 241a thereof is disposed perpendicular to the optical axis Ax1 of the objective optical system 12 that has passed through the semitransparent mirror 21. The auxiliary prism 241 is perpendicular to the side surface 241b after the optical axis Ax1 of the objective optical system 12 entering from the side surface 241a is bent twice by the inner surfaces of the two side surfaces 241b and 241c adjacent to the side surface 251a. To pass through.
[0046]
The roof prism 242 is a prism having a shape equivalent to a roof surface protruding from one side surface of the triangular prism (with its ridge line parallel to the bottom surface). Other than the roof surfaces 242f and 242g. The side surface 242d of the auxiliary prism 241 is disposed in parallel and close to the side surface 241b. Therefore, the side surface 242d of the roof prism 242 is disposed perpendicular to the optical axis Ax1 of the objective optical system 12 that has passed through the side surface 241b of the auxiliary prism 241 vertically.
[0047]
The roof prism 242 bends the optical axis Ax1 of the objective optical system 12 entering from the side surface 242d in order by the remaining side surfaces 242e adjacent to the side surface 242d, roof surfaces 242f and 242g, and the inner surfaces of the side surfaces 242d. By letting it pass perpendicularly to the side surface 242e, it goes out. At this time, the direction in which the optical axis Ax1 exits the roof prism 242 is parallel to the optical axis Ax1 before entering the auxiliary prism 242.
[0048]
As shown in FIG. 2, the image separation device 20 further includes a moving mechanism 26 and an XY stage 26 a driven by the moving mechanism 26, and the Pechan prism 24 configured as described above has an XY structure. It is fixed on the stage 26a.
[0049]
The moving mechanism 26 includes a driving actuator such as an AC motor or a DC motor and a gear mechanism for transmitting the driving force of the driving actuator to the stage for each of the X stage and the Y stage. Are driven independently. Further, the moving mechanism 26 also includes a detector such as an encoder for detecting the amount of displacement of the XY stage 26a.
[0050]
The detector and the driving actuator of the moving mechanism 26 are connected to a control device 30 described later. When the XY stage 26 a is driven, the detector transmits data indicating the amount of displacement to the control device 30, and the driving actuator controls the XY stage 26 a based on a signal output from the control device 30. To drive. Thereby, the Pechan prism 24 can be translated in the XY plane orthogonal to the optical axis Ax1 of the objective optical system 12.
[0051]
Then, when the position of the Pechan prism 24 so that the optical axis Ax1 before entering the Pechan prism 24 becomes coaxial with the optical axis Ax1 after exiting from the Peshan prism 24 is set to the “initial position”, the Pechan prism 24 is When translated in the XY plane from the initial position, the optical axis Ax1 of the objective optical system 12 exiting from the Pechan prism 24 is shifted with respect to the extension line of the optical axis Ax1 before entering the Pechan prism 24. .
[0052]
FIG. 3 also shows how the optical axis Ax1 of the objective optical system 12 is shifted by the Pechan prism 24. As shown in FIG. 3, from the initial position where the optical axes Ax1 before and after passing through the Pechan prism 24 are coaxial with each other, the optical axis Ax1 is positive in the X direction with respect to the side surface 241a as the incident end face (FIG. 3 is moved to the left) by the distance w, the optical axis Ax1 moves by the distance w in the negative direction in the X direction with respect to the side surface 242e as the exit end face.
[0053]
This movement corresponds to a case where the Pechan prism 24 is moved by a distance w in the negative direction in the X direction (rightward in FIG. 3) with respect to the optical axis Ax1 of the objective optical system 12 at a fixed position. Therefore, the optical axis Ax′1 of the objective optical system 12 after exiting the Pechan prism 24 is shifted in the negative direction in the X direction by a distance 2w with respect to the optical axis Ax′1 before entering the Pechan prism 24. Is done.
[0054]
Conversely, when the Pechan prism 24 is moved in the positive direction in the X direction, the optical axis Ax1 of the objective optical system 12 is an amount twice as large as the movement amount of the Pechan prism 24 in the positive direction in the X direction. Shifted.
[0055]
Similarly, when the Pechan prism 24 is moved in the Y direction (vertical direction in FIG. 3), the optical axis Ax ″ 1 of the objective optical system 12 after exiting the Pechan prism 24 enters the Pechan prism 24. With respect to the previous optical axis Ax ″ 1, the direction in which the Pechan prism 24 is moved is shifted by twice the moving amount of the Pechan prism 24.
[0056]
When the Pechan prism 24 is translated in the XY plane in this way, the optical axis Ax1 of the objective optical system 12 after exiting the Pechan prism 24 includes the optical axis Ax2 of the second re-imaging optical system 24. Shifted from the straight line. FIG. 2 also shows the situation at this time.
[0057]
At the initial position of the Pechan prism 24 where the optical axis Ax1 of the objective optical system 12 after exiting the Pechan prism 24 and the optical axis Ax2 of the second re-imaging optical system 25 are coaxial, the optical axis of the objective optical system 12 Even after being emitted from the Pechan prism 24, the light beam traveling on Ax1 travels on the optical axis Ax2 of the second re-imaging optical system 25 and enters the center of the imaging surface of the second camera C2.
[0058]
When the Pechan prism 24 is translated in the XY plane from this state, the optical axis Ax1 after exiting the Pechan prism 24 is the optical axis Ax2 of the second re-imaging optical system 25, as shown in FIG. Shifted from. Therefore, after the light beam traveling on the optical axis Ax1 of the objective optical system 12 is emitted from the Pechan prism 24, the second reimaging optical system is shifted from the optical axis Ax2 of the second reimaging optical system 25. 25, and enters a position shifted from the center of the imaging surface of the second camera C2. Note that the shift amount and the shift direction of the incident position of the light beam from the center of the imaging surface at this time depend on the movement amount and the movement direction of the Pechan prism 24 as described above.
[0059]
As described above, the light beam that has traveled on the optical axis Ax2 of the second re-imaging optical system 25 and reached the center of the imaging surface of the second camera C2 when the Pechan prism 24 is in the initial position, As a result of entering the position shifted from the position where the optical axis Ax2 penetrates, the range of the image captured by the second camera C2 is shifted according to the moving amount and moving direction of the Pechan prism 24.
[0060]
<Control device>
Next, the control device 30 will be described. FIG. 4 is a configuration diagram schematically showing the control device 30. The control device 30 includes a CPU 30a, a RAM 30b, a first interface circuit 30c, a second interface device 30d, and a ROM 30e, and the hardware 30a to 30e are connected to each other via a bus B. .
[0061]
The CPU 30a is a central processing unit that controls the other hardware 30b to 30e in an integrated manner. The RAM 30b is a main storage device in which a work area is expanded when the CPU 30a executes various programs.
[0062]
The first interface circuit 30 c is a device that mediates information between the CPU 30 a and the moving mechanism 26. Specifically, the first interface circuit 30c receives information indicating the amount of displacement of the XY stage 26a from the detector of the moving mechanism 26, and starts driving of the XY stage 26a in accordance with an instruction from the CPU 30a. Alternatively, information indicating the stop is transmitted to the actuator of the moving mechanism 26.
[0063]
The second interface circuit 30d is a device that controls reception of the first image data from the first camera C1, and is a relay device that outputs the received first image data to the first monitor M1 as it is. Further, the first image data acquired by the second interface circuit 30e is output to the RAM 30b.
[0064]
The ROM 30e is a read-only storage device that stores various data and various programs. The programs stored in the ROM 30e include a count program and a follow-up control program.
[0065]
Among these, the count program is a program that causes the CPU 30a to continue executing the count process while the power is on. In this counting process, every time the first interface circuit 30c receives information indicating the amount of displacement of the XY stage 26a, the CPU 30a stores the XY stage 26a stored in the RAM 30b based on the received information. Each position information of is updated. Accordingly, the latest position information of each XY stage 26a is recorded in the RAM 30b.
[0066]
The follow-up control program is a program that causes the CPU 30a to continue executing the follow-up control process as shown in the flowchart of FIG. 5 while the power is on. In this follow-up control process, the CPU 30a repeatedly executes the following processing loop (steps S101 to S106) for the first image data of each frame.
[0067]
First, when the CPU 30a acquires the first image data for one frame (S101), the CPU 30a determines the maximum from the coordinates of the plane coordinate system defined to specify the positions of all the pixels of the first image data. All the coordinates of the pixel having the luminance value are extracted (S102). Subsequently, the CPU 30a calculates barycentric coordinates based on the extracted coordinates (S103). Then, the CPU 30a calculates position information indicating each position of the XY stage 26a corresponding to the barycentric coordinates as target position information (S104), and the difference between the target position information of the XY stage 26a and the current position information in the RAM 30c. Is calculated (S105). Thereafter, the CPU 30a instructs the first interface circuit 30c to output a signal for moving the XY stage 26a by the difference to the actuator of the moving mechanism 26 (S106).
[0068]
By executing such follow-up control processing in the control device 30, the enlarged image displayed on the second monitor M2 is centered on the point having the maximum luminance in the wide-angle image displayed on the first monitor M1. The Pechan prism 24 is translated in the XY plane so as to be a partial region.
[0069]
<Treatment tool>
Next, the treatment tool 40 will be described in detail. As shown in FIG. 1, the treatment instrument 40 includes an operation unit 41 and an insertion unit 42.
[0070]
The operation unit 41 has a so-called scissor mechanism having a pair of pieces intersecting in an X shape. That is, the pair of pieces can be relatively rotated around the intersection, and a ring for inserting a finger is formed at the base end of each piece. Then, when these rings are brought into and out of contact with the operator, the tip portions of the pieces are brought into and out of contact according to the principle of leverage.
[0071]
On the other hand, the insertion portion 42 includes a narrow tube and a shaft inserted into the narrow tube. The proximal end of the thin tube is fixed to the distal end portion of one piece in the operation unit 41, and the proximal end of the shaft is fixed to the distal end portion of the other piece in the operation unit 41. When the distal end portions of the pair of pieces are brought into contact with and separated from each other in the operation unit 41, the shaft advances and retreats in the narrow tube in the insertion unit 42.
[0072]
Further, the treatment instrument 40 includes a grasping forceps 33 at the distal end of the insertion portion 42. The gripping forceps 43 are configured as a clip composed of a pair of pieces. More specifically, each of the pair of pieces is formed in a substantially semi-cylindrical shape, and when combined with each other, as a whole, it forms a cylindrical shape having substantially the same thickness as the insertion portion 42. One piece is formed integrally with the insertion portion 42 so as to protrude from the tip of the insertion portion 42, thereby functioning as a fixed piece, and the other piece is rotatable at the tip of the insertion portion 42. By being attached, it functions as a movable piece.
[0073]
The grasping forceps 43 is open when the movable piece is rotated as a whole and is closed, and closed when both pieces are in contact with each other when the movable piece is rotated. It becomes a state. The movable piece is attached to the tip of the shaft of the insertion portion 42, and comes into contact with and separates from the fixed piece according to the advance and retreat of the shaft. That is, the grasping forceps 43 are opened and closed by the two rings of the operation unit 41 being contacted and separated by the operator.
[0074]
Furthermore, the treatment instrument 40 includes a marker portion 44 in the vicinity of the distal end of the insertion portion 42. The example of FIG. 1 shows a state in which the distal end of the insertion portion 42 and the vicinity thereof are enlarged and displayed on the second monitor M2. This marker part 44 is comprised by the seal | sticker affixed on the insertion part 42 at strip | belt shape, and this seal | sticker has the layer which consists of a retroreflective material in the front side, and the layer which consists of an adhesive material in the back side have.
[0075]
FIG. 6 is a cross-sectional view schematically showing the structure of the retroreflective member 50 constituting the marker portion 44. As shown in FIG. 6, the retroreflective member 50 includes a sheet-like base 51 having a predetermined thickness, a large number of spherical beads 52, and a reflector 53 that reflects white light.
[0076]
The reflective material 53 is coated on one surface of the base 51, and a large number of beads 52 are held on the base 51 by being embedded in half on the surface coated with the reflective material 53. These many beads 52 are spread so as to fill the surface coated with the reflective material 53. The beads 52 themselves are made of glass or resin with high transmittance.
[0077]
When the illumination light is incident on the surface of the base 51 on which the beads 52 are spread, the illumination light directed toward the beads 52 out of the illumination light incident on the surface is refracted on the surface of the beads 52 and After entering the inside and being reflected at the boundary surface between the bead 52 and the reflecting material 53, it escapes to the outside of the bead 52 while being refracted on the surface of the bead 52 again.
[0078]
At this time, as indicated by an arrow in FIG. 6, the illumination light reflected by passing through the beads 52 travels in a direction opposite to the traveling direction before entering the beads 52. Released. For this reason, on the surface of a normal object, light incident on one point is scattered in all directions (see FIG. 7), whereas on the surface of a retroreflective material, most of light incident on one point is not scattered. Reflected back to the original position (see FIG. 8).
[0079]
<Operation of surgery support system>
Next, the operation of the surgery support system will be described. Since this surgery support system is configured as described above, it operates as described below.
[0080]
When performing an operation on an organ or the like in a subject's body, an operator who uses this surgery support system first drills a small hole in the subject's abdominal wall and the like, and inserts a mantle tube (tracar; T in FIG. 1) into the hole. Fit. Next, the operator inserts the distal end of the rigid endoscope 10 into the outer tube T, and the first and second monitors M1, M2, the first and second cameras C1, C2, the image separation device 20, and the light source device LS. Turn on the power in turn.
[0081]
Then, illumination light output from the light source device LS is supplied to the rigid endoscope 10 through the light guide LG, and is emitted from the distal end of the rigid endoscope 10 into the body of the subject. When the illumination light illuminates the inside of the subject in this way, the light reflected by the surface of the internal organ or the like and directed toward the tip of the rigid endoscope 10 enters the objective optical system 12. At this time, an image inside the subject's body is formed on the proximal end side of the rigid endoscope 10 by the objective optical system 12.
[0082]
Then, the image formed on the proximal end side of the rigid endoscope 10 is re-imaged on the imaging surface of the first camera C1 by the first re-imaging optical system 23 and is finally imaged after being imaged by the first camera C1. Is displayed on the first monitor M1.
[0083]
When the surgeon identifies a portion where the surgical operation is to be performed by observing the wide-angle image displayed on the first monitor M1, a small hole is made again in the abdominal wall of the subject, and another mantle tube is formed in the hole. T is inserted, and the distal end of the insertion portion 42 of the treatment instrument 40 is inserted into the outer tube T.
[0084]
Subsequently, when the surgeon operates the treatment tool 40 to cause the distal end of the insertion portion 42 of the treatment tool 40 to enter the field of view of the objective optical system 12 in the rigid endoscope 10, the first monitor M1 The tip of the insertion portion 42 of the treatment instrument 40 is projected along with the state of the body of the subject.
[0085]
At this time, on the first monitor M1, the marker portion 44 of the treatment instrument 40 is displayed with high luminance. At the same time, the control device 30 extracts the pixel having the maximum luminance from all the pixels of the first image data, and the Pechan prism 24 is moved by the XY stage 26a to a position corresponding to the barycentric position of the pixel group. Thus, the moving mechanism 26 is controlled. As a result, on the second monitor M2, an enlarged image of the marker portion 34 and its periphery is displayed with the marker portion 34 resident at the center.
[0086]
<Effect>
As described above, in the surgery support system according to the first embodiment, since the surface of the marker unit 44 is formed of a retroreflective material, the pixel group of the first image data with a low overall brightness is extremely A pixel with high luminance is formed. At this time, it is better to identify pixels with extremely high luminance from pixels with overall low luminance than to the identification rate when identifying pixels with similar luminance from pixels with low overall luminance. Since the identification rate is high, the identification rate when identifying the pixel indicating the marker is improved as compared with the conventional one. Therefore, in the control device 30, it is greatly reduced that pixels indicating the marker portion 44 are not extracted from all the pixels of the first image data.
[0087]
In the above description, the grasping forceps 43 is provided as the action portion at the distal end of the insertion portion 42 of the treatment instrument 40. However, the present invention is not limited to this. Although this treatment tool 40 is not illustrated, it may be provided with a scissors as an action part at the tip of the insertion part 42, may be provided with an injection needle, or is provided with a piercing needle. It may be a thing, and may be equipped with an electrode. In any case, it is preferable that the treatment instrument 40 has the marker portion 44 at a position as close to the action portion as possible without overlapping the action portion.
[0088]
<Modification>
In the first embodiment described above, the retroreflective material that constitutes the marker portion 44 is, as shown in FIG. 6, a large number of spherical beads on one surface of the sheet-like base 51 coated with the reflective material 53. Although 52 was spread, it may not be so. For example, as shown in the sectional view of FIG. 9, the retroreflective material may be one in which two layers 62 and 63 are laminated on one surface of a sheet-like base 61.
[0089]
However, in the retroreflective member 60 shown in FIG. 9, a number of depressions are formed on the surface of the base 61 where the two layers are laminated. This recess is formed in a triangular pyramid shape having three sides of a right-angled isosceles triangle and one bottom surface of an equilateral triangle, and an equilateral triangle opening is formed on this surface of the base 61. .
[0090]
Furthermore, as shown in FIG. 10 which is a front view of this surface of the base 61, by arranging the openings so that the sides of the base 61 are adjacent to each other and the adjacent equilateral triangular openings are opposite to each other, The surface of the base 61 is densely filled.
[0091]
The layer 62 in contact with the base 61 is configured as a reflective layer made of a reflective material and has an equal thickness. Further, the layer 63 outside the reflective layer 62 is configured as a transparent layer made of a transparent material such as a resin, and the outer surface of the transparent layer 63 opposite to the side where the reflective layer 62 is provided is the base 61. Like the surface on the side opposite to the side where each depression is present, it is formed flat.
[0092]
Then, when viewed from the direction perpendicular to the paper surface of FIG. 9, the illumination light emitted toward the transparent layer 63 enters the inside of the transparent layer 63 while being refracted on the surface of the transparent layer 63. After being reflected twice at the boundary surface between the transparent layer 63 and the reflective layer 62, it escapes to the outside of the transparent layer 63 while being refracted on the surface of the transparent layer 63 again.
[0093]
At this time, as indicated by an arrow in FIG. 9, the illumination light reflected by passing through the transparent layer 63 travels in a direction opposite to the traveling direction before entering the transparent layer 63. To be released.
[0094]
In this modification as well, the marker portion 44 is configured by a seal pasted in the vicinity of the distal end of the insertion portion 42 of the treatment instrument 40, and the front side layer of the seal is made of the retroreflective material of FIG. Has been created. As a result, regardless of the posture of the treatment instrument 40, most of the illumination light incident on the retroreflective member of the marker unit 44 is reflected so as to be inverted by 180 ° and enters the objective optical system 12. Therefore, a high-luminance point is formed in the marker portion 44 shown in the wide-angle image based on the first image data.
[0095]
Embodiment 2
In the second embodiment, the marker unit 44 is configured to reflect only light of a predetermined color, and the control device 30 extracts pixels of that color from all the pixels of the first image data. Has the same configuration as the first embodiment. Therefore, only the points different from the first embodiment will be described below.
[0096]
FIG. 11 is a cross-sectional view schematically showing the structure of the retroreflective material employed in the marker portion 44 of the second embodiment. As shown in FIG. 11, the retroreflective member 50 ′ employed in the marker unit 44 of the second embodiment is a coating material 54 that transmits only green light and reflects light of other colors. The surface of the retroreflective material shown in FIG. 6 is coated.
[0097]
That is, in the retroreflective member 50 ′ of the second embodiment, half of a large number of transparent spherical beads 52 are embedded in one surface of the base 51 coated with the reflector 53, and the base 51 The coating material 54 is coated on the outer surface of the reflecting material 53 not sandwiched between the beads 52 and the outer surface of the beads 52 with a uniform thickness.
[0098]
Since the retroreflective member 50 ′ of the second embodiment is configured as described above, the light other than the green band in the illumination light traveling toward the beads 52 of the retroreflective member 50 ′ is coated 54. Is reflected in a direction unrelated to the traveling direction (see FIG. 12). On the other hand, as shown by the arrow in FIG. 11, the light in the green band passes through the coating material 54, enters the inside while being refracted on the surface of the bead 52, and the boundary surface between the bead 52 and the reflecting material 53. , And refracts on the surface of the bead 52, escapes to the outside, passes through the coating material 54 again, and proceeds toward the incident direction.
[0099]
As a result, in the marker portion 44 of the treatment instrument 40, only the green component is reflected in substantially the same direction without being scattered. In the pixel group of the first image data, a green pixel having extremely high luminance is formed, and the green marker portion 44 is displayed on the first monitor M1.
[0100]
On the other hand, the ROM 30e of the control device 30 of the second embodiment records a follow-up control program whose contents are slightly changed from those of the first embodiment. This follow-up control program is a program that causes the CPU 30a to continue executing the follow-up control process as shown in the flowchart of FIG. 13 while the power is on. In this follow-up control process, the CPU 30a repeatedly executes the following processing loop (steps S201 to S207) for the first image data of each frame.
[0101]
First, when the CPU 30a obtains the first image data for one frame (S201), the green color is selected from the coordinates of the planar coordinate system defined for specifying the positions of all the pixels of the first image data. All the coordinates of the pixel having the maximum luminance value are extracted (S202). Subsequently, the CPU 30a excludes the coordinates of pixels whose luminance values for red and blue exceed “0” from the extracted coordinates (S203), and calculates the barycentric coordinates based on the remaining coordinates (S204). ). Then, the CPU 30a calculates position information indicating each position of the XY stage 26a corresponding to the barycentric coordinates as target position information (S205), and the difference between the target position information of the XY stage 26a and the current position information in the RAM 30c. Is calculated (S206). Thereafter, the CPU 30a instructs the first interface circuit 30c to output a signal for moving the XY stage 26a by the difference to the actuator of the moving mechanism 26 (S207).
[0102]
By executing such follow-up control processing in the control device 30, the surgery support system of the second embodiment operates as described below.
[0103]
That is, since the surface of an internal organ or the like is usually wet with body fluid or the like, as shown in the wide-angle image screen example of FIG. is there. However, according to the follow-up control process described above, the points having the red component and the blue component (two large circled portions in the screen example of FIG. 14) are excluded from the processing target and have a low possibility of being present in the body. It is determined that a point having only a high luminance component (small ◯ portion in the screen example of FIG. 14) is a pixel indicating the marker portion 44. Then, the XY stage 26a is controlled so that the Pechan prism 24 moves toward a position corresponding to the position of the point.
[0104]
As described above, according to the second embodiment, even when a large number of high-luminance points are generated, high-luminance points having a red component and a blue component are excluded. Therefore, compared to the first embodiment, Pixels indicating the marker portion 44 can be identified with a higher identification rate.
[0105]
<Modification>
In the second embodiment described above, the retroreflective material constituting the marker portion 44 is composed of a number of spherical beads on one surface of the sheet-like base 51 coated with the reflective material 53, as shown in FIG. 52 is spread and the surface of the bead 42 is coated with the coating material 54, but this need not be the case. For example, as shown in FIG. 15, the retroreflective member 50 ″ may be configured such that the reflective member 53 between the base 51 and the bead 52 reflects only light in the green band. At this time, the light in the other band is absorbed by the base 51. Thus, it is not necessary to use the coating material 54 as shown in FIG.
[0106]
【The invention's effect】
As described above, according to the present invention, when an area extracted from a wide-angle image is displayed as an enlarged image and the extracted area is made to follow the position of the marker in the wide-angle image, It is possible to improve the identification rate when identifying the pixel indicating the marker from all the pixels of the wide-angle image.
[Brief description of the drawings]
FIG. 1 is a configuration diagram schematically illustrating a surgery support system according to a first embodiment.
FIG. 2 is a block diagram schematically showing a rigid endoscope and an image separation device.
FIG. 3 is a perspective view of a Pechan prism.
FIG. 4 is a block diagram schematically showing a control device.
FIG. 5 is a flowchart for explaining follow-up control processing;
FIG. 6 is a cross-sectional view for explaining the structure of a retroreflective material
FIG. 7 is an explanatory diagram showing a state in which light is scattered on the object surface.
FIG. 8 is an explanatory view showing a state in which light is reflected on the surface of the retroreflective material.
FIG. 9 is a cross-sectional view for explaining the structure of a retroreflective material that can be employed as a modification.
FIG. 10 is a front view for explaining the arrangement of the recesses of the retroreflective member according to the modification.
FIG. 11 is a cross-sectional view for explaining the structure of a retroreflective member according to the second embodiment.
FIG. 12 is a cross-sectional view showing a state in which light in another band is reflected by a retroreflective material.
FIG. 13 is a flowchart for explaining follow-up control processing;
FIG. 14 is an exemplary diagram showing an example in which a high-luminance point is displayed in the wide-angle image indicated by the first image data.
FIG. 15 is a cross-sectional view for explaining the structure of a retroreflective material that can be employed as a modification.
[Explanation of symbols]
M1 first monitor
M2 second monitor
C1 first camera
C2 Second camera
LS light source device
LG Light Guide
10 Rigid endoscope
11 Optical fiber
12 Objective optical system
12a Objective lens group
12b Relay lens group
20 Image separator
21 translucent mirror
22 Folding mirror
23 First Reimaging Optical System
24 Peshan Prism
25 Second Reimaging Optical System
26 Movement mechanism
26a XY stage
30 Control device
30a CPU
30b RAM
30f ROM
40 treatment tools
41 Operation unit
42 Insertion part
43 Grasping forceps
44 Marker section

Claims (11)

A treatment instrument having an elongated insertion portion in which a marker including a retroreflective material and exhibiting a predetermined characteristic is disposed in the vicinity of the tip thereof;
An endoscope having an elongated optical insertion section having a thickness that can be inserted into the body of a subject and an objective optical system in the insertion section that forms an image of a subject facing the distal end on the proximal end side;
A first imaging device that generates first image data by capturing an image formed by the objective optical system and outputs the first image data in the form of a signal;
A second imaging device that generates second image data by capturing at least a part of an image formed by the objective optical system and outputs the second image data in the form of a signal;
A shift device that relatively shifts the optical axis of the objective optical system and the second imaging device;
A feature point specifying unit for specifying coordinates of a pixel group having the predetermined feature from all coordinates of a coordinate system defined for all pixels in the first image data;
A control unit that positions the pixel group having the predetermined feature at substantially the center of all the pixels in the second image data by controlling the shift device based on the coordinates specified by the feature point specifying unit; An endoscope apparatus comprising the endoscope apparatus. The endoscope apparatus according to claim 1, wherein the predetermined feature exhibited by the marker is high-intensity reflection of white light on the retroreflecting material. The endoscope apparatus according to claim 1, wherein the predetermined feature exhibited by the marker is high-intensity reflection of light only in a predetermined wavelength band in the retroreflective material. 4. The endoscope apparatus according to claim 3, wherein the light of only the predetermined wavelength band is light extracted from white light by a coating material coated on a surface of the retroreflective material. 4. The endoscope apparatus according to claim 3, wherein the light of only the predetermined wavelength band is light extracted from white light by a reflective material inside the retroreflective material. The endoscope apparatus according to claim 1, further comprising a magnifying optical system that magnifies an image formed by the objective optical system and causes the second imaging device to capture a part of the magnified image. The shift device is moved in a plane perpendicular to the optical axis of the objective optical system to shift the optical axis of the objective optical system, and the movement of moving the optical axis shift optical element. The endoscope apparatus according to claim 1, further comprising: an apparatus. The endoscope apparatus according to claim 7, wherein the optical axis shift optical element is an image inverting optical system having at least four reflecting surfaces. 2. The treatment instrument according to claim 1, wherein the treatment instrument is for being inserted into a hole different from a hole for inserting the insertion portion of the endoscope into the body of the subject. Endoscopic device. The endoscope apparatus according to claim 9, wherein the endoscope is a rigid endoscope. A light source device for supplying illumination light for illuminating the front end of the endoscope to the endoscope;
The endoscope apparatus according to claim 1, further comprising an illumination optical system that guides illumination light supplied from the light source device to a distal end of the insertion portion.

Images