JP6613903B2 - Imaging device - Google Patents

Description

The present invention is irradiated with excitation light to the fluorescent dye is administered into the body of a subject, about Louis Imaging device to photograph the fluorescence emitted from the fluorescent dye.

  A technique called near-infrared fluorescence imaging is used for angiography in surgery. In this near-infrared fluorescence imaging, indocyanine green (ICG), which is a fluorescent dye, is injected into an affected area by injecting it with an injector or the like. When indocyanine green is irradiated with near infrared light having a wavelength of about 600 to 850 nm (nanometer) as excitation light, indocyanine green emits near infrared fluorescence having a wavelength of about 750 to 900 nm. This fluorescence is photographed by an imaging device capable of detecting near infrared light, and the image is displayed on a display unit such as a liquid crystal display panel. According to this near-infrared fluorescence imaging, it is possible to observe blood vessels, lymph vessels, and the like existing at a depth of about 20 mm from the body surface.

  In recent years, attention has been focused on a technique in which a tumor is fluorescently labeled and used for surgical navigation. As a fluorescent labeling agent for fluorescently labeling the tumor, 5-aminolevulinic acid (5-ALA / 5-Aminolevulinic Acid) is used. When this 5-aminolevulinic acid (hereinafter abbreviated as “5-ALA”) is administered to a subject, 5-ALA is metabolized to a fluorescent dye, PpIX (protoporphyrinIX / protoporphyrinine). . This PpIX accumulates specifically in cancer cells. When PpIX, which is a metabolite of 5-ALA, is irradiated with visible light having a wavelength of about 410 nm, red visible light having a wavelength of about 630 nm is emitted as fluorescence from PpIX. By observing the fluorescence from this PpIX, cancer cells can be confirmed.

  Patent Document 1 discloses an intensity distribution image of near-infrared fluorescence obtained by irradiating an indocyanine green excitation light to a living organ to which indocyanine green is administered, and before indocyanine green administration. Compared with the cancer lesion distribution image obtained by applying X-ray, nuclear magnetic resonance or ultrasound to the subject's organs, it is detected by the intensity distribution image of near-infrared fluorescence, but the cancer lesion distribution image Discloses a data collection method for collecting data of a region that is not detected as secondary lesion region data of cancer.

  In such an imaging device that captures fluorescence from a fluorescent dye that has entered the body, the fluorescence from the subject and the visible image of the subject are simultaneously stored as a moving image, and the captured image recorded by the video recorder is stored. It is configured to play video. As described above, the conventional imaging apparatus records and reproduces an image captured at a predetermined frame rate as a moving image, so that a blood vessel / lymph vessel travels after administration of a fluorescent dye such as ICG in a bright external illumination environment. Observation and confirmation of cancer lesion area.

  Such recorded data can be used not only for reference but also for obtaining new knowledge by using it for analysis. For example, in TIC (Time Intensity Curve) analysis that draws a signal change curve in the time direction of ROI (Region Of Interest), indocyanine green is obtained by obtaining the time until the pixel value of ROI reaches a peak. It is possible to quantitatively evaluate the contrast time of a fluorescent dye such as. For example, to check blood flow after cardiac coronary artery bypass surgery, it is effective to obtain a temporal change curve of the ROI pixel value by TIC.

On the other hand, when a region such as a heart that accompanies body movement of the subject is analyzed as an ROI, it is necessary to perform correction associated with body movement of the subject. For example, when analyzing the myocardial part as an ROI, if there is cardiovascular movement accompanying pulsation, the cardiovascular is mixed into the analysis region, so TIC
The periodic component of the cardiovascular region accompanying the body movement of the subject is superimposed on the curve.

  Patent Document 2 discloses an ultrasonic diagnostic apparatus that corrects a motion associated with body movement of a subject using vector calculation.

International Publication No. 2009/139466 JP 2010-51729 A

  For example, in the TIC analysis in which the vicinity of the myocardial part of the heart is ROI, the position of the blood vessel part needs to be fixed (locked), but in the vector calculation described in Patent Document 2, the blood vessel is positioned in all regions. It is impossible to fix. That is, in the vector calculation described in Patent Document 2, it takes a long time for correction calculation involving spatial movement, and it is difficult to fix the position of the blood vessel part in the entire region.

The present invention has been made to solve the above-described problem, and it is possible to easily obtain a temporal change curve of a pixel value of a region of interest even for a region of interest that moves as the subject moves. and to provide a capable Imaging device.

According to the first aspect of the present invention, the excitation light for irradiating the subject with excitation light for exciting the fluorescent dye administered to the subject, and the excitation light irradiates the excitation light. An imaging unit that acquires a fluorescence image by imaging fluorescence generated from the fluorescent dye, and an image storage unit that stores the fluorescence image that changes with the body movement of the subject over time. In the imaging apparatus, the small region extracting unit that extracts a small region including a region of interest and a reference object in the vicinity of the region of interest from at least one of the fluorescent images stored with time in the image storage unit; Based on the marking unit for marking the position of the reference object in the small region extracted by the region extraction unit, and the fluorescence image stored over time based on the position of the reference object marked by the marking unit. A position specifying unit for specifying the position of the reference object with respect to the small area, and storing the reference object specified by the position specifying unit over time so that the position of the reference object is a constant position in the small area. A shift unit that shifts the position of the pixel in the small area in the fluorescent image, and a pixel in the region of interest in the small area image in which the position of the pixel is shifted by the shift unit in the fluorescent image stored over time A pixel value measuring unit that measures a value, and a change curve creating unit that creates a temporal change curve of the pixel value of the region of interest based on the pixel value measured by the pixel value measuring unit. And

According to a second aspect of the present invention, in the first aspect of the invention, the small region is a rectangular region, and the reference object is a blood vessel or lymph that passes through two opposite sides of the rectangular region. It is a tube.

According to the first aspect of the present invention, it is possible to easily obtain a temporal change curve of the pixel value of the region of interest in the fluorescence image, even for the region of interest that moves with the body movement of the subject. It becomes.

  According to the second aspect of the present invention, it is possible to obtain a temporal change curve of the pixel value of the region of interest near the blood vessel or lymph vessel.

1 is a perspective view of an imaging apparatus according to the present invention. 1 is a side view of an imaging apparatus according to the present invention. 1 is a plan view of an imaging apparatus according to the present invention. 2 is a schematic diagram of an illumination / photographing unit 12. FIG. 2 is a schematic diagram of a camera 21 in an illumination / photographing unit 12. FIG. It is a block diagram which shows the main control systems of the imaging device which concerns on this invention. 6 is a schematic diagram showing an image of the vicinity of a heart 91 displayed on a monitor 15. FIG. It is a flowchart which shows the imaging process by the imaging device concerning this invention. It is an enlarged view of the extracted small region 100. FIG. 10 is a graph showing a result of measuring pixel values in a region corresponding to the ROI shown in FIG. 9 over time. It is a schematic diagram which shows the state which performs marking. It is an enlarged view of the small area | region 100 after shift execution. It is a graph which shows the result of having measured the pixel value of the field corresponding to ROI shown in Drawing 12 with time.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a perspective view of an imaging apparatus according to the present invention. FIG. 2 is a side view of the imaging apparatus according to the present invention. FIG. 3 is a plan view of the imaging apparatus according to the present invention.

  The imaging apparatus according to the present invention is for irradiating excitation light to indocyanine green as a fluorescent dye injected into the body of a subject and photographing fluorescence emitted from this indocyanine green, A carriage 11 having four wheels 13, an arm mechanism 30 disposed near the front of the carriage 11 in the traveling direction on the upper surface of the carriage 11 (left direction in FIGS. 2 and 3), and the arm mechanism 30 Are provided with an illumination / photographing unit 12 and a monitor 15 arranged via a sub arm 41. A handle 14 used when moving the carriage 11 is attached to the rear of the carriage 11 in the traveling direction. A recess 16 for mounting a remote control for remotely operating the imaging apparatus is formed on the top surface of the carriage 11.

  The arm mechanism 30 described above is disposed on the front side in the traveling direction of the carriage 11. The arm mechanism 30 includes a first arm member 31 connected to a support portion 37 disposed on a support column 36 erected on the front side in the traveling direction of the carriage 11 by a hinge portion 33. The first arm member 31 can swing with respect to the carriage 11 through the support column 36 and the support portion 37 by the action of the hinge portion 33. The monitor 15 described above is attached to the support column 36.

  A second arm member 32 is connected to the upper end of the first arm member 31 by a hinge portion 34. The second arm member 32 can swing with respect to the first arm member 31 by the action of the hinge portion 34. For this reason, the first arm member 31 and the second arm member 32 are the first arm member 31 and the second arm member 32, as indicated by the phantom line with the reference C in FIG. 1 and 3 and a solid line marked with a reference A in FIG. 1 to FIG. 3, and a first arm member 31 It is possible to take a standby posture in which the second arm member 32 approaches.

  A support portion 43 is connected to the lower end of the second arm member 32 by a hinge portion 35. The support portion 43 can swing with respect to the second arm member 32 by the action of the hinge portion 35. A rotating shaft 42 is supported on the support portion 43. Then, the sub arm 41 that supports the illumination / photographing unit 12 rotates around the rotation shaft 42 disposed at the tip of the second arm member 32. For this reason, the illumination / photographing unit 12 shoots by rotating the sub arm 41 as shown by a solid line with a symbol A in FIGS. 1 to 3 or as a virtual line with a symbol C in FIG. When the carriage 11 is moved as shown by the position in front of the carriage 11 in the traveling direction with respect to the arm mechanism 30 for taking the attitude or the standby attitude, and the phantom line marked with the symbol B in FIGS. It moves between the position of the rear side in the advancing direction of the carriage 11 with respect to the arm mechanism 30 that is in the posture.

  FIG. 4 is a schematic diagram of the illumination / imaging unit 12.

  The illumination / imaging unit 12 includes a camera 21 having a plurality of imaging elements capable of capturing visible light and near-infrared light, a visible light source 22 disposed on the outer periphery of the camera 21, and a visible light source 22. And an excitation light source 23 disposed on the outer periphery. The visible light source 22 emits white light (visible light). The excitation light source 23 emits near-infrared light having a wavelength of 810 nm, which is excitation light for exciting indocyanine green as a fluorescent dye. From indocyanine green irradiated with near-infrared light of 810 nm, near-infrared light having a peak of about 845 nm is emitted as fluorescence.

  In this embodiment, the illumination / photographing unit 12 in which the visible light source 22, the excitation light source 23, and the camera 21 are integrated is used. However, the visible light source 22, the excitation light source 23, and the camera 21 May be arranged individually.

  FIG. 5 is a schematic diagram of the camera 21 in the illumination / photographing unit 12.

The camera 21 includes a movable lens 54 that reciprocates for focusing, a wavelength selection filter 53, a visible light image sensor 51, and a fluorescence image sensor 52. The visible light image sensor 51 and the fluorescence image sensor 52 are composed of a CMOS or a CCD. Visible light and fluorescence incident on the camera 21 coaxially along the optical axis L pass through the movable lens 54 constituting the focusing mechanism, and then reach the wavelength selection filter 53. Of visible light and fluorescence incident coaxially, visible light is reflected by the wavelength selection filter 53 and enters the visible light image sensor 51. Of the coaxial visible light and fluorescence, the fluorescence passes through the wavelength selection filter 53 and enters the fluorescence imaging device 52. At this time, the visible light is focused on the visible light imaging element 51 and the fluorescence is focused on the fluorescent imaging element 52 by the action of the focusing mechanism including the movable lens 54.

  FIG. 6 is a block diagram showing a main control system of the imaging apparatus according to the present invention.

  This imaging apparatus is composed of a CPU that performs logical operations, a ROM that stores operation programs necessary for controlling the apparatus, a RAM that temporarily stores data during control, and the like, and a control unit that controls the entire apparatus 60. The control unit 60 includes an image processing unit 70 that executes various types of image processing on the fluorescent image and the visible image.

  The control unit 60 is connected to an input unit 62 through which various types of information are input by an operator. The control unit 60 is connected to the monitor 15 described above. The input unit 62 may be disposed on a remote controller for remotely operating the imaging apparatus, and may be disposed on the screen of the monitor 15 when the monitor 15 is a touch panel type. It may be disposed on the carriage 11.

  The control unit 60 is connected to the illumination / photographing unit 12 including the camera 21, the visible light source 22, and the excitation light source 23. The control unit 60 is also connected to an image storage unit 63 that stores images taken by the camera 21 over time. The image storage unit 63 includes a fluorescent image storage unit 64 that stores fluorescent images over time, and a visible image storage unit 65 that stores visible images over time. Instead of including the fluorescent image storage unit 64 and the visible image storage unit 65, a combined image storage unit that stores an image obtained by combining (merging) the visible image and the fluorescent image with time may be provided.

  Further, as will be described later, the image processing unit 70 in the control unit 60 uses a reference object in the vicinity of the ROI (region of interest) and ROI from at least one of the fluorescent images stored in the fluorescent image storage unit 64 over time. A small region extracting unit 71 for extracting a small region including the cardiovascular, a marking unit 72 for marking a cardiovascular position in the small region extracted by the small region extracting unit 71 as a master line, and a marking by the marking unit 72 A position specifying unit 73 for specifying the position of the cardiovascular with respect to a small region in the fluorescence image stored over time based on the position of the cardiovascular, and the position of the cardiovascular specified by the position specifying unit 73 is small A shift unit 74 that shifts the position of the pixel in the small area in the fluorescent image stored over time so as to be a constant position in the area; A pixel value measuring unit 75 that measures the pixel value of the ROI in the image in the small area shifted by the shift unit 74 in the fluorescence image, and the ROI pixel value over time based on the pixel value measured by the pixel value measuring unit 75 A change curve creating unit 76 for creating a simple change curve.

  Next, by using the imaging apparatus having the above-described configuration, for example, changes in the pixel values of the ROI fluorescence image over time using TIC analysis for blood flow confirmation after cardiac coronary artery bypass surgery An imaging operation for obtaining a curve will be described. FIG. 7 is a schematic diagram showing an image near the heart 91 displayed on the monitor 15.

In this embodiment, the myocardial portion of the heart 91 in the vicinity of the cardiovascular 92 is used as the ROI, and the change over time of the pixel value of the fluorescence image is measured. The TIC analysis in which the myocardial portion is the ROI and the change in the pixel value of the fluorescence image over time is observed is an effective means for confirming the blood flow after cardiac coronary artery bypass surgery. On the other hand, when analyzing the myocardial region in the vicinity of the cardiovascular 92 as an ROI, the cardiovascular 92 is mixed into the pixel value measurement region due to the movement of the cardiovascular 92 due to pulsation. The periodic components of the cardiovascular region image accompanying the body movement are superimposed. For this reason, the imaging apparatus according to the present invention employs a configuration in which the small region 100 is extracted from the fluorescent image stored in the fluorescent image storage unit 64 and image processing is executed on the small region 100. .

  FIG. 8 is a flowchart showing an imaging process by the imaging apparatus according to the present invention.

  In this imaging apparatus, a fluorescence image including the heart 91 is previously captured by the fluorescence imaging element 52 in the camera 21 and stored in the fluorescence image storage unit 64 (step S1). This fluorescent image is taken at a frame rate of 60 fps (frame per second), for example. Here, when the pixel value is measured for 30 seconds at a frame rate of 60 fps, 1800 fluorescent images are stored.

  When performing the TIC analysis, first, a rectangular small region 100 including the ROI and the cardiovascular 92 in the vicinity of the ROI is selected from at least one of the fluorescent images stored with time in the fluorescent image storage unit 64. Extract (step S2). The rectangular small region 100 includes an ROI and a cardiovascular 92 used as a marker in the vicinity of the ROI, and the cardiovascular 92 penetrates the small region 100 in the vertical direction (Y direction) in FIG. Thus, the rectangular small region 100 is extracted so as to pass through two sides facing the X direction. The setting of the small area 100 is executed by the small area extracting unit 71 shown in FIG. However, the extraction of the small area 100 may be performed manually by the operator.

  FIG. 9 is an enlarged view of the extracted small region 100.

  As shown in this figure, the rectangular small region 100 includes an ROI and a cardiovascular 92 that is used as a reference object in the vicinity of the ROI. When this small region 100 is selected, the cardiovascular 92 penetrates the small region 100 in the Y direction so that the small region 100 passes through two sides facing the X direction. In addition, when the change over time of the pixel value in the ROI is measured in such a state, the cardiovascular 92 is mixed into the pixel value measurement region due to the movement of the cardiovascular 92 accompanying the pulsation of the subject. Therefore, the periodic component of the cardiovascular region image accompanying the body movement of the subject is superimposed on the TIC curve.

  FIG. 10 is a graph showing the results of measuring the pixel values in the region corresponding to the ROI shown in FIG. 9 over time.

  Here, the region corresponding to the ROI is a region corresponding to the position of the ROI when there is no body movement of the subject. In the TIC analysis, it is necessary to measure the pixel value of the ROI with time, but in reality, the position of the ROI changes with the pulsation of the subject. FIG. 10 shows the result of measuring the pixel value at the position where the region where the ROI exists when there is no pulsation of the subject is the region corresponding to the ROI. As apparent from FIG. 10, since the cardiovascular 92 is mixed in the region corresponding to the ROI due to the movement of the ROI accompanying the pulsation of the subject, the curve L1 representing the pixel value of the region corresponding to the ROI is The up and down movement is repeated at a period equivalent to the pulsation of the subject.

  Next, the position of the cardiovascular 92 as a reference in the small area 100 after extraction is marked (step S3).

  FIG. 11 is a schematic diagram illustrating a state in which marking is performed.

In this marking, an image of the cardiovascular 92 is extracted by executing image processing using a filter on the small region 100. Then, a line passing through the center of the cardiovascular 92 is drawn as a master line 93 that is used as a marker. This marking is performed by the marking unit 72 shown in FIG. However, this marking may be performed manually by the operator.

  Next, the position of the cardiovascular 92 is specified with respect to the fluorescence image memorize | stored in the fluorescence image memory | storage part 64 by using the master line 93 (step S4). That is, for each fluorescent image stored in the fluorescent image storage unit 64 over time, a search in the vicinity of the master line 93 is sequentially performed, and a region having a high pixel value is specified as the cardiovascular 92. Then, a new master line 93 is obtained by executing a process similar to the marking process described above with reference to the position of the detected cardiovascular 92. Such an operation is sequentially performed on each fluorescent image stored in the fluorescent image storage unit 64 over time. The specification of the position of the cardiovascular 92 and the update of the master line are executed by the position specifying unit 73 shown in FIG.

  Next, for each fluorescent image in which the master line 93 is specified, the position of the master line 93, that is, the position of the cardiovascular 92 is a constant position in the small area 100. The position of the pixel in 100 is shifted (step S5). More specifically, the position of the pixel in the small region 100 is set so that the master line 93 penetrating in the Y direction in the small region 100 is a straight line that faces the Y direction at the center position in the X direction in the small region 100. Shift in the X direction. Here, that the position of the master line 93 (the position of the cardiovascular 92) is a constant position in the small region 100 is that the position in the X direction among the positions (X, Y) of the pixels constituting the master line 93. It will be constant. The shift of the image in the X direction is executed by the shift unit 74 shown in FIG. When the cardiovascular 92 passes through the small region 100 in the X direction, each pixel may be shifted in the Y direction.

  FIG. 12 is an enlarged view of the small region 100 after the shift is executed.

  As shown in the figure, after the shift is executed, the positional relationship between the ROI and the cardiovascular 92 can be made constant. For this reason, even when the cardiovascular 92 is moved due to the pulsation of the subject, it is possible to prevent the pixel value of the cardiovascular 92 from being mixed into the measurement region.

  Next, the pixel value of the region corresponding to the ROI shown in FIG. 12 is measured (step S6). That is, the pixel value of the area corresponding to the ROI is measured for each fluorescent image that is stored in the fluorescent image storage unit 64 with time and shifted. The measurement of the pixel value is executed by the pixel value measuring unit 75 shown in FIG.

  Then, a change curve with time of the pixel value of ROI is created based on the pixel value after measurement (step S7). The creation of the change curve is executed by the change curve creation unit 76 shown in FIG.

  FIG. 13 is a graph showing the results of measuring the pixel values in the region corresponding to the ROI shown in FIG. 12 over time.

  At this time, it is possible to prevent the periodic component of the cardiovascular region image accompanying the body movement of the subject from being superimposed on the TIC curve. For this reason, the change curve L2 shown in FIG. 13 reflects only the movement of the myocardial portion accompanying the pulsation of the subject. Therefore, the change curve L2 shown in FIG. 13 reflects the change in the pixel value of the myocardial portion more accurately than the change curve L1 shown in FIG.

In the above-described embodiment, the cardiovascular 92 is used as a reference for measuring a change in the pixel value of the myocardial portion. However, instead of using a blood vessel as a reference, a lymph vessel may be used as a reference. Furthermore, the present invention can be similarly applied not only to the heart 91 but also to a region with body movement such as a diaphragm.

  In the above-described embodiment, indocyanine green is used as a material containing a fluorescent dye, and the indocyanine green is irradiated with near-infrared light of about 600 nm to 850 nm as excitation light. The case of emitting fluorescence in the near-infrared region having a peak at approximately 810 nm from the above has been described, but light other than near-infrared light may be used.

  Further, instead of using indocyanine green as the fluorescent dye, other fluorescent dyes such as the above-mentioned 5-ALA may be used.

DESCRIPTION OF SYMBOLS 12 Illumination / imaging | photography part 21 Camera 22 Visible light source 23 Excitation light source 30 Arm mechanism 51 Imaging element for visible light 52 Imaging element for fluorescence 60 Control part 62 Input part 63 Image storage part 64 Fluorescence image storage part 65 Visible image storage part 70 Image Processing unit 71 Small region extraction unit 72 Marking unit 73 Position specifying unit 74 Shift unit 75 Pixel value measurement unit 76 Change curve creation unit 91 Heart 92 Cardiovascular 93 Master line 100 Small region

Claims (2)

An excitation light source for irradiating the subject with excitation light for exciting the fluorescent dye administered to the subject;
An imaging unit that acquires a fluorescence image by imaging fluorescence generated from the fluorescent dye by being irradiated with excitation light; and
An image storage unit that stores the fluorescence image that changes with the body movement of the subject over time;
In an imaging apparatus comprising:
A small region extraction unit that extracts a small region including the region of interest and a reference object in the vicinity of the region of interest from at least one of the fluorescent images stored over time in the image storage unit;
A marking unit for marking the position of the reference object in the small region extracted by the small region extraction unit;
Based on the position of the reference object marked by the marking part, a position specifying part for specifying the position of the reference object with respect to the small region in the fluorescent image stored over time;
A shift unit that shifts the position of the pixel in the small region in the fluorescent image stored over time so that the position of the reference object specified by the position specifying unit is a constant position in the small region;
A pixel value measuring unit that measures a pixel value of a region of interest in an image of a small region in which a pixel position is shifted by the shift unit in a fluorescence image stored over time;
Based on the pixel value measured in the pixel value measurement unit, a change curve creation unit that creates a change curve over time of the pixel value of the region of interest;
An imaging apparatus comprising: The imaging apparatus according to claim 1, wherein
The small area is a rectangular area,
The imaging device, wherein the reference object is a blood vessel or a lymph vessel that passes through two opposite sides of the rectangular region.

Images