WO2016039002A1

WO2016039002A1 - Imaging device and imaging method

Info

Publication number: WO2016039002A1
Application number: PCT/JP2015/069607
Authority: WO
Inventors: 足立　晋
Original assignee: 株式会社島津製作所
Priority date: 2014-09-09
Filing date: 2015-07-08
Publication date: 2016-03-17

Abstract

　A relational expression indicating a relationship between the position of a focusing lens and a focus value of a fluorescence image captured by a fluorescence imaging element is computed by a relational expression computing unit 42, a precision index of the relational expression computed by the relational expression computing unit 42 is computed by a precision index computing unit 43, a region in which a focal position is present on the basis of the relational expression when the precision index computed by the precision index computing unit 43 is equal to or greater than a predetermined value is determined by a lens position control unit 44, and the focusing lens is moved to the position at which the focus value is maximized in the region.

Description

Imaging apparatus and imaging method

The present invention relates to an imaging apparatus and an imaging method for photographing fluorescence generated from a fluorescent dye by irradiating a fluorescent dye injected into a subject with excitation light.

In recent years, a technique called near-infrared fluorescence imaging has been used for surgery. In this near-infrared fluorescence imaging, indocyanine green (ICG) as a fluorescent dye is injected into the affected area. When indocyanine green is irradiated with near infrared light of 750 to 810 nm (nanometers) as excitation light, indocyanine green emits fluorescence in the near infrared region having a peak at about 800 nm. This fluorescence is photographed by a camera capable of detecting near infrared light, and the image is displayed on a display unit such as a liquid crystal display panel. According to this near-infrared fluorescence imaging, it is possible to observe blood vessels, lymph vessels, and the like existing at a depth of about 20 mm from the body surface.

Patent Document 1 discloses a near-infrared fluorescence intensity distribution image obtained by irradiating an indocyanine green excitation light to a living organ to which indocyanine green is administered, and indocyanine green administration. Compared with the cancer lesion distribution image obtained by applying X-rays, nuclear magnetic resonance or ultrasound to the previous test organ, it is detected by the intensity distribution image of near-infrared fluorescence, A data collection method is disclosed in which data of a region that is not detected in a cancer lesion distribution image is collected as cancer secondary lesion region data.

In Patent Literature 2, excitation light and visible light are alternately irradiated on a subject to which an angiographic contrast agent is administered, and a fluorescence image and a visible image irradiated with excitation light are alternately acquired by an imaging unit. In addition, a surgical support method is disclosed in which a fluorescent image is subjected to threshold processing with a predetermined threshold to extract a blood vessel image, and a composite image in which the visible image and the extracted blood vessel image are superimposed is created.

Also, when such an object is imaged by the image sensor, it is necessary to focus the image of the object on the image sensor. In Patent Document 3, a lens is driven in an arbitrary direction from a focusing operation start point, a contrast value is obtained at n points during this time, and the majority value is determined by comparing the contrast value at the n point with the contrast value at the starting point. Discloses a hill-climbing autofocus device that determines an initial drive direction. The autofocus device described in Patent Document 3 employs a configuration in which a focus value, which is a parameter corresponding to contrast, is calculated from a captured image and the focusing lens is moved to a position where the focus value is maximized. Has been.

International Publication No. 2009/139466 JP 2009-226072 A Japanese Patent No. 3179138

Fluorescence from indocyanine green is very weak compared to visible light. For this reason, when this fluorescence is imaged by the image sensor, the noise component of the image sensor occupies a large proportion of the fluorescence image signal.

FIG. 8 is a graph schematically showing the relationship between the rotational position of the motor for moving the focusing lens and the focus value at that time. In this graph, the vertical axis and the horizontal axis indicate arbitrary units (au).

As shown in FIG. 8, in the relationship between the rotational position of the motor and the focus value, the fluorescence is weak, and a lot of noise components are included in the signal from the image sensor, so that the relationship fluctuates. For this reason, when the configuration like the autofocus device described in Patent Document 3 described above is adopted, it is difficult to determine the position where the focus value is maximized, and it is not possible to perform focusing accurately. Will occur.

The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an imaging apparatus capable of performing accurate focusing even when photographing fluorescence.

In the first invention, an excitation light source that irradiates the subject with an excitation light beam for exciting the fluorescent dye injected into the subject, and a fluorescent image that is generated from the fluorescent dye when irradiated with the excitation light beam An image pickup device capable of picking up an image, a focusing mechanism for focusing a fluorescent image generated from the fluorescent dye by movement of the focusing lens on the image pickup device, and a motor for moving the focusing lens In the imaging apparatus for capturing a fluorescent image of the subject, the focusing lens is moved by the motor, and the position of the focusing lens at that time and the focus value of the fluorescent image captured by the imaging element A relational expression calculation unit for obtaining a relational expression representing the relationship between the relational expression and an accuracy index for calculating an accuracy index of the relational expression obtained by the relational expression calculation unit When the accuracy index calculated by the calculation unit and the accuracy index calculation unit is equal to or greater than a predetermined value, an area where the in-focus position exists is determined based on the relational expression obtained by the relational expression calculation unit, and this area A focusing lens position control unit that moves the focusing lens to a position where the focus value is maximized.

In the second invention, the relational expression is a linear expression, and the focusing lens position control unit determines that an area where the inclination of the straight line represented by the linear expression is inverted is an area where the in-focus position exists. .

In the third invention, the focus value is calculated by applying a sharpening filter to the fluorescence image captured by the image sensor.

In the fourth aspect of the invention, an excitation beam for exciting the fluorescent dye injected into the subject is irradiated toward the subject, and the fluorescence generated from the fluorescent dye is focused on the image sensor by moving the focusing lens. In the imaging method for obtaining an image of excitation light together, an excitation light irradiation step of irradiating the subject with an excitation light beam, and a fluorescent image generated from the fluorescent dye while moving the focusing lens, A relational expression calculating step for obtaining a relational expression representing a relation between the position of the focusing lens and the focus value of the fluorescent image; and an accuracy index calculating step for calculating a precision index of the relational expression calculated in the relational expression calculating step; The relational expression calculation step is continuously executed until the accuracy index calculated in the accuracy index calculation step reaches a set value, and the accuracy index becomes equal to or greater than the set value. The focusing step of determining a region where the in-focus position exists based on the relational expression calculated in the relational expression calculating step, and moving the focusing lens to a position where the focus value is maximum in this region And including.

According to the first and fourth inventions, even when weak fluorescence is photographed, it is possible to prevent the influence of noise and perform accurate focusing by using an accuracy index. Become.

According to the second invention, it is possible to easily determine the in-focus position using the primary expression.

According to the third invention, the focus value can be easily calculated by the sharpening filter.

1 is a schematic diagram of an imaging apparatus according to the present invention. 2 is a perspective view of an illumination / photographing unit 12. FIG. 2 is a schematic diagram of a camera 21. FIG. It is a block diagram which shows the main control systems of the imaging device which concerns on this invention. 3 is a functional block diagram of a fluorescence image processing unit 32. FIG. It is a flowchart which shows the imaging method which concerns on this invention. It is explanatory drawing of a Laplacian filter. It is a graph which shows typically the relation between the rotation position of the motor for moving the focusing lens, and the focus value at that time.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram of an imaging apparatus according to the present invention.

The imaging apparatus includes an input unit 11 such as a touch panel, and includes a main body 10 incorporating a control unit 30 and the like described later, an illumination / photographing unit 12 supported movably by an arm 13, and a liquid crystal display panel. A display unit 14 and a treatment table 16 on which a patient 17 is placed are provided. The illumination / imaging unit 12 is not limited to the one supported by the arm 13, and may be carried by the surgeon in hand. The display unit 14 may be a glasses-type wearable device instead of a liquid crystal display panel.

FIG. 2 is a perspective view of the illumination / photographing unit 12 described above.

The illumination / photographing unit 12 includes a camera 21 capable of detecting near-infrared rays and visible light, a visible light source 22 including a large number of LEDs disposed on the outer peripheral portion of the camera 21, and an outer peripheral portion of the visible light source 22. And an excitation light source 23 made up of a number of arranged LEDs. The visible light source 22 emits visible light. The excitation light source 23 irradiates near infrared light having a wavelength of 760 nm, which is excitation light for exciting indocyanine green.

FIG. 3 is a schematic diagram of the camera 21.

The camera 21 includes a focusing lens 71 that reciprocates for focusing, a wavelength selection filter 53, a visible light image sensor 51, and a fluorescence image sensor 52. The visible light image sensor 51 and the fluorescence image sensor 52 are composed of a CMOS or a CCD. Visible light and fluorescence incident on the camera 21 coaxially along the optical axis L pass through the focusing lens 71 constituting the focusing mechanism, and then reach the wavelength selection filter 53. Of visible light and fluorescence incident coaxially, visible light is reflected by the wavelength selection filter 53 and enters the visible light image sensor 51. Of the coaxial visible light and fluorescence, the fluorescence passes through the wavelength selection filter 53 and enters the fluorescence imaging device 52. At this time, the visible light is focused on the visible light imaging element 51 and the fluorescence is focused on the fluorescent imaging element 52 by the action of the focusing mechanism including the focusing lens 71.

FIG. 4 is a block diagram showing a main control system of the imaging apparatus according to the present invention.

This imaging apparatus is composed of a CPU that performs logical operations, a ROM that stores operation programs necessary for controlling the apparatus, a RAM that temporarily stores data during control, and the like, and a control unit that controls the entire apparatus 30. The control unit 30 synthesizes the visible image and the fluorescent image with the visible image processing unit 31 for processing the visible image, the fluorescent image processing unit 32 for processing the fluorescent image generated from indocyanine green, and the like. Image synthesizing unit 33.

The control unit 30 is connected to the input unit 11 and the display unit 14 described above. The control unit 30 is connected to an illumination / photographing unit 12 including a camera 21, a visible light source 22, and an excitation light source 23. Further, the control unit 30 is connected to a motor 39 for moving the focusing lens 71 described above. The visible image processing unit 31 in the control unit 30 transmits a control signal to the visible light source 22 and the visible light imaging device 51 and receives an image signal of a visible image from the visible light imaging device 51. The fluorescence image processing unit 32 in the control unit 30 transmits a control signal to the excitation light source 23 and the fluorescence imaging element 52, receives an image signal of the fluorescence image from the fluorescence imaging element 52, and The rotation of the motor 39 that moves the focusing lens 71 is controlled.

When performing an operation using the imaging apparatus having the above-described configuration, indocyanine green is injected into the patient 17 who is supine on the treatment table 16 by injection. Then, near infrared rays are emitted from the excitation light source 23 toward the subject including the affected part, and visible light is emitted from the visible light source 22. As the near infrared light emitted from the excitation light source 23, as described above, 760 nm near infrared light acting as excitation light for indocyanine green to emit fluorescence is employed. As a result, indocyanine green generates fluorescence in the near infrared region having a peak at about 800 nm.

Then, the camera 21 captures the vicinity of the affected part of the patient 17. As described above, the camera 21 can detect fluorescence and visible light. An image obtained by irradiating the patient 17 with visible light and photographing this with the camera 21 becomes a visible image, and an image obtained by irradiating the patient 17 with near-infrared light and photographing fluorescence from indocyanine green with the camera 21 is a fluorescent image. Become. A fluorescent image captured by the camera 21 at a predetermined frame rate is processed by the fluorescent image processing unit 32, and a visible image captured by the camera 21 at a predetermined frame rate is processed by the visible image processing unit 31. Then, the processed fluorescence image data and visible image data are synthesized by the image synthesis unit 33, and a synthesized image in which the visible image and the fluorescence image are fused is created. On the display unit 14, a fluorescent image, a visible image, and a composite image are displayed simultaneously in a divided area or selectively.

In such a configuration, noise is superimposed on an image captured by the visible light image sensor 51 and the fluorescence image sensor 52. The presence of this noise becomes a problem particularly when photographing fluorescence from indocyanine green, which is very weak compared to visible light, when the amplification factor of the signal of the fluorescence image sensor 52 is increased. This will affect the focus operation.

For this reason, in the imaging apparatus according to the present invention, a relational expression representing a relation between the position of the focusing lens 71 and the focus value of the fluorescent image captured by the fluorescent imaging element 52 is calculated by the relational expression calculation unit. The accuracy index of the relational expression calculated by the relational expression calculation unit is calculated by the accuracy index calculation unit, and the in-focus position exists based on the relational expression when the accuracy index calculated by the accuracy index calculation unit is a predetermined value or more. A region is determined, and the focusing lens 71 is moved to a focus position at a position where the focus value is maximum in this region.

FIG. 5 is a functional block diagram of the fluorescent image processing unit 32 described above.

The fluorescent image processing unit 32 includes an arithmetic unit 41 that executes various types of calculations described later, an image memory 48 for storing fluorescent images, a motor driver 49 for driving the motor 39, and a bus controller. 46. The computing unit 41, the image memory 48, and the bus controller 46 are connected by a data bus 47. Further, as a functional configuration, the calculator 41 represents the relationship between the position of the focusing lens 71 by moving the focusing lens 71 by the motor 39 and the focus value of the fluorescence image captured by the fluorescence imaging element 52. A relational expression calculating unit 42 for obtaining a relational expression, an accuracy index calculating part 43 for calculating an accuracy index of the relational expression calculated by the relational expression calculating part 42, and an accuracy index for the relational expression obtained by the relational expression calculating unit 42 When the accuracy index calculated by the calculation unit 43 is equal to or greater than a predetermined value, an area where the in-focus position exists is determined based on the relational expression, and the focus value is set to a position where the focus value is maximum in this area. A lens position control unit 44 for moving the lens 71.

Next, the focusing operation at the time of fluorescent image photographing by the above-described imaging apparatus will be described. FIG. 6 is a flowchart showing a focusing operation at the time of photographing a fluorescent image by the imaging apparatus according to the present invention.

When performing the focusing operation, the motor 39 shown in FIG. 4 is driven to start the movement of the focusing lens 71 shown in FIG. 3 toward the farther side of the both ends of the stroke from the current position (step). S1).

Next, focus values are acquired at n positions while moving the focusing lens 71 in the direction (step S2). This focus value is acquired at n positions with respect to the fluorescence image captured by the fluorescence imaging element 52 in the camera 21 while moving the focusing lens 71. This focus value is calculated by applying a sharpening filter to the fluorescence image captured by the fluorescence image sensor 52.

FIG. 7 is an explanatory diagram of a Laplacian filter as a sharpening filter.

In this embodiment, a 3 × 3 Laplacian filter is used as the sharpening filter. The sharpening filter is also called a sharpening filter, and is a filter that extracts edge information of a captured fluorescent image. The focus value at this position is calculated by applying the sharpening filter shown in FIG. 7 to each pixel of the fluorescence image captured by the fluorescence imaging element 52. Note that the coefficient and matrix size of the Laplacian filter can be changed as appropriate. In addition, another sharpening filter may be used instead of the Laplacian filter.

Referring to FIG. 6 again, next, a relation representing the relationship between the position of the focusing lens 71 and the focus value of the fluorescent image captured by the fluorescent imaging element 52 by the relational expression calculation unit 42 shown in FIG. While calculating | requiring a type | formula as a primary type | formula, the coefficient showing the inclination in this relational expression is determined (step S4). That is, it is determined whether the inclination is in the upward direction, the downward direction, or whether the inclination cannot be determined.

More specifically, first, when obtaining this relational expression, n focus values y _i (i = 1, 2, 3... N) of the fluorescence image captured by the fluorescence imaging element 52 and at that time For the position x _i (i = 1, 2, 3,... N) of the focusing lens 71, fitting is performed using the least square method on the linear function expressed by the following formulas 1 and 2.

At this time, the coefficient a indicating the slope of the linear expression shown in Expression 2 is calculated based on Expression 3 below by the accuracy index calculation unit 43 shown in FIG.

A determination coefficient R ² representing the accuracy of fitting of this mathematical expression is expressed by the following mathematical expressions 4 and 5. The value of the coefficient of determination R ² is a value between 0 and 1, the closer to 1, the higher the statistical confidence.

Then, in the inclination judgment step (step S4), and the coefficient of determination R ² values are predetermined threshold (e.g., 0.8) and above the, and determines the coefficient a is the increase, if a positive value, the coefficient of determination and the value of R ² is above a predetermined threshold value, and determines the coefficient a is a lowered if a negative value, determines the value of the coefficient of determination R ² is impossible determined equal to or smaller than a predetermined threshold value .

When the determination of “down” is made in the tilt determination step (step S4), the moving direction of the focusing lens 71 is set in the reverse direction (step S3), and the above-described step S2 is executed again. In addition, when the determination of “rise” is made in the inclination determination step (step S4), the process proceeds to step S6 described later. On the other hand, when it is determined that “determination is impossible” in the tilt determination step (step S4), one of n data indicating the relationship between the focus value y _i and the position x _i of the focusing lens 71. And the focus data is updated by acquiring the next data (step S5), and step S4 is repeated until the determination becomes possible.

When it is determined that “increase” in the tilt determination step (step S4), one of the n data indicating the relationship with the position x _i of the focus value y _i focusing lens 71 is discarded and the next After the focus data is updated by acquiring the data (step S6), the tilt determination is performed again (step S7). When the determination “rising” is made, or when the determination is “impossible to judge”, the n values of the data indicating the relationship with the position x _i of the focus value y _i focusing lens 71 are obtained. While discarding one, the next data is acquired, and step S6 and step S7 are repeated until a determination of “down” is obtained.

When the determination of “down” is made, it is determined by the lens position control unit 44 shown in FIG. 5 that the in-focus position exists in the region from the rise to the fall. Then, the focusing lens 71 is moved to the position where the focus value is maximized in steps S6 and S7, which is the process from the rise to the fall. As a result, the focusing lens 71 moves to the in-focus position, and the focusing of the fluorescent image is completed.

In the above-described embodiment, the inclination determination is performed by using a linear function. However, other functions such as a quadratic function may be used.

In the above-described embodiment, the focus of the fluorescent image has been described. However, the focus result of the fluorescent image may be used for the visible image. Further, the fluorescent image and the visible image may be focused by separate optical systems.

In the above-described embodiment, the excitation light source 23 that emits near-infrared light having a wavelength of 760 nm is used. However, the excitation light source 23 excites indocyanine green to generate excitation light. Those that irradiate near-infrared light having a wavelength of about 750 nm to 810 nm capable of generating light can be used.

In the above-described embodiment, indocyanine green is used as a material containing a fluorescent dye, and the indocyanine green is irradiated with near-infrared light of 760 nm as excitation light, thereby approximately 800 nm from indocyanine green. Although the case of emitting fluorescence in the near-infrared region having a peak at, light other than near-infrared light may be used.

For example, 5-ALA (5-aminolevulinic acid / 5-aminolevulinic acid) can be used as a fluorescent dye. When 5-ALA is used, 5-ALA that has entered the body of the patient 17 changes to a protoporphyrin IX / PpIX that is a fluorescent substance. When visible light of about 400 nm is irradiated toward the protoporphyrin, red visible light is irradiated as fluorescence from the protoporphyrin. Therefore, when 5-ALA is used, an excitation light source that emits visible light having a wavelength of about 400 nm may be used, and a light source for confirmation emits fluorescence from protoporphyrin. What irradiates red visible light to be used may be used.

DESCRIPTION OF SYMBOLS 10 Main body 11 Input part 12 Illumination / imaging | photography part 13 Arm 14 Display part 16 Treatment table 17 Patient 21 Camera 22 Visible light source 23 Excitation light source 30 Control part 31 Visible image process part 32 Fluorescence image process part 33 Image composition part 41 Calculator 42 Relational expression calculation unit 43 Accuracy index calculation unit 44 Lens position control unit 51 Image sensor for visible light 52 Image sensor for fluorescence 71 Lens for focusing

Claims

An excitation light source that irradiates the subject with an excitation beam for exciting the fluorescent dye injected into the subject;
An imaging device capable of capturing a fluorescent image generated from the fluorescent dye by being irradiated with excitation light; and
A focusing mechanism that focuses the fluorescent image generated from the fluorescent dye by the movement of the focusing lens with respect to the imaging device;
A motor for moving the focusing lens,
In an imaging device that captures a fluorescent image of the subject,
A relational expression calculation unit that obtains a relational expression representing a relation between a position of the focusing lens at that time and a focus value of a fluorescent image captured by the imaging element by moving the focusing lens by the motor;
An accuracy index calculation unit for calculating the accuracy index of the relational expression obtained by the relational expression calculation unit;
When the accuracy index calculated by the accuracy index calculation unit is equal to or greater than a predetermined value, an area where the in-focus position exists is determined based on the relational expression obtained by the relational expression calculation unit, and the focus is within this area A focusing lens position controller that moves the focusing lens to a position where the value is maximized;
An imaging apparatus comprising:
The imaging apparatus according to claim 1, wherein
The relational expression is a linear expression,
The focusing lens position control unit is an imaging apparatus that determines an area where the inclination of a straight line represented by the linear expression is reversed as an area where the in-focus position exists.
The imaging apparatus according to claim 1, wherein
The imaging apparatus in which the focus value is calculated by applying a sharpening filter to a fluorescent image captured by the image sensor.
The excitation light for exciting the fluorescent dye injected into the subject is irradiated toward the subject, and the fluorescence generated from the fluorescent dye is focused on the image pickup device by moving the focusing lens to generate the excitation light. In an imaging method for obtaining an image,
An excitation light irradiation step of irradiating the subject with excitation light;
Taking a fluorescent image generated from the fluorescent dye while moving the focusing lens, a relational expression calculating step for obtaining a relational expression representing a relation between a position of the focusing lens and a focus value of the fluorescent image;
An accuracy index calculation step of calculating an accuracy index of the relational expression calculated in the relational expression calculation step;
The relational expression calculation step is continuously executed until the accuracy index calculated in the accuracy index calculation step becomes a set value, and the relationship calculated in the relational expression calculation step when the accuracy index becomes equal to or greater than the set value. A focusing step of determining a region where the in-focus position exists based on the equation, and moving the focusing lens to a position where the focus value is maximum in this region;
An imaging method comprising: