JP2020110291A - Observation device, observation method, and endoscope apparatus - Google Patents

Abstract

To detect a transition part from a normal tissue to a cancer tissue.SOLUTION: An observation device 100 includes a polarization detection part 40 for detecting a polarization state of light from an object, and a determination part 23a for analysing principal components of a value representing the polarization state so as to determine a state of the object.SELECTED DRAWING: Figure 2

Description

The present invention relates to an observation device, an observation method, and an endoscope device.

Using polarized light, calculate the thickness of the mucosal layer based on the ratio of unpolarized light of the returning light returning from the digestive organs, in particular, the mucosal layer having the polarization anisotropy of the stomach wall, that is, the degree of polarization of the returning light. Therefore, it is known that a change in the thickness of the mucosal layer of the stomach wall can be detected, and thereby the degree of cancer invasion can be diagnosed (see Patent Document 1).

Japanese Patent No. 5501155

However, with the technique disclosed in Patent Document 1 described above, it is difficult to detect a transition part in the state of a transition process from a non-cancerous tissue, that is, a normal tissue to a cancerous tissue.

According to the first aspect, the observation device determines the state of the object by performing a principal component analysis on the polarization detection unit that detects the polarization state of the light from the object and the value indicating the polarization state. With.
According to the second aspect, the observation method determines the state of the object by detecting the polarization state of the light from the object and performing a principal component analysis on the value indicating the polarization state.

It is a perspective view showing the important section of the endoscope device to which the observation device which is an embodiment is applied. It is a block diagram showing a schematic structure of an endoscope system to which an observation device which is an embodiment is applied. It is sectional drawing which shows the principal part of the observation apparatus which is embodiment. It is a perspective view showing the important section of the observation device which is an embodiment. It is a perspective view showing the important section of the observation device which is an embodiment. It is a figure which shows schematic structure of the front-end|tip part of the insertion part of the endoscope apparatus which concerns on embodiment. It is a figure which shows the relationship of the polarization state of the light radiate|emitted from the polarization generation part in the observation apparatus which is embodiment. It is a figure which shows the relationship of a normal tissue, a cancer tissue, and a transition part. It is a figure for demonstrating the method of the principal component analysis by the observation apparatus which is embodiment. It is a figure for demonstrating the method of the principal component analysis by the observation apparatus which is embodiment. It is a figure for demonstrating the method of the principal component analysis by the observation apparatus which is embodiment. 6 is a flowchart for explaining the operation of the observation device according to the embodiment. It is the figure which expanded the three-dimensional space represented by the 2nd principal component, the 3rd principal component, and the 4th principal component which are the results of principal component analysis by the Mercator projection.

An endoscope apparatus and an endoscope system to which an observation apparatus according to an embodiment is applied will be described with reference to FIGS. 1 and 2.

FIG. 1 is a perspective view showing a main part of an endoscope apparatus to which an observation apparatus according to an embodiment is applied, and FIG. 2 shows a schematic configuration of an endoscope system to which the observation apparatus according to the embodiment is applied. It is a block diagram shown.

In these figures, the endoscope system 1 according to the present embodiment has an endoscope device 10 and a main body 20.

In the endoscope device 10, the distal end portion 12 of the insertion unit 11 is inserted into the body cavity of a subject such as a human body and irradiates an object such as the inner surface of the mucous membrane in the subject with light. In addition, the endoscope device 10 images the return light from the target object and acquires the image signal of the target object image.

The main body section 20 has a light source 26 that emits light emitted from the distal end section 12 of the endoscope apparatus 10, and performs various image processing on the image signal acquired by the endoscope apparatus 10.

As shown in FIG. 1, the endoscope device 10 includes an insertion portion 11, an operation portion 13, and a connector portion 14. The insertion portion 11 has a flexible (that is, soft) tube member 11a and has an elongated shape. The operation unit 13 operates a bending operation and an observation operation performed by the distal end portion 12 of the insertion unit 11. The connector unit 14 detachably connects the endoscope device 10 to the main body unit 20.

As shown in FIG. 2, the main body unit 20 includes a control unit 21, a storage unit 22, an image processing unit 23, a detection unit driver 24, a light source driver 25, a light source 26, a condenser lens 27, and a display. It has a controller 28 and an input interface (I/F) 29. The main body 20 is connected to the endoscope device 10 by an optical connector 30 and an electric connector 31.

The control unit 21 has an arithmetic element such as a CPU, and a control program (not shown) stored in the storage unit 22 is read at the time of startup and executed by the control unit 21. Thereby, the control unit 21 controls the entire endoscope system 1 including the image processing unit 23, the detection unit driver 24, the light source driver 25, the display controller 28, and the input interface 29.

The storage unit 22 includes a large-capacity storage medium such as a hard disk drive and a semiconductor storage medium such as ROM and RAM. The storage unit 22 stores the above-described control program and also temporarily stores various data required for the control operation of the control unit 21. Further, the storage unit 22 stores the image data processed by the image processing unit 23.

The image processing unit 23 acquires an image signal of the object image captured by the detection unit 50 of the endoscope device 10 and performs various image processing on the image signal. The image processing unit 23 outputs the image processing result to the display controller 28.

The image processing unit 23 has a determination unit 23a. The determination unit 23a performs a principal component analysis on the value indicating the polarization state of the light from the target object based on the image signal of the target object image captured by the detection unit 50. Then, the determination unit 23a determines the state of the target object based on the result of the principal component analysis.

Details of the image processing performed by the detection unit 50 and the image processing unit 23 will be described later.

The detector driver 24 generates a signal for driving the detector 50 and supplies the signal to the detector 50.

The light source 26 generates light emitted from the tip portion 12 of the endoscope device 10, and is driven by the light source driver 25. The light source 26 according to the present embodiment generates light with a narrow band, low spatial coherence, and speckles removed. The wavelength band of the light generated by the light source 26 may be selected from one wavelength or a plurality of wavelengths between 350 nm and 850 nm depending on the purpose.

However, it is known that the light absorption spectrum of hemoglobin contained in the biological tissue has a peculiar mountain near 400 nm. For this reason, the light from the light source 26 is preferably light in the wavelength band of 450 nm or more.

The bandwidth of the light generated by the light source 26 is 5 nm or less, preferably about 3 nm. Due to bandwidth concerns, such a light source 26 preferably comprises a laser light source. On the other hand, a narrow band light can be similarly generated even if an LED light source is added with a bandpass filter.

The wavelength band and bandwidth of the light generated by the light source 26 can be determined by the number and width of interference fringes, which will be described later, generated on the image sensor 57 of the detection unit 50.

The light generated by the light source 26 is converged into a light flux having a constant width by the condenser lens 27, and is guided through the optical connector 30 into the optical fiber 15 of the endoscope device 10.

Here, only one light source 26 is provided in the main body portion 20 of the endoscope system 1 according to the present embodiment. On the other hand, as will be described later, the number of optical fibers 15 is equal to the number of polarization generation units 60 of the polarization detection unit 40 described later. In the present embodiment, the number of polarization generation units 60 is six, so the number of optical fibers 15 is six. Therefore, a switching unit 26a is provided between the light source 26 and the condenser lens 27. The switching unit 26a guides the light from the light source 26 into any one of the optical fibers 15. The switching unit 26a blocks light to the optical fibers 15 other than the optical fiber 15 that guides light.

The display controller 28 generates a display drive signal for displaying the image signal output from the image processing unit 23 on the screen of the display 32 provided outside the main body 20, and supplies this to the display 32. ..

The input interface 29 receives an operation input signal from an input device 33 such as a keyboard provided outside the main body unit 20 and supplies it to the control unit 21.

The polarization detection unit 40 of the observation device according to the present embodiment is provided at the distal end portion 12 of the insertion unit 11 of the endoscope device 10, as shown in FIGS. 3 and 6.

The polarization detector 40 will be described below with reference to FIGS. Note that, in the following description, the optical element and the optical path are mainly described, and therefore, in FIGS. 3 to 7, other configurations such as an electric circuit that connects the detection unit 50 and the main body unit 20 and a coating of the endoscope device 10 are provided. A detailed description of is omitted.

The polarization detection unit 40 is arranged around the detection unit 50 that detects light from the target object, and six polarization generation units that generate light having different polarization states and irradiate the target object. 60 and. As shown in FIGS. 3 and 6, the detection unit 50 and the polarization generation unit 60 are housed in the tube member 11a of the insertion unit 11 of the endoscope device 10.

Note that, as illustrated in FIG. 3 and subsequent figures, the x axis is taken in the optical axis direction of the detection unit 50. Then, the positive direction of the x-axis is the direction toward the object.

As shown in FIG. 3, the detection unit 50 includes a front lens group 51, a diaphragm 52, a polarization separation unit 53, an imaging lens 54, a polarizing plate 55, a protective glass 56, and an image sensor 57.

The front lens group 51 is arranged on the object side (right end in FIG. 3) in the detection unit 50. The diaphragm 52 is formed in an annular shape and is arranged on the emission side of the front group lens 51. The light emitted from the diaphragm 52 is incident on the polarization separation unit 53. The image forming lens 54 is a rear lens group that forms an image of interference fringes of the light emitted from the polarization splitting unit 53 on an image pickup surface 57a of an image sensor 57 described later. The polarizing plate 55 and the protective glass 56, which are analyzers, are sequentially arranged in front of the imaging surface 57a of the image sensor 57 (right side in FIG. 6). The image sensor 57 is an image pickup element having a plurality of pixels, and picks up an image of the interference fringes formed on the image pickup surface 57a.

The detection unit 50 is designed so that at least a part of the common part of the illumination range of the object by the six polarization generation units 60 described later will be the detection range, and more preferably, the six polarization generation units. The imaging range is set to be wider than the entire illumination range of 60. Alternatively, depending on the detection range of the detection unit 50, the polarization generation unit 60 may be designed so that this detection range is at least a part of the common part of the illumination range of the object by the polarization generation unit 60.

The respective emission angles of the plurality of polarization generation units 60 may be set so that the generated polarization is irradiated to the detection range of the detection unit 50. In this case, the arrangement state of each of the plurality of polarization generation units 60 may be adjusted. By setting the illumination range of the object by the polarization generation unit 60 and the detection range of the detection unit 50 in this way, the detection unit 50 can reliably detect the return light from the irradiation range of the object.

The polarization splitting section 53 has two Savart plates 53c and a ½ wavelength plate 53d arranged between the Savart plates 53c. The Savart plate 53c in the present embodiment includes two parallel plane plates 53a (for example, YVO ₄ ) made of uniaxial crystal having birefringence and a half-wave plate arranged between the parallel plane plates 53a. 53b.

The Savart plate 53c is formed by bonding a pair of parallel plane plates 53a having birefringence so that their optical axes differ by 90°. When the light incident on the Savart plate 53c is light in which different polarization states are overlapped, the Savart plate 53c separates and outputs the light beams having different polarization states.

The light emitted from the polarization splitting unit 53 is converged by the imaging lens 54, and is imaged on the imaging surface 57a of the image sensor 57 through the polarizing plate 55 and the protective glass 56. Then, lights having different polarization states interfere with each other on the imaging surface 57a to form interference fringes.

The image sensor 57 images the light imaged on the imaging surface 57a and outputs the result as an image signal. Since the image sensor 57 images the interference fringes formed on the image pickup surface 57a as described above, it has a resolution capable of picking up fine interference fringes. The number and width of the interference fringes formed on the imaging surface 57a depend on the wavelength band and bandwidth of the light generated by the light source 26, as described above.

As shown in FIGS. 4 and 5, each of the plurality of polarization generation units 60 includes one polarizing plate (polarization conversion element) 61a to 61f and a plano-concave lens (arranged on the exit side of the polarizing plates 61a to 61f ( Divergence optical element) 62. Further, a quarter-wave plate (polarizing optical element) 63 is arranged between some of the polarizing plates 61 a and 61 b and the plano-concave lens 62. In the case where the quarter-wave plate 63 is arranged, the polarization generation unit 60 is configured by the single polarizing plates 61 a and 61 b, the quarter-wave plate 63, and the plano-concave lens 62. In the following description and the drawings, when the positions of the polarizing plates 61a to 61f are generally shown without being specified, the reference numeral 61 is representatively described.

As shown in FIGS. 4 and 5, the six polarization generation units 60 are divided into two groups and are arranged around the detection unit 50. More specifically, the three polarization generation units 60 are arranged above the polarization separation unit 53 in the drawing, and the three polarization generation units 60 are arranged below the polarization separation unit 53 in the drawing. Further, the six polarization generation units 60 are arranged in the same plane orthogonal to the x-axis which is the optical axis direction of the detection unit 50.

Further, as shown in FIGS. 4 and 5, when viewed along the x-axis, a part of each of the plurality of polarization generation units 60 overlaps a part of the image sensor 57. This makes it possible to make the outer diameter of the polarization detection unit 40 of the present embodiment viewed from the optical axis direction of the detection unit 50 more compact. As a result, it is possible to contribute to downsizing of the polarization detection unit 40 as a whole. It is sufficient that at least a part of at least one of the plurality of polarization generation units 60 is arranged so as to overlap the image sensor 57 in the optical axis direction of the detection unit 50.

The six polarization generation units 60 irradiate the object with lights having different polarization states. This point will be described in detail below.

The polarization state of light can be described by a Stokes vector, and this Stokes vector can be expressed by a Stokes matrix of 4 rows and 1 column as in the following equation.

In each component of the Stokes matrix, s ₀ is light intensity, s ₁ is xy linearly polarized light, s ₂ is 45° linearly polarized light, and s ₃ is circularly polarized light. Since s ₀ represents intensity, it takes a positive value. s ₁ =s ₀ means 0° linearly polarized light, and s ₁ =−s ₀ means 90° linearly polarized light. Further, s ₂ =s ₀ means 45° linearly polarized light, and s ₂ =−s ₀ means −45° linearly polarized light. Further, s ₃ =s ₀ means right-handed circularly polarized light, and s ₃ =−s ₀ means left-handed circularly polarized light.

The respective polarization generation units 60, more specifically, the respective combinations of the polarizing plates 61a to 61f and the quarter wave plate 63, are composed of six types of s ₁ =±1, s ₂ =±1, and s ₃ =±1. It is set to generate light having any one of the polarization states.

Specifically, in the six polarization generation units 60, as shown in FIG. 7, the polarization plates 61a to 61f are arranged so that the Stokes components of the light emitted from the respective polarization generation units 60 become the illustrated components. The position of the quarter wave plate 63 selected is determined.

It is possible to arbitrarily determine how the polarization state of the light emitted from each polarization generation unit 60 is made different, in other words, what kind of Stokes component is emitted.

Here, since the Stokes component s ₁ has a smaller change in the polarization state with respect to the angles than the other components s ₂ and s ₃ , the polarizing plate 61c and the Stokes component s ₁ =− that irradiate the light having the Stokes component s ₁ =+1. There is no problem even if the polarizing plates 61f for irradiating light having 1 are arranged apart from each other.

In addition, regarding the polarizing plates 61a and 61b and the quarter-wave plate 63 that irradiate the light having the Stokes component s ₃ =±1, the quarter-wave plate 63 arranged on the emission side of these polarizing plates 61a and 61b. There is an advantage in arranging these polarizing plates 61a and 61b side by side from the viewpoint that they can be shared and a holder (not shown) that holds the quarter-wave plate 63 can be shared.

The lens 62 is a plano-concave lens that diverges the light emitted from the polarizing plate 61 and the quarter-wave plate 63 and illuminates a predetermined area of the object. Here, the plano-concave lens 62 is optically designed so that the half field angle of the light emitted from the plano-concave lens 62 is within 15 degrees. This is because if the half angle of view is within 15 degrees, the polarization state of the light emitted from the polarization generation unit 60 does not change significantly. Further, even if the numerical aperture (NA: numerical aperture) of the optical fiber 15, which will be described in detail later, is taken into consideration, the half field angle of the light emitted from this optical fiber 15 is also within 15 degrees.

The plano-concave lens 62 as a diverging optical element used in the polarization generation unit 60 has better polarization performance than other optical elements that also have a light divergence effect, such as a ball lens. It is suitable.

Light is guided to the polarization generation unit 60 by the optical fiber 15 of the endoscope device 10, as shown in detail in FIGS. 3 and 6. As shown in FIG. 3, the optical fiber 15 extends in the tube member 11 a of the insertion portion 11 of the endoscope device 10 along the longitudinal direction of the insertion portion 11 and reaches the distal end portion 12 of the insertion portion 11. The exit end 15a is arranged near the entrance end (the left end in FIG. 3) of the polarizing plate 61.

The optical fiber 15 eliminates the polarization state of the light from the light source 26 (polarization scrambles) and guides it to the polarization generation unit 60. In the present embodiment, as shown in FIG. 6, the optical fiber 15 is a multimode optical fiber in which the core 15b has a substantially square cross section and the cladding 15c has a substantially circular cross-sectional outer shape. The optical fiber 15 having such a configuration can supply depolarized and stable light to the incident surface of the polarizing plate 61, and therefore the polarization state that has passed through the polarizing plate 61 maintains a spatially and temporally stable light intensity. be able to.

In the present embodiment, the endoscope device 10 is provided with the same number of the polarization generation units 60, that is, six optical fibers 15 (only two are shown in FIG. 3), and these optical fibers 15 are provided. Light is commonly guided from a single light source 26. Therefore, the polarized light generation unit 60 irradiates the object with light having different polarization states in a time division manner.

Each element that constitutes the polarization detection unit 40 according to the present embodiment is formed of a material that can maintain high polarization measurement accuracy even when sterilized in a high temperature environment. Specifically, the half-wave plates 53b and 53d and the quarter-wave plate 63 are made of quartz. The plano-concave lens 62 is made of optical glass. Further, the polarizing plate 61 is made of an inorganic material or a wire grid. The plane parallel plate 53a forming the Savart plate 53c is made of YVO ₄ .

Here, the autoclave for sterilization is performed under conditions of, for example, 115° C. for 30 minutes, 121° C. for 20 minutes, 126° C. for 15 minutes, and 134° C. for 10 minutes.

The observation apparatus 100 according to the present embodiment is composed of the polarization detection unit 40 and the image processing unit 23, especially the determination unit 23a.

Next, the principle of the method for measuring the Stokes components s _{0 to} s ₃ by the endoscope system 1 using the polarization detection unit 40 will be described (K. Oka and N. Saito, "Snapshot complete imaging polarimeter using Savart plates", See SPIE 6295-9, 1 (2006)).

The intensity distribution of the light imaged by the image sensor 57 is I(x, y). If the two-dimensional distributions of the Stokes components contained in the measurement light are s ₀ (x, y), s ₁ (x, y), s ₂ (x, y), and s ₃ (x, y), these Stokes components are The light intensity distribution is expressed by using the following two-dimensional distribution.

Here, arg is a function indicating the argument of a complex number, and U ₁ and U ₂ are spatial carrier frequencies introduced by the two Savart plates 53c, respectively.

S ₀ (x, y), s ₂ (x, y), and s ₁₃ (x, y) in the above equation are different from each other (especially for s ₁₃ (x, y), a real number component and an imaginary number component) 4 Since it has one spatial carrier frequency f _y =0, U ₂ , U ₂ −U ₁ , and U ₂ +U ₁ , it is obtained by spatial frequency filtering the light intensity distribution I(x,y). Then, the two-dimensional distribution of these Stokes components can be obtained from the amplitude and phase of the extracted components. At this time, the spatial frequency filtering and the modulation of the amplitude and the phase can be performed at one time by the Fourier transform technique which has a shape suitable for the modulation of the two-dimensional distribution of the Stokes component.

Therefore, the image processing unit 23 (including the determination unit 23a) of the endoscope system 1 acquires the two-dimensional distribution of the intensity of the image signal output from the detection unit 50, and Fourier transforms the two-dimensional distribution of the intensity. Then, the polarization state of the return light from the object, specifically, the two-dimensional distribution of the Stokes components s _{0 to} s ₃ is obtained.

Then, the determination unit 23a of the image processing unit 23 uses the two-dimensional distribution of the Stokes components s _{0 to} s ₃ to obtain the polarization degree of the object. Then, the determination unit 23a calculates the thickness of the mucosal layer of the target object based on the degree of polarization, and thereby diagnoses the degree of cancer invasion.

Let the Stokes matrix of the incident light on the object be S=(s ₀ , s ₁ , s ₂ , s ₃ ), and the Stokes matrix of the return light from this object be S′=(s′ ₀ , s′ ₁ , s′ ₂ and s′ ₃ ), the relation between these Stokes matrices is represented by a Mueller matrix M of 4 rows and 4 columns. That is,

Here, although it is difficult to make a strict correspondence between each of the 16 elements m _{00 to} m ₃₃ of the Mueller matrix M and the physical characteristics of the polarized light, the rough relationship is that the element m ₀₀ represents luminance and all 16 elements Elements m _{00 to} m ₃₃ of the above represent the degree of polarization, elements m ₀₁ , m ₀₂ , m ₁₀ and m ₂₀ represent dichroism (linear double absorption), and elements m ₀₃ and m ₃₀ are circular dichroism (circle). Double absorption), the elements m ₁₁ , m ₁₂ , m ₂₁ and m ₂₂ represent optical activity (circular birefringence), and the elements m _{11 to} m ₁₃ , m _{21 to} m ₂₃ and m _{31 to} m ₃₃ are birefringent. It represents the property (linear birefringence).

As described above, according to the polarization detection unit 40 of the present embodiment, six types of light having different polarization states, in other words, Stokes having different Stokes components (all Stokes components are known) are used. It is possible to detect the Stokes component of the return light from the object, which is obtained by making light represented by a matrix (which is referred to as S _{1 to} S ₆ ) incident on the object and reflecting each light. Stokes matrix of Stokes component placing and _{S '1} ~S' _6). Since the Mueller matrix M of the object is the same, each component of the Mueller matrix M can be obtained from these Stokes matrices S _{1 to} S ₆ and S′ _{1 to} S′ ₆ .

That is, in the polarization detecting unit 40 according to the present embodiment, when four of the six polarization generating units 60 are used, each component of the Mueller matrix M is uniquely determined by solving four determinants. Further, if all the six polarization generation units 60 are used, each component of the Mueller matrix M can be obtained by using the least square method.

Then, since the two-dimensional distribution of the Stokes components of the Stokes matrix S′=(s′ ₀ , s′ ₁ , s′ ₂ , s′ ₃ ) of the returning light is known, each component of the Mueller matrix M of the object is known. A two-dimensional distribution can also be obtained. Thereby, the two-dimensional distribution of the polarization degree of the object can be obtained. Then, the two-dimensional distribution of the thickness of the mucosal layer of the target object is calculated based on the two-dimensional distribution of the polarization degree, whereby the invasion degree of cancer, which is one of the states of the target object, can be diagnosed. ..

In addition, according to the determination unit 23a included in the observation device 100 according to the present embodiment, not only the normal tissue and the cancer tissue are determined based on the above-described principle, but also the normal tissue is changed to the cancer tissue. It is possible to detect the transition part in the state of being open. The detection principle will be described below.

FIG. 8 is a diagram showing a relationship between a cancer tissue, a normal tissue, and a transition portion, taking the digestive tract as an example of an object. 8(a) and 8(b) each show a state in which the digestive tract is incised in the longitudinal direction and expanded.

As shown in FIG. 8A, it is assumed that there is a region 201 where cancer tissue exists on the inner surface of the digestive tract 200. The doctor visually confirms the cancer tissue, or cuts out the biological tissue on the inner surface of the digestive tract 200 as a sample, performs a pathological examination, and then makes a definitive diagnosis. In the figure, reference numeral 202 indicates a sampling site of the cancer tissue.

Cancer tissue spreads to the periphery as the cancer cells grow. After confirming the region 201 in which the cancer tissue exists, the doctor needs to prevent the region 201 from expanding and further prevent metastasis to other than the digestive tract. Therefore, the doctor sets an excision region 203 including normal tissue existing around the region 201 (left-right direction in the drawing), and excises the digestive tract in the excision region 203.

In FIG. 8A, reference numeral 204 indicates a sample collection place diagnosed as a normal tissue, and reference numeral 205 indicates a sample collection place diagnosed as a transition part.

As described above, the determination as to whether the tissue is a normal tissue or a cancer tissue can be performed based on the values of 16 elements of the Mueller matrix M, in other words, each component. On the other hand, the determination as to whether or not it is a transition part could not be made directly from the values of 16 elements of the Mueller matrix M.

Therefore, the inventors performed various analyzes based on the values of the 16 elements of the Mueller matrix M actually calculated from the normal tissue, the cancer tissue, and the transitional part, and as a result, based on a predetermined procedure, whether it is the transitional part or not. We have come to the knowledge that it is possible to judge.

More specifically, a predetermined region of the object is irradiated with polarized light from each of the six polarization generation units 60, and the image sensor 57 receives light from the irradiated region. The determination unit 23a calculates the Mueller matrix M for each pixel using a plurality of images (six images in this embodiment) of the image sensor 57. As a result, the Mueller matrix M is acquired by the number of pixels that have received the light from the irradiation area of the object. The determination unit 23a regards each of the 16 elements of the acquired Mueller matrix M as 16th-order data. As a result, 16th-order data of the same number as the number of pixels receiving the light are obtained. The determination unit 23a represents each 16-dimensional data (Muller matrix) by linear combination of 16 principal component vectors by performing a principal component analysis using this 16th-order data group.

The outline of the principal component analysis performed by the determination unit 23a will be described below. Since the principal component analysis itself is a known method, only the outline will be described.

Principal component analysis is multidimensional (n-dimensional) data

When there is a variance-covariance matrix of each component


This is a multivariate analysis method in which coordinate transformation is performed so that becomes a diagonal matrix (no correlation).

A suitable orthogonal matrix

And x

Convert to a new variable by. At this time, each data

Is represented. Variance-covariance matrix of new variable y


Principal component analysis is to find an orthogonal matrix P such that is a diagonal matrix. At this time, the diagonal components (ε ₁₁ ,..., ε _nn ) (which are eigenvalues of the variance-covariance matrix) are 0 or a positive number due to the property of the variance-covariance matrix (symmetric matrix).

Are referred to as the first principal component (vector),..., Nth principal component (vector) in order of increasing eigenvalues of the variance-covariance matrix.

New variable

Is called the score of the principal component (vector).

Since the eigenvalues of the variance-covariance matrix are the variances of the new variable, a small value means that the contribution of all data is small. Therefore, even if the score that is smaller than the others is ignored, the information that the data has is hardly lost.

In the case of measurement data, if the inherent value is the degree of measurement error, the variable can be regarded as noise.

Therefore, decide an appropriate dimension m<n with less information loss,

If it is approximated by, even if the n-dimensional data is dimensionally compressed into the m-dimensional data, the information of the original data is hardly lost.

It is necessary to confirm that the Mueller matrix compressed by principal component analysis has optical meaning. The method described below is used as the confirmation method. As a target of dimension compression, a physical quantity that can be converted from a Mueller matrix, for example, a covariant matrix or a coherent matrix, or a matrix composed of absolute values of its matrix elements is also effective.

Therefore, the determination unit 23a of the observation device 100 according to the present embodiment expresses the data (Muller matrix) by using a part of the 16 principal component vectors obtained by the principal component analysis, so that 16-dimensional Also projects into a low-dimensional space. That is, it is converted into a value of a specific dimension lower than 16 dimensions by the principal component analysis. As a result, the 16-dimensional value that is originally the dimension of the Mueller matrix is compressed into a lower-dimensional value. In the present embodiment, n=16 and m=4 in the above description, that is, a 16-dimensional value is compressed into a 4-dimensional value. Next, the determination unit 23a expresses the data by using the principal component vector (the fifth to 16th principal component vectors in this embodiment) not used for the compression. That is, the determination unit 23a calculates a value of five dimensions or more (hereinafter, referred to as a residual), which is a value obtained by removing the values of one to four dimensions from the values of one to six dimensions. Here, the 1- to 4-dimensional values are values obtained by adding the first term including the first principal component to the fourth term including the fourth principal component on the right side of Expression 9. Then, the determination unit 23a compares this residual with a predetermined threshold value, and when the residual becomes equal to or greater than the threshold, the region in which the 16 elements of the Mueller matrix M used to calculate the residual are obtained Judge as the transition part.

The determination unit 23a determines whether or not it is a transition portion, instead of determining whether or not the entire irradiation area of the object is a transition portion, a residual difference is obtained for each of a plurality of pixels that receive light from the irradiation area. May be obtained, and the residual difference may be compared with the threshold value for each pixel, and it may be determined that the portion of the irradiation region corresponding to the pixel having the residual difference equal to or larger than the threshold value is the transition portion.

The experimental results of the inventors have revealed that the four-dimensional value includes, on average, 95% or more of polarization information for determining whether the tissue is a cancer tissue or a normal tissue. On the other hand, when the residuals obtained by removing the 1st to 4th dimension values from the 1st to nth dimension (n>4) values are compared with the normal tissue, the cancer tissue and the transition portion, this residual error is large only in the transition portion Was also clarified by the experimental results of the inventors. Therefore, it is possible to calculate the residuals based on the 1- to 4-dimensional values obtained as the result of the principal component analysis, and to determine the tissue having the residuals equal to or larger than the threshold as the transition portion.

The threshold used for determining whether or not it is a transition part may be set as follows, for example. The histogram of each element of the Mueller matrix M calculated for each of the plurality of pixels of the image sensor 57 is taken. Then, a multiple of the maximum width of this histogram with respect to the maximum width of the histogram when normal (non-cancerous) is set as the threshold value. The multiple is, for example, double or triple. The histogram is created for each of a plurality of elements of the Mueller matrix M, and the value of the element is on the horizontal axis and the frequency is on the vertical axis. In each element, the same number as the number of pixels used in the principal component analysis or The same value as the number of pixels included in a specific image is used.

According to the method described above, the image processing unit 23 including the determination unit 23a can obtain the two-dimensional distribution of 16 elements of the Mueller matrix M, so the determination unit 23a can also obtain the two-dimensional distribution of the transition part. .. As a result, as shown in FIG. 8B, the transition region 206 where the transition portion exists can be determined and detected as an image.

The operation of the determination unit 23a of the observation device 100 according to this embodiment will be described with reference to FIGS. 9 to 11. However, since it is difficult to represent a four-dimensional value in a three-dimensional manner, a case where the three-dimensional value is compressed into a two-dimensional value by principal component analysis will be described.

FIG. 9 is a diagram showing the distribution of the components of the three elements m ₁₁ , m ₂₂ , and m ₃₃ out of the 16 elements of the Mueller matrix M. Note that the examples shown in FIGS. 9 to 11 are virtual and different from actual values. In FIG. 9, the component 300 is assumed to have a distribution as shown. The axes of the first principal component, the second principal component, and the third principal component are also shown in the figure.

FIG. 10A is obtained by rewriting the distribution shown in FIG. 9 using the axes of the first principal component, the second principal component, and the third principal component which are orthogonal to each other. Since FIG. 10A is a three-dimensional (three-dimensional) diagram, a plane (two-dimensional) in which the plane including the axis of the first principal component and the axis of the second principal component matches the plane of the figure. FIG. 10(b) can be obtained by rewriting it into a "target". Further, when the plane including the axis of the first principal component and the axis of the third principal component is rewritten into a planar view such that it coincides with the plane of the drawing, it becomes as shown in FIG.

FIG. 10C is considered to be an expression in which the dimension is compressed from three dimensions to two dimensions by so-called principal component analysis. Therefore, the components sandwiched by the arrows in the figure correspond to residuals of values of two or more dimensions.

According to the analysis by the inventors, as shown in FIG. 11(a), the two-dimensional or more residuals (indicated as two-dimensional residuals in the figure) for the normal tissue and the cancer tissue are small. On the other hand, as shown in FIG. 11(b), the two-dimensional residual at the transition portion is large. Therefore, the determination unit 23a sets a threshold that can distinguish the two-dimensional residual shown in FIG. 11A from the two-dimensional residual shown in FIG. 11B, and determines whether the two-dimensional residual is equal to or greater than the threshold. The transition part is determined based on whether or not it is.

The inventors presume the reason why the transition part can be determined by the above-described method as follows.

The size of the nucleus of the cells that constitute normal tissue is almost constant. Although the size of the nucleus of the cells that make up the cancer tissue is larger than that of the normal tissue, it is also almost constant. Therefore, it is considered that the normal tissue and the cancer tissue each act as a polarizer having a specific polarization state. Since the normal tissue and the cancer tissue have different polarization states, they can be determined based on the values of 16 elements of the Mueller matrix M.

On the other hand, it is considered that cells that can be regarded as normal tissue and cells that can be regarded as cancer tissue are mixed in the transition portion. Therefore, it is presumed that when the transition part is irradiated with light, complicated scattered light is generated. This scattering is considered to be so-called Mie scattering because the nucleus of the cell is about 10 times the wavelength of light. The scattering intensity of Mie scattering depends on the particle size (here, the size of the nucleus). Therefore, it is not easy to theoretically obtain the scattering intensity from a system in which nuclei having different diameters are nonuniformly distributed.

Moreover, since the cells that can be regarded as normal tissues and the cells that can be regarded as cancer tissues are unevenly distributed, the scattering is not uniform. Therefore, variations in the values of the 16 elements of the Mueller matrix M become large, and it is difficult to determine that the 16 elements have the same values as the transition portion.

Therefore, it is considered that the transition part can be relatively easily determined by extracting the variation in the value in the form of a residual and determining whether or not it is the transition part due to the large variation.

Next, the operation of the endoscope system 1 including the observation device 100 according to the present embodiment will be described with reference to the flowchart in FIG.

First, in step S1, the light source driver 25 drives the light source 26, the light source 26 generates light, and the polarization generation unit 60 generates light in a predetermined polarization state to illuminate the object. In step S2, the return light from the object is captured by the image sensor 57 of the detection unit 50.

In step S3, the image processing unit 23 calculates a two-dimensional distribution of 16 elements of the Mueller matrix M for each pixel of the image sensor 57 based on the image pickup signal captured by the image sensor 57 in step S2.

In step S4, the determination unit 23a plots the values of 16 elements of the Mueller matrix M obtained in step S3 in the 16-dimensional space. In step S5, the determination unit 23a converts the 16 element values of the Mueller matrix M plotted in step S4 into four-dimensional values by principal component analysis. In step S6, the determination unit 23a calculates the residual difference based on the four-dimensional values obtained in step S5.

In step S7, the determination unit 23a sets the threshold value for determination in step S7. Note that this threshold may be set in advance. In step S8, the residual obtained in step S6 is compared with the threshold, information below the threshold is removed, and information below the threshold is removed to obtain information of the remaining value, that is, pixels above the threshold. Pixels above this threshold correspond to transition regions. In step S9, the image processing unit 23 images the value obtained in step S8. The image obtained in step S9 is an image showing the distribution of the transition portion. The image showing the distribution of the transition portion is formed, for example, by displaying pixels in which the residual is equal to or more than the threshold value in red in the original image, thereby indicating the transition region in red. In this case, you may judge the boundary of the transition area and draw an outline to make the red part semi-transparent, or remove the information below the threshold value and change the color density according to the size of the remaining value. May be.

The observation device 100 according to the present embodiment configured as described above includes the polarization detection unit 40 that detects the polarization state of the light from the object and the object by performing the principal component analysis of the value indicating the polarization state. And a determination unit 23a that determines the state of.

Therefore, the transition portion can be accurately determined and detected by using the determination result of the determination unit 23a.

Further, according to the observation device 100 of the present embodiment, the polarization state of the object can be observed without using a complicated mechanism. Furthermore, in the observation device 100 according to the present embodiment, downsizing of the entire device can be realized.

The inventors have found that optical entropy can be calculated based on the values of 16 elements of the Mueller matrix M, and normal tissue and cancer tissue can be discriminated by the value of this entropy. ..

The procedure for calculating the entropy using the eigenvalue obtained by performing the eigenvalue decomposition after converting the Mueller matrix into the coherent matrix is as follows.

First, the Mueller matrix M (4×4 real number matrix) is converted into a coherent matrix C (4×4 complex number matrix). In the following description, the 16 elements of the Mueller matrix M are expressed as the following equation.

The coherent matrix C is expressed by the following equation.

The 16 elements of the coherent matrix C are given by the following equation using the 16 elements of the Mueller matrix M.

Next, the coherent matrix C is diagonalized and decomposed into eigenvalues.

Then, using the components λ _{1 to} λ ₄ of this square matrix Λ, the entropy H _T of the coherent matrix C is calculated by the following equation.

Since the coherent matrix C is a Hermitian matrix, its eigenvalue is always a real number. When λ _i <0, λ _i =0. If a coherent matrix with negative eigenvalues is obtained, the original Mueller matrix M has no optical meaning. This property is used to judge whether the Mueller matrix M has an optical meaning.

Based on the above description, calculation of entropy H _T from Mueller matrix M, the entropy H _T based on 16 elements of the Mueller matrix M obtained from the cancer tissue, the Mueller matrix M obtained from other tissues 16 Experimental results were obtained that were lower than the element-based entropy H _T.

It was also found that the sum of the mean value of the entropy of the cancer tissue and the unbiased standard deviation is smaller than the difference between the mean value of the entropy of the normal tissue and the unbiased standard deviation. Here, the average value of the entropy is the average value of the entropy calculated for each pixel for each image, and is calculated for each image by (sum of [entropy of each pixel])/(total number of pixels). Unbiased standard deviation is one index of the spread of data, and is the square root of {(([entropy of each pixel]-[average of entropy]] ² ) / (total number of pixels ^-1 )} Is calculated for each image as.

Therefore, in the observation device 100 according to the present embodiment, the determination unit 23a can calculate the residual to determine and detect the transition portion, and also calculate the entropy HT to determine and detect the cancer tissue.

Further, among the results obtained by the principal component analysis of the 16 elements of the Mueller matrix M obtained for the normal tissue, the cancer tissue, and the transition portion, the inventors obtained the second principal component, the third principal component, and When the fourth principal component was plotted on the three-dimensional space, it was found that the cancer tissue and the transition part can be distinguished.

First, as described above, the 16 elements of the Mueller matrix M obtained for the normal tissue, the cancer tissue, and the transition portion were subjected to principal component analysis and dimensionally compressed into four dimensions. Then, a search was performed using a technique such as clustering on the four-dimensional space consisting of the four-dimensional values. As a result, it has been found that the first principal component, which is the eigenvalue of the dispersion-covariance matrix, is almost a unit Jones matrix, that is, polarization intensity information.

Therefore, the inventors searched for a three-dimensional value excluding the first principal component among the four-dimensional values, that is, a value rich in only the polarization conversion action component.

FIG. 13 shows a three-dimensional space represented by the second principal component (principal axis), the third principal component (principal axis), and the fourth principal component (principal axis), which are the results of the principal component analysis, developed in two dimensions by the Mercator projection. It is a thing. More specifically, the value of the principal component analysis obtained from the transition part and the cancer tissue in a certain patient is normalized by the value of the normal tissue in the same patient, and the normalized value is plotted on a three-dimensional spherical surface. Is. In the figure, the vertical axis, that is, the latitude on the sphere, is the value of the second principal component. In the figure, the upper half is the |0> direction in the Bloch sphere and the lower half is the |1> direction in the Bloch sphere. In the figure, the horizontal axis, that is, the longitude on the spherical surface is the value of the third principal component and the fourth principal component.

In the figure, white circles are values of cancer tissues, and black circles are values of transition portions. As shown in the figure, it was found that nearly 80% of the values of cancer tissues exist at high latitudes (40° or more) in the northern hemisphere. Therefore, by drawing a diagram as shown in FIG. 13 and using the latitude as the threshold value, it is possible to determine and detect the cancer tissue and the transition portion separately.

Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to the embodiments and the examples, and design changes that do not deviate from the gist of the present invention are disclosed in the present disclosure. included.

As an example, the polarization detection unit 40 according to the above-described embodiment is provided with the six polarization generation units 60, but instead of this, s ₁ =±1, s ₂ =1 among the Stokes components of the incident light, s 3 _{= 1} a may be a four-polarized light generator 60 for generating respectively. Then, of these four polarization generation units 60, one polarization generation unit 60 has a quarter-wave plate 63.

Further, although the light is sequentially switched from the light source 26 and made incident on the plurality of polarization generation units 60, frequency-time-divided light may be made incident on each polarization generation unit 60. However, if the time-division cycle is too long, the degree of polarization of the object may change. Therefore, it is desirable that the time-division cycle be invisible to humans.

Further, the arrangement position of the polarization generation unit 60 is not limited to the above-mentioned respective embodiments, and may be arranged around the detection unit 50. However, since the image pickup surface 57a of the image sensor 57 is usually formed in a rectangular shape, in consideration of downsizing of the entire observation apparatus 100, polarized light is generated above and below the detection unit 50 or in a ring shape as in the above-described embodiment. It is preferable to arrange the portion 60.

Further, in the polarization generation unit 60 in the above-described embodiment, the polarizing plate 61, the diverging lens 62, and the quarter wavelength plate 63 may be adhered to each other and integrated. In this case, the quarter wavelength plate 63 is not provided on the object side of some of the polarizing plates 61 (61c to 61f in the above-described embodiment). Therefore, if the thicknesses of the polarizing plates 61c to 61f are added by the amount corresponding to the quarter wavelength plate 63, the optical path lengths of the respective polarization generating sections 60 can be made equal.

Similarly, the emitting end 15a of the optical fiber 15 and the polarizing plate 61 may be bonded.

Further, from the viewpoint of downsizing the observation device 100, the polarization conversion element of the polarization generation unit 60 is preferably a polarization conversion element in which the polarization state obtained by this polarization conversion element does not change, that is, a passive polarization conversion element. From this viewpoint, the polarizing plate 61 is used in the above-described embodiment. However, it suffices that the polarization state does not change only when the object is irradiated with light. Therefore, for example, a liquid crystal element that is electrically driven to maintain the polarization state in a predetermined state can be used as the polarization conversion element.

Further, although the example in which the endoscope device 10 includes the determination unit 23a is shown, the determination unit 23a may be provided as an external device instead of a part of the endoscope device 10 instead. In this case, the endoscope device 10 may be provided with a transmission unit that transmits the image signal processed by the image processing unit 23 of the endoscope device 10 to the determination unit 23a.

Further, although an example in which the Mueller matrix is used as a value representing the polarization state of the observation object is shown, the Jones matrix may be used instead of this. In this case, the number of types of polarization generated by the polarization generation unit 60, the dimension value compressed by the principal component analysis in the determination unit 23a, the residual value, and the threshold value can be set appropriately.

In addition, in the determination unit 23a, an example is shown in which higher-order data is expressed in a lower dimensional space by linear combination of a plurality of principal component vectors by performing a principal component analysis using a higher-order data group. Alternatively, kernel principal component analysis may be used. In this case, for example, the determining unit 23a first performs dimension reduction by principal component analysis, and when the ratio of the information in the dimension after compression to the information in the dimension before compression is less than or equal to a predetermined ratio, the kernel It may be judged that the principal component analysis is performed. The predetermined ratio is set according to the required information amount, and is 90%, for example. As the positive definite kernel used for the kernel principal component analysis, for example, a linear kernel, a Gaussian kernel, a polynomial kernel, or the like is used.

If it is a method that statistically handles multivariate data consisting of multiple result variables, it can be determined by distinguishing cancer tissue, non-cancer tissue, and transition part using multivariate analysis other than principal component analysis. Good. Examples of multivariate analysis methods other than principal component analysis include regression analysis (including non-linearity) and factor analysis that are mathematically almost equivalent to principal component analysis and kernel principal component analysis.

The determination unit 23a may determine a cancer tissue, a non-cancerous tissue, and a transition part as follows. The determination unit 23a calculates the Mueller matrix of the object using the image signal output from the image sensor 57 that images the light from the predetermined area of the observation object for each different irradiation direction. The determination unit 23a compares the calculated Mueller matrix of the target object with the Mueller matrix obtained by actually measuring a sample having an abnormality in advance, and determines whether or not there is an abnormality of the target object based on the comparison result. In this case, the Mueller matrix (polarization characteristics) of the sample and the state of the sample (type of abnormality and degree of abnormality) are associated in advance. As a result of the comparison, the determination unit 23a sets the state associated with the sample Mueller matrix having a high degree of similarity with the Mueller matrix (polarization characteristic) of the actual measurement result obtained by actually measuring the target object as the measured target object state. judge.

When calculating the Mueller matrix serving as a reference for determining the state of the target object, the determination unit 23a uses a plurality of Mueller matrices of the above-described sample that are known in advance to be cancer tissue, non-cancer tissue, or a transition part. May be input to the neural network for learning, and the feature amount of each state may be extracted. In this case, the Mueller matrix or matrix element having the characteristics of cancer tissue, the Mueller matrix or matrix element having the characteristics of non-cancer tissue, and the Mueller matrix or matrix element having the characteristics of the transition part are calculated, and It may be stored in association with each other. Further, in this case, a plurality of images showing a cancer tissue, a non-cancerous tissue, and a transition portion are input to a neural network for learning, and the feature amount of the image corresponding to each state is extracted, and the Mueller matrix or Matrix elements and images may be stored in association with each other.

The determination unit 23a may determine the state of the object by comparing the Mueller matrix or the matrix element of the object of the state determination with the Mueller matrix calculated by the neural network or the matrix element thereof. it can.

The Mueller matrix of the above sample is a Mueller matrix acquired for different irradiation directions and different irradiation angles, and may be stored in the storage unit in advance. In this case, the determination unit 23a reads out a sample Mueller matrix having the same irradiation direction and irradiation angle of the calculated Mueller matrix from the plurality of sample Mueller matrices, and performs comparison. The control unit 21 may display the result of the determination by the determination unit 23a (that is, the result of the abnormality of the target object) on the display 32 or may store the result in the storage medium. The presence or absence of abnormality of the object may be determined by acquiring a sample Mueller matrix with one irradiation direction and one irradiation angle and comparing each of a plurality of actually measured Mueller matrices with the sample Mueller matrix.

DESCRIPTION OF SYMBOLS 1 Endoscope system 10 Endoscope device 11 Insertion part 11a Tube member 13 Operation part 15 Optical fiber 20 Main body part 26 Light source 40 Polarization detection part 50 Detection part 57 Image sensor 60 Polarization generation part 100 Observation device

Claims (9)

A polarization detection unit that detects the polarization state of light from the object,
By a principal component analysis of the value indicating the polarization state, a determination unit for determining the state of the object,
An observing device comprising: The value indicating the polarization state is a value of a specific dimension,
The observation apparatus according to claim 1, wherein the determination unit converts the value of the specific dimension into a value of a predetermined dimension lower than the specific dimension by a principal component analysis. The observation device according to claim 2, wherein the value indicating the polarization state is a matrix value of the specific dimension. The observation device according to claim 3, wherein the value indicating the polarization state is a matrix value of a Mueller matrix. The determination unit calculates a difference between the value of the predetermined dimension and a value of a dimension higher than the predetermined dimension, and determines the state of the target object based on the difference. The observation device according to any one of 1. The observation device according to claim 5, wherein the determination unit determines the state of the target object based on a magnitude relationship between the difference and a threshold value. The polarization detector is
A plurality of polarization generation units each generating light having a different polarization state,
The observation device according to any one of claims 1 to 6, further comprising: a detection unit that detects light from the object irradiated with the light generated by the plurality of polarization generation units. Detects the polarization state of light from the object,
An observation method, characterized in that the state of the object is determined by performing a principal component analysis on a value indicating the polarization state. An insertion portion having a tube member that is inserted into the body cavity of the subject;
An operation unit for operating the insertion unit,
An endoscope apparatus, wherein the polarization detection section of the observation apparatus according to any one of claims 1 to 7 is arranged in the insertion section.