JPH01209415A - Endoscope device with measuring function - Google Patents

Abstract

PURPOSE:To facilitate the three-dimensional measurement of an object to be measured which utilizes an endoscope by indicating coordinates of one or plural optional points on the surface of the object to be measured on a screen and obtaining array information. CONSTITUTION:A projecting means 2 projects projection light which has a two-dimensional regular pattern on the object in an object of observation from the tip part of a scope 3, a pattern projection image obtained by picking up the reflected light of the light projected on the object is displayed by a specifying means 10, and one or plural optional points are specified on the projection image to calculate their coordinate values. Further, the array information on the pattern corresponding to the one or plural specified points is obtained and an arithmetic means 16 calculates at least three-dimensional coordinate values regarding the one or plural points from the array information on those points and the coordinate values of the points specified by the specifying means 10 on the projection image. Consequently, three-dimensional measurement information on the unevenness of the object surface is easily obtained.

Description

[Detailed Description of the Invention] [Purpose of the Invention (Industrial Application Field) This invention captures an image by irradiating it with a two-dimensional regular pattern of light, and non-contactly measures the unevenness of the subject from the distortion of the pattern. The present invention relates to an endoscope device with a measurement function.

(Prior art) Generally, an endoscope device is an endoscope that is introduced into the inside of an object to be observed such as a living body or a mechanical device (hereinafter referred to as the inside of a body cavity), and an optical device installed at the tip of the scope. An imaging system is used to image and observe a subject such as inside a body cavity.

This endoscope is equipped with a means for projecting a two-dimensional regular pattern of light, such as a laser spot array or a deformed grid, onto the subject, and the distortion of the pattern in the captured image is measured to create a three-dimensional image of each part of the subject's surface. There is an endoscope device that has a measurement function that obtains positional information and shape information such as unevenness.

An example of a method for obtaining two-dimensional regular pattern light is shown in FIG.
A diffraction grating 23 is used, which is made by laminating two sheets 26 and 27 of approximately square shape so that the fibers of each layer are orthogonal to each other. As shown in FIG. 9, a laser beam 29 emitted from an oscillator 28 of a laser light source is perpendicularly incident on this diffraction grating 23, so that a square lattice-shaped bright spot array (hereinafter referred to as
(referred to as spot light) 25 can be generated.

Using such a pattern light generating means, as shown in FIGS. 6 and 7, a spot light 25 is projected from the endoscope tip 21 onto the subject 24, and the spot light 25 is projected onto the subject 24 with the parallax Pa. Photographs are taken using an element (CCD) 22.

Note that, although an optical system such as an objective lens is actually arranged just in front of the image sensor 22, it is omitted from illustration here to simplify the explanation, and will be described as one with the image sensor 22. Further, in FIG. 6, the endoscope tip 21 is provided with an illumination light guide (not shown), an illumination hole in which an irradiation lens is fitted, an air/water supply hole, a forceps hole that also serves as a suction hole, and the like.

The diffraction grating 23 and the light guide 34, which guides projected light from a laser light source oscillator or the like as a pattern projection light source inside the endoscope body, are connected to the diffraction grating 23 and the light guide 34, which guides the projected light from a laser light source oscillator or the like as a pattern projection light source inside the endoscope body to the diffraction grating 23. It may be introduced up to the section 21, or it may be provided separately for exclusive use.

FIG. 5 is an example of an image captured by projecting the spotlight 25 onto the subject 24 in this manner.

In addition to such spot light, a vertical striped pattern may be generated as regular patterned light and projected onto the subject. FIG. 4 is an example of an image captured using the vertically striped patterned light.

゛If the subject has irregularities, the pattern of the captured image will be distorted due to the parallax Pa. By measuring the degree of this distortion, the three-dimensional position of each point on the surface of the subject can be calculated. The principle of this calculation will be described later.

(Problem to be solved by the invention) However, in order to automatically detect pattern distortion as described above, it is necessary to correctly recognize the correspondence between each point of the projected pattern and the pattern on the photographed image. .

As described above, distortion of the pattern light on the image occurs due to the parallax Pa between the pattern light projection section (diffraction grating 23) and the imaging section (imaging element 22 including a lens etc.), but this distortion is caused by the parallax Pa. It appears only in the direction parallel to (the left-right direction in Fig. 5).

In determining each spot on the pattern, spot rows that are in the same series in the direction perpendicular to the parallax Pa (vertical direction in FIG. 5) are called, for example, the same order or homogeneous term.

Then, the spot row at the exact center is taken as the 0th order term, and the +1st order term, +2nd order term, etc. are sequentially added to the right side of Fig. 5, and the 5th order term.
They are named and divided into linear terms, -quadratic terms, . . . in the left direction of the figure.

At this time, if terms of any order are projected with approximately the same amount of light,
When detecting a pattern by converting a captured image into binary data having only values of 0 and 1, patterns of any order have approximately the same size, making it difficult to distinguish the order. For this reason, in order to make it easier to distinguish between orders, conventionally a method has been adopted in which only the spot array of a specific order, for example, the 0th order term, is projected with a larger light intensity than the spot lights of other orders. .

However, the surface of the subject is naturally not flat;
Surface conditions are not always suitable for photographing, such as unevenness, distance, etc., and wet mucous membranes in the case of internal organs of the human body.For this reason, when detecting patterns using binarization processing, emphasis is placed on In some cases, the imaged pattern of the zero-order term may ooze out, making it difficult to separate it from the adjacent pattern of the +1-order term. Furthermore, since the light emitted from the same light source is projected such that a larger amount of light is distributed to the 0th-order term, the amount of light in the surrounding multi-order terms may be insufficient, and a spot may not be detected. Further, in the above detection, complex image processing is required for the binarized image data, such as performing regression on a data region having a value of 1, thinning the line, and extracting the center point.

As described above, extremely complicated and difficult processing is required to correctly recognize the correspondence between points on an image. Furthermore, it naturally takes a long time to execute such complicated and difficult processing.

The present invention aims to solve this problem in view of the conventional circumstances, and it is an object of the present invention to solve this problem without performing complex image processing such as extracting the center point of a pattern on a captured image.
An endoscope with a measurement function that can easily and quickly calculate at least three-dimensional position information of a desired part of the subject, and based on this information, can obtain information such as distance and height between multiple points. An object of the invention is to provide a mirror device.

[Structure of the Invention] (Means for Solving the Problems) In order to achieve such an object, the endoscope device with a measurement function according to the present invention includes
A projection means for projecting illumination light having a two-dimensional regular pattern onto the object from the tip of a scope for imaging the object, and a pattern projection obtained by imaging the reflected light of the light projected onto the object by the projection means. a designating means for displaying an image, accepting designation of one or more arbitrary points on the projected image and calculating the coordinate values thereof; Obtain array information on the pattern, and calculate at least three-dimensional coordinate values regarding the one or more points from the array information of each point and the coordinate values of each point designated by the designation means on the projected image. The present invention is characterized by comprising a calculation means.

(effect).

According to the endoscope device with a measurement function that includes each of the configuration means described above, the projection means projects illumination light having a two-dimensional regular pattern from the distal end of the scope to the subject inside the observation object. The designation means displays a pattern projection image obtained by capturing the reflected light of the light projected onto the object, and also accepts designation of one or more arbitrary points on this projection image and calculates the coordinate values thereof. calculate. Further, array information on the pattern corresponding to each of the designated one or more points is obtained, and the array information of each point and the coordinate value of each point designated by the designation means on the projected image are obtained. From this, at least three-dimensional coordinate values regarding one or more points are calculated by the calculation means.

Based on the three-dimensional coordinate values of each point calculated in this way, it is also possible to easily determine and display distances, elevation differences, etc. between multiple points.

In this way, measurement information such as irregularities on the surface of the subject can be easily obtained. In this way, it becomes possible to solve the above-mentioned problems.

(Embodiment) FIG. 1 is a block diagram showing the configuration of a main part of an endoscope apparatus with a measurement function according to an embodiment of the present invention. In the device of this embodiment, two points are designated on a photographed image of a projection pattern as shown in FIG. 2, and the distance and height difference between the two points are determined and displayed. .

In FIG. 1, in addition to a normal endoscope irradiation light source 1, a pattern projection light source 2, which is a light source for generating irradiation light having a two-dimensional regular pattern, is provided. The pattern projection light source 2 can be configured by, for example, an oscillator that emits a laser beam of a specific wavelength.

In addition, a camera control unit (hereinafter abbreviated as CCU) 4, which is a component of the main body of a normal endoscope device.
, decoder 5, A/D6, frame memory 7, D/A8
, and the monitor 9, various units are provided for specifying points on the photographed image, inputting array information, calculations, and displaying the results.

That is, there is a mouse 10 which is a pointing device for specifying two points on a photographed image of a projection pattern as shown in FIG. 2 displayed on a monitor 9. Also, this is the array information for those two points. There is a keyboard 11 used for inputting the order of the spot manually. Furthermore, there are processing units numbered 12 to 17 in the figure.

Further, FIG. 6 is a diagram showing an example of the configuration of the main parts of the distal end portion of the endoscope. This example shows a side-view scope that observes a subject in a direction perpendicular to the longitudinal direction of the scope.

Laser light of a specific wavelength, for example, emitted from the pattern projection light source 2 shown in FIG. 1 is introduced into one end of the light guide 34 shown in FIG. Diffraction grating 2 from the other end of guide 34
It is radiated towards 3. Then, it passes through the diffraction grating 23,
The light becomes a two-dimensional regular pattern light (spot light) 25 and is projected onto the subject 24. At this time, by devising the method of applying the laser light to the diffraction grating 23, it is possible to obtain a spot light 25 in which only the 0th order term is emphasized more than each term of other orders.

The spot light 25 projected onto the subject 24 is diffusely reflected on the surface of the subject 24, and is received by the image sensor 22 through an optical system that does not show the reflected light and is imaged. An image of the subject captured by the spotlight 25 by the image sensor 22 is sent to the endoscope body via a signal line (not shown).

Note that the scope tip 21 includes other parts not shown in the figure.
There are an illumination light guide for guiding illumination light from the endoscope irradiation light source 1 shown in FIG. 1, air/water supply holes, suction holes, forceps holes, etc.

An image signal of the subject that is incident on the image sensor 22, captured, and sent to the endoscope body via a signal line (not shown) is sent to the camera control unit (CCU) 4 shown in FIG. The CCU 4 processes this image signal and converts it into, for example, a luminance signal Ey of an analog video signal and two color difference signals (Ey-
Er) and (Ey-Eb) are output to the decoder 5.

The decoder 5 converts the analog video signal into, for example, R<
Separate and reorganize into color signals of red), G (green), and B (blue). After being converted into a digital signal by A/D6,
The images are stored in the R, G, and B frame memories 7, respectively.

The color video information stored in the frame memory 7 is
The signal is converted into an analog signal again at A8 and displayed on the monitor 9.

FIG. 2 is a diagram showing an example of a captured image of a projection pattern in this embodiment, and FIG. 3 is a diagram showing an optical arrangement of an irradiation system, an imaging system, etc. in the rotation device.

The coordinates of the center point of an arbitrary spot on the photographing screen of the spotlight as shown in Figure 2 (referred to as the photographing point) and the actual corresponding point on the surface of the subject corresponding to this point 3
The relationship with dimensional coordinate values will be explained. In FIG. 3, the coordinates of the shooting point on the shooting screen indicated by the UV plane are (U, V), and the three-dimensional coordinates of the corresponding point in the XYZ coordinate space centered on the lens 20 are (x, y , z): z=d/(tanθ1 +tanθ2)−・・・■x=
u-z/1 ・・・・・・・・・・・・・・・・・・
・・・ψ■y＝v−2/l ・・・・・・・・・・・・
・・・・・・・・・・Represented by ■. Here, l is the distance between the objective lens 20 and the image sensor 22 (that is, the parallax Pa)
, and θ2 is the angle between the corresponding point and the Z axis as seen from the lens 20;
...It is determined by ■. θ1 is the direction in which the pattern light (spot light) is projected, depending on the order of the pattern light; θ1 = sin' (h・λ/D) ...
....■ (where λ: laser light wavelength, D=diffraction grating spacing).

Therefore, when performing three-dimensional measurement, if the coordinates (u, v) of the center point of each pattern light on the photographing screen and the order of that point are recognized, the three-dimensional coordinate values can be calculated. can.

However, automatically performing such processing requires difficult and complicated processing such as accurate extraction of patterns from photographed images and correspondence of their orders, and of course requires a long processing time.

Therefore, in the present invention, the above-mentioned extraction etc. can be performed by specifying and inputting the coordinates and the corresponding order on the image of one or more arbitrary points on the subject image using a mouse, keyboard, etc. This saves the time required for complicated processing and allows at least the three-dimensional coordinate values of a desired point to be calculated simply and quickly.

Furthermore, in this embodiment device, by specifying and inputting the above coordinates and orders for any two points,
Calculate each three-dimensional coordinate value of these two points, and calculate 2 from these values.
It is configured to calculate and display the distance and height difference between points. With this configuration, the distance and height difference between any two points on the surface of the subject, which are most frequently requested, can be easily and instantly obtained.

For example, suppose that a patterned light distribution as shown in FIG. 2 is obtained on the display screen when patterned light is projected onto an object to be measured and observed. And the two points A and B on Figure 2
If you want to measure the distance and height difference between the two, first move the cursor on the screen (not shown) using the mouse
Two points A and B are set on the screen of the monitor 9.

In FIG. 1, "coordinate designation of two points A and B" 12 causes the cursor on the screen of the monitor 9 to follow and move according to the movement of the mouse 10. Then, use the mouse 10 to make two points A and B.
When specified, the coordinates on each screen are read in ``Specify coordinates of two points A and B'' 12 and sent to ``Calculate ru, V, θ2'' 14. In "Calculation of ru, V, θ2" 14, the coordinates u', v on the image sensor 22 and the angle θ2 are calculated from the coordinates on the screen for each of the two points A and B according to the above-mentioned formula (■), etc. .

On the other hand, the order of each of the two points A and B is determined visually by the observer, and the value of each order is input from the keyboard 11.For example, in the case of FIG. 2, point A is +2 Next, point B becomes quadratic. Then, the input value is received at the "order input of two points A and B" 13 and sent to the "calculation of θ1" 15. In "Calculation of θ1" 15, the angle θ1 is calculated from the value of this order using the above-mentioned formula (2).

Coordinates u, v and angle θ2 and angle θ1 obtained in this way
Based on the values of , the three-dimensional coordinate values x, y, and z of the two points A and B are calculated by the "x, y, and zK outputs of the two points A and B" 16 according to the above formulas ■ to ■. .

From the three-dimensional coordinate values x, y, and z of these two points A and B, the distance and height difference between the two points are calculated in "Calculation of height difference and distance" 17. Further, in "Calculation of height difference and distance" 17, the information is edited into numerical information or easily recognizable figures. The easily recognizable display image thus formed is displayed on the monitor 9 together with the original image held in the frame memory 7.

In this way, information on the height difference and distance between two points on the surface of a subject such as the inside of a body cavity to be observed can be easily measured.

Of course, instead of just finding the height difference and distance between two points in this way, you can also calculate the three-dimensional coordinates by specifying one point and inputting the order using the above formulas. It is possible. Alternatively, it may be configured such that a plurality of points are specified and input at the same time, and the three-dimensional coordinate values of each point are calculated and displayed, and information such as the distance between each point is obtained and displayed using numerical values or figures.

Note that the present invention is not limited to the configuration of the above-described embodiment, and can also be applied to, for example, a vertical striped projection pattern as shown in FIG. In that case, the method of specifying points and inputting the order may be exactly the same as in the above embodiment.

In addition, the method of realizing a two-dimensional regular pattern of projected light is not limited to the method described above, which uses a diffraction grating structure in which sheets of transmissive glass fibers are arranged in parallel in a single layer are stacked vertically and horizontally. Of course, it is also possible to use other methods such as, for example, a deformed grid.

[Effects of the Invention] As explained in detail above, the invention has the following advantages: According to the endoscopic device with this function, it is possible to specify the coordinates on the screen of any one or more points on the surface of the subject to be measured, and to obtain the arrangement information, making it easy to extract patterns and arrange them. Difficult, complicated, and time-consuming processing for detecting information can be omitted. Therefore, at least the dimensional coordinate values of any one or more points of the measurement target can be calculated easily and quickly, and desired information can be easily and immediately obtained. In this way, three-dimensional measurement of a subject using an endoscope can be performed very easily, making it possible to improve the efficiency of endoscopic medical treatment.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of essential parts of an endoscope device with a measurement function according to an embodiment of the present invention, FIG. 2 is an explanatory diagram of an example of a photographed image of a projection pattern in an embodiment, and FIG. 3 is an embodiment of the present invention. A diagram showing the optical arrangement of the irradiation system, imaging system, etc. in the device,
FIG. 4 is a diagram showing an example of an image taken with a vertical striped pattern of projected light; FIG. 5 is a diagram showing an example of an image taken with spot light;
FIGS. 6 and 7 are diagrams of the relationship between the tip of the endoscope and the subject, and FIGS. 8 and 9 are diagrams illustrating the structure of the fiber diffraction grating and the state of spot light generation. 1... Light source 2... Pattern projection light source 3... Scope 4... Camera control unit 5...
・Decoder 6...A/D 7...Frame memory 8...D/A 9...Monitor 10...Mouse 11...Keyboard 12...Coordinate specification of two points A and B 13 ...order human power of 2 points A and B 14...
・Calculation of U, V, θ2 15...Calculation of θ1 16...
・X and Y of 2 points A and B.

Claims (1)

[Claims] (1) In an endoscope device that images a subject by introducing a scope into the object to be observed, a projection means for projecting illumination light having a two-dimensional regular pattern onto the subject; A designation means for displaying a pattern projection image obtained by capturing reflected light of the projected light, accepting designation of one or more arbitrary points on the projection image, and calculating coordinate values thereof; and this designation means Obtaining array information on the pattern corresponding to each of the one or more points specified by, and from this array information of each point and the coordinate values of each point specified by the specifying means on the projected image, An endoscope device with a measurement function, comprising: arithmetic means for calculating at least three-dimensional coordinate values regarding the one or more points.