JPH0547812B2 - - Google Patents

Description

[Detailed Description of the Invention] [Objective of the Invention] (Industrial Application Field) This invention captures an image by irradiating it with a two-dimensional regular pattern of light, and extracts shape information such as unevenness of the subject from the distortion of the pattern. This invention relates to a measuring endoscope device that measures by contact.

(Prior art) Generally, an endoscope device introduces a scope into a living body cavity or a mechanical device (hereinafter referred to as "inside the body cavity"), and uses an optical imaging system installed at the tip of the scope to capture the inside of the body cavity. It is used to image and observe objects.

This endoscope device is equipped with a means for generating a two-dimensional regular pattern of light, such as a laser spot array or a deformed grid, and projecting it onto the subject, and measures the distortion of the pattern in the captured image to determine the surface of the subject. There is a so-called measuring endoscope device that obtains three-dimensional positional information of each part and obtains shape information such as unevenness.

FIG. 13 shows an example of a method for obtaining two-dimensional regular pattern light. For example, two sheets 26 of several mm square are made by arranging transparent glass fibers 26a and 27a with a diameter of about 20 μm in parallel without any gaps,
A diffraction grating 23 is used in which 27 are bonded together so that the fibers of each layer are orthogonal to each other. This diffraction grating 23
As shown in FIG. 14, by vertically entering laser light 29 emitted from an oscillator 28 of a laser light source, a square lattice-shaped bright spot array (hereinafter referred to as spot light) 25 is generated in front of the laser light 29. I can do it.

Using such a pattern light generating means, the first
As shown in FIGS. 1 and 12, a spot light 25 is projected onto a subject 24 from an endoscope distal end 21, and an image is captured with a parallax Pa by an imaging unit (for example, a solid-state imaging device; CCD) 22.

Note that, although an optical system such as an objective lens is actually arranged just in front of the image sensor 22, it is omitted from illustration here to simplify the explanation, and will be described as one with the image sensor 22. Further, in FIG. 11, the endoscope scope tip 21 is equipped with an illumination light guide (not shown), an illumination hole in which an irradiation lens is fitted, an air/water supply hole, a forceps hole that also serves as a suction hole, etc. . A diffraction grating 23 and a light guide 34 that guides irradiated light from a laser light source oscillator or the like as a pattern projection light source inside the endoscope body to the diffraction grating 23
That is, the forceps may be introduced to the distal end portion 21 of the scope using the forceps hole and its passage, or may be provided separately for exclusive use.

FIG. 10 shows an example of an image captured by projecting the spot light 25 onto the subject 24 in this manner.

In addition to such spot light, a vertical striped pattern may be generated as a regular pattern of light and projected onto the subject. FIG. 9 is an example of an image captured using vertical striped pattern light.

If the subject has irregularities, the parallax Pa will cause distortion in the pattern of the captured image. By measuring the degree of this distortion, the three-dimensional position of each point on the surface of the subject can be calculated. Since the principle of this calculation is well known, its explanation will be omitted.

(Problem to be Solved by the Invention) However, in order to accurately detect pattern distortion as described above, the calculation result will be incorrect if the correspondence between each point of the projected pattern and the pattern on the captured image is not correct. Errors occur.

As mentioned above, the pattern light projection section (diffraction grating 2
3) and the imaging unit (image sensor 22 including lenses, etc.) causes distortion of the pattern light on the image, but this distortion occurs in a direction parallel to the parallax Pa (left and right direction in Fig. 10). ) appears only in

Parallax is used to distinguish each spot on the pattern.
Spot rows that are in the same series in the direction perpendicular to Pa (vertical direction in FIG. 10) are called, for example, the same order or the same order term. Then, let the spot row at the exact center be the 0th order term, and sequentially name it to the right in Figure 10 as +1st order term, +2nd order term, ..., and to the left in Figure 10 as 11th order term, -2nd order term, etc. Classify by.

At this time, if terms of all orders are projected with substantially the same amount of light, when the captured image is binarized and patterns are detected, the patterns of all orders will have substantially the same size, making it difficult to distinguish the orders. For this reason, in order to facilitate the discrimination of orders, a method has conventionally been adopted in which only the spot array of a specific order, for example, the 0th order term, is projected with a larger light intensity than the spot light of other orders.

However, the surface of the object is naturally not flat, and has irregularities, distance, and distance, and the surface condition is not always suitable for imaging, such as wet mucous membranes in the case of internal organs of the human body. For this reason, when detecting a pattern by binarization processing, an imaged pattern with an emphasized zero-order term may ooze out, and it may become difficult to separate it from an adjacent pattern with a ±1-order term. Furthermore, since the light emitted from the same light source is projected so that a larger amount of light is distributed to the zero-order term, the amount of light in the surrounding multi-order terms may be insufficient, which may result in spot detection failure. For this reason, the correspondence between each point on the image cannot be determined correctly, causing errors in measurement.

The present invention aims to solve this problem in view of the conventional circumstances, and provides a binarization level when extracting a pattern on a captured image using a function, and also uses this function on the image. It is made variable so that it is determined according to the brightness distribution of
An object of the invention is to provide a measurement endoscope device.

[Structure of the Invention] (Means for Solving the Problems) In order to achieve the above object, the measurement endoscope device according to the present invention projects illumination light having a two-dimensional regular pattern onto a subject. function determining means for determining an intensity function of a binarization level for extracting the pattern according to the luminance distribution on the image of a projected image obtained by capturing the reflected light; and a binary value determined by the function determining means. The present invention is characterized by comprising an extracting means for extracting the pattern on the projected image according to a level given by an intensity function of a conversion level.

(Function) According to the measuring endoscope device equipped with each of the configuration means described above, a function is determined in a projected image obtained by projecting irradiation light having a two-dimensional regular pattern onto a subject and capturing the reflected light. The means determines an intensity function of a binarization level for extracting the pattern according to the luminance distribution on the image, and according to the binarization level given by this intensity function, the pattern on the projected image is It comes to be extracted by the extraction means. This makes it possible to easily extract patterns by changing the binarization level depending on the screen position according to the brightness level of the image, making it possible to more accurately measure irregularities on the surface of the subject. . In this way, it becomes possible to solve the above-mentioned problems.

(Example) Hereinafter, each example of the measuring endoscope apparatus to which the present invention is applied will be described in detail based on the drawings.

FIG. 1 is a block diagram showing the main structure of a measuring endoscope apparatus according to an embodiment of the present invention.

In FIG. 1, in addition to a normal endoscope irradiation light source 1, a pattern projection light source 2, which is a light source for generating irradiation light having a two-dimensional regular pattern, is provided.

In addition, the camera control unit (hereinafter referred to as CCU) is a component of the main body of a normal endoscope device.
) 4, decoder 5, A/D 6, frame memory 7, D/A 8, and monitor 9,
Each section is provided to perform measurement, calculation, and display of the results.

That is, there is an A/D 10 that converts analog video signals from the CCU 4 into digital signal values, and an image memory 11 that stores images captured by irradiation light having a two-dimensional regular pattern. When converting image data stored in this image memory 11, for example having a depth of 8 bits per pixel, into binary data of only 1 and 0 depending on whether it is larger or smaller than a predetermined level value, A reference level generation section 13 that provides a reference level for value conversion, a comparison section 14 that compares this reference level with each of the above-mentioned image data, and a binary data that is expressed only by 0 or 1 according to the comparison result. Binarization processing unit 15 that holds
There is.

There is a thinning processing section 16 and a spot center detection section 17 that detect the center point (or center line) of the pattern based on the binarized image obtained in this way.

Further, there is a calculation unit 18 that calculates the three-dimensional coordinate value of each point from the amount of distortion of the coordinate value of the center point of this pattern etc. from the original pattern. From the calculated three-dimensional coordinate values of each point, a display image edited into numerical information or an easily recognizable figure is formed and displayed on the monitor 9.
There is an output processing unit 19 that displays the image.

Furthermore, on the display screen of the patterned light photographed image as shown in FIG.
There is 1. There is a binarization level setting section 32 and a profile creation section 12 that set the peak value of the binarization level and the profile of the function according to the selected Y coordinate on the screen. The profile creation section 12 supplies the above-mentioned reference level generation section 13 with the value of the binarization level corresponding to each point on the Y coordinate. Note that it is also possible to specify the setting of the binarization level using the mouse 30.
The profile display 33 organizes a display screen that displays the peak value and profile of the binarization level set above, superimposed on the photographed image.

Next, the operation of the measuring endoscope device of this embodiment will be explained.

Uniform irradiation light emitted from a patterned light projection light source 2 provided separately from the endoscope light source 1 is introduced from one end of a light guiding light guide (not shown), passes through the inside of the endoscope scope, and is directed to the scope 3. The light is guided to the tip of the Note that the patterned light projection light source 2 that generates uniform irradiation light can be configured by, for example, an oscillator that emits laser light.

FIG. 11 is an explanatory diagram schematically showing the internal configuration of the distal end portion 21 of the endoscope. This figure shows an example of a side-viewing endoscope that observes in the side direction perpendicular to the longitudinal direction of the scope.

The diffraction grating 23 is connected from the other end of the light guide 34 that has guided the irradiation light from the pattern light projection light source 2.
The uniform irradiation light is emitted. Diffraction grating 23
The emitted light becomes a spot light 25 having a two-dimensional regular pattern and is projected onto the subject 24.

The light projected onto the subject 24 is reflected on the subject surface, enters the image sensor 22 disposed at the distal end portion 21 of the endoscope 3, and is imaged.

The subject image captured by the image sensor 22 is sent as an electrical signal to the camera control unit (CCU) 4 shown in FIG. 1. The CCU 4 processes this electrical signal and outputs it as, for example, a luminance signal Ey of an analog video signal and two color difference signals (Ey-Er) and (Ey-Eb).

On the one hand, the analog video signal outputted from the CCU 4 is processed by the decoder 5 into R (red), G
(green), B (blue), and A/
After being digitized at D6, it is stored in the R, G, and B frame memories 7, and the image is displayed on the monitor 9 via the D/A8. On the other hand, for example, the luminance signal Ey is digitized by the A/D 10, stored in the image memory 11 in synchronization with the timing of irradiation and imaging, and used for pattern detection and calculation processing of three-dimensional position from the amount of distortion. Served.

To detect a pattern, image data stored in the image memory 11, which has a depth of, for example, 8 bits per pixel, is converted into binary data containing only 1 and 0 depending on whether it is larger or smaller than a predetermined level value. Binarization processing is used.

If the binarization level, which is the reference level for binarization processing, is too large compared to the output signal level of the pattern light, the pattern light is likely to drop out, and if the binarization level is too small, the pattern Light is no longer separated. Therefore, it is necessary to make the binarization level variable according to the signal level of the photographed image.

FIGS. 2 and 4 are diagrams showing examples of images taken with patterned light.

The intensity distribution of the patterned light is, for example, as shown in FIGS. 2 and 4, so that the zero-order term can be easily distinguished.
The 0th order term is set to be extremely bright and irradiated. For this reason, the amount of light decreases rapidly after the ±1st order terms, and as the terms become higher order terms, there is a tendency for the light to gradually become darker.

However, the above-mentioned tendency also differs depending on the shape of the subject, the shooting direction, etc. When a relatively flat object is photographed from the front, an image is obtained with an intensity distribution that does not differ much up to the periphery, as shown in FIG. 2. Furthermore, if the screen is gently raised near the center, the image will be bright at the center and the amount of light decreases toward the periphery, as shown in FIG. 4.

For an image like the one shown in Figure 2, the intensity of the pattern light on a line whose Y coordinate is Yi, for example, is
The distribution is as shown by the curve in Figure 3. In such a case, even if the function of the binarization level is as shown by the horizontal straight line in FIG. 3, it is possible to extract the pattern without error as long as the reference level is selected correctly.

However, in the case of an image like that shown in Figure 4, the intensity of the pattern light on a line whose Y coordinate is Yi, for example, will be distributed as shown by the curve in Figure 5, and will be a function of the binarization level. If we take it as a straight line,
As shown in FIG. 5, a pattern cannot be extracted correctly no matter where the reference level is set.

For this reason, as shown in Figures 6 to 8,
The function of the binarization level must be determined as a curve corresponding to the intensity distribution of the pattern light.

First, by detecting the maximum value and its position of the intensity distribution of pattern light, the position Pmax and intensity Imax of the zero-order term are determined. Based on this intensity Imax, a maximum value Bmax of the binarization level is set, and then a binarization level profile B-profile is determined according to the position Pmax of the zero-order term.

At this time, depending on the coordinate value on the screen of the position Pmax of the 0th order term, the reference distance from the pattern light projection area on the endoscopic scope and the imaging area to the object surface differs, and the pattern interval of each order differs. It becomes necessary to change the B-profile as well. That is, where the zero-order term is far away, the binarization level profile B-profile is made steep, as shown in FIG. Furthermore, if the distance between the zero-order terms is short, the binarization level profile B-profile is made gentle as shown in FIG.

The setting of the reference value of the binarization level and the creation of the profile are carried out by the binarization level setting section 32 and the profile creation section 12 in FIG.
It is executed by Note that these settings and creation, etc., do not necessarily have to be configured to be performed automatically in each of the above parts, but, for example, the mouse 30
Alternatively, the operator of the measurement endoscope may perform the settings as desired while visually viewing the measurement endoscope.

The operator specifies, using the mouse 30, the Y coordinate corresponding to the line on the subject to be measured while observing the photographed image of the patterned light on the monitor 9. By moving the mouse 30 on a desk or the like, the Y coordinate selection section 31 detects this, and while moving the straight line indicating the Y coordinate, the pattern light on the monitor 9 is displayed through the profile creation section 12 and the profile display section 33. Overlay the image on the display screen. When the Y coordinate is selected, the coordinate value is transferred to the profile creation section 12.

The profile creation unit 12 and the binarization level setting unit 32 perform the steps described above.
A reference level for the binarization level is set, a profile of the binarization level function is created, and the profile is displayed on the monitor 9 via the profile display section 33.

At this time, it is also possible to specify the reference level of the binarization level and the profile of the binarization level function using the mouse 30.

When the profile of the binarization level function is created in this way, the level value of each point on the Y coordinate is sent from the profile creation section 12 to the reference level generation section 13.

Then, the reference level generation section 13 outputs to the comparison section 14 the binary level at which the data is converted into binary data of only 1 and 0. The comparator 14 compares the value of each pixel of the image data in the image memory 11 that stores the projected image by the patterned light with respect to this binarization level. Depending on the results of this comparison,
The binary data is converted into binary data represented by only 0 or 1 by the binarization processing unit 15 and is held.

Based on the binarized image thus obtained, the thinning processing section 16 performs thinning processing to reduce the peripheral portion of the data area having a value of 1.

Further, the spot center detecting section 17 detects the center point (or center line) of the pattern, and also discriminates the center point (or center line) of the corresponding patterns of the two detected images.

Then, in the calculation unit 18, the two-dimensional coordinates of each point are calculated from the amount of distortion between the coordinate value of the center point of the pattern, etc., and the corresponding point of the original pattern.

Here, the relationship between the spot light coordinates and the distance of the corresponding point on the surface of the subject is as follows: Let the spot light coordinates for the h-th term be Xs(h), the center coordinate be Xc, and the viewing angle be 2β, and in Fig. 11, for the h-th term, If the angle formed by the line connecting the corresponding point on the object surface and the center point O of the image sensor 22 and the Z-axis at the center of the screen is θh; it is expressed by tanθh=(Xs(h)−Xc) tanβ/Xc. .

Also, the vertical distance Zh to the surface of the subject in the hth order
is expressed as; Zh=Pa/(tanθh−tanψh). Here, Pa is the parallax and ψh is the diffraction angle of the h-order term.

The three-dimensional coordinate values of each point calculated in this way are edited into numerical information or easily recognizable figures in the output processing section 19. The easily recognizable display image thus formed is displayed on the monitor 9.

In this way, shape information such as irregularities on the surface of the subject is calculated from the photographed image of the subject field onto which patterned light is projected, and is displayed on the screen of the monitor 9 in an easy-to-read form.

Note that the present invention is not limited to the configuration of the above-described embodiment, and can of course be applied to, for example, a vertical striped projection pattern as shown in FIG.

Furthermore, the endoscope is not limited to the side-viewing scope shown in the above example, but may of course be a forward-viewing or oblique-viewing scope.

Furthermore, a method to realize a two-dimensional regular pattern of projected light is to use a laser beam, as described above, to create a diffraction grating, which has a structure in which sheets of transmissive glass fibers are stacked vertically and horizontally in parallel. It is of course possible to use other methods such as a deformed lattice, for example.

[Effects of the Invention] As explained in detail above, the measuring endoscope device according to the present invention can improve the brightness distribution on a projected image captured by projecting a two-dimensional regular pattern of illumination light onto a subject. The structure is such that the binarization level of the binarization process for extracting patterns can be changed depending on the pattern. In other words, since the function that determines the reference level value of the binarization level and the profile of the binarization level can be variably set in accordance with the luminance data, an appropriate binarization level can be obtained and more accurate It becomes possible to extract patterns without errors. In this way, the measuring endoscope device can more easily and quickly measure the shape of the object surface, such as accurate distance.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the main parts of a measuring endoscope device according to an embodiment of the present invention, FIGS. 2 and 4 are diagrams showing examples of projected images of patterned light, and FIGS. FIG. 8 shows the second
An explanatory diagram for setting the binarization level function according to each image example shown in Fig. 4 and Fig. 4. Fig. 9 is a diagram showing an example of an image taken with vertical striped pattern projection light, and Fig. 10 is taken with spot light. Diagram showing an example of an image, No. 11
1 and 12 are diagrams of the relationship between the tip of the endoscope and the subject, and FIGS. 13 and 14 are explanatory diagrams of the structure of the fiber diffraction grating and the state of spot light generation. 1...Light source, 2...Pattern projection light source, 3...
Scope, 4...Camera control unit,
5...Decoder, 6...A/D, 7...Frame memory, 8...D/A, 9...Monitor, 10...
A/D, 11...image memory, 12...profile creation section, 13...reference level generation section, 14...
... Comparison section, 15 ... Binarization processing section, 16 ... Thinning processing section, 17 ... Spot center detection section, 18 ...
... Arithmetic unit, 19... Output processing unit, 20... Lens, 21... Endoscope tip, 22... Image pickup element, 23... Diffraction grating, 24... Subject, 2
5... Spot light, 26, 27... One-dimensional fiber diffraction grating, 26a, 27a... Glass fiber, 28... Oscillator, 29... Laser light, 3
0... Mouse, 31... Y coordinate selection section, 32...
Binarization level setting section, 33...profile display section.

Claims (1)

[Scope of Claims] 1. Irradiation light having a two-dimensional regular pattern is projected onto the subject, and the reflected light is captured on the projected image. The shape of the subject, such as unevenness, is determined from the distortion of the pattern from the reference point. In the measurement endoscope device for recognizing information, a function determining means determines an intensity function of a binarization level for extracting the pattern according to the luminance distribution on the projected image; A measurement endoscope apparatus comprising: extraction means for extracting the pattern on the projection image according to a level given by an intensity function of a value level.