KR20080043767A - Projection of subsurface structure onto an object's surface - Google Patents

Abstract

An imaging system illuminates an object with infrared light to enhance visibility of buried structure beneath the surface of the object, and projects a visible light image of the buried structure onto the surface of the object. The system may include an infrared light source for generating the infrared light and a structure for diffusing the infrared light. The diffusing structure may include one or more layers of diffusing material for diffusing the light. The system further includes a video imaging device for receiving the infrared light reflected from the object and for generating a video image of the buried structure based on the reflected infrared light. The buried structure may be a subcutaneous blood vessel. A calibration procedure is described as well as embodiments for ensuring that the object is maintained in focus at the correct distance.

Description

Apparatus and method for projecting an undersurface structure onto an object surface {PROJECTION OF SUBSURFACE STRUCTURE ONTO AN OBJECT'S SURFACE}

TECHNICAL FIELD The present invention relates to infrared diffusion, and more particularly, to a system for projecting a video image of a structure hidden under the surface of an object based on reflected infrared light by using diffused infrared light.

Medical treatments and treatments require the ability to find blood vessels in the patient's arms or legs, but finding them can be difficult when the vessels are thin or significantly below the subcutaneous fat or other tissues. Conventional imaging systems designed to help find these vessels have low performance.

Thus, there is a need for a system that improves visual contrast between subcutaneous blood vessels and surrounding body tissue.

Summary of the Invention

For example, if the object is a patient, there is a need for a device that improves the clarity of subcutaneous blood vessels by shining diffuse light on the patient. In one example, a light source equipped with such a device emits infrared light toward an object. Powering the light source from the power source emits infrared light. In addition, a diffusion structure having one or more diffusion portions is used. Different diffusions are different for each diffusion.

On the other hand, an apparatus for illuminating diffused light on an object is also introduced. The device illuminates the infrared light from each light source towards the object. In addition, a diffusion structure that provides various degrees of diffusion is used. The first diffuser plate of the diffuser structure is located next to the light source, causing one-step diffusion into the light emitted from the light source, and the second diffuser plate away from the first diffuser causes two-step diffusion. Light from the light source is polarized in the polarizer.

Another embodiment introduces a device for illuminating diffused light on an object. The first diffusion plate having the first diffusion surface receives light from the light source in the middle and diffuses infrared rays emitted from the light source in one step. The video camera receives the light reflected from the object and creates an image of the object.

In another embodiment of the device, the LED groups are arranged in a pattern to form an LED plane. Each LED illuminates an object and sends an electrical signal to the LED. The LED is activated using the control signal of the controller. The diffusion structure is arranged to receive and diffuse infrared light emitted from the LED in the middle.

Using the present invention described above, the subcutaneous blood vessels that are difficult or impossible to see with white light or non-diffuse infrared light can be easily seen as video images. Subcutaneous blood vessels appear as dark lines and the surrounding body tissues appear as light backgrounds.

In addition, the present invention also includes light sources of various configurations, a camera for viewing an image of an object hidden under the surface of an illuminated object, and a projector for projecting the processed image onto the object surface. Since the image is projected back to the surface of the object rather than a screen or monitor away from the surface of the object, the observer using the present invention is not subjected to parallax errors, which occur in conventional devices when the focus is out of focus. In short, because the projection is on the surface of the object, not the screen away from the surface of the object, the image remains in the same position on the surface of the object even if the observer is out of focus. Another important feature is that the image of the object seen by the camera is in the first spectrum, which is outside the range of the second spectrum projected back to the object's surface, so that the camera does not operate on the projected image. Since the spectrum of the captured image and the projected image do not overlap, the image processing process of the object may be free from interference caused by the projected image. Since the projected image is in the second spectrum (eg the visible light spectrum) and the illumination light for the camera is in the first spectrum (eg the infrared spectrum), the two spectra do not overlap at all. In other cases where the object is not primarily illuminated by the light of the first spectrum, the object is illuminated by broadband spectral ambient light, and the first spectral bandpass filter is placed in front of the camera to remove all spectral components other than the first spectrum. Only the camera can see it. If the present invention is used in the medical field to find subcutaneous blood vessels, the first spectrum will be the infrared spectrum.

Also included in the present invention is an image processing method in which unwanted small artifacts, such as hair, are removed from the photographed image before being projected onto the object surface.

During the calibration process, the projector projects a green target pattern onto the fluorescent screen, which can then be converted into deep red light for viewing with an infrared camera. The computer program records the position of the shooting pattern and calculates the correction factor to be used for the dual linear transformation to correct translation misalignment, rotation, and magnification between the camera and the projector.

Technical features of the invention

From a technical point of view, the present invention is needed in the medical field for finding blood vessels in the arm or leg of a patient. Conventionally, blood vessels under a thick subcutaneous fat layer are particularly difficult to find. Conventional imaging systems designed to assist in finding these vessels have been degraded. Accordingly, it is a technical problem underlying the present invention to provide an apparatus and method for improving visual contrast between subcutaneous blood vessels and surrounding body tissues.

This problem is solved with a device that improves the sharpness of the structure hidden under the surface of the object. The camera of the medical device receives the diffused light reflected from the object to make an image, and the video projector projects the visible light image of the hidden structure onto the surface of the object.

The technical concept of the present invention allows for changes in using two lasers to photograph the underlying structure and to fire the beams respectively, wherein the two beams intersect at a desired target distance without being parallel to each other and the object surface at this intersection. If present, the focus is on the hidden structure. As a result, the difficult task of finding blood vessels in the limbs of a patient is much easier because the blood vessels can be seen as images projected onto the skin.

The video projector projects the visible light image onto the object surface to display the underlying structure.

The invention also relates to a method for calibrating a device that improves the clarity of a structure hidden under an object surface. The apparatus includes an imager for receiving an image of diffuse light in the first spectral range reflected from an object to produce an image, and a video projector for projecting a visible light image of a structure in a second spectral range that does not overlap the first spectrum onto the surface of the object. The above method using the device comprises the steps of: luminous by projecting a pattern of points separated from each other in the second spectral range onto a fluorescent screen; Observing the position of the point of the fluorescent pattern; Calculating a correction factor for the double linear transformation using the observed position; And correcting the subsequent projected image using a double linear transformation having the correction coefficient.

The present invention also relates to another method of improving the sharpness of a structure hidden under the surface of an object, the method comprising: providing an imager that receives diffused light reflected from an object; Creating a received image; Removing the small and dark artifacts from the received image, but leaving the large and dark objects intact to create a processed image; And projecting the processed image onto an object surface in the visible light spectral range. In this method, it is preferable to perform adaptive contour enhancement on the processed image before projecting the processed image.

In addition, the present invention provides a device for improving the sharpness of the structure hidden under the surface of the object, the device receives the diffused light reflected from the object to produce an image, and a video for projecting a visible light image of the hidden structure on the surface of the object Includes a projector, but noticeably adds a pattern that is independent of the hidden structure to the projected visible light image. It is preferable that the added pattern is a sentence border.

On the other hand, according to the present invention, another apparatus for improving the sharpness of the structure hidden under the object surface, the imager receives the diffused light reflected from the object to create an image, and a video projector for projecting the visible light image of the hidden structure on the object surface It includes an infrared filter between the camera and the object.

According to the present invention, another apparatus for improving the sharpness of a structure hidden under the surface of an object includes an imager for receiving an image of diffused light reflected from the object, a video projector for projecting a visible light image of the hidden structure onto an object surface, and an object. It includes a shining light source. At this time, the camera receives only the diffuse reflected light of the first spectral range, and the visible light image has a second spectrum that does not overlap with the first spectrum. Also, an infrared filter is placed between the camera and the object.

On the other hand, according to the present invention, another device for improving the sharpness of the structure hidden under the object surface is an imager for receiving the diffused light reflected from the object to create an image, a video projector for projecting a visible light image of the hidden structure on the object surface, And an infrared filter between the camera and the object.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is an imaging system for viewing an object in infrared light according to a preferred embodiment of the present invention;

2 is a perspective view of an imaging system using diffused infrared light;

3-4 are cross-sectional views of an imaging system;

5 is a block diagram of an imaging system;

6A is a perspective view of another imaging system using diffuse infrared;

6B is a cross-sectional view of FIG. 6A;

7A is a perspective view of another imaging system;

FIG. 7B is a sectional view of FIG. 7A;

8 is a perspective view showing another feature of the imaging system;

9 is a cross-sectional view taken along the line A-A of FIG. 8;

10 is a cross-sectional view taken along the line B-B in FIG. 9;

11 is a block diagram of an imaging system;

12 is a perspective view of a third imaging system;

13 is a perspective view of a fourth imaging system;

14 is a schematic diagram of a fourth imaging system;

15 is an interior view of a fifth imaging system using ambient light;

16 is an artifact removal program for processing a received image;

17 is a C ++ programming language for the artifact removal program of the received image;

18 is a perspective view showing a method of arranging an object to be photographed using a pair of laser pointers;

19 is a perspective view showing a calibration process of an imaging system;

20 is a photograph of processed images of subcutaneous blood vessels projected onto body tissues covering blood vessels;

21 is a photograph of a projected image surrounded by a border of sentences;

FIG. 22 is a picture surrounded by a border of text similar to FIG. 21, but the object is not in position so that the text is out of focus;

23 is a frame border image coupled to a projected image to project onto an object and leave it in place;

FIG. 24 is a photograph of a processed image of a subcutaneous blood vessel projected on the hand by the present invention similarly to FIGS. 20 to 21, showing that the rim of the hand is not in focus;

25 is a computer listing showing a solution for obtaining the double linear transformation coefficients of the calibration process of the imaging system of the present invention;

FIG. 26 is a program listing of a C ++ programming language for performing runtime calibration on photographed images of an object using coefficients determined in the calibration process. FIG.

Skin and other body tissues reflect infrared light in the near infrared range of 700-900 nm, but blood absorbs infrared light in this range. Thus, in video images of body tissues taken under infrared light, blood vessels appear as dark lines and the surrounding flesh is seen with a lighter background. However, due to the reflectance of the subcutaneous fat, blood vessels located significantly below the subcutaneous tissue are difficult or impossible to see when illuminated directly, ie when light comes from one direction.

In the near-infrared range of body tissues where subcutaneous fat has accumulated significantly, the contrast between blood vessels and surrounding flesh is significantly higher than in direct infrared light. Although the present invention is not limited to any particular theory, most of the diffuse infrared rays reflected from the subcutaneous fat are far from the viewing direction. Thus, when illuminating tissue using highly diffused infrared light, the contrast between blood vessels and surrounding flesh can be maintained as desired.

The imaging system 2 of FIG. 1 is a device for illuminating an object 32 such as body tissue with high diffusive infrared rays and making a video image of the object based on the infrared rays reflected from the object 32. As will be described in detail later, when the object 32 is a body tissue, blood vessels under the subcutaneous fat in the tissue can be clearly seen in a video image made by the imaging system 2.

The imaging system 2 includes a luminaire 10 that illuminates the object 32 in infrared in various directions. A plurality of infrared rays providers 10a to f of the luminaire 10 illuminates the infrared rays to the object 32 in various directions. The infrared direction of each infrared provider is indicated by 4a-f in FIG. 1. These infrared directions cover the range from perpendicular to parallel to the surface of the object 32. Infrared rays that illuminate the object 32 in such a wide range are referred to as highly diffused infrared rays.

As will be described in detail later, the reflectors 10a to f are preferred reflectors that direct light from a single light source toward the object 32, but each may be a separate light source or a combination of light sources and reflectors.

The imaging system 2 includes a video camera 38 for shooting an object 32. The camera 38 sees the object 32 in the viewing direction indicated by the arrow 6 in FIG. 1, and receives the diffuse infrared rays reflected from the object 32 to generate an electronic video image of the object 32.

2 shows a luminaire 10, and FIG. 3 is a cross-sectional view taken along the line A-A of FIG. 2. The light source 12 of the luminaire 10 sends light to one end of the light diffusion structure 14. The inner surface of the elongated case 16 of the light diffusing structure 14 is a reflective surface. The inner surface of the case 16 is preferably white, but may be a mirror surface or a combination of a white surface and a mirror surface. At the opposite end of the light diffusion structure 14 there is a hollow light guide 22. As will be described in detail later, the light guide 22 serves as an outlet for diffused light.

The elongated case 16 is composed of first and second sections 16a to b, each having a large end and a small end. The first and second sections 16a to b each have a pyramid shape. Since the four trapezoidal faces of the sections 16a to b are the same, each end of the sections 16a to b forms a square hole. As shown in FIG. 2, the ends 16 of the sections 16a to b are opposed to each other to form a case 16.

The small end of the first section 16a forms an inlet 18 consisting of short sides of four trapezoidal surfaces. When the light source 12 is connected to the inlet 18 of the small end of the first section 16a, the light of the light source 12 enters the case 16 through the inlet 18 to illuminate the inner surface.

The small end of the second section 16b forms an outlet 20 consisting of short sides of four trapezoidal surfaces. A hollow light guide 22 is connected to the outlet 20, and the inner judge surface of the light guide is white like the inner face of the case 16.

The luminaire 10 includes an elongated inner reflector 24 disposed coaxially with the case 16. For convenience of explanation, the internal reflector 24 of FIG. 2B is removed from the case 16. The internal reflector 24 connects the pyramid 24b to the square tube 24a, and four sides of the pyramid 24b are triangular. Referring to FIG. 3, the vertices of the pyramid 24b are disposed near the inlet 18 of the case 16. The inner reflector 24 has a white reflective surface as well as the inner surface of the case.

The characteristics of the light diffusing structure 14 are best shown in FIG. 3. Inside the light source 12 is a lamp 26, such as a crystal-halogen bulb, and a gold plated reflector, such as Gilway's L517A-G. The lamp 26 emits electromagnetic radiation in the form of white light.

For convenience of explanation, the lamp 26 is regarded as a point emitting light in various directions such as 28 and 30. The light 28 is reflected at the inner surface of the section 16b of the case 16, passes through the outlet 20, passes through the light guide 22, and is reflected at the inner surface several times and exits the outlet 23. Light 30 exiting light source 12 at a different angle from light 28 is reflected at inner reflector 24, reflected at the inner surface of section 16b, and then exited through light guide 20 through exit 20. 22), likewise reflected several times on the inner side of the light guide and then exit exit 23, but exit at an angle different from the angle at which light 28 exits.

Placing the object 32 near the outlet 23 causes the two lights 28, 30 to reach the object 32 at different angles. The light exiting the light source 12 may be referred to as a myriad of light reflected by hitting the inner surface of the case 16 and the reflector 24 at a myriad of angles. Therefore, the light exiting the outlet 23 reaches the object 32 at various angles, and therefore is highly diffused light. This angle of arrival covers from perpendicular to parallel to the plane of the outlet 23. Since the diffusion structure 14 is three-dimensional, the light reflected from the inner surface of the case 16 and the inner reflector 24 is three-dimensional, and is also reflected perpendicularly to the surface of FIG. 3. Thus, the light from the outlet 23 of the luminaire 10 is also highly diffused and appears to be emitted from a myriad of light sources.

Due to the arrangement of the inner reflecting surface of the case 16 and the outer reflecting surface of the inner reflector 24, the diffusion structure 14 effectively transmits the light of the lamp 26 to the outlet 23. Thus, a substantial portion of the light emitted from the lamp 26 hits the object 32 so that there is little energy loss.

As will be described in detail later, the lighting fixture 10 is also used to diffuse light for medical use, but the scope of the present invention is not limited to medical use. The lighting fixture 10 can also be used for photography purposes.

The cold mirror 34 is disposed between the lamp 26 of the light source 12 and the inlet 18 of the case 16. The cold mirror 34 reflects almost all light having a wavelength outside the infrared wavelength in the 700-1000 nm range. The infrared transmission filter 36 disposed between the cold mirror 34 and the inlet 18 filters out light other than infrared rays but transmits infrared rays. Therefore, the light entering the case 16 through the cold mirror 34 and the filter 36 may be infrared.

The configuration in which the light source 12 emits infrared rays may be different. For example, the light source 12 may be made of an infrared LED. Therefore, the light source 12 demonstrated in FIG. 3 is only an example to the last.

4 shows the dimensions of the luminaire 10. The total length of the light diffusion structure 14 is 34.82 inches, the height and width of the case 16 is 10.04 inches at the connection of the first and second sections 16a to b, the length of the light guide 22 is 14 inches, The length of the inner reflector 24 is 15.86 inches, the length of the square tube 24a of the inner reflector 24 is 7.93 inches, the height and width of the square tube is 3.5 inches, the height and width of the light source is 2.11 inches.

In the present invention, the lens 40 is used together with the camera 38 to produce a video image of the object based on the diffused light reflected from the object 32. The camera 38 is preferably a CCD video camera of model number 631520010000 manufactured by Cohu. The lens is a 25mm f-0.95 movie camera made by Amgenieux.

The camera 38 and the lens 40 are installed in the square tube 24a of the inner reflector 24. The camera 38 and the lens 40 are disposed toward the hole formed by the open end of the square tube 24a, and the light guide 22 is aimed at the viewing center of the camera 38. In this case, light reflected from the object 32 and sequentially passed through the holes of the light guide 22, the case 26, and the square tube 24a enters the camera 38.

When the infrared transmission filter 42 is disposed at the open end of the square tube 24a, almost all light having a wavelength other than the wavelength of 700 to 1000 nm from the light reflected from the object 32 and entering the case 16 is filtered out. Filters out light with wavelengths other than 800-850 nm. Therefore, since the light entering the lens 40 through the filter 42 is infrared rays having a desired wavelength, the camera 38 is also emitted from the luminaire 10 to receive the infrared rays reflected from the object 32.

Using the light reflected from the object 32, the camera 38 makes an image of the object in the form of an electrical video signal. Referring to FIG. 5, when a video signal enters the video enhancement board 44 manufactured by DigiVision, Inc., using the model number ICE-3000, a video video signal with improved video is generated based on the video signal of the camera 38. Enters a video capture / display card 46 made by Miro on a Model 20-TD live card. The card 46 captures still images from the video signals stored in digital format in the digital storage device and also formats the video video signals for real-time display of the video monitor 48.

The luminaire 10 may use other means to produce diffuse infrared light in accordance with the present invention. For example, the infrared provider 10a-f of FIG. 1 may be implemented with a flash, or the LEDs arranged in a circle may be used to illuminate a plastic diffuser near the surface of the object 32. In the latter case each LED would be an infrared provider 10a-f.

6 shows another embodiment of the present invention. The projector 50 of the imaging system 2 is to increase the contrast of the light and dark portions of the object 32 by projecting the image of the object 32 onto the object 32. As introduced in US Pat. No. 5,969,754, "Contrast Enhancing Illuminator," the feature of the object is visually improved by superimposing the feature of the visible light image onto which the object is projected. Overlapping the visible image makes the brighter parts brighter but the darker ones.

In FIG. 6, the infrared ray 52 is reflected on the object 32 in a manner similar to the above-described method, but the optical path is folded, thereby bending the outlet 23 of the light guide 22 by 90 degrees compared to the outlets of FIGS. 1 to 3.

Referring to FIG. 6B, infrared light 52 emitted from the interior of the light diffusion structure 14 is reflected from the hot mirror 54 to face the object 32 through the light guide 22. The hot mirror 54 may receive an infrared image of the object 32 indicated by 56 and reflect it toward the camera 38. The hot mirror 54 may receive the visible light image 58 of the projector 50 and send it to the object 32 through the light guide 22.

As detailed in US Pat. No. 5,969,754, the video output signal of the video camera 38 is provided as a video input signal of the projector 50. The projector 50 projects the visible light image 58 of the object 32 onto the hot mirror 54 based on the video input signal. The hot mirror 54 sends the visible light image 58 to the object 32 through the light guide 22. When the visible light image 58 projected by the projector 50 matches the infrared image 56 of the object detected by the camera 38, the visible light image 58 overlaps with the features of the visible light image 58 and the features of the object 32.

When the object 32 is body tissue and the present invention is used to find subcutaneous blood vessels in the body tissue, the blood vessels appear as dark lines in the projected visible light image 58. Therefore, when the visible light image 58 is projected onto the body tissue, the subcutaneous blood vessel is placed just below the dark line in the projected visible light image 58. In this way, the present invention improves the performance of finding subcutaneous blood vessels while minimizing patient discomfort.

7 shows another example of the invention used as an illuminator to enhance contrast. The example of FIG. 7 is similar to FIG. 6, but with the camera 38 positioned outside the light diffusion structure 14. In order to cover placing the camera 38 in a different position, the hot mirror 54 of FIG. 7 was turned 90 degrees clockwise relative to FIG. 6. The hot mirror 54 functions similar to that described in FIG. The camera positions are different, and the infrared transmission filter 42 is installed on the wall surface of the light guide 22. The reflector 60 is provided to direct the light of the light source 12 to the light guide 22 and the outlet 23. Reflector 60 is preferably flat and a hole for passing light between the object 32 and the camera 38 and the projector 50 is preferable.

A smaller and more reliable imaging system 70 is shown in Figures 8-11. The imaging system 70 has the most desirable structure for illuminating an object 71 such as body tissue and making an image of the object based on the infrared rays reflected from the object. All the features are arranged in the housing 72 of the imaging system 70.

The housing 72 of FIG. 8 is a cuboid, 3 to 5 inches long and 3.5 inches wide. As will be appreciated by those skilled in the art, the imaging system 70 can have a variety of configurations, and the structure described herein is merely an example. Although the housing of FIG. 8 is a rectangular parallelepiped, it is possible to have a size different from other shapes, such as a circle and a polygon.

The video camera 74 with the lens 75 and the video processing component are located in the housing 72. The camera 74 and the video processing component serve to detect infrared light and process the infrared light detected by the object 71. The camera 74 makes an image based on the infrared rays reflected from the object 71. 8-9, the camera 74 is installed in the hole 76 of the mounting wall 78 and the lens 75 enters the interior 77 of the housing. In particular, the camera 74 is installed symmetrically in the center of the housing 72. Positioning the camera symmetrically in this way can maximize the amount of infrared detection, thereby improving the image quality produced by the system 70 and, ultimately, improving vascular illumination under the subcutaneous fat in body tissues.

The housing 72 contains various elements that transmit diffused light to the object 71. Arrow 80 is diffused light from system 70 and arrow 82 is reflected light reflected from object 71. As shown in Fig. 9, an LCD array 85 is provided on the wall 78 in which several LEDs 84 are arranged to emit infrared rays. LED array 85 forms an LED reference plane. In operation, each LED 84 emits light having a wavelength of 740 nm, preferably an LED produced by Roithmer Lasertechnik of Austria under model number ELD-740-524.

The LED 84 of FIG. 10 is provided on a circuit board 86 near the wall 78. 9, it is desirable to arrange eight groups of LEDs 92 and 94 concentrically around the imaging system 70, in which case the diffused light of the system 70 can be dispersed and transmitted to the maximum. The number of LEDs 84 of the LED groups 92 and 94 is preferably 10 or more, but the number may vary depending on system characteristics. In addition, the number of LED groups of the LED array 85 may be changed.

In FIG. 9, four LED groups 92 are located at the edge 96 of the LED array 85, where the number of LEDs 84 is preferably 15 or more. Four LED groups 94 disposed on the side 98 of the LED array 85 are located between the LED groups 92 at the corners.

It is best to arrange the LED array 85 on the circuit board 86. The control circuit of the circuit board 86 together with the controller 90 controls the operation of the LED 84. Referring to the block diagram of FIG. 11, a controller 90, such as a microprocessor, and a power source 88 are connected to a circuit board 86. The LEDs can be controlled without the controller 90, for example, by turning the power supply 88 on and off. It is also conceivable to use pulse modulation techniques to control the LEDs 84, such as the power supply 88, to suit the duty cycle. Here, the "on" time for the "off" time of the LED is called the duty cycle.

In the imaging system 70 of FIG. 11, an LED array 85 is connected to a power supply 88 and a controller 90 via a circuit board 86. The controller 90 controls the LED array 85 to emit infrared light towards the object 71. The controller 90 controls the LEDs 84 in the LED array 85 to emit infrared light continuously or intermittently. That is, the LED 84 is selected to emit infrared light toward the object 71 intermittently or continuously. Thus, the imaging system 70 is configured to emit infrared light from the LED array through various arrangements or combinations of the LEDs 84 or LED groups 92,94.

Referring to FIG. 10, the first diffusion plate 100 is adhered to the light emitting surface 102 of the LED 84 in the LED array 85 to diffuse the light from the LED 84. As the first diffusion layer 100, a holographic 20 degree diffusion plate such as LSD20PC10-F10x10 / PSA manufactured by Physical Optics Corporation of Torrance, California is most preferred. The first diffusion plate 100 is 3.5 inches long, 3.5 inches wide, 0.1 inches thick is good. When the LED 84 is activated, the first diffuser 100 diffuses the infrared rays of the LED array 85.

When the inner surface 104 of the housing 72 is reflective-coated with white paint or the like, light diffused from the diffuser plate 100 is reflected and further diffused. The second diffuser plate 106 is separated from the first diffuser plate by LDD, and this distance (LDD) is best to be about 3 inches. Similarly, the second diffuser 106 is preferably a holographic 20 diffuser with a length and width of 3.5 inches and a thickness of 0.1 inches.

The second diffuser plate 106 further diffuses the diffused light supplied from the first diffuser plate 100 and reflected from the inner side 104. 8, two diffuser plates 100 and 106 are flat and parallel to each other. This allows a constant quantification of the amount of diffused light exiting system 70 when LED 84 is activated.

The backing material 108 of FIG. 10 is preferably a material manufactured by DuPont and sold under the trademark LUCITE, and placed next to the second diffuser plate 106, with a thickness of 0.125 inches. The polarizer 110 next to the backing material 108 is most preferably sold as VP-GS-12U by Visual Pursuits, Illinois, with a thickness of 0.075 inches.

Thus, the imaging system 70 passes through the first diffuser plate 100 and reflects off the inner surface 104 of the first compartment 72a and then twice the WO diffuser plate 106, the backing material 108, and the polarizer plate 110. It can pass through a variety of diffusion effects. After the light passes through the first diffuser plate 100, a certain level of diffusion occurs, and the light already diffused in the diffuser plate 100 is reflected to another level by reflection at the inner surface 104 of the first compartment 72a. Diffusion occurs and another level of diffusion occurs through the second diffuser plate 106.

According to FIG. 8, it is preferable that a circular central portion 112 having a diameter of 1 inch is located at the center of the polarizing plate 110. The shape of the central portion 112 preferably matches the camera lens 75. The degree of polarization of the central portion 112 of the polarizer 110 is inclined by about 90 degrees compared to the degree of polarization of the periphery 114. The camera lens 75 contacts the backing material 108. In FIG. 8, the position of the lens 75 inside the housing 72 coincides or partially overlaps the central portion 112 of the polarizer 110. When the front of the lens 75 and the central portion 112 of the polarizer are matched, all surface glare (specular reflection) is removed from the camera image.

Referring to FIG. 10, the backing material 108 and the polarizer 110 are disposed in the same direction as the diffusion plates 100 and 106. The first diffuser plate 100, the inner surface 104, the second diffuser plate 106, the backing material 108, and the polarizer 110 form a diffuser structure 116 for shining diffused light on the object 71. This diffusion structure is only an example and does not limit the present invention. For example, the polarizer 110 may or may not be provided with one of the diffuser plates 100 and 106, and there may be only two diffuser plates 100 and 106 without the polarizer 110.

The system 70 is activated and then shines the diffused light 80 onto the object 71 and makes a video image of the object with the camera 74. Specifically, when the power source 88 is connected, infrared light is emitted from the light emitting surface 102 of the LED 84, infrared light is diffused in one step from the first diffusion layer 100, and diffused in two steps from the inner surface, and the backing is performed. Existing diffused light is diffused through the ash 108 and the polarizing plate to illuminate the object 71 in three stages in the second diffusion layer 106. As described above, the diffused light 80 is reflected by the object 71, The diffuse reflected light 82 is made, and the diffuse reflected light is captured by the camera 74. The camera 74 then creates a video image of the object, thus emitting diffused light according to the imaging system 70 of the present invention. If the properties of the object 71 is different, such as blood vessels and body tissues can be found by separating the objects.

Those skilled in the art will appreciate from the foregoing description and drawings, but those described above may be changed or changed. For example, the diffusion strengths of the system 70 may be varied so that the diffusion plates 100, 106 are not parallel. In addition, although the LED array 85 is best parallel to the first diffuser plate 100, it will be apparent to those skilled in the art to arrange them at various angles. Therefore, it should be understood that the above description is merely exemplary and does not limit the scope of the present invention.

20 are photographs of images projected by blood vessels covered with body tissues.

A description will now be given of a camera, an illumination source for photographing a structure below the surface of an illuminated object, and another configuration of the projector for projecting the processed image back to the surface of the object. Common features of various embodiments of the present invention will be omitted, and only those described above will be understood by those skilled in the art.

Because the present invention differs from the prior art in projecting a hidden structure image back to an object surface (not a monitor or screen far away from the object surface), the observer using the present invention does not experience the parallax problem that occurs when the conventional axis is misaligned. Do not. An important feature of all embodiments is that the camera is fed to the image that is projected back to the object surface, while the first spectrum of the hidden structure image taken by the camera is outside the second spectrum of the image that is projected back to the object surface. Since the spectrum of the captured image and the projection image of the hidden structure do not overlap, it is possible to effectively process the image of the hidden structure without interference of the projection image. Since the projected image is in the visible spectrum and the illumination of the camera is in the infrared spectrum, the two spectra do not overlap. On the other hand, by illuminating the object with a broad spectrum of ambient light instead of the light having an infrared spectrum and installing an infrared filter in front of the camera may filter out all the spectrum other than the infrared spectrum, so that only the infrared component of the light reflected from the object can be seen.

A third imaging system 130 is shown in FIG. The existing near-infrared illumination CCD camera 132 with a lens is used as a photographing machine as in the conventional example. As described above, the second polarization filter 134 is disposed between the CCD camera and the reflected light of the photographed object to reduce the specular reflection of the object surface. An illumination source, a first polarization filter, a holographic illumination diffusion ring, and an optically neutral glass cover constitute the assembly 136, which will be described in the fourth embodiment with reference to FIGS.

This embodiment also uses a conventional video projector 138, called a "light engine," to project an image onto the object under inspection. The video projector 138 preferably has a high output light intensity, because the output light intensity is a determinant factor for viewing the projected image well under normal room lighting. The video projector 138 has a high-intensity LED light source 140 that sends light to the existing prism assembly 142, so that the light is bent backward by the internal reflection of the prism assembly 142, the existing digital light processing (DLP) device The DLP device is also referred to as a digital mirror device (DMD), and compact mirrors are concentrated to adjust the direction of the reflected light to each mirror through the existing projection lens 146. By not illuminating the target object or causing the light to shine on the target object, the light can be turned on and off to the pixel base as is known to those skilled in the art. Using the prism assembly 142 can further reduce the size of the imaging system, it is well known to those skilled in the art.

As described above, the hot mirror 148 is disposed at 45 degrees to reflect infrared rays reflected from the object toward the camera 132. The hot mirror 148 acts as a mirror for long wavelength light (eg infrared), but high frequency light, such as green light from the projector 138, is not reflected and passes through the hot mirror to face the object.

The imaging system 130 uses the first and second lasers 150 and 152 to properly position the target at the focus of the camera 132, which will be described later.

A fourth image system 154 of the present invention will be described with reference to FIGS. 13 to 14.

Installation of imaging system 154 on pole 156 extending upward from cart 158 facilitates transportation. The fine focus 160 may be properly positioned on the target object O while moving up and down the imaging system 154. Like other embodiments, a 525 nm green LED light source 164 (photon engine) illuminating the DMD / DLP chip 166 is disposed in the video projector 162. As the light source 164 used in this embodiment, a PE09-G illuminator, which is a Teledyne model with an output intensity of 85 lumens, is suitable. DMD chip 166 is a 0.7SVGA SDR DMD chip manufactured by Texas Instruments with a resolution of 848x600 pixels, a mirror tilt of 10 degrees, and a frame rate of 30 Hz. Similarly, the existing prism assembly 168 reflects the light of the light source 164 inward toward the DMD chip 166 and then toward the object O. The DMD chip 166 is controlled by an existing electronic board 167 manufactured by Optical Science Corp.

Placing the condenser lens 170 between the light source 164 and the prism assembly 168, the BK7 bioconvex lens 013-2790-A55 with a BBAR / AR coating surface sold by OptoSigma is preferred and the coating surface is 425 ~. It is suitable for 675nm wavelength. The light from the projector passes through the prism assembly 168, the projection lens 172, Besler's 8680 magnification lens, and the conventional high-pass filter hot mirror 174, where the infrared image of the object O is the second polarized light. Reflected through the filter 178 to the camera 176. The camera 176 is a Firefly camera FIRE-BW-XX sold by Point Gray Research, which uses a 640x480 CCD chip and a Sony ICX084AL, which can be imaged to a computer 180 via an IEEE-1394 ("FireWire") interface. Send it. Several interface signals 181 of the computer 180 communicate with the imaging system in a conventional manner. As briefly mentioned in the third embodiment, the fourth embodiment also uses two lasers 150 and 152 to properly position the target object O in the focal region of the camera 176.

Like the imaging system 130 of FIG. 12, the assembly 136 of the fourth imaging system 154 of FIGS. 12 to 14 also diffuses the light of the infrared light source 12, the first polarization filter 184, and the LED 190. It has a holographic illumination diffusion ring 186 and an optical neutral glass cover 188, the polarizing filter has a hole in the center does not affect the projection image of the projector 162 or the image of the object, the diffusion ring is also central A hole is drilled in and the projected image and object image pass through this hole. The infrared light source 182 is an annular array of LEDs, and the projection image and the object image pass through the central hole. As the LED, the near-infrared LED 190 of 740 nm that illuminates the object O is preferable, and studies have shown that such a structure can provide diffuse infrared light enough to illuminate the object O.

A fifth image system 192 of the present invention will now be described with reference to FIG. 15. The difference from other imaging systems is that sunlight (S), which has a broader spectral range than conventional infrared light sources, does not use an integrated diffuse infrared light source (e.g. light source 182 with LED 190 ring) to illuminate the object. Or ambient light (L). Although the ambient light contains infrared light and can be diffused, the intensity of the infrared light is lower than that of the infrared light source described so far. Therefore, better performance is needed for a more sensitive camera.

Similarly, the fifth imaging system 190 includes a projector 162, a light source 164, a prism assembly 168, a projector lens 172, and a DMD chip 166. For miniaturization, a fold mirror 194 is disposed between the light source 164 and the prism assembly 168 at a right angle. Of course there is also a hot mirror 174.

An infrared filter 196 is disposed in the optical path between the camera 198 and the object O to filter out all infrared components of the captured image. The camera 198 is a CMOS camera made by Basler Vision Technologies of Germany, A600-HDR, which uses an IEEE 1394 (FireWire) interface and can capture images up to 112 dB. An advantage of this embodiment is that it can be used even in bright rooms.

Experiments show that some people have a lot of hair on their limbs that make it difficult to clearly understand the subcutaneous structure. All hairs, even white hairs, appear black in the near infrared. Therefore, small dark artifacts such as body hair should be removed from the image, but the remaining large and dark objects should be left to process the received image further. 16a to b are artifact removal programs of the received image. The program is run twice, followed by a conventional edge enhancement procedure. For example, unsharp masking followed by smoothing removes the hair artifacts. The accompanying program is well known to those skilled in the art of image processing.

A received image having an integer unit pixel value in the range of 0 ... 255 is converted into a floating point value of 0.0 to 1.0. The image is finalized using a Gaussian convolution of eight sigma pixels. This image has a fairly small sigma value and retains only very small hairs. Subtract the Gaussian image from the original image to create a "difference image" with pixel values concentrated around 1.0. Since even white hairs appear black in near infrared, negative pixel values indicate body hair, thereby replacing those pixels with corresponding pixels in a Gaussian image. This is the first step in processing the received image. Next, create an array with 0.0 or 1.0 values by setting all negative pixel positions ("hair" position) to 1.0 in the "other image" and setting the remaining pixel positions to 0, where all "hair pixels" Is 1.0 and the remaining pixels are zero. The original image "im1" having a pixel value of 0.0 to 1.0 is "boost" by 0.015 at all "hair pixel" positions.

Gaussian smoothing of eight sigma pixels, another image generation, negative pixel position checking, and "boosting" image with negative pixels are once again performed to complete the image with sigma 64 pixel Gaussian convolution. If you create a third different image and subtract the finished image from the "boosted" image once more, you get an image that consists of the absolute values of all the pixels of the third different image. The third image is divided into an absolute value image obtained by finishing the absolute value image with the Gaussian convolution of 64 sigma pixels, and the image segmented by the Gaussian convolution of the four sigma pixels.

According to the above artifact removal program, the contrast can be set by the contrast of subcutaneous blood vessels, and the artifacts (hair) can be ignored, and an image with improved outline can be produced. Factors such as sigma values, thresholds, and the like may vary depending on the age of the subject, degree of coloration, and the like.

FIG. 17 is a program written in the C ++ programming language for performing artifact elimination image processing of a received image based on the program of FIG. 16, and uses the Intel image processing library to speed up mathematical operations.

All embodiments of the present invention employ a mechanism for maintaining an image of a hidden structure using a camera having a suitable lens-object distance. 18, in this mechanism, a pair of lasers 150, 152 are used, and beams 200, 202 are fired out of parallel in each laser, and these two lasers if the target is at a suitable distance (e.g. 206 plane) from the camera. The beams intersect at one point 204 of the intersection plane 206. If the target is closer to the camera (e.g. 208 plane) or farther away (e.g. 210 plane) than the proper distance, the two laser beams do not intersect at one point 204, but a pair of points 212,214 of 208 plane Points 216 and 218 in the 210 plane appear on the surface of the object, meaning that the hidden object is not focused on the camera and must be refocused by changing the distance to the camera. Lasers 150 and 152 can also be seen in FIGS. Suitable for the present invention is a model LM-03 laser manufactured by Roithner Lasertechnik, Vienna, Austria.

The second mechanism adds visible light patterns (eg borders) to the projected image regardless of the hidden structure to be photographed. Since the projected recognition pattern can be seen by the human eye only when it appears on the surface of the target object at the proper distance, the hidden structure under the target is also positioned at the proper distance from the camera. This recognition pattern may be a cartoon or a logo or name of a hospital or a medical center, as desired. The projected image of the hidden structure is slightly blurred through the artifact removal process, but if this pattern is out of focus, the human eye can recognize it quickly. The advantage of the second mechanism is that there is no risk of blindness or injury by projecting visible light patterns without the use of a laser, which can be done without proper safety measures.

21 is a photograph of a projected image represented by a frame border. FIG. 22 is similar to FIG. 21, but is a photograph showing that the object is not properly positioned because the border of the sentence is not in focus by changing the position of the object. 23 shows an example of a sentence border combined with a projected image to properly position an object. In some cases, the border of the sentence looks reversed because it reflects the image inside the prism assembly, but when you project it, it looks right. Trim the projected image properly before combining it with the border so that the border is clear and clear in the projected image.

24 is a photograph of a processed image of a subcutaneous blood vessel when projecting light onto the hand in accordance with the present invention. This picture is similar to FIG. 21 and FIG. 20 (without a border), but shows that the border is out of focus and the hands are not in place.

As shown in FIG. 19, when the green pattern 222 is projected on the fluorescent screen 222 by the video projectors 138 and 162, four green dots are changed into red dots that can be viewed by the infrared camera 132. The computer program has four projection points (P1-4) with Cartesian coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) and, if properly aligned, the "real" position (X1). , Y1), (X2, Y2), (X3, Y3), (X4, Y4), and calculate the correction coefficients (a to f) to be used for the dual linear transformation equation, and calculate the size, angle of rotation, and camera and projector. Correct misalignment between Figures 25A-B show a full MAPLE 9 computer equation solver for double linear transformation coefficients as a function of the values measured during calibration. This calibration factor is used to convert the coordinate system (x, y) of the image into the calibration coordinate system (X, Y) needed to produce the calibrated image during operation of the device. FIG. 26 illustrates a method of performing high-performance image alignment using a double linear transformation equation by using coordinates once calculated as variables for an existing image processing library math routine provided by an integrated circuit to be used in a processor. This run-time correction is done using scale integer calculations rather than floating point calculations for faster image processing.

During the calibration process, a test pattern 220 consisting of four points (P1-4) is projected onto the fluorescent screen, each of which has a 25 pixel radius (as seen from the camera) at a rectangular edge of 320x240 pixels (as seen from the camera). Have For example, the resolution of camera 132 is 640x480 pixels while the resolution of projector 138 is 1024x780 pixels. Experiments in which the radius of the point varied between 4 and 50 pixels showed that the standard deviation of 100 samples rapidly decreased from a radius of 5 pixels to 25 pixels but much slower to 50 pixels.

For the calibration method of the present invention, a test pattern of four points P1 to 4 in the first spectrum, preferably in the green light range, is projected onto the fluorescent screen 222, followed by light in the vicinity of or within the infrared spectrum, the second spectrum, for example The camera 132 emits light, such as visible red light, and even transmits infrared light through the filter so that the camera can see the target object. Next, the calibration software measures four observation positions, calculates the calibration coefficients (a, b, c, d, g, f, h, k) of the double linear transformation equation, and uses these coefficients in the linear transformation equation. Correct the misalignment (rotation, translation, magnification) between the camera and the projector. At this time, the projected image is corrected by warping the image prior to projection. This process can correct several different misalignments in the horizontal and vertical directions, and also correct different translation errors in the horizontal and vertical directions.

According to the test, the misalignment to ± 25.4mm can be corrected within the interpolation range of the camera pixel size during this calibration process. The alignment is best around four points in the test pattern, but overall improves significantly.

Claims (12)

In a device for improving the sharpness of a structure hidden under the surface of an object: A photographing apparatus that receives an image of diffused light reflected from an object and makes an image; A first laser emitting a first beam; And A second laser emitting a second beam; And the first and second beams are not parallel to each other and intersect at a desired target distance, and when the intersection is on the surface of the object, the focus of the hidden structure is brought to the camera. 2. The apparatus of claim 1, further comprising a video projector for projecting a visible light image of the hidden structure onto the surface of the object. An imager for receiving an image of diffused light in a first spectral range reflected from an object to make an image; And a video projector for projecting a visible light image of a structure in a second spectral range that does not overlap with the first spectrum to the surface of the object, the method comprising the steps of: (B) projecting a pattern of points separated from each other in the second spectral range onto a fluorescent screen to make it shine; 위치 observing the position of the point of the fluorescent pattern; Calculating the correction factor for the double linear transformation using the observed position; And And calibrating subsequent projection images using the dual linear transformation having the correction coefficient. In the method of improving the sharpness of the structure hidden under the surface of the object: Providing a camera that receives diffused light reflected from an object; Creating a received image; 작 removing small and dark artifacts from the received image, but leaving the large and dark objects intact to create a processed image; And Projecting the processed image onto a surface of an object in the visible light spectral range. 5. A method according to claim 4, characterized in that an adaptive contour enhancement of the processed image is performed after projecting the processed image and before projecting, i.e., between step V and step V. In a device for improving the sharpness of a structure hidden under the surface of an object: A photographing apparatus that receives an image of diffused light reflected from an object and makes an image; And And a video projector for projecting a visible light image of the hidden structure onto the surface of the object. Apparatus for remarkably adding a pattern independent of the hidden structure to the projected visible light image. The apparatus of claim 6, wherein the added pattern is a border. In a device for improving the sharpness of a structure hidden under the surface of an object: A photographing apparatus that receives an image of diffused light reflected from an object and makes an image; A video projector for projecting a visible light image of a hidden structure onto an object surface; And And an infrared filter between the imager and the object. In a device for improving the sharpness of a structure hidden under the surface of an object: A photographing apparatus that receives an image of diffused light reflected from an object and makes an image; A video projector for projecting a visible light image of a hidden structure onto an object surface; And Apparatus comprising a; light source for illuminating the object. 10. The apparatus of claim 9, wherein the camera receives only diffuse reflected light in a first spectral range and the visible light image has a second spectrum that does not overlap with the first spectrum. 10. The device of claim 9, wherein an infrared filter is disposed between the imager and the object. In a device for improving the sharpness of a structure hidden under the surface of an object: A photographing apparatus that receives an image of diffused light reflected from an object and makes an image; A video projector for projecting a visible light image of a hidden structure onto an object surface; And And an infrared filter between the imager and the object.

Images