JP6183625B2 - Transparent display and fluoroscopic image observation apparatus using the same - Google Patents

Description

  The present invention provides a display image and an image from the back surface by providing pixels for a light transmitting window that transmits light from the back surface in addition to the display pixels in the display surface pixels of the display device such as a liquid crystal display device. The present invention relates to a transmissive display that can be seen in a superimposed manner, and to a transparent image observation apparatus using the transmissive display.

  Conventionally, when transmissive display is performed, a half mirror can be used to superimpose an image of light entering the mirror at an angle of 45 ° and an image of light that is perpendicular to it and that extends from the direction of the line of sight. Yes, in Patent Document 1, a blood vessel visualization device using the same is shown. However, this appears to be a composite of the display image and the naked eye image, but a half mirror is arranged between the practitioner and the affected part of the patient, and the display device is horizontal (example in FIG. 2 of Patent Document 2) or half There has been a demand for a simpler transmissive image display device that is disposed between the mirror and the practitioner (the example of FIG. 3 of Patent Document 2) and that is obstructive to the practitioner.

JP 2004-267534 A

An object of the present invention is to provide a transmissive display that allows a user of the display to see the back side image through the display surface so as to be directly superimposed on the display image on the display surface, and It is to provide a fluoroscopic image observation apparatus using this.

In the transmissive display of the present invention, the display surface of a display such as a liquid crystal display or an EL (electroluminescence) display is alternately provided with an image display portion and a light transmission window portion that transmits light from the back surface. A fluoroscopic image observation apparatus characterized by having an image acquisition device, an image processing device, and a controller for controlling them.

The invention according to claim 1 is a transmissive display,
When a display and an image memory are provided, and an image display unit and a light transmission window that transmits light from the back surface of the display surface are arranged on the display surface of the display,
In the image memory, an image data portion corresponding to the image display portion and a transmission data portion corresponding to the light transmission window portion are alternately arranged for each pixel or a plurality of pixels, thereby displaying the display surface on the image display. by the light transmission window portion and parts, seen superimposed display image displaying the image data in the image display section, an image formed by transmitted light coming from the back of the display surface coming transmitted through the light transmitting portion It is possible to do this.

The invention according to claim 2 is the transmissive display according to claim 1,
The image data portion and the transparent data portion on the image memory are switched for each image display screen or a plurality of screens.

The invention according to claim 3 is a fluoroscopic image observation device,
An image acquisition device for acquiring an image of an observation object to be observed, an image processor for performing processing to display an image acquired from the image acquisition device, and the display for displaying a result of the processing And the operation of the transmission display capable of observing the object to be observed existing through the display surface with the naked eye, the image acquisition device, the image processing device, and the transmission display. A control unit for controlling the display image of the object to be observed acquired by the processing of the image processor, and the observation image can be superposed on the fluoroscopic image with the naked eye. Is a transmissive display device according to claim 1 or claim 2.

Since it is configured as described above, in the transmissive display of the present invention, the display image displayed on the image display unit of the display surface and the back side image transmitted from the light transmission window of the display surface are directly superimposed. Therefore, when the head mounted display is used, the naked eye image and the display image can be viewed simultaneously.

It is a figure which shows one embodiment of the transmission display by this invention. It is a figure which shows the general structure of the liquid crystal display which can be used for the transmissive display by this invention. It is a figure which shows one Example of the blood-vessel visualization apparatus which is a fluoroscopic image observation apparatus using the transmission display by this invention. It is a figure which shows one Example of the welding protective spectacles apparatus which is a fluoroscopic image observation apparatus using the transmission display by this invention.

The transmission display according to the present invention is characterized in that the image display unit and the light transmission window unit that transmits light from the back surface are alternately arranged on the display surface of the display unit, and the image display unit and the light transmission window unit are arranged. There are two ways to do this. 1 is a physical arrangement of the image display unit and the light transmission window itself, and 2 is an image memory for storing a transmission type display such as a transmission type liquid crystal display and image data to be displayed on the display (for example, The frame is also called a frame memory), and the image data portion and the transparent data portion in which white data (becomes the transmissive portion) is always placed in the image memory are alternately placed for each pixel or a plurality of pixels. In this case, the image data on the display surface is replaced with the image data portion and the transmission data portion of the image memory for each frame (screen) of the image display. The display unit and the light transmission window unit can be interchanged.
Hereinafter, examples will be described with reference to the drawings.

FIG. 1 is a diagram showing an embodiment of a transmissive display according to the present invention.
In 1-A, the display surface 101 of the transmissive display device has a large number of display pixels (pixels).
On the enlarged display surface 102, a part of which is enlarged, an image display unit 103 that displays image data and a light transmission window unit 104 that is always white (transparent and capable of transmitting light from the back side) are alternately arranged. Has been.
In 1-B, the display surface 101 of the transmissive display similarly has a large number of display pixels (pixels). However, in this case, in the image memory 105 that stores image data to be displayed on the display surface 101, the image data unit 107 and the white (transparent part) are always displayed as seen by the image memory enlargement units 106A and 106B. ) Are alternately arranged, and as a result of display corresponding thereto, image display portions and light transmission window portions of the display surface 101 are alternately configured. Here, the image display section and the light transmission window section may be fixed as in the case of only the image memory expansion section 106A, and in this case, this is substantially the same as the case of 1-A.
If, for example, switching is performed between the image memory expansion units 106A and 106B for each frame to be displayed, the display image and the transmission image do not appear at different positions to the eyes, and it is possible to see them completely overlapping. .
In both 1-A and 1-B, the image display unit and the light transmission window unit may be alternately arranged for each pixel or a plurality of pixels.

1-C is a block diagram of a transmissive display device. The transmissive display device includes a light transmissive display element integrated device 110, and the light transmissive display element integrated device 110 includes an image display unit on its own display surface. The light transmission window portions are alternately formed, or an existing transmission type (in this case, display by controlling the passage of light from the backlight), a transmission type display such as a liquid crystal display, and the image processor 111. The image memory 105 can be used, and either the image data portion 107 and the white data (becomes a transmissive portion) can always be stored in the image memory 105. These perform predetermined operations by the controller 112.

FIG. 2 is a diagram showing a general structure of a liquid crystal display that can be used in a transmissive display according to the present invention.
This figure shows a cross section of a general transmission type (booklight type) liquid crystal display device, which was cited from materials of Sharp Corporation. First, an outline is given using explanations from the materials.
A voltage is applied to the liquid crystal array substrate 201 in which the liquid crystal is filled in the space sandwiched by the transparent dielectrics by the conductive transparent electrodes 202A and 202B such as ITO films. The liquid crystal is rotated by the voltage, and the passage of light can be controlled. A color filter substrate 203 is on the right side of the transparent electrode 202B, and a polarizing plate 204 and a transparent light guide plate 205 are disposed on the right side and the left side of the other transparent electrode 202A. Although this shows a cross section, each pixel is arranged side by side on the display surface with such a structure. For the backlight light entering from the left, the voltage corresponding to the data of each pixel is related to the transparent electrodes 202A and 202B, and the degree of light transmission is determined. As a result, the color and its brightness are determined by the intensity of light passing through the R (red), G (green), and B (blue) filters of the color filter substrate 203.

When this general liquid crystal display is used for the transmissive display according to the present invention, when the image data 105 and the white data (which becomes the transmissive part) are always stored in the image memory 105, the liquid crystal display is It can be used as it is without any changes. This is because the image memory 105 side determines the image display portion and the light transmission window portion.
When creating an image display part and a light transmission window part on the display surface itself, two means can be considered. The first is that the electrodes are forcibly biased so that a voltage is applied between the transparent electrodes of the light transmission window so that the liquid crystal is completely transmitted. It is not much different from the previous example in the sense that there are few changes. This can be done by changing the electrode connection of the liquid crystal display.
The second is to actually take a space of only a transparent member between the liquid crystal display pixels on the display surface and use this as a light transmission window portion. In this case, it is necessary to create a new display device layout design.

An example of a fluoroscopic image observation apparatus using the above transmissive display will be described with reference to FIGS.
FIG. 3 is a diagram showing an embodiment of a blood vessel visualization device which is a fluoroscopic image observation device using a transmission display according to the present invention. In 3-A, a blood vessel visualization device 300, a treatment section 301 (arm here) and a treatment tool 302 (here syringe) are shown. The blood vessel visualization device 300 is mounted on the head 303 of a practitioner (doctor or nurse), and an illumination light source 304 that emits infrared light, preferably near-infrared light (wavelength 760 to 2000 nm). (A visible light source may also be included if necessary), an image acquisition unit 305 that captures the reflected light of the light treatment unit 301 as an image, an image processing unit 306 that processes this image, and an image processing unit 306 It has a head mounted fluoroscopic display 307 capable of displaying an image output on a display surface and seeing through the display surface and viewing the treatment portion, and a controller 308 for controlling these operations. The processing device 306 is
The image arithmetic unit 309 and the image memory 312 have an image matching unit 310, and the image matching unit 310 displays the image on the display surface of the fluoroscopic display 307 (from the image acquisition unit 305). The image of the treatment part including the blood vessel image obtained by image acquisition is obtained by aligning the obtained treatment part image with the (visual) treatment part image obtained visually through the fluoroscopic display 307. Thus, it is possible to perform the treatment while the display image and the visual image are matched.

3B shows a block diagram of a configuration example of the blood vessel visualization device. The operation will be described using this. As the illumination light source 304, there is at least an infrared light lamp 304A, preferably a near-infrared light (wavelength 760 to 2000 nm) lamp, which is applied to the treatment portion 301 of the person to be treated. The treatment unit 301 reflects the irradiation light and is acquired as an image by the image acquisition unit 305. The image acquisition unit 305 is a camera having sensitivity to at least infrared light or infrared light transmission that transmits infrared light. It has a camera with a filter.
The reflected light from the treatment part 301 of the infrared light lamp or near-infrared light lamp becomes darker because the portion with blood vessels is absorbed in comparison with the other portions without blood vessels, and it is inside the skin of the treatment portion 301. A certain blood vessel image can be obtained.
In addition, when a visible light camera is added to the image acquisition unit 305, it is better to switch between the infrared light transmission filter and the visible light transmission filter at a high speed with one camera, rather than providing each, The scale, orientation, etc. are consistent and convenient.
When a visible light camera is added, an infrared light image with a blood vessel image and a visible light camera image (substantially the same as the naked eye image) are obtained. If there is one visible light image and one infrared light image Since the two images match, the images are superimposed and integrated as it is, so that the display image is easy to see. As described above, the visible light camera is not essential, but if it is present, it gives an advantage that an image close to the naked eye can be taken and the display is easy to see.

The image obtained by the image acquisition unit 305 is processed by the image processing unit 306, and is displayed on the fluoroscopic display unit 307 that can see through the display surface and visually observe the treatment portion. In this case, in the image processor 306, the image calculation unit 309 includes an image matching unit 310 and an image processing unit 311, which are essential components in the image matching unit 310, and are obtained from the image acquisition unit 305. When displaying the image of the treated portion on the fluoroscopic display 307, the position and size (scale) of the image is set so that the practitioner can see the treated portion through the fluoroscopic display 307 and match the macroscopic image. Then, an image whose orientation is adjusted is created, or a numerical value to be corrected is determined, stored in the image memory 312, and displayed on the fluoroscopic display 307. As an example of the operation of the image matching unit 310, an infrared light image or a visible light image of the treatment unit may be matched with a naked eye image seen through the fluoroscopic display 307. However, as described above, the treatment unit is like an arm. Since the outline is unclear, the clear image of the mark and the fluoroscopic display are temporarily placed on the treatment part with the normal space between the blood vessel visualization device mounted on the practitioner and the treatment part. The position, size (scale), and orientation are adjusted so that the image of the mark with the naked eye seen through 307 matches, and the correction values are stored in the numerical table of the image memory 312. If so, since it is often used in the same state with the patient, it can always be corrected by applying this numerical value. The correction numerical value input in this case may include a distance and a specified value obtained from theoretical calculation of the camera image system, but it is convenient if there is an adjustment input 313 so that the numerical value can be adjusted from the outside.

In the matching of the image matching unit 310, the numerical values entered in the numerical value table may include distances and specified values obtained from the theoretical calculation of the camera image system, but if there is an adjustment input 313 so that the numerical values can be adjusted from the outside. As mentioned above, it is convenient. Considering that the practitioner's hand is blocked, the adjustment input 313 preferably has a configuration that does not use a hand such as a foot switch. Up / down / left / right movement (position), scale (size), direction (tilt angle) are corrected by the switch and the number of steps and the time of stepping, and the reference mark image data is corrected (for example, pressing the right turn switch once) And the like, and the final correction value is stored in a numerical table. The same data is applied every time when the correction input is fixed without moving.
Also, if the distance between the practitioner and the treatment site is large, the image taken with the camera and the naked eye image will be small, but there is a certain relationship between the distance and the size of the image. If the correction numerical value is obtained and entered in the numerical value table, it is possible to automatically correct with reference to this data.

The image processing means 311 is a generally-known image processing means, and it is convenient if it is not essential. In order to make it easy to see the blood vessel image of the image acquired by the image acquisition unit 305, processing for increasing the contrast or colorization (the infrared light image itself is black and white, for example, the blood vessel image is colored in red). Or contour enhancement processing (the color of the contour is darkened or blackened) to clarify the contour of the blood vessel image, or the brightness and darkness of the naked eye image and display image viewed through the fluoroscopic display 307 are easily adjusted, or The obtained image can be displayed in a transparent manner, and many commercially available programs can be used.

The controller 308 controls the operations of the illumination light source 304, the image acquisition unit 305, the image processing unit 306, and the fluoroscopic display unit 307, and usually includes a computer, a processing program, a switch, a power source, and the like.

The fluoroscopic display 307 has an image display portion and a transmission window portion through which reflected light of an object on the back side of the display surface is alternately distributed on the display plane, so that it can be viewed from the front side of the display surface. In this case, the display image and the transparent image on the back surface can be seen overlapping. As an example of this, an appropriate transmissive window always transmits light, and a display unit is arranged around it, which is alternately repeated. For example, when a liquid crystal display is used, the electrode of the liquid crystal interferes with the transmissive window. It is possible to use a structure in which a transparent window such as an ITO film or a transparent transparent window is used so as not to be formed. Further, the display pixels of the liquid crystal are divided into a display portion and a transmission window portion, which are configured by alternately arranged portions. The display unit displays an image. The transmissive window is in a state where the liquid crystal pixels are always fully open so that the light on the back side passes.
In this way, a transmissive display 307 can be realized by using a transmissive liquid crystal (in this case, the meaning is different from the transmissive window (meaning that the back side can be seen through) in the sense that the backlight is transmitted). In addition, the data part and the white part are exchanged for each frame or a plurality of frames (for example, in the case of 30 frames / second for normal image broadcasting, 15 frames per second, the data part and the white part for each frame). Since the screens in which the screens are exchanged can be alternately displayed, the eyes appear to overlap the display and the fluoroscopic image on the entire display surface as described above with reference to FIG.

It is a figure which shows one Example of the welding protective spectacles apparatus which is a fluoroscopic image observation apparatus using the transmission display by this invention. In 4-A, the welding operator performs welding by placing the welding rod 401 against the working portion to be welded. Here, the head is covered with the welding protective eyeglass device 400, which includes an image acquisition unit 402 (camera) that takes an image of the welded portion and an image memory that processes the obtained image and stores the image. An image processor 403, a transmission display 404 for displaying the result, a filter 405 for removing light harmful to eyes when welding through the transmission display 404 with the naked eye, and a mask 406 for preventing contact with sparks Have. A block 4-B of the construction of this welding protective eyeglass device is shown. The image acquisition device 402, the image processing device 403, the transmissive display 404, a controller 407 for controlling them, and a filter 405 in front of the transmissive display 404 are provided. The operation is as described above. The functions of the image processor 403 and the filter 405 are added.
In the image processor 403, the actual welded image enters the image acquisition unit 402 together with an extremely large flash of light and light from the image of the weld buried therein. Since this light contains a large amount of ultraviolet rays, an ultraviolet cut filter is inserted in front of the image acquisition device 402 and as a filter 405. Furthermore, since the intensity of visible light is extremely high, a filter that suppresses the intensity of visible light is also used, or the intensity is compressed in the image processor 403 so that it falls within the sensitivity range of the human eye. indicate. The display image does not need to display all the images as they are, and may be an image having a character necessary for welding. Therefore, it is not always necessary to display a bright image such as a flash. From the naked eye, the weld can be observed through the transmission display 404 and the filter 405.

  As described above, the transmissive display device according to the present invention can realize a fluoroscopic image observation apparatus that can observe the image acquired by the image acquisition device and the naked eye image seen through the transmissive display device. Industrial applicability is extremely large in the application of requirements.

DESCRIPTION OF SYMBOLS 101 Display surface 102 Enlarged display surface 103 Image display part 104 Light transmission window part 105 Image memory 106A, 106B Image memory expansion part 107 Image data part 108 Transmission data part 110 Light transmission display element integrated device 111 Image processor 112 Controller 201 Liquid crystal Array substrate 202A, 202B Conductive transparent electrode 203 Color filter substrate 204 Polarizing plate 205 Transparent light guide plate 300 Blood vessel visualization device 301 Treatment unit 302 Treatment tool 303 Head 304 Illumination light source 305 Image acquisition device 306 Image processor 307 Fluoroscopic display device 308, 407 Controller 309 Image operation unit 310 Image matching unit 311 Image processing unit 312 Image memory 313 Adjustment input 317 External display monitor 400 Welding protection glasses device 401 Welding rod 402 Image acquisition unit 403 Image processing unit 404 Transmission display unit 405 Filter 406 Mask


that's all.

Claims (3)

When a display and an image memory are provided, and an image display unit and a light transmission window that transmits light from the back surface of the display surface are arranged on the display surface of the display,
In the image memory, an image data portion corresponding to the image display portion and a transmission data portion corresponding to the light transmission window portion are alternately arranged for each pixel or a plurality of pixels, thereby displaying the display surface on the image display. by the light transmission window portion and parts, seen superimposed display image displaying the image data in the image display section, an image formed by transmitted light coming from the back of the display surface coming transmitted through the light transmitting portion A transmissive display characterized by being able to. 2. The transmissive display device according to claim 1, wherein the image data portion and the transmissive data portion on the image memory are exchanged for each image display screen or a plurality of screens. An image acquisition device for acquiring an image of an observation object to be observed, an image processor for performing processing to display an image acquired from the image acquisition device, and the display for displaying a result of the processing And the operation of the transmission display capable of observing the object to be observed existing through the display surface with the naked eye, the image acquisition device, the image processing device, and the transmission display. A control unit for controlling the display image of the object to be observed acquired by the processing of the image processor, and the observation image can be superposed on the fluoroscopic image with the naked eye. A transmissive display device according to claim 1 or 2, characterized in that a fluoroscopic image observation device.

Images