JPH08114753A - Image display device - Google Patents

Abstract

PURPOSE: To improve convenience of specimen observation by displaying that any position of a specimen is observed. CONSTITUTION: A PDA 33 is lighted by an LED 35, and its reflected image is image picked up by a video camera 40 through a retroreflection optical system and a branch mirror 23. An image processor 50 stores positions of respective photodiodes placing in an image as positional information from the image pickup result. An input image is made incident on the video camera 40 through the branch mirror 23, and the image processor stores the image data. When the specimen is observed, an observed image made incident from a microscope 10 is detected by the PDA 33, and the rough divided image data are obtained. The image processor 50 superimposes the rough divided image data on the image data of the beforehand stored specimen based on the positional information of the photodiodes to display them on a monitor 53.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image display device used for measuring the response of a cell membrane to a stimulus.

[0002]

2. Description of the Related Art In order to observe the response of a biological sample to a stimulus, a dye that fluoresces in response to an electric potential is usually incorporated into the biological sample and observed using a microscope equipped with a fluorescence device. are doing. When fluorescence is observed with a microscope, this biological sample changes its fluorescence intensity according to the potential of its cell membrane. Since the electric potential of the cell membrane appears as a response to the stimulus, by observing the electric potential of the cell membrane, the reaction state of the biological sample can be grasped.

Since this reaction occurs at an extremely high speed and the amount of change in fluorescence is extremely small, a detector operating at a high speed and having a wide dynamic range is required to observe this phenomenon. Therefore, a photodiode array (PDA) is used as a detector that satisfies this severe condition.

[0004]

This PDA is provided with a photodetecting surface in which photodiodes are arranged in a grid of, for example, 16 × 16, and is extremely high speed (0.5 m per frame).
sec) and has a wide dynamic range of 16 bits (65536 gradations). However, since the number of detection pixels is 16 × 16, which is coarse, it is often difficult to know which part of the sample is being observed in the image detected by the PDA.

The present invention has been made to solve such a problem, and an object thereof is to easily determine which part of the sample is being observed even when the sample is observed with a PDA or the like. An object of the present invention is to provide an image display device that can be grasped and improves the convenience of sample observation.

[0006]

Therefore, an image display apparatus according to the present invention is an optical information detecting means for detecting optical information obtained from a specific portion of a sample, and a relay optical system for guiding an input image of the sample to the optical information detecting means. System, a branching optical system for branching an input image guided to the relay optical system, and a retroreflection optical system for branching the reflected image of the optical information detecting means returning through the relay optical system from the relay optical system to enter the branching optical system. Equipped with. Further, the reflected image of the optical information detecting means or the input image of the sample guided by the branch optical system is picked up by the image pickup means, and the image processing means picks up the image at this time based on the image pickup result obtained by the image pickup means. The position information of the optical information detecting means in the image is obtained, and the input image of the sample imaged by the imaging means and the optical information detected by the optical information detecting means are superposed on the basis of the obtained position information. To display.

[0007]

When the input image of the sample and the optical information detected by the optical information detecting means are displayed in a superimposed manner, the following processing is performed by the image pickup means and the image processing means.

First, an input image incident through the branch optical system is previously imaged by an image pickup means, and optical information (image data, spectral characteristics, etc.) of the input image is secured. In addition, the position of the optical information detecting unit in the captured image is secured as position information based on the reflected image of the optical information detecting unit that is incident on the image capturing unit.

Then, the input image of the sample is formed on the image detecting means to obtain optical information. The image processing means superimposes the optical information detected by the optical information detection means on the input image previously picked up by the image pickup means based on the position information already secured.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A description will be given below with reference to the accompanying drawings.

<Embodiment 1> FIG. 1 shows the overall construction of an apparatus according to this embodiment, in which an image display device is attached to a TV camera port of a microscope apparatus 10.

The microscope device 10 is mainly used when measuring the electric potential of a cell membrane in a biological sample, and is provided with a fluorescence device 11 for performing fluorescence observation. Sample stage 12
A biological sample 13 to be observed is set on the top, and the biological sample 13 is loaded with a dye that emits fluorescence in accordance with the potential. Therefore, the fluorescence intensity changes according to the potential of the membrane. The change in the electric potential of the membrane appears as a response to the stimulus, and therefore, by observing the fluorescence, it can be observed that the biological sample 13 responds to the stimulus.

On the other hand, the image display device is branched from the relay optical system 20 for guiding the observation image of the microscope device 10, the detector 30 for detecting the change of the observation image guided by the relay optical system 20, and the relay optical system 20. A video camera 40 for taking an image,
The detector 30 and the image processing device 50 for processing the image obtained by the video camera 40 are included.

FIG. 2 shows a microscope apparatus 1 of the detector 30.
The photodetection surface 31 on which an observation image of 0 is formed is enlarged and shown. Photodiodes 32 are arranged on the photodetection surface 31 in a 16 × 16 (8 × 8 for convenience of illustration in FIG. 2) grid pattern to form a photodiode array (PDA) 3.
3 is formed, and by utilizing the characteristics of the photodiode, a wide detector having a high speed (1 frame: 0.5 msec) and a dynamic range of 16 bits (65536 gradations) is configured. The observation image described above is formed on the light detection surface 31, and the observation image is obtained from each photodiode 32 as image data roughly divided.

The relay optical system 20, as shown in FIG.
This is a lens system for forming an observation image on the light detection surface 31 by a relay lens (eyepiece) 21 and an imaging lens 22, and in this lens system, a branch mirror 23 composed of a half mirror, and the same. A reflection mirror 24 composed of a half mirror is arranged. Both mirrors 23, 2
4 is integrally supported by a support arm 25, and has a mechanism of sliding on the base 27 along the groove 27a by moving the optical path switching knob 26 in and out along the arrow direction. Therefore, the optical path switching knob 26
Both mirrors 23, 24 can be moved into or out of this optical path by operating.

When the branch mirror 23 is located in the optical path, the image of the sample transmitted through the reflection mirror 24 is split by the branch mirror 23, and is passed through the mirror 28 and the imaging lens 29 to the video camera 40. It is incident on the photocathode. Therefore,
The observation image of the microscope device 10 is displayed on the video camera 40.
It can be directly incident.

Furthermore, the branching mirror 23 and the reflecting mirror 24
Means that a so-called retroreflective optical system is configured before the sample observation. That is, when the LED 35 provided in the vicinity of the light detection surface 31 is turned on and the light detection surface 31 is illuminated, the image (reflected / scattered light) of the PDA 33 at this portion is formed by the imaging lens 22, the branch mirror 23, and the relay lens 21. To the reflection mirror 24. The image of PDA33 is
A part of the light is reflected by the reflection mirror 24 and goes backward in this optical path, and the relay lens 21, the branch mirror 23, and the mirror 28
And the video camera 40 via the imaging lens 29. Therefore, by going through this retroreflective optical system,
An image of the PDA 33 on which an observation image is formed can be made incident on the video camera 40.

In the optical system of the present embodiment, the field of view of the video camera 40 is a square whose diagonal is the diameter of the field of view of the microscope device 10, and the field of view of the PDA 33 is 2/3 of that.
The imaging lens is selected so that

On the other hand, the result detected by the PDA 33 of the detector 30 is converted into an image signal by the PDA control unit 51 and given to the image processing device 50. Further, the result captured by the video camera 40 is converted into an image signal by the camera control unit 52, and is similarly given to the image processing device 50.

The image processing apparatus 50 performs image processing based on the captured image of the PDA 33 given through the camera control unit 52, and specifies each position in the captured image for each photodiode constituting the photocathode. . As specific processing, the photocathode of the PDA 33 is imaged by the video camera 40, and its representative points (for example, since the photocathode is a square, for example, its three vertices, the center and the other two vertices) are displayed on the screen. Select using the cursor. By doing so, it is possible to grasp which position in the video camera image the photocathode of the PDA 33 occupies. And
This position information is stored in the memory of this device. Also,
Image information captured by the video camera 40 is stored in a frame memory in the device.

Further, in the image processing device 50, based on the specified position information of each photodiode 32, the P of the image of the video camera 40 in the frame memory is set.
The images obtained by the DA 33 are superposed and displayed on the monitor 53. This is a processing operation called so-called superimpose.

The processing operation will be outlined below. After obtaining the position information of the photocathode of the PDA 33 by the method described above, the light from the sample is picked up by the PDA 33 and superimposed on a black and white video camera image,
Display in pseudo color.

As described above, the image of the sample taken by the video camera is stored in the frame memory of about 512 × 512. On the other hand, the signal from the PDA 33 is temporarily 16 ×.
Based on the obtained position information stored in 16 memories, arbitrary arithmetic processing such as enlargement, parallel movement, and rotation is performed, and every other position is written in the position corresponding to the PDA photocathode of the frame memory. At this time, the image signal of the video camera at the address where the PDA signal is written is erased, and the data at the other addresses are left as they are. The contents of this frame memory are displayed on the screen.

By carrying out such processing, the image of the video camera and the image of the PDA are displayed on the screen.
Only the corresponding position is displayed in a superimposed manner. As the method of writing every other one, there are a method of writing in a stripe shape vertically or horizontally alternately on the screen and a method of writing in a staggered manner. In this embodiment, as a method of changing the address of the PDA memory to the address of the frame memory of the video camera, an operator for conversion is first obtained from the position information described above. If that operator is applied to the PDA position coordinates as it is, a "dropout" occurs, so first perform the inverse conversion by applying the inverse operator to the video camera position coordinates, and check if it corresponds to the PDA position. To do. After that, a method of writing the PDA signals in a zigzag pattern only every other position corresponding to the PDA is adopted (FIGS. 10A and 10B).

The operation procedure in the image processing apparatus 50 will be described below in order. First, the LED 35 located in front of the PDA 33 is turned on while the optical path switching knob 26 is pushed in (the mirrors 23 and 24 are positioned in the optical path), and the light detection surface 31 of the detector 30 is illuminated. To do. As a result, the image of the PDA 33 is displayed on the photocathode 41 of the video camera 40.
Tied up. This imaging result is given to the image processing apparatus 50 via the camera control unit 52, and the position of the photocathode of each photodiode 32 in the captured image is analyzed. Then, the result is stored in the memory as position information. Note that this imaging result is shown in FIG.
As shown in (a), it is displayed on the monitor 53. In the figure, an area indicated by reference numeral 53a is an image display unit, and
The area indicated by 3b is the menu display section.

After turning off the LED 35, the microscope apparatus 10 is turned on to turn on the illumination light source 14 or 11. As a result, the image of the biological sample 13 on the sample stage 12 is formed at the input image position by the microscope objective lens. As described above, the transmitted light flux passes through the relay lens 21 and the like and is coupled onto the photoelectric surface 41 of the video camera 40. The image pickup result is given to the image processing apparatus 50 via the camera control unit 52 and stored in the frame memory as image information (see FIG. 4B).

Next, pull the optical path switching knob 26 toward you,
The two mirrors 23 and 24 are moved out of the optical path, and the fluorescent device 11 of the microscope device 10 is turned on. By this operation, the fluorescence observation image of the biological sample 13 set on the sample stage 12 is formed on the light detection surface 31 of the detector 30.
At this time, the biological sample 13 is stimulated and the observation is continued. The fluorescence image as the observation result is sequentially transmitted from the PDA 33 on the light detection surface 31 to the PDA control unit 51. For reference, in FIG.
An image obtained by the PDA 33 is shown below.

Next, the image processing apparatus 50 writes this fluorescence image at each corresponding image position based on the position information of each photodiode 32 stored in the frame memory. In this way, the image obtained by the PDA 33 is superimposed on the image of the biological sample 13 that has been captured in advance and is displayed on the monitor 53 (see FIG. 4D). In this way, the biological sample 13 can be displayed by superimposing it on the monitor.
It can be easily recognized which part of the PDA image is in the PDA image.

When observing other parts of the biological sample 13, the sample stage 12 is moved, or
After the magnification of the objective lens of the microscope is reduced to widen the observation range, the above-mentioned operation may be performed again.

Among the mirrors shown in this embodiment, a reflecting mirror 24 as a half mirror constituting a retroreflective optical system.
May be like a bandpass filter, and in that case, it is desirable to efficiently reflect the light flux of the LED and transmit other light. When an ordinary non-transparent mirror is used as the reflection mirror 24, the branch mirror 2
In addition to the structure shown in FIG. 3, it is possible to insert and remove, and when taking an image of a biological sample, only the reflection mirror 24 may be removed from the optical path and used.

Further, as described above, since the PDA 33 has a high speed (one frame is 5 msec), it cannot be followed by a normal image display device (about one frame 1/30 sec), and as shown in FIG. In addition, the information of each pixel can be displayed by the image processing support computer 54.

<Embodiment 2> FIG. 5 shows another embodiment of the relay optical system shown in FIG. Note that the other configurations are the same as those shown in FIG. 1, and the same components are designated by the same reference numerals and the description thereof will be omitted. The relay optical system 20 ′ of this embodiment is a retroreflector 60 that forms a retroreflection optical system.

The principle of the optical system using the retro reflector 60 will be described below with reference to FIG. In the figure,
Components corresponding to those in FIG. 5 are designated by the same reference numerals.

An input image as an observation image of the microscope apparatus 10 is imaged on the photocathode of the PDA 33 by the lens 1 and the imaging lens 22, and by the lens 1 and the imaging lens 29, the photocathode 41 of the video camera 40. Image on top. By reflecting the mirror twice on the photocathode 41 of the video camera 40 (branch mirror 23, mirror 28), the PDA 33
The same image (not a mirror image) as above is formed. The magnification at this time is determined by the ratio of the focal lengths of the imaging lens 29 and the imaging lens 22. In addition, branch mirror (half mirror)
23, the mirror 28 not only prevents the mirror image,
It is also used to adjust the image formation position on the photocathode 41 of the video camera 40.

The image on the photocathode of the PDA 33 is the image forming lens 2.
An image is formed on the mirror surface 62 in the retro reflector 60 through the branch mirror 23 by the lens 2 and the lens 61, and further, through the branch mirror 23 by the lens 61 and the imaging lens 29, on the photoelectric surface of the video camera 40. Form an image. The retro reflector 60 causes the pupil of the imaging lens 22 to move to the imaging lens 2
The image on the photocathode of the PDA 33 is
It is tied to the photocathode 41 of the video camera 40 at a magnification according to the ratio of the focal lengths of the imaging lens 29 and the imaging lens 22.

FIG. 7 shows a plan view of FIG. In FIG. 7, image points 1 and 2 are the photocathode of the PDA 33,
It corresponds to the photocathode of the video camera 40. The optical path from the object point a through the lens 1, the branching mirror 23, the mirror 28, and the imaging lens 29 to the image point 2 is the object point a → lens 1 → branching mirror 23 → imaging lens 22 → image point 1 → imaging. The optical path is lens 22 → branch mirror 23 → lens 61 → image point 3 → lens 61 → branch mirror 23 → mirror 28 → imaging lens 29 → image point 2 of which the optical path from branch mirror 23 to image point 2 is The same applies when the branch mirror 23 is tilted. In FIG. 7, the lenses 1 and 61 and the imaging lenses 22 and 29 are shown.
Although the pupil of is on the branch mirror surface 23, it is not necessarily limited to this example, and the retro-reflector 60 including the lens 61 and the mirror 62 is
The pupils of the imaging lenses 22 and 29 are relayed to the same size, and the lens 1
It is only necessary that the pupil of (1) coincides with the pupils of the imaging lenses 22 and 29.

By adopting such a structure, the image of the photocathode of the PDA 33 and the image of the biological sample 13 can be observed by the video camera 40 without being displaced from each other.

Another configuration example is shown in FIG. In this case, a half mirror 71 is arranged instead of the mirror 28, and the light flux transmitted through this half mirror 71 is reflected by the mirror 72 and guided to the imaging lens 73. With this configuration, in addition to the detectors 81 and 82 corresponding to the positions of the detector and the video camera in FIG. 6,
A third detector 83 can be provided. Even in this case, the observation can be performed as in the above-mentioned embodiment.

FIG. 9 shows still another configuration example. The optical system of FIG. 9 is arranged by inverting the direction of the branch mirror 23 by 90 °. In this case, the optical path of the input image → PDA 33 does not change, but the optical path of the input image reaching the video camera is as follows: input image → lens 1 → branching mirror 23 → retro reflector 6
0 → branch mirror 23 → mirror 28 → imaging lens 29 → video camera 40. When observing the image of the photocathode of the PDA 33 with the video camera 40, the optical path at this time is PDA 33 → imaging lens 22 → branching mirror 23 → mirror 28 → imaging lens 29 → video camera 40.

<Third Embodiment> Another embodiment of a detector for detecting a change in an observed image will be described with reference to FIG. The same components as those of the device shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.

The detector 130 of this embodiment is arranged at a position where an input image is formed, and has a pinhole plate 131 having a pinhole 131a, a spherical diffraction grating 132 for separating a light flux passing through the pinhole 131a, and The CCD line sensor 133 constitutes a spectroscope.

The sequence of processing operations is as follows:
This is almost the same as the case of using 33. First, the video camera 40 captures an image of the pinhole plate 131 serving as the input unit of the spectroscope. Then, using the mouse, the outline of the pinhole or the pinhole position is designated by the "+" mark on the screen of the monitor 153. This outline is stored in the memory as an overlay image and is always displayed on the screen.

Next, the optical path is switched, and the input image of the biological sample 13 is picked up by the video camera 40. This picked-up image is
It is sent to the image processing apparatus 150 via the camera control unit 152. On the other hand, the light flux that has transmitted through the half mirror inside the retroreflective optical system and further transmitted through the pinhole 131a is dispersed by the diffraction grating 132 and forms spots on the CCD line sensor 133 at positions corresponding to the respective wavelengths. This is read by the CCD line sensor 133, and the computer 1 for displaying / analyzing the spectral spectrum
The spectroscopic characteristics of the biological sample 13 are known by performing an analysis at 54 and displaying it as a graph on the monitor 153.

On the other hand, in the image processing apparatus 150, the input from the video camera 40 is AD-converted and stored in the memory. The computer overlays the overlay, thins out pixels, and transfers them to the computer. Also, the input from the spectrometer is
If necessary, log the intensity axis and send it to the computer. In the computer 154, the image of the video camera 40 and the graph of the spectroscope output are simultaneously displayed on the monitor 153, and the outline of the pinhole 131a is drawn on the image captured by the video camera 40 in the overlay. This video image and graph are constantly rewritten, and biological sample 1
By moving 3, it is easy to know which position of the biological sample 13 is being observed.

An example of the display on the monitor 153 is shown in FIG. Reference number 1 in the figure
Reference numeral 55 is an imaged image of a video camera displayed on a monitor, and a spectral detection point 156 detected by the detector 130 is displayed on the biological sample 13, whereby the position of the biological sample is determined. It is easy to see if you are observing. Further, reference numeral 157 indicates the detector 1.
The output of 30 is displayed as a graph, and biological sample 1
The spectral characteristics of this part in No. 3 are displayed. Reference numeral 158 indicates a menu screen.

Further, the detector 230 can be constructed by using the FT type spectroscope 132 shown in FIG. This detector uses an optical connector 231a as an input unit that allows a part of the input image of the biological sample 13 to pass therethrough, and the input image incident on this optical connector 231a is an FT spectroscope 232 via an optical fiber 231. Have been led to.

The FT spectroscope 232 has an optical fiber 231.
A deflector 233 for deflecting light incident through the components into components in directions orthogonal to each other, a Savart plate 234 that does not give a phase difference between the respective deflection components, and provides only so-called lateral displacement,
An analyzer 235 and an analyzer 235 for aligning and outputting the deflection directions.
Imaging lens 23 for focusing two light beams output via
6. The array detector 237 is arranged in a region where the two light fluxes overlap each other at a distance from the image plane of the two light fluxes and detects the intensity of the received light. For other configurations,
Since the configuration is the same as that of FIG. 11 described above, description thereof will be omitted.

[0048]

As described above, according to the image device of the present invention, the optical information relating to the input image detected by the image detecting means is superposed on the input image of the biological sample or the like and displayed. Therefore, it is possible to easily recognize from which part of the input image this optical information is obtained.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a configuration of a microscope observation device including an image display device according to an embodiment.

FIG. 2 is a partially enlarged view showing an enlarged photodetection surface of the PDA.

FIG. 3 is a perspective view schematically showing a configuration of a relay optical system in the device configuration of FIG.

4A is an explanatory view showing a state where a photocathode of a PDA is displayed on a monitor, FIG. 4B is an explanatory diagram showing a state where a biological sample is displayed on a monitor, and FIG. 4C is obtained by a PDA. Explanatory drawing showing an image, (d) shows P on a captured image of a biological sample.
It is explanatory drawing which shows the state which overlapped and displayed the detection image of DA.

FIG. 5 is a perspective view showing another configuration example of a relay optical system.

FIG. 6 is an explanatory diagram showing the principle of a relay optical system.

FIG. 7 is a plan view of FIG. 6;

FIG. 8 is a perspective view showing another configuration example of the relay optical system.

FIG. 9 is a perspective view showing another configuration example of the relay optical system.

10A is a diagram showing a state in which images of a PDA and a biological sample are displayed on a monitor screen, and FIG. 10B is a diagram showing each pixel. Note that in (b), the shaded area indicates PDA signal pixels, and the white area indicates video camera signal pixels.

FIG. 11 is a diagram showing another configuration example of a microscope observing apparatus including an image display device.

12 is a diagram showing a display example of a monitor in the image display device of FIG.

FIG. 13 is a diagram showing another configuration example of a microscope observing apparatus including an image display device.

[Explanation of symbols]

10 ... Microscope device, 20 ... Relay optical system, 23 ... Branching mirror, 24 ... Reflecting mirror, 25 ... Support arm, 26 ... Optical path switching knob, 30 ... Detector (image detecting means), 31 ... Photodetecting surface, 32 ... Photodiode, 33 ... Photodiode array, 40 ... Video camera, 50 ... Image processing device.

Claims (4)

[Claims] 1. An optical information detecting means for detecting optical information obtained from a specific portion of a sample, a relay optical system for guiding an input image of the sample to the optical information detecting means, and an input for guiding to the relay optical system. A branching optical system for branching an image, a retroreflecting optical system for branching the reflected image of the optical information detecting means returning through the relay optical system from the relay optical system to enter the branching optical system, and the branching Based on an image pickup means for picking up a reflection image of the optical information detecting means or an input image of the sample guided by an optical system, and an image pickup result obtained by the image pickup means, the optical information in the image pickup image at this time The position information of the detection means is obtained, and based on the obtained position information, the input image of the sample imaged by the imaging means and the optical information detected by the optical information detection means are superposed and displayed. Image display device characterized by comprising an image processing means for. 2. The light information detecting means comprises a light detecting surface composed of a single photocathode or a plurality of arranged photocathodes, and the light information detected by the light information detecting means is The image display device according to claim 1, wherein the image data is rough image data regarding the input image formed on the light detection surface. 3. The optical information detecting means includes an input section for passing a part of the input image, and a spectroscopic means for splitting the light passing through the input section into light of each wavelength component, The image display device according to claim 1, wherein the optical information is a spectral characteristic of light transmitted through the input unit. 4. The image according to any one of claims 1, 2 and 3, further comprising a transition means for displacing the retroreflective optical system into and out of the relay optical system. Display device.