JP2010223676A - Observation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract easily a dark part when generating a synthetic image, concerning an observation device capable of observing a transparent observation object such as a biological cell in a wide visual field, by adopting for illumination, a surface light source formed by arranging a bright part and the dark part repeatedly. <P>SOLUTION: This device includes: an illumination means for illuminating by transmission, an observation object stored in a transparent container by a surface light source formed by providing a bright part and a dark part alternately; an imaging optical system for imaging transmission light from the observation object illuminated by the illumination means; an imaging means for acquiring an image of the observation object formed by the imaging optical system; a control means for allowing the imaging means to acquire a plurality of images, while shifting a phase of brightness/darkness of the surface light source; and a correction means for extracting each position of three or more corresponding points to a linear pattern from an image acquiring a linear characteristic pattern constituted of the bright part and the dark part of the illumination means, and correcting distortion of the acquired image so that each corresponding point is positioned on one straight line. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an observation apparatus capable of observing a transparent object such as a living cell in a wide field of view by employing a surface light source in which a bright part and a dark part are repeatedly arranged for illumination.

  A stripe illumination type observation device (see Patent Document 1) obliquely illuminates a portion of an observation object facing a dark portion of a stripe light source with two bright portions adjacent to the dark portion. Can be obtained. Therefore, if a plurality of images are acquired while shifting the light-dark phase of the stripe light source and only the dark image is synthesized, for example, a dark field image overlooking the cell group scattered in the culture vessel is generated. You can also

JP 2007-178426 A

  However, in the stripe illumination type observation apparatus, a linear stripe pattern may be deformed into an arc shape or the like due to the meniscus effect of the culture medium present in the culture vessel, and in such a case, There is a problem that it is difficult to extract and synthesize only a dark part image from a plurality of images picked up while shifting the phase.

  The present invention has been made to solve such a conventional problem, and an object of the present invention is to provide an observation apparatus that can easily extract a dark part when creating a composite image.

  An observation apparatus according to a first aspect of the invention includes an illuminating unit that transmits and illuminates an object to be observed contained in a transparent container by a surface light source in which bright portions and dark portions are alternately provided, and the object illuminated by the illuminating unit. An imaging optical system that forms an image of transmitted light from an observation object, an imaging unit that acquires an image of the observation object formed by the imaging optical system, and the imaging while shifting the phase of light and dark of the surface light source 3 or more corresponding points with respect to the linear pattern from an image obtained by acquiring a linear characteristic pattern composed of a bright part and a dark part of the illumination means, and a control means for causing the means to acquire a plurality of images And correcting means for correcting the distortion of the acquired image so that the corresponding points are positioned on a straight line.

  The observation device according to a second invention is characterized in that, in the observation device according to the first invention, the correction means extracts positions of three or more corresponding points for each of the plurality of dark portions.

  An observation apparatus according to a third aspect of the present invention is the observation apparatus according to the first or second aspect, wherein the transparent container is a well plate having a plurality of wells.

  The observation device according to a fourth aspect is the observation device according to the first aspect, wherein the surface light source is composed of a mask member having a stripe pattern or a checker pattern and an illumination light source that uniformly illuminates the mask member. Features.

  The observation apparatus according to a fifth aspect is the observation apparatus according to the first aspect, wherein the correction unit corrects distortion of the image by superimposing the plurality of images acquired by the imaging unit. .

  ADVANTAGE OF THE INVENTION According to this invention, the observation apparatus which can perform a dark part extraction easily at the time of preparation of a synthesized image can be provided.

It is explanatory drawing which shows 1st Embodiment of the observation apparatus of this invention. It is a block diagram which shows the controller of the observation apparatus of FIG. It is explanatory drawing explaining the background luminance distribution of a dark field image. It is explanatory drawing which shows the stripe pattern displayed during a phase shift. It is explanatory drawing which shows the image acquired during the phase shift. It is explanatory drawing which shows the dark field image formed by synthesize | combining each effective area | region of FIG. It is explanatory drawing which shows operation | movement of CPU regarding observation. It is explanatory drawing which shows the image of the stripe pattern acquired by the image pick-up element. It is explanatory drawing for demonstrating the meniscus effect by a culture solution. It is explanatory drawing which shows the image by which distortion correction was carried out. It is explanatory drawing which shows a corresponding point. It is explanatory drawing which shows the waveform of the brightness | luminance of a X direction. It is explanatory drawing which shows the change of the correlation value of a X direction. It is explanatory drawing which shows the method of distortion correction. It is explanatory drawing which shows the waveform of the brightness | luminance of the X direction in 2nd Embodiment. It is explanatory drawing which shows the change of the differential value of a X direction. It is explanatory drawing which shows a checker flag pattern.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
FIG. 1 shows a first embodiment of the observation apparatus of the present invention. This observation apparatus includes a transmissive liquid crystal panel 11, a stage (specimen stage) 13, an imaging optical system 15, an image sensor 17, a panel controller 19, a controller 21, and a display 23.

  A transparent culture vessel (96 well plate in the figure) 25 is placed on the stage 13. In the well 25a of the culture vessel 25, for example, an object to be observed such as unstained cells is accommodated.

  The transmissive liquid crystal panel 11 has a backlight and illuminates substantially the entire area of the culture vessel 25 uniformly. The portion of the stage 13 on which the culture vessel 25 is placed is made of a transparent member such as glass or is hollow so that the illumination light is not obstructed.

  The visual field of the imaging optical system 15 is sufficiently large, and light emitted from substantially the entire area of the culture vessel 25 is captured by the imaging optical system 15 and forms an image on the imaging surface of the imaging element 17. The imaging element 17 acquires an image (luminance distribution) on the imaging surface in response to an instruction from the controller 21. The image is taken into the controller 21 and displayed on the display 23 after image processing. The controller 21 can store an image before image processing and an image after image processing as necessary. In addition, an image before image processing can be displayed on the display 23 as necessary.

  In response to an instruction from the controller 21, the panel controller 19 displays a stripe pattern in which bright portions and dark portions are repeatedly arranged on the transmissive liquid crystal panel 11. A stripe light / dark pattern is formed by irradiating the stripe pattern formed on the transmissive liquid crystal panel 11 with a surface light source (backlight) that uniformly illuminates.

  FIG. 2 is a block diagram showing the configuration of the controller 21. The controller 21 includes an input unit 27, a stage control unit 29, a panel control unit 31, an illumination control unit 33, an image sensor control unit 35, and a CPU 37.

  The input unit 27 inputs an instruction from the user. Information that can be input by the user via the input unit 27 includes various instructions such as start of observation.

  The stage control unit 29 controls the movement of the stage 13 and sets the position of the stage 13 in the optical axis direction (Z direction position) and the position in the direction perpendicular to the optical axis direction (XY direction position).

  The panel control unit 31 moves the transmissive liquid crystal panel 11 to set a position in the optical axis direction (Z direction position) and a position in the direction perpendicular to the optical axis direction (XY direction position). .

  The illumination control unit 33 controls the panel controller 21 to control the phase and the like of the stripe pattern displayed on the transmissive liquid crystal panel 11.

  The image sensor control unit 35 performs drive control of the image sensor 17.

  The CPU 37 controls the stage control unit 29, the panel control unit 31, the illumination control unit 33, and the image sensor control unit 35.

  FIG. 3 is a diagram for explaining the principle of the observation apparatus described above. FIG. 3 shows a state where the focusing plane P1 of the imaging lens 4 of the imaging optical system 15 is in the vicinity of the bottom surface of the well 25a of the culture vessel 25 and the cells 3a attached to the bottom surface are observed. In FIG. 3, reference numeral 1d indicates one dark part of the stripe pattern 1, and reference numeral 1b indicates bright parts located on both sides of the dark part 1d.

  In the well 25a of the culture vessel 25, focusing on the partial region 3d facing the dark portion 1d, the partial region 3d is illuminated from the oblique direction by the light Lb emitted from the two bright portions 1b, but light is emitted from the dark portion 1d. Since it does not inject, the partial area 3d is not illuminated from the front. That is, the partial area 3d is obliquely illuminated by the light Lb. Therefore, a dark field image 5d of the partial region 3d is formed in a region conjugate with the partial region 3d in the image sensor 17. In this dark field image 5d, an image of the cell 3a existing in the partial region 3d appears. This image (cell image) is not an image by direct light transmitted through the cell 3a but an image by scattered light scattered at the edge of the cell 3a.

  In the well 25a of the culture vessel 25, when attention is paid to the partial region 3b that faces the bright portion 1b, the partial region 3b is illuminated from the front by the bright portion 1b. Therefore, a bright field image 5b of the partial region 3b is formed in a region conjugate with the partial region 3b in the image sensor 17. In the bright field image 5b, an image of the cell 3a existing in the partial region 3b does not appear.

  Therefore, a stripe image in which the dark field image 5d including the cell image and the bright field image 5b not including the cell image are repeatedly arranged is formed on the image sensor 17. Therefore, if the imaging element 17 repeatedly performs imaging while shifting the phase of the stripe pattern displayed on the transmissive liquid crystal panel 11 (phase shift), and the dark field images 5d of a plurality of images obtained thereby are connected, the culture A composite image can be obtained in which the cells scattered throughout the container 25 can be viewed.

  For example, as shown in FIGS. 4A, 4B, and 4C, an image is picked up by the image pickup device 17 while shifting the phase of the stripe pattern by the shift pitch Δ, and the image shown in FIG. If the effective areas of the three images shown in A), (B), and (C) (the portion of width Ba ′ in FIG. 4) are connected, as shown in FIG. A composite image with a panoramic view of the cell group can be obtained.

  FIG. 7 is a flowchart showing the operation of the CPU 37 of the controller 21 relating to the observation of the object to be observed. This flow starts when the user inputs an observation start to the input unit 27.

  Step S1: The CPU 37 displays a linear stripe pattern as shown in FIG.

  Step S2: The CPU 37 captures an image of the observation object in the culture vessel 25 and acquires an image.

  FIG. 8 shows an image G <b> 1 acquired by the image sensor 17. This image G1 is an image of one well 25a when a 96-well plate is used as the transparent culture vessel 25. The straight stripe pattern displayed on the transmissive liquid crystal panel 11 is deformed into an arc shape due to the meniscus effect of the culture solution accommodated in the well 25a. In this way, when a linear stripe pattern is deformed into an arc shape or the like and imaged, only the dark part image is extracted from a plurality of images captured while shifting the phase as shown in FIG. It becomes difficult to do.

  That is, as shown in FIG. 9, when the culture solution 41 is stored in the well 25a of the 96-well plate, the outer peripheral portion of the surface of the culture solution 41 rises due to the surface tension. Then, due to the meniscus effect caused by the rise of the surface of the culture solution 41, the linear stripe pattern imaged on the image sensor 17 via the imaging optical system 15 is deformed into an arc shape. In the 96-well plate, the diameter of the well 25a is 6 mm, and the shape of the surface of the culture solution 41 is close to an arc-shaped lens shape. Therefore, the magnification of the object to be observed is maintained at a substantially uniform magnification.

  Step S3: The CPU 37 corrects the distortion of the acquired image G1. By this distortion correction, the distortion of the image G1 is corrected, and an image G2 having a linear stripe pattern as shown in FIG. 10 can be obtained. Details of the distortion correction will be described later.

  Step S4: The CPU 37 repeats a series of processes including image acquisition and phase shift for the well 25a, and acquires m (for example, three) images set in advance.

  Step S5: The CPU 37 creates one composite image by arranging and connecting the effective areas of the acquired m images in the order of acquisition.

  Step S6: The CPU 37 displays the composite image created in step S5 on the display 15, and ends the flow.

  Hereinafter, the details of the distortion correction in step S3 will be described in detail.

  In this distortion correction, first, as shown in FIG. 11, the position of the corresponding point T corresponding to the straight stripe pattern displayed on the transmissive liquid crystal panel 11 is extracted. In the image G1 shown in FIG. 11, the linear stripe pattern displayed on the transmissive liquid crystal panel 11 is deformed into an arc shape due to the meniscus effect of the culture solution accommodated in the well 25a. The longitudinal direction of the stripe pattern is taken as the Y axis, and the direction perpendicular to the Y axis is taken as the X axis. The coordinates Y1, Y2,..., Yn of the corresponding point T in the Y-axis direction are set in advance, and the position of the Y coordinate of the corresponding point T is known. Therefore, the CPU 37 obtains the position of each corresponding point T by measuring the X coordinate position of each corresponding point T of the specific stripe image in the image G1.

  The measurement of the X coordinate position of each corresponding point T is performed using a luminance waveform on each Y coordinate of the image G1, as shown in FIG. FIG. 12 shows a luminance waveform on a line segment passing through the position Y2 in FIG. 11 and parallel to the X axis, for example. As for this waveform, it is desirable to create a waveform obtained by integrating several pixels before and after each Y coordinate in the Y direction in order to improve the S / N ratio. For this integrated waveform, a symmetry correlation calculation window W1 having about twice the line width of the stripe pattern is prepared. Then, as shown in FIG. 13, a position where the correlation value in the window W <b> 1 reaches a peak is determined under the condition that it is in the dark part of the stripe pattern. This peak position corresponds to the X coordinate of the center position of the dark part of the stripe pattern shown in FIG. Thereby, the deviation amount of the point group that should be a straight line, that is, the deviation amount of the corresponding point T can be measured.

  Next, self-calibration is performed using this point group. In this case, since the distortion is point-symmetric, for example, the Tardiff method (J Tardif, P Sturm, and S Roy.Self-calibration of a general radially symmetric distortion model.In Proc. Of the .9th European Conference on Computer Vision, Vol. 4, pp. 186 2006.) and the like can be used to estimate the position of the point cloud when there is no distortion and to correct the distortion. Here is a brief description.

  Taliff's method is originally intended to correct center-symmetric distortion such as a fisheye lens correctly, and this distortion is bent with respect to a certain center point when a stripe pattern that is a straight line in real space is captured in a two-dimensional image. It means that it is displayed. The distortion caused by the fisheye lens and the distortion caused by the surface tension (meniscus effect), which is a problem in the present invention, will be considered.

As shown in FIG. 14, it is assumed that a point P ′ that should originally be imaged at the coordinates (x ′, y ′) is imaged at the coordinates (x, y) due to the effect of distortion. The coordinates C of the strain center and (c _x, c _y),

And R ′ is a function f (r), and r ′ = r / f (r) (1)
It can be expressed as

Three points P ₁ ′, P ₂ ′, and P ₃ ′ arranged on a straight line in real space become image coordinates (x ₁ , y ₁ ), (x ₂ , y ₂ ), and (x ₃ , y ₃ ). Assume that each image is captured.

  According to (1), these three points are constraint conditions.

Meet. A plurality of sets of three points arranged on a straight line are obtained, and f (r), c _x , and _cy are obtained by simultaneously solving Equation (2).

For coordinates (x ′, y ′) after distortion correction, f (r), c _x , and _cy are used.

It can be expressed as.

In the above-described embodiment, the image G1 is acquired by the imaging element 17 in a state where the dark part of the transmissive liquid crystal panel 11 is linear, and three or more corresponding points T corresponding to the linear dark part are acquired from the acquired image G1. Since the distortion of the image G1 acquired so that the corresponding point T is positioned on a straight line is corrected, the dark part can be easily extracted when the composite image is created.
(Second Embodiment)
In the first embodiment described above, the X coordinate position of the corresponding point T is set to the center position of the dark portion of the stripe pattern. However, in this second embodiment, the X coordinate position of the corresponding point T is the dark portion of the stripe pattern. Edge position. In this embodiment, the same elements as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In this embodiment, the measurement of the X coordinate position of each corresponding point T is performed using a luminance waveform on each Y coordinate of the acquired image G1, as shown in FIG. For this waveform, an edge calculation window W2 having about twice the line width of the stripe pattern is prepared. Then, as shown in FIG. 16, the position where the differential value of the luminance sum in the window W2 (corresponding to the value obtained by subtracting the luminance sum of the right half from the luminance sum of the left half of the window W2) becomes a peak. decide. This peak position corresponds to the X coordinate of the edge position of the dark part of the stripe pattern shown in FIG. Thereby, the deviation amount of the point group that should be a straight line, that is, the deviation amount of the corresponding point T can be measured.

  In this embodiment, the edge position of the dark part of the stripe pattern can be obtained easily and reliably.

Note that the stripe pattern is also inverted in the operation of the stripe illumination, so the differential peak is not used, and the intersection of the differential waveform at the normal image and the differential waveform at the inversion (in this case, not the edge of the dark part but the dark part) (The center position of the line width) may be used.
(Supplementary items of the embodiment)
As mentioned above, although this invention was demonstrated by embodiment mentioned above, the technical scope of this invention is not limited to embodiment mentioned above, For example, the following forms may be sufficient.

  (1) In the above-described embodiment, an example in which the present invention is applied to an observation apparatus having a light and dark shape in a stripe pattern has been described. However, for example, the present invention is also applied to an observation apparatus having a light and dark shape in a checker pattern as shown in FIG. can do. And the position of a corresponding point can be calculated | required with higher precision by utilizing the lattice point of the checker pattern located in a straight line as a corresponding point.

  (2) In the above-described embodiment, an example in which one image of each stripe pattern is acquired has been described. However, for example, an S / N ratio is improved by acquiring and superimposing a plurality of images of each stripe pattern. This makes it possible to determine the position of the corresponding point more accurately.

  (3) In the Taldiff method in the above-described embodiment, the accuracy of distortion correction can be improved as the number of corresponding points is increased. Therefore, the number of corresponding points may be increased by shifting the stripe pattern by a movement amount smaller than the spatial period (line width) of the stripe pattern.

  (4) In the above-described embodiment, the transmissive liquid crystal panel 11 with the backlight is used to generate the surface light source. However, other devices may be used as long as the same surface light source can be generated. . For example, a large number of small light sources such as LEDs may be prepared, and these may be densely arranged in an array to form a surface light source, and a mask member having a stripe-shaped opening may be disposed on the surface light source.

  (5) In the above-described embodiment, the example in which the present invention is applied to the observation of the culture container 25 composed of a 96-well plate has been described. For example, at the corner of the flask, the linear stripe pattern is relatively largely deformed by the meniscus effect, but can be processed in the same manner as in the above-described embodiment.

DESCRIPTION OF SYMBOLS 11 ... Transmission-type liquid crystal panel, 13 ... Stage, 15 ... Imaging optical system, 17 ... Imaging element, 21 ... Controller, 25 ... Culture container, 25a ... Well.

Claims (5)

Illumination means for transmitting and illuminating an object to be observed contained in a transparent container by a surface light source in which bright portions and dark portions are alternately provided,
An imaging optical system that forms an image of transmitted light from the object illuminated by the illumination means;
Imaging means for acquiring an image of the object to be observed formed by the imaging optical system;
Control means for causing the imaging means to acquire a plurality of images while shifting the light-dark phase of the surface light source;
The positions of three or more corresponding points with respect to the linear pattern are extracted from an image obtained by acquiring a linear characteristic pattern composed of a bright part and a dark part of the illumination means, and the corresponding points are in a straight line. Correction means for correcting distortion of the acquired image so as to be positioned;
An observation apparatus comprising: The observation apparatus according to claim 1,
The said correction | amendment means extracts the position of three or more corresponding points with respect to each of the said some dark part, The observation apparatus characterized by the above-mentioned. The observation device according to claim 1 or 2,
The observation apparatus, wherein the transparent container is a well plate having a plurality of wells. The observation apparatus according to claim 1,
The observation apparatus, wherein the surface light source includes a mask member having a stripe pattern or a checker pattern and an illumination light source that uniformly illuminates the mask member. The observation apparatus according to claim 1,
The observation unit, wherein the correction unit corrects distortion of the image by superimposing a plurality of images acquired by the imaging unit.

Images