JP6317003B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method for correcting image distortion when imaging contents such as a fluid injected into a recess.

  In a medical or biological science experiment, for example, a liquid or gel-like fluid flows into each well of a plate-like instrument (for example, called a microplate or a microtiter plate) provided with a large number of depressions called wells. A body (for example, a culture solution, a medium, etc.) is injected, and the cells cultured here are observed and measured as a sample. For example, in the techniques described in Patent Documents 1 and 2, a chemical solution or a fluorescent dye for facilitating observation is injected into a sample, and the number of cells that react with the chemical solution in the sample is measured.

  In recent years, a sample is imaged with a CCD camera or the like without using a chemical solution for observation, and converted into data, and various image analysis techniques are applied to the image data for observation and analysis. It is becoming. As an example of such a technique, for example, Patent Document 3 discloses a technique of imaging the contents of a well by scanning and moving an optical system provided above the well relative to the well.

JP 05-184579 A JP 2000-316596 A US Pat. No. 7,262,858

  When imaging is performed from above the well as in the technique described in Patent Document 3 described above, the image may be distorted depending on the surface state (surface irregularities, meniscus, etc.) of the fluid injected into the well. This distortion causes a problem that the shape or size of the imaged cell or the like is recognized as being different from the original one. This problem is particularly remarkable when the viscosity of the fluid is high or the cross-sectional size of the well is small. The state of the fluid surface varies from sample to sample depending on the amount of fluid to be injected, wettability to the well wall surface, the state at the time of injection, and the like. For this reason, the mode of distortion appearing in the image also varies from sample to sample.

  However, a technique capable of correcting image distortions that differ from sample to sample in this way has not been established so far.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique capable of easily and effectively reducing distortion of an image of a sample formed by injecting a fluid into a recess.

  In the following, “fluid” is a generic term for liquids, gel-like or semi-fluid solids, and those that are injected into a well in a fluid state such as soft agar and then solidified.

  In one aspect of the imaging apparatus according to the present invention, in order to achieve the above object, a sample holding plate having a recess capable of holding a liquid provided on the upper surface thereof is arranged substantially horizontally, and a bottom surface of the recess is set to a predetermined position. Holding means for holding in contact with a transparent flat plate-like or sheet-like pattern forming body on which a strain detection pattern, which is a regular pattern having a transmittance for light of a wavelength lower than that of the bottom surface, is formed, and the depression Illuminating means for irradiating light to the recess through the pattern forming body from below, and receiving light transmitted through the recess above the recess to image the contents of the recess And an image correction unit that corrects distortion of the processing target image based on an image of the distortion detection pattern imaged by the imaging unit.

  In one aspect of the imaging method according to the present invention, in order to achieve the above object, a sample holding plate provided with a recess capable of holding a liquid on the upper surface is disposed substantially horizontally, and the bottom surface of the recess is The sample holding plate is held in contact with a transparent flat plate or sheet-like pattern forming body on which a strain detection pattern having a regular pattern whose transmittance for light of a predetermined wavelength is lower than that of the bottom surface. The recess is irradiated with light from below the pattern forming body, the light transmitted above the recess is received, and the contents of the recess are imaged to obtain a processing target image. An imaging step, a detection step of detecting distortion of an image of the distortion detection pattern obtained by imaging the depression, and an image correction step of correcting distortion of the processing target image based on a detection result in the detection step; It has.

  In the invention thus configured, the contents of the recess illuminated from below via the distortion detection pattern are imaged above. Therefore, both the contents formed by injecting the fluid into the recess and the distortion detection pattern are imaged through the surface of the same fluid. Therefore, the distortion of the image due to the surface state of the fluid appears in the same manner with respect to the contents of the recess and the distortion detection pattern. If the distortion detection pattern is known, it is possible to easily grasp the distortion mode appearing in the image from the result of imaging the image, and based on this, the processing target image obtained by imaging the contents of the depression is corrected. Therefore, it is possible to remove distortion from the processing target image relatively easily and stably regardless of variations in the surface state of the contents.

  For example, it is possible to detect the distortion of the distortion detection pattern from the image of the distortion detection pattern and perform correction necessary for canceling the distortion on the processing target image. Since the same distortion occurs between the distortion detection pattern and the contents of the depression, applying the correction for canceling the distortion in the distortion detection pattern image to the image to be processed effectively Can be corrected automatically.

  More specifically, for example, an image obtained by adjusting the positional relationship of the image elements in the processing target image based on the positional relationship of the distortion detection pattern components grasped from the distortion detection pattern image is corrected. Image. It is considered that the distortion of the image due to the irregularities on the surface of the fluid appears mainly as a change in the position of the elements constituting the image. Therefore, it is possible to grasp the degree of positional fluctuation due to distortion depending on the positional relationship in the captured image of each component of the distortion detection pattern whose positional relationship is known in advance. Since each image element in the processing target image is considered to be displaced in the same manner, by adjusting the positional relationship, a corrected image in which the positional fluctuation due to distortion is canceled is obtained.

  In these inventions, for example, a wavelength component in the visible region of the irradiated light is received to capture an image to be processed, while a wavelength component outside the visible region is received to capture a distortion detection pattern image. You may do it. In this way, the distortion detection pattern may be transparent to visible light, and as a result, it is possible to acquire an image without the distortion detection pattern reflected as a processing target image. Become. Therefore, it is possible to prevent the distortion detection pattern from being hindered in observation and analysis based on the image.

  In this case, for example, the first light including the wavelength component in the visible region is irradiated toward the recess to capture the processing target image, while the second light including only the wavelength component outside the visible region is input to the recess. The image of the distortion detection pattern may be picked up by irradiation. By individually performing the imaging of the contents of the depression and the imaging of the distortion detection pattern, the imaging conditions for each imaging can be individually set.

  Or, for example, the light containing both the wavelength component in the visible region and the wavelength component outside the visible region is irradiated toward the concave portion, and the light transmitted through the concave portion is in the visible region and the wavelength component outside the visible region. The wavelength component in the visible region is received to obtain the processing target image, while the wavelength component outside the visible region is received to obtain the distortion detection pattern image. In this way, the processing target image and the distortion detection pattern image can be acquired simultaneously by one imaging, and the time required for imaging can be shortened.

  In the present invention, for example, the sample holding plate may be held by a transparent stage on which a distortion detection pattern is formed, and the transparent stage may function as a pattern forming body. Thus, by combining the function of holding the sample holding plate and the function of arranging the strain detection pattern below the recess in the transparent stage, the apparatus can be reduced in size and cost. In addition, since the shape and size of the distortion detection pattern are unique to the imaging apparatus, it is possible to efficiently grasp the distortion of the image based on the distortion detection pattern image.

  In these inventions, for example, a grid pattern with a constant interval can be used as the distortion detection pattern. Distortion at each part in the image can be detected for each position from the grid arranged at regular intervals, and distortion at each part of the image can be effectively corrected based on the result.

  Further, for example, the distortion detection pattern may be transparent to visible light, and the transmittance for light having a predetermined wavelength outside the visible region may be lower than the transmittance for visible light. In this case, the distortion detection pattern is not reflected in the image in the imaging with visible light, and the image of the distortion detection pattern can be captured in the imaging under the light irradiation having the wavelength component outside the visible region. it can. For this reason, it is possible to appropriately perform distortion correction based on the distortion detection pattern image while avoiding the distortion detection pattern from appearing in the processing target image.

  According to this invention, when the fluid is injected into the recess of the sample holding plate and used for imaging, the degree of distortion of the image due to the surface state of the fluid is determined according to a known strain detection pattern. Since it is possible to estimate from the image, the distortion of the image can be easily and effectively reduced.

1 is a diagram illustrating a schematic configuration of an imaging apparatus according to the present invention. It is a figure explaining the distortion resulting from the surface state of a fluid. It is a figure which shows the principal part of 1st Embodiment of the imaging device concerning this invention. It is a figure which shows the example of the distortion which appears in the image of a grid pattern. It is a flowchart which shows the imaging operation in 1st Embodiment. It is a figure which shows the principal part of 2nd Embodiment of the imaging device concerning this invention. It is a flowchart which shows the imaging operation in 2nd Embodiment. It is a figure which shows the structure of the microplate in 3rd Embodiment.

<Overall configuration of device>
FIG. 1 is a diagram showing a schematic configuration of an imaging apparatus according to the present invention. As shown in FIG. 1A, the imaging device 1 has a substantially circular cross section, and a plurality of, for example, 96 (12 × 8, for example) liquids such as a culture solution, a culture medium, and a reagent are injected into each. A stage 15 on which a sample (microplate) M in which wells W in a matrix arrangement) are formed, a holder 11 that holds the stage 11, a light source 12 provided below the holder 11, and a holder 11 Is provided with an imaging unit 13 provided above and a control unit 10 that controls the imaging unit 13 to execute a predetermined operation. In FIG. 1A, the vertical direction is the vertical direction. The diameter and depth of each well W in the microplate M are typically about 10 mm or less. Note that the size of the microplate and the number of wells targeted by the imaging apparatus 1 are not limited to these and are arbitrary.

  The stage 15 is a flat plate formed of a transparent material such as quartz or resin, and is held by the holder 11 so that the upper surface is substantially horizontal. The planar size of the stage 15 is approximately the same as or slightly larger than that of the microplate M. The microplate M is held in a substantially horizontal state by being placed on the upper surface of the stage 15. The stage 15 can be removed from the holder 11 as necessary. In this state, the holder 11 can directly hold the microplate M by abutting against the peripheral edge of the lower surface of the microplate M. The stage 15 is provided with a distortion detection pattern for image distortion correction, which will be described in detail later.

  The light source 12 is controlled by a light source control unit 112 provided in the control unit 10 and is collectively applied to a plurality of wells W from below the microplate M held on the stage 15 according to a control command from the light source control unit 112. Then, the light L is irradiated. The irradiated light L passes through the transparent stage 15 and enters the well W from its bottom surface.

  The imaging unit 13 functions as a camera that captures an image of the microplate M by receiving the transmitted light Lt emitted from the light source 12 and transmitted above the microplate M. The imaging resolution is, for example, about 2400 dpi (dots per inch). The imaging unit 13 is connected to a camera driving mechanism 113 provided in the control unit 10, and the camera driving mechanism 113 scans and moves the imaging unit 13 in the horizontal plane along the lower surface of the microplate M held by the holder 11. Let

  That is, in this embodiment, the imaging unit 13 can be scanned and moved along the lower surface of the microplate M. Here, the imaging unit 13 moves relative to the microplate M. However, it is sufficient that the relative movement between the imaging unit 13 and the microplate M is realized. In this sense, the microplate M is moved relative to the imaging unit 13. You may make it move.

  The imaging unit 13 has a line sensor (not shown) in which a large number of fine imaging elements are arranged one-dimensionally in a uniaxial direction in a horizontal plane, and as shown in FIG. 1B, in the longitudinal direction of the line sensor. At least one whole well W, more desirably, a plurality of (three in the figure) wells W can be included in the imaging range SR at a time.

  Further, as shown in FIG. 1B, the scanning movement direction of the line sensor by the camera driving mechanism 113 is a direction orthogonal to the longitudinal direction of the imaging range SR (the arrangement direction of the imaging elements of the line sensor). As described above, the line sensor in which the imaging elements are arranged along a predetermined direction is scanned and moved in the scanning movement direction along the upper surface of the microplate M, thereby capturing a two-dimensional image of the microplate M viewed from the upper surface side. Is possible.

  Image data picked up by the image pickup unit 13 is given to the image processing unit 114. The image processing unit 114 appropriately performs image processing on the image data from the imaging unit 13 or executes predetermined calculation processing based on the image data. Data before and after processing is stored and saved in the storage unit 115 as necessary. Further, the detection processing unit 116 performs a predetermined detection process based on the image data given from the image processing unit 114, and detects a characteristic part included in the image. This detection process is a process for detecting areas in which the optical characteristics are different from the surrounding area in the image by analyzing luminance data of the image, for example, and by calculating a feature amount for the area, It is possible to classify the origin and type of the area. Since various techniques are known for the process of identifying and detecting a part having a certain feature from the image and the feature amount suitable for such a process, detailed description thereof is omitted here.

  The detection result by the detection processing unit 116 is also stored in the storage unit 115. Further, the image processing unit 114 may perform image processing based on the detection result by the detection processing unit 116 as necessary. Then, the image data subjected to appropriate image processing is given to a display unit 118 having display means such as a liquid crystal display, and the display unit 118 displays an image corresponding to the given image data and presents it to the user. . Further, the image display device 1 includes an input receiving unit 117 for receiving an operation instruction input from the user regarding the contents of the image processing, the display mode, and the like. The input receiving unit 117 is an input receiving unit such as a keyboard, a mouse, and a touch pad, or a combination of them. The input receiving unit 117 receives an instruction input from the user, and the control unit 10 reflects this in the operation of the apparatus. The function desired by the user is realized.

  The imaging apparatus 1 is configured such that a fluid (in this specification, a liquid, a gel-like or semi-fluid solid, and a fluid having a fluidity such as soft agar is held in each well W. And an optical image of an object to be imaged such as cells contained therein, or having a predetermined optical characteristic from the optical image, more specifically, in the well W The present invention can be applied to an application in which a specific portion having an optical characteristic different from that of a retained liquid or the like is detected using a difference in the optical characteristic. For example, it can be suitably used for the purpose of imaging a cell or a cell clump (spheroid) in a culture solution or a medium as an imaging object, or automatically detecting such a cell or the like by image processing.

  In this imaging device 1, light is irradiated from below the well W into which the fluid is injected, and the light transmitted above the well W is received and imaged. As shown in FIG. 1A, in a general microplate M, the side wall surface of the well W is tapered, and the horizontal cross-sectional area of the well W opening is smaller as it is closer to the bottom. When imaging is performed above the well W, the contents are not shielded by the side wall surface of the well W. On the other hand, since the contents of the well W are imaged through the liquid level of the fluid, distortion due to the surface state of the fluid may occur in the captured image as described below.

  FIG. 2 is a diagram for explaining distortion caused by the surface state of the fluid. Even when the microplate M is held horizontally, the surface S of the fluid FL is not necessarily horizontal due to factors such as the viscosity of the fluid FL injected into the well W, wettability with the wall surface of the well W, and meniscus. In many cases, it has unevenness. Further, this type of fluid FL is dispensed into the well W using, for example, a pipette, but unevenness may occur on the surface of the fluid FL due to variations in work at that time.

  As indicated by a one-dot chain line arrow in FIG. 2, even if the illumination light L incident from below the well W is parallel light, the refraction at the surface S of the fluid FL having such irregularities causes the upper part of the well W to be The direction of the transmitted light Lt that passes through is disturbed. For this reason, in the captured image, the observation object in the fluid FL, for example, the cell clump Sp, is distorted in its position, size, shape, etc., which may hinder observation and analysis. .

  Therefore, in this embodiment, this distortion is detected as described below, and the distortion is corrected using the result. Here, two embodiments that enable detection and correction of distortion in the imaging apparatus 1 having the above-described configuration will be described, and the principle and specific operation will be described in order. In the embodiments, some of the configurations of the imaging apparatus 1 illustrated in FIG. 1A are more specific, and the basic configuration is as described above. Below, the same code | symbol is attached | subjected about the structure same as the said structure.

<First Embodiment>
FIG. 3 is a diagram showing a main part of the first embodiment of the imaging apparatus according to the present invention. This embodiment is characterized by a configuration corresponding to the imaging unit 13, the light source 12, the light source control unit 112, and the image processing unit 114 shown in FIG. An imaging unit, a light source, a light source control unit, and an image processing unit in this embodiment are denoted by reference numerals 13a, 12a, 112a, and 114a, respectively.

  The light source 12a in this embodiment includes a white light source 121 that emits white light and an infrared light source 122 that emits infrared light, and these are controlled by the light source control unit 112a. More specifically, a lighting signal is selectively given to the white light source 121 and the infrared light source 122 from the light source control unit 112a, and one of the white light source 121 and the infrared light source 122 is selectively turned on accordingly. To do.

  Light (white light or infrared light) L emitted from the light source 12 a enters the microplate M via the stage 15. As described above, the stage 15 is formed of a transparent material and has a regular distortion detection pattern. The strain detection pattern is formed on the upper surface, lower surface or inside of the stage 15 by printing, stamping or etching. As the pattern, a grid pattern in which the regularity of the pattern can be easily read from an image obtained by capturing the pattern, for example, a pattern GP1 in which dots are provided at each of lattice points having a constant pitch as shown in FIG. As shown in FIG. 3C, a pattern GP2 or the like in which two sets of parallel lines having a constant pitch are orthogonal to each other can be used.

  The transmitted light Lt transmitted upward from the stage 15 enters the imaging unit 13a. The imaging unit 13a includes a CCD imaging device 132 that constitutes a line sensor, an imaging optical system 131 that converges the transmitted light Lt on the CCD imaging device 132, and an AD converter that converts an analog output signal of the CCD imaging device 132 into a digital signal. (A / D) 133. The image picked up by the CCD image pickup device 132 includes the image of the grid pattern described above in addition to the image corresponding to the contents of the well W.

  The distortion that appears in the image of the captured grid pattern is quantitatively estimated by comparing the positional relationship of the components (dots and lines) in the known pattern with the positional relationship in the image in which they are actually captured. Is possible. Since distortion due to the surface state of the fluid FL should appear in the same manner in the grid pattern image and the well contents image, detecting the grid pattern image distortion is equivalent to the well contents. This corresponds to detecting distortion appearing in the image of the object.

  The circles indicated by dotted lines in FIG. 3B and FIG. 3C represent the contour of the well W, and thus by including a large number of grid points inside the contour of the well, The amount of distortion in each part can be detected finely. On the other hand, each grid point must be clearly separable in the image, and there is a problem even if the grid pitch is too small. The grid pitch P that satisfies these requirements can be set to, for example, about 0.2 mm. In FIG. 3B and FIG. 3C, the grid pitch P is the same in the vertical direction and the horizontal direction, but the present invention is not limited to this.

  Further, the formation position of the grid pattern in the thickness direction of the stage 15 may in principle be provided on any of the upper surface, the lower surface, and the inside of the stage 15. However, by providing it as close as possible to the well contents to be imaged, it is possible to take an image with both the well contents and the grid pattern in focus, and the quality and distortion correction of the captured image Accuracy can be improved. In this sense, it is desirable to provide a grid pattern on the upper surface of the stage 15, and it is desirable to hold the microplate 15 with the upper surface of the stage 15 and the bottom surface of the well W as close as possible.

  FIG. 4 is a diagram showing an example of distortion appearing in the grid pattern image. Here, the X coordinate axis is set in the right direction and the Y coordinate axis is set in the downward direction with the upper left corner of the image as the origin. FIG. 4A is an example of a grid pattern image captured without injecting liquid into the well W, and FIG. 4B is an example of a grid pattern image captured by injecting liquid. In FIG. 4A, the dots of the grid pattern are aligned in a straight line at a substantially constant pitch in the X and Y directions throughout the circle corresponding to the outline of the well, and almost no distortion is observed.

  On the other hand, in the image of FIG. 4B, the dots are evenly aligned at the center of the well, but at the peripheral edge of the well, the line becomes a curve and the pitch also varies. Specifically, the dot pitch is clogged as it approaches the periphery. This is a distortion mainly caused by the meniscus. In other words, the liquid surface rises and the light refraction increases as it approaches the peripheral edge of the well, and in the imaging result, the dot pitch becomes smaller than the actual pitch.

  The distortion shown in FIG. 4B can be canceled by moving the dots so that the dots are arranged in a straight line in both the X and Y directions at a constant pitch. . Also for each pixel between dots, the respective movement amounts necessary for correction can be obtained by interpolation from the movement amounts of neighboring dots. The image distortion can be corrected by applying the movement of each pixel with the movement amount thus obtained to each pixel of the image obtained by imaging the well contents. FIG. 4C is obtained by correcting the image shown in FIG. 4B by applying a correction process described later, and the image is close to FIG. I can ask.

  In such correction based on the “pixel position adjustment” principle, the amount of pixel movement necessary for correction is not necessarily an integral multiple of the pixel pitch. Thereby, the pixel in the image before correction and the pixel in the image after correction may not correspond one to one. In order to deal with this problem, for example, the pixel value of each pixel in the image after correction is determined based on the pixel value of the corresponding pixel before correction and the pixel values of surrounding pixels, for example. What is necessary is just to obtain | require by a simple average or a load average.

  In this embodiment, as shown in FIG. 3A, an output signal from the CCD imaging device 132 is converted into digital data by an AD converter 133, and the digital data is converted into an image acquisition unit 1141 of the image processing unit 114a and a grid pattern. This is given to the calculation unit 1142. The grid pattern calculation unit 1142 calculates the movement amount of each pixel necessary for correcting the distortion from the positional relationship of the captured grid pattern dots. On the other hand, the correction by applying the movement of each pixel obtained by the grid pattern calculation unit 1142 to the image of the well contents acquired from the data given to the image acquisition unit 1141 is performed by the image calculation unit 1143. It is executed by. In this way, a corrected image in which image distortion is canceled is obtained.

  It is desirable that the grid pattern is not originally included in the imaging result. Therefore, in this embodiment, the grid pattern provided on the stage 15 is made transparent to visible light and opaque to infrared light, and an image of the grid pattern is acquired from the imaging result under infrared light irradiation. The image of the well contents is acquired from the imaging result under white light irradiation. Specifically: The grid pattern does not need to be completely transparent to visible light, but at least for specific light outside the visible region (for example, infrared light) with respect to the transmittance of visible light. It is desirable to make the transmittance lower.

  FIG. 5 is a flowchart showing the imaging operation in the first embodiment. First, the microplate M in which a fluid is injected into each well W is set in the apparatus (step S101), and infrared light is designated as a light source (step S102). That is, the light source control unit 112a outputs a lighting signal only to the infrared light source 122 of the two light sources provided in the light source 12a, and turns on the infrared light source 122. Thereby, infrared light is irradiated from the light source 12a toward the stage 15. By imaging in this state, an image of a grid pattern that is opaque to infrared light is acquired (step S103). From the positional relationship of the individual dots in the grid pattern thus obtained, the movement amount of each pixel necessary for removing image distortion is obtained as correction data (step S104).

  Next, white light is designated as the light source (step S105). That is, the light source control unit 112a outputs a lighting signal only to the white light source 121 of the two light sources provided in the light source 12a, and turns on the white light source 121. Thereby, white light is irradiated from the light source 12a toward the stage 15. By performing imaging in this state, it is possible to obtain an image in which only the well contents are shown without showing a grid pattern transparent to visible light (step S106). The image of the well contents obtained in this way is used as a processing target image, and the correction data obtained previously is applied to the image to be corrected (step S107), whereby the corrected image from which distortion has been removed is obtained. can get.

  As described above, in this embodiment, the well W is imaged in a state where the transparent stage 15 in which the grid pattern is formed is interposed between the light source 12a and the well W, and distortion of the imaged grid pattern is detected. The amount of movement of each pixel for correcting this is obtained. Then, the distortion of the image can be effectively removed by applying the movement amount to the image of the well contents and performing the correction. Since the distortion amount of each part of the image is easily grasped from the image of the grid pattern, the distortion can be reduced by a relatively simple correction.

  Here, the grid pattern is transparent to visible light and can be imaged only with infrared light, and the grid pattern and well contents are individually imaged by switching the light source. By doing so, the grid pattern is not reflected in the final image, and the influence on observation and analysis is prevented.

  By separately capturing the grid pattern and the well contents, the imaging unit 13a needs to be scanned twice with respect to the microplate M, which increases the processing time. However, the grid pattern is captured by the positional relationship between the components. Therefore, there is an advantage that the configuration of the imaging unit 13a is simplified because the imaging can be performed using the same imaging device. Further, since correction data is obtained prior to imaging of the well contents, it is possible to perform correction in real time according to the progress of imaging of the well contents.

Second Embodiment
FIG. 6 is a diagram showing a main part of a second embodiment of the imaging apparatus according to the present invention. An imaging unit, a light source, a light source control unit, and an image processing unit in this embodiment are denoted by reference numerals 13b, 12b, 112b, and 114b, respectively.

  The light source 12b of this embodiment also includes a white light source 123 and an infrared light source 124. However, these are turned on / off all at once by the light source controller 112b. Therefore, the light L on which infrared light and white light are superimposed is emitted from the light source 12b toward the stage 15.

  The light Lt transmitted above the stage 15 is incident on a spectroscope, for example, a dichroic mirror 135, through an imaging optical system 134 provided in the imaging unit 13b of this embodiment. The dichroic mirror 135 splits incident light into an infrared component and a visible component. Of the split light, the visible component enters the CCD image sensor 136, and the infrared component enters the other CCD image sensor 137. To do. The output signal of the CCD image pickup device 136 is converted into digital data by the AD converter 137 and given to the image acquisition unit 1141 of the image processing unit 114b. On the other hand, the output signal of the CCD image sensor 138 is converted into digital data by the AD converter 139, and is supplied to the grid pattern correction calculation unit 1142 of the image processing unit 114b. The functions of the image acquisition unit 1141, the grid pattern correction calculation unit 1142, and the image calculation unit 1143 provided in the image processing unit 114b are the same as those provided in the first embodiment.

  In such a configuration, an infrared component corresponding to the image of the grid pattern and a visible component corresponding to the image of the well contents are simultaneously acquired and dispersed by the dichroic mirror 135, whereby each image is individually obtained. Imaged. This also enables distortion detection in the grid pattern image and correction of the well content image based thereon.

  FIG. 7 is a flowchart showing an imaging operation in the second embodiment. In this embodiment, after setting the microplate M in the apparatus (step S201), infrared light and white light are simultaneously irradiated onto the microplate M (step S202). Then, the grid pattern is captured as an image of the infrared component after the spectrum by the dichroic mirror 135, and the well contents are captured as an image of the visible light component (step S203).

  Of these, correction data is obtained from an infrared component image, that is, a grid pattern image (step S204), and this is applied to the well contents image to perform correction (step S205), which is the same as in the first embodiment. The same.

  In this embodiment, light including both infrared light and visible light is irradiated toward the well W, the transmitted light is dispersed, and an image of the grid pattern from the infrared component and an image of the contents of the well from the visible component. Get at the same time. Thereby, since both images can be acquired by one scanning, the processing time required for the operation can be shortened. Then, by detecting the distortion of the image of the grid pattern and correcting the image of the well contents based on the result, it is possible to effectively remove the distortion of the image as in the first embodiment.

<Third Embodiment>
Next explained is the third embodiment of the invention. In the imaging devices of the first and second embodiments described above, the grid pattern is provided on the stage 15 provided between the illumination light source 12 (12a, 12b) and the microplate M. By doing so, the shape and dimensions of the grid pattern, the positional relationship with the imaging unit 13 and the like are unique to the apparatus, and it is avoided that product variations affect distortion correction. Moreover, since the special specification is not requested | required for the microplate M, the general thing currently marketed can be used.

  On the other hand, in a third embodiment described below, a grid pattern is directly provided on the bottom surface of the microplate. As a result, the stage used in the first and second embodiments can be omitted, and imaging can be performed by directly holding the microplate by the holder. Note that this embodiment is characterized by the microplate itself, and the configuration and operation of the imaging apparatus are the same as those described in any of the first and second embodiments described above. However, as described above, the following microplate is directly held by the holder 11 without using the stage 15.

  FIG. 8 is a diagram showing the structure of the microplate in the third embodiment. More specifically, FIG. 8A is an exploded perspective view of the microplate in this embodiment, and FIG. 8B is a partial cross-sectional view of the microplate. The microplate 2 has an upper portion in which through holes 211 having a substantially cylindrical side shape (more precisely, a taper whose sectional area gradually decreases toward the bottom surface) are regularly arranged in a two-dimensional matrix at a constant pitch. It has the plate 21 and the lower surface sheet 22 affixed on the lower surface of the upper plate 21 so as to block each through hole 211.

  As shown in FIG. 8B, the lower surface sheet 22 is closely attached to the lower surface of the upper plate 21, and liquid is supplied to the space surrounded by the side surface of the through hole 211 of the upper plate 21 and the lower surface sheet 22. It is possible to hold. That is, this space functions as a well W for holding a fluid, and the side surface of the through hole 211 forms the side wall surface of the well W, and the lower surface sheet 22 forms the bottom surface of the well W.

  The lower surface sheet 22 is a sheet body formed of a transparent resin, for example, PET (polyethylene terephthalate) resin, and a regular grid pattern GP3 is provided on the upper surface or the lower surface thereof as shown in FIG. . As the pattern shape of the grid pattern, a grid pattern in which the regularity of the pattern can be easily read from an image obtained by capturing the grid pattern, for example, the same one as shown in FIG. 3B or 3C can be used. Further, the forming method can be, for example, printing or engraving. Like the grid pattern provided on the stage 15 of the first and second embodiments, it is more preferable that the pattern is transparent to visible light and can be imaged by light outside the visible region, for example, infrared light.

  In the microplate 2 configured as described above, since the grid pattern is formed in advance on the bottom surface of the well, imaging can be performed without using a stage provided with the grid pattern. For this reason, it is possible to use what was comprised more compactly as an imaging device. In addition, since the grid pattern is arranged at a position very close to the well contents, both the well contents and the grid pattern can be stored within the depth of field of the imaging unit, and a clear image can be obtained. Better results in quality and correction accuracy.

  In addition, since the grid pattern is formed on the microplate, a conventional apparatus that does not have a function of arranging the grid pattern can be applied as the imaging apparatus. Even in imaging with such an apparatus, image distortion can be effectively removed by applying the imaging method according to the present invention.

<Others>
As described above, in each of the above embodiments, the microplates M and 2 correspond to the “sample holding plate” of the present invention, and the well W corresponds to the “recessed portion” of the present invention. The upper plate 21 and the lower surface sheet 22 of the microplate 2 correspond to the “plate” and “transparent sheet” of the present invention, respectively.

  Further, in each of the above embodiments, the holder 11 and the stage 15 function as the “holding means” of the present invention, and the stage 15 also has the functions of the “transparent stage” and the “pattern forming body” of the present invention. . In the above embodiment, the light source 12 (12a, 12b) functions as the “illuminating unit” of the present invention, and the imaging unit 13 (13a, 13b) functions as the “imaging unit” of the present invention. Further, the image processing unit 114 (114a, 114b) functions as the “image correction unit” of the present invention. In addition, the dichroic mirror 135 in the second embodiment functions as the “spectrometer” of the present invention.

  In the operation of the above embodiment, steps S102 to S103, S105 to S106, and S202 to S203 correspond to the “imaging process” of the present invention. Among these, steps S102 to S103 and S105 to S106 correspond to the “test pattern imaging step” and “processing target image imaging step” of the present invention, respectively. Steps S104 and S204 correspond to the “detection step” of the present invention, while steps S107 and S205 correspond to the “correction step” of the present invention.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, image distortion correction is performed using a grid pattern that is transparent to visible light, but the grid pattern is transparent to visible light as long as it can be imaged by some optical method. That is not an essential requirement. It is permissible for the image of the grid pattern to be reflected in the image as long as there is no problem in observing the contents of the well, and since the pattern is known, the image of the grid pattern can be deleted from the image by image processing after imaging. Is also possible.

  In the second embodiment, the dichroic mirror 135 is used as the “spectral section” of the present invention. For example, the grid pattern image and the well contents image are separated by an appropriate combination of optical filters. May be. Alternatively, the image of the grid pattern and the image of the well contents may be individually captured by switching the optical filter and performing imaging twice.

  In the third embodiment, the lower surface sheet 22 on which the grid pattern is formed is provided so as to block the bottom of the through hole 211 of the upper plate 21 so that the lower surface sheet 22 functions as the bottom surface of the well. However, the present invention is not limited to this, and a transparent sheet having a grid pattern may be pasted on a microplate having a bottom formed in advance. In this case, the sheet may be attached to either the inner bottom surface or the outer bottom surface of the well.

  Moreover, in the said embodiment, although the stage 15 in which the grid pattern was formed is detachable with respect to the holder 11, making the stage 15 removable is not an essential requirement. On the other hand, the pattern forming body on which the grid pattern is formed may be prepared as an independent instrument separate from the imaging device, and used as necessary. Alternatively, imaging may be performed by placing the microplate 2 having the grid pattern according to the third embodiment on a transparent stage without a grid pattern.

  Further, for example, in the above-described embodiment, imaging is performed by moving a line sensor having a one-dimensional array of image sensors relative to the microplate M, but a CCD array in which the image sensors are two-dimensionally arranged in a matrix is used. It is also possible to perform imaging.

  The present invention can be particularly suitably applied to a field that requires observation of a sample in which a fluid is injected into a recess, such as a well on a microplate used in the medical / biological science field. The application field is not limited to the medical / biological science field.

2, M microplate (sample holding plate)
11 Holder (holding means)
12, 12a, 12b Light source (illumination means)
13, 13a, 13b Imaging unit (imaging means)
15 stages (holding means, transparent stage, pattern forming body)
21 Upper plate (plate)
22 Bottom sheet (transparent sheet)
114, 114a, 114b Image processing unit (image correction means)
135 Dichroic mirror (spectral part)
GP1, GP2, GP3 Grid pattern (distortion detection pattern)
S102 to S103 Imaging process, test pattern imaging process S105 to S106 Imaging process, processing target image imaging process S202 to S203 Imaging process S104, S204 Detection process S107, S205 Correction process W well (recessed part)

Claims (12)

Distortion detection in which a sample holding plate having a recess capable of holding a liquid on its upper surface is substantially horizontal, and the bottom surface of the recess is a regular pattern having a lower transmittance for light of a predetermined wavelength than the bottom surface. Holding means for abutting and holding a transparent flat plate-like or sheet-like pattern forming body on which a pattern is formed;
Illumination means for irradiating light to the recess from below the recess via the pattern forming body,
An imaging unit that receives light transmitted through the depression and images the contents of the depression and obtains a processing target image above the depression;
An image pickup apparatus comprising: an image correction unit that corrects distortion of the processing target image based on an image of the distortion detection pattern imaged by the image pickup unit.   The holding means has a transparent stage on which the distortion detection pattern is formed, holds the sample holding plate placed on the transparent stage, and the transparent stage functions as the pattern forming body. The imaging apparatus according to 1.   The imaging apparatus according to claim 2, wherein the distortion detection pattern is transparent to visible light, and has a lower transmittance for light of a predetermined wavelength outside the visible region than the transmittance for visible light.   The imaging apparatus according to claim 1, wherein the imaging unit images the contents of the depression simultaneously with the distortion detection pattern.   5. The image correction unit detects distortion of the distortion detection pattern from the image of the distortion detection pattern and performs correction necessary for canceling the distortion on the processing target image. The imaging device according to any one of the above.   The image correction unit corrects an image in which the positional relationship of the image elements in the processing target image is adjusted based on the positional relationship of the components of the distortion detection pattern grasped from the image of the distortion detection pattern. The imaging apparatus according to claim 5, which is a later image.   The imaging unit receives the wavelength component in the visible region of the light emitted from the illumination unit and images the processing target image, while the wavelength component outside the visible region of the light emitted from the illumination unit. The imaging apparatus according to claim 1, wherein the image of the distortion detection pattern is captured by receiving light.   The illumination unit selectively irradiates the first light including a wavelength component in a visible region and the second light including only a wavelength component outside the visible region toward the recess, and the imaging unit includes: The image of the processing target image is captured in a state where the first light is irradiated from the illumination unit, and the image of the distortion detection pattern is captured in a state where the second light is irradiated from the illumination unit. Item 8. The imaging device according to Item 7.   The illuminating unit irradiates light that includes both a wavelength component in the visible region and a wavelength component outside the visible region toward the recess, while the imaging unit transmits light that passes through the recess in the visible region. 8. The light source according to claim 7, further comprising: a spectral unit that splits the wavelength component into a wavelength component outside the visible region and a wavelength component outside the visible region, and separately receiving the split light to capture the image of the processing target image and the distortion detection pattern. Imaging device. Distortion detection in which a sample holding plate having a recess capable of holding a liquid on its upper surface is substantially horizontal, and the bottom surface of the recess is a regular pattern having a lower transmittance for light of a predetermined wavelength than the bottom surface. A transparent flat plate-like or sheet-like pattern forming body on which a pattern for use is formed and held, and light is applied to the recess from the lower side of the sample holding plate via the pattern forming body, An imaging step of receiving light transmitted above the depression and imaging the contents of the depression to obtain a processing target image;
A detection step of detecting distortion of an image of the distortion detection pattern obtained by imaging the depression;
An image correction method comprising: an image correction step of correcting distortion of the processing target image based on a detection result in the detection step. The imaging step includes a processing target image imaging step of capturing the processing target image by irradiating the first light including a wavelength component in the visible region toward the depression, and a first target including only the wavelength component outside the visible region. The image pickup method according to claim 10 , further comprising: a distortion detection pattern imaging step of irradiating the light of 2 toward the depression and capturing an image of the distortion detection pattern. In the imaging step, the light including both the wavelength component in the visible region and the wavelength component outside the visible region is irradiated toward the recess, and the light transmitted through the recess is in the visible region and the visible region. The spectral component is separated into outside wavelength components, and the wavelength component in the visible region is received to obtain the processing target image, while the wavelength component outside the visible region is received to obtain the distortion detection pattern image. The imaging method according to 10 .