JPWO2018088460A1 - Well address acquisition system, well address acquisition method, and program - Google Patents

Abstract

Provided is a well address acquisition system that allows an operator to easily grasp the address of a well to be dispensed. The well address acquisition system transforms a master image representing an image of the array surface of the wells of the microplate according to a reference image generated by causing the imaging device to image the array surface, and each well region in the master image Is generated by causing the imaging device to image the array plane in a state where it is dispensed onto the microplate by a dispensing pipette A plurality of wells based on a tip detection unit that detects a tip position of a chip image in the target image, a tip position in the target image, and pixel position information associated with each well region And an address acquisition unit for acquiring the address of the well where the tip of the chip is located.

Description

  The present invention relates to a well address acquisition system, a well address acquisition method, and a program for supporting a dispensing operation of dispensing into a microplate using a dispensing pipette.

  Conventionally, in the medical field, the biochemical field, and the like, a microplate provided with a plurality of wells for storing liquids is used for examinations and experiments. In a general microplate, a large number of wells are arranged in a matrix, and numbers and alphabets indicating the rows and columns of the wells are arranged at the end portions (for example, near the left side and the upper side) of the wells. It is displayed. The position of each well (hereinafter also referred to as a well address or simply an address) is represented by, for example, “A row 1 column” or “A1” using these symbols. Note that the microplate is also referred to as a well plate, a microwell plate, or the like.

  When manually dispensing using a dispensing pipette, the operator searches for a well to be dispensed from among a number of wells based on the symbol displayed at the end, and dispenses it. Such work requires a high concentration over a long period of time, which is burdensome and stressful for the operator. In addition, there is a possibility that an artificial mistake of accidentally dispensing into wells that should not be dispensed may occur.

  For such a problem, Patent Document 1 discloses a technique that allows an operator to visually recognize whether or not liquid is contained in a well by irradiating each well with inspection light from the bottom. ing.

JP 2010-96754 A

  To visually confirm the address of the well, the operator must determine whether the well into which the tip of the pipette pipette tip has been inserted is the correct well to be dispensed. It's not easy. In this regard, in the above-mentioned Patent Document 1, although the well in which the liquid has already been stored and the well in which the liquid has not yet been stored are identified, the well to be dispensed by the operator is specified. I don't do it.

  The present invention has been made in view of the above, and provides a well address acquisition system, a well address acquisition method, and a program that enable an operator to easily grasp the address of a well to be dispensed. Objective.

  In order to solve the above problems, a well address acquisition system according to one embodiment of the present invention includes an imaging element that generates an electrical signal by photoelectrically converting light incident on a light receiving surface, and an image based on the electrical signal. An imaging device that generates and outputs data; and an information processing device that generates an image based on image data output from the imaging device, wherein the information processing device includes an array of a plurality of wells included in the microplate Generated by causing the imaging device to image the first image data generated by causing the imaging device to image a surface and the array surface in a state of being dispensed to the microplate by a dispensing pipette. An image acquisition unit for acquiring second image data, and a master image representing an image of the array plane, each corresponding to the plurality of wells By transforming a master image including a number of well regions according to a reference image, which is an image generated based on the first image data, and mapping it to a pixel region corresponding to the light receiving surface, each well region A tip position of the chip image in the target image is detected from a mapping processing unit that associates the position information of the pixel in the pixel region with the target image that is an image generated based on the second image data. Based on the tip detection unit, the tip position in the target image, and the position information of the pixel associated with each well region, the well region to which the tip position belongs is extracted. An address acquisition unit that acquires an address of a well in which the tip of the chip is located.

  In the well address acquisition system, the information processing apparatus further includes a feature point acquisition unit that acquires a plurality of feature points in the image of the array plane in the reference image, and the mapping processing unit includes the plurality of features. Image conversion may be performed by associating a plurality of points in the master image in which images corresponding to the points are represented with the plurality of feature points.

  In the well address acquisition system, the tip detection unit may generate a difference image between the reference image and the target image and detect the tip position from the difference image.

  The well address acquisition system may further include a surface light source that illuminates at least the array surface.

  The well address acquisition system further includes a slit light source that is provided with a slit parallel to the array surface of the microplate, and emits light that travels along an optical path covering the array surface, An image of a light emitting point that emits light when the chip crosses the light may be detected from a target image, and the position of the image may be detected as the tip position.

  In the well address acquisition system, the address acquisition unit extracts a well region associated with a pixel in the pixel region corresponding to the tip position, and acquires a well address corresponding to the extracted well region. May be.

  In the well address acquisition system, the address acquisition unit may acquire an address of a well corresponding to the same well region when the tip position remains in the same well region for a predetermined time or more.

  The well address acquisition system may further include a jig that determines a position of the microplate with respect to the imaging device.

  A well address acquisition method according to another aspect of the present invention includes an imaging device that generates an electrical signal by photoelectrically converting light incident on a light receiving surface, and generates and outputs image data based on the electrical signal. A well address acquisition method comprising: an imaging apparatus; and an information processing apparatus that generates an image based on image data output from the imaging apparatus. A step (a) of acquiring first image data generated by causing the imaging device to image an array surface of a plurality of wells, and a master image representing an image of the array surface, A master image including a plurality of well regions, each corresponding to a well, is an image generated based on the first image data. (B) associating each well region with pixel position information in the pixel region by deforming it in accordance with a reference image and mapping it to a pixel region corresponding to the light receiving surface; and the microplate by means of a dispensing pipette A step (c) of acquiring second image data generated by causing the imaging device to image the array surface in a state of being dispensed into the image, and an image generated based on the second image data (D) detecting the tip position of the image of the chip in the target image from the target image, and the position information of the pixels associated with the well regions and the tip position in the target image. By extracting the well region to which the tip position belongs, the address of the well where the tip of the chip is located among the plurality of wells And step (e) to obtain, is intended to include.

  A program according to still another aspect of the present invention includes an imaging device that generates an electrical signal by photoelectrically converting light incident on a light receiving surface, and generates and outputs image data based on the electrical signal And a well address acquisition system comprising: an information processing device that generates an image based on image data output from the imaging device; and a program to be executed by the information processing device, the plurality of wells of the microplate A step (a) of acquiring first image data generated by causing the imaging device to image an array plane; and a master image representing an image of the array plane, each corresponding to the plurality of wells A master image including a plurality of well regions is matched with a reference image that is an image generated based on the first image data. (B) associating each well region with the position information of the pixel in the pixel region by deforming and mapping the pixel region corresponding to the light receiving surface, and dispensing the microplate using a dispensing pipette A step (c) of acquiring second image data generated by causing the imaging device to image the array surface in a state of being present, and a target image that is an image generated based on the second image data Detecting the tip position of the tip image of the chip in the target image, and the tip position based on the tip position in the target image and the position information of the pixels associated with each well region. Extracting a well region to which the tip of the chip is located among the plurality of wells by extracting a well region to which the chip belongs And e), it is intended to run.

  According to the present invention, the position of the tip of the chip in the target image is detected, and based on this position and the position information of the pixel in the pixel region associated with each well region in the master image, the tip of the chip Since the address of the well where is located is acquired, the operator can easily grasp the address of the well to be dispensed.

It is a mimetic diagram showing a schematic structure of a well address acquisition system concerning a 1st embodiment of the present invention. It is a schematic diagram for demonstrating the positional relationship between the apparatuses shown in FIG. It is a schematic diagram for demonstrating the positional relationship between the apparatuses shown in FIG. It is a block diagram which shows the main structures of the information processing apparatus shown in FIG. It is a partial enlarged view of the arrangement surface of a microplate. It is a schematic diagram which illustrates a master image. It is a flowchart which shows operation | movement of the well address acquisition system which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the registration process of a microplate. It is a schematic diagram which illustrates the input screen of the product information of a microplate. It is a schematic diagram which illustrates a reference image. It is a schematic diagram which illustrates the master image to which image conversion was performed. It is a schematic diagram for demonstrating the mapping process of a master image. It is a schematic diagram which illustrates a target image. It is a schematic diagram which illustrates a difference image. It is a schematic diagram which illustrates a dispensing assistance screen. It is a flowchart which shows the acquisition process of a well address. It is a schematic diagram for demonstrating the acquisition process of a well address. It is a schematic diagram for demonstrating the determination process of a well address. It is a schematic diagram which illustrates a dispensing assistance screen (when the position of a chip | tip is incorrect). It is a schematic diagram which illustrates a dispensing assistance screen (when the position of a chip | tip is correct). It is a schematic diagram for demonstrating the modification of the registration process of product information. It is a schematic diagram for demonstrating the 1st modification of the acquisition process of a feature point. It is a schematic diagram for demonstrating the 2nd modification of the acquisition process of a feature point. It is a schematic diagram for demonstrating the 2nd modification of the acquisition process of a feature point. It is a schematic diagram for demonstrating the 1st modification of the acquisition process of the position of a chip | tip tip. It is a schematic diagram for demonstrating the 2nd modification of the acquisition process of the position of a chip | tip tip. It is a schematic diagram for demonstrating the 2nd modification of the acquisition process of the position of a chip | tip tip. It is a schematic diagram for demonstrating the 3rd modification of the acquisition process of the position of a chip | tip tip. It is a schematic diagram for demonstrating the 3rd modification of the acquisition process of the position of a chip | tip tip. It is a schematic diagram which shows schematic structure of the well address acquisition system which concerns on the 2nd Embodiment of this invention. It is a schematic diagram which illustrates the target image produced | generated in the 2nd Embodiment of this invention.

  Hereinafter, a well address acquisition system, a well address acquisition method, and a program according to embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited by these embodiments. Moreover, in description of each drawing, the same code | symbol is attached | subjected and shown to the same part.

  The drawings referred to in the following description only schematically show the shape, size, and positional relationship so that the content of the present invention can be understood. That is, the present invention is not limited only to the shape, size, and positional relationship illustrated in each drawing. Also, there are cases in which parts having different dimensional relationships and ratios are included between the drawings.

(First embodiment)
FIG. 1 is a diagram showing a schematic configuration of a well address acquisition system according to the first embodiment of the present invention. 2 and 3 are schematic diagrams for explaining the positional relationship between the apparatuses shown in FIG.

  In the well address acquisition system 1 shown in FIG. 1, the tip of the tip 3a of the dispensing pipette 3 is positioned in the dispensing operation in which the dispensing pipette 3 is used to dispense the microplate 2 in which a plurality of wells 2a are arranged. This is a system for acquiring the address of the well 2a (hereinafter also referred to as well address or simply address). As shown in the figure, the well address acquisition system 1 captures the image data output from the camera 4 that images the microplate 2 and outputs image data, the illumination device 5 that illuminates the microplate 2, and the camera 4. An information processing apparatus 6 that captures and generates an image and obtains a well address based on the image;

  The microplate 2 is an instrument in which depressions (wells 2a) for arranging specimens and reagents are arranged in a matrix on a plate-like substrate formed of a resin such as polyethylene, polypropylene, polystyrene, or glass. Also called plate or microwell plate. In the present embodiment, as an example, a case where a 96-well microplate 2 in which wells 2a are arranged in 8 rows and 12 columns is used will be described. It can be applied when using all kinds of microplates regardless of manufacturers.

  The dispensing pipette 3 has a pipette body 3b and a tip 3a attached to the tip of the pipette body 3b. The pipette main body 3b is provided with a syringe, and by reciprocating the piston along the axial direction in the syringe, the liquid is sucked from the tip of the tip 3a and the liquid is poured out from the tip.

  As the pipetting pipette 3, various products are known such as a type in which the piston is manually operated and a type in which the piston is driven by an electric motor. However, the present invention is a piston driving method, pipette size, manufacturer, etc. Regardless, the single-channel type can be applied when using any kind of pipetting pipette. For example, a pipette pipette that communicates with the information processing apparatus 6 via a communication network such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) can be used. When such a pipetting pipette is used, the pipetting quantity can be automatically adjusted by transmitting a control signal for controlling the driving amount of the electric motor from the information processing device 6 to the pipetting pipette. Alternatively, by transmitting a detection signal for detecting the drive of the electric motor from the pipette pipette to the information processing apparatus 6, the information processing apparatus 6 can grasp that the dispensing has been performed.

  The camera 4 is an imaging device that has an imaging element 4a such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and that can capture a moving image at a predetermined imaging frame rate (for example, 15 fps or more). The image sensor 4a receives light (subject image) incident on the camera 4 and imaged by the optical system on the light receiving surface, and generates an electrical signal by performing photoelectric conversion. The camera 4 generates and outputs image data by performing predetermined signal processing such as amplification and A / D conversion on the electrical signal.

  As the camera 4, it is preferable to use a camera capable of generating a color image in which color filters are arranged in a predetermined arrangement (for example, a Bayer arrangement) on the light receiving surface of the image sensor 4 a. More preferably, a small camera that does not interfere with the dispensing operation, for example, a USB camera that can be connected to the information processing apparatus 6 via a USB (Universal Serial Bus) cable may be used.

  As shown in FIG. 2, the camera 4 is arranged so that the entire arrangement surface 2 b of the microplate 2 is accommodated in the visual field V. In FIGS. 1 and 2, the camera 4 is installed on the illumination device 5, but the installation method of the camera 4 is not limited to this. For example, a stand may be installed on a work table for dispensing work or a stage 7 described later, and the camera 4 may be attached to the stand. In any case, the camera 4 is installed at a position where the entire array surface 2b can be imaged and does not interfere with the dispensing operation. In particular, it is necessary to determine the position and angle of the camera 4 so that the field of view of the camera is not blocked by the operator's hand and the tip of the chip 3a is not reflected on the camera 4 during the dispensing operation.

  The illumination device 5 is a light source device that illuminates the arrangement surface 2 b of the microplate 2. As the illuminating device 5, it is preferable to use a surface light source having a planar light emitting surface 5 a so that the entire array surface 2 b can be illuminated with substantially uniform illuminance. More preferably, a high-luminance light source is used to suppress the influence of ambient light such as room light. In this embodiment, high-luminance LED surface illumination (flat illumination) (for example, chip LED surface illumination IDHM-62 / 122DWT manufactured by Immac Co., Ltd.) is used as the illumination device 5.

  The size of the illumination device 5 and the positional relationship with the microplate 2 are preferably adjusted as appropriate according to the size of the microplate 2. As shown in FIG. 2, the distance L from the light emitting surface 5a to the microplate 2 should be a certain amount (for example, 5 cm or more) so that the front side of the microplate 2 can be sufficiently illuminated when viewed from the light emitting surface 5a. good. Further, the height H of the light emitting surface 5a is preferably several times the height h of the arrangement surface 2b so that the front end side of the microplate 2 can be sufficiently illuminated when viewed from the light emitting surface 5a. When the general-purpose microplate 2 is used, the height H of the light emitting surface 5a is preferably 7 cm or more. Furthermore, as shown in FIG. 3, the width W of the light emitting surface 5a is preferably set to be equal to or larger than the width w of the microplate 2 so that the left and right ends of the arrangement surface 2b can be sufficiently illuminated.

  The illumination device 5 may be omitted if the periphery of the microplate 2 is bright enough to capture the entire array surface 2b by the camera 4.

  The microplate 2 is preferably placed on the stage 7 painted with a uniform black surface. Since the microplate 2 is often made of a transparent or translucent material, when the microplate 2 is imaged by the camera 4, the background becomes black, and the contrast of the image is increased. Accuracy can be improved.

  A jig 8 for positioning the microplate 2 may be installed on the stage 7. The jig 8 is a bar-shaped or wall-shaped member having an L shape, for example, and is fixed to the stage 7. By bringing two orthogonal side surfaces of the microplate 2 into contact with the jig 8, the microplate 2 on the stage 7 is determined. By fixing the positions of the camera 4 and the illumination device 5 with respect to the jig 8, the microplate 2, the camera 4, and the illumination device can be simply mounted on the stage 7 according to the jig 8. 5 can be maintained in an appropriate state. In FIG. 1, the illumination device 5 is placed on the stage 7, but the illumination device 5 may be installed outside the stage 7.

  FIG. 4 is a block diagram showing the main configuration of the information processing apparatus 6. The information processing apparatus 6 is configured by, for example, a personal computer (PC), a notebook PC, or the like, and executes various programs to support dispensing work for the microplate 2 in cooperation with the camera 4 by executing a predetermined program. Realize the function.

  As illustrated in FIG. 4, the information processing apparatus 6 includes an image acquisition unit 61, a display unit 62, an input unit 63, a communication unit 64, a storage unit 65, and a calculation unit 66. The image acquisition unit 61 is an external interface that inputs and outputs image data and various signals to and from various external devices such as the camera 4. Specifically, a USB interface and the like are included.

  The display unit 62 is, for example, a liquid crystal display or an organic EL display, and displays a predetermined screen under the control of the calculation unit 66.

  The input unit 63 includes an input device such as a keyboard, a mouse, and a touch panel, and inputs information corresponding to an operation performed by the user to the calculation unit 66.

  The communication unit 64 is a communication interface that connects the information processing apparatus 6 to a communication network and transmits / receives information to / from other devices via the communication network. Specifically, an optical modem, a soft modem, a cable modem, etc. are included. The communication network may be, for example, any one of WiFi (Wireless Fidelity), Bluetooth (registered trademark), the Internet, a LAN, a telephone line, and the like.

  The storage unit 65 is configured using a computer-readable storage medium such as a semiconductor memory such as a disk drive, ROM, or RAM. In addition to the operating system program and the driver program, the storage unit 65 stores a program for causing the information processing apparatus 6 to execute a predetermined operation, various data used during the execution of the program, setting information, and the like. Specifically, the storage unit 65 includes a program storage unit 651, a master information storage unit 652, an image storage unit 653, and a dispensing setting information storage unit 654. Among these, the program storage unit 651 stores a well address acquisition program to be executed by the calculation unit 66.

  The master information storage unit 652 stores product information of the microplate 2 used in the dispensing operation and a master image that is an image schematically representing the arrangement surface 2b of the microplate 2. FIG. 5 is a partially enlarged view of the arrangement surface 2 b of the microplate 2. FIG. 6 is a schematic view illustrating a master image.

As shown in FIG. 5, the product information includes the well 2a (for example, A) closest to the origin when a predetermined point on the arrangement surface 2b (for example, the angle at which the left side and the upper side intersect) is the origin (0, 0). Microplate including center coordinates (X _A1 , Y _A1 ), well 2a pitch (ΔX, ΔY), well 2a diameter φ, number of wells 2a (n _X columns, n _Y rows) 2 includes information on the size of each part. Such product information may be input from the input unit 63 every time a dispensing operation is performed, or may be stored in advance in the master information storage unit 652 and read when the dispensing operation is performed. In the present embodiment, the former case will be described.

As shown in FIG. 6, the master image M1 is obtained by arranging well regions R _A1 , R _A2 ,... Representing the well 2a in a rectangular region representing the outline of the arrangement surface 2b based on the product information described above. is there. Such a master image M1 may be created on the basis of product information each time a dispensing operation is performed, or a master image storage unit 652 stores a pre-created image for each manufacturer and product number of a microplate. You can leave it.

  The image storage unit 653 stores an image generated by the image processing unit 662 described later based on the image data output from the camera 4.

  The dispensing setting information storage unit 654 stores setting information (dispensing setting information) for performing dispensing work. The dispensing setting information includes information such as the address and order of the wells 2a to be dispensed, the type of liquid dispensed to each well 2a, and the amount dispensed to each well 2a.

  The calculation unit 66 is configured by using hardware such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and reads and executes a program stored in the program storage unit 651, whereby the information processing apparatus 6 The data transfer to each unit and instructions are performed, and the operation of the information processing apparatus 6 is comprehensively controlled. In addition, the calculation unit 66 generates an image based on the image data acquired from the camera 4 by executing the well address acquisition program, and the well 2a in which the tip of the tip 3a of the pipetting pipette 3 is located from the image. Execute the calculation process to get the address. Specifically, the functional units realized by the calculation unit 66 executing the well address acquisition program include a master image acquisition unit 661, an image processing unit 662, an address acquisition unit 663, a dispensing instruction unit 664, and display control. A portion 665 is included.

  The master image acquisition unit 661 registers product information of the microplate 2 used for dispensing work based on information input from the input unit 63, and acquires a master image based on the product information.

  The image processing unit 662 generates an image by performing predetermined image processing such as demosaicing, white balance processing, and gamma correction on the image data output from the camera 4 and acquired by the image acquisition unit 61. The image processing unit 662 includes a feature point acquisition unit 662a, a mapping processing unit 662b, and a tip detection unit 662c, and performs predetermined processing using the generated image.

  The feature point acquisition unit 662a includes a plurality of images in the array surface 2b in an image generated before the start of the dispensing operation (that is, an image captured when the dispensing pipette 3 is not within the field of view of the camera 4). Get feature points.

  Based on the plurality of feature points, the mapping processing unit 662b deforms the master image M1 according to the image of the array surface 2b in the image, and the deformed master image is a pixel corresponding to the light receiving surface of the image sensor 4a. By mapping to the region, the well region in the master image M1 is associated with the position information of the pixel in the pixel region.

  The tip detector 662c is also referred to as the tip of the image of the tip 3a in the image (hereinafter also abbreviated to the tip of the tip) from the image generated during the dispensing operation (ie, the image showing the arrangement surface 2b and the dispensing pipette 3). ) Position (tip position).

  The address acquisition unit 663 uses the tip of the chip 3a in the actual microplate 2 based on the position of the tip of the chip detected by the tip detection unit 662c and the position information of the pixels in the pixel region associated with each well region. The address of the well 2a in which is located is acquired.

  The dispensing instruction unit 664 generates instruction information for instructing the operator to dispense the microplate 2 based on the dispensing setting information stored in the dispensing setting information storage unit 654.

  The display control unit 665 generates a screen to be displayed on the display unit 62 in a series of operations of the well address acquisition system 1.

  Next, the operation of the well address acquisition system 1 will be described. FIG. 7 is a flowchart showing the operation of the well address acquisition system 1. First, as preparation for the dispensing operation, the operator places the microplate 2 to be dispensed on the stage 7. At this time, as shown in FIG. 1, positioning is performed by bringing the left and upper side surfaces of the microplate 2 into contact with the jig 8. When the jig 8 is not installed on the stage 7, the position of the microplate 2 is fixed or the microplate 2 is mounted so that the position of the microplate 2 is not shifted during the dispensing operation. It is good to mark the place where it was placed.

  In step S <b> 10, the calculation unit 66 registers the microplate 2. FIG. 8 is a flowchart showing a microplate registration process.

In step S100, the display control unit 665 causes the display unit 62 to display a screen for inputting the product information of the microplate 2. FIG. 9 is a schematic view illustrating an input screen for product information on a microplate. The input screen P1 shown in FIG. 9 includes an input field p11 for the center coordinates (X _A1 , Y _A1 ) of the well (well A1) in the A row and the first column viewed from the origin of the microplate 2 (see FIG. 5), and the well 2a. Input column p12 for the pitch (column pitch ΔX, row pitch ΔY), an input column p13 for the number of wells 2a (number of columns and rows), an input column p14 for the well diameter φ, and an OK button p15. . The operator inputs product information of the microplate 2 to be dispensed into these input fields p11 to p14, and further performs a predetermined operation (for example, a click operation with a mouse) on the OK button p15. When the input unit 63 detects an operation on the OK button p15, the input unit 63 inputs the information input in the input fields p11 to p14 to the master image acquisition unit 661.

  In step S <b> 101, the master image acquisition unit 661 registers the information input from the input unit 63 as product information of the microplate 2.

In subsequent step S102, the master image acquisition unit 661 acquires a master image based on the product information registered in step S101. Specifically, in the master image M1 shown in FIG. 6, the well regions R _A1 , R _A2 ,... Are rearranged based on the origin based on the product information, and the diameter φ ′ of each well region is adjusted.

  In subsequent step S <b> 103, the information processing apparatus 6 causes the camera 4 to image the microplate 2 and generates an image based on image data (first image data) output from the camera 4. At this time, since the dispensing operation has not yet started, the dispensing pipette 3 is not shown in the image, and an image showing the arrangement surface 2b of the microplate 2 without being missing is obtained. The display control unit 665 displays this image on the display unit 62. Note that the image acquired at this time may be a still image. Hereinafter, the image acquired in the registration process of the microplate 2 is referred to as a reference image.

  FIG. 10 is a schematic diagram illustrating a reference image. Here, as shown in FIG. 1, in this embodiment, since the microplate 2 is imaged by the camera 4 from an oblique direction, the reference image M2 shown in FIG. The array of well images m21 is distorted in a trapezoidal shape, and the image m21 of each well is flattened. An image generated by imaging with the camera 4 including the reference image M2 has an image size corresponding to the pixel region I1 corresponding to the light receiving surface of the imaging element 4a.

In subsequent step S104, the feature point acquisition unit 662a acquires the position of the specified feature point by causing the operator to specify the feature point with respect to the reference image M2 displayed on the display unit 62. Here, the feature points are points where corresponding images are captured between the master image M1 and the reference image M2, and at least four locations are designated. In the present embodiment, the center points of the well images C _A1 , C _A12 , C _H1 , and C _H12 arranged at the four corners of the plurality of well images m21 shown in FIG. The feature point is specified by, for example, a pointer operation using a mouse for the reference image M2. The feature point acquisition unit 662a performs the coordinates (x _A1 , y _A1 ), (x _A12 , y _A12 ), (x _H1 , y _H1 ), (x) of the specified feature points (the center points of the four corner well images). _xH12 , _yH12 ) are recorded.

  The feature points are not limited to the center points of the well images arranged at the four corners. For example, it may be four corners or four sides of the outline of the arrangement surface 2b, or a marker previously attached to the arrangement surface 2b. In short, it is only necessary that the positions where the corresponding images are captured can be linked between the reference image M2 generated by imaging the array surface 2b with the camera 4 and the master image M1.

In subsequent step S105, the mapping processing unit 662b associates the plurality of feature points acquired in step S104 with a plurality of points on the master image M1 in which images corresponding to these feature points are captured, respectively. Perform image conversion. Specifically, the center points of the well regions R _A1 , R _A12 , R _H1 , and R _{H12 in} the master image M1 are the center points of the well images C _A1 , C _A12 , C _H1 , and C _{H12 in} the reference image M2. And affine transformation is applied to the master image M1.

  FIG. 11 is a schematic view illustrating a master image that has been subjected to image conversion. The master image M1 'shown in FIG. 11 has a distorted shape, similar to the reference image M2 shown in FIG.

In subsequent step S106, the mapping processing unit 662b maps the master image M1 ′ subjected to the image conversion to the pixel region I1. FIG. 12 is a schematic diagram for explaining the master image mapping process. As shown in FIG. 12, by mapping the master image M1 ′ to the pixel region I1, each well region R _A1 , R _A1 ,... In the master image M1 ′ is associated with the position information of the pixels in the pixel region I1. It is done. In other words, it is specified to which of the well regions R _A1 , R _A1 ,... Each pixel in the pixel region I1 belongs.

  Thereafter, the process returns to the main routine (see FIG. 7), and the dispensing operation from the dispensing pipette 3 to the microplate 2 is started.

  In step S11 subsequent to step S10, the information processing apparatus 6 causes the camera 4 to start imaging the microplate 2, and generates an image based on image data (second image data) sequentially output from the camera 4. Thereby, images in which the arrangement surface 2b of the microplate 2 and the dispensing pipette 3 are shown are sequentially obtained. Hereinafter, an image acquired during the dispensing operation is referred to as a target image. The imaging frame rate of the camera 4 during the dispensing operation is not particularly limited. However, considering a response and a processing load in a well address acquisition process (see step S14) to be described later, as an example, the imaging frame rate is approximately 15 fps to 60 fps. It is preferable. FIG. 13 is a schematic diagram illustrating a target image.

  In step S12, the tip detection unit 662c acquires the position (tip position) of the tip of the image of the chip 3a in each target image for the target images sequentially generated in step S11.

  Specifically, first, a difference image is generated by subtracting the reference image M2 shown in FIG. 10 from the target image M3 shown in FIG. FIG. 14 is a schematic diagram illustrating a difference image. Here, only the presence or absence of the pipette pipette image m30 differs between the target image M3 and the reference image M2. Therefore, only the pipetting pipette image m30 remains in the difference image M4.

  An image m30 of the dispensing pipette is extracted from the difference image M4, and the tip of this image m30, that is, the tip tip m31 is detected. Various known methods can be used as a method of detecting the tip end m31. As an example, a binarization process is performed on the difference image M4 to extract a region of the pipette pipette image m30 and track the contour of this region. A portion having the largest curvature in the contour (a portion having the smallest curvature radius) is defined as a tip end m31. As another example, the center of gravity and the main axis of the region extracted by the binarization process may be detected, and the end farther from the center of gravity of both ends of the main axis may be used as the tip end m31. The tip detection unit 662c acquires the coordinates (x, y) in the pixel region I1 of the tip tip m31 detected in this way.

  In subsequent step S13, the dispensing instruction unit 664 acquires the dispensing setting information from the dispensing setting information storage unit 654, and instructs the well 2a to be dispensed. FIG. 15 is a schematic view illustrating a dispensing support screen on which instructions for the wells 2a to be dispensed are displayed. The dispensing support screen P2 shown in FIG. 15 includes an examination information display field p21 in which information such as the date and time of examination, the name of the sample, the name of the reagent, the product number of the microplate 2 and the product number of the dispensing pipette 3 are displayed. , A well display column p22 schematically representing the arrangement surface 2b of the microplate 2, a legend display column p23, and a message display column p24. In the well display column p22, a plurality of circle marks (hereinafter referred to as well marks) p25 arranged in the same manner as the well 2a on the arrangement surface 2b are arranged. Each well mark p25 is displayed with a color or shading according to the state of the well 2a. The state of the well 2a includes a state in which dispensing is not performed, a state in which dispensing has already been performed, a state in which dispensing is awaited, and a state in which the tip of the tip is inserted.

  The dispensing instruction unit 664 instructs dispensing by displaying the well mark p25 corresponding to the well 2a waiting to be dispensed in a predetermined color or shade (see the legend display column p23). The operator moves the tip of the tip 3a of the dispensing pipette 3 to the well 2a instructed on the dispensing support screen P2.

  In subsequent step S14, the address acquisition unit 663 acquires the well address of the well 2a where the tip of the chip 3a is currently located. FIG. 16 is a flowchart showing well address acquisition processing. FIG. 17 is a schematic diagram for explaining a well address acquisition process. FIG. 18 is a schematic diagram for explaining a well address determination process.

First, in step S141, the address acquisition unit 663 determines that the tip end m31 is one of a plurality of well regions R _A1 , R _A2 ,... Based on the position of the tip end m31 acquired in step S12 (see FIG. 14). It is determined whether it is included in the inside. Specifically, it is determined whether or not the coordinates (x, y) of the tip end m31 match any of the coordinates of the pixels in the pixel area I1 associated with each of the well areas R _A1 , R _A2 ,. To do.

When the tip end m31 is included in the well region (step S141: Yes), the address acquisition unit 663 subsequently extracts the well region including the tip end m31 (step S142). For example, in FIG. 17, the well region R _F4 is extracted.

  On the other hand, when the tip end m31 is not included in any of the well regions (step S141: No), the address acquisition unit 663 repeatedly performs the determination on the sequentially generated target images (step S141).

  In step S143 following step S142, the address acquisition unit 663 determines whether or not a predetermined time has elapsed with the tip end m31 remaining in the same well region (step S143).

  Here, as shown in FIG. 18, when the array surface 2b of the microplate 2 is imaged by the camera 4, a two-dimensional image is actually obtained even when the tip portion of the chip 3a is floating from the array surface 2b. In FIG. 2, since the chip 3a appears to overlap the array surface 2b, there is a possibility that the tip of the chip 3a is erroneously detected as being in any one of the wells 2a. Therefore, in the present embodiment, it is determined whether or not the tip end m31 corresponding to the tip of the tip 3a remains in the same well region for a predetermined time (for example, 1 second) or longer. When the operator puts the tip of the tip 3a in the well 2a to be dispensed, the operator should hold the dispensing pipette 3 in that state for a while, and when the tip of the tip 3a is floating This is because the dispensing pipette 3 should be moved toward the well 2a to be dispensed. What is necessary is just to set suitably the time determined in step S143 by a worker's liking.

  If the tip end m31 has moved from the well region extracted in step S142 before the predetermined time has elapsed (step S143: No), the process returns to step S141. On the other hand, when the predetermined time has passed with the tip end m31 remaining in the well region (step S143: Yes), the address of the well region is determined as the address of the well 2a where the tip portion of the chip 3a is located. (Step S144). Thereafter, the process returns to the main routine (see FIG. 7).

  In step S15 subsequent to step S14, the dispensing instruction unit 664 determines whether or not the address acquired in step S14 matches the address of the well 2a instructed to dispense in step S13.

  If the addresses do not match (step S15: No), the dispensing instruction unit 664 issues a warning. Specifically, as illustrated in FIG. 19, in the well display field p22 of the dispensing support screen P2, the well mark p25 corresponding to the address acquired in step S14 is displayed in a predetermined color or shade (legend display field p23). And a warning that the position of the chip 2a is incorrect is displayed in the message display field p24 (step S16). Alternatively, a warning sound may be generated or a warning message may be read out.

  On the other hand, when the addresses match (step S15: Yes), the dispensing instruction unit 664 permits dispensing (step S17). Specifically, as illustrated in FIG. 20, in the well display column p22 of the dispensing support screen P2, the well mark p25 corresponding to the address acquired in step S14 is displayed in a predetermined color or shade (legend display column p23). In addition, a message indicating that the tip position is correct and a message prompting dispensing are displayed in the message display field p24 (step S17). Thereafter, the dispensing instruction unit 664 changes the color or shading of the well mark p25 to indicate that dispensing has been completed.

  When a dispensing pipette that can communicate with the information processing device 6 is used, the dispensing instruction unit 664 receives the detection signal for detecting the driving of the electric motor from the dispensing pipette, The shading may be changed to indicate that dispensing has been completed.

  In subsequent step S18, the dispensing instruction unit 664 determines whether or not dispensing to all the wells 2a set in the dispensing setting information has been completed. If dispensing has not been completed (step S18: No), the process returns to step S13. On the other hand, when the dispensing is finished (step S18: Yes), the operation of the well address acquisition system 1 is finished.

  As described above, according to the present embodiment, it is possible to specify the address of a well that an operator is to dispense in a microplate in which a plurality of wells are arranged. Therefore, the operator can easily determine whether or not the well is a correct well to be dispensed by confirming the address of the identified well. Accordingly, it is possible to reduce an artificial mistake in the dispensing work, to efficiently advance the dispensing work, and to suppress worker fatigue.

  In addition, according to the present embodiment, the well address acquisition system 1 can be configured in a compact manner using the camera 4 and the information processing device 6 and preferably the illumination device 5. Therefore, the operator can perform a dispensing operation using the general-purpose microplate 2 and the dispensing pipette 3 with the same feeling as before.

  Moreover, according to this embodiment, since the well address acquisition system 1 is configured by a combination of general-purpose hardware such as the camera 4, the illumination device 5, and the information processing device 6, an operator can easily and inexpensively configure the system. Can be used.

(Modification)
Next, a modification of the first embodiment of the present invention will be described.
FIG. 21 is a schematic diagram for explaining a modification of the microplate product information registration process (see step S101 in FIG. 8). In the first embodiment, when registering product information, the operator inputs each numerical value on the product information input screen (see FIG. 9). (See FIG. 4).

  Specifically, as shown in FIG. 21, a product information database DB is created for each microplate product and stored in the storage unit 65. The product information database DB includes product identification information (product ID), manufacturer name, product number, number of wells, and center of well (well A1) in row A and column 1 as seen from the origin of the microplate (see FIG. 5). It includes information such as coordinates, well pitch, and well diameter.

  In this case, by causing the operator to input the product ID or manufacturer and product number from the input unit 63, the master image acquisition unit 661 extracts the information of the corresponding product from the product information database, and various information regarding the product is obtained. Automatic input is made on the input screen shown in FIG.

  Alternatively, a master image may be created in advance for each product for which information is stored in the product information database DB and stored in the storage unit 65. In this case, the master image acquisition unit 661 reads out the master image corresponding to the corresponding product from the storage unit 65 by allowing the worker to input the product ID or manufacturer and product number.

  FIG. 22 is a schematic diagram for explaining a first modification of the feature point acquisition process (see step S104 in FIG. 8). In the first embodiment, when acquiring the feature point, the operator manually designates the center points of the four wells for the reference image M2 displayed on the display unit 62. The feature point acquisition unit 662a may automatically extract feature points.

  Specifically, as shown in FIG. 22, the feature point acquisition unit 662a detects four corners m51 of the outline of the array surface 2b with respect to the reference image M5 generated by causing the camera 4 to pick up an image of the microplate 2. To do. Various known methods can be used as a method of detecting the angle m51. As an example, a straight line is detected from the reference image M5 by the Hough transform, and four intersections close to the center of the reference image M5 are selected from the intersections of the straight lines. In this case, the mapping processing unit 662b performs image conversion by associating the four corners of the master image M1 (see FIG. 6) with the four corners m51 detected from the reference image M5.

  23 and 24 are schematic diagrams for explaining a second modification of the feature point acquisition process. When acquiring a feature point, it is good also as detecting automatically the marker previously attached to the microplate.

  Specifically, as shown in FIG. 23, markers 2c having a predetermined color are attached in advance to predetermined four locations on the arrangement surface of the microplate 2 '. As a result, as shown in FIG. 24, the image m61 of the marker 2c is reflected in the reference image M6 generated by causing the camera 4 to pick up an image of the microplate 2 '. An image m61 of the marker 2c is detected from this reference image M6. As a detection method of the image m61, for example, various known methods such as detecting a region having the same color feature amount as the marker 2c can be used. In this case, a marker is displayed at a position corresponding to the marker 2c in the master image, and the mapping processing unit 662b performs image conversion by associating the marker in the master image with the image m61 in the reference image M6.

  Here, in FIG. 23, the marker 2c is attached in the vicinity of the four corners of the arrangement surface, but the position of the marker 2c is not limited to this. In short, it is only necessary to perform image conversion by associating positions where corresponding images are captured between the reference image M6 and the master image.

  In this modification, since the marker is attached to the microplate in advance, it is possible to improve the accuracy of mapping of the master image and thus the well address acquisition accuracy. That is, when a microplate with a marker is used in the well address acquisition system 1, the address acquisition accuracy can be ensured. That is, the marker attached to the microplate can be used as a precision guarantee.

  FIG. 25 is a schematic diagram for explaining a first modification of the tip tip position acquisition process (see step S12 in FIG. 7). In the first embodiment, a difference image (see FIG. 14) obtained by subtracting the reference image from the target image is generated, and the tip end m31 is detected from the pipette pipette image m30 shown in the difference image. However, when the chip is formed of a transparent or white translucent member, as shown in FIG. 25, the chip portion in the difference image M4 'may not be clearly visible.

  In such a case, the image m32 of the pipette body is extracted from the difference image M4 ', and the contour of the image m33 of the nozzle (the part into which the tip is fitted) on the tip side of the pipette body is extracted. A point where the extended line m34 of the contour intersects is estimated as the position of the tip end m35.

  FIG. 26 and FIG. 27 are schematic diagrams for explaining a second modification of the processing for obtaining the position of the tip of the chip. As described above, when the tip of the pipetting pipette is formed of a transparent or white translucent member and the tip portion is not clearly visible in the image, the tip position of the tip in the target image may be obtained by calculation.

  First, as shown in FIG. 26, a distance L1 from the base end of the pipette body 3b to the tip of the tip 3a is measured in advance. Then, as shown in FIG. 27, a difference image M7 obtained by subtracting the reference image from the target image is generated, an image m70 of the pipette body 3b is extracted from the difference image M7, and a base end portion m71 of the image m70 is detected. The point extended from the base end m71 by the distance L1 'on the difference image M7 corresponding to the distance L1 is set as the position of the tip end m72. The distance L1 'may be determined in consideration of the inclination of the dispensing pipette 3 in addition to the scale of the image. Alternatively, a marker serving as an index of the distance L1 may be attached to the pipette body 3b side, and the distance L1 'may be determined based on the length of the marker shown in the difference image M7.

  FIG. 28 and FIG. 29 are schematic diagrams for explaining a third modification of the tip tip position acquisition process. As described above, when the tip of the pipetting pipette is formed of a transparent or white translucent member and the tip portion is not clearly visible in the image, a means for indicating the tip position may be provided on the tip side. good.

  For example, as shown in FIG. 28, a step 3d is formed in a part of a tip 3c attached to the pipette body 3b, and a distance L2 from the step 3d to the tip of the tip 3c is measured in advance. Then, an image of the step 3d is detected from the difference image obtained by subtracting the reference image from the target image, and the point extended from the step 3d by the distance on the difference image corresponding to the distance L2 is set as the tip end position.

  Alternatively, as shown in FIG. 29, a color marker 3f may be attached to a part of the chip 3e. In this case, the distance L3 from the marker 3f to the tip of the chip 3e is measured in advance. Then, the image of the marker 3f is detected from the difference image obtained by subtracting the reference image from the target image, and the point extended from the image of the marker 3f by the distance on the difference image corresponding to the distance L3 is set as the tip end position. . In addition, when attaching the color marker 3f, you may detect the image of the marker 3f directly from a target image.

  As described above, when the tip itself is provided with means (steps or markers) that serve as a guide for the tip position, the detection accuracy of the tip tip position can be improved. In other words, the use accuracy of the well address can be ensured by using a chip provided with a step or a marker. That is, it is possible to use a step or marker provided on the chip as a guarantee of accuracy.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. FIG. 30 is a schematic diagram showing a schematic configuration of a well address acquisition system according to the present embodiment. FIG. 31 is a schematic view illustrating a target image generated in the present embodiment.

  A well address acquisition system 9 shown in FIG. 30 includes an illumination device 10 instead of the illumination device 5 in the well address acquisition system 1 shown in FIG. The configuration of each part of the well address acquisition system 9 other than the illumination device 10 is the same as that of the first embodiment.

  The illuminating device 10 has a slit parallel to the array surface 2b of the microplate 2, and emits slit light 10a traveling in an optical path parallel to the array surface 2b so as to cover the array surface 2b. .

  The operation of the well address acquisition system 9 is the same as that of the first embodiment as a whole, and only the tip tip position acquisition process (step S12 in FIG. 7) is different from that of the first embodiment. In the well address acquisition system 9, when the tip 3a of the dispensing pipette 3 is inserted into the well 2a, a part of the slit light 10a is reflected at the point where the tip 3a crosses the slit light 10a, and this point emits light. When this state is imaged by the camera 4, a state in which the tip of the image m80 of the chip 3a emits light can be captured like a target image M8 shown in FIG. Therefore, the position of the tip of the chip 3a can be acquired by detecting the light emission point m81 from the target image M8. The light emission point m81 can be detected by extracting a region having a color feature amount corresponding to the emission color (for example, red) of the slit light 10a.

  According to the present embodiment, the position of the tip portion of the chip can be directly detected from the target image, so that it is possible to reduce the calculation load for image processing. In addition, according to the present embodiment, even when a transparent or white translucent chip is used, the tip of the chip can be detected with high accuracy.

  The present invention is not limited to the first and second embodiments and modifications described above, and various inventions can be made by appropriately combining a plurality of components disclosed in the embodiments and modifications. Can be formed. For example, some constituent elements may be excluded from all the constituent elements shown in the respective embodiments and modifications, or constituent elements shown in different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1, 9 Well address acquisition system 2, 2 'Microplate 2a Well 2b Arrangement surface 2c Marker 3 Dispensing pipette 3a, 3c Tip 3b Pipette main body 3d Step 3e Tip 3f Marker 4 Camera 4a Imaging element 5, 10 Illuminating device 5a Light emitting surface 6 Information processing device 7 Stage 8 Jig 10a Slit light 61 Image acquisition unit 62 Display unit 63 Input unit 64 Communication unit 65 Storage unit 66 Calculation unit 651 Program storage unit 652 Master information storage unit 653 Image storage unit 654 Dispensing setting information storage Unit 661 Master image acquisition unit 662 Image processing unit 662a Feature point acquisition unit 662b Mapping processing unit 662c Tip detection unit 663 Address acquisition unit 664 Dispensing instruction unit 665 Display control unit

Claims (10)

An imaging device that has an imaging device that generates an electrical signal by photoelectrically converting light incident on the light receiving surface, and that generates and outputs image data based on the electrical signal;
An information processing device for generating an image based on image data output from the imaging device;
The information processing apparatus comprises:
The first image data generated by causing the imaging device to image the array surface of a plurality of wells included in the microplate, and the imaging of the array surface in a state of being dispensed to the microplate by a dispensing pipette An image acquisition unit that acquires second image data generated by causing the device to capture an image;
A master image representing an image of the array plane, and a master image including a plurality of well regions that are regions respectively corresponding to the plurality of wells, is a reference that is an image generated based on the first image data A mapping processing unit that associates each well region with the position information of the pixel in the pixel region by deforming according to the image and mapping the pixel region to the pixel region corresponding to the light receiving surface,
A tip detection unit that detects a tip position of the image of the chip in the target image from a target image that is an image generated based on the second image data;
Extracting the well region to which the tip position belongs based on the tip position in the target image and pixel position information associated with each well region, thereby leading the tip of the chip among the plurality of wells An address acquisition unit for acquiring an address of a well in which
Well address acquisition system having. The information processing apparatus further includes a feature point acquisition unit that acquires a plurality of feature points in the image of the array plane in the reference image,
The well according to claim 1, wherein the mapping processing unit performs image conversion by associating a plurality of points in the master image in which images corresponding to the plurality of feature points are represented with the plurality of feature points, respectively. Address acquisition system.   The well address acquisition system according to claim 1 or 2, wherein the tip detection unit generates a difference image between the reference image and the target image, and detects the tip position from the difference image.   The well address acquisition system according to claim 1, further comprising a surface light source that illuminates at least the array surface. A slit parallel to the array surface of the microplate is provided, further comprising a slit light source that emits light traveling along an optical path covering the array surface,
The said front-end | tip detection part detects the image of the light emission point light-emitted when the said chip crosses the said light from the said target image, and detects the position of this image as the said front-end | tip position. Well address acquisition system.   The address acquisition unit extracts a well region associated with a pixel in the pixel region corresponding to the tip position, and acquires an address of a well corresponding to the extracted well region. The well address acquisition system according to any one of the above items.   The well address acquisition system according to claim 6, wherein the address acquisition unit acquires an address of a well corresponding to the same well region when the tip position remains in the same well region for a predetermined time or more. .   The well address acquisition system according to claim 1, further comprising a jig that determines a position of the microplate with respect to the imaging device. An imaging device that generates an electrical signal by photoelectrically converting light incident on the light receiving surface, generates and outputs image data based on the electrical signal, and image data output from the imaging device In a well address acquisition system comprising an information processing device that generates an image based on the well address acquisition method executed by the information processing device,
A step (a) of acquiring first image data generated by causing the imaging device to image an array surface of a plurality of wells included in the microplate;
A master image representing an image of the array plane, and a master image including a plurality of well regions that are regions respectively corresponding to the plurality of wells, is a reference that is an image generated based on the first image data (B) associating each well region with position information of a pixel in the pixel region by deforming it in accordance with an image and mapping it to a pixel region corresponding to the light receiving surface;
(C) obtaining second image data generated by causing the imaging device to image the array surface in a state where the microplate is being dispensed by a dispensing pipette;
Detecting a tip position of an image of the chip in the target image from a target image that is an image generated based on the second image data; and
Extracting the well region to which the tip position belongs based on the tip position in the target image and pixel position information associated with each well region, thereby leading the tip of the chip among the plurality of wells Obtaining the address of the well where is located, (e),
Well address acquisition method including: An imaging device that generates an electrical signal by photoelectrically converting light incident on the light receiving surface, generates and outputs image data based on the electrical signal, and image data output from the imaging device In a well address acquisition system comprising: an information processing device that generates an image based on the program;
A step (a) of acquiring first image data generated by causing the imaging device to image an array surface of a plurality of wells included in the microplate;
A master image representing an image of the array plane, and a master image including a plurality of well regions that are regions respectively corresponding to the plurality of wells, is a reference that is an image generated based on the first image data (B) associating each well region with position information of a pixel in the pixel region by deforming it in accordance with an image and mapping it to a pixel region corresponding to the light receiving surface;
(C) obtaining second image data generated by causing the imaging device to image the array surface in a state where the microplate is being dispensed by a dispensing pipette;
Detecting a tip position of an image of the chip in the target image from a target image that is an image generated based on the second image data; and
Extracting the well region to which the tip position belongs based on the tip position in the target image and pixel position information associated with each well region, thereby leading the tip of the chip among the plurality of wells Obtaining the address of the well where is located, (e),
A program that executes

Images