JP2010281800A - Cell analyzer and method of analyzing cell - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for obtaining a situation in a cell (electron microscopy expressed by an entire biomatter) under diversified environment (morbid state) conditions by suppressing destruction of cells when producing observation samples and for correlating position information of specific substances in the same visual field. <P>SOLUTION: An analyzer of cells includes: a storage section for storing a first image including information on cell structure of the cells, and a second image that includes the first image and includes information on the inside of the cells and information on a substance related to the cell structure of the cells; a position identifying section for extracting the cell structure from the first image, extracting cell structure related to the substance from the second image, comparing each extracted cell structure, and identifying to which position of the first image the second image corresponds; and a coupling section for overlapping the second image to the identified position of the first image, based on the identification result by the position identifying section. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a cell analysis device and a cell analysis method.

  One of the daily researches to advance medicine is to observe specific proteins in cells. As a merit of this research, it is possible to observe how specific proteins are localized in cells that are abnormal when they become ill, and when different types of proteins are adjacent to each other, It is possible to observe what kind of effect it has. By proceeding with the above research, it is possible to understand the state of cells in more detail and to elucidate various symptoms appearing in the human body. As a method for observing a specific protein in a cell, a fluorescent antibody staining method in which a fluorescent dye is labeled on an antibody and observed with a fluorescence microscope is known (see, for example, Patent Document 1).

International Publication No. 2005/035751 Pamphlet

  Research has been conducted to observe specific proteins in cells. The merit of this research is that it is possible to observe the localization of specific proteins in cells during disease. Therefore, the above research is medically expected to be able to understand the state of cells in more detail and to elucidate various symptoms appearing in the human body.

  As a conventional observation method, a fluorescent antibody staining method in which a fluorescent dye is labeled on an antibody and observed with a fluorescence microscope is known. With the recent improvement in the performance of microscopes, it has become possible to estimate the approximate position of the labeled protein in the intracellular microstructure. However, the method of observing with a fluorescence microscope cannot obtain the exact position of the labeled protein in the cell.

  On the other hand, a protein position analysis method using an electron microscope has been established as an immunoelectron microscopic method using a colloidal gold secondary antibody. However, in this method, there is a concern that the observation image may have lost the fine structure due to the remarkable destruction of the cell membrane during the preparation of the observation sample.

If the image obtained by the fluorescence microscope (light microscopic image) and the image obtained by the electron microscope (electron microscopic image) can be overlaid, the positional information of the substance in the cell can be obtained, which is considered useful for research and diagnosis. . However, both images have extremely different observable magnification and resolution (the field of view of about several percent of the light microscope image corresponds to one image of the electron microscope image). In addition to making mistakes, the calculation of the rotation angle is an iterative fitting operation, and the process of accurately superimposing the light microscope image and the electron microscope image is a very troublesome and difficult task.
In view of the background described above, by solving the problem of the overlaying process, the present invention suppresses cell destruction during preparation of the observation sample, while suppressing intracellular conditions (total biological materials) under various environmental (pathological conditions) conditions. It is an object to provide a cell analysis method that correlates position information of a specific substance in the same field of view.

  In the present invention, in order to solve the above-described problems, development of an image analysis program for automating the superimposing process of images including information with different fields of view has been started. A first image (e.g., an electron microscope image) having information on cell structure expressed in all biological materials, and information on a specific substance including the first image and associated with the cell structure of the cell The second image (for example, an optical microscope image) was overlapped. By observing the two images in a superimposed manner, a more accurate position of the substance in the cell can be obtained. In addition, analysis can be performed without damaging the cell membrane as in immunoelectron microscopy using a colloidal gold secondary antibody.

  More specifically, the present invention is an analysis device for specifying the position of a substance in a cell, the first image having information on a plurality of substances contained in the cell and information on the cell structure of the cell, A storage unit that stores the second image that includes the first image and has information on a substance among substances contained in the cell and has information related to the cell structure of the cell. And extracting a cell structure associated with a substance contained in the second image from both the first image and the second image, collating each extracted cell structure, and the second image is the first image. A position specifying unit that specifies which position of one image corresponds to, and a combining unit that superimposes the second image on the specified position of the first image based on a specifying result by the position specifying unit. .

  In the present invention, by superimposing images having different properties, the position of the substance in the cell can be obtained more accurately without damaging the cell. One of the images having different properties is a first image having information on the cell structure of the cell. The first image preferably includes all information of the cell including information on a plurality of substances contained in the cell and information on the cell structure. On the other hand, an image having a property different from that of the first image is an image that includes the first image and has information on a substance associated with the cell structure of the cell. The inclusion of the first image means that the second image is captured in a wider range than the first image, and the first image is included in the second image. By superimposing images having different properties from each other in this way, it is possible to observe the trend of the substance on the first image having information on the cell structure. As a result, it is possible to obtain an accurate position of the substance.

When overlapping images having different properties, it is necessary to specify which position of the first image the second image corresponds to. When specifying to which position of the first image the second image corresponds, not only the position of the second image with respect to the first image but also the angle of the second image with respect to the first image must be specified. Such identification can be performed by humans based on experience. However, experience is a prerequisite, and even an experienced person may find it difficult to identify. In the present invention, the position specifying unit specifies which position of the first image the second image corresponds to by collating cell structures as information common to images having different properties. That is, although the first image and the second image are images having different properties, they have information regarding the cell structure (for example, the shape of the cell membrane) as common information. Therefore, the specifying unit specifies which position of the first image the second image corresponds to by comparing the cell structure as common information. Since the substance included in the second image is associated with the cell structure, the position of the substance can be specified on the first image when the coupling unit overlaps the first image and the second image. .
“Associated with a cell structure” means that a substance is localized at a site showing the outline of a cell, such as a cell membrane or a cell wall, and information on the cell structure can be obtained by imaging the substance.

  Here, in the present invention, the storage unit may store the first image as an electron microscope image and the second image as a label image that captures the substance to be labeled. The electron microscope image includes information on all substances inside and outside the cell. On the other hand, in the labeled image, for example, a specific substance can be projected by fluorescent antibody staining. Therefore, by superimposing such images, it is possible to confirm the trend of the substance (stained substance) labeled on the electron microscope image.

  In the present invention, the second image may include information on a specific protein as information on the substance. According to the present invention, the position of a specific protein in a cell can be specified more accurately.

  Further, in the present invention, the position specifying unit obtains information on the shape of the cell membrane adjacent to the protein as a cell structure related to the protein as the substance, collates the shape of each extracted cell membrane, You may make it pinpoint which position of said 1st image the said 2nd image respond | corresponds. Proteins, particularly talin, are characterized by being present adjacent to the cell membrane. In other words, the position where the protein is present is closely related to the shape of the cell membrane. Therefore, by using such a relationship, it is possible to collate two images (first image and second image) from the shape of the cell membrane.

  The position specifying unit includes a ratio of overlapping black pixels in the first image among black pixels in the second image, and a ratio of overlapping white pixels in the first image among white pixels in the second image; Calculated based on the proportion of black pixels in the second image that overlap with white pixels in the first image and the proportion of white pixels in the second image that overlap with black pixels in the first image An index may be used to determine whether or not the first image and the second image match, and the position of the first image corresponding to the second image may be specified. By using such an index, the accuracy at the time of collating the first image and the second image can be further improved.

Here, in the present invention, the storage unit is a third image that is obtained from the same sample as the second image and is associated with the second image, and is a third image that copies a specific substance different from the substance. An image is further stored, and the position specifying unit collates the cell structure extracted from the second image associated with the third image with the cell structure extracted from the first image, and The position where the three images correspond to the first image is specified, and the combining unit further sets the third image to the specified position of the first image based on the specifying result by the position specifying unit. You may make it overlap.
Examples of the specific substance include a protein whose localization in the cell is unknown.

  In the present invention, a third image different from the second image can be further superimposed. The third image is related to the second image by being obtained from the same sample as the second image. Therefore, unlike the second image, the third image does not have information on the cell structure, but is associated with the second image, so the information on the cell structure included in the second image Verification is possible. Therefore, according to the present invention, it is possible to confirm a specific substance on the first image as long as it is an image obtained from the same sample as the second image, even if the substance is not associated with the cell structure. Become. That is, the position of the specific substance in the cell can be specified. The substance and the specific substance may be displayed in different colors by, for example, staining with antibodies labeled with different fluorescent substances and changing the wavelength of the laser to be irradiated.

  Here, this invention is good also as an analysis method which has a function and an effect equivalent to the analysis apparatus mentioned above. Specifically, the present invention is an analysis method for analyzing a cell, wherein the computer includes, in a storage device, a first image having information on a cell structure of the cell, the first image, and A storage step of storing information relating to a substance contained in a cell, the second image having information relating to a substance associated with the cell structure of the cell, and extracting the cell structure from the first image, A position specifying step of extracting a cell structure associated with the substance from a second image, collating each of the extracted cell structures, and specifying which position of the first image the second image corresponds to; And a combining step of superimposing the second image on the specified position of the first image based on the specifying result of the position specifying step.

  The present invention may be a program for realizing the analysis method. Furthermore, the present invention may be a computer-readable recording medium that records such a program. In this case, the function can be provided by causing a computer or the like to read and execute the program of the recording medium. Note that a computer-readable recording medium is a recording medium that accumulates information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read from a computer or the like. Say.

  According to the present invention, it is possible to obtain intracellular conditions (electron microscopic images expressed by all biological materials) under various environmental (pathological conditions) conditions while suppressing cell destruction during observation sample preparation, It is possible to provide a technique for correlating position information of specific substances.

1 shows a schematic configuration of an analysis apparatus according to an embodiment. The functional block diagram of the analyzer which concerns on embodiment is shown. An example of an electron microscope image is shown. An example of a confocal laser microscope image is shown. An example of the confocal laser microscope image cut out is shown. An example of the image which overlap | superposed the extracted confocal laser microscope image and the electron microscope image is shown. The analysis processing flow concerning an embodiment is shown. An example of the trace line image which performed the expansion process with respect to the electron microscope image shown in FIG. 3 is shown. A confocal laser microscope image (threshold value 32) that has been binarized is shown. A confocal laser microscope image (threshold 40) subjected to binarization is shown. An example of false detection is shown. An example of a confocal laser microscope image that has been binarized and has a high index region is shown. An example of the image which traced the cell membrane is shown. An example of the image which shows the area | region compared in order to obtain a parameter | index is shown. An example of the superimposed image is shown. An example of the image which shows the location where black pixels overlapped is shown. An example of the image which shows the location where white pixels overlapped is shown. An example of the image which shows the location where the black pixel in the image of FIG. 13 and the white pixel in the image of FIG. 12 overlapped is shown. An example of the image which shows the location where the white pixel in the image of FIG. 13 and the black pixel in the image of FIG. 12 overlapped is shown. An example of a cut out specific confocal laser microscope image showing a protein different from talin is shown. An example of the image which overlap | superposed the extracted confocal laser microscope image and the electron microscope image is shown. An example of the image which overlap | superposed the extracted confocal laser microscope image, the extracted specific confocal laser microscope image, and the electron microscope image is shown. The analysis processing flow concerning a modification is shown. An example of the image which shows the specific (search) range is shown. An example of a search result is shown. An example of the searched image is shown. The relationship between an input image and an output image is shown. The index embedding (1) in the output image is shown. The embedding of the index into the result image (2) is shown. The embedding of the index into the result image (3) is shown. An example of the original image is shown. An example of the output image after conversion is shown. The result (1) of an image with a magnification of 5 is shown. The result (2) of an image with a magnification of 5 is shown. The result of an image with a magnification of 10 is shown. The result of an image with a magnification of 20 is shown.

  Next, an embodiment of an analysis apparatus according to the present invention will be described with reference to the drawings. In the following description, a case where a protein in a cell is observed will be described as an example.

<Configuration>
FIG. 1 shows a schematic configuration of an analysis apparatus 100 according to the embodiment. FIG. 2 shows a functional block diagram of the analysis apparatus 100. The analysis device 100 includes a housing 1 that stores the control unit 10, a display device 2 such as a display, an input device 3 such as a pointing device and a keyboard, and an interface 4 that can be connected to an external device.

  The display device 2 is, for example, a liquid crystal display device, a plasma display panel, a CRT (Cathode Ray Tube), an electroluminescence panel, or the like. Although not shown, the display device 2 includes a RAM that stores image data and a drive circuit that drives the display device 2 based on the data in the RAM. However, a RAM for storing image data, a drive circuit for driving the display device 2, and the like may be provided independently of the display device 2 as an image processing board.

  The input device 3 is a computer input device, and includes, for example, a keyboard and a pointing device. The type of pointing device is not particularly limited, and a mouse (trackball), a dial-type operation device, a device that moves a pointer on a display in a stick form, or a finger of a user (hereinafter also referred to as an analyst) by capacitance. A device that detects an operation, a touch panel, a joystick, or the like may be used depending on the needs of the user.

  The keyboard transmits an electrical signal corresponding to the input key to a keyboard controller (not shown) of the keyboard in response to a user input operation. The keyboard controller transmits a code corresponding to the electrical signal to the CPU 11. The device driver of the CPU 11 forms a font outline corresponding to the code based on font data (coordinates of several points for drawing the outline and a curve equation connecting them) built in the OS (True). Display on the display device 2. Further, the CPU 11 displays a character cursor indicating a character input destination on the screen in accordance with a user input operation, and moves the screen.

  The pointing device detects a user operation and transmits an operation signal to a pointing device control apparatus (not shown) (for example, a mouse controller not shown or the interface 4). Upon receiving the operation signal, the pointing device control apparatus transmits information for generating the operation direction and the operation amount to the CPU 11. The device driver of the CPU 11 displays a pointer on the screen on the display device 2 based on an operation signal from the pointing device control device, and moves the screen.

  The interface 4 may be a serial interface such as USB, or PCI (Peripheral Component Interconnect), ISA (Industry Standard Architecture), EISA (Extended ISA), ATA (AT Attachment STI, E13 Any of parallel interfaces such as Computer System Interface may be used.

  The control unit 10 includes a CPU (central processing unit) 11, a memory 12, a position specifying unit 13, a combining unit 14, an output unit 15, and a storage unit 16. The CPU 11 is connected to each hardware such as the storage unit 16 described above via the bus 17. The CPU 11 controls hardware such as the storage unit 16 and executes predetermined processing according to a control program stored in the memory 12, for example.

  The memory 12 includes a volatile RAM (Random Access Memory) and a nonvolatile ROM (Read Only Memory). The ROM includes a rewritable semiconductor memory such as a flash memory, an EPROM (Erasable Programmable Read-Only Memory), and an EEPROM (Electrically Erasable Programmable Read-Only Memory).

  The storage unit 16 can be configured by a hard disk drive (hereinafter referred to as HDD) or a rewritable semiconductor memory. The storage unit 16 stores an electron microscope image corresponding to the first image of the present invention, a marker image corresponding to the second image of the present invention, and the like.

  Here, FIG. 3 shows an example of an electron microscope image. The electron microscope image of FIG. 3 includes all cell information. That is, since an electron microscope image cannot project only a specific protein, it has all the information constituting the cell such as several types of proteins contained in the cell, cell membrane, and cell wall. On the other hand, FIG. 4 shows an example of a confocal laser microscope image. The confocal laser microscope image shown in FIG. 4 shows only a specific protein (in this embodiment, talin). That is, the confocal laser microscope image can be copied only by subjecting the protein to be observed to a special treatment (for example, fluorescent antibody staining or fluorescent labeling). Further, the confocal laser microscope image shown in FIG. 4 shows a wider range than the electron microscope image. That is, the electron microscope image of FIG. 3 is included in the confocal laser microscope image.

  The position specifying unit 13 cuts out the shape of the cell membrane associated with talin contained in the confocal laser microscope image from both the electron microscope image and the confocal laser microscope image, collates the shape of each cut out cell membrane, The position of the electron microscope image corresponding to the focused laser microscope image is specified. FIG. 5 is an example of a confocal laser microscope image cut out. The position specifying unit 13 compares the confocal laser microscope image and the electron microscope image cut out in this manner, and specifies which position of the electron microscope image the confocal laser microscope image corresponds to.

  The coupling unit 14 superimposes the confocal laser microscope image on the electron microscope image based on the identification result by the position identifying unit 13. FIG. 6 shows an example of an image obtained by superimposing the extracted confocal laser microscope image and the electron microscope image. As shown in FIG. 6, the position of talin can be specified on an electron microscope image showing all the cells. The storage unit 16 also stores the clipped image and the superimposed image. The details of the analysis processing by the analysis apparatus 100 including the processing by the position specifying unit 13 and the combining unit 14 will be described later.

  The output unit 15 outputs various images stored in the storage unit 16. Specifically, the output unit 15 causes the display device 2 to display an electron microscope image, a confocal laser microscope image, a specification result by the position specifying unit 13, an image superimposed by the combining unit 14, and the like. The output unit 15 can also output various images to the outside of the analysis apparatus 100 via the interface 4.

<Analysis processing>
FIG. 7 shows an analysis processing flow according to the embodiment. Hereinafter, a case where the internal configuration of the control unit 10 is realized by a program executed by the CPU 11 will be described as an example. However, instead of such a configuration, any one or more of the internal configurations of the control unit 10 may be realized by a processor, a hardware circuit, or the like different from the CPU 11. That is, depending on the processing capability and processing amount of the CPU 11, each component shown in FIG. 2 may be configured by a different processor or digital circuit. Even when a processor or a digital circuit other than the CPU 11 is included, the processing procedure is the same as that shown in FIG. That is, the analysis apparatus 100 according to the present embodiment can be realized as a program of the CPU 11, a dedicated processor, or a dedicated digital circuit.

  In step S01, the position specifying unit 13 accesses the storage unit 16 and acquires an electron microscope image and a confocal laser microscope image. The image to be analyzed can be selected by the analyst via the input device 3 while referring to the display device 2. If an image is acquired, it will progress to step S02.

  In step S02, the position specifying unit 13 extracts the shape of the cell membrane from the electron microscope image. Extraction of the shape of the cell membrane can be performed by an analyst tracing the cell membrane via the input device 3 while referring to the display device 2. The position specifying unit 13 extracts the shape of the cell membrane (hereinafter also referred to as a trace line) by receiving a signal from the operating device 3 that reflects the intention of the analyst. The position specifying unit 13 may determine the difference in luminance at the boundary of the cell membrane and automatically determine and extract the shape of the cell membrane. Note that the electron microscope image has a noise-like pattern other than the cell membrane, for example, on the cell wall or on the cell. These patterns are also intracellular substances. However, in the case of automatic extraction, these patterns are also extracted, and it may be impossible to accurately extract only the shape of the cell membrane. Therefore, an analyst can trace the cell membrane and create an image of only the traced line, thereby realizing accurate extraction of the shape of the cell membrane.

  It is assumed that the electron microscope image does not exactly match the talin of the confocal laser microscope image depending on the state of the cell at the time of photographing. Therefore, the extracted trace line may be subjected to an expansion process for making the trace line thicker so as to cope with some errors. FIG. 8 shows an example of a trace line image obtained by performing expansion processing on the electron microscope image shown in FIG.

Here, the expansion process will be described in more detail. The expansion process is a process for enlarging the trace line corresponding to the cell membrane by one pixel. When the output image is g and the target pixel of the output image is g _y , this operation can be defined by the equation shown in Equation 1. Here, 1-pixel has a pixel value of 0, and 0-pixel has a pixel value of 255. The 4-neighbor is X ₁ , X ₃ , X ₅ , X ₇ when X ₀ in Table 1 is the target pixel, and the 8-neighbor is all X _{1 to} X _8. .

  When the shape of the cell membrane is extracted from the electron microscope image, the process proceeds to step S03.

In step S03, the position specifying unit 13 extracts the shape of the cell membrane from the confocal laser microscope image. In the confocal laser microscope image, talin is colored and displayed (see FIG. 4). Therefore, extraction of the shape of the cell membrane from the confocal laser microscope image may be performed by extracting the colored portion of the confocal laser microscope image. This process can be automatically extracted by binarization. That is, when the original image is f, the target pixel is f (i, j), the resulting image is f _t , the target pixel is f _t (i, j), and the reference value (threshold) is t Binarization is performed by the equation shown in Equation 2. In addition, 1-pixel and 0-pixel are the same as the expansion process described above.

  Here, FIG. 9 shows a confocal laser microscope image (threshold 32) that has been binarized. FIG. 10 shows a confocal laser microscope image (threshold 40) that has been binarized. Tallinn is originally distributed at a distance from each other, and if the threshold value t is set too high, a portion taken out as a cell membrane may be interrupted as shown in FIG. Therefore, the threshold is preferably about 32, for example. When the shape of the cell membrane is extracted from the confocal laser microscope image, the process proceeds to step S04.

  In step S04, the position specifying unit 13 compares the extracted shapes of the cell membranes and specifies which position of the electron microscope image the confocal laser microscope image corresponds to. More specifically, the position specifying unit 13 performs the process relating to the position specification by a processing method called template matching. Template matching is a search method in which an input image and a template image are overlapped and compared and collated while moving the template image to determine whether they match. In this embodiment, the trace line image (see FIG. 8) obtained by executing step 02 is used as the template image, and the binarization obtained by executing step S03 is performed as the input image. The confocal laser microscope image (see FIG. 9) is used.

  In this embodiment, an index is used when determining whether or not the template image and the input image match. As the index (score), a ratio of matching and non-matching of the pixels of the template image and the input image is used. The index can be expressed by the equation shown in Equation 3.

  Here, when the overlapping portion of the template image and the input image is similar, the ratio of the black pixels of the template image that overlap with the black pixels of the input image, and the input image of the white pixels of the template image The ratio of overlapping with the white pixel of becomes higher, and as a result, the index becomes higher. Conversely, if the overlapping part of the template image and the input image is not similar, the ratio of the black pixels of the template image that overlap with the white pixels of the input image, the black of the input image of the white pixels of the template image Since the ratio of overlapping with the pixel of this becomes higher, the index becomes lower.

  Note that the above-described index can be determined whether or not the template image and the input image match by using the ratio of only the black pixels overlapping if the shape is originally compared. However, for example, by subtracting the ratio of overlapping pixels of different colors and adding white to the determination, it is possible to prevent the index from being erroneously increased even if it overlaps the black region. For example, when trying to identify (search) the position of 探索 as shown in FIG. 11, if it is ◯, the index does not increase even when calculating only black pixels. However, in the case of ●, since the black pixels overlap each other, it is assumed that the index is high even though it is different from the template image. However, by subtracting the proportion of the colors that do not match here, for example, it is possible to suppress an increase in the index at the time of ●.

  Here, the process using the above-described index will be described with a specific example. Hereinafter, a case where the position specifying unit 13 specifies (searches) the position of the image of FIG. 13 on the image of FIG. 12 will be described. The area surrounded by the square X in FIG. 12 is an area where the highest index is obtained when the images in FIG. 13 are superimposed. Note that a region a in FIG. 13 is a margin generated when the image is rotated, and the overlapping of the regions a is not related to the calculation of the index. That is, the index is obtained by referring to the pixels in the range of the area a1 in FIG.

(Calculation of bb)
Here, FIG. 15A shows an image obtained by superimposing the image shown in FIG. 13 on the image shown in FIG. A black area b is a binarized Tallinn image (input image shown in FIG. 12), and a light gray area c is an image obtained by tracing a cell membrane (template image shown in FIG. 13). As a ratio necessary for obtaining the index, first, a value of bb (a ratio of the template image overlapping the black of the input image) is obtained. Specifically, among the black pixels of the image (template image) of FIG. 13, a portion overlapping with the black pixels of the image (input image) of FIG. 12 is obtained. This overlapping portion is obtained by comparing / collating the image of FIG. 12 with the image of FIG. FIG. 15B shows the comparison / collation result, and the overlapping portion is indicated by a region d in FIG. 15B. From the image shown in FIG. 15B, the area d occupies 76% of the black pixels in FIG. Therefore, bb = 76.

(Calculation of ww)
Next, the value of ww (the ratio of the white of the template image that overlaps the white of the input image) is obtained as the ratio necessary for obtaining the index. This overlapping portion is obtained by comparing / collating the image of FIG. 12 with the image of FIG. FIG. 15C shows a portion where white pixels in the image (template image) in FIG. 13 overlap with white pixels in the image (input image) in FIG. A portion where white pixels overlap each other is indicated by a region e in FIG. 15C. In FIG. 15C, the region e occupies 60% of the white pixels in FIG. Therefore, ww = 60.

(Calculation of wb)
Next, as a ratio necessary for obtaining the index, a value of wb (a ratio of black of the template image that overlaps with white of the input image) is obtained. FIG. 15D shows a location where black pixels in the image (template image) of FIG. 13 overlap with white pixels in the image (input image) of FIG. A portion where the black pixel in the image of FIG. 13 and the white pixel in the image of FIG. 12 overlap is indicated by a region f in FIG. 15D. In FIG. 15D, the region f occupies 24% of the black pixels in FIG. Therefore, wb = 24.

  Next, as a ratio necessary for obtaining the index, a value of bw (a ratio of the white of the template image overlapping the black of the input image) is obtained. FIG. 15E shows a portion where a white pixel in the image (template image) in FIG. 13 and a black pixel in the image (input image) in FIG. 12 overlap. A portion where the white pixel in the image of FIG. 13 and the black pixel in the image of FIG. 12 overlap is indicated by a region g in FIG. 15E. In FIG. 15E, the region g occupies 40% of the white pixel in FIG. Therefore, bw = 40.

  From the above results, index (score) = bb−wb + ww−bw = 76−24 + 60−40 = 72. That is, the index in this embodiment is 72. In addition, when the overlapping part of the image (template image) which traced the cell membrane and the binarized Tallinn image (input image) are similar, the value of bb and ww becomes high, and as a result, the index also becomes high. On the other hand, when the overlapping part of the image obtained by tracing the cell membrane and the Tallinn image are not similar, the values of wb and bw are high and the index is low. Therefore, it is possible to determine how similar the overlapping portion of the image obtained by tracing the cell membrane and the Tallinn image is based on the index.

  The index preferably takes a value of −200 ≦ score ≦ 200. Therefore, when the index is 72 as described above, approximately 70% of the index matches as a whole. For example, when the index is 8, approximately 50% coincides.

  When specifying the position by the above method, specify the angle at which the cell membrane of the electron microscope image treated as the template image appears on the input image, and further, an error may occur in the scale of the template image on the input image. It is necessary to consider that there is. Therefore, in the present embodiment, the image (template image) in FIG. 13 is rotated in the range of 0 ° to 359 °, and the scale of the image in FIG. 13 is increased to change the size of the image in FIG. Is executed. Regarding the width of the scale ratio, when the scale ratio of the horizontal length of the image is X and the scale ratio of the vertical length of the image is Y, 0.07 ≦ X ≦ 0.11, 0.06 ≦ Y ≦ It is preferable to take the range of 0.10.

  When the extracted shapes of the cell membranes are collated and the position of the confocal laser microscope image corresponding to the electron microscope image is specified, the process proceeds to step S05.

  In step S <b> 05, the combining unit 14 superimposes the confocal laser microscope image on the electron microscope image based on the specification result by the position specifying unit 13. Subsequently, in step S06, the output unit 15 outputs the superimposed image (see FIG. 6) to the display device 2. By performing the analysis processing described above, it is possible to specify only the position of talin on an electron microscope image showing all the cells.

<Modification>
The analysis apparatus 100 according to the modified example superimposes a specific confocal laser microscope image (fluorescent antibody staining: corresponding to the third image of the present invention) that captures a protein different from talin on an electron microscope image. As a result, according to the analysis apparatus 100 according to the modified example, it is possible to specify the position of a protein other than talin on the electron microscope image. The basic configuration of the analysis apparatus 100 according to the modified example is the same as that of the analysis apparatus 100 described above, and the description thereof is omitted.

  Explaining the difference, the storage unit 16 of the analysis apparatus 100 according to the modified example further stores a specific confocal laser microscope image showing a protein different from Tallin. This specific confocal laser microscope image is obtained from the same sample as the confocal laser microscope image, but by changing the type of fluorescent material and changing the wavelength of the laser to be irradiated, a protein different from talin is copied. . FIG. 16 shows an example of a cut out specific confocal laser microscope image showing a protein different from talin. A region surrounded by diagonal lines in FIG. 16 indicates a protein different from talin. The protein different from talin is not particularly limited as long as it is required to confirm the position and can be displayed in a color different from that of talin by irradiation with a laser having a predetermined wavelength. In addition, as long as it can be obtained from the same sample as the confocal laser microscope image, another substance may be projected instead of a protein different from talin.

  On the other hand, FIG. 17 shows an example of an image obtained by superimposing the extracted specific confocal laser microscope image and the electron microscope image. According to FIG. 17, it can be confirmed that a protein different from talin is present inside the cell. FIG. 18 shows an example of an image obtained by superimposing the extracted confocal laser microscope image, the extracted specific confocal laser microscope image, and the electron microscope image. According to FIG. 18, the positions of talin and proteins other than talin can be confirmed on an electron microscope image.

  Here, an analysis process performed by the analysis apparatus 100 according to the modification will be described. FIG. 19 shows an analysis processing flow according to the modification. The process flow of FIG. 19 shows the process in the case of superimposing the confocal laser microscope image and the specific confocal laser microscope image on the electron microscope image. In addition, about the process same as the process flow shown in FIG. 7, detailed description is omitted by attaching | subjecting the same step number.

  In step S01-1, the position specifying unit 13 accesses the storage unit 16 and acquires an electron microscope image, a confocal laser microscope image, and a specific confocal laser microscope image. Next, in step S02, the position specifying unit 13 extracts the shape of the cell membrane from the electron microscope image. Next, in step S03, the position specifying unit 13 extracts the shape of the cell membrane from the confocal laser microscope image. Next, in step S04, the position specifying unit 13 compares the extracted shape of each cell membrane to specify which position of the confocal laser microscope image corresponds to the electron microscope image. . When the position is specified, the process proceeds to step S05-1.

  In step S05-1, the combining unit 14 superimposes the confocal laser microscope image and the specific confocal laser microscope image on the electron microscope image based on the identification result by the position identifying unit 13. The specific confocal laser microscope image and the confocal laser microscope image are obtained from the same sample, and the resolution and imaging range of the image are the same except that the wavelength of the irradiated laser is different. In other words, the specific confocal laser microscope image and the confocal laser microscope image are overlaid in advance. Therefore, in step S04, when the position of the confocal laser microscope image corresponding to the electron microscope image is specified, the position of the specific confocal laser microscope image corresponding to the electron microscope image can be specified. Become. Therefore, in step S05-1, the combining unit 14 can superimpose the confocal laser microscope image and the specific confocal laser microscope image on the electron microscope image based on the specification result by the position specifying unit 13.

  As a modification, the case where three images are overlaid has been described, but only the specific confocal laser microscope image may be overlaid on the electron microscope image (see FIG. 17). In this case, for example, information regarding the shape of the cell membrane included in the confocal laser microscope image may be copied to the specific confocal laser microscope image, and the specific confocal laser microscope image may include information regarding the shape of the cell membrane. Thereby, it becomes possible to collate the shape of the cell membrane of the electron microscope image with the shape of the cell membrane of the specific confocal laser microscope image. As a result, it is possible to specify which position of the electron microscope image the specific confocal laser microscope image corresponds to. Note that the number of specific confocal laser microscope images may be increased to two or three. As a result, it is possible to confirm the positions of a plurality of specific proteins on the electron microscope image.

  In step S06, the output unit 15 outputs an image (see FIG. 18) in which the confocal laser microscope image and the specific confocal laser microscope image are superimposed on the electron microscope image to the display device 2. By performing the analysis process described above, it is possible to specify not only the position of talin but also the position of a protein different from talin on an electron microscope image showing all the cells (see FIG. 18).

<Test>
Next, a test for specifying the cell position of the electron microscope image actually performed by the above-described analyzer will be described.

<Test conditions>
In this test, electron microscope images having a plurality of magnifications were prepared. Specifically, two electron microscope images are 5 ×, one image is 10 ×, and one image is 20 ×. The graphic software Paint (registered trademark) attached to Windows (registered trademark) was used for tracing the cell membrane of the electron microscope image. As the setting of the processing contents, the number of times of expansion processing is changed according to the magnification, and is 20 times for a magnification of 5 times, 40 times for a magnification of 10 times, and 80 times for a magnification of 20 times. . As for the change in the image size due to the magnification, the template image is not reduced according to the magnification, but the input image is enlarged.

  In this test, a confocal laser microscope image binarized with a threshold value 32 was used as an input image. Further, the range of specification (search) is the range shown in FIG. Since this range is too wide to fit in one image, it was divided into two images, and two of them were searched.

<About output image>
In this test, an image as shown in FIG. 21 was created as an output image for judging the result. This image is displayed by changing the color according to the height of the index at the place where the template image and the input image overlap. The image shown in FIG. 21 is a result of actual matching with the template image of FIG. And in the image of FIG. 21, the area | region where a parameter | index is high is shown by the area | region h.

  Note that the example of the image in FIG. 21 is an example in which the identification is effectively performed, and only one region with a high index is identified. In addition, the indices other than the area where the identified index is high are low, and the difference between the area where the index is high and the area where the index is low is significant. However, it is also assumed that a plurality of regions with high indexes are specified. In such a case, in addition to the determination by the analyzer, the tester's determination may be used secondarily.

  Next, a method for creating an output image will be described. In the description, it is assumed that the input image is f, the pixel value of the input image is f (i, j), the output image is g, the pixel value of the output image is g (i, j), and the obtained index is s. Further, the coordinates of the upper left point of the template image are (a, b), and the coordinates of the lower right point are (c, d). These are shown in FIG.

  First, an image having the same size as the input image is prepared as an output image, and all pixel values are set to zero. Next, an index s is obtained in the range of a ≦ i ≦ c and b ≦ j ≦ d of the input image f, that is, in a portion overlapping the template image. Then, the obtained index s is reflected in the output image using the equation shown in Equation 4.

  The above processing will be described below with specific examples. First, consider the state shown in FIG. The index at the position of the template image in FIG. The pixel value of the output image in the same range as the range from which the index is obtained is compared with the index 51, and the larger one is set as the pixel value. As a result, the output image is in the state shown in FIG.

  Next, consider a state in which the template image is moved from the state of FIG. Here, it is assumed that the index is 63. At this time, the output image is in the state shown in FIG.

  Next, consider a state in which the column is moved. It is assumed that the index is 60 at this time. At this time, the corresponding pixel values have three types of values of 0, 51, and 63. Since the index this time is 60, pixel values are rewritten for those other than the pixel value 63. Then, the output image is in the state shown in FIG.

  In this way, in this test, a process of writing the maximum value of the index obtained including the pixels to each pixel value was performed. Then, pixel values were converted for the obtained image. The conversion was performed using the formula shown below. In the formula, max represents the maximum pixel value in the image, and min represents the minimum pixel value in the image.

  When this conversion is performed, the pixel value takes a range of 0 ≦ g (i, j) ≦ 255, and the higher the index, the closer the pixel value is to 0 (the color on the image is black) and the lower the pixel value is. The closer the pixel value is to 255 (the color on the image is white).

  Thereafter, the converted image and the image of FIG. 27 are compared, and only the black region of the image of FIG. 27 is extracted from the converted output image. At this time, all other regions have a pixel value of 255. The image is shown in FIG. The image of FIG. 28 is the result of searching the image of FIG. 12 used in the section of this method and output image. Furthermore, when pixel values other than the pixel value 255 in FIG. 28 are set to the maximum values that can be taken by H, S, and V in the HSV color space, they are converted into the RGB color space and displayed as shown in FIG.

<Result>
As a result of searching with the respective electron microscope images in FIG. 29, FIG. 30, FIG. 31, and FIG. 32 below, the corresponding portions of the original electron microscope image, the cell membrane trace image used as the template image, and the confocal laser microscope image are shown. An enlarged image is shown. A portion surrounded by a square on the output image 1 or the output image 2 is a corresponding portion. In addition, all confocal laser microscope images show Tallinn.

  Looking at the results, it was confirmed that all magnifications are located at relatively high indices. In particular, the one with a magnification of 5 (1) in FIG. 29 is well searched. In addition, it is thought that the thing of the high magnification of FIG. 31, FIG. 32 is more preferable to add a tester's judgment secondary because the parameter | index becomes high as a whole. In order to improve matching accuracy, the number of expansion processes may be further increased or decreased. In addition, the improvement in accuracy can be expected by connecting broken lines on the input image.

<Effect>
As described above, according to the analysis apparatus according to the embodiment, by superimposing the electron microscope image and the confocal laser microscope image, it is possible to more accurately obtain the position of talin in the cell without damaging the cell. Can do. That is, the position of talin in the cell can be obtained more accurately on the electron microscope image. As a result, the cellular state can be grasped in more detail, and it can contribute to the elucidation of various symptoms appearing in the human body.

DESCRIPTION OF SYMBOLS 1 ... Case 2 ... Display apparatus 3 ... Input device 4 ... Interface 10 ... Control part 11 ... CPU
DESCRIPTION OF SYMBOLS 12 ... Memory 13 ... Position specification part 14 ... Coupling part 15 ... Output part 16 ... Memory | storage part 100 ... Analysis apparatus

Claims (7)

A cell analysis device,
A first image having information relating to a cell structure of the cell; and information relating to a substance including the first image and contained in the cell, the information relating to a substance associated with the cell structure of the cell. A storage unit for storing two images;
The cell structure is extracted from the first image, the cell structure associated with the substance is extracted from the second image, the extracted cell structures are collated, and the second image is the first image. A position specifying unit for specifying which position corresponds to;
An analysis apparatus comprising: a combining unit that superimposes the second image on the specified position of the first image based on a specifying result by the position specifying unit.   The analysis device according to claim 1, wherein the storage unit stores the first image as an electron microscope image and the second image as a label image that images the substance to be labeled.   The analysis apparatus according to claim 1, wherein the second image includes protein information as the substance information.   The position specifying unit obtains information on the shape of the cell membrane adjacent to the protein as a cell structure related to the protein as the substance, collates the extracted shape of each cell membrane, and the second image is The analysis apparatus according to claim 3, wherein the position corresponding to the first image is specified.   The position specifying unit includes a ratio of overlapping black pixels in the first image among black pixels in the second image, and a ratio of overlapping white pixels in the first image among white pixels in the second image; Calculated based on the proportion of black pixels in the second image that overlap with white pixels in the first image and the proportion of white pixels in the second image that overlap with black pixels in the first image 5. The method according to claim 1, wherein it is determined whether or not the first image and the second image match using an index, and the position of the first image corresponding to the second image is specified. An analysis device according to any one of the above. The storage unit is a third image associated with the second image by being obtained from the same sample as the second image, and further stores a third image showing a specific substance different from the substance,
The position specifying unit collates the cell structure extracted from the second image associated with the third image with the cell structure extracted from the first image, and the third image is Identify which position in the image corresponds to,
The analysis device according to any one of claims 1 to 5, wherein the combining unit further superimposes the third image on a specified position of the first image based on a specifying result by the position specifying unit. An analysis method for analyzing cells,
Computer
A first image having information relating to a cell structure of the cell in a storage device; and information relating to a substance that includes the first image and is contained in the cell, the substance being associated with the cell structure of the cell A storage step for storing a second image having information;
The cell structure is extracted from the first image, the cell structure associated with the substance is extracted from the second image, the extracted cell structures are collated, and the second image is the first image. A position identifying step for identifying which position corresponds to;
An analysis method for executing a combining step of superimposing the second image on the specified position of the first image based on the specifying result of the position specifying step.

Images