JP2004054347A - Image processing method, image processing program, and image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To promptly and effectively extract an outline of a cell and to automatically and promptly detect the change in shape. <P>SOLUTION: A phase difference image and a fluorescent image are processed for an identical field. In the processing of the phase difference image, the edge of the phase difference image is extracted (step S101), and a GVF is created based on the extracted edge (step S102). In the processing of the fluorescent image, an initial outline is created (step S103). Based on the processing results, the outline of the cell is extracted by using the GVF Snakes (step S104). <P>COPYRIGHT: (C)2004,JPO

Description

を最小にするｕ（ｘ，ｙ），ｖ（ｘ，ｙ）と定義する。
【００３４】
エッジ付近では、
【数２】

そして、エッジから離れた場所では、
【数３】

の項の影響が大きくなるのでｕ，ｖがゆっくり変化する。したがって、広範囲に力がおよびエッジ付近ではエッジに引き付けられる力が強いので、正確に目的とするエッジを抽出することができる。また、パラメータμの値を大きくするとＧＶＦ　Ｓｎａｋｅｓの処理の際に輪郭の収束は遅くなるが、ノイズの影響を受け難くなる。図７は、図６のエッジより求めたＧＶＦを画像化したものである。
【００３５】
つぎに、蛍光画像４２１を読み込む（ステップＳ４０５）。従来のように細胞を染色し鮮明な画像を得るという方法をとることができないため、解析に使用する画像を得るために、９６穴プレート内の細胞に蛍光マーカーを付与したｃＤＮＡを注入・培養し、光学顕微鏡３５０によって撮影した蛍光画像４２１を読み込む。ｃＤＮＡを注入してもすべての細胞にｃＤＮＡが入り込むわけではない。図８に示すように、注入に成功した細胞の核のみが光っている。
【００３６】
蛍光画像４２１を注目細胞（ｃＤＮＡの注入に成功した細胞）のマーカーとして利用すると同時に、Ｓｎａｋｅｓの初期輪郭に利用する。蛍光画像４２１を二値化することによって核の輪郭を抽出し、Ｓｎａｋｅｓの初期輪郭とする（ステップＳ４０６）。画像の二値化はヒストグラムによるものが一般的であるが、９６穴プレートで撮影した画像は背景部分の輝度が画像内の場所に依存してゆるやかに変化することがあるので、望ましい結果を得ることは難しい。
【００３７】
図９は、蛍光画像のある位置の水平方向の輝度分布を示す説明図である。このように輝度分布に勾配がある場合が多い。そこで、ここでは前処理としてモルフォロジを利用することによって、安定して二値化画像を得られるようにした。
【００３８】
まず、蛍光画像を核より小さい構造要素でｃｌｏｓｉｎｇ処理をおこない、その後大きい構造要素（ここでは核と同程度の大きさのものを使用）でｏｐｅｎｉｎｇ処理をおこなう。これによって、図１０に示すように背景画像を得ることができる。図１０は、図９と同じ場所における輝度分布を示す説明図である。
【００３９】
ここで、ｏｐｅｎｉｎｇ処理のみでも背景画像を得られるが、ｃｌｏｓｉｎｇ処理をおこなうことによってノイズを減らすことができる。つぎに、図１１に示すように、元画像からこの背景画像を引くことによって発光部分の輝度だけが残り、背景部分の輝度は０以下となるので閾値は簡単に決めることができる。図１１は図９と同じ場所の輝度分布を示す説明図であり、この処理で値が０以下になる場合は輝度を０にしている。
【００４０】
このように画像を二値化することにより核の型が得られる。この核の輪郭をＳｎａｋｅｓの初期輪郭４２２とする。図１２は、図８の蛍光画像の処理結果を示す説明図である。また、この方法で二値化すると密集している細胞の核どうしが繋がってしまう場合がある。そこで、図１３に示すように、モルフォロジを利用た分水嶺アルゴリズムにより、繋がっている核を分離する処理を加える。
【００４１】
つぎに、ＧＶＦ　Ｓｎａｋｅｓによって細胞の輪郭の抽出処理をおこなう（ステップＳ４０７）。ＧＶＦ　Ｓｎａｋｅｓは、
輪郭点を、
【数４】

ＧＶＦを、
【数５】

とするとき、以下の式を繰り返し計算して目的の輪郭に収束させる。
【数６】

（α：張力パラメータ、β：平滑パラメータ、γ：ＧＶＦのパラメータ、ｔ：時間）
【００４２】
ただし、初期輪郭は核の輪郭であり、目的とする輪郭の内側にある。そこで、輪郭を膨らませる力が必要なので、
【数７】

を（２）式に加えた以下の式を利用する。
【数８】

【００４３】
上記ステップＳ４０６における初期輪郭をこの式によって、輪郭の変化がある程度小さくなるまで計算し、細胞の輪郭を抽出する。図１４は図５、図８の画像より細胞の輪郭抽出をおこなった結果を示す説明図である。ここで、実線は初期輪郭を示しており、点線は処理後の輪郭を示している。
【００４４】
なお、ステップＳ４０１の位相差画像４１１の読み込みと、ステップＳ４０５の蛍光画像４２１の読み込みは、いずれかが先であってもよく、また、同時であってもよい。したがって、図４に示した順序で位相差画像４１１の処理が終了した後、蛍光画像４２１の処理をおこなってもよく、その逆であってもよい。さらに、両者を同時並行でおこなってもよい。
【００４５】
（具体的実験例）
つぎに、発明者が実際におこなった具体的な実験例について説明する。この実験例で、使用した画像（６９６×５２０Ｐｉｘｅｌ、対物２０倍）は、Ｈｅｌａ細胞という癌細胞である。この細胞の平均的なサイズは７０×５０Ｐｉｘｅｌ程度である。この細胞画像１６０枚（１９６３個の細胞）について、輪郭抽出をおこない評価をおこなった。蛍光画像はｃＤＮＡの注入が成功し、核が発光しているものを使用した。
【００４６】
また、つぎのようにパラメータを設定して実験をおこなった。まず、エッジの抽出処理については、使用したモルフォロジの構造要素は３×３、５×５、７×７、９×９Ｐｉｘｅｌの４種類であり、それぞれの構造要素でエッジを抽出しその結果をマージした。ここで大きさの異なる構造要素を複数使うことによって、より正確にエッジの抽出をすることができた。
【００４７】
また、核の輪郭抽出処理については、使用したモルフォロジの構造要素はｃｌｏｓｉｎｇに１１×１１Ｐｉｘｅｌ、ｏｐｅｎｉｎｇに３１×３１Ｐｉｘｅｌの構造要素を使用した。二値化は、輝度（２５６階調）が０を背景、１以上を核の領域としておこなった。また、ＧＶＦ　Ｓｎａｋｅｓについては、０α＝０．６、β＝　０．０、γ＝２．０、κ＝１．８、μ＝　０．１５とした。
【００４８】
評価方法は位相差画像と輪郭抽出をおこなった処理画像を重ねあわせ、目視により評価した。結果として約６６％（１９６３個のうち１２９４個）の細胞の輪郭抽出に成功した。抽出できていないものは位相差画像においてハローがはっきりと出ておらず、細胞と培地の境界が不鮮明なものである。図１５〜図１７に結果例を示す。図１５〜図１７において、初期輪郭は処理結果の実線で示されており、処理後の輪郭は点線で示されている。
【００４９】
図１５は、画像中央にある細胞の下部分のハローが不鮮明なので輪郭を抽出できなかった例である。図１６に示すようにハローが出ている細胞に関しては正確に細胞の輪郭を抽出できている。図１７は、図１３（ｄ）を初期輪郭として処理したもので、右から二番目の細胞はノイズにより上部の輪郭抽出ができなかったが、それ以外は比較的きれいに輪郭を抽出できた。
【００５０】
以上説明したように、本実施の形態によれば、細胞にｃＤＮＡを注入したときに引き起こされる形態変化を検出するために、画像処理のＧＶＦ　Ｓｎａｋｅｓにより細胞の輪郭抽出を試みた。まず、蛍光画像を用いてＧＶＦ　Ｓｎａｋｅｓの初期輪郭を作成した。ここでモルフォロジを利用した二値化により撮影条件の影響を受けずに安定して初期輪郭を作成できた。つぎに、位相差画像のハローを抽出してＧＶＦ　Ｓｎａｋｅｓのエッジ情報としたことで、正確に細胞輪郭を抽出することに成功した。このように蛍光画像と位相差画像を利用することによって、注目細胞の輪郭抽出を自動的におこなうことに成功した。
【００５１】
また、本実施の形態における画像処理方法は、あらかじめ用意されたコンピュータ読み取り可能なプログラムであってもよく、またそのプログラムをパーソナルコンピュータやワークステーションなどのコンピュータで実行することによって実現される。このプログラムは、ＨＤ、ＦＤ、ＣＤ−ＲＯＭ、ＭＯ、ＤＶＤなどのコンピュータで読み取り可能な記録媒体に記録され、コンピュータによって記録媒体から読み出されることによって実行される。また、このプログラムは、インターネットなどのネットワークを介して配布することが可能な伝送媒体であってもよい。
【００５２】
（付記１）同一視野における細胞の位相差画像に関する情報および蛍光画像に関する情報の入力を受け付ける画像情報入力工程と、
前記画像情報入力工程によって入力が受け付けられた位相差画像に関する情報および蛍光画像に関する情報に基づいて前記細胞の輪郭を抽出する輪郭抽出工程と、
前記輪郭抽出工程によって抽出された輪郭に関する情報を出力する出力工程と、
を含んだことを特徴とする画像処理方法。
【００５３】
（付記２）前記画像情報入力工程によって入力が受け付けられた位相差画像に関する情報に対して、細胞輪郭のエッジを抽出する位相差画像処理工程を含み、
前記輪郭抽出工程は、前記位相差画像処理工程によって抽出された細胞輪郭のエッジに基づいて前記細胞の輪郭を抽出することを特徴とする付記１に記載の画像処理方法。
【００５４】
（付記３）前記位相差画像処理工程は、抽出された細胞辺縁のハローを用いて細胞輪郭のエッジを算出し、算出されたエッジを用いてＧＶＦ（ｇｒａｄｉｅｎｔ
ｖｅｃｔｏｒ　ｆｌｏｗ）を作成し、
前記輪郭抽出工程は、前記位相差画像処理工程によって作成されたＧＶＦに基づいて前記細胞の輪郭を抽出することを特徴とする付記２に記載の画像処理方法。
【００５５】
（付記４）前記画像情報入力工程によって入力が受け付けられた蛍光画像に関する情報に対して、モルフォロジを利用して核の輪郭を抽出する蛍光画像処理工程を含み、
前記輪郭抽出工程は、前記蛍光画像処理工程によって抽出された核の輪郭に基づいて前記細胞の輪郭を抽出することを特徴とする付記１〜３のいずれか一つに記載の画像処理方法。
【００５６】
（付記５）前記蛍光画像は、蛍光マーカーを付与したｃＤＮＡを注入した細胞に関する画像であることを特徴とする付記１〜４のいずれか一つに記載の画像処理方法。
【００５７】
（付記６）前記蛍光画像を注目細胞のマーカーとして用いることを特徴とする付記５に記載の画像処理方法。
【００５８】
（付記７）同一視野における細胞の位相差画像に関する情報および蛍光画像に関する情報の入力を受け付けさせる画像情報入力工程と、
前記画像情報入力工程によって入力が受け付けられた位相差画像に関する情報および蛍光画像に関する情報に基づいて前記細胞の輪郭を抽出させる輪郭抽出工程と、
前記輪郭抽出工程によって抽出された輪郭に関する情報を出力させる出力工程と、
をコンピュータに実行させることを特徴とする画像処理プログラム。
【００５９】
（付記８）同一視野における細胞の位相差画像に関する情報および蛍光画像に関する情報を入力する画像情報入力手段と、
前記画像情報入力手段によって入力された位相差画像に関する情報および蛍光画像に関する情報に基づいて前記細胞の輪郭を抽出する輪郭抽出手段と、
前記輪郭抽出手段によって抽出された輪郭に関する情報を出力する出力手段と、
を備えたことを特徴とする画像処理装置。
【００６０】
【発明の効果】
以上説明したように、この発明によれば、目視に頼ることなく、迅速にかつ効率的に細胞輪郭を抽出することができ、これによって、形態変化の検出を自動的にかつ所定期間で大量におこなうことが可能な画像処理方法、画像処理プログラム、画像処理装置が得られるという効果を奏する。
【図面の簡単な説明】
【図１】この発明の本実施の形態にかかる画像処理方法における細胞輪郭抽出の概要を示す説明図である。
【図２】この発明の本実施の形態にかかる画像処理装置のハードウエア構成の一例を示すブロック図である。
【図３】この発明の本実施の形態にかかる画像処理装置の機能的構成の一例を示すブロック図である。
【図４】この発明の本実施の形態にかかる画像処理装置の処理の内容を示すフローチャートである。
【図５】位相差画像の一例を示す説明図である。
【図６】図５の位相差画像の処理結果を示す説明図である。
【図７】図６のエッジより求めたＧＶＦを画像化したものを示す説明図である。
【図８】蛍光画像の一例を示す説明図である。
【図９】蛍光画像のある位置の水平方向の輝度分布を示す説明図である。
【図１０】図９と同じ場所における輝度分布を示す説明図である。
【図１１】図９と同じ場所の輝度分布を示す説明図である。
【図１２】図８の蛍光画像の処理結果を示す説明図である。
【図１３】繋がっている核を分離する処理を示す説明図である。
【図１４】図５、図８の画像より細胞の輪郭抽出をおこなった結果を示す説明図である。
【図１５】細胞の輪郭抽出の処理（抽出失敗）を示す説明図である。
【図１６】別の細胞の輪郭抽出の処理（抽出成功）を示す説明図である。
【図１７】別の細胞の輪郭抽出の処理（抽出成功）を示す説明図である。
【符号の説明】
３００　制御部
３０１　画像外部入力部
３０２　位相差画像処理部
３０３　蛍光画像処理部
３０４　輪郭抽出処理部
３１０　記憶部
３１１　画像情報記憶部
３１２　エッジ情報記憶部
３１３　初期輪郭情報記憶部
３１４　輪郭画像情報記憶部
３５０　光学顕微鏡
３５１　モニタ
４１１　位相差画像
４１２　画像のエッジ
４１３　ＧＶＦ　ｆｉｅｌｄ
４２１　蛍光画像
４２２　核の輪郭（ＧＶＦ　Ｓｎａｋｅｓの初期輪郭）[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method, an image processing program, and an image processing device for extracting an outline of a cell in an image of the cell taken by an optical microscope or the like.
[0002]
[Prior art]
In recent years, the outline of human genetic information has been deciphered, and as the next stage, attention has been paid to functional analysis of DNA for application to the medical and drug discovery fields. As one of the means, there is an attempt to elucidate the function of DNA by quantitatively analyzing the morphological change caused when cDNA is injected into cells. Conventionally, a method of visually screening for morphological changes has been used for the analysis.
[0003]
[Problems to be solved by the invention]
However, in the above-described conventional technology, there is a problem that a great amount of time and cost are required for the analysis process. In particular, it is necessary to analyze a large amount of morphological changes in order to apply it to the drug discovery field (for example, 10,000 samples or more per day). Therefore, it is necessary to automatically detect and analyze morphological changes by image processing. It has been demanded.
[0004]
In addition, since it is necessary to handle living cells for analysis, it is not possible to take a method of staining cells and obtaining a clear image as in the related art. Furthermore, when a phase difference image is used, there are many cases in which the outline of a cell is unclear or contains noise, and there has been a problem that it is difficult to extract the outline by the conventional edge extraction method.
[0005]
The present invention solves the above-mentioned problem by rapidly and efficiently extracting the contours of cells, whereby an image processing method and an image processing method capable of detecting a morphological change automatically and in a large amount in a predetermined period are provided. It is an object to provide a program and an image processing device.
[0006]
[Means for Solving the Problems]
In order to solve the above-described problem and achieve the object, an image processing method, an image processing program, and an image processing apparatus according to the present invention are configured to input information on a phase difference image of a cell in the same field of view and information on a fluorescent image, and input the information. The method is characterized in that the contour of the cell is extracted based on the information on the phase difference image and the information on the fluorescence image, and information on the extracted contour is output.
[0007]
According to these inventions, a cell contour can be quickly and efficiently extracted without relying on visual observation.
[0008]
Further, with respect to the input information on the phase difference image, an edge of the cell contour is calculated using the extracted cell edge halo, and a GVF (gradient vector flow) is created using the calculated edge. The morphology is used to extract the outline of the nucleus from the input information about the fluorescent image, and the outline of the cell can be extracted based on the created GVF and the extracted nucleus outline. Furthermore, a fluorescent image can be used as a marker for a cell of interest.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Preferred embodiments of an image processing method, an image processing program, and an image processing apparatus according to the present invention will be described below in detail with reference to the accompanying drawings.
[0010]
(Overview of cell contour extraction processing)
First, an outline of a cell contour extraction process in the image processing method according to the embodiment of the present invention will be described. FIG. 1 is an explanatory diagram showing an outline of cell contour extraction in the image processing method according to the embodiment of the present invention.
[0011]
In FIG. 1, the image processing method according to the present embodiment includes processing of a phase difference image and a fluorescence image in the same visual field. As the processing of the phase difference image, first, an edge of the phase difference image is extracted (step S101), and a GVF is created based on the extracted edge (step S102). In the processing of the fluorescent image, an initial contour is created (step S103). The processing of the phase difference image and the processing of the fluorescence image can be performed in parallel. Then, based on the results obtained from both processes, cell contours are extracted by GVF Snakes (step S104).
[0012]
As described above, an edge is extracted by using the feature of the phase difference image, and the edge of the cell is extracted using the edge by GVF Snakes of an image processing technique. The fluorescent image is used as a marker for the cell of interest, and is used to create an initial outline of Snakes. Thereby, the cell contour can be accurately extracted from the image.
[0013]
(Hardware configuration of image processing device)
Next, a hardware configuration of the image processing apparatus according to the embodiment of the present invention will be described. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the image processing apparatus according to the embodiment of the present invention.
[0014]
2, an image processing apparatus includes a CPU 201, a ROM 202, a RAM 203, an HDD 204, an HD 205, an FDD (flexible disk drive) 206, an FD (flexible disk) 207 as an example of a removable recording medium, A display 208, an I / F (interface) 209, a keyboard 211, a mouse 212, a scanner 213, and a printer 214 are provided. Each component is connected by a bus 200.
[0015]
Here, the CPU 201 governs overall control of the image processing apparatus. The ROM 202 stores programs such as a boot program. The RAM 203 is used as a work area of the CPU 201. The HDD 204 controls reading / writing of data from / to the HD 205 under the control of the CPU 201. The HD 205 stores data written under the control of the HDD 204.
[0016]
The FDD 206 controls reading / writing of data from / to the FD 207 under the control of the CPU 201. The FD 207 stores the data written under the control of the FDD 206, and causes the information processing device to read the data recorded in the FD 207. In addition to the FD 207, the removable recording medium may be a CD-ROM (CD-R, CD-RW), MO, DVD (Digital Versatile Disk), memory card, or the like. The display 208 displays data such as documents, images, function information, etc., in addition to a cursor, icons, or tool boxes. For example, a CRT, a TFT liquid crystal display, a plasma display, etc.
[0017]
An I / F (interface) 209 is connected to a network such as a LAN or the Internet via a communication line, and is connected to another server or an information processing device via the network. The I / F 209 manages an interface between the network and the inside, and controls input and output of data from other servers and information terminal devices. I / F 209 is, for example, a modem.
[0018]
The keyboard 211 includes keys for inputting characters, numerals, various instructions, and the like, and inputs data. It may be a touch panel type input pad or a numeric keypad. The mouse 212 is used to move the cursor, select a range, or move and change the size of windows. A trackball, a joystick, or the like may be used as long as the pointing device has a similar function.
[0019]
The scanner 213 optically reads an image such as a driver image and takes in image data into the information processing device. Further, an OCR function is provided, and printed information can be read and converted into data by the OCR function. The printer 214 prints image data such as contour image information and document data. For example, a laser printer, an inkjet printer, or the like is used.
[0020]
(Functional configuration of image processing device)
Next, a functional configuration of the image processing apparatus will be described. FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus according to the embodiment of the present invention. 3, the image processing apparatus includes a control unit 300, an external image input unit 301, a phase difference image processing unit 302, a fluorescent image processing unit 303, a contour extraction processing unit 304, and a storage unit 310. Contains. The storage unit 310 includes an image information storage unit 311, an edge information storage unit 312, an initial contour information storage unit 313, and a contour image information storage unit 314.
[0021]
Here, the control unit 300 controls the components 301 to 314 and controls the entire apparatus. Further, the image external input unit 301 receives the input of the phase difference image information and the fluorescence image information of the same visual field from the optical microscope 350, and stores them in the image information storage unit 311.
[0022]
Further, the phase difference image processing unit 302 extracts the edge of the cell contour from the phase difference image information stored in the image information storage unit 311 and stores the edge information in the edge information storage unit 312. More specifically, a cell edge halo is extracted from the phase difference image information stored in the image information storage unit 311, an edge of a cell contour is calculated using the extracted halo, and the calculated edge information is calculated. The information is stored in the edge information storage unit 312. The details of the halo extraction process and the edge information calculation process will be described later.
[0023]
Further, the fluorescent image processing unit 303 extracts the outline of the nucleus from the fluorescent image information stored in the image information storage unit 311 using morphology, and stores the extracted outline information in the initial outline information storage unit 313. Remember.
[0024]
In addition, the contour extraction processing unit 304 extracts a cell contour based on the edge information of the cell contour stored in the edge information storage unit 312 and the contour information stored in the initial contour information storage unit 313, and includes the information. The contour image information is stored in the contour image information storage unit 314. The contour image information stored in the contour image information storage unit 314 can be output to a monitor 351 (for example, the display 208 shown in FIG. 2).
[0025]
Further, although not shown, extraction of a characteristic amount (for example, the degree of circularity or irregularity of a cell) and detection of a morphological change may be performed based on the outline image information stored in the outline image information storage unit 314. it can. The contour image information stored in the contour image information storage unit 314 can be output to the outside by, for example, the FD 207 and FDD 206, the I / F 209, the printer 214, and the like illustrated in FIG.
[0026]
The control unit 300, the phase difference image processing unit 302, the fluorescence image processing unit 303, and the contour extraction processing unit 304 realize their functions by the CPU 201 executing a program stored in the ROM 202, the RAM 203, the HD 205, or the FD 207. .
[0027]
The function of the storage unit 310 (image information storage unit 311, edge information storage unit 312, initial contour information storage unit 313, contour image information storage unit 314) is realized by RAM 203, HD 205 and HDD 204, or FD 207 and FDD 206. .
[0028]
(Processing procedure of image processing device)
Next, the procedure of the process of the image processing apparatus will be described. FIG. 4 is a flowchart illustrating a processing procedure of the image processing apparatus according to the embodiment of the present invention. In the flowchart of FIG. 4, first, the phase difference image 411 is read (step S401). FIG. 5 is an explanatory diagram illustrating an example of the phase difference image 411. As shown in FIG. 5, the phase difference image 411 often includes an image in which the outline of a cell is unclear or includes noise.
[0029]
As an effective method of contour extraction, there is an image processing technique called Snakes. This is because a certain degree of contour can be extracted by taking the edge of the image, and the broken part can be complemented by the Snakes processing. However, the phase difference image 411 used this time has an edge in the cell and becomes noise. Therefore, by extracting only the halo characteristic of the phase difference image 411, a part of the cell contour with less noise can be extracted.
[0030]
As shown in FIG. 5, the phase difference image 411 has a bright border at the edge of the cell. Here, by using a morphological close-opening filter, this bright portion can be extracted.
[0031]
Specifically, first, a close-opening filter is applied to the phase difference image 411. Next, a halo is extracted by calculating a luminance difference between the image to which the filter is applied and the original image (step S402). Thereafter, binarization and thinning processing are performed to obtain an edge 412 of the cell contour (step S403). FIG. 6 shows the processing result of the phase difference image of FIG. Here, as the size of the used structural element increases, noise increases. Therefore, a structural element small enough to extract a halo portion is selected. By this processing, some edges can be extracted.
[0032]
The halo is extracted and the broken part is supplemented by Snakes. However, in general Snakes, it is difficult to accurately obtain the contours of the corners and slender portions of the cells. Therefore, the contours of the cells are extracted by GVF Snakes to which GVF having a wide range of attraction to edges is applied.
[0033]
The GVF field 413 is calculated using the edge 412 (FIG. 6) of the image extracted in step S403 (step S404). In particular,
(Equation 1)

Are defined as u (x, y) and v (x, y).
[0034]
Near the edge,
(Equation 2)

And where away from the edge,
[Equation 3]

Since the influence of the term becomes large, u and v change slowly. Therefore, since the force is applied over a wide range and the force attracted to the edge is strong near the edge, the target edge can be accurately extracted. Further, when the value of the parameter μ is increased, the convergence of the contour is slowed down in the processing of the GVF Snakes, but the influence of noise is reduced. FIG. 7 is an image of the GVF obtained from the edge in FIG.
[0035]
Next, the fluorescent image 421 is read (step S405). Since it is not possible to take a method of staining cells and obtaining a clear image as in the past, in order to obtain an image to be used for analysis, the cells in a 96-well plate are injected and cultured with cDNA to which a fluorescent marker has been added. The fluorescence image 421 captured by the optical microscope 350 is read. Injecting the cDNA does not mean that the cDNA enters all cells. As shown in FIG. 8, only the nuclei of successfully injected cells glow.
[0036]
The fluorescence image 421 is used as a marker for a cell of interest (a cell into which cDNA has been successfully injected) and at the same time as an initial contour of Snakes. The outline of the nucleus is extracted by binarizing the fluorescence image 421, and is used as the initial outline of Snakes (step S406). The binarization of an image is generally based on a histogram, but an image taken with a 96-hole plate has a desirable result because the luminance of the background portion may change slowly depending on the position in the image. It is difficult.
[0037]
FIG. 9 is an explanatory diagram showing a horizontal luminance distribution at a certain position of the fluorescent image. As described above, the luminance distribution often has a gradient. Therefore, here, by using morphology as preprocessing, a binarized image can be stably obtained.
[0038]
First, the fluorescent image is subjected to a closing process with a structural element smaller than the nucleus, and thereafter, an opening process is performed with a large structural element (here, the same size as the nucleus is used). Thus, a background image can be obtained as shown in FIG. FIG. 10 is an explanatory diagram showing a luminance distribution at the same place as in FIG.
[0039]
Here, the background image can be obtained only by the opening process, but the noise can be reduced by performing the closing process. Next, as shown in FIG. 11, by subtracting this background image from the original image, only the luminance of the light emitting portion remains, and the luminance of the background portion becomes 0 or less, so that the threshold value can be easily determined. FIG. 11 is an explanatory diagram showing a luminance distribution at the same place as in FIG. 9. When the value becomes 0 or less in this processing, the luminance is set to 0.
[0040]
By binarizing the image in this manner, a nucleus type is obtained. The outline of this nucleus is defined as the initial outline 422 of Snakes. FIG. 12 is an explanatory diagram showing a processing result of the fluorescence image of FIG. When binarization is performed by this method, nuclei of densely packed cells may be connected to each other. Thus, as shown in FIG. 13, a process of separating connected nuclei is added by a watershed algorithm using morphology.
[0041]
Next, a cell outline extraction process is performed by GVF Snakes (step S407). GVF Snakes
Contour points,
(Equation 4)

GVF,
(Equation 5)

Then, the following equation is repeatedly calculated to converge on the target contour.
(Equation 6)

(Α: tension parameter, β: smoothing parameter, γ: GVF parameter, t: time)
[0042]
However, the initial contour is the contour of the nucleus and is inside the target contour. So we need the power to inflate the contour,
(Equation 7)

Is added to the expression (2).
(Equation 8)

[0043]
The initial contour in step S406 is calculated using this equation until the change in the contour becomes small to some extent, and the contour of the cell is extracted. FIG. 14 is an explanatory diagram showing the result of extracting the outline of a cell from the images of FIGS. Here, the solid line indicates the initial contour, and the dotted line indicates the processed contour.
[0044]
Note that the reading of the phase difference image 411 in step S401 and the reading of the fluorescence image 421 in step S405 may be performed first, or may be performed simultaneously. Therefore, after the processing of the phase difference image 411 is completed in the order shown in FIG. 4, the processing of the fluorescent light image 421 may be performed, and vice versa. Further, both may be performed simultaneously and in parallel.
[0045]
(Specific experimental example)
Next, specific experimental examples actually performed by the inventor will be described. In this experimental example, the image used (696 × 520 Pixel, 20 × objective) is a cancer cell called Hela cell. The average size of these cells is about 70 × 50 Pixel. An outline was extracted and evaluated for 160 cell images (1963 cells). The fluorescence image used was one in which the injection of cDNA was successful and the nucleus emitted light.
[0046]
In addition, an experiment was performed by setting parameters as follows. First, regarding the edge extraction processing, there are four types of morphological structural elements used: 3 × 3, 5 × 5, 7 × 7, and 9 × 9 Pixels. Edges are extracted from each structural element, and the results are merged. did. Here, the edge could be more accurately extracted by using a plurality of structural elements having different sizes.
[0047]
As for the contour extraction processing of the nucleus, the morphological structural elements used were 11 × 11 Pixels for closing and 31 × 31 Pixels for opening. The binarization was performed with a luminance (256 gradations) of 0 as a background and 1 or more as a core region. In addition, GVF Snakes were set to 0α = 0.6, β = 0.0, γ = 2.0, κ = 1.8, and μ = 0.15.
[0048]
In the evaluation method, the phase difference image and the processed image subjected to the contour extraction were superimposed and evaluated visually. As a result, about 66% (1,294 out of 1963) of cells were successfully extracted. Those that could not be extracted have no clear halo in the phase contrast image, and the boundary between the cells and the medium is unclear. 15 to 17 show examples of the results. 15 to 17, the initial contour is indicated by a solid line of the processing result, and the contour after the processing is indicated by a dotted line.
[0049]
FIG. 15 shows an example in which the outline cannot be extracted because the halo in the lower part of the cell at the center of the image is unclear. As shown in FIG. 16, the outline of the cell can be accurately extracted from the cell with halo. FIG. 17 shows the result of processing FIG. 13 (d) as an initial contour. In the second cell from the right, the contour of the upper part could not be extracted due to noise.
[0050]
As described above, according to the present embodiment, in order to detect a morphological change caused by injecting cDNA into cells, contour extraction of cells was attempted by GVF Snakes of image processing. First, an initial contour of GVF Snakes was created using a fluorescent image. Here, the initial contour could be created stably without being affected by the imaging conditions by binarization using morphology. Next, the halo of the phase difference image was extracted and used as the edge information of GVF Snakes, thereby successfully extracting the cell contour accurately. As described above, by using the fluorescence image and the phase difference image, the outline of the cell of interest was automatically extracted.
[0051]
The image processing method according to the present embodiment may be a computer-readable program prepared in advance, and is realized by executing the program on a computer such as a personal computer or a workstation. This program is recorded on a computer-readable recording medium such as HD, FD, CD-ROM, MO, and DVD, and is executed by being read from the recording medium by the computer. Further, the program may be a transmission medium that can be distributed via a network such as the Internet.
[0052]
(Supplementary Note 1) an image information inputting step of receiving input of information on a phase contrast image and information on a fluorescence image of cells in the same visual field;
A contour extraction step of extracting a contour of the cell based on information on the phase difference image and the fluorescence image received by the image information input step;
An output step of outputting information about the contour extracted by the contour extraction step,
An image processing method comprising:
[0053]
(Supplementary Note 2) A phase difference image processing step of extracting an edge of a cell contour with respect to the information on the phase difference image whose input has been received in the image information input step,
The image processing method according to claim 1, wherein the contour extracting step extracts a contour of the cell based on an edge of the cell contour extracted in the phase difference image processing step.
[0054]
(Supplementary Note 3) In the phase difference image processing step, an edge of a cell contour is calculated using the extracted halo of the cell edge, and a GVF (gradient) is calculated using the calculated edge.
vector flow),
3. The image processing method according to claim 2, wherein the contour extracting step extracts a contour of the cell based on the GVF created in the phase difference image processing step.
[0055]
(Supplementary Note 4) A fluorescence image processing step of extracting a nucleus contour using morphology with respect to the information on the fluorescence image whose input has been received in the image information input step,
The image processing method according to any one of supplementary notes 1 to 3, wherein the contour extracting step extracts a contour of the cell based on a contour of a nucleus extracted in the fluorescence image processing step.
[0056]
(Supplementary note 5) The image processing method according to any one of Supplementary notes 1 to 4, wherein the fluorescent image is an image relating to a cell into which cDNA with a fluorescent marker has been injected.
[0057]
(Supplementary note 6) The image processing method according to supplementary note 5, wherein the fluorescence image is used as a marker of a cell of interest.
[0058]
(Supplementary Note 7) an image information inputting step of receiving input of information on a phase contrast image and information on a fluorescence image of cells in the same visual field;
An outline extraction step of extracting the outline of the cell based on the information on the phase difference image and the information on the fluorescence image whose input has been accepted by the image information input step,
An output step of outputting information on the contour extracted by the contour extraction step,
An image processing program for causing a computer to execute the following.
[0059]
(Supplementary Note 8) Image information input means for inputting information on a phase contrast image of cells and information on a fluorescent image in the same visual field,
Contour extraction means for extracting a contour of the cell based on information on a phase difference image and information on a fluorescence image inputted by the image information input means,
Output means for outputting information about the contour extracted by the contour extraction means,
An image processing apparatus comprising:
[0060]
【The invention's effect】
As described above, according to the present invention, it is possible to quickly and efficiently extract a cell contour without relying on visual observation, thereby automatically detecting a morphological change in a large amount in a predetermined period. There is an effect that an image processing method, an image processing program, and an image processing device that can be performed are obtained.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing an outline of cell contour extraction in an image processing method according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an example of a hardware configuration of the image processing apparatus according to the embodiment of the present invention;
FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus according to the embodiment of the present invention;
FIG. 4 is a flowchart showing the contents of processing of the image processing apparatus according to the embodiment of the present invention;
FIG. 5 is an explanatory diagram illustrating an example of a phase difference image.
FIG. 6 is an explanatory diagram showing a processing result of the phase difference image of FIG. 5;
FIG. 7 is an explanatory diagram showing an image of the GVF obtained from the edge in FIG. 6;
FIG. 8 is an explanatory diagram illustrating an example of a fluorescent image.
FIG. 9 is an explanatory diagram showing a luminance distribution in a horizontal direction at a certain position of a fluorescent image.
FIG. 10 is an explanatory diagram showing a luminance distribution at the same place as in FIG. 9;
FIG. 11 is an explanatory diagram showing a luminance distribution at the same place as in FIG. 9;
FIG. 12 is an explanatory diagram showing a processing result of the fluorescence image of FIG. 8;
FIG. 13 is an explanatory diagram showing a process of separating connected nuclei.
FIG. 14 is an explanatory diagram showing a result of extracting a contour of a cell from the images of FIGS. 5 and 8;
FIG. 15 is an explanatory view showing a process of extracting a contour of a cell (extraction failure).
FIG. 16 is an explanatory diagram showing another cell contour extraction process (successful extraction).
FIG. 17 is an explanatory diagram showing another cell contour extraction process (successful extraction).
[Explanation of symbols]
300 control unit 301 image external input unit 302 phase difference image processing unit 303 fluorescent image processing unit 304 contour extraction processing unit 310 storage unit 311 image information storage unit 312 edge information storage unit 313 initial contour information storage unit 314 contour image information storage unit 350 Optical microscope 351 Monitor 411 Phase difference image 412 Image edge 413 GVF field
421 Fluorescence image 422 Nuclear contour (initial contour of GVF Snakes)

Claims (5)

An image information input step of receiving information on a phase difference image of cells in the same field of view and information on a fluorescent image,
A contour extraction step of extracting a contour of the cell based on information on the phase difference image and the information on the fluorescence image received by the image information input step,
An output step of outputting information about the contour extracted by the contour extraction step,
An image processing method comprising: For the information on the phase difference image input is received by the image information input step, including a phase difference image processing step of extracting the edge of the cell contour,
The image processing method according to claim 1, wherein the contour extracting step extracts a contour of the cell based on an edge of the cell contour extracted in the phase difference image processing step. For the information on the fluorescence image input is received by the image information input step, including a fluorescence image processing step of extracting the outline of the nucleus using morphology,
The image processing method according to claim 1, wherein the contour extracting step extracts a contour of the cell based on a contour of a nucleus extracted in the fluorescent image processing step. An image information inputting step of receiving information on a phase difference image of cells in the same visual field and information on a fluorescent image,
A contour extraction step of extracting a contour of the cell based on information on the phase difference image and the information on the fluorescence image whose input has been accepted by the image information input step,
An output step of outputting information on the contour extracted by the contour extraction step,
An image processing program for causing a computer to execute the following. Image information input means for inputting information about a phase difference image of cells in the same field of view and information about a fluorescent image,
Contour extraction means for extracting a contour of the cell based on the information on the phase difference image and the information on the fluorescence image input by the image information input means,
Output means for outputting information about the contour extracted by the contour extraction means,
An image processing apparatus comprising:

Images