JP2009003016A - Microscope and image acquisition system - Google Patents

Microscope and image acquisition system Download PDF

Info

Publication number
JP2009003016A
JP2009003016A JP2007161551A JP2007161551A JP2009003016A JP 2009003016 A JP2009003016 A JP 2009003016A JP 2007161551 A JP2007161551 A JP 2007161551A JP 2007161551 A JP2007161551 A JP 2007161551A JP 2009003016 A JP2009003016 A JP 2009003016A
Authority
JP
Japan
Prior art keywords
light receiving
sample
receiving element
microscope
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2007161551A
Other languages
Japanese (ja)
Inventor
Hiroyuki Tsukamoto
宏之 塚本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nikon Corp
Original Assignee
Nikon Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nikon Corp filed Critical Nikon Corp
Priority to JP2007161551A priority Critical patent/JP2009003016A/en
Publication of JP2009003016A publication Critical patent/JP2009003016A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Microscoopes, Condenser (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To provide a microscope capable of imaging an object to be imaged which is too large to be held in field, with high resolution at a high speed in order to form a virtual slide image, and to provide an image acquiring system. <P>SOLUTION: The microscope has an objective 8, and a photodetector group 9 comprising a plurality of two-dimensional photodetectors 11 provided so as to fit in the field of the objective 8, and the observed image of a sample 6 is acquired, by imaging the area of the sample 6 in the filed of the objective 8 by two or more number of times, while the position of the sample 6 or the photodetector group 9 is changed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

本発明は、顕微鏡及び画像取得システムに関するものである。   The present invention relates to a microscope and an image acquisition system.

近年、被検試料の全体や注目する一部の観察画像を電子化(バーチャルスライド画像として保存)し、これを任意にモニタ上に表示して観察することが可能な所謂バーチャルスライドシステムが注目されている(例えば、特許文献1を参照。)。
特開平2−45734号公報
In recent years, a so-called virtual slide system that can digitize (save as a virtual slide image) the entire test sample and a part of the observation image of interest and display it on a monitor for observation is drawing attention. (For example, refer to Patent Document 1).
JP-A-2-45734

しかしながら、撮像素子の画素ピッチは数μm程度であるため、撮像の際、高倍率の対物レンズを用いることが必要となるが、高倍率の対物レンズはその視野が小さい。したがって、バーチャルスライド画像を作成するには、撮像対象を分割して撮像し、撮像した画像をタイリングすることが必要となる。このようにバーチャルスライドシステムでは、撮像対象を多数に分割して撮像する必要があるため、現状ではバーチャルスライド画像の作成に多くの時間を要しているという問題がある。   However, since the pixel pitch of the imaging element is about several μm, it is necessary to use a high-magnification objective lens for imaging, but the high-magnification objective lens has a small field of view. Therefore, in order to create a virtual slide image, it is necessary to divide the imaging target and capture images, and to tile the captured images. As described above, in the virtual slide system, it is necessary to divide the imaging target into a large number of images, and there is a problem that a large amount of time is currently required to create a virtual slide image.

そこで本発明は上記問題点に鑑みてなされたものであり、バーチャルスライド画像を作成するために撮像対象を高速に撮像することが可能な顕微鏡、及び画像取得システムを提供することを目的とする。   Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a microscope and an image acquisition system that can capture an imaging target at high speed in order to create a virtual slide image.

上記課題を解決するために本発明は、
対物レンズと、前記対物レンズの視野内に収まるように設けられた複数の二次元受光素子からなる受光素子群と、を有しており、
試料の観察画像は、前記対物レンズの視野内の当該試料の領域を、前記試料又は前記受光素子群の位置を変更しながら複数回撮像することで取得されることを特徴とする顕微鏡を提供する。
In order to solve the above problems, the present invention
An objective lens, and a light receiving element group composed of a plurality of two-dimensional light receiving elements provided so as to be within the field of view of the objective lens,
An observation image of a sample is obtained by imaging the region of the sample in the field of view of the objective lens a plurality of times while changing the position of the sample or the light receiving element group. .

また本発明は、
上記顕微鏡と、該顕微鏡で取得された前記試料の観察画像を処理する制御装置と、前記試料の観察画像を表示する表示装置とを有し、
前記顕微鏡は、前記試料の観察画像の取得を、前記試料を順に移動させながら繰り返し行い、
前記制御装置は、前記顕微鏡で取得された前記観察画像どうしを接続することで、前記試料の注目する領域又は前記試料全体の観察画像を作成し、これを前記表示装置に表示させることを特徴とする画像取得システムを提供する。
The present invention also provides
The microscope, a control device that processes the observation image of the sample acquired by the microscope, and a display device that displays the observation image of the sample,
The microscope repeatedly obtains an observation image of the sample while moving the sample in order,
The control device creates an observation image of the region of interest of the sample or the entire sample by connecting the observation images acquired by the microscope, and displays the observation image on the display device. An image acquisition system is provided.

本発明によれば、バーチャルスライド画像を作成するために撮像対象を高速に撮像することが可能な顕微鏡、及び画像取得システムを提供することができる。   ADVANTAGE OF THE INVENTION According to this invention, in order to produce a virtual slide image, the microscope which can image an imaging target at high speed, and an image acquisition system can be provided.

以下、本願の実施形態に係る画像取得システムについて添付図面に基づいて詳細に説明する。
はじめに、本画像取得システムの全体的な構成について図1を参照して説明する。図1は、本願の実施形態に係る画像取得システムの構成を示す図である。
本実施形態に係る画像取得システム1は、バーチャルスライドシステムに好適なシステムであって、図1に示すように顕微鏡2と、該顕微鏡2を制御する制御装置3と、表示装置4とからなる。
Hereinafter, an image acquisition system according to an embodiment of the present application will be described in detail with reference to the accompanying drawings.
First, the overall configuration of the image acquisition system will be described with reference to FIG. FIG. 1 is a diagram illustrating a configuration of an image acquisition system according to an embodiment of the present application.
The image acquisition system 1 according to the present embodiment is a system suitable for a virtual slide system, and includes a microscope 2, a control device 3 that controls the microscope 2, and a display device 4, as shown in FIG.

顕微鏡2は、光源5aとレンズ系5bを含む照明装置5と、該照明装置5側から順に、被検試料6を載置するための物体側ステージ7と、対物レンズ8と、受光素子群9を載置した像側ステージ10とを有してなる。
受光素子群9は、対物レンズ8の視野内に収まるように像側ステージ10上に配置された複数の二次元受光素子11からなる。なお、この受光素子群9については、後で詳細に説明する。そして、物体側ステージ7及び像側ステージ10は、それぞれ駆動部7a,10aを備えており、制御装置3の指示に基づきXY方向(水平方向)、及びZ方向(垂直方向即ち光軸方向)へ移動することが可能である。さらに、物体側ステージ7には、観察光を通過させるための開口が設けられている。
The microscope 2 includes an illuminating device 5 including a light source 5a and a lens system 5b, an object-side stage 7 on which the test sample 6 is placed in order from the illuminating device 5 side, an objective lens 8, and a light receiving element group 9. And an image side stage 10 on which is mounted.
The light receiving element group 9 includes a plurality of two-dimensional light receiving elements 11 disposed on the image side stage 10 so as to be within the field of view of the objective lens 8. The light receiving element group 9 will be described in detail later. The object-side stage 7 and the image-side stage 10 include drive units 7a and 10a, respectively, in the XY direction (horizontal direction) and the Z direction (vertical direction, that is, the optical axis direction) based on instructions from the control device 3. It is possible to move. Furthermore, the object side stage 7 is provided with an opening for allowing observation light to pass therethrough.

斯かる構成の下、本顕微鏡2において、照明装置5によって照明された被検試料6からの光は、対物レンズ8によって結像され、これを像側ステージ10上の受光素子群9が受光することで、被検試料6の画像が撮像されることとなる。ここで、受光素子群9における複数の二次元受光素子11は、それぞれの受光領域12どうしを隙間なく隣接して配置することがその構成上不可能であるため、得られる被検試料6の画像は、受光領域12どうしの隙間に対応する部分が抜け落ちたものとなってしまう。このため、この受光領域12どうしの隙間を埋めるべく本顕微鏡2では、像側ステージ10を駆動して受光素子群9の位置を変更しながら被検試料6の同一領域の撮像を複数回行う。そして、撮像された複数の画像どうしを制御装置3が接続することで、抜けのない被検試料6の観察画像を取得することができる。   Under such a configuration, in the microscope 2, the light from the test sample 6 illuminated by the illumination device 5 is imaged by the objective lens 8, and this is received by the light receiving element group 9 on the image side stage 10. Thus, an image of the test sample 6 is taken. Here, since the plurality of two-dimensional light receiving elements 11 in the light receiving element group 9 cannot be disposed adjacent to each other without any gap between the respective light receiving regions 12, the obtained image of the test sample 6 is obtained. Will be a portion where the portion corresponding to the gap between the light receiving regions 12 is missing. Therefore, in the present microscope 2, the same region of the test sample 6 is imaged a plurality of times while driving the image side stage 10 to change the position of the light receiving element group 9 in order to fill the gap between the light receiving regions 12. Then, the control device 3 connects a plurality of captured images so that an observation image of the test sample 6 without missing can be acquired.

次に、本画像取得システム1の特徴的な構成をより詳細に説明する。
通常、顕微鏡の視野数は20〜25程度であるため、この視野をカバーし得るサイズの受光素子は存在する。しかしながら、被検試料の撮像効率を考慮すれば、対物レンズには実視野が大きな比較的低倍率のものを用いることが好ましい。例えば、対物レンズの倍率が20倍である場合、被検試料上での長さ0.25μmは像面内での長さ5μmに相当するが、視野数25は5000pixに相当することとなるため、1個の二次元受光素子によって視野全体をカバーすることは困難である。そして、被検試料の撮像の更なる高速化を考慮した場合には、顕微鏡の視野数を大きくすることが有効であるものの、大きくした視野全体をカバー可能な受光素子を開発することは困難である。
Next, the characteristic configuration of the image acquisition system 1 will be described in more detail.
Usually, since the number of fields of the microscope is about 20 to 25, there is a light receiving element having a size capable of covering this field of view. However, considering the imaging efficiency of the test sample, it is preferable to use an objective lens having a large real field of view and a relatively low magnification. For example, when the magnification of the objective lens is 20 times, a length of 0.25 μm on the test sample corresponds to a length of 5 μm in the image plane, but a field number of 25 corresponds to 5000 pix. It is difficult to cover the entire field of view with a single two-dimensional light receiving element. In consideration of further speeding up of imaging of the test sample, it is effective to increase the number of fields of the microscope, but it is difficult to develop a light receiving element that can cover the entire enlarged field of view. is there.

そこで本画像取得システム1は、上述したように対物レンズ8の視野内に複数の二次元受光素子11からなる受光素子群9を配置し、一度に複数の画像を撮像する構成としており、これによって撮像に要する時間の大幅な短縮化を図ることができる。
また、受光素子群9における複数の二次元受光素子11は、上述のように受光領域12どうしを隙間なく隣接して配置することがその構成上不可能であるため、得られる被検試料6の画像は、受光領域12どうしの隙間に対応する部分が抜け落ちたものとなってしまう。そこで本画像取得システム1は、この受光領域12どうしの隙間を埋めるべく、像側ステージ10を駆動して二次元受光素子11の位置を変更しながら被検試料6の位置を変更することなく撮像を複数回行うことで、抜けのない被検試料6の観察画像を取得する構成としている。
In view of this, the image acquisition system 1 has a configuration in which the light receiving element group 9 including the plurality of two-dimensional light receiving elements 11 is arranged in the field of view of the objective lens 8 as described above, and a plurality of images are captured at a time. The time required for imaging can be greatly shortened.
Further, since the plurality of two-dimensional light receiving elements 11 in the light receiving element group 9 cannot be disposed adjacent to each other without any gap as described above, the obtained specimen 6 to be obtained is not suitable. In the image, the part corresponding to the gap between the light receiving areas 12 is missing. Therefore, the present image acquisition system 1 captures an image without changing the position of the test sample 6 while driving the image side stage 10 to change the position of the two-dimensional light receiving element 11 so as to fill the gap between the light receiving areas 12. Is performed multiple times to obtain an observation image of the test sample 6 without omission.

また、本画像取得システム1において、前述のように被検試料6の位置を変更することなく撮像を二次元受光素子11の位置を変更しながら複数回行う際には、二次元受光素子11どうしを後述する適切な所定の間隔をあけて配置しておき、二次元受光素子11の位置を変更する前の受光領域12と変更した後の受光領域12を一部重複させて被検試料6の撮像を行うことが好ましい。これは、二次元受光素子11の設置位置の誤差、各ステージ7,10によって二次元受光素子11や被検試料6を移動させた際の移動誤差、及び対物レンズ8等の歪曲収差等の影響を解消するためである。また、このように被検試料6の位置を変更せずに同一領域の撮像を行うに際して、二次元受光素子11の位置変更前後で受光領域12を一部重複させる、即ち受光領域12に重複領域を設けることによって、被検試料6の同じ部分が複数の二次元受光素子11で撮像されることとなる。したがって制御装置3は、この情報を用いて画像どうしの位置合わせを行い高精度に接続することが可能となる。   In the image acquisition system 1, when imaging is performed a plurality of times while changing the position of the two-dimensional light receiving element 11 without changing the position of the test sample 6 as described above, the two-dimensional light receiving elements 11 are connected to each other. Are arranged at an appropriate predetermined interval, which will be described later, and the light receiving region 12 before the position of the two-dimensional light receiving element 11 is changed and the light receiving region 12 after the change are partially overlapped. It is preferable to perform imaging. This is because of an influence of an installation position error of the two-dimensional light receiving element 11, a movement error when the two-dimensional light receiving element 11 and the test sample 6 are moved by the stages 7, 10, and distortion aberration of the objective lens 8 or the like. This is to eliminate the problem. Further, when imaging the same region without changing the position of the test sample 6 in this way, the light receiving region 12 is partially overlapped before and after the position change of the two-dimensional light receiving element 11, that is, the light receiving region 12 is overlapped. As a result, the same portion of the test sample 6 is imaged by the plurality of two-dimensional light receiving elements 11. Therefore, the control device 3 can align images with each other using this information and can connect with high accuracy.

ここで、当該重複領域は、大き過ぎれば撮像効率が低下することとなってしまうため、極力小さいことが好ましい。通常、二次元受光素子における受光領域の各辺の長さは数mm以上であるのに対し、設置誤差や移動誤差は約1mm以内に収めることができ、また歪曲収差の大きさは数%以内である。このため、必要な重複領域は、受光領域12の一辺当たり当該受光領域12の面積の15%程度、即ち両辺で30%程度で十分である。   Here, if the overlapping region is too large, the imaging efficiency is reduced, and therefore it is preferable that the overlapping region be as small as possible. Normally, the length of each side of the light receiving area in a two-dimensional light receiving element is several millimeters or more, but the installation error and movement error can be kept within about 1 mm, and the magnitude of distortion is within several percent. It is. For this reason, it is sufficient that the necessary overlapping region is about 15% of the area of the light receiving region 12 per side of the light receiving region 12, that is, about 30% on both sides.

したがって、本画像取得システム1において、必要な重複領域を確保しながら撮像効率の向上を図るためには、受光素子群9を構成する複数の二次元受光素子11の受光領域12どうしの上記所定の間隔(各受光素子の中心間の距離)が、以下の条件式(1),(2)を満足することが望ましい。
(1) 0.7p<u<p
(2) 0.7q<v<q
但し、
p :前記複数の二次元受光素子の前記受光領域のX方向の大きさ
q :前記複数の二次元受光素子の前記受光領域のY方向の大きさ
u :X方向において対向する前記受光領域どうしの間隔(各受光素子の中心間の距離)
v :Y方向において対向する前記受光領域どうしの間隔(各受光素子の中心間の距離)
Therefore, in the present image acquisition system 1, in order to improve the imaging efficiency while securing a necessary overlapping area, the predetermined predetermined values of the light receiving areas 12 of the plurality of two-dimensional light receiving elements 11 constituting the light receiving element group 9 are used. It is desirable that the distance (the distance between the centers of the light receiving elements) satisfies the following conditional expressions (1) and (2).
(1) 0.7p <u <p
(2) 0.7q <v <q
However,
p: size in the X direction of the light receiving region of the plurality of two-dimensional light receiving elements q: size in the Y direction of the light receiving region of the plurality of two-dimensional light receiving elements u: between the light receiving regions facing each other in the X direction Spacing (distance between the centers of each light receiving element)
v: Distance between the light receiving regions facing each other in the Y direction (distance between the centers of the light receiving elements)

なお、本画像取得システム1において、二次元受光素子11の受光領域12どうしの所定の間隔が、上記条件式(1),(2)の上限値を上回ると、重複領域が無くなり隙間のない画像接続が不可能になってしまう。また、上記条件式(1),(2)の下限値を下回ると、重複領域が大きくなり過ぎて、撮像効率が低下してしまう。   In the present image acquisition system 1, when the predetermined interval between the light receiving regions 12 of the two-dimensional light receiving element 11 exceeds the upper limit value of the conditional expressions (1) and (2), there is no overlapping region and there is no gap. Connection becomes impossible. If the lower limit value of the conditional expressions (1) and (2) is not reached, the overlapping area becomes too large and the imaging efficiency is lowered.

ここで、二次元受光素子11の外形に比して受光領域12が小さい場合には、上記条件式(1),(2)を満足することができない場合がある。そこで斯かる場合には、本画像取得システム1は、以下の条件式(3),(4)を満足することが望ましい。
(3) 0.7ep<u<ep
(4) 0.7fq<v<fq
但し、
p :前記複数の二次元受光素子の前記受光領域のX方向の大きさ
q :前記複数の二次元受光素子の前記受光領域のY方向の大きさ
u :X方向において対向する前記受光領域どうしの間隔
v :Y方向において対向する前記受光領域どうしの間隔
e,f:4以下の自然数
Here, when the light receiving region 12 is smaller than the outer shape of the two-dimensional light receiving element 11, the conditional expressions (1) and (2) may not be satisfied. Therefore, in such a case, it is desirable that the image acquisition system 1 satisfies the following conditional expressions (3) and (4).
(3) 0.7ep <u <ep
(4) 0.7fq <v <fq
However,
p: size in the X direction of the light receiving region of the plurality of two-dimensional light receiving elements q: size in the Y direction of the light receiving region of the plurality of two-dimensional light receiving elements u: between the light receiving regions facing each other in the X direction Interval v: Interval e between the light receiving areas facing each other in the Y direction, f: Natural number of 4 or less

なお、斯かる場合には、X方向における受光領域どうしの隙間を埋めるためには、像側ステージ10によって二次元受光素子11をX方向へe回移動させながら撮像を行い、Y方向における隙間を埋めるためには、二次元受光素子11をY方向へf回移動させながら撮像を行う必要がある。このため、e,fが大きくなり過ぎると、視野内において受光領域の占める比率が低下し、撮像効率の悪化の影響が大きくなるため、e,fは4以下の自然数とする。なお、本画像取得システム1において、二次元受光素子11の受光領域12どうしの所定の間隔(各受光素子の中心間の距離)が、当該条件式(3),(4)の上限値又は下限値を越えると、上記条件式(1),(2)を満足しない場合と同様の結果を招くこととなってしまう。   In such a case, in order to fill the gap between the light receiving regions in the X direction, the image side stage 10 performs imaging while moving the two-dimensional light receiving element 11 in the X direction e times, and the gap in the Y direction is set. In order to fill, it is necessary to perform imaging while moving the two-dimensional light receiving element 11 f times in the Y direction. For this reason, if e and f become too large, the ratio of the light receiving area in the field of view decreases and the influence of deterioration in imaging efficiency increases, so e and f are natural numbers of 4 or less. In the image acquisition system 1, the predetermined interval between the light receiving regions 12 of the two-dimensional light receiving element 11 (the distance between the centers of the light receiving elements) is the upper limit value or lower limit of the conditional expressions (3) and (4). If the value is exceeded, the same result as in the case where the conditional expressions (1) and (2) are not satisfied is brought about.

以上の条件を満足する本画像取得システム1の数値実施例を以下に示す。
本画像取得システム1は、被検試料6の10×10mmの範囲を撮像対象とし、これを0.25μmの撮像分解能で撮像するものとする。
顕微鏡2の受光素子群9を構成する二次元受光素子11には、5μmピッチで1392×1040個の画素を有するものを採用しており、受光領域12のサイズは6.96mm(=X方向の大きさp)×5.20mm(=Y方向の大きさq)である。
これに対し対物レンズ8には、前述の撮像分解能を達成するために、倍率が20倍で視野直径が2.4mmのものを採用している。
A numerical example of the image acquisition system 1 that satisfies the above conditions is shown below.
This image acquisition system 1 assumes that a 10 × 10 mm range of the sample 6 to be imaged and images this with an imaging resolution of 0.25 μm.
The two-dimensional light receiving elements 11 constituting the light receiving element group 9 of the microscope 2 are those having 1392 × 1040 pixels at a pitch of 5 μm, and the size of the light receiving region 12 is 6.96 mm (= X direction) Size p) × 5.20 mm (= size q in the Y direction).
On the other hand, the objective lens 8 has a magnification of 20 times and a field diameter of 2.4 mm in order to achieve the above-described imaging resolution.

受光素子群9は、前述したサイズの二次元受光素子11を対物レンズ8の視野内に収まるように像側ステージ10上に長方形状に3行3列、計9個配置してなる。詳しくは、この9個の二次元受光素子11は、それぞれの受光領域12が図2に示す撮像領域Hのポジションaに位置するように配置されており、X方向において対向する受光領域12どうしの間隔uが12.8mm、Y方向において対向する受光領域12どうしの間隔vが9.6mmとなるように相対位置が固定されている。
なお、図2は、本願の実施形態に係る画像取得システム1における顕微鏡2の撮像領域H、受光素子群9を構成する各二次元受光素子11の受光領域12の配置、及び撮像時の当該受光領域12の移動方向を示す図である。また、図1中には、簡単のために二次元受光素子11は4つのみ示されている。
この場合、像面上での視野直径は48mmとなり、当該対物レンズ8のイメージサークルは、1つの二次元受光素子11の受光領域12の大きさの6×6倍に相当する。
The light receiving element group 9 is formed by arranging a total of nine two-dimensional light receiving elements 11 having the above-mentioned size in a rectangular shape on three rows and three columns on the image side stage 10 so as to be within the field of view of the objective lens 8. Specifically, the nine two-dimensional light receiving elements 11 are arranged so that each light receiving region 12 is positioned at a position a of the imaging region H shown in FIG. 2, and the light receiving regions 12 facing each other in the X direction are arranged. The relative position is fixed so that the distance u is 12.8 mm and the distance v between the light receiving regions 12 facing each other in the Y direction is 9.6 mm.
2 shows the imaging area H of the microscope 2 in the image acquisition system 1 according to the embodiment of the present application, the arrangement of the light receiving areas 12 of the two-dimensional light receiving elements 11 constituting the light receiving element group 9, and the light reception at the time of imaging. It is a figure which shows the moving direction of the area | region 12. FIG. In FIG. 1, only four two-dimensional light receiving elements 11 are shown for simplicity.
In this case, the field diameter on the image plane is 48 mm, and the image circle of the objective lens 8 corresponds to 6 × 6 times the size of the light receiving region 12 of one two-dimensional light receiving element 11.

以上の構成の下、本画像取得システム1で被検試料6の撮像を行う際の撮像手順を説明する。
予め使用者は、物体側ステージ7によって被検試料6を移動させて位置を調整し、その位置を固定する。この下で本画像取得システム1は、像側ステージ10によって図2のポジションaに位置する各二次元受光素子11を−Y方向(ポジションb)、X方向(ポジションc)、Y方向(ポジションd)へと順に移動させ、各ポジションにおいて全ての二次元受光素子11が被検試料6の撮像を行う。
このように本画像取得システム1は、被検試料6の対物レンズ8の視野内の特定領域の撮像を受光素子群9の位置を変更(又は被検試料6の位置を変更)しながら計4回行うことで、二次元受光素子11の受光領域12どうしの隙間をカバーして撮像領域H全体の撮像が実現されるため、抜けのない被検試料6の観察画像を取得することができる。
An imaging procedure when imaging the test sample 6 with the image acquisition system 1 under the above configuration will be described.
The user adjusts the position by moving the test sample 6 with the object side stage 7 in advance, and fixes the position. Under this condition, the image acquisition system 1 causes the image side stage 10 to move each two-dimensional light receiving element 11 positioned at the position a in FIG. 2 in the −Y direction (position b), the X direction (position c), and the Y direction (position d). ) In order, and all the two-dimensional light receiving elements 11 image the test sample 6 at each position.
Thus, the present image acquisition system 1 captures a specific area within the field of view of the objective lens 8 of the test sample 6 while changing the position of the light receiving element group 9 (or changing the position of the test sample 6). Since the imaging of the entire imaging region H is realized by covering the gap between the light receiving regions 12 of the two-dimensional light receiving element 11, the observation image of the test sample 6 with no omission can be acquired.

なお、図2には、各二次元受光素子11のポジションの変更が、ポジション変更後の受光領域12が変更前の受光領域12と重複しないように行われているように描かれている。しかしながら実際には、各二次元受光素子11のポジションの変更は、上述したように像側ステージ10の移動誤差等の影響を解消するべく、ポジション変更前後の受光領域12どうしが一部重複するように、詳しくは重複領域が受光領域12の15%程度となるように行われる。   In FIG. 2, the change of the position of each two-dimensional light receiving element 11 is depicted so that the light receiving area 12 after the position change does not overlap with the light receiving area 12 before the change. However, in actuality, when the position of each two-dimensional light receiving element 11 is changed, the light receiving areas 12 before and after the position change partially overlap in order to eliminate the influence of the movement error of the image side stage 10 as described above. More specifically, the overlapping area is about 15% of the light receiving area 12.

次に、本画像取得システム1は、物体側ステージ7によって被検試料6の位置を変更して前述した撮像を再度行い、これを繰り返すことで被検試料6全体の撮像を行う。
ここで、本実施形態において、図2に示した本画像取得システム1の撮像領域Hの大きさは被検試料6上に換算して1.92×1.44mm(重複を除いた値)であり、撮像対象は上述したように被検試料6の10×10mmの領域である。
したがって、本画像取得システム1は、被検試料6の位置の変更をして行った撮像を1セットとすると6×7回のセットで繰り返し実施することで、撮像対象全体を撮像できることとなる。なお、本実施形態において、物体側ステージ7による被検試料6の位置の変更は、図3に示す順序で行われる。図3は、本願の実施形態に係る画像取得システム1の顕微鏡2において、被検試料6の位置の変更順序を示す図である。
以上より、本画像取得システム1は、単一の受光素子を用いた従来の顕微鏡に比して、約9倍の速度で被検試料6の撮像を行うことが可能となる。
Next, the image acquisition system 1 changes the position of the test sample 6 by the object side stage 7 and performs the above-described imaging again, and repeats this to perform imaging of the entire test sample 6.
Here, in the present embodiment, the size of the imaging region H of the present image acquisition system 1 shown in FIG. 2 is 1.92 × 1.44 mm (value excluding duplication) when converted on the test sample 6. The imaging target is a 10 × 10 mm region of the test sample 6 as described above.
Therefore, the image acquisition system 1 can capture the entire imaging target by repeatedly performing 6 × 7 sets of imaging performed by changing the position of the test sample 6 as one set. In the present embodiment, the change of the position of the test sample 6 by the object side stage 7 is performed in the order shown in FIG. FIG. 3 is a diagram showing a change order of the position of the test sample 6 in the microscope 2 of the image acquisition system 1 according to the embodiment of the present application.
As described above, the present image acquisition system 1 can capture an image of the test sample 6 at a speed about nine times that of a conventional microscope using a single light receiving element.

本画像取得システム1において制御装置3は、各二次元受光素子11で撮像された画像を当該制御装置3内の不図示の記憶部に保存する際には、各二次元受光素子11の相互位置の情報や、物体側ステージ7及び像側ステージ10の移動量の情報等に基づいて画像どうしをタイリングし、1枚の観察画像として保存する。これにより、本画像取得システム1は、表示装置4に観察画像を表示する際には、比較的高倍率でありシームレスな見やすいマクロ観察画像を表示することが可能となる。これは特に、被検試料6の広い領域の観察画像を表示する場合や、被検試料6の狭い領域を拡大して表示する場合に有効である。   In the image acquisition system 1, when the control device 3 saves an image captured by each two-dimensional light receiving element 11 in a storage unit (not shown) in the control device 3, the mutual position of each two-dimensional light receiving element 11. The images are tiled based on the above information, the information on the movement amounts of the object side stage 7 and the image side stage 10, and the like, and stored as one observation image. As a result, when displaying the observation image on the display device 4, the present image acquisition system 1 can display a macro observation image that is relatively high in magnification and seamless and easy to see. This is particularly effective when an observation image of a wide area of the test sample 6 is displayed or when a narrow area of the test sample 6 is enlarged and displayed.

なお、本画像取得システム1は、各二次元受光素子11で撮像された画像を接続して比較的少ない枚数の画像として保存、又は画像を接続せずに個別に保存して、表示装置4に観察画像を表示する際には、制御装置3がその都度画像を見かけ上タイリングして表示する構成とすることもできる。この構成は、記憶部に保存される画像の数は多くなるものの、巨大な画像を作成せずに済むという利点がある。   The image acquisition system 1 connects the images picked up by the two-dimensional light receiving elements 11 and stores them as a relatively small number of images, or stores them individually without connecting the images, and stores them in the display device 4. When the observation image is displayed, the control device 3 may be configured so that the image is apparently tiled and displayed each time. Although this configuration increases the number of images stored in the storage unit, there is an advantage that it is not necessary to create a huge image.

また、本画像取得システム1では、顕微鏡2の受光素子群9は上述した構成に限られるものではない。例えば、2行1列の二次元受光素子11に3行1列の二次元受光素子11をY方向へずらして配置してなる組み合わせを、対物レンズ8の視野内に収まるように像側ステージ10上に3組並べた計15個の二次元受光素子11によって受光素子群9を構成することもできる。詳しくは、この15個の二次元受光素子11は、それぞれの受光領域12が図4に示す撮像領域Hのポジションaに位置するように配置されており、X方向において対向する受光領域12どうしの間隔uが12.8mm、Y方向において対向する受光領域12どうしの間隔vが7.2mmとなるように相対位置が固定されている。なお、図4に示すように15個の二次元受光素子11のうちの3個は、撮像領域外に位置している。   In the image acquisition system 1, the light receiving element group 9 of the microscope 2 is not limited to the above-described configuration. For example, the image side stage 10 so that a combination of the two-dimensional light-receiving element 11 of 2 rows and 1 column and the two-dimensional light-receiving element 11 of 3 rows and 1 column arranged in the Y direction is within the field of view of the objective lens 8. The light receiving element group 9 can also be constituted by a total of 15 two-dimensional light receiving elements 11 arranged in three groups. Specifically, the fifteen two-dimensional light receiving elements 11 are arranged such that each light receiving region 12 is positioned at a position a of the imaging region H shown in FIG. 4, and the light receiving regions 12 facing each other in the X direction are arranged. The relative position is fixed so that the distance u is 12.8 mm and the distance v between the light receiving regions 12 facing each other in the Y direction is 7.2 mm. As shown in FIG. 4, three of the 15 two-dimensional light receiving elements 11 are located outside the imaging region.

斯かる構成の場合、本画像取得システム1による被検試料6の撮像は、像側ステージ10によって図4のポジションaに位置する各二次元受光素子11をY方向(ポジションb)、Y方向(ポジションc)へ順に移動させ、各ポジションにおいて全ての二次元受光素子11が被検試料6の撮像を行うことで達成される。
したがって、本画像取得システム1は、各二次元受光素子11の受光領域12どうしがY方向から見て隙間なく配置されており、受光素子群9の移動方向を一方向のみとすることができるため、像側ステージ10の機構を簡略化することができ、また移動時間の短縮化を図ることもできる。また、被検試料6の同一領域の撮像を計3回行うことで、抜けのない被検試料6の観察画像を得ることができるため、単一の受光素子を用いた従来の顕微鏡に比して、約12倍の速度で被検試料6の撮像を行うことができる。
なお、図4は、本願の実施形態に係る画像取得システム1における顕微鏡2の撮像領域H、受光素子群9を構成する各二次元受光素子11の受光領域12の配置の変形例、及び撮像時の当該受光領域12の移動方向を示す図である。
In the case of such a configuration, imaging of the test sample 6 by the image acquisition system 1 is performed by moving each two-dimensional light receiving element 11 positioned at the position a in FIG. 4 by the image side stage 10 in the Y direction (position b) and the Y direction ( This is achieved by sequentially moving to the position c) and imaging the test sample 6 by all the two-dimensional light receiving elements 11 at each position.
Therefore, in the present image acquisition system 1, the light receiving regions 12 of the two-dimensional light receiving elements 11 are arranged with no gap when viewed from the Y direction, and the moving direction of the light receiving element group 9 can be only one direction. The mechanism of the image side stage 10 can be simplified, and the moving time can be shortened. In addition, since an observation image of the test sample 6 without omission can be obtained by performing the imaging of the same region of the test sample 6 a total of three times, compared with a conventional microscope using a single light receiving element. Thus, imaging of the test sample 6 can be performed at about 12 times the speed.
4 shows a modification of the arrangement of the imaging region H of the microscope 2 and the light receiving regions 12 of the two-dimensional light receiving elements 11 constituting the light receiving element group 9 in the image acquisition system 1 according to the embodiment of the present application, and during imaging. It is a figure which shows the moving direction of the said light reception area | region 12.

また、本画像取得システム1では、上述のように被検試料6の撮像は、受光素子群9を構成する各二次元受光素子11の受光領域12どうしの隙間をカバーするために、像側ステージ10によって受光素子群9の位置を変更させながら被検試料6の同一領域を複数回撮像することで行われる。しかしながら本画像取得システム1の構成はこれに限られず、像側ステージ10の位置を固定し、物体側ステージ7によって被検試料6の位置を変更させながら被検試料6の同一領域を複数回撮像する構成としてもよい。   In the image acquisition system 1, as described above, the imaging of the test sample 6 is performed on the image-side stage in order to cover the gap between the light receiving regions 12 of the two-dimensional light receiving elements 11 constituting the light receiving element group 9. 10 is performed by imaging the same region of the test sample 6 a plurality of times while changing the position of the light receiving element group 9. However, the configuration of the image acquisition system 1 is not limited to this, and the same region of the test sample 6 is imaged a plurality of times while the position of the image side stage 10 is fixed and the position of the test sample 6 is changed by the object side stage 7. It is good also as composition to do.

以上、本実施形態によれば、バーチャルスライド画像を作成するために、視野内に収まらない大きな撮像対象を高解像度で高速に撮像することが可能な顕微鏡、及び画像取得システムを実現することができる。またこれにより、観察画像の電子化の高速化が図られることとなる。   As described above, according to the present embodiment, it is possible to realize a microscope and an image acquisition system that can capture a large imaging target that does not fit in the field of view at a high speed in order to create a virtual slide image. . As a result, the observation image can be digitized at high speed.

本願の実施形態に係る画像取得システム1の構成を示す図である。It is a figure which shows the structure of the image acquisition system 1 which concerns on embodiment of this application. 本願の実施形態に係る画像取得システム1における顕微鏡2の撮像領域H、受光素子群9を構成する各二次元受光素子11の受光領域12の配置、及び撮像時の当該受光領域12の移動方向を示す図である。The imaging area H of the microscope 2 in the image acquisition system 1 according to the embodiment of the present application, the arrangement of the light receiving areas 12 of the two-dimensional light receiving elements 11 constituting the light receiving element group 9, and the movement direction of the light receiving area 12 at the time of imaging. FIG. 本願の実施形態に係る画像取得システム1の顕微鏡2において、被検試料6の位置の変更順序を示す図である。It is a figure which shows the change order of the position of the test sample 6 in the microscope 2 of the image acquisition system 1 which concerns on embodiment of this application. 本願の実施形態に係る画像取得システム1における顕微鏡2の撮像領域H、受光素子群9を構成する各二次元受光素子11の受光領域12の配置の変形例、及び撮像時の当該受光領域12の移動方向を示す図である。The imaging region H of the microscope 2 in the image acquisition system 1 according to the embodiment of the present application, the modification of the arrangement of the light receiving regions 12 of the two-dimensional light receiving elements 11 constituting the light receiving element group 9, and the light receiving region 12 at the time of imaging It is a figure which shows a moving direction.

符号の説明Explanation of symbols

1 画像取得システム
2 顕微鏡
3 制御装置
4 表示装置
5 照明装置
6 被検試料
7 物体側ステージ
8 対物レンズ
9 受光素子群
10 像側ステージ
11 受光素子
12 受光領域
H 撮像領域
DESCRIPTION OF SYMBOLS 1 Image acquisition system 2 Microscope 3 Control apparatus 4 Display apparatus 5 Illumination apparatus 6 Test sample 7 Object side stage 8 Objective lens 9 Light receiving element group 10 Image side stage 11 Light receiving element 12 Light receiving area H Imaging area

Claims (6)

対物レンズと、前記対物レンズの視野内に収まるように設けられた複数の二次元受光素子からなる受光素子群と、を有しており、
試料の観察画像は、前記対物レンズの視野内の当該試料の領域を、前記試料又は前記受光素子群の位置を変更しながら複数回撮像することで取得されることを特徴とする顕微鏡。
An objective lens, and a light receiving element group composed of a plurality of two-dimensional light receiving elements provided so as to be within the field of view of the objective lens,
The observation image of a sample is acquired by imaging the region of the sample in the field of view of the objective lens a plurality of times while changing the position of the sample or the light receiving element group.
前記受光素子群を載置してXY方向へ移動させる受光素子用ステージ、又は、前記試料を載置してXY方向へ移動させる試料用ステージを有していることを特徴とする請求項1に記載の顕微鏡。   2. A light receiving element stage on which the light receiving element group is placed and moved in the XY direction, or a sample stage on which the sample is placed and moved in the XY direction. The microscope described. 前記試料の観察画像の取得に際する前記受光素子群の位置の変更は、変更する前の前記受光領域と変更した後の前記受光領域とが一部重複するように行われることを特徴とする請求項1又は請求項2に記載の顕微鏡。   The change of the position of the light receiving element group when acquiring the observation image of the sample is performed so that the light receiving area before the change and the light receiving area after the change partially overlap. The microscope according to claim 1 or 2. 前記複数の二次元受光素子は、受光領域どうしが所定の間隔をあけて配置されており、
前記受光素子群を構成する前記複数の二次元受光素子の前記受光領域どうしの前記所定の間隔は、以下の条件式を満足することを特徴とする請求項1から請求項3に記載の顕微鏡。
0.7ep<u<ep
0.7fq<v<fq
但し、
p :前記複数の二次元受光素子の前記受光領域のX方向の大きさ
q :前記複数の二次元受光素子の前記受光領域のY方向の大きさ
u :X方向において対向する前記受光領域どうしの間隔
v :Y方向において対向する前記受光領域どうしの間隔
e,f:4以下の自然数
In the plurality of two-dimensional light receiving elements, light receiving regions are arranged with a predetermined interval between them,
The microscope according to any one of claims 1 to 3, wherein the predetermined interval between the light receiving regions of the plurality of two-dimensional light receiving elements constituting the light receiving element group satisfies the following conditional expression.
0.7ep <u <ep
0.7fq <v <fq
However,
p: size in the X direction of the light receiving region of the plurality of two-dimensional light receiving elements q: size in the Y direction of the light receiving region of the plurality of two-dimensional light receiving elements u: between the light receiving regions facing each other in the X direction Interval v: Interval e between the light receiving areas facing each other in the Y direction, f: Natural number of 4 or less
前記複数の二次元受光素子は、前記受光領域どうしがX方向又はY方向から見て隙間なく配置されており、
前記試料の観察画像は、前記対物レンズの視野内の当該試料の領域を、前記試料又は前記受光素子群の位置を一方向へのみ変更しながら複数回撮像することで取得されることを特徴とする請求項1から請求項4のいずれか1項に記載の顕微鏡。
In the plurality of two-dimensional light receiving elements, the light receiving regions are arranged without gaps when viewed from the X direction or the Y direction,
The observation image of the sample is acquired by imaging the region of the sample in the field of view of the objective lens a plurality of times while changing the position of the sample or the light receiving element group only in one direction. The microscope according to any one of claims 1 to 4.
請求項1から請求項5のいずれか1項に記載の顕微鏡と、該顕微鏡で取得された前記試料の観察画像を処理する制御装置と、前記試料の観察画像を表示する表示装置とを有し、
前記顕微鏡は、前記試料の観察画像の取得を、前記試料を順に移動させながら繰り返し行い、
前記制御装置は、前記顕微鏡で取得された前記観察画像どうしを接続することで、前記試料の注目する領域又は前記試料全体の観察画像を作成し、これを前記表示装置に表示させることを特徴とする画像取得システム。
A microscope according to any one of claims 1 to 5, a control device that processes an observation image of the sample acquired by the microscope, and a display device that displays the observation image of the sample. ,
The microscope repeatedly obtains an observation image of the sample while moving the sample in order,
The control device creates an observation image of the region of interest of the sample or the entire sample by connecting the observation images acquired by the microscope, and displays the observation image on the display device. Image acquisition system.
JP2007161551A 2007-06-19 2007-06-19 Microscope and image acquisition system Pending JP2009003016A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2007161551A JP2009003016A (en) 2007-06-19 2007-06-19 Microscope and image acquisition system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2007161551A JP2009003016A (en) 2007-06-19 2007-06-19 Microscope and image acquisition system

Publications (1)

Publication Number Publication Date
JP2009003016A true JP2009003016A (en) 2009-01-08

Family

ID=40319501

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2007161551A Pending JP2009003016A (en) 2007-06-19 2007-06-19 Microscope and image acquisition system

Country Status (1)

Country Link
JP (1) JP2009003016A (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011148636A1 (en) * 2010-05-27 2011-12-01 Canon Kabushiki Kaisha Image pickup apparatus
US20120056991A1 (en) * 2010-09-04 2012-03-08 Leica Microsystems (Schweiz) Ag Image sensor, video camera, and microscope
WO2012033219A1 (en) * 2010-09-10 2012-03-15 Canon Kabushiki Kaisha Imaging apparatus
WO2012056920A1 (en) 2010-10-29 2012-05-03 Canon Kabushiki Kaisha Microscope, image acquisition apparatus, and image acquisition system
JP2012138891A (en) * 2010-12-08 2012-07-19 Canon Inc Imaging apparatus
WO2013051147A1 (en) * 2011-10-07 2013-04-11 キヤノン株式会社 Image acquisition apparatus adjustment method, image acquisition apparatus, and image acquisition apparatus manufacturing method
JP2013130686A (en) * 2011-12-21 2013-07-04 Canon Inc Imaging apparatus
WO2013100029A1 (en) * 2011-12-27 2013-07-04 キヤノン株式会社 Image processing device, image display system, image processing method, and image processing program
US8605144B2 (en) 2010-09-08 2013-12-10 Canon Kabushiki Kaisha Imaging apparatus
US8654223B2 (en) 2010-06-15 2014-02-18 Canon Kabushiki Kaisha Image pickup apparatus
US9386210B2 (en) 2011-03-18 2016-07-05 Canon Kabushiki Kaisha Imaging apparatus

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09281405A (en) * 1996-04-17 1997-10-31 Olympus Optical Co Ltd Microscopic system
JP2003058868A (en) * 2001-08-21 2003-02-28 Dainippon Printing Co Ltd Image input device
JP2007132757A (en) * 2005-11-09 2007-05-31 Osaka City Visual examination method and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09281405A (en) * 1996-04-17 1997-10-31 Olympus Optical Co Ltd Microscopic system
JP2003058868A (en) * 2001-08-21 2003-02-28 Dainippon Printing Co Ltd Image input device
JP2007132757A (en) * 2005-11-09 2007-05-31 Osaka City Visual examination method and device

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011248123A (en) * 2010-05-27 2011-12-08 Canon Inc Imaging apparatus
US9261691B2 (en) 2010-05-27 2016-02-16 Canon Kabushiki Kaisha Image pickup apparatus
WO2011148636A1 (en) * 2010-05-27 2011-12-01 Canon Kabushiki Kaisha Image pickup apparatus
US8654223B2 (en) 2010-06-15 2014-02-18 Canon Kabushiki Kaisha Image pickup apparatus
US20120056991A1 (en) * 2010-09-04 2012-03-08 Leica Microsystems (Schweiz) Ag Image sensor, video camera, and microscope
US8605144B2 (en) 2010-09-08 2013-12-10 Canon Kabushiki Kaisha Imaging apparatus
JP2012078784A (en) * 2010-09-10 2012-04-19 Canon Inc Imaging device
CN103080809A (en) * 2010-09-10 2013-05-01 佳能株式会社 Imaging apparatus
CN103080809B (en) * 2010-09-10 2015-07-01 佳能株式会社 Imaging apparatus
WO2012033219A1 (en) * 2010-09-10 2012-03-15 Canon Kabushiki Kaisha Imaging apparatus
US9332166B2 (en) 2010-09-10 2016-05-03 Canon Kabushiki Kaisha Imaging apparatus equipped with illumination optical system having a plurality of optical integrators that guide light
WO2012056920A1 (en) 2010-10-29 2012-05-03 Canon Kabushiki Kaisha Microscope, image acquisition apparatus, and image acquisition system
JP2012138891A (en) * 2010-12-08 2012-07-19 Canon Inc Imaging apparatus
US9386210B2 (en) 2011-03-18 2016-07-05 Canon Kabushiki Kaisha Imaging apparatus
WO2013051147A1 (en) * 2011-10-07 2013-04-11 キヤノン株式会社 Image acquisition apparatus adjustment method, image acquisition apparatus, and image acquisition apparatus manufacturing method
JP2013130686A (en) * 2011-12-21 2013-07-04 Canon Inc Imaging apparatus
WO2013100029A1 (en) * 2011-12-27 2013-07-04 キヤノン株式会社 Image processing device, image display system, image processing method, and image processing program

Similar Documents

Publication Publication Date Title
JP2009003016A (en) Microscope and image acquisition system
US9088729B2 (en) Imaging apparatus and method of controlling same
JP4441831B2 (en) Microscope equipment
JP2007233098A (en) Image acquisition device, image acquisition method, and image acquisition program
JP2009036969A (en) Cover glass, slide glass, preparation, observation method, and microscopic device
US20130088589A1 (en) Method for measuring luminance of light-emitting display panel
JP2010117705A (en) Microscope for virtual-slide creating system
US10133047B2 (en) Image forming apparatus, image forming method, image forming system, and recording medium
JP2007251143A (en) Visual inspection system
KR100803043B1 (en) Apparatus and method for inspecting defects of display device
JP2010223676A (en) Observation device
WO2007055082A1 (en) Cofocal microscope
CN111527438B (en) Shock rescanning system
US10133052B2 (en) Image acquiring method and image acquiring apparatus using the same
CN107037576B (en) Optical microscopic imaging device and imaging method thereof
WO2006011526A1 (en) Imaging apparatus and microscope apparatus using the same
JP4574758B2 (en) Microscope image observation device
JP5665369B2 (en) Imaging device
JP2010039262A (en) Observation device
JP2009015047A (en) Microscope device and observation method
US7864413B2 (en) Examination apparatus with multiple image acquisition devices
US8654223B2 (en) Image pickup apparatus
CN111279242B (en) Dual processor image processing
JP2008306525A (en) Image pickup apparatus, image processing apparatus, image processing method, and program
JP6068010B2 (en) Microscope system

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20100608

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20120229

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20120306

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20120626