JP2019000340A - Pathological diagnosis support apparatus - Google Patents

Abstract

To provide a pathological diagnosis support apparatus for obtaining reliable pathological diagnosis results while reducing a doctor's burden for making a pathological diagnosis.SOLUTION: A pathological diagnosis support apparatus includes: image data memory means for storing a three-dimensional address on a surgical operation extraction material of a divided area by associating it with the photographed image data of the divided area in a pathology sample obtained by cutting out the surgical operation extraction material; classification means for classifying the stored image data based on the feature value of the image; means for transmitting at least a representative image of every classification image to a site on the Internet, the site that stores a teacher image used for the pathological diagnosis and determines the type of the classification image; means for receiving information of the type for every classification image from the site; memory means for storing the type information for every received classification image; selection means for selecting one among the classification images; using multiple image data included in the selection classification image and the three-dimensional address associated and memorized; composite image generating means for generating a three-dimensional composite image of the selected classification image; and display means for displaying the three-dimensional composite image.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a pathological diagnosis support apparatus that supports pathological diagnosis by a pathologist.

  In recent years, in the field of pathology and the like, an apparatus for converting an image observed with an optical microscope into digital data and observing it on a display is used. As such an apparatus, a digital microscope that is an optical microscope that observes a pathological specimen on a display via a digital camera, a remote microscope that adds a digital camera and an electric stage to a conventional optical microscope, and performs microscopic observation by remote operation, In addition, there is a hole slide image creation device (WSI creation device) that captures the entire pathological specimen and generates digital image data.

  For example, Patent Document 1 describes a hole slide image creation apparatus that captures a plurality of images at a high magnification while sequentially moving a movable frame on which a pathological specimen is placed.

JP 2010-20329 A

  By the way, in the pathological diagnosis, the pathologist identifies the lesion by observing the digitalized image. However, when the surgically extracted material collected from the patient is large, a large amount of image data is observed. There is a problem that the pathologist has a heavy burden because it is necessary to perform the operation.

  An object of the present invention is to provide a pathological diagnosis support apparatus for obtaining a highly reliable pathological diagnosis result while reducing the burden on a pathologist when performing pathological diagnosis.

  The pathological diagnosis support apparatus according to the present invention assigns a three-dimensional address on the surgically-extracted material of the segmented region to image data for each segmented region obtained by imaging the segmented region on the pathological specimen obtained by cutting out the surgically-extracted material. Image data storage means for storing in association, classification means for classifying the image data stored by the image data storage means based on image feature values, and at least for each of the classified images classified by the classification means Transmitting means for transmitting representative images to a site on the Internet for storing a teacher image used for pathological diagnosis and determining the type of the classified image, and receiving for receiving information on the type of each classified image from the site Means for storing information of each type of classification image received by the reception means, and a selection for selecting one of the classification images. Selecting by the selecting means using a plurality of image data included in the classification image selected by the selecting means and the three-dimensional address on the surgically-extracted material stored in association with the image data And a display unit for displaying the three-dimensional composite image generated by the composite image generation unit. The composite image generation unit generates a three-dimensional composite image of the classified image.

  The pathological diagnosis support apparatus according to the present invention assigns a three-dimensional address on the surgically-extracted material of the segmented region to image data for each segmented region obtained by imaging the segmented region on the pathological specimen obtained by cutting out the surgically-extracted material. Image data storage means for storing in association, classification means for classifying the image data stored by the image data storage means based on image feature values, and teacher image storage for storing a teacher image used for pathological diagnosis Means for determining the type of the classified image by referring to the teacher image for each of the classified images classified by the classifying unit, and storage means for storing information on the type of each classified image Selection means for selecting one of the classified images, a plurality of the image data included in the classified image selected by the selection means, and the image data Composite image generation means for generating the three-dimensional composite image of the classification image selected by the selection means, using the three-dimensional address on the surgically-extracted material stored in association with the data, and the composite image generation Display means for displaying a three-dimensional composite image generated by the means.

  ADVANTAGE OF THE INVENTION According to this invention, the pathological diagnosis assistance apparatus for obtaining the reliable pathological diagnosis result can be provided, reducing the burden of the pathologist at the time of performing pathological diagnosis.

It is a figure which shows the surgical extraction material for producing the preparation observed with the WSI preparation apparatus which concerns on embodiment. It is a figure which shows the surgical extraction material for producing the preparation observed with the WSI preparation apparatus which concerns on embodiment. It is a figure which shows the surgical extraction material for producing the preparation observed with the WSI preparation apparatus which concerns on embodiment. It is a figure which shows the preparation observed with the WSI preparation apparatus which concerns on embodiment. It is a block diagram which shows the structure of the pathological-diagnosis assistance apparatus which concerns on embodiment. It is a flowchart which shows the preparation image data preparation process in the pathological diagnosis support apparatus which concerns on embodiment. It is a flowchart which shows the classification | category process of the image data in the pathological diagnosis assistance apparatus which concerns on embodiment. It is a figure which shows the classification | category of the image data in the pathological diagnosis assistance apparatus concerning embodiment, and the reply of a teacher image site. It is a flowchart which shows the display process of the image data in the pathological diagnosis assistance apparatus which concerns on embodiment. It is a figure which shows the display process of the three-dimensional synthesized image of the classification image in the pathological diagnosis assistance apparatus which concerns on embodiment.

  Hereinafter, a pathological diagnosis support apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the pathological diagnosis support apparatus 14 (see FIG. 5) according to the present embodiment, pathological diagnosis is performed using the image data of the preparation 4 (see FIG. 4) photographed using the hole slide image (WSI) creating apparatus 16. Support is provided.

(Preparation preparation)
First, after the surgically removed material excised by surgery is fixed, the surgically removed material is cut out. Fixation is performed using formalin etc. according to the cancer handling regulations of each organ. The surgically removed material after fixation is, for example, a size of 2 cm × 3 cm and a thickness of about 4 mm, which fits in a commonly used cover glass (for example, 2.5 cm × 5.0 cm) and will be described later. Cut into a thickness considering the penetration of paraffin in paraffin embedding and put into cassette. Here, for example, an address for recognizing the position of the excised material piece or organ piece in the surgically-extracted material by attaching a symbol to the cassette using ink or attaching a seal on which the symbol is printed. (3D address) is assigned.

  Here, in breast cancer, thyroid cancer, etc., the entire lesioned part or the entire surgically-extracted material is cut out. For example, as the cutting process, first, the surgically-extracted material 2 shown in FIG. 1 is cut out to a thickness of about 4 mm to obtain slices A to C shown in FIG. Next, as shown in FIGS. 3A to 3C, each of the slices A to C is cut into a size of 2 cm × 3 cm to obtain a material piece or an organ piece. 3A is a code indicating an address, and in the present embodiment, the address of “1,1” of the slice A is A (1,1). Represent. In addition, the area | regions 5 and 7 shown in FIGS. 1-3 each show the lesioned part.

  Moreover, the cutting-out process may be automated. That is, as shown in FIG. 2, the material is first cut into slices A to C, then cut with a cutting device that cuts into a size of 2 cm × 3 cm, etc. as shown in FIG. 3, and further cut out using this device. Pieces or organ pieces may be placed in their respective cassettes. The cassette may be pre-addressed with a symbol using ink, or with a seal printed with the symbol, or may be cut when or after the material piece is placed in the cassette. An address may be given by a printing unit provided in the delivery apparatus. As an address, a one-dimensional barcode or a two-dimensional barcode indicating a matrix may be attached instead of or in addition to the notation in a matrix such as A (2, 2).

  Next, paraffin embedding of the piece of material or organ piece put in the cassette is performed. In the paraffin embedding process, a dehydration process in which water is replaced with alcohol such as ethanol, an intermediate treatment process in which the alcohol is replaced with an organic solvent such as xylene or chloroform, and the organic solvent is replaced with paraffin. A paraffin infiltration process for infiltrating the paraffin into the piece and an embedding process for solidifying the infiltrated paraffin are performed. Here, the dehydration step, the intermediate agent treatment step, and the infiltration step are performed using an embedding device, and the embedding processing step is performed using an embedding center (embedding stage).

  Next, after the paraffin embedding process, each block obtained by the paraffin embedding process is sliced into a thickness of several μm using a microtome to obtain a section. Next, as shown in FIG. 4, the sliced section 3 is affixed to a glass slide, followed by a deparaffinization operation and staining to obtain a preparation (tissue specimen) 4.

  As shown in FIG. 4, the specimen 4 is provided with a specimen number such as a specimen ID 6 and a pathological diagnosis request number 8 and an address (three-dimensional address) 10 by a numeral and a two-dimensional barcode 12. Here, the specimen number is a unique number assigned to one surgical extraction material, and the specimen ID 6 and the request number 8 are in a one-to-one correspondence.

(Pathological diagnosis support device)
FIG. 5 is a block diagram illustrating a configuration of a pathological diagnosis support apparatus including the WSI creation apparatus 16. The pathological diagnosis support apparatus 14 includes a WSI creation apparatus 16, a server 18, and an external terminal 20.

  The WSI creation device 16 is provided in a medical institution such as a hospital or a clinic, and includes a control unit 22 that controls each part of the WSI creation device 16 in an integrated manner. The control unit 22 includes a stage control unit 26 that moves the stage 24 for placing the preparation 4, a lens driving unit 30 that controls the driving of the objective lens 28 for observing the preparation 4, a CCD, and the like. Image sensor 32 that captures the subject image formed on the imaging surface by the objective lens 28, digital camera 34 that captures the entire image including the entire area of the slide 4, image data captured by the image sensor 32 and the digital camera 34 An image storage unit 36 that temporarily stores the image, a display unit 38 that displays an image based on image data captured by the image sensor 32, and an operation unit 40 that includes a start button (not shown) for starting a series of shooting operations. A communication unit 42 for transmitting image data to the server 18 via a network such as the Internet, and an image storage unit Image combining unit 44 for generating image data of one image are connected by causing Awa connecting a plurality of image data stored in the 6. Here, the objective lens 28 includes the objective lens 28 for observing the slice 3 (see FIG. 4) at least at a basic magnification (for example, 10 times the objective).

  The server 18 includes a control unit 46, which stores the WSI creation device 16, the external terminal 20, and teacher images used for pathological diagnosis via a network such as the Internet, and determines the type of classification image to be described later. A communication unit 48 that transmits and receives image data to and from a teacher image site (not shown) on the Internet, a database 50 that stores image data and the like, and an information storage unit 52 that stores information on the types of classified images. I have.

  The external terminal 20 is provided in an examination room where a pathologist performs a pathological diagnosis. The external terminal 20 includes a control unit 54. The control unit 54 receives image data from the server 18 via a network such as the Internet, and transmits / receives information on the type of classified image to / from the server 18. Unit 56, a display unit 58 that displays an image based on image data, a three-dimensional composite image, which will be described later, and a display range switching button (non-display button) for moving the display range displayed on the display unit 58 and changing the display magnification. A classification designation button (not shown) for designating the type of the classification image, an observation start button (not shown) for starting observation of the image of the slide having a specific address, and an observation end button for ending the observation An operation unit 60 including, for example, (not shown) and the like is connected. Here, the display unit 58 has a resolution of about 50 million pixels as a whole, and is configured by arranging about 2 to 3 vertical displays and 3 to 4 horizontal displays. In addition, when the same display size is obtained, you may comprise by one display.

  Next, a preparation image data creation process in the WSI creation device 16 of the pathological diagnosis support apparatus according to the embodiment will be described with reference to the flowchart shown in FIG. First, of the prepared slides 4 prepared as described above, one prepared slide 4 having a specific address (three-dimensional address) is set at a predetermined position on the stage 24 (step S1), and the start button is operated. Then, the control unit 22 starts photographing processing (step S2). That is, the control unit 22 controls the stage control unit 26 and moves the stage 24 so that the preparation 4 is positioned immediately below the digital camera 34.

  Next, the control unit 22 causes the digital camera 34 to capture a preparation image including the entire area of the preparation 4 and causes the image storage unit 36 to store the preparation image data of the preparation image. Then, the control unit 22 recognizes the sample number and the address 10 from the number indicating the sample number and the address (three-dimensional address) 10 described in the preparation 4 or the two-dimensional barcode 12 based on the preparation image data (Step S3). ). Moreover, the control part 22 recognizes the total number of the preparations 4 produced from one surgical extraction material in the preparation production process.

  Next, the control unit 22 reads out the preparatory image data from the image storage unit 36, and based on the image data, an area where the section 3 is pasted in the preparation 4 (see FIG. 4) (hereinafter referred to as a specimen area). Is specified (step S4).

  Next, the controller 22 divides the specimen region into a plurality of square lattice-like regions (hereinafter referred to as segmented regions) in advance before photographing the slice 3 at the basic magnification using the objective lens 28. Recognize the position of each segmented region in the sample region. For example, the position of the segmented region located at the leftmost end of the upper edge of the sample region is recognized as position “1, 1”, and the position of the segmented region located right next to the leftmost end of the upper edge of the sample region is positioned Recognized as “1,2”. The same processing is repeated until the position of the segmented region located at the rightmost end of the lower edge of the sample region is recognized as the position “M, N”.

  Next, the control unit 22 photographs the section 3 at the basic magnification using the objective lens 28. First, the control unit 22 controls the stage control unit 26 to move the stage 24 so that the segmented region of the position “1, 1” to be imaged first in the sample region is located immediately below the objective lens 28, and further, the stage 24. Is moved in the vertical direction to focus on a predetermined focal position (hereinafter referred to as a reference plane) in the segmented region of the position “1, 1”. Next, the segmented region at the position “1, 1” is photographed at a basic magnification of 10 times the objective (step S5), and the position information is added to the image data and stored in the image storage unit 36. That is, for example, a three-dimensional address on the surgically-extracted material represented by A (1,1) “1,1” or the like is added to the image data and stored in the storage unit 36. Similarly, the control unit 22 sequentially moves the stage 24 to photograph the segment area from the position “1, 2” to the position “M, N” at the basic magnification, and the position information is included in the image data of the image of the segment area. (3D address on the surgically extracted material) is added and stored in the image storage unit 36.

  Next, the control unit 22 reads out the image data of the images of the respective divided areas taken at the basic magnification from the image storage unit 36, and the image composition unit 44 joins all the divided areas for the preparation 4 at the specific address. Image data of the entire section 3 is generated (step S6). At this time, the image composition unit 44 connects the segmented regions based on the position information added to the image data of the segmented regions and stores the segmented regions in the image storage unit 36.

  Here, when the entire image data of the section 3 is stored, the control unit 22 stores information on the specimen number, address (three-dimensional address) recognized in step S3, and all of the preparation 4 obtained in the preparation preparation step. The information on the number of sheets is added to the entire image data of the section 3 and stored in the image storage unit 36. When the entire image data is generated, the control unit 22 ends photographing of one slide 4 having a specific address (three-dimensional address) among the slides 4 obtained in the slide preparation process.

  Next, based on the specimen number recognized in step S3 and information on the total number of slides added to the address (three-dimensional address), it is determined whether or not all the slides 4 obtained in the slide preparation process have been imaged. (Step S7).

  If not all of the preparations obtained in the preparation preparation process have been photographed, the process returns to the process shown in step S1, and the photographing process of the preparation 4 having another address is performed.

  When all of the slides 4 obtained in the slide preparation process are shot, the control unit 22 sends the image data of all the slides 4 shot through the communication unit 48 to the server 18 via a network such as the Internet. Then, the image data creation process shown in the flowchart of FIG. 7 is terminated.

  At this time, when the server 18 receives image data for all the preparations 4 via the communication unit 48, the control unit 46 stores the received image data in the database 50.

  Next, image data classification in the pathological diagnosis support apparatus according to the embodiment will be described with reference to the flowchart shown in FIG. The control unit 46 of the server 18 stores image data of an image having a specific specimen number stored in the database 50, that is, image data of a segmented image having a three-dimensional address on the surgically-extracted material having the specific specimen number (segmented image). ) Are sequentially read out, and the images are classified into classified images 1 to n based on the feature values of the images (step S10). Here, the feature value of the image includes color, brightness, distribution, shape, arrangement, solidity, and the like of the image. FIG. 8 is a diagram illustrating a state in which the divided images are classified into the classified images 1 to n. As shown in this figure, all the divided images having a three-dimensional address on the surgically-extracted material having a specific specimen number are classified into the classified images 1 to n based on the feature values.

  Next, when all the image data having a specific specimen number have been classified into the classified images 1 to n (step S11), the control unit 46 transmits a teacher image site (not shown) on the Internet via the communication unit 48. At least one representative image of each of the classified images 1 to n is transmitted (step S12). In the teacher image site, the received representative images of the classified images 1 to n are matched with the teacher images stored in the teacher image site, and the type of each of the classified images 1 to n is determined.

  The control unit 46 receives an answer (including possibility information) of the type determination for each of the classified images 1 to n from the teacher image site via the communication unit 48 (step S13). As shown in FIG. 8, the classification determination received from the teacher image site is that classification image 1 is adipose tissue (98% possibility of adipose tissue) and classification image 2 is adenocarcinoma (99% possibility of adenocarcinoma). ), The classification image 3 is a fiber tissue (possibility of fiber tissue 80%), the classification image 4 is fiberized (possibility of fibrosis 90%), and the classification image 5 is smooth muscle (possibility of smooth muscle 90). %)... Classification image n is unknown (classification is impossible).

  The control unit 46 stores, in the information storage unit 52, the determination responses (including possibility information) of the classification images 1 to n received from the teacher image site via the communication unit 48 (step S14). .

  Next, image data display processing during pathological diagnosis in the pathological diagnosis support apparatus according to the embodiment will be described with reference to the flowchart of FIG. When the pathologist performs a pathological diagnosis, if the instruction to display the image of the slide 4 at a specific address (three-dimensional address) is given using the observation start button of the operation unit 60 of the external terminal 20, the control unit 54 A transmission request for image data at the designated address (three-dimensional address) is transmitted to the server 18 (step S20).

  When receiving the transmission request, the control unit 46 of the server 18 transmits the entire image data of the intercept 3 of the address (three-dimensional address) designated via the communication unit 48 to the external terminal 20 (step S21).

  When the entire image data of the intercept 3 for the address (three-dimensional address) designated by the external terminal 20 is received (step S22), the control unit 54 displays the image of the received image data on the display unit 58 (step S23). ).

  Here, the pathologist can change the display range and display magnification of the image displayed on the display unit 58 using the display range switching button of the operation unit 60. The pathologist observes the image displayed on the display unit 58 and makes a pathological diagnosis on the lesioned part and the like.

  If the pathologist wants to observe a three-dimensional composite image of a specific type of classification image (for example, an adenocarcinoma image, a fiber tissue image, a fibrosis image, etc.), any one of the classification images 1 to n Operate the classification specification button to specify. As a result, information specifying the specimen number of the section currently being observed and the classification images 1 to n is transmitted to the server 18 via the communication unit 56 of the external terminal 20 (step S30 in FIG. 10).

  When the server 18 receives information specifying the sample number and any one of the classified images 1 to n via the communication unit 48 (step S31), the image included in the received classified image of the specified sample number. Data is transmitted to the external terminal 20 (step S31).

  The external terminal 20 synthesizes the image data of the received classified images to generate a three-dimensional synthesized image of a specific type of classified image (step S33) and displays it on the display unit 58 (step S34).

  Therefore, a pathologist can refer to a three-dimensional composite image of a specific type of classification image (for example, an adenocarcinoma image, a fiber tissue image, a fibrosis image, ...) when performing pathological diagnosis of a lesion site. Therefore, it becomes easy to grasp a specific type of classification image three-dimensionally, especially when the entire lesion, such as breast cancer or thyroid cancer, or the entire surgically-extracted material is cut out, thereby improving the reliability of pathological diagnosis be able to.

  When the pathologist operates the observation end button of the operation unit 60, the control unit 54 ends the display of the preparation image of the specific address (three-dimensional address) displayed on the display unit 58 (step S24). Next, the control unit 54 of the external terminal 20 stores the image data having the same specimen number stored in the database 50 based on the information on the total number of the slides obtained in the slide preparation process added to the image data. It is determined whether or not all the images have been observed (step S25).

  If all the images of the image data having the same specimen number have not been observed, the control unit 54 transmits a transmission request for image data at an unobserved address to the server 18 (step S26).

  When the transmission request is received, the control unit 46 of the server 18 transmits the entire image data of the intercept 3 of the address (three-dimensional address) designated via the communication unit 48 to the external terminal 20 (step S27). .

  When the external terminal 20 receives the entire image data of the intercept 3 for the designated address from the server 18 (step S28), the control unit 54 causes the display unit 58 to display the received image data image (step S29). Then, the process returns to step S24. When all the images of the image data having the same specimen number are observed, the control unit 22 ends the display of the image data.

  According to the pathological diagnosis support apparatus according to the present embodiment, a highly reliable pathological diagnosis result can be obtained while reducing the burden on the pathologist when performing pathological diagnosis.

  In the above-described embodiment, the server 18 transmits a representative image of the classified image to the teacher image site on the Internet, determines the classified image at the teacher image site, and receives the result. The server may be provided with a teacher image storage unit for storing a teacher image used for pathological diagnosis, and the server may determine the type of the classification image by referring to the teacher image for each classification image.

DESCRIPTION OF SYMBOLS 4 ... Preparation, 14 ... Pathological diagnosis support apparatus, 16 ... WSI preparation apparatus, 18 ... Server, 20 ... External terminal, 40 ... Operation part, 50 ... Database, 52 ... Information storage part

Claims (2)

Image data storage means for storing, in association with the image data for each segmented region obtained by imaging the segmented region on the pathological specimen obtained by cutting out the surgically-extracted material, a three-dimensional address on the surgically-extracted material of the segmented region ,
Classification means for classifying the image data stored by the image data storage means based on image feature values;
Transmitting means for storing at least a representative image for each of the classified images classified by the classification means, transmitting a teacher image used for pathological diagnosis to a site on the Internet for determining the type of the classified image;
Receiving means for receiving the type of information for each of the classified images from the site;
Storage means for storing the type of information for each of the classified images received by the receiving means;
Selecting means for selecting one of the classified images;
The plurality of image data included in the classification image selected by the selection unit and the three-dimensional address on the surgically-extracted material stored in association with the image data are used to select the selection unit. A composite image generating means for generating a three-dimensional composite image of the classified image;
A pathological diagnosis support apparatus comprising: display means for displaying the three-dimensional composite image generated by the composite image generation means. Image data storage means for storing, in association with the image data for each segmented region obtained by imaging the segmented region on the pathological specimen obtained by cutting out the surgically-extracted material, a three-dimensional address on the surgically-extracted material of the segmented region ,
Classification means for classifying the image data stored by the image data storage means based on image feature values;
Teacher image storage means for storing a teacher image used for pathological diagnosis;
Determining means for determining the type of the classified image by referring to the teacher image for each of the classified images classified by the classifying means;
Storage means for storing the type of information for each classified image;
Selecting means for selecting one of the classified images;
The plurality of image data included in the classification image selected by the selection unit and the three-dimensional address on the surgically-extracted material stored in association with the image data are used to select the selection unit. A composite image generating means for generating the three-dimensional composite image of the classified image;
A pathological diagnosis support apparatus comprising: display means for displaying a three-dimensional composite image generated by the composite image generation means.