WO2021206053A1 - データ生成装置、データ生成方法及びプログラム - Google Patents
データ生成装置、データ生成方法及びプログラム Download PDFInfo
- Publication number
- WO2021206053A1 WO2021206053A1 PCT/JP2021/014494 JP2021014494W WO2021206053A1 WO 2021206053 A1 WO2021206053 A1 WO 2021206053A1 JP 2021014494 W JP2021014494 W JP 2021014494W WO 2021206053 A1 WO2021206053 A1 WO 2021206053A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- images
- label
- partial images
- input
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 30
- 238000012549 training Methods 0.000 claims description 9
- 239000003086 colorant Substances 0.000 claims description 6
- 230000001575 pathological effect Effects 0.000 description 91
- 238000012545 processing Methods 0.000 description 26
- 206010028980 Neoplasm Diseases 0.000 description 12
- 238000002372 labelling Methods 0.000 description 9
- 230000005856 abnormality Effects 0.000 description 8
- 230000007170 pathology Effects 0.000 description 8
- 238000010186 staining Methods 0.000 description 8
- 210000004881 tumor cell Anatomy 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- LPXQRXLUHJKZIE-UHFFFAOYSA-N 8-azaguanine Chemical compound NC1=NC(O)=C2NN=NC2=N1 LPXQRXLUHJKZIE-UHFFFAOYSA-N 0.000 description 1
- 101100322583 Caenorhabditis elegans add-2 gene Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000010827 pathological analysis Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000007447 staining method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Definitions
- the present invention relates to a data generator, a data generation method and a program.
- Patent Document 1 a system that analyzes pathological images using artificial intelligence and performs pathological diagnosis is known.
- a machine learning algorithm is trained using a plurality of annotated digital pathological images, and an abnormal image pattern corresponds to a pathological abnormality using a discriminative model generated by the learning. It is described to identify whether or not.
- an object of the present invention is to provide a technique that enables easier generation of learning data related to images.
- the data generation device includes a display control unit for displaying an image and a boundary line of a plurality of partial images generated by dividing the image into a plurality of images on a screen, and a plurality of partial images.
- Learning to train a learning model by associating the input unit that accepts the input of the label given to each of the above, each of the plurality of partial images, and the label given to each of the plurality of partial images. It has a generation unit that generates data for use.
- FIG. 1 is a diagram showing an example of an image processing system 1 for determining whether or not an abnormality is present in a pathological image.
- the data generation device 10 is a device that generates learning data (teacher data) for learning a learning model included in the diagnostic device 20 by using the pathological image input to the data generation device 10.
- the diagnostic device 20 is a device that determines whether or not an abnormality is present in the pathological image by using a learned model trained using the learning data generated by the data generation device 10.
- the data generation device 10 and the diagnostic device 20 can communicate with each other via a wireless or wired communication network N.
- the data generation device 10 superimposes the pathological image input to the data generation device 10 and the boundary line of a plurality of partial images generated by dividing the pathological image into a plurality of images and displays them on the screen.
- the partial image may be referred to as a tile.
- the size of the partial image (the number of pixels in the vertical direction and the horizontal direction) is the same size as the training data that can be input to the training model.
- the data generation device 10 assigns each partial image displayed on the screen to the partial image by a pathologist or the like (hereinafter, referred to as “user”) who uses the data generation device 10. Accepts the input of the label related to.
- the label will be described on the premise that it is two types of labels indicating whether or not a pathological abnormality is present in the partial image, but the present embodiment is not limited thereto. This embodiment can also be applied to the case where three or more types of labels are attached.
- the data generation device 10 may generate learning data from one pathological image, or may generate learning data from a plurality of pathological images. When generating learning data from a plurality of pathological images, the data generation device 10 repeats a process of dividing the pathological image into a plurality of partial images and accepting input of a label for each pathological image. When the user has completed labeling of all the pathological images, the data generation device 10 generates learning data by associating the image data of each partial image with the label assigned to each partial image. The generated learning data is sent to the diagnostic device 20.
- the diagnostic device 20 trains (trains) a learning model using the learning data sent from the data generation device 10.
- the learning model is, for example, a neural network capable of having a predetermined ability by learning.
- the diagnostic device 20 inputs a pathological image to be diagnosed into the learned model generated by learning, and determines whether or not an abnormality exists in the pathological image based on the output result from the learned model.
- the data generation device 10 is illustrated as one information processing device, but the present embodiment is not limited to this.
- the data generation device 10 may be configured by one or a plurality of physical servers or the like, or may be configured by using a virtual server operating on a hypervisor. It may be configured using a cloud server.
- FIG. 2 is a diagram showing a hardware configuration example of the data generation device 10.
- the data generation device 10 includes a processor 11 such as a CPU (Central Processing Unit) and a GPU (Graphical processing unit), a storage device 12 such as a memory, an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), and a wired or wireless device. It has a communication IF (Interface) 13 for communication, an input device 14 for accepting input operations, and an output device 15 for outputting information.
- the input device 14 is, for example, a keyboard, a touch panel, a mouse and / or a microphone.
- the output device 15 is, for example, a display, a touch panel, and / or a speaker.
- FIG. 3 is a diagram showing an example of a functional block configuration of the data generation device 10.
- the data generation device 10 includes a storage unit 100, a display control unit 101, an input unit 102, a generation unit 103, and an image processing unit 104.
- the storage unit 100 can be realized by using the storage device 12 included in the data generation device 10.
- the display control unit 101, the input unit 102, the generation unit 103, and the image processing unit 104 are realized by the processor 11 of the data generation device 10 executing the program stored in the storage device 12. Can be done.
- the program can be stored in a storage medium.
- the storage medium in which the program is stored may be a non-transitory computer readable medium.
- the non-temporary storage medium is not particularly limited, but may be, for example, a storage medium such as a USB memory or a CD-ROM.
- the storage unit 100 is generated as a pathological image DB (image DB) for storing one or more pathological images (images) used for generating learning data, and a label DB for storing labels given to partial images by the user. It stores a learning data DB that stores the learning data.
- image DB pathological image DB
- label DB label DB
- the display control unit 101 causes an output device 15 such as a display, another information processing device that communicates with the data generation device 10, and the like to display various screens according to the present embodiment. Further, the display control unit 101 causes the pathological image (image) to display a screen on which the boundary line of the partial image is superimposed.
- the input unit 102 receives various inputs from the user. Further, the input unit 102 receives input of a label to be given to each of the plurality of partial images from the user via the input device 14.
- the input unit 102 stores the label given to each partial image in the label DB.
- the input unit 102 may store the partial image ID that uniquely identifies each partial image in the label DB in association with the label given to each partial image.
- the label may be input, for example, by selecting a partial image for inputting a label from each of the partial images displayed on the screen and accepting the designation of the label to be given to the partial image.
- the input unit 102 when the input unit 102 generates learning data from a plurality of pathological images (plurality of images), the input unit 102 includes a plurality of pathological types corresponding to the plurality of pathological images (pathological types common to the plurality of pathological images). You may accept the input of the information about the image type corresponding to the image).
- the generation unit 103 generates learning data for training the learning model by associating each of the plurality of partial images with the labels given to each of the plurality of partial images by the input unit 102. For example, the generation unit 103 acquires a partial image ID and a label from the label DB, and also obtains a partial image corresponding to the partial image ID from the pathological image (image) stored in the pathological image DB (image DB). Extract image data. Subsequently, the generation unit 103 generates learning data by combining the image data of the extracted partial image and the label corresponding to the partial image ID.
- the image processing unit 104 changes the colors of a plurality of pathological images (plurality of images) according to a color changing method determined according to the type of pathology (image type) input to the input unit 102.
- the changing method includes a method of changing the RGB value of each pixel in a plurality of pathological images (plural images) to a standardized value, and a method of randomly changing the RGB value of each pixel in a plurality of pathological images (plural images). It may include a way to change it.
- FIG. 4 is a flowchart showing an example of a processing procedure performed by the data generation device 10.
- the input unit 102 receives input of a pathological image used for generating learning data from the user, and stores the input pathological image in the pathological image DB (S10).
- Pathological images are usually stained to make it easier to determine if they are abnormal. Examples of the staining method include hematoxylin-eosin staining, PAS staining, May-Gimza staining, Alcian blue staining, papanicorow staining, Azan staining, Elastica Wangison staining, and Elastica Masson staining.
- the user When inputting a plurality of pathological images, the user inputs pathological images having the same pathological type. In the following description, it is assumed that a plurality of pathological images have been input by the user.
- the input unit 102 receives input of information regarding the type of pathology from the user for the plurality of pathological images input in the processing procedure of step S10 (S11).
- the type of pathology that can be input may be, for example, either "tumor” or "supermutation".
- the user inputs "tumor” as the type of pathology when each of the plurality of pathological images is a tumor image, and when each of the plurality of pathological images is a hypervariant image, the user inputs "supermutation" as the type of pathology. ".
- the image processing unit 104 changes the colors of the plurality of pathological images according to the color changing method according to the type of pathology input in the processing procedure of step S11 (S12).
- the method of changing the color may be to change the RGB value of each pixel in a plurality of pathological images to a standardized value.
- the image processing unit 104 sets the average value and standard deviation of the R (Red) value, the average value and standard deviation of the G (Green) value, and the average of the B (Blue) values for all the pixels in the plurality of pathological images. Calculate the value and standard deviation.
- the image processing unit 104 standardizes the R values of all the pixels in the plurality of pathological images by using the average value and the standard deviation of the R values.
- the color changing method may be a method of randomly shifting the RGB value of each pixel in a plurality of pathological images for each pathological image.
- the image processing unit 104 randomly determines the number of RGB values to be shifted for each pathological image, and shifts the R value, the G value, and the B value by the determined values. For example, when the image processing unit 104 determines that the number of RGB values to be shifted is "5" for the first pathological image, the R value, G value, and B value of each pixel included in the pathological image are set to "5". Add 5 to.
- the image processing unit 104 determines that the number for shifting the RGB values of the second pathological image is "-2", the R value, G value, and B value of each pixel included in the pathological image. Add -2 to.
- the image processing unit 104 repeats the same processing for all pathological images. This makes it possible to disperse the colors of a plurality of pathological images.
- the display control unit 101 selects one pathological image from the plurality of input pathological images, and displays a screen on which the boundary line of the partial image is superimposed on the selected pathological image (S13).
- the input unit 102 accepts the input of the label to be given for each partial image (S14).
- the label given to each partial image by the user is a label indicating that the partial image is a tumor image, or the partial image is not a tumor image. Is one of the labels indicating.
- the label given to each partial image by the user is a label indicating that the partial image is a super-mutated image, or the partial image is super-mutated. It is one of the labels indicating that it is not an image of.
- the input unit 102 stores the received label in the label DB in association with the partial image ID.
- the data generation device 10 repeats the processing procedure of step S14.
- the generation unit 103 generates learning data by associating the assigned label with the image data of each partial image for which labeling has been completed. (S16).
- the determination of whether or not the label input is completed depends on whether or not the input unit 102 detects that a predetermined button (for example, a button for starting the generation of learning data) has been pressed on the screen. You may be told. Alternatively, it may be performed depending on whether or not a label is attached to a predetermined number of partial images.
- a label is given to a predetermined number of partial images and the ratio of the given label (ratio for each type of label) is within a predetermined range (for example, a label showing an image of a tumor and an image of a tumor).
- the ratio of the label to the label indicating that the label is not the same is approximately 1: 4, etc.), and it may be automatically determined that the label has been attached.
- the generation unit 103 may divide the generated learning data into folders for each label and store them in the learning data DB. For example, the image data of the partial image labeled with a tumor and the image data of the partial image labeled without a tumor may be stored in separate folders.
- the partial image in which the portion where the tissue does not exist is a predetermined ratio or more (the partial image satisfying the predetermined condition) may be excluded from the learning data.
- the input unit 102 inputs a label for a partial image in which a specific color portion (for example, a white portion) is a predetermined ratio or more among the plurality of partial images in each pathological image. You may not accept it.
- the generation unit 103 may not include the partial image in which the input of the label is not accepted by the input unit 102 among the plurality of partial images in the learning data to be generated. .. As a result, it is possible to prevent the generation of learning data inappropriate for learning the learning model.
- FIG. 5 is a diagram showing an example of a screen for labeling.
- FIG. 5 shows an example when the pathological image is an image containing tumor cells.
- the selection menu M10 is a menu for designating the type of pathological image input by the user.
- the selection menu M11 is a menu for designating a pathological image to be displayed on the screen when a plurality of pathological images are input.
- 40 pathological images related to tumor cells have been input, and it is shown that the third pathological image is currently displayed.
- a grid-like boundary line is superimposed on an enlarged image of a part of the pathological image.
- one area surrounded by a boundary line corresponds to one partial image.
- an image of the entire pathological image and a display frame V11 indicating an area of the pathological image that is enlarged and displayed in the display area W10 are displayed.
- the position or size of the display frame V11 may be arbitrarily changed by, for example, operating a mouse.
- the display control unit 101 enlarges and displays the pathological image in the display area W10 according to the changed position and size of the display frame V11. To change.
- the size of the partial image (the number of pixels in the vertical direction and the horizontal direction) is predetermined according to the learning model of the learning target, the user may not be able to change it arbitrarily.
- the size of the partial image is 300 pixels (pixels) in the vertical direction and 300 pixels (pixels) in the horizontal direction.
- a display frame T10 indicating the position of the partial image that accepts the input of the label from the user is displayed.
- the position of the display frame T10 can be arbitrarily changed by selecting a partial image in which the user wants to input a label. By displaying the display frame T10, the user can recognize at which position the partial image is to be labeled.
- the information indicating the input label is displayed superimposed on the partial image.
- the letter "T” is displayed in the upper left, indicating that the partial image is not an image of a tumor cell.
- the letter "N” is displayed in the upper left of the partial image to which the indicated label is attached.
- the ratio of the labels already assigned is displayed as information indicating the actual value of the ratio of each label assigned to the plurality of partial images. That is, in the example of FIG. 5, it is desirable that the ratio of the number of partial images labeled with T to the number of partial images labeled with N is 1: 4, but at present, the portion labeled with T. It is shown that the ratio of the number of images to the number of partial images labeled N is 1: 2. Therefore, the user can recognize that it is necessary to find and label a partial image that is not an image of tumor cells until the ratio displayed in the display area N10 is 1: 4.
- the display control unit 101 causes the labeling screen to display both the target value of the number of partial images to be labeled and the information indicating the actual value of the number of partial images to which the label is attached. You may. For example, it is assumed that there is a condition that the recognition accuracy of the learning model becomes high when 2,000 images are trained as the learning data.
- a standard of "the target number of tiles to be labeled is 2,000" is displayed as a target value. This indicates that it is desirable for the user who gives the label to give the label to 2,000 partial images.
- the display area N11 the actual value of the number of partial images to which the label has been given is displayed. This allows the user to recognize that an additional 1,000 partial images need to be labeled.
- the plurality of partial images are divided from a reference point (for example, the upper left pixel) in the pathological image at predetermined intervals in the right direction and the downward direction (for example, 300 pixel intervals in the right direction and 300 pixel intervals in the downward direction). It may be an image generated by the above.
- the plurality of partial images are not limited to this, and the points deviated by a predetermined distance from the reference point in the pathological image are spaced apart by a predetermined distance in the left-right direction and the up-down direction (for example, an interval of 300 pixels in the right-left direction and an interval of 300 pixels in the up-down direction). It may be an image generated by dividing by (etc.).
- the pathological image is divided in the right direction and the downward direction at predetermined intervals based on the points (pixels) shifted to the right and / or downward by a predetermined distance from the reference point (upper left image) in the pathological image.
- the image may be generated by the above.
- the offset value designation area M12 is an area for designating how many pixels the reference point in the pathological image is shifted to the right.
- the offset value designation area M13 is an area for designating how many pixels the reference point in the pathological image is shifted downward.
- the partial image P1 shows the partial image located at the upper left of the pathological image when the partial image is generated by dividing the pathological image in the right direction and the downward direction at predetermined intervals with reference to the upper left of the pathological image.
- the image P6 shows a partial image to the right of the partial image P1.
- 50 pixels and 0 pixels are input to the offset value designation areas M12 and M13 in FIG. 5, respectively.
- the partial image P2 is generated based on the point shifted by 50 pixels from the upper left to the right of the pathological image.
- 100 pixels and 0 pixels are input to the offset value designation areas M12 and M13 in FIG. 5, respectively.
- the partial image P3 is generated based on the point shifted by 100 pixels from the upper left to the right of the pathological image.
- the data generation device 10 divides the image into a plurality of partial images and displays them, and accepts a label input from the user for each partial image. This makes it possible to more easily generate learning data related to images.
- the data generation device 10 changes the pixel values related to the colors of the plurality of pathological images according to the method of changing the pixel values determined according to the image types corresponding to the plurality of images.
- the color of a plurality of images can be changed to a more appropriate color suitable for learning the learning model according to the image type, and a learning model with higher recognition accuracy can be generated. ..
- the data generation device 10 displays a target value and an actual value for the ratio of each type of label given to each partial image.
- the user can label each partial image so that the actual value approaches the target value, so that the learning data for generating the learning model with higher recognition accuracy can be efficiently generated. It will be possible to do.
- the data generation device 10 makes it possible to divide the image into partial images based on arbitrary points (pixels) on the image. As a result, after labeling, the user repeats the work of shifting the reference point when dividing the image into partial images and performing the labeling again, so that the number of input images is small. Can also generate a large number of training data.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
図1は、病理画像に異常が存在するか否かを判定する画像処理システム1の一例を示す図である。データ生成装置10は、データ生成装置10に入力された病理画像を用いて、診断装置20が備える学習モデルを学習させるための学習用データ(教師データ)を生成する装置である。診断装置20は、データ生成装置10で生成された学習用データを用いて学習した学習済モデルを用いて、病理画像に異常が存在するか否かの判定を行う装置である。データ生成装置10及び診断装置20は、無線又は有線の通信ネットワークNを介してお互いに通信することができる。
図2は、データ生成装置10のハードウェア構成例を示す図である。データ生成装置10は、CPU(Central Processing Unit)、GPU(Graphical processing unit)等のプロセッサ11、メモリ、HDD(Hard Disk Drive)及び/又はSSD(Solid State Drive)等の記憶装置12、有線又は無線通信を行う通信IF(Interface)13、入力操作を受け付ける入力デバイス14、及び情報の出力を行う出力デバイス15を有する。入力デバイス14は、例えば、キーボード、タッチパネル、マウス及び/又はマイク等である。出力デバイス15は、例えば、ディスプレイ、タッチパネル及び/又はスピーカ等である。
図3は、データ生成装置10の機能ブロック構成例を示す図である。データ生成装置10は、記憶部100と、表示制御部101と、入力部102と、生成部103と、画像処理部104とを含む。記憶部100は、データ生成装置10が備える記憶装置12を用いて実現することができる。また、表示制御部101と、入力部102と、生成部103と、画像処理部104とは、データ生成装置10のプロセッサ11が、記憶装置12に記憶されたプログラムを実行することにより実現することができる。また、当該プログラムは、記憶媒体に格納することができる。当該プログラムを格納した記憶媒体は、コンピュータ読み取り可能な非一時的な記憶媒体(Non-transitory computer readable medium)であってもよい。非一時的な記憶媒体は特に限定されないが、例えば、USBメモリ又はCD-ROM等の記憶媒体であってもよい。
図4は、データ生成装置10が行う処理手順の一例を示すフローチャートである。まず、入力部102は、ユーザから、学習用データの生成に用いる病理画像の入力を受け付け、入力された病理画像を病理画像DBに格納する(S10)。病理画像は、異常かどうかの判断を付けやすくするため、染色されることが通常行われる。染色方法としては、ヘマトキシリン・エオジン染色、PAS染色、メイ・ギムザ染色、アルシアンブルー染色、パパニコロウ染色、アザン染色、エラスチカ・ワンギーソン染色、エラスチカ・マッソン染色などが例として挙げられる。なお、ユーザは、複数の病理画像を入力する場合、病理の種別が同一である病理画像を入力する。以下の説明では、ユーザにより複数の病理画像が入力されたものとする。
以上説明した実施形態によれば、データ生成装置10は、画像を複数の部分画像に分割して表示し、部分画像ごとに、ユーザからラベルの入力を受け付けるようにした。これにより、画像に関する学習用データを、より簡易に生成することが可能になる。
Claims (8)
- 画像と、前記画像を複数に分割することで生成される複数の部分画像の境界線とを重ねて画面に表示させる表示制御部と、
前記複数の部分画像の各々に付与される、ラベルの入力を受け付ける入力部と、
前記複数の部分画像の各々と、前記複数の部分画像の各々に付与されたラベルとを対応づけることで、学習モデルを学習させるための学習用データを生成する生成部と、
を有するデータ生成装置。 - 前記画像は、複数の画像を含み、
前記入力部は、前記複数の画像に対応する画像種別に関する情報の入力を受け付け、
前記入力部に入力された前記画像種別に応じて定められた変更方法に従って、前記複数の画像の色に関する画素値を変更する変更部、を更に有する、
請求項1に記載のデータ生成装置。 - 前記変更方法には、前記複数の画像における各画素のRGB値を標準化した値に変更する方法、及び、前記複数の画像における各画素のRGB値をランダムに変更する方法が含まれる、
請求項2に記載のデータ生成装置。 - 前記表示制御部は、前記複数の部分画像の各々に付与するラベルの目標値であるラベルごとの比率を示す情報と、前記複数の部分画像に対して付与されたラベルごとの比率の実績値を示す情報との両方を表示させる、
請求項1~3のいずれか一項に記載のデータ生成装置。 - 前記入力部は、前記複数の部分画像のうち、所定条件を満たす部分画像については、ラベルの入力を受け付けないようにし、
前記生成部は、前記複数の部分画像のうち、ラベルの入力が受け付けられなかった部分画像については、前記学習用データに含めないようにする、
請求項1~4のいずれか一項に記載のデータ生成装置。 - 前記複数の部分画像は、前記画像内における基準となる点から所定距離ずらした点を基準に、前記画像を右左方向及び上下方向に所定間隔で分割することで生成される画像であり、
前記入力部は、前記所定距離の指定を受け付ける、
請求項1~5のいずれか一項に記載のデータ生成装置。 - データ生成装置が実行するデータ生成方法であって、
画像と、前記画像を複数に分割することで生成される複数の部分画像の境界線とを重ねて画面に表示させるステップと、
前記複数の部分画像の各々に付与される、ラベルの入力を受け付けるステップと、
前記複数の部分画像の各々と、前記複数の部分画像の各々に付与されたラベルとを対応づけることで、学習モデルを学習させるための学習用データを生成するステップと、
を含むデータ生成方法。 - コンピュータに、
画像と、前記画像を複数に分割することで生成される複数の部分画像の境界線とを重ねて画面に表示させるステップと、
前記複数の部分画像の各々に付与される、ラベルの入力を受け付けるステップと、
前記複数の部分画像の各々と、前記複数の部分画像の各々に付与されたラベルとを対応づけることで、学習モデルを学習させるための学習用データを生成するステップと、
を実行させるためのプログラム。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2022514063A JP7236202B2 (ja) | 2020-04-06 | 2021-04-05 | データ生成装置、データ生成方法及びプログラム |
KR1020227038232A KR20220164755A (ko) | 2020-04-06 | 2021-04-05 | 데이터 생성 장치, 데이터 생성 방법 및 프로그램 |
EP21785651.7A EP4134909A4 (en) | 2020-04-06 | 2021-04-05 | DATA GENERATION DEVICE, DATA GENERATION METHOD AND PROGRAM |
US17/917,542 US20230148243A1 (en) | 2020-04-06 | 2021-04-05 | Data generation apparatus, data generation method, and program |
CN202180024507.9A CN115380307A (zh) | 2020-04-06 | 2021-04-05 | 数据生成装置、数据生成方法及程序 |
JP2023025993A JP2023058726A (ja) | 2020-04-06 | 2023-02-22 | データ生成装置、データ生成方法及びプログラム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020068413 | 2020-04-06 | ||
JP2020-068413 | 2020-04-06 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021206053A1 true WO2021206053A1 (ja) | 2021-10-14 |
Family
ID=78023254
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/014494 WO2021206053A1 (ja) | 2020-04-06 | 2021-04-05 | データ生成装置、データ生成方法及びプログラム |
Country Status (6)
Country | Link |
---|---|
US (1) | US20230148243A1 (ja) |
EP (1) | EP4134909A4 (ja) |
JP (2) | JP7236202B2 (ja) |
KR (1) | KR20220164755A (ja) |
CN (1) | CN115380307A (ja) |
WO (1) | WO2021206053A1 (ja) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005185560A (ja) * | 2003-12-25 | 2005-07-14 | Konica Minolta Medical & Graphic Inc | 医用画像処理装置及び医用画像処理システム |
JP2014018332A (ja) * | 2012-07-17 | 2014-02-03 | Hoya Corp | 画像処理装置及び内視鏡装置 |
WO2019159821A1 (ja) * | 2018-02-15 | 2019-08-22 | 国立大学法人新潟大学 | 高頻度変異型癌の判別システム、プログラム及び方法 |
JP2019525151A (ja) | 2016-06-28 | 2019-09-05 | コンテクストビジョン、アクチボラグContextvision Ab | デジタル病理画像における病理学的異常を検出するための方法及びシステム並びに組織スライドに注釈を付けるための方法 |
JP2019146935A (ja) * | 2018-02-28 | 2019-09-05 | 富士フイルム株式会社 | 学習用データ作成支援装置、学習用データ作成支援方法、および学習用データ作成支援プログラム |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017099721A (ja) | 2015-12-02 | 2017-06-08 | パナソニックIpマネジメント株式会社 | 画像処理装置及び画像処理方法 |
JP6702716B2 (ja) | 2015-12-21 | 2020-06-03 | キヤノン株式会社 | 画像処理装置、画像処理方法及びプログラム |
JP6623851B2 (ja) | 2016-03-08 | 2019-12-25 | 富士通株式会社 | 学習方法、情報処理装置および学習プログラム |
JP7027070B2 (ja) | 2017-08-31 | 2022-03-01 | キヤノン株式会社 | 情報処理装置、情報処理方法、及びプログラム |
JP7235272B2 (ja) | 2018-05-16 | 2023-03-08 | 株式会社アドダイス | 画像処理装置及び検査システム |
JP7458328B2 (ja) * | 2018-05-21 | 2024-03-29 | コリスタ・エルエルシー | マルチ分解能登録を介したマルチサンプル全体スライド画像処理 |
-
2021
- 2021-04-05 EP EP21785651.7A patent/EP4134909A4/en active Pending
- 2021-04-05 KR KR1020227038232A patent/KR20220164755A/ko unknown
- 2021-04-05 WO PCT/JP2021/014494 patent/WO2021206053A1/ja unknown
- 2021-04-05 CN CN202180024507.9A patent/CN115380307A/zh active Pending
- 2021-04-05 JP JP2022514063A patent/JP7236202B2/ja active Active
- 2021-04-05 US US17/917,542 patent/US20230148243A1/en active Pending
-
2023
- 2023-02-22 JP JP2023025993A patent/JP2023058726A/ja active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005185560A (ja) * | 2003-12-25 | 2005-07-14 | Konica Minolta Medical & Graphic Inc | 医用画像処理装置及び医用画像処理システム |
JP2014018332A (ja) * | 2012-07-17 | 2014-02-03 | Hoya Corp | 画像処理装置及び内視鏡装置 |
JP2019525151A (ja) | 2016-06-28 | 2019-09-05 | コンテクストビジョン、アクチボラグContextvision Ab | デジタル病理画像における病理学的異常を検出するための方法及びシステム並びに組織スライドに注釈を付けるための方法 |
WO2019159821A1 (ja) * | 2018-02-15 | 2019-08-22 | 国立大学法人新潟大学 | 高頻度変異型癌の判別システム、プログラム及び方法 |
JP2019146935A (ja) * | 2018-02-28 | 2019-09-05 | 富士フイルム株式会社 | 学習用データ作成支援装置、学習用データ作成支援方法、および学習用データ作成支援プログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP4134909A4 |
Also Published As
Publication number | Publication date |
---|---|
CN115380307A (zh) | 2022-11-22 |
EP4134909A4 (en) | 2024-02-21 |
US20230148243A1 (en) | 2023-05-11 |
JP2023058726A (ja) | 2023-04-25 |
KR20220164755A (ko) | 2022-12-13 |
JP7236202B2 (ja) | 2023-03-09 |
JPWO2021206053A1 (ja) | 2021-10-14 |
EP4134909A1 (en) | 2023-02-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102208115B (zh) | 基于三维医学图像而生成立体视图的技术 | |
US20170115834A1 (en) | Information processing apparatus, method for processing information, and non-transitory computer readable medium storing program | |
CN107633478B (zh) | 图像处理装置、图像处理方法以及计算机可读介质 | |
JP2005296156A (ja) | 医用画像表示装置 | |
WO2021206053A1 (ja) | データ生成装置、データ生成方法及びプログラム | |
Waldin et al. | Chameleon: dynamic color mapping for multi-scale structural biology models | |
JP2020101860A (ja) | 医用画像処理装置、システム及びプログラム | |
JP5857606B2 (ja) | 奥行き製作支援装置、奥行き製作支援方法、およびプログラム | |
US11163504B2 (en) | Dividing a spanning region of adjacent sub-images to generate print data | |
CN107742316A (zh) | 图像拼接点获取方法及获取装置 | |
JP2000347786A (ja) | カラー表示におけるポインタ表示色生成方法および装置、記録媒体 | |
JP6828586B2 (ja) | 表示プログラム、表示装置及び表示方法 | |
JP2014178905A (ja) | 画像表示装置、画像表示方法およびプログラム | |
WO2024038651A1 (ja) | プログラム、画像処理方法及び画像処理装置 | |
JP2009225247A (ja) | 画像表示装置及び画像表示方法 | |
US7587086B2 (en) | Identifying selected pixels in a digital image | |
WO2022004319A1 (ja) | 情報処理装置、情報処理方法、及びプログラム | |
JP7076791B2 (ja) | カラーマッピング装置、カラーマッピング方法及びプログラム | |
JP6656571B1 (ja) | 翻訳された病院内の案内板を表示する、プログラム、方法および装置 | |
WO2022070493A1 (ja) | データ処理システム | |
Waldin et al. | Dynamic color mapping for multi-scale structural biology models | |
JP5023001B2 (ja) | 編集装置、編集方法及びプログラム | |
JP2005312774A (ja) | 医用画像表示システム及びその表示方法 | |
JPS63118833A (ja) | マルチウインドウ表示・操作方式 | |
JPWO2021206053A5 (ja) |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21785651 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022514063 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20227038232 Country of ref document: KR Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2021785651 Country of ref document: EP Effective date: 20221107 |