WO2023228229A1 - 画像処理装置、画像処理方法、およびプログラム - Google Patents

画像処理装置、画像処理方法、およびプログラム Download PDF

Info

Publication number
WO2023228229A1
WO2023228229A1 PCT/JP2022/021048 JP2022021048W WO2023228229A1 WO 2023228229 A1 WO2023228229 A1 WO 2023228229A1 JP 2022021048 W JP2022021048 W JP 2022021048W WO 2023228229 A1 WO2023228229 A1 WO 2023228229A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
cell
characteristic information
target image
image processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/021048
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
康夫 尾見
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to EP22943638.1A priority Critical patent/EP4530627A4/en
Priority to JP2024522727A priority patent/JPWO2023228229A1/ja
Priority to PCT/JP2022/021048 priority patent/WO2023228229A1/ja
Publication of WO2023228229A1 publication Critical patent/WO2023228229A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N33/00—Investigating or analysing materials by specific methods not covered by groups G01N1/00 - G01N31/00
    • G01N33/48—Biological material, e.g. blood, urine; Haemocytometers
    • G01N33/50—Chemical analysis of biological material, e.g. blood, urine; Testing involving biospecific ligand binding methods; Immunological testing
    • G01N33/53—Immunoassay; Biospecific binding assay; Materials therefor
    • G01N33/575—Immunoassay; Biospecific binding assay; Materials therefor for cancer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/695—Preprocessing, e.g. image segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/698—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Definitions

  • the present invention relates to an image processing device, an image processing method, and a program.
  • Patent Document 1 describes a technology that normalizes WSI data by preprocessing each of multiple WSIs (Whole Slide Images) with different dyes to suppress sample variations, and applies the normalized WSI data to a machine learning model. Disclosed.
  • Patent Document 1 does not take into account the relationship between images for learning and images input to a machine learning model for image recognition in pathological diagnosis. Therefore, the technique disclosed in Patent Document 1 has a problem in that the accuracy of image recognition of an image input to a machine learning model may be lowered.
  • One aspect of the present invention has been made in view of the above problems, and one example of its purpose is to provide a technique that improves the accuracy of classifying sample cells as benign or malignant in pathological diagnosis.
  • An image processing device includes a first acquisition unit that acquires a target image including a sample cell as an object, and an image processing device that inputs an image including a cell as an object, and determines whether the cell is a benign cell or a malignant cell.
  • a second acquisition means for acquiring characteristic information indicating the characteristics of the training image used for learning the trained estimation model to estimate whether the target image is a characteristic of the target image; generating means for generating an image in which the image is changed.
  • an image processing device acquires a target image including a sample cell as a subject, receives an image including a cell as a subject, and determines whether the cell is a benign cell or a malignant cell. acquiring characteristic information indicating characteristics of a training image used for learning a trained estimation model to estimate whether the image is a cell; and changing characteristics of the target image by referring to the characteristic information. and generating an image.
  • a program according to one aspect of the present invention is a program that causes a computer to function as an image processing device, and the program causes the computer to function as a first acquisition means for acquiring a target image including sample cells as a subject;
  • the first step is to obtain characteristic information indicating the characteristics of the training image used to train the trained estimation model to estimate whether the cell in question is a benign cell or a malignant cell. 2
  • a generating means that generates an image with changed characteristics of the target image by referring to the characteristic information.
  • FIG. 1 is a block diagram showing the configuration of an image processing device according to an exemplary embodiment 1 of the present invention.
  • FIG. 1 is a flow diagram of an image processing method according to exemplary embodiment 1 of the present invention;
  • FIG. 2 is a block diagram showing the configuration of a classification device according to a second exemplary embodiment of the present invention.
  • FIG. 7 is a flowchart illustrating an example of a process flow in which a generation unit according to exemplary embodiment 2 of the present invention generates an image with a changed noise level of a target image.
  • FIG. 7 is a flowchart illustrating an example of a process flow in which the generation unit according to the second exemplary embodiment of the present invention generates an image in which the hue of the target image is changed.
  • FIG. 1 is a block diagram showing the configuration of an image processing device according to an exemplary embodiment 1 of the present invention.
  • FIG. 1 is a flow diagram of an image processing method according to exemplary embodiment 1 of the present invention;
  • FIG. 2 is a block diagram
  • FIG. 7 is a diagram showing an example of a hue histogram in exemplary embodiment 2 of the present invention.
  • FIG. 7 is a flowchart illustrating an example of a process flow in which the generation unit changes the resolution of a target image according to the second exemplary embodiment of the present invention.
  • FIG. 7 is a diagram showing an example of a micrometer image in exemplary embodiment 2 of the present invention.
  • FIG. 7 is a flow diagram showing another example of the flow of processing in which the generation unit changes the resolution of the target image according to the second exemplary embodiment of the present invention.
  • 1 is a block diagram illustrating an example of the hardware configuration of an image processing device and a classification device in each exemplary embodiment of the present invention.
  • the image processing apparatus 1 is an image processing apparatus that generates an image in which the characteristics of a target image containing sample cells as a subject are changed.
  • the image processing device 1 inputs an image including a cell as a subject, and uses a learning image used to train an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell.
  • An image is generated with the characteristics of the target image changed according to the characteristics of the target image.
  • Image characteristics refer to the properties and characteristics that an image has. Examples include the noise level, which indicates the amount of noise contained in an image; the hue, which consists of the hue, brightness, and saturation of the image; the resolution, which indicates the number of pixels included in a given length in the image; Examples include information indicating the size of a predetermined object included.
  • the specific configuration of the estimation model does not limit the present exemplary embodiment, but as an example, CNN (Convolution Neural Network), RNN (Recurrent Neural Network), or a combination thereof can be used.
  • CNN Convolution Neural Network
  • RNN Recurrent Neural Network
  • a non-neural network model such as a random forest or a support vector machine may be used.
  • FIG. 1 is a block diagram showing the configuration of an image processing device 1 according to this exemplary embodiment.
  • the image processing device 1 includes a first acquisition section 11, a second acquisition section 12, and a generation section 13.
  • the first acquisition section 11, the second acquisition section 12, and the generation section 13 are configured to realize a first acquisition means, a second acquisition means, and a generation means, respectively, in this exemplary embodiment.
  • the first acquisition unit 11 acquires a target image that includes sample cells as a subject.
  • the first acquisition unit 11 supplies the acquired target image to the generation unit 13.
  • the second acquisition unit 12 receives an image including a cell as an object, and generates a training image used for training a trained estimation model to estimate whether the cell is a benign cell or a malignant cell. Obtain characteristic information indicating characteristics. The second acquisition unit 12 supplies the acquired characteristic information to the generation unit 13.
  • the generation unit 13 refers to the characteristic information supplied from the second acquisition unit 12 and generates an image with changed characteristics of the target image supplied from the first acquisition unit 11.
  • the first acquisition unit 11 acquires a target image including a sample cell as a subject, and the first acquisition unit 11 receives an image including a cell as a subject, and a second acquisition unit 12 that acquires characteristic information indicating the characteristics of the learning image used for learning the pre-trained estimation model to estimate whether the cell is a benign cell or a malignant cell;
  • a configuration is adopted that includes a generation unit 13 that generates an image with changed characteristics of the target image.
  • the image processing device 1 generates an image in which the characteristics of the target image are changed in accordance with the characteristics of the learning image used for learning the estimation model. According to the image processing device 1, it is possible to improve the accuracy of classifying sample cells as benign or malignant in pathological diagnosis.
  • FIG. 2 is a flow diagram showing the flow of the image processing method S1 according to the exemplary embodiment.
  • Step S11 the first acquisition unit 11 acquires a target image including sample cells as a subject.
  • the first acquisition unit 11 supplies the acquired target image to the generation unit 13.
  • Step S12 the second acquisition unit 12 inputs an image containing a cell as an object, and uses it to train a pre-trained estimation model to estimate whether the cell is a benign cell or a malignant cell. Obtain characteristic information indicating the characteristics of the learning image. The second acquisition unit 12 supplies the acquired characteristic information to the generation unit 13.
  • step S13 the generation unit 13 refers to the characteristic information supplied from the second acquisition unit 12 and generates an image with changed characteristics of the target image supplied from the first acquisition unit 11.
  • a target image including a sample cell as a subject is acquired, an image including a cell as a subject is input, and the cell is a benign cell.
  • a configuration including: generating an image is adopted. Therefore, according to the image processing method S1 according to the present exemplary embodiment, the same effects as the image processing apparatus 1 described above can be obtained.
  • Example Embodiment 2 A second exemplary embodiment of the invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.
  • the classification device 2 is a device that classifies sample cells as benign cells or malignant cells.
  • the classification device 2 inputs an image containing a cell as an object, and inputs an image containing a sample cell as an object into an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell. By inputting , the sample cells are classified as benign cells or malignant cells.
  • the classification device 2 acquires a target image that includes sample cells as a subject, and generates an image that matches the characteristics of the target image to the characteristics of the learning image used for learning the estimation model. Then, the classification device 2 inputs the generated image into the estimation model, and refers to the results of the estimation model to classify whether the sample cell is a benign cell or a malignant cell. That is, the classification device 2 includes the configuration of the image processing device 1 described above, and determines whether a sample cell is a benign cell or a malignant cell by inputting an image generated by the image processing device 1 into an estimation model. Classify.
  • the learning image is an image obtained by digitizing an image that includes a sample cell as a subject
  • the target image is an image obtained by photographing an image of the sample cell displayed using a microscope.
  • the training images are digitized images, many training images can be acquired, so the classification device 2 can increase the accuracy of the estimation model.
  • the target image is an image obtained by photographing an image projected by a microscope
  • the classification device 2 can be used for cytodiagnosis in rapid on-site evaluation (ROSE).
  • the image characteristics and estimation model are as described above.
  • FIG. 3 is a block diagram showing the configuration of the classification device 2 according to this exemplary embodiment.
  • the classification device 2 includes a control section 10, a storage section 21, and an input/output section 22.
  • the storage unit 21 stores data referenced by the control unit 10, which will be described later. Examples of data stored in the storage unit 21 include characteristic information PI indicating the characteristics of the learning image TP used for learning the estimation model, a target image SP containing sample cells as a subject, and a learning image TP. .
  • the input/output unit 22 is an interface that acquires data from other connected devices or outputs data to other connected devices.
  • the input/output unit 22 supplies data acquired from other devices to the control unit 10, and outputs data supplied from the control unit 10 to other devices.
  • the input/output unit 22 may be a communication module that communicates with other devices via a network.
  • a network includes a wireless LAN (Local Area Network), wired LAN, WAN (Wide Area Network), public line network, mobile data communication network, or , a combination of these networks can be used.
  • the control unit 10 controls each component included in the classification device 2.
  • the control unit 10 also functions as a first acquisition unit 11, a second acquisition unit 12, a generation unit 13, and a classification unit 14, as shown in FIG.
  • the first acquisition section 11, the second acquisition section 12, the generation section 13, and the classification section 14 each include a first acquisition means, a second acquisition means, a generation means, and a classification means in this exemplary embodiment. This is the configuration that achieves this.
  • the first acquisition unit 11 acquires a target image SP that includes sample cells as a subject.
  • the first acquisition unit 11 stores the acquired target image SP in the storage unit 21.
  • the second acquisition unit 12 receives as input an image including a cell as a subject, and uses a training image TP used for learning an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell. Obtain characteristic information PI indicating the characteristics of.
  • the second acquisition unit 12 may be configured to acquire the characteristic information PI itself, or acquire one or more learning images TP, and derive the characteristic information PI from the one or more learning images TP. It may be obtained by The second acquisition unit 12 stores the acquired characteristic information PI (and the learning image TP) in the storage unit 21.
  • characteristic information PI indicating a characteristic obtained by averaging the characteristics of each of a plurality of learning images TP, and a characteristic that is the median value of the variation of characteristics in each of a plurality of learning images TP.
  • the second acquisition unit 12 includes a derivation unit 121.
  • the deriving unit 121 derives characteristic information PI from the learning image TP.
  • the derivation unit 121 extracts the characteristics of the learning image TP, and derives characteristic information PI indicating the extracted characteristics.
  • the derivation unit 121 extracts the characteristics of each of the plurality of learning images TP, and calculates a characteristic obtained by averaging the extracted characteristics, or a characteristic that is the median value of the variation of the extracted characteristics.
  • the characteristic information PI shown is derived.
  • the generation unit 13 generates an image with changed characteristics of the target image SP by referring to the characteristic information PI.
  • the generation unit 13 supplies the generated image to the classification unit 14.
  • the generation unit 13 generates an image in which the noise level of the target image SP is changed. As another example, the generation unit 13 generates an image in which the hue of the target image SP is changed. As yet another example, the generation unit 13 generates an image in which the resolution of the target image SP is changed. As yet another example, the generation unit 13 generates an image in which the size of the object included in the target image SP is changed.
  • the generation unit 13 may generate an image in which a plurality of characteristics of the target image SP are changed.
  • the generation unit 13 generates an image in which the noise level, color tone, and resolution of the target image SP are changed.
  • the generation unit 13 may generate images in which the characteristics are changed in a predetermined order, or may generate images in which a plurality of characteristics of the target image SP are changed regardless of the order. An image with changed characteristics may be generated.
  • the generation unit 13 first generates an image with the noise level of the target image SP changed. Next, the generation unit 13 performs a process of changing the hue on the image after the noise level has been changed, and generates an image with the changed hue. Then, the generation unit 13 performs a process of changing the resolution on the image after changing the hue, and generates an image with the changed resolution.
  • the classification unit 14 inputs the image supplied from the generation unit 13 into the estimation model, and classifies whether the sample cells included as subjects in the image supplied from the generation unit 13 are benign cells or malignant cells.
  • the classification unit 14 outputs the classified results via the input/output unit 22.
  • the classification unit 14 determines that the sample cell included as a subject in the image supplied from the generation unit 13 is a benign cell. Classify. On the other hand, if the estimation result output from the estimation model indicates that the sample cell is a malignant cell, the classification unit 14 classifies the sample cell included as a subject in the image supplied from the generation unit 13 as a malignant cell. do.
  • FIG. 4 is a flow diagram illustrating an example of the flow of processing S2 in which the generation unit 13 according to the present exemplary embodiment generates an image with a changed noise level of the target image SP.
  • the characteristic information PI is information indicating the noise level in the learning image TP, and is information indicating the noise level in a background area that does not include an object as a subject, and the generation unit 13 applies a Gaussian filter. The process of calculating the noise level will be explained.
  • Step S21 the generation unit 13 extracts a background region, which is a region that does not include an object as a subject, from the target image SP.
  • a background region which is a region that does not include an object as a subject.
  • the method by which the generation unit 13 extracts the background region is not particularly limited, one example is a method in which a region having a predetermined pixel value or less is extracted as the background region.
  • Step S22 the generation unit 13 applies a Gaussian filter to the extracted background region.
  • the sigma value of the Gaussian filter applied by the generation unit 13 is not particularly limited, but examples thereof include values such as 1.0 and 1.5.
  • step S23 the generation unit 13 calculates the noise level of the background area after applying the Gaussian filter.
  • the generation unit 13 calculates, as the noise level, the standard deviation of the pixel values in the background region after applying the Gaussian filter. Then, the generation unit 13 calculates the difference between the calculated standard deviation of pixel values in the background region and the standard deviation indicated by the characteristic information PI.
  • step S24 the generation unit 13 determines whether the calculated difference is within a predetermined range.
  • step S25 If it is determined in step S24 that the calculated difference is not within the predetermined range (step S24: NO), in step S25, the generation unit 13 changes the sigma value of the Gaussian filter to be applied.
  • the generation unit 13 increases the sigma value by 10%.
  • the generation unit 13 lowers the sigma value by 10%. That is, the generation unit 13 derives a sigma value such that the difference between the noise level indicated by the characteristic information PI and the noise level in the background region of the target image SP is within a predetermined range.
  • the generation unit 13 returns to the process of step S22 in order to apply the Gaussian filter with the changed sigma value.
  • step S26 In step S24, if it is determined that the calculated difference is within the predetermined range (step S24: YES), in step S26, the generation unit 13 calculates the noise level indicated by the characteristic information PI and the background of the target image SP.
  • a Gaussian filter with a sigma value whose difference from the noise level in the region is within a predetermined range is applied to the target image SP to generate an image with a changed noise level. That is, the generation unit 13 generates an image in which the noise level of the target image SP is changed so that the noise level in the background region of the extracted target image SP approaches the noise level indicated by the characteristic information PI.
  • the generation unit 13 generates an image in which the noise level of the target image SP is changed so that the noise level in the background region of the extracted target image SP approaches the noise level indicated by the characteristic information PI. Therefore, the generation unit 13 can input an image having a noise level equivalent to the noise level of the learning image TP to the estimation model, thereby improving the accuracy of classifying sample cells as benign or malignant in pathological diagnosis. be able to.
  • FIG. 5 is a flow diagram illustrating an example of the flow of processing S3 in which the generation unit 13 according to the present exemplary embodiment generates an image in which the hue of the target image SP is changed.
  • the characteristic information PI is information indicating the hue in the learning image TP, and is information indicating the hue of an object area that includes an object as a subject. A case will be described in which the information indicates histograms of hue, saturation, and lightness in the object area of the TP.
  • Step S31 the generation unit 13 extracts an object region, which is a region including an object as a subject, in the target image SP.
  • the method by which the generation unit 13 extracts an object region is not particularly limited, but one example is a method in which a region having a predetermined pixel value or more is extracted as an object region.
  • step S32 the generation unit 13 generates a histogram of the hue of the extracted object area as information indicating the hue of the extracted object area. As an example, the generation unit 13 generates histograms for each of the hue, saturation, and lightness of the extracted object area.
  • step S33 the generation unit 13 generates an image in which the hue in the object area of the extracted target image SP is changed so as to approach the hue indicated by the characteristic information PI.
  • An example of the process of the generation unit 13 in step S33 will be described with reference to FIG. 6.
  • FIG. 6 is a diagram illustrating an example of a hue histogram in this exemplary embodiment. Although the hue histogram will be described below, the same applies to the saturation and lightness histograms.
  • the hue histogram in the object region of the target image SP generated in step S32 is the middle histogram in FIG. Mean sp1 , the left end a sp1 and the right end b sp1 of the 95% confidence interval are calculated.
  • the generation unit 13 calculates the changed value Out using the following equation (1).
  • Each variable in equation (1) is as follows.
  • Var tar Value obtained by subtracting the lower limit from the upper limit of the 95% confidence interval of the hue value In in the target histogram. That is, right end b pi - left end a pi .
  • Var poi1 A value obtained by subtracting the lower limit value from the upper limit value of the 95% confidence interval of the hue value In in the histogram of the target image SP. That is, the right end b sp1 - the left end a sp1 .
  • the generation unit 13 uses equation (1) to calculate the lower limit value from the upper limit value of the 95% confidence interval for a certain hue value In in the target image SP in the 95% confidence interval of each histogram.
  • the subtracted value Var poi1 is changed to match the target histogram Var tar , thereby converting it into a target hue value.
  • a hue histogram in the object area of the image after conversion is shown at the bottom of FIG.
  • the mean value mean sp2 of the hue histogram in the object region after conversion, the left end a sp2 and the right end b sp2 of the 95% confidence interval are the mean value mean sp2 of the hue histogram in the object region of the target image SP.
  • the mean value mean pi of the histogram indicated by the characteristic information PI is closer to the left end a pi and the right end b pi of the 95% confidence interval.
  • the generation unit 13 generates an image in which the hue in the object area of the target image SP is changed so as to approach the hue indicated by the characteristic information PI. Therefore, the generation unit 13 can input an image having a hue similar to that of the learning image TP to the estimation model, thereby improving the accuracy of classifying sample cells as benign or malignant in pathological diagnosis.
  • the generation unit 13 may change the hue of the background area of the target image SP in addition to the object area of the target image SP.
  • the generation unit 13 may be configured to refer to the background area of the learning image TP.
  • the generation unit 13 may acquire the average value of the pixel values of the background area of the learning image TP, and change the pixel value of the background area of the target image SP to the average value.
  • the generation unit 13 extracts the background regions of each of the plurality of learning images TP, and calculates the average value of the pixel values of each of the extracted background regions. The generation unit 13 may then change the pixel value of the background area of the target image SP to the calculated average value. In this case, the generation unit 13 may use the median value of variations in pixel values instead of the average value of pixel values.
  • FIG. 7 is a flow diagram illustrating an example of the process S4 in which the generation unit 13 changes the resolution of the target image SP according to the present exemplary embodiment.
  • the characteristic information PI is information indicating the resolution of the learning image TP, and a process in which the generation unit 13 refers to an image including a micrometer as an object will be described below.
  • Step S41 the generation unit 13 acquires a micrometer image taken at the same resolution as the target image SP.
  • An example of a micrometer image will be described with reference to FIG. 8.
  • FIG. 8 is a diagram illustrating an example of a micrometer image in this exemplary embodiment.
  • the generation unit 13 acquires an image including a micrometer as an object, as shown in image P1 of FIG.
  • step S42 the generation unit 13 calculates the resolution of the micrometer image.
  • the generation unit 13 first generates an image obtained by binarizing the image P1, as shown in the image P2 of FIG. Next, as shown in image P3 in FIG. 8, the generation unit 13 performs closing processing by performing dilation processing and erosion processing on image P2. Then, as shown in the image P4 of FIG. 8, the generation unit 13 generates the coordinates of the leftmost pixel of the micrometer ( X L , Y L ) and the coordinates (X R , Y R ) of the rightmost pixel of the micrometer are detected, and the resolution is calculated using the following equation (2).
  • L in equation (2) is the length in micrometers.
  • the generation unit 13 calculates the resolution by calculating the number of pixels of the target image SP included in the predetermined length using equation (2).
  • step S43 the generation unit 13 generates an image with the calculated resolution changed to the resolution indicated by the characteristic information PI.
  • the generation unit 13 generates an image with the resolution of the target image SP changed so that the resolution indicated by the characteristic information PI and the resolution of the target image SP are the same. Therefore, it is possible to input into the estimation model an image in which the size of the object included as a subject in the training image TP is equal to the size of the object included as a subject in the target image SP. The accuracy of benign or malignant classification can be improved.
  • FIG. 9 is a flowchart showing another example of the process S5 in which the generation unit 13 changes the resolution of the target image SP according to the present exemplary embodiment.
  • the characteristic information PI is information indicating the size of a predetermined object included as a subject in the learning image TP
  • the generation unit 13 describes a configuration for extracting a predetermined object included as a subject in the target image SP. explain.
  • the predetermined object is not particularly limited, but objects with small individual differences in size are preferable, and examples include red blood cells and lymphocytes.
  • the generation unit 13 uses red blood cells or lymphocytes, which have small individual differences, as a reference for the size of the object included in the target image SP, so that the size of the object can be suitably calculated.
  • the predetermined object is a red blood cell
  • Step S51 the generation unit 13 extracts red blood cells included as a subject in the target image SP as a predetermined object included as a subject in the target image SP.
  • An example of a method for the generation unit 13 to extract red blood cells included as subjects in the target image SP is a method using known deep learning segmentation (DL segmentation).
  • Step S52 the generation unit 13 calculates the size of the extracted red blood cells.
  • the generation unit 13 may approximate the extracted red blood cells to an ellipse and calculate the major axis and the minor axis.
  • the generation unit 13 may calculate the major axis and minor axis of the red blood cell with the largest area. As another example, the generation unit 13 may calculate the major axis and minor axis of the red blood cells, which are the median values of the variations in area of the plurality of red blood cells. As yet another example, the generation unit 13 may calculate the average value of each of the major axis and minor axis of a plurality of red blood cells.
  • step S53 the generation unit 13 generates an image in which the resolution of the target image SP is changed so that the calculated red blood cell size (longer axis and shorter axis) becomes the size (longer axis and shorter axis) indicated by the characteristic information PI. generate.
  • the generation unit 13 changes the resolution of the target image SP so that the size of the predetermined object indicated by the characteristic information PI is equal to the size of the predetermined object included as a subject in the target image SP. Generate an image. Therefore, it is possible to input into the estimation model an image in which the size of the object included as a subject in the training image TP is equal to the size of the object included as a subject in the target image SP. The accuracy of benign or malignant classification can be improved.
  • an image containing a cell as an object is input, and a trained estimation is performed to estimate whether the cell is a benign cell or a malignant cell.
  • characteristic information PI indicating the characteristics of the learning image TP used for model learning
  • an image is generated in which the characteristics of the target image SP containing the sample cells as a subject are changed.
  • the classification device 2 employs a configuration in which the generated image is input to the estimation model.
  • the classification device 2 according to the present exemplary embodiment generates an image in which the characteristics of the target image SP are changed in accordance with the characteristics of the training image TP used for learning the estimation model, and then uses the image as the estimation model. Therefore, according to the classification device 2 according to the present exemplary embodiment, it is possible to improve the accuracy of classifying sample cells as benign or malignant in pathological diagnosis.
  • Some or all of the functions of the image processing device 1 and the classification device 2 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.
  • the image processing device 1 and the classification device 2 are realized, for example, by a computer that executes instructions of a program that is software that realizes each function.
  • a computer that executes instructions of a program that is software that realizes each function.
  • An example of such a computer (hereinafter referred to as computer C) is shown in FIG.
  • Computer C includes at least one processor C1 and at least one memory C2.
  • a program P for operating the computer C as the image processing device 1 and the classification device 2 is recorded in the memory C2.
  • the processor C1 reads the program P from the memory C2 and executes it, thereby realizing the functions of the image processing device 1 and the classification device 2.
  • Examples of the processor C1 include a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), and PPU (Physics Processing Unit). , a microcontroller, or a combination thereof.
  • a flash memory for example, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination thereof can be used.
  • the computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and temporarily storing various data. Further, the computer C may further include a communication interface for transmitting and receiving data with other devices. Further, the computer C may further include an input/output interface for connecting input/output devices such as a keyboard, a mouse, a display, and a printer.
  • RAM Random Access Memory
  • the program P can be recorded on a non-temporary tangible recording medium M that is readable by the computer C.
  • a recording medium M for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit can be used.
  • Computer C can acquire program P via such recording medium M.
  • the program P can be transmitted via a transmission medium.
  • a transmission medium for example, a communication network or broadcast waves can be used.
  • Computer C can also obtain program P via such a transmission medium.
  • a first acquisition means that acquires a target image containing sample cells as a subject; and a trained estimation device that receives an image containing cells as a subject and estimates whether the cell is a benign cell or a malignant cell.
  • a second acquisition means for acquiring characteristic information indicating characteristics of a training image used for model learning; and a generation means for generating an image with changed characteristics of the target image by referring to the characteristic information.
  • the characteristic information is information indicating a noise level in the learning image, and is information indicating a noise level in a background area that does not include an object as a subject, and the generating means is configured to extract the background from the target image.
  • the generating means calculates a noise level by applying a Gaussian filter to a background region of the target image, and calculates a noise level when the difference between the noise level indicated by the characteristic information and the noise level in the background region of the target image is a predetermined value.
  • the image processing device which derives a sigma value that falls within the range.
  • the characteristic information is information indicating the color tone in the learning image, and is information indicating the color tone in an object area that includes an object as a subject, and the generating means extracts the object area from the target image.
  • the image processing apparatus according to any one of Supplementary Notes 1 to 3, which generates an image in which a hue in an object area of the extracted target image is changed to be closer to a hue indicated by the characteristic information.
  • appendix 5 The image processing device according to appendix 4, wherein the characteristic information is information indicating histograms of hue, saturation, and brightness in the object area of the learning image.
  • the characteristic information is information indicating the resolution of the learning image
  • the generating means refers to a micrometer image taken at the same resolution as the target image, and converts the resolution of the target image into the characteristic information.
  • the image processing device according to any one of Supplementary Notes 1 to 5, which generates an image whose resolution is changed to that indicated by .
  • the characteristic information is information indicating the size of a predetermined object included as a subject in the learning image, and the generating means extracts the predetermined object included as a subject in the target image, and
  • the image processing device according to any one of Supplementary Notes 1 to 6, which generates an image in which the resolution of the target image is changed so that the size of the object in the image becomes the size indicated by the characteristic information.
  • Appendix 8 The image processing device according to appendix 7, wherein the predetermined object is a red blood cell or a lymphocyte.
  • the second acquisition means acquires characteristic information indicating a characteristic obtained by averaging the characteristics of each of the plurality of learning images, or characteristic information indicating a characteristic that is the median value of the variation in characteristics in each of the plurality of learning images.
  • the image processing device according to any one of Supplementary Notes 1 to 9 to be acquired.
  • the learning image is an image obtained by digitizing an image containing a sample cell as a subject, and the target image is an image obtained by photographing an image of a sample cell reflected under a microscope.
  • a classification device comprising a classification means for classifying.
  • An image processing device acquires a target image that includes a sample cell as an object, and receives an image that includes a cell as an object and uses a trained device to estimate whether the cell is a benign cell or a malignant cell.
  • An image processing method comprising: acquiring characteristic information indicating characteristics of a training image used for learning an estimation model; and generating an image with changed characteristics of the target image by referring to the characteristic information. .
  • a program that causes a computer to function as an image processing device includes a first acquisition means that acquires a target image containing sample cells as a subject, and an image that includes cells as a subject as an input, a second acquisition means for acquiring characteristic information indicating characteristics of a training image used for learning a trained estimation model to estimate whether a cell is a benign cell or a malignant cell; A program that functions as a generating means for generating an image with changed characteristics of the target image by referring to the target image.
  • the processor includes at least one processor, and the processor performs a first acquisition process of acquiring a target image containing a sample cell as an object, and inputs an image including a cell as an object, and determines whether the cell is a benign cell or a malignant cell. a second acquisition process that acquires characteristic information indicating the characteristics of the training image used for learning the trained estimation model to estimate whether or not the target image exists; An image processing device that executes generation processing that generates a modified image.
  • this image processing device may further include a memory, and this memory includes a memory for causing the processor to execute the first acquisition process, the second acquisition process, and the generation process.
  • the program may be stored. Further, this program may be recorded on a computer-readable non-transitory tangible recording medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Immunology (AREA)
  • Hematology (AREA)
  • Urology & Nephrology (AREA)
  • Chemical & Material Sciences (AREA)
  • Multimedia (AREA)
  • Pathology (AREA)
  • Food Science & Technology (AREA)
  • Biochemistry (AREA)
  • Cell Biology (AREA)
  • Biotechnology (AREA)
  • Microbiology (AREA)
  • Medicinal Chemistry (AREA)
  • Analytical Chemistry (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
PCT/JP2022/021048 2022-05-23 2022-05-23 画像処理装置、画像処理方法、およびプログラム Ceased WO2023228229A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP22943638.1A EP4530627A4 (en) 2022-05-23 2022-05-23 IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM
JP2024522727A JPWO2023228229A1 (https=) 2022-05-23 2022-05-23
PCT/JP2022/021048 WO2023228229A1 (ja) 2022-05-23 2022-05-23 画像処理装置、画像処理方法、およびプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/021048 WO2023228229A1 (ja) 2022-05-23 2022-05-23 画像処理装置、画像処理方法、およびプログラム

Publications (1)

Publication Number Publication Date
WO2023228229A1 true WO2023228229A1 (ja) 2023-11-30

Family

ID=88918809

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/021048 Ceased WO2023228229A1 (ja) 2022-05-23 2022-05-23 画像処理装置、画像処理方法、およびプログラム

Country Status (3)

Country Link
EP (1) EP4530627A4 (https=)
JP (1) JPWO2023228229A1 (https=)
WO (1) WO2023228229A1 (https=)

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108172278A (zh) * 2018-01-24 2018-06-15 北京航空航天大学 一种he染色病理图像颜色归一化方法
JP2018535471A (ja) * 2015-09-23 2018-11-29 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 正規化及びアーチファクト補正のための画像処理方法及び画像処理装置
JP2019036255A (ja) * 2017-08-21 2019-03-07 有限会社ワッツ 癌等の遠隔診断用複合サーバ装置、癌等の遠隔診断方法および遠隔診断プログラム
WO2019102829A1 (ja) * 2017-11-24 2019-05-31 国立大学法人大阪大学 画像解析方法、画像解析装置、画像解析システム、画像解析プログラム、記録媒体
JP2019139370A (ja) * 2018-02-07 2019-08-22 パナソニックＩｐマネジメント株式会社 人物検知装置
WO2019159821A1 (ja) * 2018-02-15 2019-08-22 国立大学法人新潟大学 高頻度変異型癌の判別システム、プログラム及び方法
US20200167910A1 (en) * 2018-11-28 2020-05-28 International Business Machines Corporation Recognizing pathological images captured by alternate image capturing devices
JP2020518915A (ja) * 2017-04-27 2020-06-25 パスハラキス スタブロスＰＡＳＣＨＡＬＡＫＩＳ， Ｓｔａｖｒｏｓ 自動眼底画像分析用のシステムおよび方法
JP2020107239A (ja) * 2018-12-28 2020-07-09 株式会社Ｃｙｇａｍｅｓ 画像処理方法、画像処理システム及びプログラム
CN111564205A (zh) * 2020-05-14 2020-08-21 湖南品信生物工程有限公司 一种病理图像染色归一化方法及装置
CN112215801A (zh) * 2020-09-14 2021-01-12 北京航空航天大学 一种基于深度学习和机器学习的病理图像分类方法及系统
CN112348059A (zh) * 2020-10-23 2021-02-09 北京航空航天大学 基于深度学习的多种染色病理图像分类方法及系统
JP2021506022A (ja) * 2017-12-07 2021-02-18 ベンタナ メディカル システムズ， インコーポレイテッド 生体画像における連帯的細胞および領域分類のための深層学習システムならびに方法
CN112435743A (zh) * 2020-12-09 2021-03-02 上海市第一人民医院 一种基于机器学习的膀胱癌病理组学智能诊断方法及其预后模型
JP2021515240A (ja) * 2018-04-12 2021-06-17 グーグル エルエルシーＧｏｏｇｌｅ ＬＬＣ 定量的バイオマーカデータのオーバレイを有する病理学用拡張現実顕微鏡
JP2021532350A (ja) * 2018-07-31 2021-11-25 ザ・リージエンツ・オブ・ザ・ユニバーシテイ・オブ・コロラド、ア・ボデイー・コーポレイト 機械学習を適用して高スループットシステムにおけるマイクロコピー画像を分析するためのシステムおよび方法
WO2022064763A1 (ja) * 2020-09-28 2022-03-31 富士フイルム株式会社 学習装置、方法およびプログラム、画像生成装置、方法およびプログラム、学習済みモデル、仮想画像並びに記録媒体
WO2022070491A1 (ja) * 2020-09-29 2022-04-07 株式会社島津製作所 画像解析装置

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018535471A (ja) * 2015-09-23 2018-11-29 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 正規化及びアーチファクト補正のための画像処理方法及び画像処理装置
JP2020518915A (ja) * 2017-04-27 2020-06-25 パスハラキス スタブロスＰＡＳＣＨＡＬＡＫＩＳ， Ｓｔａｖｒｏｓ 自動眼底画像分析用のシステムおよび方法
JP2019036255A (ja) * 2017-08-21 2019-03-07 有限会社ワッツ 癌等の遠隔診断用複合サーバ装置、癌等の遠隔診断方法および遠隔診断プログラム
WO2019102829A1 (ja) * 2017-11-24 2019-05-31 国立大学法人大阪大学 画像解析方法、画像解析装置、画像解析システム、画像解析プログラム、記録媒体
JP2021506022A (ja) * 2017-12-07 2021-02-18 ベンタナ メディカル システムズ， インコーポレイテッド 生体画像における連帯的細胞および領域分類のための深層学習システムならびに方法
CN108172278A (zh) * 2018-01-24 2018-06-15 北京航空航天大学 一种he染色病理图像颜色归一化方法
JP2019139370A (ja) * 2018-02-07 2019-08-22 パナソニックＩｐマネジメント株式会社 人物検知装置
WO2019159821A1 (ja) * 2018-02-15 2019-08-22 国立大学法人新潟大学 高頻度変異型癌の判別システム、プログラム及び方法
JP2021515240A (ja) * 2018-04-12 2021-06-17 グーグル エルエルシーＧｏｏｇｌｅ ＬＬＣ 定量的バイオマーカデータのオーバレイを有する病理学用拡張現実顕微鏡
JP2021532350A (ja) * 2018-07-31 2021-11-25 ザ・リージエンツ・オブ・ザ・ユニバーシテイ・オブ・コロラド、ア・ボデイー・コーポレイト 機械学習を適用して高スループットシステムにおけるマイクロコピー画像を分析するためのシステムおよび方法
US20200167910A1 (en) * 2018-11-28 2020-05-28 International Business Machines Corporation Recognizing pathological images captured by alternate image capturing devices
JP2020107239A (ja) * 2018-12-28 2020-07-09 株式会社Ｃｙｇａｍｅｓ 画像処理方法、画像処理システム及びプログラム
CN111564205A (zh) * 2020-05-14 2020-08-21 湖南品信生物工程有限公司 一种病理图像染色归一化方法及装置
CN112215801A (zh) * 2020-09-14 2021-01-12 北京航空航天大学 一种基于深度学习和机器学习的病理图像分类方法及系统
WO2022064763A1 (ja) * 2020-09-28 2022-03-31 富士フイルム株式会社 学習装置、方法およびプログラム、画像生成装置、方法およびプログラム、学習済みモデル、仮想画像並びに記録媒体
WO2022070491A1 (ja) * 2020-09-29 2022-04-07 株式会社島津製作所 画像解析装置
CN112348059A (zh) * 2020-10-23 2021-02-09 北京航空航天大学 基于深度学习的多种染色病理图像分类方法及系统
CN112435743A (zh) * 2020-12-09 2021-03-02 上海市第一人民医院 一种基于机器学习的膀胱癌病理组学智能诊断方法及其预后模型

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4530627A4

Also Published As

Publication number Publication date
JPWO2023228229A1 (https=) 2023-11-30
EP4530627A4 (en) 2025-09-24
EP4530627A1 (en) 2025-04-02

Similar Documents

Publication Publication Date Title
Zhao et al. Dermoscopy image classification based on StyleGAN and DenseNet201
JP5801379B2 (ja) 確率密度関数推定器
US20200125889A1 (en) Information processing device, recording medium recording information processing program, and information processing method
JP7294695B2 (ja) 学習済モデルによるプログラム、情報記録媒体、分類装置、ならびに、分類方法
CN110059700B (zh) 图像摩尔纹识别方法、装置、计算机设备及存储介质
CN119741312B (zh) 一种基于深度学习的mri医学图像分割方法及系统
CN116503858B (zh) 一种基于生成模型的免疫荧光图像分类方法及系统
CN118279667A (zh) 一种面向皮肤镜图像的深度学习白癜风识别方法
CN113344028A (zh) 乳腺超声序列图像分类方法及装置
CN118097755A (zh) 一种基于yolo网络的智慧人脸身份识别方法
CN117315702B (zh) 基于集合预测的文本检测方法、系统及介质
CN110766657A (zh) 一种激光干扰图像质量评价方法
WO2016192213A1 (zh) 一种图像特征提取方法和装置、存储介质
Azaza et al. Context proposals for saliency detection
CN111127400A (zh) 一种乳腺病变检测方法和装置
Li et al. A modified histogram equalization approach for image contrast enhancement
CN109948706B (zh) 结合深度学习与特征多尺度融合的微钙化簇检测方法
CN110458024A (zh) 活体检测方法及装置和电子设备
WO2023228229A1 (ja) 画像処理装置、画像処理方法、およびプログラム
Li et al. Sublingual vein extraction algorithm based on hyperspectral tongue imaging technology
JPWO2023228229A5 (https=)
Oyewole et al. Bridging communication gap among people with hearing impairment: An application of image processing and artificial neural network
US9918686B2 (en) Automated fibro-glandular (FG) tissue segmentation in digital mammography using fuzzy logic
Ahmad et al. Brain tumor detection using image segmentation and classification
CN115424163A (zh) 唇形修改伪造视频检测方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22943638

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18863714

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2024522727

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2022943638

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022943638

Country of ref document: EP

Effective date: 20241223

WWP Wipo information: published in national office

Ref document number: 2022943638

Country of ref document: EP