WO2023100475A1 - 医療画像処理装置及びその作動方法 - Google Patents

医療画像処理装置及びその作動方法 Download PDF

Info

Publication number
WO2023100475A1
WO2023100475A1 PCT/JP2022/037653 JP2022037653W WO2023100475A1 WO 2023100475 A1 WO2023100475 A1 WO 2023100475A1 JP 2022037653 W JP2022037653 W JP 2022037653W WO 2023100475 A1 WO2023100475 A1 WO 2023100475A1
Authority
WO
WIPO (PCT)
Prior art keywords
recognition result
medical image
reliability
recognition
mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/037653
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
美沙紀 後藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2023564765A priority Critical patent/JPWO2023100475A1/ja
Publication of WO2023100475A1 publication Critical patent/WO2023100475A1/ja
Priority to US18/675,095 priority patent/US20240312200A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/987—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns with the intervention of an operator
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the present invention relates to a medical image processing apparatus and its operating method.
  • enhancement processing is performed on the detected lesion candidate region to generate an image that allows the operator to estimate the possibility of false detection of the lesion candidate region.
  • lesion candidate regions detected from a predetermined feature amount are color-coded according to likelihood information (parameters).
  • a notification image is displayed to notify the presence of a lesion candidate region based on the highlighting used.
  • character/image data is obtained from a paper medium on which test results of an eye to be examined are printed, and if there is an erroneous recognition in the obtained results, the user corrects them as appropriate.
  • image recognition matching processing is performed using misrecognition results accumulated by machine learning.
  • Patent No. 6246431 Japanese Unexamined Patent Application Publication No. 2021-115238
  • Patent Document 1 the display mode of the image is changed using the number of times of continuous recognition of the detected lesion candidate area, and the image display is performed so that the possibility of false detection can be estimated.
  • the recognition is determined, and the user corrects the incorrect recognition result on the confirmation/correction screen.
  • Patent Documents 1 and 2 the possibility of misdetection or misrecognition of an image is displayed.
  • it is inefficient to make corrections by inputting arbitrary phrases every time. Therefore, there is a demand for efficient correction with less burden on the user when correcting recognition results of medical images acquired by examinations such as endoscopy and ultrasound.
  • the present invention is a medical image processing apparatus that calculates a reliability indicating the possibility of misrecognition of a specific subject in a medical image, reduces the user's burden based on the reliability, and efficiently corrects the recognition result. and a method of operating the same.
  • the medical image processing apparatus of the present invention comprises a processor, the processor acquires a medical image, performs recognition processing for recognizing a specific subject on the medical image, calculates the reliability of the recognition result of the specific subject, Based on the reliability, an acceptance mode for accepting correction of the recognition result is determined, the correction of the recognition result is accepted in the determined acceptance mode, and the corrected recognition result is displayed.
  • the acceptance mode accepts corrections to the recognition results from the user, and automatically confirms the corrections to the recognition results when a predetermined condition is met.
  • the correction candidates of the recognition result are preferably displayed as options on the monitor, and the user is allowed to select the correction of the recognition result from the correction candidates.
  • the acceptance mode include a second mode in which correction of the recognition result is not accepted from the user and the recognition result is fixed.
  • the reception mode is determined to be the first mode when the reliability is less than the first threshold, and the reception mode is determined to be the second mode when the reliability is equal to or greater than the first threshold.
  • the acceptance mode preferably includes a third mode in which correction of the recognition result is accepted from the user and the recognition result is manually confirmed. It is preferable to determine the reception mode to be the first mode when the reliability is equal to or greater than the second threshold, and to determine the reception mode to be the third mode when the reliability is less than the second threshold.
  • Acquire time-series images which are time-series medical images, perform recognition processing for each medical image that makes up the time-series images, and trust based on the number of times a specific subject is recognized with respect to the number of medical images that make up the time-series images. It is preferable to calculate degrees. It is preferable to calculate the reliability based on the number of times an object different from the specific object is recognized with respect to the number of medical images forming the time-series images.
  • a first recognition process for recognizing a specific subject and a second recognition process for recognizing a subject different from the specific subject are executed, a first recognition result is obtained by the first recognition process, and a second recognition process is performed. It is preferable to acquire a second recognition result by recognition processing, compare the correspondence relationship between the first recognition result and the second recognition result, and calculate the first reliability and the second reliability of the first recognition result.
  • At least one of a first threshold of reliability and a second threshold lower than the first threshold is set for each type of specific subject, and the reliability of at least one of the first threshold and the second threshold is set.
  • the acceptance mode is determined based on
  • a method of operating a medical image processing apparatus includes the steps of acquiring a medical image, executing recognition processing for recognizing a specific subject in the medical image, and calculating the reliability of the recognition result of the specific subject. determining an acceptance mode for accepting correction of the recognition result based on the reliability; accepting the correction of the recognition result in the decided acceptance mode; and displaying the corrected recognition result. have.
  • the present invention it is possible to calculate the reliability indicating the possibility of misrecognition of a specific subject in a medical image, reduce the user's burden based on the reliability, and efficiently correct the recognition result. can.
  • FIG. 4 is an explanatory diagram showing connected devices of the medical image processing apparatus; 1 is a block diagram showing functions of a medical image processing apparatus; FIG. FIG. 4 is an explanatory diagram of a medical image acquired in recognition processing; FIG. 4 is an explanatory diagram of a group of medical images for which recognition processing is performed; FIG. 4 is an explanatory diagram of recognition processing for a group of medical images 41; FIG. 4 is an explanatory diagram of a first mode, which is a reception mode; FIG. 4 is an explanatory diagram of a second mode, which is a reception mode; It is explanatory drawing of the 3rd mode which is reception mode. FIG. 4 is an explanatory diagram of characteristics of each reception mode; FIG.
  • FIG. 11 is an explanatory diagram of a modification of acceptance mode control; It is a flow chart which shows a series of flows of the present invention.
  • FIG. 11 is an explanatory diagram of a function for determining reliability implemented in the second embodiment;
  • FIG. 11 is an explanatory diagram for controlling a reception mode using time-series image recognition results in the second embodiment;
  • FIG. 11 is an explanatory diagram of functions of a recognition processing unit and a reliability determination unit realized in the third embodiment;
  • FIG. 11 is an explanatory diagram of a correspondence relationship between recognition results and scene determination results detected in the third embodiment;
  • FIG. 11 is an explanatory diagram of a screen for receiving correction of incompatible correspondence in the third embodiment;
  • FIG. 1 is a diagram showing devices connected to a medical image processing apparatus 11 in a medical image processing system 10 according to an embodiment of the present invention.
  • the medical image processing system 10 has a medical image processing device 11 , a database 12 , an endoscope system 13 including an endoscope 13 a , a display 14 and a user interface (UI) 15 .
  • Medical image processing apparatus 11 is in electrical communication with database 12 , endoscope system 13 , display 14 and user interface 15 .
  • the database 12 is a device that stores acquired images and can transmit and receive data to and from the medical image processing apparatus 11, and may be a recording medium such as USB (Universal Serial Bus) or HDD (Hard Disc Drive).
  • the display 14 displays images acquired by the medical image processing apparatus 11 .
  • the user interface 15 is an input device for inputting to the medical image processing apparatus 11, and may use a foot pedal, a gesture recognizer, or a voice recognizer in addition to or instead of a keyboard or mouse. Input may be performed using input means provided in the medical device, such as a switch of the endoscope 13a, without being limited to the user interface 15.
  • the database 12 stores videos and still images of medical examinations created by the endoscope system 13 and other medical equipment. Unless otherwise specified, use white light as the illumination light for photographing medical examinations, obtain a video signal of 60 frames per second (60 fps (frame per second)), and record the photographing time. Also, when the video signal is 60 fps, it is preferable to count the time in units of 1/100 seconds.
  • a program in a program memory is operated by a central control unit 20 configured by an image control processor, and an image acquisition unit 21 and an input reception unit 22 are operated.
  • the storage memory 23, the output control unit 24, and the recognition unit 30 are realized.
  • the functions of the recognition unit 30 the functions of the recognition processing unit 31, the reliability determination unit 32, the acceptance mode control unit 33, the recognition result correction unit 34, and the recognition result determination unit 35 are implemented.
  • the image acquisition unit 21 receives data such as images acquired by the endoscope system 13 and images stored in the database 12 and transmits the data to the recognition unit 30 .
  • the input reception unit 22 is connected to the user interface 15.
  • a storage memory 23 temporarily stores an image to be subjected to recognition processing. Instead of the storage memory 23, the database 12 may have the function of temporary storage.
  • the output control unit 24 performs control to display an image on the display 14 .
  • programs related to processing such as image processing are stored in a program memory (not shown).
  • a medical image 40 is an in vivo image captured by an endoscope 13a or the like.
  • a specific object is an object to be recognized during the recognition process, for example, a lesion such as a tumor or inflammation detected as a region of interest R, a treatment tool S such as forceps or a snare, or an observed pyloric region of the stomach or rectum.
  • the medical image 40 after recognition processing accepts correction of the recognition result based on the degree of reliability, which will be described later.
  • the medical image 40 is a frame forming a moving image or a still image.
  • the data to be input to the recognition unit 30 may be a single image for each recognition process.
  • Group 41 may be entered.
  • a time-series medical image 40 refers to a plurality of temporally consecutive medical images 40 . It is preferable that the medical image group 41 to be subjected to recognition processing is not an image of the entire endoscopy, but an image obtained by dividing the range or narrowing the range.
  • the medical image processing apparatus 11 acquires the medical image 40 or the group of medical images 41 from the image acquiring unit 21 and transmits the acquired medical image 40 to the recognizing unit 30 .
  • the recognition processing unit 31 executes recognition processing of a specific subject on the medical image 40 or the medical image group 41 received by the recognition unit 30 using the content learned in advance.
  • the recognition processing unit 31 has a function of a trained model necessary for recognition processing. That is, the recognition processing unit 31 is a computer algorithm composed of a neural network that performs machine learning, and determines whether or not a specific subject is present in each medical image 40 input according to the learning content, and determines whether or not there is a specific subject. , and acquire the recognition result.
  • the recognition result in addition to the name of the specific subject, information such as the matching rate with the content learned in advance is also obtained. Concordance rate is used to calculate reliability.
  • the recognition process can recognize multiple results, not just one result.
  • the recognition result for the region of interest R may be "tumor: 95%” or “tumor: 65%, bleeding: 20%, gastritis: 10%”.
  • the recognition processing unit 31 determines whether or not each of the medical images 40 constituting the group of medical images 41 has a specific object learned in advance. , and if so, recognition processing is performed on the name of the specific subject that has been learned, and a recognition result is obtained.
  • the recognition results include, for example, recognition of a region of interest R such as a lesion in the medical image 40a, non-recognition of a specific subject in the medical image 40b, and recognition of the treatment tool S such as a snare in the medical image 40c.
  • Reliability of the recognition result is represented by a reliability level, which is linked to the medical image 40 together with the recognition result and transmitted to the reliability determination unit 32 .
  • the medical image group 41 which is a group of frame images or a group of still images, may be input to the recognition processing unit 31, or the medical images 40 may be individually input one by one.
  • the reliability determining unit 32 determines the reliability of the recognition result obtained by the recognition processing unit 31.
  • the degree of reliability is determined by using the matching rate with the previously learned learning contents calculated by the recognition processing unit 31, the false detection rate for each recognition target, image quality information, and the like. If the confidence is high, the recognition result is less likely to be correct, ie, less likely to require correction of the recognition result. If the reliability is low, there is a high possibility that the recognition result is incorrect, that is, there is a high possibility that the recognition result needs to be corrected.
  • the degree of reliability is expressed, for example, as a percentage (%).
  • the acceptance mode control unit 33 controls the display mode of the recognition result of the medical image 40 and acceptance of correction based on the determined reliability.
  • the reception mode is determined according to the reliability of the recognition result obtained for each medical image.
  • a medical image 40 is displayed together with the recognition result in a display mode for each reception mode. Each reception mode will be described later.
  • the recognition result correction unit 34 accepts correction of the recognition result according to the acceptance mode. In the correction, selection of a plurality of detected candidate recognition result options or input of correction content in arbitrary words is performed. The correction is made by user input via the user interface 15 and the input contents are reflected on the display 14 . If the correction is not accepted, the recognition result correction unit 34 is not used. The corrected content is saved by receiving a confirmation instruction in the recognition result confirmation unit 35 .
  • the recognition result confirmation unit 35 confirms the recognition result in accordance with the recognition result confirmation instruction for each reception mode.
  • the confirmation result is stored in association with the medical image 40 .
  • the medical images 40 may be transmitted to the database 12 one by one, or may be temporarily stored in the storage memory 23 and then collectively transmitted to the database 12 .
  • the output control unit 24 controls the display mode of the display 14 according to the reception mode determined by the reception mode control unit 33 based on the reliability.
  • the recognition processing of the medical image 40 will be explained.
  • a medical image 40 acquired from the database 12 or the endoscope system 13 by the image acquisition unit 21 is transmitted to the recognition unit 30 and recognition processing is performed. Acquisition of a recognition result of a specific subject in the medical image 40 and calculation of its reliability are performed by the recognition processing.
  • the medical image 40 after recognition processing is displayed on the display 14 in different acceptance modes according to the calculated reliability.
  • Reliability is an index that indicates the reliability of the recognition result calculated by inference of the learning model, and is preferably expressed using a percentage (%).
  • Control of acceptance of corrections based on reliability is realized by switching acceptance modes.
  • the reliability for each medical image 40 used for switching the acceptance mode the highest reliability among the reliability calculated for each item such as the lesion name acquired in the recognition process is used. For example, if the recognition result of the medical image 40 after recognition processing is "tumor: 65%, bleeding: 20%, gastritis: 15%", the medical image 40 is treated as a recognition result with a reliability of 65%. In this case, the screen is displayed and the correction of the recognition result is accepted in the acceptance mode corresponding to the reliability of 65%. Items with low reliability, for example, less than 10% may not be added to the recognition results.
  • the reception mode is determined based on the reliability of the recognition result for the specific subject obtained by recognition processing for each medical image 40 , and the medical image 40 is displayed on the display 14 . Accepts corrections to recognition results obtained by correction methods controlled for each acceptance mode. The corrected recognition result is displayed on the display 14.
  • FIG. 1 is a diagrammatic representation of the recognition result for the specific subject obtained by recognition processing for each medical image 40 .
  • the acceptance mode is set step by step based on the reliability of each medical image 40 after recognition processing.
  • the mode of screen display differs for each reception mode, and the display mode is in accordance with the contents of acceptance of correction in the reception mode.
  • the reliability indicates the possibility of misrecognition.
  • a recognition result with a reliability close to 100% has a low need for correction, and a recognition result with a reliability lower than 50% has a possibility of misrecognition.
  • the acceptance mode is controlled so that acceptance of correction is restricted for the medical image 40 of the recognition result with high reliability, and acceptance of correction is accepted for the medical image 40 of the recognition result with low reliability. Reflect the acceptance status of corrections on the screen display.
  • a display mode for accepting corrections is determined based on the degree of reliability.
  • the acceptance mode is determined according to the reliability of the recognition results detected by the recognition process. Specifically, the first mode when the reliability is a value within a predetermined range, the second mode when the value is greater than the predetermined range, or the second mode when the value is less than the predetermined range Three types of patterns in any one of the three modes are discriminated.
  • the predetermined range of reliability is, for example, 50% or more and less than 90%.
  • the acceptance mode accepts user input for correcting the recognition result, and when a predetermined condition is satisfied, The mode is switched to the first mode for automatically confirming the correction of the recognition result.
  • the predetermined condition is, for example, elapse of a certain period of time.
  • the predetermined condition can be preset for each recognition result according to the importance of the recognition result. For example, if the recognition result is "tumor", the recognition result may be automatically confirmed after 30 seconds, and if the recognition result is "inflammation", the recognition result may be automatically confirmed after 10 seconds.
  • the medical image 40 that has undergone recognition processing and items such as acquired lesion names for a specific subject are displayed on the display 14 in the recognition result display field 50 in the first mode screen.
  • a recognition result is determined by selecting items displayed by user input. For example, if the recognition results for the region of interest R are "tumor: 65%”, “bleeding: 20%”, and “no lesion: 10%", the three items of tumor, bleeding, and no lesion and the option of "other" are displayed. 14, and one of the options is selected by user input. If “others" is selected, the recognition result may be corrected to "unknown", or the acceptance mode may be switched to a third mode for accepting input of arbitrary words, which will be described later. Selection of options by user input includes a method of operating a cursor C displayed on the display 14 using a mouse.
  • the recognition result is confirmed not when an option is selected, but when a predetermined condition is met. If none of the options is selected when the predetermined condition is satisfied, the correction candidate with the highest reliability is automatically determined. Therefore, if the predetermined condition is that a certain period of time has elapsed, the option can be reselected within the certain period of time, and it is possible to prevent the user from spending too much time making judgments due to incomplete selection of correction candidates. If the recognition result is finalized after a certain period of time has elapsed, a countdown to finalization may be displayed on the display 14 . When a correction candidate is selected, the selected correction candidate is determined as the content of correction after a certain period of time has passed since the selection.
  • the mode is switched to the second mode in which the recognition result is fixed without accepting the correction.
  • the display 14 displays the medical image 40 that has undergone recognition processing and the confirmed recognition result in the recognition result display field 50 .
  • the second mode is preferably terminated after a certain period of time has passed or by user input.
  • the user input is preferably a simple action such as pressing a foot pedal or clicking a mouse.
  • means such as a switch button for switching between the first mode and the third mode may be provided in order to supplement the recognition result, correct the detailed name, or deal with the case of obvious misrecognition.
  • Whether or not the reliability is higher than the predetermined range is determined based on the upper threshold of the predetermined range set in advance. This threshold is used as the first threshold, and when the reliability is equal to or higher than the first threshold, the second mode is selected. Since the second mode is an acceptance mode in which the recognition result is fixed without accepting correction, it is preferable that the user sets the first threshold to 90% or higher.
  • the mode is switched to the third mode to accept any input by the user.
  • the display 14 displays a medical image 40 that has undergone recognition processing, an arbitrary input reception field 51 , a correction candidate display field 52 , and a confirmation button 53 .
  • the arbitrary input reception column 51 receives input of arbitrary words by the user via the user interface 15 .
  • the correction candidate display field 52 displays the recognition result obtained by the recognition process as reference information for the user to make arbitrary input. Since the reliability is low, selection of correction candidates obtained as recognition results is not accepted unlike the first mode.
  • the confirm button 53 is a button for confirming the recognition result after the user observes the medical image 40 and inputs the name of a specific subject in the arbitrary input acceptance field 51, and terminates the third mode. The user selects the confirm button 53 by mouse operation or the like.
  • the reliability is lower than the predetermined range.
  • This threshold is the second threshold, and when the reliability is less than the second threshold, the third mode is selected. Since the third mode is an acceptance mode in which the recognition result is determined by user's arbitrary input when the recognition result cannot be determined automatically, the second threshold is preferably less than 50%. Also, even if the reliability is less than the first threshold and greater than or equal to the second threshold, if other options are not recognized during recognition processing, for example, "tumor: 60%" may be switched to the third mode.
  • Selection and determination of items in the first to third modes are transmitted to the recognition result correction unit 34 or the recognition result determination unit 35 via the user interface 15 .
  • User operations are input using a mouse or keyboard, but may be input using other means. For example, selection of an option in the first mode using a foot switch, switching of reception screens in the second mode using a gesture operation, input of arbitrary text in the third mode using voice input, and the like.
  • the procedure for finalizing the reliability value, display mode, and recognition result corresponding to each reception mode and ending the reception mode differs.
  • the user selects a recognition result from a plurality of recognition result options.
  • the second mode when the reliability is greater than or equal to the first threshold does not accept user operations on the recognition result.
  • the third mode when the reliability is less than the second threshold, the reference information of the recognition result obtained from the recognition result is displayed, and the user's input of arbitrary words for the recognition result is accepted.
  • the type of the specific subject that is the recognition target in the recognition result is at least one of lesion, treatment tool, and observation site.
  • the type of specific subject to be detected in recognition processing may be set in advance before performing recognition processing. For example, there are lesion detection for detecting only lesions, treatment tool detection for detecting only treatment tools, and scene determination for determining scenes in which medical images 40 such as parts and organs are shown.
  • At least one of a first threshold of reliability and a second threshold lower than the first threshold may be set.
  • a first threshold of reliability and a second threshold lower than the first threshold may be set.
  • treatment instruments have few items displayed as options, and the selection of options can be narrowed down using endoscopy information. Therefore, even if the second threshold is lowered, it is unlikely that the recognition results will be difficult to determine from the options. . However, even if they are set individually, the second threshold is lower than the first threshold.
  • the reception mode may be controlled in two stages using only the first threshold or the second threshold.
  • FIG. 10(A) is the control of the three-step reception mode using the above-described first and second thresholds
  • FIG. 10(B) uses only the first threshold and the reliability is less than the first threshold. This is a control for accepting correction of the recognition result in either the first mode or the second mode that is equal to or higher than the first threshold. This is control for receiving correction of the recognition result in either a certain first mode or a third mode that is equal to or greater than the second threshold.
  • the control does not use the third mode, and the recognition result is confirmed when predetermined conditions such as automatic confirmation or a certain period of time are satisfied. It is possible to reduce the amount of time required and confirm many recognition results in a short time. It is preferable to use this method when the number of recognition objects to be classified in recognition processing is small, such as treatment tools.
  • the control does not use the first mode.
  • observation of the recognized image is performed.
  • the medical image processing apparatus 11 acquires the medical image 40 captured by the medical examination from the database 12 and the endoscope system 13 (step ST110). Recognition processing for recognizing a specific subject included in the acquired medical image 40 is executed (step ST120). The recognition result in the medical image 40 is obtained by the recognition process together with the calculated reliability (step ST130). Based on the reliability of the medical image 40, an acceptance mode for controlling acceptance of corrections to recognition results is determined (step ST140).
  • step ST150 It is determined whether the reliability is within a predetermined range, that is, less than the first threshold and greater than or equal to the second threshold (step ST150). If less than the first threshold and greater than or equal to the second threshold (Y in step ST150), the acceptance mode is switched to the first mode (step ST210). In the first mode, options that are candidates for the recognition result are displayed on the screen, and the user observes the medical image 40 to determine the recognition result from any of the options (step ST220). In the first mode, the recognition result is determined when a predetermined condition such as elapse of a certain time period is satisfied. (Step ST230). If the user does not select the recognition result, the item with the highest reliability among the plurality of recognition results is determined as the recognition result.
  • a predetermined condition such as elapse of a certain time period
  • step ST150 If the reliability is not within the predetermined range (N in step ST150), it is determined whether the reliability is greater than or less than the predetermined range (step ST160). If the reliability is greater than the predetermined range, that is, equal to or greater than the first threshold (Y in step ST160), the acceptance mode is switched to the second mode (step ST310). In the second mode, correction of the recognition result is not accepted, and the recognition result is automatically determined (step ST320).
  • the acceptance mode is switched to the third mode (step ST410).
  • the third mode correction of the recognition result with arbitrary wording by user input is accepted (step ST420).
  • the user confirms the recognition result by an operation such as pressing a confirmation button (step ST430).
  • the medical image 40 linked with the information of the confirmed recognition result is stored in the database 12 or the storage memory 23 (step ST510). If the recognition results of all the medical images 40 that have undergone recognition processing have not been finalized (N in step ST520), an acceptance mode for controlling acceptance of corrections to the recognition results based on the reliability of the medical images 40. is determined, and the process of determining the recognition result is continued (step ST140). If the recognition results of all the medical images 40 that have undergone recognition processing have been determined (Y in step ST520), the medical image processing ends.
  • the reliability determination unit 32 when performing recognition processing for the medical image group 41, implements the functions of a recognition result totalization unit 32a and a reliability update unit 32b.
  • the recognition result totaling unit 32 a totals the recognition results of the constituent medical images 40 .
  • the recognition result of the medical image group 41 is determined using the number of times a specific subject with the same result has been recognized for each medical image 40 that constitutes it.
  • the reliability update unit 32b determines the reliability of each medical image 40 linked in time series based on the recognition result of the medical image group 41 by updating the value calculated in the recognition process. Controls acceptance of corrections to recognition results.
  • a medical image group 41 which is a time-series medical image 40, is acquired from the database 12 or the endoscope system 13, and recognition processing is performed for each medical image 40 in recognition processing for the medical image group 41, thereby forming the medical image group 41.
  • Reliability is calculated based on the number of recognition times of a specific subject with respect to the number of medical images 40 to be processed. The calculated reliability is used to control acceptance of correction of recognition results for each medical image 40 .
  • the same recognition result that accounts for the majority of the number of times of recognition can be determined as the recognition result of the medical image group 41 .
  • the number of consecutive occurrences of the same recognition result may be used even if it does not constitute a majority.
  • the reliability is updated depending on whether the recognition results of the medical image group 41 and the medical image 40 match, and the reception mode is determined based on the updated reliability.
  • the reception mode may be determined based on whether or not the image matches the medical image group 41 without updating the reliability.
  • the recognition processing of a medical image group 41 consisting of four time-series images will be described as an example.
  • the medical images 40d, 40e, and 40g of "snare” and the medical image 40f of "forceps" are acquired by the recognition processing for detection of the treatment instrument, the recognition result of the medical image group 41 of "snare” three out of four times is The recognition result is "snare”.
  • Medical images 40d, 40e, and 40g which are the same recognition results as the medical image group 41, have increased reliability and become equal to or greater than the first threshold, and the reception mode is the second mode.
  • a medical image 40f, which is a recognition result different from that of the medical image group 41 has a reduced reliability that is less than the first threshold and greater than or equal to the second threshold, and the acceptance mode becomes the first mode.
  • the method of acquiring the recognition result as the medical image group 41, which is the time-series medical images 40, in the recognition result totaling unit 32a is to obtain the same recognition result that occupies a certain percentage or more of the number of the medical images 40, that is, a certain number of images per unit time.
  • the number of times of recognition equal to or more than the number of times of recognition, or the same recognition result that continues for a certain number of times or more is used. For example, when the medical image group 41 is captured at a frame rate of 60 fps and the unit time is 1 second, the recognition processing of 60 medical images 40 constituting the medical image group 41 is performed. , the same recognition result is obtained 30 times or more, or the same recognition result is obtained 20 times or more in succession.
  • the reliability update unit 32b can update the reliability of each of the time-series medical images 40 from the recognition results of the medical image group 41.
  • the reliability of the recognition result of the medical image 40 that is the same as the recognition result of the medical image group 41 is updated to a higher value. For example, when recognition processing of a treatment tool is performed and "forceps" is acquired as a recognition result of the medical image group 41, the reliability of "forceps" in each medical image 40 constituting the medical image group 41 is updated to a larger value.
  • the value to be updated increases as the number of recognition times per unit time with respect to the number of medical images 40 forming the medical image group 41 or the number of continuous recognition times increases.
  • the reliability of each time-series medical image 40 if the recognition result differs from the recognition result of the medical image group 41, the reliability can be updated to a lower value.
  • the value to be updated becomes smaller as the number of times of recognition per unit time with respect to the number of medical images 40 forming the group of medical images 41 or the number of times of continuous recognition becomes smaller.
  • the reliability update unit 32b By updating the reliability by the reliability update unit 32b, the reliability of the recognition result of each medical image 40 constituting the medical image group 41 becomes higher than before the update when the recognition result of the medical image group 41 is the same. , if different, it will be smaller than before update. As a result, reception mode control can be performed with higher accuracy. In addition, the recognition result may change due to fluctuations in reliability.
  • part information is acquired, and a combination thereof is used to detect both types of lesions. Highly accurate reliability calculation and recognition result acquisition can be performed for a specific subject.
  • the functions of the first processing unit 31a and the second processing unit 31b in the recognition processing unit 31 correspond to the functions of the second processing unit 31b in the reliability determination unit 32.
  • the function of the collation part 32c is implement
  • the reliability of the medical image 40 obtained by calculating the reliability of a plurality of specific subjects is represented for each acquired recognition result.
  • the first processing unit 31a performs first recognition processing for recognizing a specific subject on the medical image 40 in the same manner as the recognition processing unit 31 in the above embodiment, and acquires a first recognition result.
  • the second processing unit 31b executes a second recognition process for recognizing a specific subject on the medical image 40 and obtains a second recognition result. In the second recognition process, a specific subject of a different type from the specific subject recognized in the first recognition process is recognized.
  • the medical image 40 for which the first recognition result and the second recognition result have been acquired is sent to the reliability determination unit 32 .
  • the reliability determination unit 32 determines a first reliability indicating the possibility of misrecognition of the first recognition result of the medical image 40 and a second reliability indicating the possibility of misrecognition of the second recognition result.
  • the correspondence matching unit 32c stores in advance correspondences of different types of specific subjects, for example, combinations of lesion information and site information.
  • the correspondence matching unit 32c checks whether the correspondence between the first recognition result and the second recognition result matches the contents stored in advance.
  • the first reliability and the second reliability are calculated based on the results of matching.
  • the recognition results of the medical image 40 subjected to the two recognition processes are as follows: lesion: "tumor: 65%, bleeding: 20%, gastritis: 15%”; cardia: 15%, gastric corpus: 15%", the medical image 40 is treated as having a “lesion reliability of 65%” and a "partial reliability of 40%".
  • the acceptance mode is switched for each recognition process executed, and the acceptance of corrections for each recognition result is controlled step by step. For example, for the same medical image 40, after acceptance of correction by switching the acceptance mode of the lesion, acceptance of correction is performed by switching the acceptance mode of the part.
  • the order of correction may be determined by reliability or by user's operation.
  • each reliability is equal or higher than when the matching result is not used, and when it is not matching, each reliability is less than the same value as when the matching result is not used. If the correspondence does not match, each reliability is at most less than the first threshold, ie, the value of the first mode or third mode that accepts correction, and correction is performed. Alternatively, if they do not match, at least one of them is erroneous recognition, so each reliability is set to 50%.
  • lesion detection processing for detecting lesions is executed to obtain lesion detection results. to obtain the scene determination result.
  • the correspondence matching unit 32c stores in advance information on the combination of parts that can be detected for a specific lesion type, and the correspondence between the acquired lesion detection result and the scene determination result matches the stored information on the combination. or match.
  • a first reliability level of the lesion detection result and a second reliability level of the scene determination result are each determined using the collation result as to whether or not the correspondence relationship matches the stored information.
  • the reception mode control unit 33 selects the first mode. Alternatively, the reception mode is switched to the third mode.
  • FIG. 16 shows a screen display of the display 14 for correcting the scene determination result when the first reliability of "stomach cancer" is higher than the second reliability of "rectum”.
  • the recognition result display column 50 displayed side by side with the medical image 40, in addition to the optional input reception column 51 and the correction candidate display column 52, the other recognition result is displayed.
  • none of "ileocecal region”, “sigmoid colon”, and “descending colon” detected by the scene determination process match the correspondence relationship with "stomach cancer”. is entered in the arbitrary input acceptance column 51 to correct the scene determination result.
  • the correction candidate display field 52 each part of the stomach stored as a combination of "stomach cancer” is displayed as a correction candidate instead of the scene determination result.
  • the input arbitrary wording can be confirmed by selecting the confirmation button 53 . If the first reliability level of "stomach cancer" is lower than the second reliability level of "rectum”, it is preferable to switch to the reception mode in which the lesion detection result is corrected as the scene determination result is "rectum". In the correction of the correspondence relationship, whether to correct the lesion detection result, the scene determination result, or both may be determined by user operation such as selection of the correction target switching button 54 .
  • Combinations of recognition results and second recognition results include combinations of lesions and parts/organs, as well as combinations of lesions and treatment tools.
  • the first processing unit 31a is set to detect lesions
  • the second processing unit 31b is set to perform treatment tool detection processing
  • the correspondence matching unit 32c determines the type of treatment tool that can be used for a specific lesion. memorize. Note that the settings of the first processing unit 31a and the second processing unit 31b may be reversed.
  • Reliability may be calculated individually to control switching of reception modes.
  • the first processing unit 31a performs lesion detection processing as the first recognition processing
  • the second processing unit 31b performs scene determination processing as the second recognition processing. Control acceptance of corrections. That is, for one medical image 40, the reception mode is controlled for each recognition result that is not linked, and correction is received.
  • the function of a third processing unit (not shown) is realized, and recognition processing is performed on three types of specific subjects in one medical image 40. , the reliability of each may be calculated separately.
  • the database 12 connected to the medical image processing apparatus 11 is connected to the endoscope system 13, and an example in which the endoscope inspection image acquired by the endoscope 13a is processed has been described.
  • the invention is not limited to this, and the medical image 40 acquired by other medical inspection devices such as an ultrasound imaging device and a radiography device may be subjected to processing for recognizing the presence or absence of a specific subject and the name of the subject. good.
  • the central control unit 20 the image acquisition unit 21, the output control unit 24, the input reception unit 22, and the recognition processing unit 31 included in the recognition unit 30, the reliability determination unit 32, the acceptance mode control unit 33, the recognition
  • the hardware structure of a processing unit that executes various processes such as the result correction unit 34 and the recognition result determination unit 35 is various processors as shown below.
  • Various processors include CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), etc., which are general-purpose processors that run software (programs) and function as various processing units.
  • Programmable Logic Devices which are processors, and dedicated electric circuits, which are processors with circuit configurations specifically designed to perform various types of processing.
  • One processing unit may be composed of one of these various processors, or composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA).
  • a plurality of processing units may be configured by one processor.
  • a plurality of processing units may be configured by one processor.
  • this processor functions as a plurality of processing units.
  • SoC System On Chip
  • SoC System On Chip
  • the various processing units are configured using one or more of the above various processors as a hardware structure.
  • the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements.
  • the hardware structure of the storage unit is a storage device such as an HDD (hard disc drive) or SSD (solid state drive).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Optics & Photonics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
PCT/JP2022/037653 2021-11-30 2022-10-07 医療画像処理装置及びその作動方法 Ceased WO2023100475A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023564765A JPWO2023100475A1 (https=) 2021-11-30 2022-10-07
US18/675,095 US20240312200A1 (en) 2021-11-30 2024-05-27 Medical image processing apparatus and method for operating the same

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-194738 2021-11-30
JP2021194738 2021-11-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/675,095 Continuation US20240312200A1 (en) 2021-11-30 2024-05-27 Medical image processing apparatus and method for operating the same

Publications (1)

Publication Number Publication Date
WO2023100475A1 true WO2023100475A1 (ja) 2023-06-08

Family

ID=86611991

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/037653 Ceased WO2023100475A1 (ja) 2021-11-30 2022-10-07 医療画像処理装置及びその作動方法

Country Status (3)

Country Link
US (1) US20240312200A1 (https=)
JP (1) JPWO2023100475A1 (https=)
WO (1) WO2023100475A1 (https=)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011142974A (ja) * 2010-01-13 2011-07-28 Fujifilm Corp 医用画像表示装置および方法、並びにプログラム
WO2016088187A1 (ja) * 2014-12-02 2016-06-09 オリンパス株式会社 フォーカス制御装置、内視鏡装置及びフォーカス制御装置の制御方法
WO2019054045A1 (ja) * 2017-09-15 2019-03-21 富士フイルム株式会社 医療画像処理装置、医療画像処理方法及び医療画像処理プログラム
JP2021100555A (ja) * 2019-12-24 2021-07-08 富士フイルム株式会社 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011142974A (ja) * 2010-01-13 2011-07-28 Fujifilm Corp 医用画像表示装置および方法、並びにプログラム
WO2016088187A1 (ja) * 2014-12-02 2016-06-09 オリンパス株式会社 フォーカス制御装置、内視鏡装置及びフォーカス制御装置の制御方法
WO2019054045A1 (ja) * 2017-09-15 2019-03-21 富士フイルム株式会社 医療画像処理装置、医療画像処理方法及び医療画像処理プログラム
JP2021100555A (ja) * 2019-12-24 2021-07-08 富士フイルム株式会社 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム

Also Published As

Publication number Publication date
JPWO2023100475A1 (https=) 2023-06-08
US20240312200A1 (en) 2024-09-19

Similar Documents

Publication Publication Date Title
JP6890184B2 (ja) 医療画像処理装置及び医療画像処理プログラム
US11914918B2 (en) Medical information processing apparatus, medical information processing method, and non-transitory computer-readable storage medium
CN112435740B (zh) 信息处理设备、检查系统、信息处理方法和存储介质
WO2006123455A1 (ja) 画像表示装置
US20240304311A1 (en) Medical image processing apparatus, medical image proces sing method, program, and diagnosis support apparatus
CN113543694A (zh) 医用图像处理装置、处理器装置、内窥镜系统、医用图像处理方法、及程序
WO2007119297A1 (ja) 医療用画像処理装置及び医療用画像処理方法
WO2020054543A1 (ja) 医療画像処理装置及び方法、内視鏡システム、プロセッサ装置、診断支援装置並びにプログラム
US20200090548A1 (en) Image processing apparatus, image processing method, and computer-readable recording medium
CN112969402A (zh) 内窥镜系统以及用于内窥镜系统的图像处理装置和图像处理方法
US20240312019A1 (en) Medical image processing apparatus, method for operating medical image processing apparatus, and non-transitory computer readable medium
JPWO2019087969A1 (ja) 内視鏡システム、報知方法、及びプログラム
JP7260060B2 (ja) 情報処理装置、表示方法、及びプログラム
JP2019111040A (ja) 画像解析装置及び画像解析方法
WO2023100475A1 (ja) 医療画像処理装置及びその作動方法
US20240386701A1 (en) Medical image processing apparatus and endoscope system
US20240148235A1 (en) Information processing apparatus, information processing method, endoscope system, and report creation support device
JP7289241B2 (ja) ファイリング装置、ファイリング方法及びプログラム
US20230410304A1 (en) Medical image processing apparatus, medical image processing method, and program
CN115038374A (zh) 图像处理装置、图像处理方法以及程序
EP4213093B1 (en) Medical image processing device, ultrasonic diagnostic apparatus, and program
CN120152648A (zh) 医疗支援装置、内窥镜及医疗支援方法
CN113950278B (zh) 图像分析装置及图像分析方法
WO2021241735A1 (ja) 内視鏡プロセッサ装置
CN119183359A (zh) 第二内窥镜系统、第一内窥镜系统及内窥镜检查方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22900909

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023564765

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22900909

Country of ref document: EP

Kind code of ref document: A1