WO2023100446A1 - 医療画像処理装置、医療画像処理装置の作動方法、及び医療画像処理装置用プログラム - Google Patents

医療画像処理装置、医療画像処理装置の作動方法、及び医療画像処理装置用プログラム Download PDF

Info

Publication number
WO2023100446A1
WO2023100446A1 PCT/JP2022/034598 JP2022034598W WO2023100446A1 WO 2023100446 A1 WO2023100446 A1 WO 2023100446A1 JP 2022034598 W JP2022034598 W JP 2022034598W WO 2023100446 A1 WO2023100446 A1 WO 2023100446A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection
medical image
post
image processing
subject
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/034598
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
勝之 比嘉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2023564748A priority Critical patent/JPWO2023100446A1/ja
Priority to CN202280079439.0A priority patent/CN118338833A/zh
Publication of WO2023100446A1 publication Critical patent/WO2023100446A1/ja
Priority to US18/676,391 priority patent/US20240312019A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/12—Arrangements for detecting or locating foreign bodies
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G06T7/0014—Biomedical image inspection using an image reference approach
    • G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present invention relates to a medical image processing apparatus, a method of operating the medical image processing apparatus, and a program for the medical image processing apparatus.
  • Patent Document 1 a notification unit that determines an operation on a subject in a medical image, sets the recognition unit to valid or invalid using the determination result, and notifies the valid or invalid state of the recognition unit.
  • a cancellation mechanism that invalidates the image detection process may be used when appropriate information cannot be obtained in the image detection process.
  • the cancellation mechanism is enabled and the image detection processing is disabled, if the image detection processing is stopped continuously for a long period of time after that, diagnostic support information cannot be obtained during the continuation period.
  • the cancellation mechanism may have to be activated many times after the duration ends, which is troublesome.
  • the medical image processing apparatus of the present invention includes a processor.
  • the processor obtains a plurality of time-series medical images, performs detection processing for detecting a subject from each of the plurality of medical images, and when the subject is detected, the content of the processing corresponding to the detected subject.
  • Predetermined post-detection processing is performed, and the post-detection processing includes at least processing for notifying the user of the detected subject. is set to an invalid mode in which post-detection processing for at least the detected subject is invalid during a preset invalid period.
  • the detection process is preferably at least one of a lesion detection process for detecting a lesion, a treatment instrument detection process for detecting a treatment instrument, and a part detection process for detecting a part.
  • the lesion detection processing is processing for detecting lesions by type
  • the treatment instrument detection processing is processing for detecting treatment instruments by type.
  • Post-detection processing preferably includes processing for recording the results of detection processing in the recording unit.
  • the post-detection processing preferably includes processing for displaying predetermined options on the display corresponding to the detected subject.
  • the processor preferably accepts input from the user and determines whether or not there is a specific instruction based on the input.
  • the processor determines that a specific instruction has been given when the input continues for a continuous input determination period set in advance during the input acceptance period or longer.
  • the processor determines that a specific instruction has been given when the input continues for a preset continuous input determination period or longer during the period in which the post-detection processing for the subject is disabled.
  • the processor preferably determines that a specific instruction has been given when an input is made at least once during an input acceptance period.
  • the input acceptance period is preferably the period during which post-detection processing is performed.
  • the processor preferably determines that a specific instruction has been given when a preset number of inputs or more are made during a preset multiple-input determination period.
  • the processor preferably disables post-detection processing for lesions and/or multiple types of treatment instruments in the disabled mode.
  • the processor preferably disables at least one of the processes included in the post-detection processing in the disable mode.
  • the processor preferably disables post-detection processing related to lesion detection processing in the disabled mode when a lesion is detected.
  • the processor disables post-detection processing related to the treatment tool detection process in the invalid mode when the treatment tool is detected.
  • the processor disables post-detection processing related to part detection processing in the invalid mode when a part is detected.
  • the processor preferably disables detection processing for the detected subject in the disabled mode.
  • the processor preferably disables detection processing for lesions and/or multiple types of treatment tools in the disabled mode.
  • the processor preferably disables at least one of the processes included in the detection process in the disable mode.
  • the processor preferably sets the invalid period to be the period from when a specific instruction is given to when a specific cancellation condition is met.
  • the release condition is preferably that a subject different from the detected subject has been detected in the medical image.
  • the processor calculates a degree of similarity that indicates the degree to which the plurality of medical images are similar to each other, and the cancellation condition is that the degree of similarity between the plurality of recently acquired medical images is equal to or less than a preset value. It is preferable that
  • the processor preferably notifies the user when receiving a specific instruction from the user, when setting the disabled mode, or when canceling the setting of the disabled mode.
  • a method of operating a medical image processing apparatus of the present invention includes steps of acquiring a plurality of time-series medical images, performing detection processing for detecting a subject from each of the plurality of medical images, and detecting a subject when the subject is detected. performing a post-detection process including at least a process of notifying a user of the detected subject, the content of the process being determined in advance corresponding to the detected subject; and setting the operation mode to an invalid mode in which post-detection processing for at least the detected subject is invalid during a preset invalid period when instructed.
  • a program for a medical image processing apparatus of the present invention has a function of acquiring a plurality of time-series medical images, a function of performing detection processing for detecting a subject from each of the plurality of medical images, and when a subject is detected, A function of performing post-detection processing including at least a process of notifying the user of the detected subject, in which the content of the processing is determined in advance corresponding to the detected subject;
  • the computer is caused to execute a function of setting the operation mode to at least a detection process for a detected subject or an invalid mode in which the operation mode is invalid during a preset invalid period.
  • FIG. 1 is a block diagram showing functions of a medical image processing apparatus;
  • FIG. It is a block diagram explaining a structure of a medical image processing apparatus.
  • 4 is a block diagram showing functions of a detection processing unit;
  • FIG. 3 is a block diagram showing respective functions of a lesion detection section, a treatment instrument detection section, and a site detection section;
  • FIG. 10 is an explanatory diagram illustrating a detection process in which a treatment instrument detector detects a treatment instrument and a post-detection process in which a detection result is notified;
  • FIG. 4 is an explanatory diagram illustrating detection processing in which a treatment instrument detector detects a treatment instrument, and post-detection processing in which a detection result is recorded together with an endoscopic image; It is an explanatory view explaining post-detection processing which displays an option.
  • FIG. 9 is an explanatory diagram for explaining a case where the detection process is erroneous in the detection process in which the treatment instrument detector detects the treatment instrument and in the post-detection process in which the treatment instrument detector notifies the detection result;
  • FIG. 11 is an explanatory diagram of a case where a determination method of a specific instruction is a method using a continuous input determination period;
  • FIG. 10 is an explanatory diagram of a case where enabling of a canceling mechanism and transition to an invalid mode are successively performed;
  • FIG. 10 is an explanatory diagram of a case where a method of determining a specific instruction is a method in which input is performed at least once during an input acceptance period;
  • FIG. 11 is an explanatory diagram of a case where a determination method of a specific instruction is a method using a multiple input determination period;
  • FIG. 13(a) is an explanatory diagram illustrating a case where the display of the detection result of false detection is not displayed in the invalid mode and the indicator is displayed only for a predetermined period, and
  • FIG. 10 is an explanatory diagram illustrating a case where display of a detection result of erroneous detection is continuously displayed and an indicator is displayed only for a predetermined period;
  • FIG. 14(a) is an explanatory diagram for explaining a case in which detection processing for forceps, which is a detected subject, is not performed in the invalid mode, and
  • FIG. 14(b) shows detection processing, etc. for forceps, which is a detected subject.
  • FIG. 10 is an explanatory diagram for explaining a case where detection processing of a snare other than the detected subject is performed, although detection processing is not performed;
  • FIG. 15(a) is an explanatory diagram for explaining a case in which the detection processing of the treatment tool as the detected subject is not performed in the invalid mode, and FIG.
  • FIG. 15(b) shows the detection of the treatment tool as the detected subject.
  • FIG. 10 is an explanatory diagram for explaining a case in which a lesion other than the detected object is detected, although no processing is performed;
  • FIG. 12 is an explanatory diagram illustrating that the invalid mode is released when the invalid mode release condition is satisfied;
  • FIG. 11 is an explanatory diagram illustrating a case where a condition for canceling an invalid mode is satisfied when a subject different from the detected subject is detected;
  • 2 is a block diagram showing functions of a medical image processing apparatus including a similarity calculator;
  • FIG. FIG. 11 is an explanatory diagram illustrating a case where a cancellation condition for an invalid mode is satisfied when the similarity between endoscopic images is low;
  • 4 is a flow chart showing an example of the flow of processing of the medical image processing apparatus;
  • the medical image processing apparatus 10 includes a medical image acquisition unit 11 , a detection processing unit 12 , a post-detection processing unit 13 , an input processing unit 14 and a mode setting unit 15 .
  • the medical image processing apparatus 10 includes an endoscope apparatus 16, various modalities such as X-ray examination (not shown), RIS (Radiology Information Systems) or an examination information system such as an endoscope information system (Fig. not shown), a device capable of outputting medical image data such as PACS (Picture Archiving and Communication System) 17, a display device such as display 18, and/or a keyboard (not shown), display 18, or an input device 19 such as a foot switch.
  • PACS Picture Archiving and Communication System
  • the medical image processing apparatus 10 performs detection processing for detecting subjects from each of a plurality of time-series medical images acquired from the endoscope apparatus 16 or the like.
  • a medical image is, for example, a medical image handled by the PACS 17, and is mainly an image obtained by examination. Specifically, an X-ray image obtained by X-ray examination, an MRI obtained by MR examination, a CT image obtained by CT examination, an endoscopic image obtained by endoscopy, or an ultrasonic image obtained by ultrasonic examination can be used.
  • a subject to be detected is a subject detected by each detection process for detecting a specific subject among the subjects in the medical image, such as various lesions, various treatment tools, or each part. Therefore, the detection process includes a lesion detection process for detecting a lesion, a treatment tool detection process for detecting a treatment tool, a part detection process for detecting each part, etc., and performing at least one of these detection processes, Two or more detection processes may be performed.
  • the detection process is performed by a diagnostic support learning model built by performing learning on a machine learning algorithm.
  • this diagnostic support learning model is trained and adjusted so as to output information about the detected subject, which is the result of the target detection process (hereinafter referred to as the detection result).
  • Learning model is the result of the target detection process.
  • Post-detection processing is performed on the detection results obtained by the detection processing.
  • the content of processing is determined in advance according to the detected subject. Then, post-detection processing based on predetermined contents is performed on the detected subject.
  • the post-detection processing includes notification processing for notifying the user according to the detected subject.
  • Post-detection processing includes, in addition to notification processing, recording processing for recording detection results in the data recording unit 37 (see FIG. 2), option display processing for displaying options corresponding to the detected subject on the display 18, and the like. including.
  • option display processing the user selects the detected subject from the displayed options.
  • the recorded detection results or the selected contents of the detected subject are used for report creation and the like.
  • Notification is by display on the display 18, for example. Therefore, when the subject is detected, the user is notified by displaying on the display 18 the information about the detected subject, which is the detection result, such as the name of the subject. Since the acquired medical image is displayed on the display 18 during the examination, the medical image and the detection result are displayed on the display 18 . Therefore, the user can know the detection result by looking at the display 18 during observation during the examination. The user uses the detection result displayed on the display 18 as diagnostic support information for examination, diagnosis, or the like.
  • the user may look at the medical image and the detection result and determine that the detection result is incorrect.
  • the user can give a specific instruction to set the operation mode of the medical image processing apparatus 10 to the disabled mode.
  • the contents of the invalid mode can be set in advance.
  • the medical image processing apparatus 10 sets an invalid mode to invalidate post-detection processing when there is a specific instruction from the user.
  • the disable mode is an operation mode that disables post-detection processing for at least the detected subject.
  • post-detection processing is disabled in the disabled mode, predetermined post-detection processing corresponding to the detected subject, including at least notification processing, is not performed.
  • the disabled mode is applied for a preset period of time.
  • the medical image processing apparatus 10 releases the invalid mode when a release condition for releasing the invalid mode is satisfied.
  • the user can flexibly apply the invalid mode for invalidating the post-detection processing. Also, the invalid mode can be canceled until, for example, an invalid mode cancellation condition is satisfied, and the user can set the cancellation condition.
  • a report is created after the examination or treatment.
  • AI Artificial Intelligence
  • a diagnosis support learning model based on machine learning
  • a diagnostic support function that detects, records the detection results, and creates a report using the recorded detection results.
  • the user performs a canceling operation of activating the cancellation mechanism of the detection process or the like using the input device 19 such as a foot switch, and disables the detection process or the like for a predetermined period.
  • a mode to do is conceivable.
  • the erroneous detection by AI continues, it may become troublesome for the user to perform this canceling operation many times.
  • the detection process or the like is not performed during that time, and the diagnosis support function cannot be effectively utilized.
  • the medical image processing apparatus 10 can preset the contents of the post-detection processing of how to use the detection result by AI, and in the case of erroneous detection, etc., the operation can be performed by the user giving a specific instruction.
  • the mode is shifted to a disabled mode in which post-detection processing is disabled.
  • post-detection processing and the like are invalidated.
  • the duration of the invalid mode until the invalid mode is canceled can be set in advance according to various cancellation conditions.
  • cancellation of the invalid mode can be set according to preset cancellation conditions.
  • detection processing by AI can be resumed automatically when desired, so the diagnosis support function can be used effectively while avoiding the trouble associated with cancellation work. can also be achieved.
  • cancel and invalidate have the same meaning.
  • invalidity of each process means that the intended effect of each process does not appear, and does not mean that no process is performed.
  • the medical image processing apparatus 10 it is possible to flexibly control the processing related to the detection of AI or the like in accordance with the user's wishes, and to create a report or detect an examination performed using medical images.
  • Various processing such as utilization of results can be performed more accurately and efficiently.
  • the medical image processing apparatus 10 of the present invention includes an input device 19 that is an input device, a display 18 that is mainly an output device, a control unit 21, a communication unit 22, and A storage unit 23 is a computer electrically interconnected via a data bus 24 .
  • the input device 19 is an input device such as a keyboard, a mouse, a touch panel of the display 18, a foot switch, or a scope button of the endoscope device 16.
  • a computer that configures the medical image processing apparatus 10 receives input of various instructions from the input device 19 .
  • the display 18 is a kind of output device.
  • the display 18 displays various operation screens according to the operation of the input device 19 .
  • the operation screen has an operation function by GUI (Graphical User Interface).
  • a computer that configures the medical image processing apparatus 10 can receive input from a user through an operation screen.
  • the control unit 21 includes a CPU (Central Processing Unit) 31 which is a processor, a RAM (Random Access Memory) 32, a ROM (Read Only Memory) 33, and the like.
  • the CPU 31 loads a program stored in the storage unit 23 or the like into the RAM 32 or ROM 33 and executes processing according to the program, thereby controlling each unit of the computer in an integrated manner.
  • the communication unit 22 is a network interface that controls transmission of various information via the network 25 . Note that the RAM 32 or the ROM 33 may have the function of the storage section 23 .
  • the storage unit 23 is an example of a memory, and for example, a hard disk drive, a solid state drive, or a disk in which a plurality of hard disk drives, etc., are built in the computer constituting the medical image processing apparatus 10 or connected via a cable or network. array.
  • the storage unit 23 stores control programs, various application programs, various data for use in these programs, display data for various operation screens associated with these programs, and the like.
  • the storage unit 23 of the present embodiment stores various data such as the program 34 for the medical image processing apparatus and the data 35 for the medical image processing apparatus.
  • the medical image processing apparatus program 34 or the medical image processing apparatus data 35 are programs or data for implementing various functions of the medical image processing apparatus 10, respectively.
  • the functions of the medical image processing apparatus 10 are realized by the medical image processing apparatus program 34 and the medical image processing apparatus data 35 .
  • the medical image processing apparatus data 35 also includes a temporary storage unit 36 and a data recording unit 37, and stores data temporarily stored by the medical image processing apparatus program 34, and the like.
  • the computer that constitutes the medical image processing apparatus 10 can be a specially designed device, a general-purpose server device, a PC (Personal Computer), or the like. In addition, it is sufficient that the functions of the medical image processing apparatus 10 can be exhibited, and the computer or a part of the computer may be shared with an apparatus that performs other functions. function can also be incorporated.
  • the medical image processing apparatus 10 of the present embodiment is a processor apparatus, and the medical image processing apparatus 10 stores programs related to medical image processing in the storage unit 23, which is a program memory.
  • a program in a program memory is operated by a control unit 21 configured by a processor or the like, thereby operating a medical image acquisition unit 11, a detection processing unit 12, and a post-detection processing unit 13. , an input processing unit 14 and a mode setting unit 15 (see FIG. 1).
  • the medical image acquisition unit 11 acquires a plurality of time-series medical images from a device capable of outputting medical images.
  • a medical image is an image obtained mainly through an examination, and may be obtained in real time during an examination or from a saved medical image taken during an examination.
  • a plurality of time-series medical images can be a plurality of medical images obtained from a single examination.
  • endoscopic images are acquired in real time during endoscopic examination using the endoscope device 16 . Therefore, the medical image acquisition unit 11 acquires a plurality of time-series endoscopic images.
  • An endoscopic image is a type of medical image, and is an image obtained by photographing a subject with an endoscope included in the endoscope device 16 . A case in which an endoscopic image is used as a medical image will be described below.
  • An endoscopic image is an image based on individual frame images captured by the endoscope device 16 with a preset number of frames, and mainly means a still image.
  • the display 18 displays a moving image in which a plurality of time-series frame images are sequentially displayed for endoscopy.
  • the detection processing unit 12 performs detection processing for detecting a specific subject from each of the multiple endoscopic images acquired by the medical image acquisition unit 11 .
  • detection processing is normally performed on each of a plurality of time-series endoscopic images sequentially in the order in which the endoscopic images are acquired.
  • the detection processing unit 12 includes a lesion detection unit 41, a treatment instrument detection unit 42, and a site detection unit 43.
  • the lesion detection unit 41 Based on the acquired endoscopic image, the lesion detection unit 41 performs detection processing for detecting a region of interest such as a lesion, the treatment tool detection unit 42 performs detection processing for detecting various treatment tools, and the site detection unit 43 performs a detection process for detecting a part.
  • a subject detected by the lesion detection unit 41 is a lesion
  • a subject detected by the treatment instrument detection unit 42 is a treatment instrument
  • a subject detected by the part detection part 43 is a part. Subjects of different categories such as lesions, treatment tools, and parts are detected by these detection processes.
  • the lesion detection unit 41 may perform processing for detecting lesions for each type as the lesion detection processing.
  • Objects to be detected when lesions are detected for each type may be types related to lesions, such as tumor or non-tumor, various degrees of progression, scores of various diagnostic criteria, and the like.
  • the treatment instrument detection unit 42 may perform a process of detecting treatment instruments by type as the treatment instrument detection process.
  • a subject to be detected when detecting treatment instruments by type may be any type related to the treatment instrument, for example, by name of treatment instrument, by purpose of treatment instrument, or by procedure. Snares, forceps, knives, and the like are examples of types by treatment instrument name or types by purpose.
  • Treatment instruments for each procedure can be of the type used for polypectomy, cold polypectomy, EMR (Endoscopic mucosal resection), or the like.
  • the detection processing unit 12 may perform measurement processing, classification processing, or the like as the detection processing. For example, measurement processing or classification processing for detecting the value of biological information such as oxygen saturation can be performed.
  • the detection processing unit 12 includes a measurement unit (not shown) or a classification unit (not shown).
  • a measurement unit not shown
  • a classification unit not shown
  • the measurement process a region of the living body having a specific range of oxygen saturation values is detected, and as the classification process, the area of the living body is detected for each oxygen saturation value range.
  • a detection process for classifying areas or the like may be performed.
  • the position, size, distance from the endoscope distal end, or discrimination related to these may be performed for lesions, treatment instruments, parts, or the like.
  • discrimination or the like is performed in the detection process, each subject having the discriminated content is detected.
  • one detection process may be performed, or a plurality of detection processes may be combined.
  • each detection unit included in the detection processing unit 12 can include one or more detectors corresponding to specific subjects to be detected.
  • the detector may be provided outside the medical image processing apparatus 10 .
  • Lesion detector 41 includes lesion detector 44 .
  • the lesion detector 44 is a diagnostic support learning model that detects lesions of a subject based on acquired endoscopic images. When an endoscopic image is input, the lesion detector 44 generates a detection result including at least the lesion area in the subject when the subject captured in the endoscopic image includes a lesion.
  • the detection results include, in addition to the area of the lesion, the type or classification of the lesion such as redness, tumor or non-tumor, or the stage or progression of a specific lesion. May contain information. Also, the detection result may include saved information that can be acquired from another device. Note that if the subject in the endoscopic image does not include a lesion, a detection result with the content "no lesion" may be generated.
  • the treatment instrument detector 42 includes a treatment instrument detector 45 .
  • the treatment instrument detection unit 42 is a diagnosis support learning model that detects various treatment instruments appearing in the endoscopic image based on the acquired endoscopic image.
  • the treatment instrument detector 45 When an endoscopic image is input, the treatment instrument detector 45 generates a detection result including the name of the treatment instrument when the subject in the endoscopic image includes various treatment instruments.
  • the detection result includes various information obtained by learning the treatment instrument detector 45, such as the name of the treatment instrument, the name of the manufacturer, the model number, the date of manufacture, and the date of use, in addition to the name of the instrument.
  • the detection result may include saved information or the like that can be obtained from another device. Further, when the subject captured in the endoscopic image does not include various treatment tools, a detection result with the content "no treatment tools" may be generated.
  • the part detection unit 43 includes a part detector 46 .
  • the part detection unit 43 is a diagnostic support learning model that detects the part of the subject in the endoscopic image based on the acquired endoscopic image.
  • the part detector 46 When an endoscopic image is input, the part detector 46 generates a detection result including the name of the part of the subject appearing in the endoscopic image.
  • the detection result includes various information obtained by learning the part detector 46, such as the position or orientation of the endoscope when the endoscopic image was taken, in addition to the name of the part. Also, the detection result may include saved information that can be acquired from another device. Further, when the subject captured in the endoscopic image does not include a specific part, the detection result may be generated with the content "no part detected".
  • Various detectors including the lesion detector 44, the treatment instrument detector 45, and the part detector 46 are specifically diagnostic support learning models built using machine learning algorithms, and can detect the endoscopic image 51 in various ways. It is a learning model that can detect a specific subject, which is a detection target included in the endoscope image 51, and output information about the detection result as an objective variable when input to the detector. Various detectors use a machine learning algorithm in advance so as to output information about the detection result of detecting a specific subject included in the endoscopic image 51 as an objective variable. Learning is performed using initial image data sets for various detectors, which consist of the scope image 51 and correct data such as the name of the specific subject or the position of the subject, and parameters and the like are adjusted.
  • the machine learning algorithm used for various detectors various algorithms can be used as long as they are mainly used for supervised learning, but good inference results in detection processing using images are output as objective variables It is preferable to use an algorithm that For example, it is preferable to use a multilayer neural network or a convolutional neural network, and it is preferable to use a technique called deep learning.
  • the diagnostic support learning model includes an endoscopic image 51, which is an input image, which is generally performed to improve the performance of the learning model, such as improving the accuracy of detecting a specific subject or improving the detection speed. Techniques such as processing for, using multiple learning models, and combining with unsupervised learning may also be used.
  • the detection processing unit 12 may be configured to calculate a feature amount from an endoscopic image and perform detection or the like using the calculated feature amount, instead of being configured with a detector based on machine learning.
  • an endoscopic image 51 is displayed in the form of a moving image on the display 18 during the examination.
  • the treatment instrument detector 45 performs detection processing for detecting various treatment instruments as subjects to be detected from the endoscope image 51 .
  • the detected subjects are various treatment tools.
  • the detection processing unit 12 detects the forceps 52 as one type of treatment tool, the detection result 54 is generated as the name of the treatment tool, “forceps”.
  • the post-detection processing unit 13 performs post-detection processing.
  • the post-detection processing includes various processing using the detection result 54 .
  • the content of the process is determined in advance according to the detected subject, and includes at least a notification process of notifying the user according to the detected subject.
  • the notification process may be notification related to the detected subject, and in addition to notifying the user of the name of the treatment tool that has been detected, it is also possible to notify the name of the treatment being performed at the time of detection according to the detected subject. included.
  • Post-detection processing is performed when a subject is detected.
  • the detected subjects are various types of treatment tools, so post-detection processing is performed when any treatment tool is detected.
  • the post-detection processing includes notifying processing of notifying the detection result 54 and the like, recording processing of recording the detection result 54 in the recording unit, and selection of predetermined options corresponding to the detected subject. is displayed on the display 18, and the like.
  • the option display process displays options for selecting, for example, the content to be described in the report from among various types of information related to the detected subject.
  • options including procedures using the detected treatment tool are displayed.
  • options including site names related to the detected site are displayed.
  • the options may be displayed as text or as a schema or the like. The user can automatically enter in the report the procedure using the detected treatment tool or the name of the part desired to be described in the report by selecting the name of the procedure or the part from the options.
  • the means of processing are set in advance.
  • the notification means is displayed on the display 18 .
  • the recording means records to the data recording unit 37 .
  • the option display means is to display on the display 18 .
  • post-detection processing may be performed in combination with manual processing.
  • the part name is manually input by the user selecting the part from the schematic diagram of the inspection object, and then the treatment tool is moved by the detection process. It is also possible to detect and display options related to treatment tools by post-detection processing. By appropriately combining manual processing and post-detection processing, operations suitable for various situations can be performed.
  • the detection result 54 indicating that the forceps 52 has been detected is displayed on the display 18 as the detection result display 55 by the notification process.
  • the detection result display 55 notifies the user that the medical image processing apparatus 10 has detected the forceps 52 through the detection process.
  • the recording process records the name of the treatment tool "forceps” as the detection result 54 in the data recording unit 37 in association with the detected endoscopic image 51 .
  • the options for the procedure associated with the treatment instrument name "forceps” were “Polypectomy”, “Cold Polypectomy”, and "EMR”
  • the option display processing resulted in the detection result 54 are displayed on the display 18 as the option display 56 .
  • the procedure name is confirmed and recorded.
  • “Polypectomy” is selected and confirmed. After the notification or display of options, etc., if the treatment instrument is correctly detected by the detection process, the user continues the endoscopy as before without giving a specific instruction.
  • the detection process may generate erroneous detection results.
  • “snare” is generated as the detection result 54, for example.
  • the notification process causes the detection result 54 “snare” to be displayed as the detection result display 55 on the display 18 .
  • the detection result display 55 notifies the user that the detection result 54 of the detection processing performed by the medical image processing apparatus 10 based on the endoscopic image 51 is "snare".
  • the treatment instrument name “snare”, which is the detection result 54 is recorded in the data recording unit 37 in association with the detected endoscopic image 51 .
  • the user can judge whether the detection process correctly detected or erroneously detected by looking at the endoscopic image 51 displayed on the display 18 and the detection result display 55 .
  • the user determines that the detection processing has made an erroneous detection
  • the user sets the operation mode of the medical image processing apparatus 10 to the invalid mode so that erroneous display or recording is not performed any more.
  • post-detection processing and the like can be disabled.
  • By invalidating post-detection processing, etc. it is possible to prevent unnecessary trouble such as correcting wrong records later.
  • a specific instruction is given to the medical image processing apparatus 10 .
  • a specific instruction is determined to be a specific instruction by presetting an instruction method or the like.
  • Specific instructions can be based on user input.
  • the input processing unit 14 receives an input from the user and determines whether or not a specific instruction has been given.
  • the medical image processing apparatus 10 includes an input processing unit 14 that receives user input. The user inputs using the input device 19, and the input processing unit 14 determines whether or not there is a specific instruction based on the input from the user.
  • the input device 19 As an input method using the input device 19, when the input device 19 is, for example, a keyboard, a mouse, a touch panel of the display 18, or a scope switch of the endoscope device 16, a specific button or the like is selected, clicked, or Use touch as input. If the input device 19 is a footswitch, the depression of the footswitch is used as an input. When a plurality of scope switches or foot switches are provided, a specific scope switch or foot switch is set in advance.
  • the scope switch or foot switch is a so-called momentary type switch that is turned on while being pressed or the like.
  • a momentary type switch is in an OFF state while the switch is released after the depression is completed.
  • various switches such as a so-called alternate switch that maintains the ON state may be used.
  • An alternate switch switches between an ON state and an OFF state when pressed.
  • the input processing unit 14 When determining an input from a user, the input processing unit 14 adopts a determination method selected from a plurality of determination methods by the user.
  • One of the plurality of determination methods is a method in which input is continuously performed during the input acceptance period for a preset continuous input determination period or longer.
  • the input acceptance period is a period during which the input processing unit 14 accepts input from the user in order to determine the input from the user, and can be set in advance.
  • the input reception period is a period during which post-detection processing is performed.
  • the post-detection process includes at least the notification process among the notification process, the recording process, and the option display process, as described above. Therefore, the period during which notification processing such as displaying the detection result display 55 is performed is defined as the input acceptance period.
  • the period during which the detection result display 55 is displayed is a period during which the detection result display 55 based on the plurality of detection results 54 sequentially generated from the plurality of time-series endoscopic images 51 is displayed sequentially.
  • the continuous input determination period is a period for determining whether or not the input is a specific instruction. If the input continues beyond this period, it is determined that the specific instruction has been given. Continuous input means that the ON state of the switch for input continues. For example, if the input is pressing the footswitch, the period during which the footswitch is kept pressed is the period during which the input continues.
  • the continuous input determination period is set in advance, for example, 3 seconds.
  • the continuous input determination period starts when the input is performed. Input is performed, the continuous input determination period starts, and if the continuous input determination period is canceled before the preset continuous input determination period elapses, the continuous input determination period also ends. After that, when the input is performed, the continuous input determination period starts again.
  • the user sees the detection result display 55 shown on the display 18, determines that the detection process is performing erroneous detection, and gives a specific instruction to switch to the invalid mode. .
  • time elapses from left to right, and displayed images shown on the display 18 are shown in chronological order on the right side of the description "display”.
  • periods during which each is executed are shown in bar graph form.
  • the input is performed during the period indicated by the input period 63 .
  • the post-detection processing is performed during the period indicated by the post-detection processing period 64 .
  • the post-detection processing period 64 is the same as the input acceptance period.
  • the disabled mode is executed during the period indicated by disabled mode period 65 .
  • the user starts input by pressing the foot switch at time t1 during the input reception period during which the detection result display 55 is displayed.
  • an indicator 61 is displayed on the display 18, and how long is the remaining input period 63, and if the foot switch is kept depressed, it becomes longer than the continuous input judgment period, and a specific instruction is given. indicates whether the For example, the period until the input period 63 becomes equal to or longer than the continuous input determination period is indicated by the indicator 61 with the text "2 seconds until cancellation" and a figure that changes with the passage of time.
  • the input processing unit 14 determines that a specific instruction has been given when the input period 63 by pressing the footswitch, which is an input, continues beyond the preset continuous input determination period.
  • the continuous input determination period is a period from time t1 to time t2.
  • a specific instruction is given.
  • the input processing unit 14 issues an instruction to the mode setting unit 15, and at time t2, the mode setting unit 15 switches the operation mode of the medical image processing apparatus 10 to the invalid mode.
  • post-detection processing for the detected subject is invalid. Therefore, in the present embodiment, the detection process is performed in the invalid mode, but the notification process, the recording process, or the option display process, which are included in the post-detection process, are not performed. In this embodiment, the post-detection processing is disabled in the disabled mode, so the detection result display 55 is not displayed in the disabled mode. As will be described later, it is possible to set so that the detection process is not performed in the disabled mode. Since the post-detection process is a process in which the content of the process is determined according to the detected subject, if the detection process is not performed, there is no detected subject, so the post-detection process is not performed either.
  • the indicator 61 indicates that the ineffective mode has started for a preset period of time such as several seconds, and then the indicator 61 is not displayed. You may do so.
  • the indicator 61 displays the text "invalid mode start". The disabled mode continues for a preset period of time.
  • the invalid mode mark 62 may be turned on so that the user can recognize at a glance that the invalid mode is in effect.
  • the specific instruction By issuing a specific instruction using the continuous input determination period, if the footswitch is pressed by mistake, or if another instruction is issued using the footswitch, the specific instruction may be issued against the user's intention. You can avoid being judged. Also, by looking at the indicator 61, the user can know how much time is required until a specific instruction is executed. By doing so, it is possible to cancel the transition to the invalid mode.
  • the specific instruction may be continuously input for a preset continuous input acceptance period or more while the post-detection processing for the subject is disabled.
  • the treatment instrument detector 45 may erroneously generate a treatment instrument name other than the forceps 52 as the detection result 54 .
  • the medical image processing apparatus 10 disables recording of detection results in a short period of time such as several seconds, like conventional medical image processing apparatuses that do not have a function to switch to an invalid mode. It has a cancellation mechanism that Activation of the canceling mechanism is performed by input from the input unit.
  • the cancellation mechanism is activated by an instruction different from a specific instruction, for example, when an input is performed for a preset number of times or more for the cancellation mechanism during a preset cancellation mechanism determination period.
  • the cancellation period determination period starts from the time of the first input, the cancellation mechanism determination period is 2 seconds, and the number of times of the cancellation mechanism input is 3 or more. Therefore, the cancellation mechanism is activated when the number of times of input for the cancellation mechanism is three or more times within 2 seconds, which is the judgment period for the cancellation mechanism.
  • the detection process or the post-detection process is invalidated 3 seconds after the confirmation, and the invalidation is automatically canceled after 3 seconds.
  • the activation of the canceling mechanism is activated by, for example, three inputs such as depression of the foot switch, and the user is notified that the activation has been confirmed.
  • the notification is made by the indicator 61 on the display 18, the cancellation mechanism activation mark 57, or the like.
  • the input method for activating the canceling mechanism is, in addition to performing an input equal to or more than the number of inputs for the canceling mechanism during the judgment period for the canceling mechanism, continuous input, i.e., long-pressing of the input device 19, etc. input method can be adopted. Also in this case, in order to distinguish from the transition to the invalid mode, it is preferable to enable the canceling mechanism by a determination method different from the continuous input determination period for transition to the invalid mode.
  • the cancellation mechanism is used to switch to invalid mode. For example, if the user wants to continue to disable the post-detection processing after the activation of the canceling mechanism has been confirmed, the user inputs the activation of the canceling mechanism and then inputs a specific instruction.
  • a specific instruction a method using a continuous input determination period in which continuous input (long press) is performed by the input device 19 as described above, or the like is adopted.
  • the method of continuous input is the same as described above, and the indicator 61 indicates a text such as "2 seconds until cancellation", and when shifting to the invalid mode, a message such as "invalid mode start" is displayed.
  • a graphic is displayed showing text and time remaining before transitioning to disable mode.
  • the cancellation mechanism cancels the invalidation after a short specific period, but if you want to invalidate the detection process or post-detection process again, you can save the trouble of inputting to activate the cancellation mechanism again. Therefore, in the medical image processing apparatus 10, switching to the invalid mode can be performed while maintaining the conventional canceling mechanism, so that the user can use the apparatus without any sense of incongruity, and convenience is further increased.
  • one of the multiple determination methods is a method in which input is performed at least once during the input acceptance period.
  • the input acceptance period is the post-detection processing period 64 . Therefore, from the start of the post-detection process, that is, from the start of notification of the detection result 54, if the touch panel is touched or the foot switch is pressed even once, it is determined that a specific instruction has been given. , it is determined that one input has been made when the switch is turned ON during the input acceptance period, regardless of the switch method.
  • FIG. 11 differs from FIG. 9 in the determination method for determining that the input is a specific instruction, and is otherwise the same.
  • the user performs an input by pressing the footswitch once at time t1.
  • the footswitch input 66 is indicated by a star-shaped mark.
  • the input processing unit 14 determines that a specific instruction has been performed.
  • the operation of the medical image processing apparatus 10 according to specific instructions is the same as in the case of FIG. In this way, the operation mode can be quickly switched by giving a specific instruction with one input.
  • one of the multiple determination methods is a method in which inputs are performed a preset number of times or more during a preset multiple input determination period.
  • the multiple input judgment period is a period for judging whether or not the input is a specific instruction, and it is judged that the specific instruction has been made if the input is performed more than a certain number of times within this period. .
  • the fact that one input is performed is the same as in the case of FIG. 11, and means that the ON state of the switch for input is performed once. For example, when the input is pressing of a foot switch, it is assumed that one input is performed by pressing the foot switch once.
  • the foot switch since the input from the foot switch is determined as a momentary switch, for the second input, the foot switch is released, turned off, and then the foot switch is pressed again. to the ON state.
  • the specific number of times is set in advance and is, for example, three times.
  • the multiple input determination period is set in advance, for example, 3 seconds.
  • the start of the multiple input determination period is the time point when the input is performed.
  • a specific instruction is executed when a first input is made, a multiple input determination period starts, and a specific number of inputs are made before a preset multiple input determination period has elapsed.
  • one input is defined as one input when the switch is pressed once from a state in which it has not been pressed. Therefore, when three or more inputs are made within three seconds, which is the multiple-input determination period, this condition is satisfied by performing pressing and releasing of pressing about three times within three seconds.
  • the multiple input determination period starts, and if the specified number of inputs are not made within the preset multiple input determination period, the multiple input determination period also ends. After that, when the input is performed, the multiple input determination period starts again.
  • FIG. 12 is different from FIG. 9 in the method of determining whether the input is a specific instruction, and is otherwise the same.
  • the user looks at the detection result display 55 and performs an input 66 at time t1 by pressing the foot switch once.
  • the indicator 61 is displayed on the display 18 and indicates the remaining number of times until the number of times of pressing of the foot switch becomes equal to or more than the number of times of input. For example, after the input 67, by displaying the text "One more time to cancel" and a figure whose color changes when the input is completed two times after the input is completed, the remaining number of inputs until the number of inputs is exceeded is displayed. show.
  • the multiple input determination period when the multiple input determination period, for example, 3 seconds, has passed since the first input 66, one of the figures whose color was first changed by pressing returns to the original color. As a result, it can be recognized from the indicator 61 that a specific instruction is given by continuously pressing the foot switch during the multiple input determination period.
  • the input processing unit 14 determines that a specific instruction has been given when the number of times the footswitch, which is an input, is pressed exceeds the number of inputs set in advance during the multiple input determination period.
  • the multiple-input determination period is a period from time t1 to time t3.
  • the depression of the foot switch is performed four times, input 66, input 67, input 68, and input 69, from time t1 to time t3.
  • the time t2 when the input 68, which is the third input, is performed is earlier than the time t3, and a specific instruction is performed at the time t2 when the input 68 is performed. Operations according to specific instructions are the same as in the case of FIG.
  • the input processing unit 14 notifies the user of the reception when receiving a specific instruction from the user.
  • the notification that the specific instruction has been received may be shared with the notification that the operation mode is the disabled mode.
  • the determination method for a specific instruction by input can be set by selecting from a plurality of determination methods. These determination methods allow the user to make specific instructions by inputting in a simple manner that does not interfere with the examination. In addition, switching to the invalid mode can be performed after selecting an appropriate determination method according to the application.
  • the mode setting unit 15 sets the operation mode of the medical image processing apparatus 10 to the disabled mode based on instructions from the input processing unit 14.
  • the invalid mode is an operation mode that invalidates at least the detection processing or the post-detection processing for the detected subject for a preset period.
  • a specific instruction from the user is determined by the input processing unit 14 .
  • the input processing unit 14 instructs the mode setting unit 15 to set the operation mode of the medical image processing apparatus 10 to the invalid mode.
  • the mode setting unit 15 can disable the post-detection processing for the detected subject in the disable mode, as described above.
  • the mode setting unit 15 sets a plurality of detection processes according to the detection process.
  • Post-detection processing can be disabled for lesion types and/or treatment device types.
  • the mode setting unit 15 can disable at least one of the processes included in the post-detection process in the disable mode. Also, all processes included in the post-search process may be invalidated. In this embodiment, detection processing is performed, and post-detection processing using the detection result 54 is not performed. In the invalid mode, all of the notification processing, recording processing, and option display processing set in the post-detection processing are not performed, but specific processing not included in the post-detection processing can be performed on the detection result 54. make it possible. In the disabled mode, among the notification processing, recording processing, and option display processing that are set to be included in the post-detection processing, the recording processing and option display processing may be performed without performing only the notification processing.
  • the indicator 61 when the invalid mode is set, the indicator 61 indicates "invalid mode start" for a predetermined period such as the first few seconds after the operation mode is changed from the invalid mode to the invalid mode. , etc., to inform the user that the invalid mode has started. After a predetermined period of time has elapsed, the indicator 61 is hidden so as not to interfere with observation. Also, when in the ineffective mode, it is preferable to continue displaying on the display 18 or the like as notification of the ineffective mode. For example, the invalid mode mark 62 displayed on the display 18 is lit continuously while the invalid mode is set to inform the user that the user is in the invalid mode so that the user can grasp it at a glance. may
  • the post-detection processing is disabled, but the last displayed detection result display 55 may remain displayed.
  • the case shown in FIG. 13(b) is the same as the case of FIG. 13(a) except that the detection result display 55 remains displayed.
  • the mode setting unit 15 can be set to disable the detection process for the detected subject in the disabled mode.
  • the detection process is not performed, no subject is detected, so the post-detection process is not performed either.
  • the details for the case where post-detection processing is not performed are similar to those described above.
  • various processes using the detection result cannot be performed, and the calculation resources used for the detection process are not required, so that the processing speed can be improved.
  • the mode setting unit 15 sets a plurality of detection processes according to the detection process. It is possible to disable the detection process for one type of lesion and/or multiple types of treatment tools. Moreover, when the detection process includes a plurality of processes, the mode setting unit 15 can disable at least one of the processes included in the detection process in the disabled mode. Also, all processes included in the search process may be invalidated. For example, when the detection process is set to detect a plurality of subjects in different categories such as lesions, treatment tools, or parts, the detection process is disabled for at least one of these categories, or All categories of subject detection processing can be disabled.
  • the invalidity of the detection process in the invalid mode and the invalidation of the post-detection process can be selected and set in advance. Some categories of subject detection processing are enabled, some post-detection processing is disabled, and another portion of post-detection processing is enabled. and post-detection processing can be combined.
  • the user may be notified of which is the case, for example, by making the color of the invalid mode mark 62 different depending on whether the detection process is disabled or the post-detection process is disabled. Also, which process is invalid may be displayed by a diagram such as a mark or text to inform the user.
  • the detection process By disabling the detection process or the post-detection process in the disable mode, if the detection process is performing an erroneous detection, the erroneous detection will continue and erroneous display or recording will not be performed any more. can.
  • it is possible to select whether to disable detection processing or disable post-detection processing it is possible to flexibly configure settings so that some detection processing is performed, but some post-detection processing, such as display or recording, is not performed. It is possible to deal with it. In this case, the detection result 54 is not used for post-detection processing, but the detection result 54 can be used for other purposes. Further, when the detection process is disabled, it is possible to save computational resources.
  • the mode setting unit 15 can preset how to treat the subject when the detection process or the post-detection process is disabled in the invalid mode according to the detection process.
  • a range can be set in advance.
  • the detected subject to be invalidated can be the same type of subject as the detected subject.
  • the type of detected subject is "forceps", and subjects other than forceps, such as snares, are not invalidated.
  • the detected object is the forceps 52. Therefore, in the disable mode, detection processing or post-detection processing for the forceps 52 is disabled. do. Therefore, in the disabled mode, when the forceps 52 are included in the endoscopic image 51, detection processing or post-detection processing for the forceps 52 is not performed. In addition, since the invalid mode for invalidating the detection process and the like for the forceps 52 continues, the invalid mode mark 62 is displayed on the display 18 .
  • the snare is displayed on the endoscopic image 51. 71 is included, detection processing or post-detection processing for the snare 71 is performed by the detection processing by the detection processing unit 12 . Then, for example, "snare" is generated as the detection result 54 by the detection processing for the snare 71 and displayed on the display 18 as the detection result display 55. FIG. In addition, since the invalid mode for invalidating the detection process and the like for the forceps 52 continues, the invalid mode mark 62 is displayed on the display 18 .
  • the detected subject to be invalidated is the treatment tool. It can be general equipment.
  • the subject category that includes the forceps is treatment tools. Therefore, the subject to be invalidated is the subject of the treatment instrument, and when the forceps 52 are detected, the detection processing or post-detection processing for the treatment instrument is invalidated.
  • Subjects in the category to which the detected subject belongs include, in addition to treatment tools, lesions when specific lesions such as tumors or non-tumors are detected, and specific parts such as the esophagus or duodenal bulb. and the like.
  • the detected object is the forceps 52, and the object to which the forceps 52 belongs is a treatment tool.
  • the detection processing or post-detection processing for the treatment instrument is invalidated. Therefore, in the invalid mode in this case, when the snare 71 is included in the endoscopic image 51, no detection processing or post-detection processing for the snare 71 is performed.
  • the invalid mode mark 62 is displayed on the display 18 . Since the treatment instrument detection process is performed by the treatment instrument detector 45, the invalid mode in this case may be executed by not operating the treatment instrument detector 45 among various detectors.
  • the endoscopic image 51 shows a lesion.
  • detection processing or post-detection processing for the lesion 72 is performed by the detection processing by the detection processing unit 12 .
  • a “lesion” is generated as the detection result 54 by the detection processing for the lesion 72 and displayed on the display 18 as the detection result display 55 .
  • the invalid mode mark 62 is displayed on the display 18 because the invalid mode for invalidating the detection process and the like for the treatment instrument is being continued.
  • a subject to be invalidated may be a detected individual subject. For example, when the forceps 52 are detected, the forceps 52 themselves are invalidated, and forceps other than the forceps 52 are not invalidated.
  • the subject to be invalidated may be set as the treatment tool category to which the snare belongs.
  • the invalid mode for invalidating the detection process and the like for the treatment tool may be realized by invalidating the various detection processes themselves. Therefore, in the invalid mode, one or more of lesion detection processing or lesion detection processing, treatment instrument detection processing or post-detection processing of treatment instrument detection processing, or site detection processing or post-detection processing of site detection processing is disabled. good too.
  • the detection processing or post-detection processing for the detected subject in the invalid mode By disabling the detection processing or post-detection processing for the detected subject in the invalid mode, even if, for example, erroneous detection is repeated in the detection processing, an erroneous subject name due to erroneous detection will not be displayed or recorded. You can prevent it from being broken. Further, by disabling only the post-detection processing, even in the case of the disabled mode, the detection processing or the post-detection processing can be continued depending on the subject included in the endoscopic image 51 . Therefore, it is possible to prevent an erroneous subject name from being continuously displayed or recorded due to erroneous detection, and perform notification processing or recording processing for the detection of another subject. Therefore, in the case of erroneous detection, the flexible setting saves the trouble of various corrections such as display or recording of the erroneous detection result 54, and enables effective detection processing.
  • the mode setting unit 15 cancels the invalid mode when a preset cancellation condition is satisfied. Therefore, the mode setting unit 15 sets the invalid mode for a period from when a specific instruction is given by the user to when a specific release condition is satisfied.
  • the cancellation condition can be, for example, the point in time when the invalid mode continuation time, which is a preset time, has elapsed since the invalid mode started.
  • the invalid mode continuation time can be set as appropriate, and is set to 3 minutes, for example.
  • the mode setting unit 15 sets an operation mode other than the invalid mode, for example, sets the operation mode immediately before starting the invalid mode. Note that after canceling the invalid mode, the operation mode may be changed to a preset operation mode other than the operation mode before shifting to the invalid mode.
  • the invalid mode in which the detection process or the post-detection process for the forceps 52 as a detected object is invalid, after the invalid mode starts, three minutes set as the invalid mode duration time, the invalid mode is disabled. Disabled mode is automatically released when the release condition is satisfied. Time t4 is the time when three minutes have passed since the invalid mode start time.
  • the release condition can also be based on the endoscopic image 51. In this case, since it is necessary to perform the detection process on the endoscopic image 51, in the invalid mode, only the post-detection process is invalidated, and the detection process is not invalidated.
  • the release condition is, for example, that a subject different from the detected subject is detected in the endoscopic image 51 .
  • the range of the subject different from the detected subject can be set according to the subject detection process, but as described above, it can be the same as the example of the detected subject and the subject to be invalidated. .
  • the detection processing is processing for detecting each type of subject
  • the cancellation condition is satisfied and the invalid mode is set. unlock.
  • the cancellation condition is that a subject different from the detected subject is detected, the cancellation condition is satisfied when the detection result 54 is "second snare" different from “snare”.
  • the invalid mode is released, the post-detection processing invalidated in the invalid mode is enabled, and each processing set as the post-detection processing is started.
  • the cancellation condition may use the degree of similarity between the multiple endoscopic images 51 acquired by the medical image acquisition unit 11 .
  • the medical image processing apparatus 10 includes a similarity calculator 81.
  • FIG. The similarity calculator 81 calculates a similarity indicating the degree to which a plurality of endoscopic images are similar to each other.
  • various feature amounts can be calculated from the endoscopic image, and the degree of similarity can be calculated using the calculated feature amounts.
  • the feature amount may be a feature vector in a plurality of endoscopic images 51, and the degree of similarity may be calculated based on the mutual closeness based on the feature vector.
  • the cancellation condition is that the degree of similarity between the most recently acquired endoscopic images 51 is less than or equal to a preset value.
  • the degree of similarity is the extent to which the plurality of endoscopic images 51 are similar to each other. have a low degree of similarity.
  • the plurality of recently acquired endoscopic images can be the latest acquired endoscopic image and the previously acquired endoscopic images for a preset number of frames. Note that a frame is a unit of photographing an endoscopic image, and one endoscopic image is obtained for each frame.
  • the degree of similarity between two endoscopic images is calculated in the order of the latest acquisition. If the calculated similarity is less than or equal to a preset threshold value, it is entered as “low” in the "similarity between image frames” column. Similarly, when the similarity is high and does not fall below the threshold, it is indicated as "high”. In the case of "high” where the similarity is not equal to or less than the preset threshold, the cancellation condition is not satisfied, so the invalid mode is continued without being canceled. If the similarity is "low”, i.e., equal to or lower than the threshold, the cancellation condition is satisfied, and the invalid mode is canceled. After canceling the invalid mode, it returns to the operation mode before shifting to the invalid mode.
  • the invalid mode can be automatically and appropriately canceled even if the error in the detection process continues.
  • the medical image processing apparatus 10 preferably notifies the user when receiving a specific instruction from the user, when setting the invalid mode, or when canceling the setting of the invalid mode.
  • the medical image acquisition unit 11 acquires an endoscopic image 51 acquired by the endoscope device 16 (step ST110, the detection processing unit 12 detects the endoscopic image acquired by the medical image acquisition unit 11). Detection processing is performed on the image 51 (step ST120).
  • a detection result 54 by the detection process is displayed on the display 18 as a detection result display 55 (step ST130).
  • the user looks at the detection result display 55 and the endoscopic image 51 displayed on the display 18 and determines whether or not the detection result 54 is an erroneous detection.
  • a specific instruction is given using the footswitch or the like which is the input device 19 (Y in step ST140).
  • the operation mode is switched to invalid mode (step ST160).
  • step ST170 After switching to the disabled mode, in the disabled mode of the present embodiment, detection processing is performed and post-detection processing is not performed, so an endoscopic image is acquired (step ST170) and detection processing is performed (step ST180). If the invalid mode is not canceled (N in step ST190), the endoscopic image is acquired again in the invalid mode (step ST170), and detection processing is performed (step ST180).
  • the invalid mode is canceled, the operation mode immediately before setting the invalid mode is restored, and endoscopic image acquisition is started (step ST110). If the invalid mode is not canceled (N in step ST190) and the examination is continued (Y in step ST200), the endoscopic image is acquired again and detection processing is performed in the invalid mode (step ST110). If the inspection is not to be continued (N in step ST200), the inspection ends.
  • step ST140 If no specific instruction is given (N in step ST140) and the examination is to be continued (Y in step ST150), the endoscopic image is acquired again and detection processing is performed (step ST110). If the inspection is not continued (N in step ST150), the inspection ends.
  • the above embodiments and the like are programs for a medical image processing apparatus, and include a function of acquiring a plurality of time-series medical images, a function of performing detection processing for detecting a subject from each of the plurality of medical images, and A function of performing post-detection processing with predetermined processing contents corresponding to the detected subject when it is detected, and the post-detection processing notifying the user according to the detected subject. and making the computer execute a function of setting at least post-detection processing for the detected subject to an invalid mode for invalidating at least the post-detection processing for the detected subject for a preset period when a specific instruction is received from the user after notification.
  • the hardware structure of the processing unit of is various processors as shown below.
  • Various processors include CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), etc., which are general-purpose processors that run software (programs) and function as various processing units.
  • Programmable Logic Devices which are processors, and dedicated electric circuits, which are processors with circuit configurations specifically designed to perform various types of processing.
  • One processing unit may be composed of one of these various processors, or composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA).
  • a plurality of processing units may be configured by one processor.
  • a plurality of processing units may be configured by one processor.
  • this processor functions as a plurality of processing units.
  • SoC System On Chip
  • SoC System On Chip
  • the various processing units are configured using one or more of the above various processors as a hardware structure.
  • the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Surgery (AREA)
  • Quality & Reliability (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Endoscopes (AREA)
PCT/JP2022/034598 2021-11-30 2022-09-15 医療画像処理装置、医療画像処理装置の作動方法、及び医療画像処理装置用プログラム Ceased WO2023100446A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023564748A JPWO2023100446A1 (https=) 2021-11-30 2022-09-15
CN202280079439.0A CN118338833A (zh) 2021-11-30 2022-09-15 医疗图像处理装置、医疗图像处理装置的工作方法及医疗图像处理装置用程序
US18/676,391 US20240312019A1 (en) 2021-11-30 2024-05-28 Medical image processing apparatus, method for operating medical image processing apparatus, and non-transitory computer readable medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021194538 2021-11-30
JP2021-194538 2021-11-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/676,391 Continuation US20240312019A1 (en) 2021-11-30 2024-05-28 Medical image processing apparatus, method for operating medical image processing apparatus, and non-transitory computer readable medium

Publications (1)

Publication Number Publication Date
WO2023100446A1 true WO2023100446A1 (ja) 2023-06-08

Family

ID=86611871

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/034598 Ceased WO2023100446A1 (ja) 2021-11-30 2022-09-15 医療画像処理装置、医療画像処理装置の作動方法、及び医療画像処理装置用プログラム

Country Status (4)

Country Link
US (1) US20240312019A1 (https=)
JP (1) JPWO2023100446A1 (https=)
CN (1) CN118338833A (https=)
WO (1) WO2023100446A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3838109A4 (en) * 2018-08-17 2021-10-06 FUJIFILM Corporation ENDOSCOPIC SYSTEM
JP7617622B2 (ja) * 2021-06-24 2025-01-20 株式会社Ａｉメディカルサービス 検査支援装置、検査支援方法および検査支援プログラム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020036224A1 (ja) * 2018-08-17 2020-02-20 富士フイルム株式会社 内視鏡システム
WO2020090729A1 (ja) * 2018-11-01 2020-05-07 富士フイルム株式会社 医療画像処理装置、医療画像処理方法及びプログラム、診断支援装置
JP2021100555A (ja) * 2019-12-24 2021-07-08 富士フイルム株式会社 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020036224A1 (ja) * 2018-08-17 2020-02-20 富士フイルム株式会社 内視鏡システム
WO2020090729A1 (ja) * 2018-11-01 2020-05-07 富士フイルム株式会社 医療画像処理装置、医療画像処理方法及びプログラム、診断支援装置
JP2021100555A (ja) * 2019-12-24 2021-07-08 富士フイルム株式会社 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム

Also Published As

Publication number Publication date
US20240312019A1 (en) 2024-09-19
JPWO2023100446A1 (https=) 2023-06-08
CN118338833A (zh) 2024-07-12

Similar Documents

Publication Publication Date Title
JP7525248B2 (ja) 医用情報処理装置及び医用情報処理プログラム
JP5486432B2 (ja) 画像処理装置、その作動方法およびプログラム
JP2019093144A (ja) 対話型イベントタイムラインのためのシステム及び方法
US12268354B2 (en) Image recording device, image recording method, and recording medium for recording time-series images of endoscopy
US20240312019A1 (en) Medical image processing apparatus, method for operating medical image processing apparatus, and non-transitory computer readable medium
US10163239B2 (en) Computer-aided diagnostic apparatus and method based on diagnostic intention of user
WO2020040086A1 (ja) 医療画像処理システム
CN114126493B (zh) 通过超声探头检测组织接触的系统和方法
US20240386701A1 (en) Medical image processing apparatus and endoscope system
US20240386692A1 (en) Medical image processing device, method of operating medical image processing device, non-transitory computer readable medium, and endoscope system
CN113925608B (zh) 手术辅助系统以及手术辅助方法
US20240148235A1 (en) Information processing apparatus, information processing method, endoscope system, and report creation support device
WO2018008661A1 (ja) 制御装置、制御方法、制御システム及びプログラム
EP4213093B1 (en) Medical image processing device, ultrasonic diagnostic apparatus, and program
JP2018011928A (ja) 制御装置、制御方法、制御システム及びプログラム
JP6568375B2 (ja) 眼科情報処理システム、画像処理装置、および画像処理方法
KR102829233B1 (ko) 이미지 수득의 최적 조건 자동 유도 특징을 갖는 의료영상의 광학적 병리진단 시스템
CN111354453A (zh) 医疗成像系统
US20240136034A1 (en) Information processing apparatus, information processing method, endoscope system, and report creation support device
JP2007000639A (ja) 脈管内検査を実施するための装置
JP7220542B2 (ja) 医用画像処理装置、医用画像処理方法及び医用画像処理プログラム
JP5573106B2 (ja) 医用画像表示システム及びプログラム
JP2018187440A (ja) 医用診断支援装置、方法およびプログラム
JP2024141601A (ja) 胸腔鏡下肺がん手術の胸膜浸潤プログラム
JP2024070317A (ja) 処理装置、処理プログラム、処理方法及び処理システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22900881

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023564748

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 202280079439.0

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22900881

Country of ref document: EP

Kind code of ref document: A1