CN116704288A - Training method and recognition method for appendectomy surgical instrument recognition model - Google Patents

Training method and recognition method for appendectomy surgical instrument recognition model Download PDF

Info

Publication number
CN116704288A
CN116704288A CN202310701884.2A CN202310701884A CN116704288A CN 116704288 A CN116704288 A CN 116704288A CN 202310701884 A CN202310701884 A CN 202310701884A CN 116704288 A CN116704288 A CN 116704288A
Authority
CN
China
Prior art keywords
appendectomy
training
surgical instrument
image
image data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310701884.2A
Other languages
Chinese (zh)
Inventor
邵益平
徐智龙
陶海露
鲁建厦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN202310701884.2A priority Critical patent/CN116704288A/en
Publication of CN116704288A publication Critical patent/CN116704288A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/003Navigation within 3D models or images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Remote Sensing (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to a training method and a recognition method for a recognition model of an appendectomy surgical instrument, wherein the model training method comprises the following steps: marking the position and name of the appendectomy surgical instrument in the image of the appendectomy surgery to generate an image dataset; extracting and labeling a region of interest of the image dataset; performing image preprocessing on the image data set; dividing the image dataset into a training set and a testing set; constructing a Pytorch-based self-adaptive Mask R-CNN network; inputting the training set into the self-adaptive Mask R-CNN network, and verifying by using the test set, and training the self-adaptive Mask R-CNN network until the accuracy of the self-adaptive Mask R-CNN network reaches a preset value. The appendectomy surgical instrument identification model trained by the model training method can effectively identify the category of laparoscopic surgical instruments, accurately identify the position of the surgical instruments, and has certain robustness on illumination, angles and other influence factors.

Description

Training method and recognition method for appendectomy surgical instrument recognition model
Technical Field
The application relates to the technical field of medical appliances, in particular to a training method and a recognition method for a recognition model of an appendectomy surgical instrument.
Background
Appendicitis refers to the pathological changes of appendicitis caused by bacterial infection in the appendix, such as appendicitis mucosal inflammatory reaction, full-layer penetration, appendiceal rupture, abscess formation, etc. Appendicitis is a common acute abdomen, which can cause symptoms such as abdominal pain, fever and the like. Appendicitis generally needs to be treated by surgery, and appendectomy is the main means for treating appendicitis clinically at present. However, the quality and effectiveness of appendectomy procedures depend on the experience and skill of the physician, and problems such as long surgical time, high trauma, and postoperative pain are easily encountered during the procedure. Therefore, how to improve the surgical quality and effect of appendectomy has important significance in reducing the surgical risk and improving the therapeutic effect of patients.
By evaluating various indexes of the appendectomy procedure, such as the number of times of switching appliances in the procedure, the movement range of the appliances, etc., the method has important significance for improving the surgical skills of doctors and improving the surgical results. However, the current laparoscopic surgery assessment requires a professional doctor to perform manual implementation, which requires the doctor to have professional literacy, is complicated and takes a long time, and has a certain subjectivity. Therefore, how to evaluate the laparoscopic surgical procedure by an automated technique becomes a urgent problem to be solved.
Currently, the conventional appendectomy surgical instrument identification method is mainly based on a traditional image processing algorithm. These algorithms require manual extraction of features and design rules, and then classification using a classifier. However, the accuracy of this approach is affected by a variety of factors, such as changes in the pose of the surgical instrument, illumination and background interference of the surgical scene, etc., resulting in lower stability and accuracy of the algorithm. In addition, the design and optimization of this method requires a lot of manpower and time, and lacks versatility and flexibility, and is difficult to adapt to different surgical scenarios and instrument types.
Therefore, a method for identifying appendectomy surgical instruments is needed, which has certain robustness to illumination, angles and other influencing factors and has higher identification precision.
Disclosure of Invention
Based on the above-mentioned shortcomings and drawbacks of the prior art, the present application provides a training method and an identification method for an appendectomy surgical instrument identification model that meets the above-mentioned needs.
In order to achieve the aim of the application, the application adopts the following technical scheme:
a training method for an appendectomy surgical instrument identification model, comprising the steps of:
s1, marking the position and the name of an appendectomy surgical instrument in an image of appendectomy surgery to generate an image data set;
s2, extracting and labeling a region of interest of the image dataset;
s3, performing image preprocessing on the image data set;
s4, dividing the image data set into a training set and a testing set;
s5, constructing a Pytorch-based self-adaptive Mask R-CNN network;
s6, inputting the training set into the self-adaptive Mask R-CNN network, and verifying by using the test set, and training the self-adaptive Mask R-CNN network until the accuracy of the self-adaptive Mask R-CNN network reaches a preset value.
As a preferred embodiment, step S1 includes:
s11, dividing an image of appendicitis excision operation into a plurality of divided images;
s12, marking the positions and names of appendectomy surgical instruments in a plurality of segmented images, and generating an image data set.
As a further preferred embodiment, the segmentation of step S11 segments the image of the appendectomy into 10 frames/slice of segmented image.
As a further preferred embodiment, step S12 uses labelme software to manually label the location and name of the appendectomy surgical instrument.
As a preferred embodiment, step S3 includes:
s31, carrying out image intensity range normalization and histogram equalization on the image dataset;
s32, converting the image data set into a coco data set.
As a preferred embodiment, step S3 is preceded by the further step of:
s30, carrying out data enhancement on the image data sets to obtain more image data sets.
As a preferred embodiment, step S4 is performed with 8: the scale of 2 divides the image dataset into a training set and a test set.
As a preferred embodiment, step S2 identifies the region of interest of the image dataset using a fast RCNN network.
On the other hand, the application also provides a recognition method of the appendectomy surgical instrument, and the recognition result of the appendectomy surgical instrument is obtained by recognizing the image by using the model trained by the method according to any one of the above.
Compared with the prior art, the application has the beneficial effects that:
the appendectomy surgical instrument identification model trained by the model training method can effectively identify the category of laparoscopic surgical instruments, accurately identify the position of the surgical instruments, and has certain robustness on illumination, angles and other influence factors. Based on the model trained by the method, the identification method of the appendectomy surgical instrument can effectively identify the category of the laparoscopic surgical instrument and accurately identify the position of the surgical instrument.
Drawings
FIG. 1 is a flow chart of a training method of the appendectomy surgical instrument identification model of the present application;
fig. 2 is a schematic diagram of the structure of the Mask R-CNN network of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
In the following description, various embodiments of the application are provided, and various embodiments may be substituted or combined, so that the application is intended to include all possible combinations of the same and/or different embodiments described. Thus, if one embodiment includes feature A, B, C and another embodiment includes feature B, D, then the present application should also be considered to include embodiments that include one or more of all other possible combinations including A, B, C, D, although such an embodiment may not be explicitly recited in the following.
The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements described without departing from the scope of the application. Various examples may omit, replace, or add various procedures or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.
The application provides a training method of an appendectomy surgical instrument identification model, which comprises the following steps:
s1, marking the position and the name of an appendectomy surgical instrument in an image of appendectomy surgery to generate an image data set;
s2, extracting and labeling a region of interest of the image dataset;
s3, performing image preprocessing on the image data set;
s4, dividing the image data set into a training set and a testing set;
s5, constructing a Pytorch-based self-adaptive Mask R-CNN network;
s6, inputting the training set into the self-adaptive Mask R-CNN network, and verifying by using the test set, and training the self-adaptive Mask R-CNN network until the accuracy of the self-adaptive Mask R-CNN network reaches a preset value.
In a preferred embodiment of the present application, step S1 comprises the steps of:
s11, dividing an image of appendicitis excision operation into a plurality of divided images;
s12, marking the positions and names of appendectomy surgical instruments in a plurality of segmented images, and generating an image data set.
Specifically, in step S11, the image of appendicitis excision operation is divided into a plurality of divided images by a video frame splitting method, the selected image of appendicitis excision operation is a video shot by a laparoscope in the operation process, the frame rate is 30fps, the image size is 1920×1080, and the video is divided into pictures in 10 frames/sheet.
Step S12 is to use a manual labelme software method to identify the segmented image generated in step S11, and mark the positions and names of all appliances appearing in the image in the operation process for subsequent training. In one example of this embodiment, the callout contains 9 classes of instruments: nontraumatic forceps (atraumatic grasper), aspirator (Aspirator), dissecting forceps (separation grasper), electric hook, clip applier, needle holder, scissor, trocar, bag, which account for 31.05%, 6.39%, 36.69%, 12.36%, 2.45%, 8.36%, 2.11%, 0%, 0.61% of the total number of appliances in the dataset, respectively. The test set is not processed.
In a preferred embodiment of the application, the extraction of step S2 and labeling uses the Faster RCNN network to extract the region of interest on each image in the image dataset of step S1.
Another embodiment of the present application provides a specific implementation method of step S3, including the following steps:
s31, carrying out image intensity range normalization and histogram equalization on the image dataset;
s32, converting the image data set into a coco data set.
Specifically, the coco data set format structure of the coco data set in S32 is as follows:
co2017, data set root directory;
is described-train 2017, all training image folders;
all verification image folders;
-labeling folders corresponding to the nodes;
training set annotation files corresponding to target detection and segmentation tasks;
to-objects-val2017. Json: verification set annotation files corresponding to object detection and segmentation tasks.
In another embodiment of the present application, step S3 is preceded by the further step of:
s30, carrying out data enhancement on the image data sets to obtain more image data sets. Such data enhancement methods include, but are not limited to, clipping or flipping.
In this embodiment, specifically, step S4 is performed with 8: the scale of 2 divides the image dataset into a training set and a test set.
The embodiment further provides a specific implementation of step S6, wherein a schematic structural diagram of the Mask R-CNN network is shown in fig. 1, and the feature extraction is performed on the picture of the input training set by using a backbone network (such as res net, VGG, etc.). A target detection area (anchor box) is generated using an area candidate network (RPN), while for each anchor box, a bounding box (bounding box) offset of its corresponding object is calculated, and a classification probability of whether the object is contained.
For each candidate box, a Mask branching network is introduced to generate a Mask, which branches generate a binary Mask on each candidate box for representing the exact contour of the appliance.
The suggested regions are mapped onto the feature map and a ROI Pooling layer is used to generate a fixed size feature map for each suggested region. The classification probabilities and bounding box offsets are then trained with the detected classification loss (detection classification loss) and the detected bounding box regression loss (detection bounding box regression loss). At the same time, the mask branching network is trained with segmentation loss (segmentation loss).
During training, all loss are added and all network parameters are updated by the back propagation algorithm. And gradually converging parameters of the Mask R-CNN network through repeated iterative optimization, and finally iterating to obtain a model capable of accurately detecting, classifying and dividing the appliance.
In the test stage, a test picture is input, and a target detection area and a Mask are generated through a Mask R-CNN network, so that the segmentation and detection of the appliance are realized. When the prediction accuracy of the training set reaches a preset value, finishing training; if the prediction accuracy of the training set is lower than the preset value, reconstructing the training set for training.
According to the model training method provided by the embodiment of the application, the trained appendectomy surgical instrument identification model can effectively identify the category of laparoscopic surgical instruments, accurately identify the position of the surgical instruments, and has certain robustness on the influence factors such as illumination, angles and the like.
The application also provides a recognition method of the appendectomy surgical instrument, which is used for recognizing the image by applying the model trained by the method of any embodiment to obtain the recognition result of the appendectomy surgical instrument.
The identification method of the appendectomy surgical instrument can effectively identify the category of the laparoscopic surgical instrument based on the appendectomy surgical instrument identification model trained by the embodiment, accurately identify the position of the surgical instrument, and has certain robustness on the influence factors such as illumination, angles and the like.
The foregoing is merely exemplary embodiments of the present disclosure and is not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.

Claims (9)

1. A training method for an appendectomy surgical instrument identification model, comprising the steps of:
s1, marking the position and the name of an appendectomy surgical instrument in an image of appendectomy surgery to generate an image data set;
s2, extracting and labeling a region of interest of the image dataset;
s3, performing image preprocessing on the image data set;
s4, dividing the image data set into a training set and a testing set;
s5, constructing a Pytorch-based self-adaptive Mask R-CNN network;
s6, inputting the training set into the self-adaptive Mask R-CNN network, and verifying by using a test set, and training the self-adaptive Mask R-CNN network until the accuracy of the self-adaptive Mask R-CNN network reaches a preset value.
2. The method for training an identification model of an appendectomy surgical instrument of claim 1, wherein step S1 comprises:
s11, dividing the image of the appendicitis excision operation into a plurality of divided images;
s12, marking the positions and names of appendectomy surgical instruments in the plurality of divided images, and generating an image data set.
3. The method according to claim 2, wherein the segmentation in step S11 segments the image of the appendectomy into 10 frames/slice of segmented images.
4. The method for training an identification model of an appendectomy surgical instrument according to claim 2, wherein step S12 uses labelme software to manually label the location and name of the appendectomy surgical instrument.
5. The method for training an identification model of an appendectomy surgical instrument of claim 1, wherein step S3 comprises:
s31, carrying out image intensity range normalization and histogram equalization on the image dataset;
s32, converting the image data set into a coco data set.
6. The method for training an identification model of an appendectomy surgical instrument according to claim 1, wherein step S3 is preceded by the steps of:
s30, carrying out data enhancement on the image data sets to obtain more image data sets.
7. The method for training an identification model of an appendectomy surgical instrument of claim 1, wherein step S4 is performed with a training pattern of 6: the scale of 2 divides the image dataset into a training set and a test set.
8. The method of training an appendectomy surgical instrument identification model of claim 1, wherein step S2 identifies the region of interest of the image dataset using a Faster RCNN network.
9. A method for identifying an appendectomy surgical instrument, wherein the image is identified using a model trained by the method of any one of claims 1-8, resulting in an identification of the appendectomy surgical instrument.
CN202310701884.2A 2023-06-14 2023-06-14 Training method and recognition method for appendectomy surgical instrument recognition model Pending CN116704288A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310701884.2A CN116704288A (en) 2023-06-14 2023-06-14 Training method and recognition method for appendectomy surgical instrument recognition model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310701884.2A CN116704288A (en) 2023-06-14 2023-06-14 Training method and recognition method for appendectomy surgical instrument recognition model

Publications (1)

Publication Number Publication Date
CN116704288A true CN116704288A (en) 2023-09-05

Family

ID=87832159

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310701884.2A Pending CN116704288A (en) 2023-06-14 2023-06-14 Training method and recognition method for appendectomy surgical instrument recognition model

Country Status (1)

Country Link
CN (1) CN116704288A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118154991A (en) * 2024-05-09 2024-06-07 四川大学华西医院 Fewer-sample appendix classification system based on ultrasonic image and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118154991A (en) * 2024-05-09 2024-06-07 四川大学华西医院 Fewer-sample appendix classification system based on ultrasonic image and storage medium

Similar Documents

Publication Publication Date Title
CN110458883B (en) Medical image processing system, method, device and equipment
CN108615051B (en) Diabetic retina image classification method and system based on deep learning
CN107180421B (en) Fundus image lesion detection method and device
Kowal et al. Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images
CN112634261A (en) Stomach cancer focus detection method and device based on convolutional neural network
CN112614128B (en) System and method for assisting biopsy under endoscope based on machine learning
CN110335241B (en) Method for automatically scoring intestinal tract preparation after enteroscopy
CN109635846A (en) A kind of multiclass medical image judgment method and system
CN116704288A (en) Training method and recognition method for appendectomy surgical instrument recognition model
CN109146867B (en) Oral cavity curved surface CT image biological feature extraction and matching method and device
CN110766659A (en) Medical image recognition method, apparatus, device and medium
CN111353978B (en) Method and device for identifying heart anatomy structure
CN109544528B (en) Lung nodule image identification method and device
CN109241898B (en) Method and system for positioning target of endoscopic video and storage medium
CN111882559B (en) ECG signal acquisition method and device, storage medium and electronic device
CN112950552B (en) Rib segmentation marking method and system based on convolutional neural network
CN114494798A (en) Electrocardiogram artifact confirmation method, terminal equipment and storage medium
CN116779093B (en) Method and device for generating medical image structured report and computer equipment
CN110647889B (en) Medical image recognition method, medical image recognition apparatus, terminal device, and medium
CN110443792B (en) Bone scanning image processing method and system based on parallel deep neural network
CN107705829B (en) Medical image transmission method based on intelligent identification
CN112634266B (en) Semi-automatic labeling method, medium, equipment and device for laryngoscope image
Mahalaxmi et al. Detection of Lung Cancer Using Binarization Technique.
Nahrawi et al. A Novel Nucleus Detection on Pap Smear Image Using Mathematical Morphology Approach
CN115578313A (en) Left ventricle heart chamber contour extraction method, device, medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination