WO2022246720A1 - 手术动作识别模型的训练方法、介质和设备 - Google Patents

手术动作识别模型的训练方法、介质和设备 Download PDF

Info

Publication number
WO2022246720A1
WO2022246720A1 PCT/CN2021/096244 CN2021096244W WO2022246720A1 WO 2022246720 A1 WO2022246720 A1 WO 2022246720A1 CN 2021096244 W CN2021096244 W CN 2021096244W WO 2022246720 A1 WO2022246720 A1 WO 2022246720A1
Authority
WO
WIPO (PCT)
Prior art keywords
fusion
pyramid
column
layers
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/096244
Other languages
English (en)
French (fr)
Inventor
贾富仓
徐文廷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2022246720A1 publication Critical patent/WO2022246720A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the invention belongs to the technical field of image processing, and in particular relates to a training method for a surgical action recognition model, a computer-readable storage medium, and computer equipment.
  • Surgical robot system is an intelligent computer-aided system that can assist surgeons to complete operations.
  • the processing result based on the image algorithm enables the auxiliary surgical robot to make corresponding surgical operation actions to assist the attending surgeon to complete the surgical operation.
  • the surgical robot system not only has the characteristics of minimally invasive surgery, less trauma, quick recovery, and less pain for the patient, but also because the introduction of the intelligent auxiliary robot system combines the patient's image data with the patient's anatomical parts in the actual operation, the operation can be tracked in real time during the operation
  • the real-time recognition of instruments and surgical actions enables surgeons to have a clearer understanding of real-time changes in anatomical parts, making minimally invasive surgery safer, more stable and more reliable.
  • the real-time detection of surgical actions through the intervention of the auxiliary robotic arm can largely replace the tasks of the auxiliary surgeon, reducing the demand for surgeons during the operation and the misoperation caused by improper cooperation between multiple doctors.
  • the target recognition task for surgical actions is the most basic and crucial technology in the surgical assisted robot system.
  • the real-time detection of surgical actions based on deep learning realizes the core low-level algorithm in the surgical robot system and provides key technical support for the development of semi-autonomous or fully self-service surgical robots in the future.
  • Static methods only have spatial information (image data) without any temporal context of the current frame.
  • Dynamic activity detection methods use video data that provides temporal context information for motion in the video.
  • the above methods are all applied to natural scenes and simulated surgical scenes, which are very different from surgical action detection in real scenes.
  • the scene captured with the endoscopic camera is too close to show the complete organ and its surroundings, so there is little contextual information.
  • Such a dynamic-based text detection method is difficult to effectively use the time and space information between the upper and lower frames of the surgical video, and these methods are difficult to meet the task requirements of surgical action detection.
  • the movement and orientation of the endoscope at close range makes organs appear very different from different angles, and these highly variable conditions can also cause traditional object detection algorithms to fail.
  • a training method for a surgical action recognition model includes a backbone network, a pyramid feature aggregation network, and a prediction network, wherein the pyramid feature aggregation network includes a feature map collection module and a feature map divergence module, and the feature map collection Between the input unit of the module and the output unit of the feature map divergence module, there is a skip connection fusion path, and the training method includes:
  • the hierarchical feature map is input to the pyramid feature aggregation network, and the fusion process of the feature map collection module and the feature map divergence module is sequentially processed to obtain a number of fusion feature maps of different scales;
  • the loss function is updated according to the predicted target value and the obtained real target value, and the model parameters of the surgical action recognition model are adjusted according to the updated loss function.
  • the feature map collection module includes the first column of pyramid layers, the second column of pyramid layers, and the third column of pyramid layers with the number of fusion units decreasing
  • the feature map divergence module includes the third column of pyramid layers with the number of fusion units increasing.
  • a column pyramid layer, a fourth column pyramid layer and a fifth column pyramid layer wherein the first column pyramid layer is the input unit of the feature map collection module, and the fifth column pyramid layer is the feature map
  • the output unit of the divergence module, and the number of fusion units in the first column of pyramid layers is the same as that of the fifth column of pyramid layers, and the number of fusion units in the second column of pyramid layers is the same as that of the fourth column of pyramid layers.
  • the fusion unit transmits information through a predetermined fusion path network.
  • first column of pyramid layers and the fifth column of pyramid layers each include five fusion units of different feature scales
  • second column of pyramid layers and the fourth column of pyramid layers each include three different
  • a fusion unit of a feature scale, the pyramid layer of the third column has a fusion unit.
  • the predetermined fusion path network includes:
  • the first fusion path is from bottom to top in the pyramid layer from small-scale fusion units to large-scale fusion units;
  • the second fusion path is used to diagonally connect the fusion units between two adjacent layers, and fuse the feature map information of different scales between adjacent layers by downsampling;
  • the third fusion path is used to diagonally connect the fusion units between two adjacent layers, and fuse the feature map information of different scales between adjacent layers by upsampling;
  • the fourth fusion path is used to horizontally connect the fusion units of the same layer to fuse the feature map information of the same scale;
  • the fifth fusion path is from top to bottom in the first pyramid layer from the large-scale fusion unit to the small-scale fusion unit;
  • a skip-connect fusion path is used to connect the fusion units of the same scale in the pyramid layer in the first column and the pyramid layer in the fifth column.
  • the hierarchical feature map obtained by the backbone network has three scales, and the five fusion units in the first column of pyramid layers are respectively the first fusion unit, the second fusion unit, the second fusion unit, and the second fusion unit that increase in scale from bottom to top.
  • Three fusion units, a fourth fusion unit and a fifth fusion unit, the hierarchical feature maps of three scales are respectively input to the first fusion unit, the second fusion unit and the third fusion unit; the first fusion unit
  • the fusion unit, the second fusion unit, and the third fusion unit are connected by a fifth fusion path, and the third fusion unit, the fourth fusion unit, and the fifth fusion unit are connected by a first fusion path.
  • the formula of the loss function is as follows:
  • L cls is the Focal loss function
  • L reg is the IOU loss function
  • N pos represents the number of positive samples
  • ⁇ is the balance weight of L reg and the value is 1
  • Pointing (x, y) corresponds to the category of ground-truth
  • P x, y pointing (x, y) corresponds to the predicted value category
  • Pointing (x, y) corresponds to the target box of ground-truth
  • t x, y pointing (x, y) corresponds to the target box of the predicted value
  • is an exponential function when The time value is 1, The exponential function is 0 when other values are taken.
  • the present invention also discloses a computer-readable storage medium, the computer-readable storage medium stores the training program of the surgical action recognition model, and when the training program of the surgical action recognition model is executed by the processor, the above-mentioned surgical action recognition is realized The training method of the model.
  • the present invention also discloses a computer device, which includes a computer-readable storage medium, a processor, and a training program for a surgical action recognition model stored in the computer-readable storage medium, the surgical action recognition model
  • the training program is executed by the processor, the above-mentioned training method for the surgical action recognition model is realized.
  • the invention discloses a training method for a surgical action recognition model. Compared with the traditional training method, it has the following technical effects:
  • the resulting fusion feature map can more accurately predict the surgical category and the position of the border, and solve the problem of indistinct surgical action video features.
  • Fig. 1 is the flowchart of the training method of the surgical action recognition model of embodiment one of the present invention
  • Fig. 2 is the frame diagram of the training test model of the surgical action recognition model of embodiment one of the present invention
  • Fig. 3 is the structural representation of the pyramidal feature aggregation network of embodiment one of the present invention.
  • FIG. 4 is a schematic structural diagram of a training device for a surgical action recognition model according to Embodiment 2 of the present invention.
  • FIG. 5 is a functional block diagram of a computer device according to an embodiment of the present invention.
  • the existing detection method based on deep learning needs to rely on sufficient context information, and in the real surgical scene, since the camera shooting scene is very close to , it is difficult to extract effective context information, and the classification accuracy cannot be improved.
  • This application provides a training method for a surgical action recognition model. First, hierarchical feature maps of different scales are extracted through the backbone network, and then the hierarchical feature maps are fused using the pyramid feature aggregation network, and the feature map information of different scales is Perform full fusion to obtain fusion feature maps of different scales, and finally use the prediction network to predict and use the updated loss function to adjust the model parameters of the surgical action recognition model.
  • This training method can make full use of the spatial information in the video and improve the existing model. The ability to integrate multi-scale spatial information can improve the recognition accuracy and detection speed of the model.
  • the surgical action recognition model of the first embodiment includes a backbone network, a pyramid feature aggregation network, and a prediction network, wherein the pyramid feature aggregation network includes a feature map collection module and a feature map divergence module , there is a jump-connect fusion path between the input unit of the feature map collection module and the output unit of the feature map divergence module, and the training method of the surgical action recognition model includes the following steps:
  • Step S10 Input the obtained original surgical action images into the backbone network to obtain several hierarchical feature maps of different scales
  • Step S20 input the hierarchical feature map into the pyramid feature aggregation network, and sequentially undergo the fusion processing of the feature map collection module and the feature map divergence module to obtain a number of fusion feature maps of different scales;
  • Step S30 Input several fused feature maps of different scales into the prediction network to obtain the predicted target value
  • Step S40 update the loss function according to the predicted target value and the acquired real target value, and adjust the model parameters of the surgical action recognition model according to the updated loss function.
  • step S10 the backbone network processes the original surgical action image to obtain hierarchical feature maps of three scales C3, C4, and C5, and then in step S20, input the hierarchical feature maps of corresponding scales into In the fusion unit of the corresponding scale, the feature map information is fused.
  • the feature map collection module includes the first column of pyramid layers P1, the second column of pyramid layers P2, and the third column of pyramid layers P3 with the number of fusion units decreasing, and the feature map divergence module includes the number of fusion units increasing.
  • the number of fusion units is the same, and each fusion unit transmits information through a predetermined fusion path network.
  • the entire pyramid feature aggregation network is butterfly-shaped, and the feature map information of different scales is fully fused through each fusion unit.
  • the first column of pyramid layers and the fifth column of pyramid layers each include five fusion units of different feature scales
  • the second column of pyramid layers and the fourth column of pyramid layers each include three different A fusion unit of a feature scale
  • the pyramid layer of the third column has a fusion unit.
  • the scales of the fusion units located in the same row are the same, which are also called fusion units of the same layer, and the scales of the fusion units in the same column of pyramid layers decrease from top to bottom.
  • the predetermined merged path network includes a first merged path 11 , a second merged path 12 , a third merged path 13 , a fourth merged path 14 , a fifth merged path 15 and a jump-connected merged path 16 .
  • the first fusion path 11 points from the small-scale fusion unit to the large-scale fusion unit from bottom to top in the pyramid layer;
  • the second fusion path 12 is used to diagonally connect the fusion units between two adjacent layers, through Downsampling fuses feature map information of different scales between adjacent layers;
  • the third fusion path 13 is used to diagonally connect fusion units between two adjacent layers, and fuses feature maps of different scales between adjacent layers by upsampling information;
  • the fourth fusion path 14 is used to horizontally connect the fusion units of the same layer to fuse the feature map information of the same scale;
  • the fifth fusion path 15 is from top to bottom in the first pyramid layer from the large-scale fusion unit to the small The fusion unit of the scale;
  • the skip connection fusion path 16 is used to connect the fusion units of the same scale in the first column of pyramid layers and the fifth column of pyramid layers, that is, to fuse the features between the input unit and the output unit of the same layer Graph information to retain more original information.
  • the five fusion units of the pyramid layer P1 in the first column are respectively the first fusion unit, the second fusion unit, the third fusion unit, the fourth fusion unit and the fifth fusion unit whose scale increases from bottom to top
  • the hierarchical feature maps of three scales C5, C4, and C3 are respectively input to the first fusion unit, the second fusion unit, and the third fusion unit; the first fusion unit, the second fusion unit
  • the third fusion unit is connected with the fifth fusion path, that is, the feature map information is transmitted by upsampling, and the third fusion unit, the fourth fusion unit, and the fifth fusion unit are connected through the first fusion path , that is, the feature map information is transmitted by downsampling, so that the feature map information can be further fused.
  • the prediction network includes two branch networks, which are used for classification and regression tasks respectively. After the branch network processes the fusion feature map, the predicted target value is obtained, and finally the loss function is updated according to the predicted target value, and the surgical action is adjusted according to the updated loss function.
  • the model parameters of the model are identified, and the process of adjusting the model parameters is a prior art, which will not be repeated here.
  • step S40 the formula of the loss function is as follows:
  • L cls is the Focal loss function
  • L reg is the IOU loss function
  • N pos represents the number of positive samples
  • ⁇ is the balance weight of L reg and the value is 1
  • Pointing (x, y) corresponds to the category of ground-truth
  • P x, y pointing (x, y) corresponds to the predicted value category
  • Pointing (x, y) corresponds to the target box of ground-truth
  • t x, y pointing (x, y) corresponds to the target box of the predicted value
  • is an exponential function when The time value is 1, The exponential function is 0 when other values are taken.
  • the parameter ⁇ solves the problem of unbalanced positive and negative samples
  • the confidence p t can make the model focus on samples that are difficult to classify, thus solving the problem of unbalanced sample categories.
  • the training method of the surgical action recognition model disclosed in the first embodiment fully fuses the high-level semantic information and the low-level semantic information through the improved pyramid feature aggregation network, and the obtained fusion feature map can more accurately predict the operation category and the position of the border, and solve the problem of It solves the problem that the video features of surgical actions are not obvious.
  • the second embodiment also discloses a training device for a surgical action recognition model.
  • the training device includes a first input unit 100 , a second input unit 200 , a third input unit 300 and a model training unit 400 .
  • the first input unit 100 is used to input the obtained original surgical action image to the backbone network to obtain several hierarchical feature maps of different scales;
  • the second input unit 200 is used to input the hierarchical feature map to The pyramid feature aggregation network sequentially passes through the fusion processing of the feature map collection module and the feature map divergence module to obtain a number of fusion feature maps of different scales;
  • the third input unit 300 is used to combine the fusion feature maps of several different scales Input to the prediction network to obtain the predicted target value;
  • the model training unit 400 is used to update the loss function according to the predicted target value and the acquired real target value, and adjust the model parameters of the surgical action recognition model according to the updated loss function.
  • the third embodiment also discloses a computer-readable storage medium, the computer-readable storage medium stores a training program of the surgical action recognition model, and the training program of the surgical action recognition model is implemented when the processor executes The training method of the above-mentioned surgical action recognition model.
  • the present application also discloses a computer device.
  • the computer device includes a processor 20 , an internal bus 30 , a network interface 40 , and a computer-readable storage medium 50 .
  • the processor 20 reads the corresponding computer program from the computer-readable storage medium and executes it, forming a request processing device on a logical level.
  • one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each A logic unit, which can also be a hardware or logic device.
  • the computer-readable storage medium 50 stores a training program of the surgical action recognition model, and when the training program of the surgical action recognition model is executed by the processor, the above-mentioned training method of the surgical action recognition model is realized.
  • Computer-readable storage media includes both volatile and non-permanent, removable and non-removable media by any method or technology for storage of information.
  • Information may be computer readable instructions, data structures, modules of a program, or other data.
  • Examples of computer readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage , magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by computing devices.
  • PRAM phase change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically erasable

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种手术动作识别模型的训练方法、存储介质和设备。手术动作识别模型包括主干网络、金字塔特征聚合网络和预测网络，金字塔特征聚合网络包括特征图汇集模块和特征图发散模块，训练方法包括：将获取到的原始手术动作图像输入到主干网络，得到若干不同尺度的层次化特征图（S10）；将层次化特征图输入到金字塔特征聚合网络，依次经过特征图汇集模块和特征图发散模块的融合处理，得到若干不同尺度的融合特征图（S20）；将若干不同尺度的融合特征图输入到预测网络，得到预测目标值（S30）；根据预测目标值和获取到的真实目标值更新损失函数，并调整手术动作识别模型的模型参数（S40）。该方法充分利用空间信息，融合更多尺度特征，训练得到高精度的预测模型。

Description

手术动作识别模型的训练方法、介质和设备 技术领域
本发明属于图像处理技术领域，具体地讲，涉及一种手术动作识别模型的训练方法、计算机可读存储介质、计算机设备。
背景技术
外科手术机器人系统是一种可以辅助外科医生完成手术的智能计算机辅助系统。在微创外科手术中，根据图像算法做出的处理结果使辅助手术机器人做出相应的手术操作动作，协助主治外科医生共同完成手术操作。手术机器人系统不仅具有微创手术创伤小、恢复快、患者痛苦程度轻的特点，并且因为引入智能辅助机器人系统将病人的影像数据与实际手术中患者的解剖部位相结合，手术中通过实时跟踪手术器械与手术动作的实时识别，使外科医生更加清晰的了解解剖部位的实时变化，使得微创手术更加安全、稳定、可靠。同时，通过辅助机械臂的介入对手术动作的实时检测，可以在很大程度上代替辅助外科医生的任务，减小手术过程中外科医生的需求量和多名医生相互配合不当产生的误操作。其中，针对手术动作的目标识别任务，是手术辅助机器人系统中最基本也至关重要的技术。基于深度学习的手术动作实时检测，实现手术机器人系统中核心的低层算法，为将来半自主或全自助化手术机器人的研发提供关键技术支撑。
现有的基于深度学习的检测方法，分为两大类型，基于静态的行为检测和基于动态的行为检测。静态方法仅具有空间信息(图像数据)，而没有当前帧的任何时间上下文。动态活动检测方法使用视频数据，该视频数据为视频中的运动提供了时间上下文信息。但上述方法都是应用于自然场景与模拟的手术场景，和在真实场景下的手术动作检测有很大不同。首先，人体的组织器官存在非刚体的形变，两个不同器官之间的边界、形状和颜色差异很小，基于空间信息的方法难以提取到图像中有效的特征信息，造成分类器的精度较差。其次，使用内窥镜相机拍摄的场景非常接近，无法显示完整的器官及其周围环境，因此几乎没有上下文信息。这样基于动态的行文检测方法难以有效利用手术视频上下 帧之间的时间与空间信息，这些方法就难以满足手术动作检测的任务需求。最后，内窥镜在近距离内的运动和方向使器官从不同角度呈现出很大的不同，这些变化性剧烈的情况也会造成传统的目标检测算法失效。
发明内容
(一)本发明所要解决的技术问题
在手术动作检测场景中时间上下文信息较少的情况下，如何充分利用空间信息，融合更多尺度特征，训练得到高精度的预测模型。
(二)本发明所采用的技术方案
一种手术动作识别模型的训练方法，手术动作识别模型包括主干网络、金字塔特征聚合网络和预测网络，其中，所述金字塔特征聚合网络包括特征图汇集模块和特征图发散模块，所述特征图汇集模块的输入单元和所述特征图发散模块的输出单元之间具有跳连融合路径，所述训练方法包括：
将获取到的原始手术动作图像输入到所述主干网络，得到若干不同尺度的层次化特征图；
将所述层次化特征图输入到所述金字塔特征聚合网络，依次经过所述特征图汇集模块和所述特征图发散模块的融合处理，得到若干不同尺度的融合特征图；
将若干不同尺度的融合特征图输入到所述预测网络，得到预测目标值；
根据预测目标值和获取到的真实目标值更新损失函数，并根据更新后的损失函数调整手术动作识别模型的模型参数。
可选择地，所述特征图汇集模块包括融合单元数量递减的第一列金字塔层、第二列金字塔层和第三列金字塔层，所述特征图发散模块包括融合单元数量递增的所述第三列金字塔层、第四列金字塔层和第五列金字塔层，其中，所述第一列金字塔层为所述特征图汇集模块的输入单元，所述第五列金字塔层为所述所述特征图发散模块的输出单元，且所述第一列金字塔层与所述第五列金字塔层的融合单元数量相同，所述第二列金字塔层与所述第四列金字塔层的融合单元数量相同，各个融合单元通过预定融合路径网进行信息传递。
可选择地，所述第一列金字塔层与所述第五列金字塔层均包括五个不同特征尺度的融合单元，所述第二列金字塔层与所述第四列金字塔层均包括三个不同
特征尺度的融合单元，所述第三列金字塔层具有一个融合单元。
可选择地，所述预定融合路径网包括：
第一融合路径，在金字塔层中自下而上由小尺度的融合单元指向大尺度的融合单元；
第二融合路径，用于对角连接两个相邻层之间的融合单元，通过下采样融合相邻层之间的不同尺度特征图信息；
第三融合路径，用于对角连接两个相邻层之间的融合单元，通过上采样融合相邻层之间的不同尺度特征图信息；
第四融合路径，用于水平连接同一层的融合单元，以融合相同尺度的特征图信息；
第五融合路径，在第一列金字塔层中自上而下由大尺度的融合单元指向小尺度的融合单元；
跳连融合路径，用于连接所述第一列金字塔层与所述第五列金字塔层中同一尺度的融合单元。
可选择地，所述主干网络得到的层次化特征图具有三种尺度，所述第一列金字塔层的五个融合单元分别是由下至上尺度递增的第一融合单元、第二融合单元、第三融合单元、第四融合单元和第五融合单元，三种尺度的层次化特征图分别输入到所述第一融合单元、所述第二融合单元和所述第三融合单元；所述第一融合单元、所述第二融合单元和所述第三融合单元通过第五融合路径连接，所述第三融合单元、所述第四融合单元和所述第五融合单元通过第一融合路径连接。
可选择地，所述损失函数的公式如下：
Figure PCTCN2021096244-appb-000001
其中L cls是Focal损失函数，L reg是IOU损失函数，N pos代表正样本的数量，λ是L reg的平衡权重且值是1，
Figure PCTCN2021096244-appb-000002
指对特征图上的所有点(x,y)所对应的损失进行求和，
Figure PCTCN2021096244-appb-000003
指点(x,y)对应ground-truth的类别，P x,y指点(x,y)对应预测值得类别，
Figure PCTCN2021096244-appb-000004
指点(x,y)对应ground-truth的目标框，t x,y指点(x,y)对应预测值的目标框，
Figure PCTCN2021096244-appb-000005
是指数函数，当
Figure PCTCN2021096244-appb-000006
时值为1，
Figure PCTCN2021096244-appb-000007
取其它值时指数函数为0。
本发明还公开了一种计算机可读存储介质，所述计算机可读存储介质存储有手术动作识别模型的训练程序，所述手术动作识别模型的训练程序被处理器执行时实现上述的手术动作识别模型的训练方法。
本发明还公开了一种计算机设备，所述计算机设备包括计算机可读存储介质、处理器和存储在所述计算机可读存储介质中的手术动作识别模型的训练程序，所述手术动作识别模型的训练程序被处理器执行时实现上述的手术动作识别模型的训练方法。
(三)有益效果
本发明公开了一种手术动作识别模型的训练方法，相对于传统的训练方法，具有如下技术效果：
通过改进的金字塔特征聚合网络来充分融合高层语义信息和低层语义信息，得到的融合特征图可以更加精确地预测手术类别和边框的位置，解决了手术动作视频特征不明显的问题。
附图说明
图1为本发明的实施例一的手术动作识别模型的训练方法的流程图；
图2为本发明的实施例一的手术动作识别模型的训练测模型的框架图；
图3为本发明的实施例一的金字塔特征聚合网络的结构示意图；
图4为本发明的实施例二的手术动作识别模型的训练装置的结构示意图；
图5为本发明的实施例的计算机设备原理框图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
在详细描述本申请的各个实施例之前，首先简单描述本申请的技术构思：现有的基于深度学习的检测方法，需要依赖充分的上下文信息，而在真实手术场景下，由于相机拍摄场景非常接近，很难提取到有效的上下文信息，无法提高分类精度。本申请提供了一种手术动作识别模型的训练方法，首先通过主干网络提取到不同尺度的层次化特征图，接着利用金字塔特征聚合网络对层次化特征图进行融合处理，对不同尺度的特征图信息进行充分融合，得到不同尺度的融合特征图，最后利用预测网络进行预测以及利用更新后的损失函数调整手术动作识别模型的模型参数，该训练方法能充分利用视频中的空间信息，改善现有模型融合多尺度空间信息的能力，进而提高模型的识别精度和检测速度。
具体地，如图1和图2所示，本实施例一的手术动作识别模型的包括主干网络、金字塔特征聚合网络和预测网络，其中，金字塔特征聚合网络包括特征图汇集模块和特征图发散模块，特征图汇集模块的输入单元和特征图发散模块的输出单元之间具有跳连融合路径，手术动作识别模型的训练方法包括如下步骤：
步骤S10：将获取到的原始手术动作图像输入到所述主干网络，得到若干不同尺度的层次化特征图；
步骤S20：将所述层次化特征图输入到所述金字塔特征聚合网络，依次经过所述特征图汇集模块和所述特征图发散模块的融合处理，得到若干不同尺度的融合特征图；
步骤S30：将若干不同尺度的融合特征图输入到所述预测网络，得到预测目标值；
步骤S40：根据预测目标值和获取的真实目标值更新损失函数，并根据更新后的损失函数调整手术动作识别模型的模型参数。
示例性地，在步骤S10中，主干网络对原始手术动作图像进行处理，得到C3、C4、C5三个尺度的层次化特征图，接着在步骤S20中，将相应尺度的层次化特征图输入到相应尺度的融合单元中，进行特征图信息的融合。
具体地，如图3所示，特征图汇集模块包括融合单元数量递减的第一列金字塔层P1、第二列金字塔层P2和第三列金字塔层P3，特征图发散模块包括融合单元数量递增的所述第三列金字塔层P3、第四列金字塔层P4和第五列金字塔层P5，其中，所述第一列金字塔层为所述特征图汇集模块的输入单元，所述第 五列金字塔层为所述所述特征图发散模块的输出单元，且所述第一列金字塔层与所述第五列金字塔层的融合单元数量相同，所述第二列金字塔层与所述第四列金字塔层的融合单元数量相同，各个融合单元通过预定融合路径网进行信息传递。整个金字塔特征聚合网络呈蝴蝶状，通过各个融合单元对不同尺度的特征图信息进行充分融合。
示例性地，所述第一列金字塔层与所述第五列金字塔层均包括五个不同特征尺度的融合单元，所述第二列金字塔层与所述第四列金字塔层均包括三个不同特征尺度的融合单元，所述第三列金字塔层具有一个融合单元。需要说明的是，位于同一行的融合单元的尺度相同，又称为同一层的融合单元，同一列金字塔层的各个融合单元的尺度由上至下递减。
进一步地，如图2所示，预定融合路径网包括第一融合路径11、第二融合路径12、第三融合路径13、第四融合路径14、第五融合路径15和跳连融合路径16。其中，第一融合路径11在金字塔层中自下而上由小尺度的融合单元指向大尺度的融合单元；第二融合路径12用于对角连接两个相邻层之间的融合单元，通过下采样融合相邻层之间的不同尺度特征图信息；第三融合路径13用于对角连接两个相邻层之间的融合单元，通过上采样融合相邻层之间的不同尺度特征图信息；第四融合路径14用于水平连接同一层的融合单元，以融合相同尺度的特征图信息；第五融合路径15在第一列金字塔层中自上而下由大尺度的融合单元指向小尺度的融合单元；跳连融合路径16用于连接所述第一列金字塔层与所述第五列金字塔层中同一尺度的融合单元，即用于融合同层输入单元与输出单元之间的特征图信息，以保留更多原始信息。
示例性地，所述第一列金字塔层P1的五个融合单元分别是由下至上尺度递增的第一融合单元、第二融合单元、第三融合单元、第四融合单元和第五融合单元，三种尺度C5、C4、C3的层次化特征图分别输入到所述第一融合单元、所述第二融合单元和所述第三融合单元；所述第一融合单元、所述第二融合单元和所述第三融合单元通过第五融合路径连接，即通过上采样方式传递特征图信息，所述第三融合单元、所述第四融合单元和所述第五融合单元通过第一融合路径连接，即通过下采样方式传递特征图信息，这样可以进一步融合特征图信息。
经过金字塔特征聚合模块充分利用特征图多尺度信息的融合，经过特征图信息的初始层汇集、输出层发散，利用输入层与输出层的跳连保留特征图的原 始信息，得到信息更加丰富的融合特征图。预测网络包括两条分支网络，分别用于分类与回归任务，分支网络对融合特征图进行处理之后，得到预测目标值，最后根据预测目标值更新损失函数，并根据更新后的损失函数调整手术动作识别模型的模型参数，其中调整模型参数的过程为现有技术，在此不进行赘述。
示例性地，在步骤S40中，损失函数的公式如下：
Figure PCTCN2021096244-appb-000008
其中L cls是Focal损失函数，L reg是IOU损失函数，N pos代表正样本的数量，λ是L reg的平衡权重且值是1，
Figure PCTCN2021096244-appb-000009
指对特征图上的所有点(x,y)所对应的损失进行求和，
Figure PCTCN2021096244-appb-000010
指点(x,y)对应ground-truth的类别，P x,y指点(x,y)对应预测值得类别，
Figure PCTCN2021096244-appb-000011
指点(x,y)对应ground-truth的目标框，t x,y指点(x,y)对应预测值的目标框，
Figure PCTCN2021096244-appb-000012
是指数函数，当
Figure PCTCN2021096244-appb-000013
时值为1，
Figure PCTCN2021096244-appb-000014
取其它值时指数函数为0。
上式中Focal损失函数的一般形式是：
L cls(p t)＝-α t(1-p t) γlog(p t)
其中，参数α解决正负样本不平衡的问题，置信度p t能够使模型主要关注难分类的样本，这样就解决了样本类别不均衡的问题。
本实施例一公开的手术动作识别模型的训练方法，通过改进的金字塔特征聚合网络来充分融合高层语义信息和低层语义信息，得到的融合特征图可以更加精确地预测手术类别和边框的位置，解决了手术动作视频特征不明显的问题。
本实施例二还公开了一种手术动作识别模型的训练装置，训练装置包括第一输入单元100、第二输入单元200、第三输入单元300和模型训练单元400。其中，第一输入单元100用于将获取到的原始手术动作图像输入到所述主干网络，得到若干不同尺度的层次化特征图；第二输入单元200用于将所述层次化特征图输入到所述金字塔特征聚合网络，依次经过所述特征图汇集模块和所述特征图发散模块的融合处理，得到若干不同尺度的融合特征图；第三输入单元300用于将若干不同尺度的融合特征图输入到所述预测网络，得到预测目标值；模型训练单元400用于根据预测目标值和获取到的真实目标值更新损失函数，并根据更新后的损失函数调整手术动作识别模型的模型参数。
进一步地，本实施例三还公开了一种计算机可读存储介质，所述计算机可读存储介质存储有手术动作识别模型的训练程序，所述手术动作识别模型的训练程序被处理器执行时实现上述的手术动作识别模型的训练方法。
进一步地，本申请还公开了一种计算机设备，在硬件层面，如图6所示，该计算机设备包括处理器20、内部总线30、网络接口40、计算机可读存储介质50。处理器20从计算机可读存储介质中读取对应的计算机程序然后运行，在逻辑层面上形成请求处理装置。当然，除了软件实现方式之外，本说明书一个或多个实施例并不排除其他实现方式，比如逻辑器件抑或软硬件结合的方式等等，也就是说以下处理流程的执行主体并不限定于各个逻辑单元，也可以是硬件或逻辑器件。所述计算机可读存储介质50上存储有手术动作识别模型的训练程序，所述手术动作识别模型的训练程序被处理器执行时实现上述的手术动作识别模型的训练方法。
计算机可读存储介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机可读存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带、磁盘存储、量子存储器、基于石墨烯的存储介质或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。
上面对本发明的具体实施方式进行了详细描述，虽然已表示和描述了一些实施例，但本领域技术人员应该理解，在不脱离由权利要求及其等同物限定其范围的本发明的原理和精神的情况下，可以对这些实施例进行修改和完善，这些修改和完善也应在本发明的保护范围内。

Claims (13)

  1. 一种手术动作识别模型的训练方法，其中，手术动作识别模型包括主干网络、金字塔特征聚合网络和预测网络，其中，所述金字塔特征聚合网络包括特征图汇集模块和特征图发散模块，所述特征图汇集模块的输入单元和所述特征图发散模块的输出单元之间具有跳连融合路径，所述训练方法包括：
    将获取到的原始手术动作图像输入到所述主干网络，得到若干不同尺度的层次化特征图；
    将所述层次化特征图输入到所述金字塔特征聚合网络，依次经过所述特征图汇集模块和所述特征图发散模块的融合处理，得到若干不同尺度的融合特征图；
    将若干不同尺度的融合特征图输入到所述预测网络，得到预测目标值；
    根据预测目标值和获取到的真实目标值更新损失函数，并根据更新后的损失函数调整手术动作识别模型的模型参数。
  2. 根据权利要求1所述的手术动作识别模型的训练方法，其中，所述特征图汇集模块包括融合单元数量递减的第一列金字塔层、第二列金字塔层和第三列金字塔层，所述特征图发散模块包括融合单元数量递增的所述第三列金字塔层、第四列金字塔层和第五列金字塔层，其中，所述第一列金字塔层为所述特征图汇集模块的输入单元，所述第五列金字塔层为所述所述特征图发散模块的输出单元，且所述第一列金字塔层与所述第五列金字塔层的融合单元数量相同，所述第二列金字塔层与所述第四列金字塔层的融合单元数量相同，各个融合单元通过预定融合路径网进行信息传递。
  3. 根据权利要求2所述的手术动作识别模型的训练方法，其中，所述第一列金字塔层与所述第五列金字塔层均包括五个不同特征尺度的融合单元，所述第二列金字塔层与所述第四列金字塔层均包括三个不同特征尺度的融合单元，所述第三列金字塔层具有一个融合单元。
  4. 根据权利要求3所述的手术动作识别模型的训练方法，其中，所述预定融合路径网包括：
    第一融合路径，在金字塔层中自下而上由小尺度的融合单元指向大尺度的融合单元；
    第二融合路径，用于对角连接两个相邻层之间的融合单元，通过下采样融 合相邻层之间的不同尺度特征图信息；
    第三融合路径，用于对角连接两个相邻层之间的融合单元，通过上采样融合相邻层之间的不同尺度特征图信息；
    第四融合路径，用于水平连接同一层的融合单元，以融合相同尺度的特征图信息；
    第五融合路径，在第一列金字塔层中自上而下由大尺度的融合单元指向小尺度的融合单元；
    跳连融合路径，用于连接所述第一列金字塔层与所述第五列金字塔层中同一尺度的融合单元。
  5. 根据权利要求4所述的手术动作识别模型的训练方法，其中，所述主干网络得到的层次化特征图具有三种尺度，所述第一列金字塔层的五个融合单元分别是由下至上尺度递增的第一融合单元、第二融合单元、第三融合单元、第四融合单元和第五融合单元，三种尺度的层次化特征图分别输入到所述第一融合单元、所述第二融合单元和所述第三融合单元；所述第一融合单元、所述第二融合单元和所述第三融合单元通过第五融合路径连接，所述第三融合单元、所述第四融合单元和所述第五融合单元通过第一融合路径连接。
  6. 根据权利要求4所述的手术动作识别模型的训练方法，其中，所述损失函数的公式如下：
    Figure PCTCN2021096244-appb-100001
    其中L cls是Focal损失函数，L reg是IOU损失函数，N pos代表正样本的数量，λ是L reg的平衡权重且值是1，
    Figure PCTCN2021096244-appb-100002
    指对特征图上的所有点(x,y)所对应的损失进行求和，
    Figure PCTCN2021096244-appb-100003
    指点(x,y)对应ground-truth的类别，P x,y指点(x,y)对应预测值得类别，
    Figure PCTCN2021096244-appb-100004
    指点(x,y)对应ground-truth的目标框，t x,y指点(x,y)对应预测值的目标框，
    Figure PCTCN2021096244-appb-100005
    是指数函数，当
    Figure PCTCN2021096244-appb-100006
    时值为1，
    Figure PCTCN2021096244-appb-100007
    取其它值时指数函数为0。
  7. 一种计算机可读存储介质，其中，所述计算机可读存储介质存储有手术动作识别模型的训练程序，所述手术动作识别模型的训练程序被处理器执行时实现权利要求1所述的手术动作识别模型的训练方法。
  8. 根据权利要求7所述的计算机可读存储介质，其中，所述特征图汇集模块包括融合单元数量递减的第一列金字塔层、第二列金字塔层和第三列金字塔层，所述特征图发散模块包括融合单元数量递增的所述第三列金字塔层、第四列金字塔层和第五列金字塔层，其中，所述第一列金字塔层为所述特征图汇集模块的输入单元，所述第五列金字塔层为所述所述特征图发散模块的输出单元，且所述第一列金字塔层与所述第五列金字塔层的融合单元数量相同，所述第二列金字塔层与所述第四列金字塔层的融合单元数量相同，各个融合单元通过预定融合路径网进行信息传递。
  9. 根据权利要求8所述的计算机可读存储介质，其中，所述第一列金字塔层与所述第五列金字塔层均包括五个不同特征尺度的融合单元，所述第二列金字塔层与所述第四列金字塔层均包括三个不同特征尺度的融合单元，所述第三列金字塔层具有一个融合单元。
  10. 根据权利要求9所述的计算机可读存储介质，其中，所述预定融合路径网包括：
    第一融合路径，在金字塔层中自下而上由小尺度的融合单元指向大尺度的融合单元；
    第二融合路径，用于对角连接两个相邻层之间的融合单元，通过下采样融合相邻层之间的不同尺度特征图信息；
    第三融合路径，用于对角连接两个相邻层之间的融合单元，通过上采样融合相邻层之间的不同尺度特征图信息；
    第四融合路径，用于水平连接同一层的融合单元，以融合相同尺度的特征图信息；
    第五融合路径，在第一列金字塔层中自上而下由大尺度的融合单元指向小尺度的融合单元；
    跳连融合路径，用于连接所述第一列金字塔层与所述第五列金字塔层中同一尺度的融合单元。
  11. 根据权利要求10所述的计算机可读存储介质，其中，所述主干网络得到的层次化特征图具有三种尺度，所述第一列金字塔层的五个融合单元分别是由下至上尺度递增的第一融合单元、第二融合单元、第三融合单元、第四融合单元和第五融合单元，三种尺度的层次化特征图分别输入到所述第一融合单元、 所述第二融合单元和所述第三融合单元；所述第一融合单元、所述第二融合单元和所述第三融合单元通过第五融合路径连接，所述第三融合单元、所述第四融合单元和所述第五融合单元通过第一融合路径连接。
  12. 根据权利要求10所述的计算机可读存储介质，其中，所述损失函数的公式如下：
    Figure PCTCN2021096244-appb-100008
    其中L cls是Focal损失函数，L reg是IOU损失函数，N pos代表正样本的数量，λ是L reg的平衡权重且值是1，
    Figure PCTCN2021096244-appb-100009
    指对特征图上的所有点(x,y)所对应的损失进行求和，
    Figure PCTCN2021096244-appb-100010
    指点(x,y)对应ground-truth的类别，P x,y指点(x,y)对应预测值得类别，
    Figure PCTCN2021096244-appb-100011
    指点(x,y)对应ground-truth的目标框，t x,y指点(x,y)对应预测值的目标框，
    Figure PCTCN2021096244-appb-100012
    是指数函数，当
    Figure PCTCN2021096244-appb-100013
    时值为1，
    Figure PCTCN2021096244-appb-100014
    取其它值时指数函数为0。
  13. 一种计算机设备，其中，所述计算机设备包括计算机可读存储介质、处理器和存储在所述计算机可读存储介质中的手术动作识别模型的训练程序，所述手术动作识别模型的训练程序被处理器执行时实现权利要求1的手术动作识别模型的训练方法。
PCT/CN2021/096244 2021-05-24 2021-05-27 手术动作识别模型的训练方法、介质和设备 Ceased WO2022246720A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110565266.0 2021-05-24
CN202110565266.0A CN113705320A (zh) 2021-05-24 2021-05-24 手术动作识别模型的训练方法、介质和设备

Publications (1)

Publication Number Publication Date
WO2022246720A1 true WO2022246720A1 (zh) 2022-12-01

Family

ID=78648021

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/096244 Ceased WO2022246720A1 (zh) 2021-05-24 2021-05-27 手术动作识别模型的训练方法、介质和设备

Country Status (2)

Country Link
CN (1) CN113705320A (zh)
WO (1) WO2022246720A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116935212A (zh) * 2023-06-13 2023-10-24 城云科技（中国）有限公司 应用于树木的目标检测模型构建方法、装置及应用

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118535668B (zh) * 2024-06-17 2025-06-03 浙江祉数科技有限公司 基于信息识别的园林植物地理信息查询系统及方法
CN120126050A (zh) * 2025-02-21 2025-06-10 中国医学科学院北京协和医院 一种用于手术打结动作视频识别的改进的yolov8n模型训练方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109472298A (zh) * 2018-10-19 2019-03-15 天津大学 用于小尺度目标检测的深度双向特征金字塔增强网络
CN110766632A (zh) * 2019-10-22 2020-02-07 广东启迪图卫科技股份有限公司 基于通道注意力机制和特征金字塔的图像去噪方法
CN111291739A (zh) * 2020-05-09 2020-06-16 腾讯科技（深圳）有限公司 面部检测、图像检测神经网络训练方法、装置和设备
CN111401517A (zh) * 2020-02-21 2020-07-10 华为技术有限公司 一种感知网络结构搜索方法及其装置
WO2020221990A1 (en) * 2019-04-30 2020-11-05 Huawei Technologies Co., Ltd. Facial localisation in images

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109754403A (zh) * 2018-11-29 2019-05-14 中国科学院深圳先进技术研究院 一种ct图像内的肿瘤自动分割方法及系统
CN112614571B (zh) * 2020-12-24 2023-08-18 中国科学院深圳先进技术研究院 神经网络模型的训练方法、装置、图像分类方法和介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109472298A (zh) * 2018-10-19 2019-03-15 天津大学 用于小尺度目标检测的深度双向特征金字塔增强网络
WO2020221990A1 (en) * 2019-04-30 2020-11-05 Huawei Technologies Co., Ltd. Facial localisation in images
CN110766632A (zh) * 2019-10-22 2020-02-07 广东启迪图卫科技股份有限公司 基于通道注意力机制和特征金字塔的图像去噪方法
CN111401517A (zh) * 2020-02-21 2020-07-10 华为技术有限公司 一种感知网络结构搜索方法及其装置
CN111291739A (zh) * 2020-05-09 2020-06-16 腾讯科技（深圳）有限公司 面部检测、图像检测神经网络训练方法、装置和设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116935212A (zh) * 2023-06-13 2023-10-24 城云科技（中国）有限公司 应用于树木的目标检测模型构建方法、装置及应用

Also Published As

Publication number Publication date
CN113705320A (zh) 2021-11-26

Similar Documents

Publication Publication Date Title
US12131580B2 (en) Face detection method, apparatus, and device, and training method, apparatus, and device for image detection neural network
WO2021077295A1 (zh) 三维人体模型重建方法及存储设备、控制设备
CN113222064A (zh) 一种图像目标对象实时检测方法、系统、终端及存储介质
CN112614571B (zh) 神经网络模型的训练方法、装置、图像分类方法和介质
CN116596952B (zh) 一种多层级病灶检出优化的病理切片图像分割检测方法
CN112037263A (zh) 基于卷积神经网络和长短期记忆网络的手术工具跟踪系统
CN112734769B (zh) 基于交互信息引导的深度学习方法的医学图像分割和定量分析方法、计算机设备、存储介质
CN119205802A (zh) 一种眼动引导混合数据增强的半监督医学图像分割方法
CN113705320A (zh) 手术动作识别模型的训练方法、介质和设备
CN112464860A (zh) 手势识别方法、装置、计算机设备及存储介质
Wang et al. Real-time surgical environment enhancement for robot-assisted minimally invasive surgery based on super-resolution
Liu et al. Towards surgical tools detection and operative skill assessment based on deep learning
CN108564582A (zh) 一种基于深度神经网络的mri脑肿瘤自动识别方法
Jayachandran et al. Multi-dimensional cascades neural network models for the segmentation of retinal vessels in colour fundus images
CN121392536B (zh) 一种用于医学影像分割的医学大模型知识蒸馏方法及系统
Wang et al. Diy your easynas for vision: Convolution operation merging, map channel reducing, and search space to supernet conversion tooling
Liu et al. InstrumentNet: An integrated model for real-time segmentation of intracranial surgical instruments
CN119399096B (zh) 基于超声图像的胎儿解剖结构小样本检测方法与装置
WO2022222519A1 (zh) 故障图像生成方法与装置
CN117611910B (zh) 基于食管内镜图像的自监督识别方法、系统、设备及介质
Xiao et al. A novel local-global spatial attention network for cortical cataract classification in AS-OCT
CN119206717A (zh) 一种用于血细胞图像的目标检测方法、装置、介质和设备
CN119832591A (zh) 基于机器学习的浮游动物智能识别方法和系统
Wilms et al. Segmenting Medical Instruments in Minimally Invasive Surgeries using AttentionMask
CN117831107A (zh) 一种基于高低分辨率融合模型的面部关键点检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21942300

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21942300

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27/05/2024)

122 Ep: pct application non-entry in european phase

Ref document number: 21942300

Country of ref document: EP

Kind code of ref document: A1