目的 构建融合实例分割与多目标追踪的人工智能手术流程解析体系,实现手术视频自动化结构化解析与客观质量评估。方法 回顾性分析2022年1月至2023年12月徐州市中心医院胃肠外科、南京大学医学院附属鼓楼医院普外科及苏州大学附属第四医院普外科实施的20例腹腔镜胃癌根治术的手术视频资料,共提取标注 7569 张含 11 类手术器械及术中出血区域的图像,按 8∶2 分层划分训练、验证数据集;基于 YOLO11L-seg 模型开展迁移学习,结合 ByteTrack 算法完成器械时序追踪,从内部静态图像、外部独立动态手术视频、外科医师盲评3个维度验证模型效能,评价指标包含交并比阈值为0.5时的边界框平均精度均值(mAP@0.5)、掩码(Mask)mAP@0.5、准确率、F1/Dice 系数;依托器械时序使用频率构建手术流程热图,量化区分不同术式、定位关键手术步骤。结果 模型内部静态验证边界框mAP@0.5 为 83.5%,掩码mAP@0.5 为 82.8%;外部动态视频测试集整体识别准确率 92.2%,F1/Dice 系数 0.878,术中出血识别准确率达 100%;手术热图可完全区分远端胃切除术、全胃切除术、近端胃切除术 3 种术式,关键操作步骤定位准确率>90%;单例手术视频智能审核时长较人工缩短约70%。3名高年资外科医师盲评显示器械、出血识别及术式判别准确率为100%。结论 基于 YOLO11L-seg 与 ByteTrack 的器械识别追踪方案可精准解析腹腔镜胃癌根治术时序流程,手术热图可直观量化手术操作特征,为微创手术自动化、标准化质量管控提供可行技术方案。
Objective To construct an artificial intelligence surgical workflow reconstruction system integrating instance segmentation and multi-object tracking to realize automatic structural analysis and objective quality evaluation of surgical videos. Methods A total of 20 videos of laparoscopic radical gastrectomy performed between January 2022 and December 2023 at the Department of Gastrointestinal Surgery Xuzhou Central Hospital; the Department of General Surgery, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School; and the Department of General Surgery, the Fourth Affiliated Hospital of Soochow University were retrospectively analyzed. A total of 7569 images annotated with 11 types of surgical instruments and intraoperative bleeding areas were extracted and divided into training and validation datasets at a ratio of 8:2. Transfer learning was performed based on the YOLO11L-seg model, and the ByteTrack algorithm was integrated for temporal tracking of surgical instruments. Model performance was verified from three dimensions: internal static image validation, external independent dynamic surgical video validation, and blind evaluation by clinical surgeons. Evaluation metrics included mAP@0.5, mask mAP@0.5, recognition accuracy, and F1/Dice coefficient. A surgical procedure heatmap was generated based on the temporal frequency of instrument usage to quantitatively distinguish different surgical procedures and locate key operative steps. Results In the internal static validation, the box mAP@0.5 and mask mAP@0.5 of the model reached 83.5% and 82.8%, respectively. In the external dynamic video test set, the overall recognition accuracy was 92.2% with an F1/Dice coefficient of 0.878, and the recognition accuracy of intraoperative bleeding was 100%. The surgical heatmap could completely distinguish three radical gastrectomy procedures including distal, total and proximal gastrectomy, and the localization accuracy of key operative steps exceeded 90%. The intelligent review time of a single surgical video was reduced by approximately 70% compared with manual review. Blind evaluation by three senior surgeons demonstrated an average accuracy of 100% in instrument identification, bleeding detection and surgical procedure classification. Conclusion The instrument recognition and tracking scheme based on YOLO11L-seg and ByteTrack can accurately reconstruct the temporal workflow of laparoscopic radical gastrectomy. The surgical heatmap can intuitively quantify surgical operation characteristics, providing a feasible technical strategy for automatic and standardized quality control of minimally invasive surgery.