Welcome to ALXIV Repository

Advancing Open Scholarly Communication

A Comprehensive Review of Knowledge Distillation for Lightweight Medical Image Segmentation

Burhan, Asmat and Purwono, Purwono (2024) A Comprehensive Review of Knowledge Distillation for Lightweight Medical Image Segmentation. Journal of Advanced Health Informatics Research, 2 (2): 4. pp. 95-101. ISSN 2985-6124

[thumbnail of ID+294+JAHIR+95-101+2024.pdf] Text
ID+294+JAHIR+95-101+2024.pdf - Published Version
Available under License Creative Commons Attribution Share Alike.

Download (923kB)

Abstract

Medical image segmentation plays a crucial role in computer-aided diagnosis by enabling precise identification of anatomical and pathological structures. While deep learning models have significantly improved segmentation accuracy, their high computational complexity limits deployment in resource-constrained environments, such as mobile healthcare and edge computing. Knowledge Distillation (KD) has emerged as an effective model compression technique, allowing a lightweight student model to inherit knowledge from a complex teacher model while maintaining high segmentation performance. This review systematically examines key KD techniques, including Response-Based, Feature-Based, and Relation-Based Distillation, and analyzes their advantages and limitations. Major challenges in KD, such as boundary preservation, domain generalization, and computational trade-offs, are explored in the context of lightweight model development. Additionally, emerging trends, including the integration of KD with Transformers, Federated Learning, and Self-Supervised Learning, are discussed to highlight future directions in efficient medical image segmentation. By providing a comprehensive analysis of KD for lightweight segmentation models, this review aims to guide the development of deep learning solutions that balance accuracy, efficiency, and real-world applicability in medical imaging

Item Type: Article
Uncontrolled Keywords: Knowledge Distillation, Medical Image Segmentation, Model Compression, Lightweight Deep Learning, Comprehensive Review
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science
Depositing User: Dr. Purwono ,
Date Deposited: 15 Apr 2026 04:04
Last Modified: 15 Apr 2026 04:04
URI: https://alxiv.org/id/eprint/12

Actions (login required)

View Item
View Item