Burhan, Asmat and Purwono, Purwono (2024) A Comprehensive Review of Knowledge Distillation for Lightweight Medical Image Segmentation. Journal of Advanced Health Informatics Research, 2 (2): 4. pp. 95-101. ISSN 2985-6124
ID+294+JAHIR+95-101+2024.pdf - Published Version
Available under License Creative Commons Attribution Share Alike.
Download (923kB)
Abstract
Medical image segmentation plays a crucial role in computer-aided diagnosis by enabling precise identification of anatomical and pathological structures. While deep learning models have significantly improved segmentation accuracy, their high computational complexity limits deployment in resource-constrained environments, such as mobile healthcare and edge computing. Knowledge Distillation (KD) has emerged as an effective model compression technique, allowing a lightweight student model to inherit knowledge from a complex teacher model while maintaining high segmentation performance. This review systematically examines key KD techniques, including Response-Based, Feature-Based, and Relation-Based Distillation, and analyzes their advantages and limitations. Major challenges in KD, such as boundary preservation, domain generalization, and computational trade-offs, are explored in the context of lightweight model development. Additionally, emerging trends, including the integration of KD with Transformers, Federated Learning, and Self-Supervised Learning, are discussed to highlight future directions in efficient medical image segmentation. By providing a comprehensive analysis of KD for lightweight segmentation models, this review aims to guide the development of deep learning solutions that balance accuracy, efficiency, and real-world applicability in medical imaging
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | Knowledge Distillation, Medical Image Segmentation, Model Compression, Lightweight Deep Learning, Comprehensive Review |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science |
| Depositing User: | Dr. Purwono , |
| Date Deposited: | 15 Apr 2026 04:04 |
| Last Modified: | 15 Apr 2026 04:04 |
| URI: | https://alxiv.org/id/eprint/12 |
