report.org (19307B)
1 #+title: Diabetic Retinopathy Classification -- Final Report 2 #+subtitle: EEL4759: Digital Image Processing 3 #+author: Vineet Kumar 4 #+options: toc:nil 5 #+cite_export: biblatex 6 #+bibliography: references.bib 7 #+latex_class_options: [12pt,letterpage] 8 #+latex_header: \usepackage[margin=1in]{geometry} 9 #+latex_header: \usepackage{booktabs} 10 #+latex_header: \usepackage{float} 11 #+latex_header: \usepackage{graphicx} 12 13 * Abstract 14 15 This report covers automated severity grading of diabetic retinopathy 16 (DR) from retinal images using deep convolutional neural 17 networks. Three pretrained architectures (ResNet-50, EfficientNet-B0, 18 and ViT-B/16) were fine-tuned on the Kaggle "Diabetic Retinopathy 19 Resized Arranged" dataset and evaluated on a five-class severity 20 classification task (Healthy through Proliferative DR). Each model was 21 trained with and without CLAHE (Contrast Limited Adaptive Histogram 22 Equalization) preprocessing to assess its effect on classification 23 performance. Class imbalance was addressed via sqrt-inverse-frequency 24 weighted sampling and Focal Loss. At 384\times{}384 pixel resolution, 25 ResNet-50 and EfficientNet-B0 achieved 0.80 overall accuracy and 26 weighted F1-scores of 0.77 and 0.78 respectively. ViT-B/16 was 27 constrained to 224\times{}224 and achieved 0.69 accuracy. CLAHE 28 improved detection of minority classes (particularly Mild NPDR) at a 29 small cost to overall accuracy. Mild NPDR remained the hardest class 30 to classify across all experiments, with F1-scores no higher than 0.17 31 due to its visual similarity to healthy retinal images. 32 33 * Introduction 34 35 ** Diabetic Retinopathy 36 37 Diabetic retinopathy is a progressive eye disease caused by diabetes 38 that is one of the leading causes of preventable vision loss 39 worldwide. It develops when high blood sugar damages the retinal blood 40 vessels, causing leakage, abnormal blood vessel growth, and eventually 41 retinal detachment if untreated. The disease is classified into five 42 severity grades: no retinopathy (Healthy), Mild Non-Proliferative DR 43 (NPDR), Moderate NPDR, Severe NPDR, and Proliferative DR. Early 44 detection through systematic retinal screening can prevent up to 90% 45 of severe vision loss cases. Manual grading by trained specialists is 46 accurate but expensive and slow, making automated image classification 47 useful for large-scale screening programs. 48 49 ** Convolutional Neural Networks for Retinal Image Classification 50 51 Convolutional neural networks (CNNs) have become the dominant approach 52 for medical image classification tasks. Rather than hand-crafting 53 features, CNNs learn hierarchical representations (low-level edges and 54 textures in early layers, progressively more abstract features in 55 deeper layers) directly from training data. For retinal imaging, 56 pretrained ImageNet models (transfer learning) are effective: the 57 low-level filters learned on natural images transfer well to retinal 58 images, and fine-tuning on DR data allows the network to specialize to 59 lesion-specific features such as microaneurysms, hard exudates, and 60 hemorrhages that distinguish severity grades. This project compares 61 three architectures spanning different design philosophies: ResNet-50 62 (residual connections), EfficientNet-B0 (compound scaling), and 63 ViT-B/16 (pure self-attention on image patches). 64 65 ** Existing Model Performance 66 67 The Kaggle Diabetic Retinopathy detection competitions have 68 established rough baselines for this type of task. On five-class 69 grading datasets similar to the one used here, top-performing single 70 models typically achieve quadratic weighted kappa scores in the 71 0.82--0.86 range using heavily tuned ensembles, test-time 72 augmentation, and competition-grade preprocessing. More comparable 73 single-model baselines reported on Kaggle notebooks for the "Diabetic 74 Retinopathy Resized Arranged" dataset (224\times{}224) report 75 accuracies of roughly 0.60--0.73 and kappa values around 0.50--0.58 76 for standard fine-tuned CNNs, which is consistent with the results 77 obtained in this project's 224px training run. 78 79 * Methodology 80 81 ** Dataset and Class Distribution 82 83 The dataset used is "Diabetic Retinopathy Resized Arranged" 84 [cite:@amanneo2022dr], containing 35,126 retinal images organized into 85 five class-labeled folders. The images are JPEG files at approximately 86 1024\times{}768 pixel resolution. The dataset was split into 70% 87 training, 15% validation, and 15% test sets using stratified sampling 88 to preserve the class distribution across all splits (scikit-learn's 89 =train_test_split= with =stratify=, seed 42). 90 91 #+begin_export latex 92 \begin{table}[H] 93 \centering 94 \caption{Dataset class distribution (test split, 5,269 images).} 95 \begin{tabular}{clrr} 96 \toprule 97 Class & Grade & Test Count & Test \% \\ 98 \midrule 99 0 & Healthy & 3,872 & 73.5\% \\ 100 1 & Mild NPDR & 366 & 6.9\% \\ 101 2 & Moderate NPDR & 794 & 15.1\% \\ 102 3 & Severe NPDR & 131 & 2.5\% \\ 103 4 & Proliferative DR & 106 & 2.0\% \\ 104 \bottomrule 105 \end{tabular} 106 \end{table} 107 #+end_export 108 109 The dataset is severely imbalanced: the Healthy class accounts for 110 nearly 74% of all samples, while Severe NPDR and Proliferative DR 111 together comprise only 4.5%. 112 113 ** Models 114 115 Three ImageNet-pretrained architectures were fine-tuned for 5-class DR 116 severity classification. In all cases the original classification head 117 was replaced with a linear layer mapping to 5 outputs, and all 118 backbone weights were unfrozen for full fine-tuning. 119 120 - ResNet-50 [cite:@he2016deep]: Deep residual network with skip 121 connections. The final fully connected layer (2048 \to 5) was 122 replaced. Input: 384\times{}384. 123 124 - EfficientNet-B0 [cite:@tan2019efficientnet]: Compound-scaled network 125 optimizing width, depth, and resolution simultaneously. The 126 classifier head (=Dropout(0.2)= + Linear 1280 \to 5) was replaced. 127 Input: 384\times{}384. 128 129 - ViT-B/16 [cite:@dosovitskiy2021image]: Vision Transformer that 130 splits the image into 16\times{}16 pixel patches and processes them 131 with multi-head self-attention. The projection head (768 \to 5) was 132 replaced. Input: 224\times{}224 (PyTorch's ViT-B/16 implementation 133 only supports 224\times{}224 input; higher resolutions are not 134 supported without custom modifications to the model). 135 136 ** Preprocessing and Augmentation 137 138 Images were resized from their native ~1024\times{}768 resolution to 139 the model's target input resolution (384\times{}384 for ResNet-50 and 140 EfficientNet-B0; 224\times{}224 for ViT-B/16) as the first 141 preprocessing step. 142 143 CLAHE (Contrast Limited Adaptive Histogram Equalization) 144 [cite:@zuiderveld1994clahe] was applied as an optional subsequent 145 preprocessing step. The transform converts the image from RGB to LAB 146 color space, applies =cv2.createCLAHE= (clip limit 2.0, tile grid 147 8\times{}8) to the L (lightness) channel only, and converts back to 148 RGB. Operating on the lightness channel alone 149 enhances local contrast in retinal structures such as vessels and 150 lesions without altering hue or saturation. 151 152 The training augmentation pipeline (applied after CLAHE if enabled): 153 154 1. =RandomResizedCrop= to target resolution, scale 0.8--1.0 155 2. =RandomHorizontalFlip= (p=0.5), =RandomVerticalFlip= (p=0.5) 156 3. =RandomRotation=(\pm{}30\textdegree) 157 4. =ColorJitter= (brightness 0.3, contrast 0.3, saturation 0.2, hue 158 0.02) 159 5. =GaussianBlur= (kernel 3, \sigma{} 0.1--1.0) 160 6. Normalize to ImageNet mean/std 161 162 Validation and test images were resized to the target resolution 163 deterministically (no random crop) and normalized identically. 164 165 ** Class Imbalance Handling 166 167 Two complementary mechanisms addressed the severe class imbalance: 168 169 - WeightedRandomSampler: Each training sample was assigned a weight 170 proportional to $1/\sqrt{n_c}$ where $n_c$ is the count of its 171 class. The square-root weighting provides a moderate upsampling of 172 minority classes without completely drowning the majority-class 173 signal (which full inverse-frequency weighting was found to do). 174 175 - Focal Loss [cite:@lin2017focal]: The loss function 176 $\mathrm{FL}(p_t) = -(1-p_t)^\gamma \log(p_t)$ with $\gamma=2$ 177 down-weights easy, well-classified examples and focuses gradient 178 updates on hard misclassifications, which helps with minority classes 179 where the model initially predicts low confidence. 180 181 ** Training Configuration 182 183 #+begin_export latex 184 \begin{table}[H] 185 \centering 186 \caption{Hyperparameters used for all experiments.} 187 \begin{tabular}{ll} 188 \toprule 189 Hyperparameter & Value \\ 190 \midrule 191 Optimizer & AdamW \\ 192 Head learning rate & 1e-4 \\ 193 Backbone learning rate & 1e-5 ($10\times$ lower) \\ 194 Weight decay & 1e-4 \\ 195 Batch size & 256 \\ 196 Max epochs & 50 \\ 197 Early stopping & patience 10, monitor val macro F1 \\ 198 LR warmup & 3 epochs linear ($0.1\times \to 1\times$) \\ 199 LR schedule & Cosine annealing after warmup \\ 200 Gradient clipping & max norm 1.0 \\ 201 Mixed precision & \texttt{torch.amp.autocast} + GradScaler \\ 202 Hardware & $2\times$ AMD Radeon RX 7900 XTX (ROCm) \\ 203 Random seed & 42 \\ 204 \bottomrule 205 \end{tabular} 206 \end{table} 207 #+end_export 208 209 A discriminative learning rate was used: the pretrained backbone was 210 trained at 1e-5 while the newly initialized classification head was 211 trained at the full 1e-4. This prevents the pretrained features from 212 being overwritten too quickly in early epochs. 213 214 ** Evaluation Metrics 215 216 Each trained model was evaluated on the held-out test set using: 217 218 - Accuracy: fraction of correctly classified samples 219 - Weighted F1: F1 averaged over classes weighted by support; reflects 220 overall performance on the imbalanced distribution 221 - Macro F1: F1 averaged equally over all 5 classes; better reflects 222 performance on minority classes 223 - Quadratic-weighted Cohen's Kappa: measures inter-rater agreement 224 beyond chance; the quadratic weighting penalizes predictions further 225 from the true label more heavily, which is appropriate for the 226 ordinal DR severity scale 227 228 * Results 229 230 ** Summary of Test-Set Performance 231 232 #+begin_export latex 233 \begin{table}[H] 234 \centering 235 \caption{Test-set metrics for all six experiments at $384\times384$ (ResNet-50, EfficientNet-B0) and $224\times224$ (ViT-B/16).} 236 \begin{tabular}{llccc} 237 \toprule 238 Model & CLAHE & Accuracy & Weighted F1 & Macro F1 \\ 239 \midrule 240 ResNet-50 & No & 0.80 & 0.77 & 0.51 \\ 241 ResNet-50 & Yes & 0.78 & 0.77 & 0.53 \\ 242 EfficientNet-B0 & No & 0.80 & 0.78 & 0.52 \\ 243 EfficientNet-B0 & Yes & 0.78 & 0.76 & 0.51 \\ 244 ViT-B/16 & No & 0.68 & 0.69 & 0.47 \\ 245 ViT-B/16 & Yes & 0.69 & 0.71 & 0.49 \\ 246 \bottomrule 247 \end{tabular} 248 \end{table} 249 #+end_export 250 251 ** Per-Class F1 Scores 252 253 #+begin_export latex 254 \begin{table}[H] 255 \centering 256 \caption{Per-class F1-score for all six experiments.} 257 \begin{tabular}{lccccc} 258 \toprule 259 Model / CLAHE & Healthy & Mild NPDR & Moderate NPDR & Severe NPDR & Proliferative DR \\ 260 \midrule 261 ResNet-50 / No & 0.90 & 0.09 & 0.57 & 0.50 & 0.52 \\ 262 ResNet-50 / Yes & 0.89 & 0.17 & 0.53 & 0.45 & 0.61 \\ 263 EfficientNet-B0 / No & 0.90 & 0.10 & 0.57 & 0.45 & 0.58 \\ 264 EfficientNet-B0 / Yes & 0.88 & 0.11 & 0.54 & 0.45 & 0.60 \\ 265 ViT-B/16 / No & 0.81 & 0.14 & 0.43 & 0.43 & 0.51 \\ 266 ViT-B/16 / Yes & 0.83 & 0.17 & 0.45 & 0.45 & 0.56 \\ 267 \bottomrule 268 \end{tabular} 269 \end{table} 270 #+end_export 271 272 ** Effect of Input Resolution (224px vs 384px) 273 274 The experiments were run twice: an initial run at 224\times{}224 and a 275 subsequent run at 384\times{}384 for ResNet-50 and 276 EfficientNet-B0. ViT-B/16 was kept at 224\times{}224 in both runs. The 277 resolution increase produced a substantial improvement for the CNN 278 models. 279 280 #+begin_export latex 281 \begin{table}[H] 282 \centering 283 \caption{Comparison of 224px (initial run) vs 384px (final run) test-set results for CNN models. ViT results shown for reference; both runs used 224px.} 284 \resizebox{\linewidth}{!}{% 285 \footnotesize 286 \begin{tabular}{llccccccc} 287 \toprule 288 Model & CLAHE & Acc (224px) & W-F1 (224px) & M-F1 (224px) & Kappa (224px) & Acc (384px) & W-F1 (384px) & M-F1 (384px) \\ 289 \midrule 290 ResNet-50 & No & 0.61 & 0.66 & 0.46 & 0.53 & 0.80 & 0.77 & 0.51 \\ 291 ResNet-50 & Yes & 0.61 & 0.65 & 0.44 & 0.52 & 0.78 & 0.77 & 0.53 \\ 292 EfficientNet-B0 & No & 0.61 & 0.65 & 0.46 & 0.55 & 0.80 & 0.78 & 0.52 \\ 293 EfficientNet-B0 & Yes & 0.62 & 0.66 & 0.45 & 0.54 & 0.78 & 0.76 & 0.51 \\ 294 ViT-B/16 & No & 0.68 & 0.69 & 0.47 & 0.53 & 0.68 & 0.69 & 0.47 \\ 295 ViT-B/16 & Yes & 0.67 & 0.69 & 0.48 & 0.56 & 0.69 & 0.71 & 0.49 \\ 296 \bottomrule 297 \end{tabular}% 298 } 299 \end{table} 300 #+end_export 301 302 Accuracy for ResNet-50 and EfficientNet-B0 improved by approximately 303 19 percentage points (0.61 \to 0.80) with the resolution increase. The 304 improvement is consistent across both CLAHE settings and is attributed 305 to the finer retinal microstructure (microaneurysms, hemorrhage dots, 306 exudates) that is present at 384px but lost at 224px. 307 308 ** Confusion Matrices 309 310 *** ResNet-50 without CLAHE 311 312 #+ATTR_LATEX: :width 0.75\textwidth :placement [H] 313 [[file:results/resnet50_clahe=False/confusion_matrix.png]] 314 315 *** ResNet-50 with CLAHE 316 317 #+ATTR_LATEX: :width 0.75\textwidth :placement [H] 318 [[file:results/resnet50_clahe=True/confusion_matrix.png]] 319 320 *** EfficientNet-B0 without CLAHE 321 322 #+ATTR_LATEX: :width 0.75\textwidth :placement [H] 323 [[file:results/efficientnet_b0_clahe=False/confusion_matrix.png]] 324 325 *** EfficientNet-B0 with CLAHE 326 327 #+ATTR_LATEX: :width 0.75\textwidth :placement [H] 328 [[file:results/efficientnet_b0_clahe=True/confusion_matrix.png]] 329 330 *** ViT-B/16 without CLAHE 331 332 #+ATTR_LATEX: :width 0.75\textwidth :placement [H] 333 [[file:results/vit_b_16_clahe=False/confusion_matrix.png]] 334 335 *** ViT-B/16 with CLAHE 336 337 #+ATTR_LATEX: :width 0.75\textwidth :placement [H] 338 [[file:results/vit_b_16_clahe=True/confusion_matrix.png]] 339 340 ** Training Curves 341 342 *** ResNet-50 343 344 #+ATTR_LATEX: :width \textwidth :placement [H] 345 [[file:results/resnet50_clahe=False/training_curves.png]] 346 347 #+ATTR_LATEX: :width \textwidth :placement [H] 348 [[file:results/resnet50_clahe=True/training_curves.png]] 349 350 *** EfficientNet-B0 351 352 #+ATTR_LATEX: :width \textwidth :placement [H] 353 [[file:results/efficientnet_b0_clahe=False/training_curves.png]] 354 355 #+ATTR_LATEX: :width \textwidth :placement [H] 356 [[file:results/efficientnet_b0_clahe=True/training_curves.png]] 357 358 *** ViT-B/16 359 360 #+ATTR_LATEX: :width \textwidth :placement [H] 361 [[file:results/vit_b_16_clahe=False/training_curves.png]] 362 363 #+ATTR_LATEX: :width \textwidth :placement [H] 364 [[file:results/vit_b_16_clahe=True/training_curves.png]] 365 366 * Discussion 367 368 ** Resolution Is the Dominant Factor for CNN Models 369 370 The largest improvement was the ~19 percentage point accuracy jump for 371 ResNet-50 and EfficientNet-B0 when input resolution was increased from 372 224\times{}224 to 384\times{}384. Retinal images contain small 373 features used for diagnosis (microaneurysms, dot-like red lesions as 374 small as 10--20\mu{}m, dot hemorrhages, and hard exudates) that take 375 up only a few pixels at 224px. At 384px these features are resolved 376 clearly enough for the network to learn discriminative filters for 377 them. The resolution improvement had no effect on ViT-B/16 because 378 PyTorch's ViT-B/16 implementation only supports 224\times{}224 input 379 and cannot be run at higher resolutions without custom modifications 380 to the model. 381 382 ** CLAHE Trades Accuracy for Minority-Class Recall 383 384 Applying CLAHE consistently reduced overall accuracy by approximately 385 2 percentage points for the CNN models (e.g., ResNet-50: 0.80 \to 386 0.78) while improving detection of minority classes. For ResNet-50, 387 CLAHE nearly doubled the Mild NPDR F1-score from 0.09 to 0.17, and 388 improved Proliferative DR F1 from 0.52 to 0.61. This happens because 389 CLAHE enhances the local contrast of subtle lesions, making 390 minority-class features more distinguishable, but the enhanced 391 contrast can also introduce artifacts that disrupt features the model 392 relied on for the majority Healthy class. 393 394 For ViT-B/16, CLAHE produced a small but consistent improvement across 395 most classes, and the Mild NPDR recall improved from 0.17 to 396 0.22. This suggests that CLAHE is more beneficial for ViT, possibly 397 because the transformer's attention mechanism can exploit the enhanced 398 contrast across longer-range spatial dependencies. 399 400 Whether CLAHE is desirable depends on the application: maximizing 401 overall accuracy favors CLAHE-off, while maximizing detection of the 402 higher-risk advanced DR grades favors CLAHE-on. For a clinical 403 screening tool the latter is generally preferable. 404 405 ** Mild NPDR Remains Universally Difficult 406 407 Mild NPDR (class 1) had the lowest F1-score in every experiment, 408 ranging from 0.09 to 0.17. This class is defined by only 409 microaneurysms with no other lesions, a very subtle change from the 410 healthy retina. The confusion matrices confirm that the vast majority 411 of Mild NPDR predictions are misclassified as Healthy. The visual 412 difference is small, the class is heavily underrepresented (6.9% of 413 the dataset), and the class boundaries are subjective even for trained 414 graders. Despite weighted sampling and Focal Loss, the network does 415 not reliably learn to distinguish these cases. A possible solution to 416 this is to train with a classification model that supports higher 417 resolutions such as the modified Hi-ResNet, where I hypothesize that 418 the small lesions in class 1 get obscured or even essentially erased 419 when downscaling the images during the dataset preparation for use as 420 input to the models. 421 422 ** ViT-B/16 Underperforms at 224px 423 424 ViT-B/16 achieved lower accuracy (0.68--0.69) compared to ResNet-50 425 and EfficientNet-B0 (0.78--0.80). This is mainly due to the resolution 426 constraint: ViT-B/16 uses 16\times{}16 pixel patches, so at 224px each 427 patch covers a 16\times{}16 area, which is large enough to subsume 428 entire microaneurysms. The training curves reveal severe overfitting: 429 the training loss approaches zero while validation loss increases 430 after ~10--15 epochs. ViT-B/16 likely requires either a higher input 431 resolution (which demands interpolated positional embeddings and 432 careful fine-tuning), a larger dataset, or stronger regularization to 433 generalize well on medical images. 434 435 * Conclusion 436 437 Pretrained CNN architectures achieved strong diabetic retinopathy 438 grading performance (accuracy ~0.80, weighted F1 ~0.77--0.78) when 439 fine-tuned at sufficient resolution (384\times{}384). Input resolution 440 was the most impactful factor, with a ~19 percentage point accuracy 441 improvement for ResNet-50 and EfficientNet-B0 when resolution was 442 increased from 224 to 384 pixels. CLAHE preprocessing improved 443 minority class detection at a small overall accuracy cost and is 444 recommended for clinical settings where detecting advanced DR grades 445 is the priority. EfficientNet-B0 achieved the best weighted F1 (0.78) 446 without CLAHE, while ResNet-50 with CLAHE achieved the best macro F1 447 (0.53), indicating slightly more balanced performance across 448 classes. ViT-B/16 was constrained by its fixed-resolution patch 449 embedding and underperformed the CNN models. 450 451 Mild NPDR remains the hardest class: its visual distinction from a 452 healthy retina is subtle and the class is severely 453 underrepresented. Future work could address this with: dedicated data 454 augmentation for lesion synthesis, higher-resolution ViT variants 455 (ViT-L with interpolated positional embeddings), pre-training on 456 larger retinal image datasets, or model ensembles. 457 458 * Appendix 459 The repository will be available at 460 [[https://git.vineetk.net/eel4759_finalproject_classification/]]. 461 462 #+print_bibliography: