add original proposal presentation and move into assignment/HEADmaster

1 files changed, 462 insertions, 0 deletions

diff --git a/assignment/report.org b/assignment/report.org
new file mode 100644
index 0000000..044e888
--- /dev/null
+++ b/assignment/report.org
@@ -0,0 +1,462 @@
+#+title: Diabetic Retinopathy Classification -- Final Report
+#+subtitle: EEL4759: Digital Image Processing
+#+author: Vineet Kumar
+#+options: toc:nil
+#+cite_export: biblatex
+#+bibliography: references.bib
+#+latex_class_options: [12pt,letterpage]
+#+latex_header: \usepackage[margin=1in]{geometry}
+#+latex_header: \usepackage{booktabs}
+#+latex_header: \usepackage{float}
+#+latex_header: \usepackage{graphicx}
+
+* Abstract
+
+This report covers automated severity grading of diabetic retinopathy
+(DR) from retinal images using deep convolutional neural
+networks. Three pretrained architectures (ResNet-50, EfficientNet-B0,
+and ViT-B/16) were fine-tuned on the Kaggle "Diabetic Retinopathy
+Resized Arranged" dataset and evaluated on a five-class severity
+classification task (Healthy through Proliferative DR). Each model was
+trained with and without CLAHE (Contrast Limited Adaptive Histogram
+Equalization) preprocessing to assess its effect on classification
+performance. Class imbalance was addressed via sqrt-inverse-frequency
+weighted sampling and Focal Loss. At 384\times{}384 pixel resolution,
+ResNet-50 and EfficientNet-B0 achieved 0.80 overall accuracy and
+weighted F1-scores of 0.77 and 0.78 respectively. ViT-B/16 was
+constrained to 224\times{}224 and achieved 0.69 accuracy. CLAHE
+improved detection of minority classes (particularly Mild NPDR) at a
+small cost to overall accuracy. Mild NPDR remained the hardest class
+to classify across all experiments, with F1-scores no higher than 0.17
+due to its visual similarity to healthy retinal images.
+
+* Introduction
+
+** Diabetic Retinopathy
+
+Diabetic retinopathy is a progressive eye disease caused by diabetes
+that is one of the leading causes of preventable vision loss
+worldwide. It develops when high blood sugar damages the retinal blood
+vessels, causing leakage, abnormal blood vessel growth, and eventually
+retinal detachment if untreated. The disease is classified into five
+severity grades: no retinopathy (Healthy), Mild Non-Proliferative DR
+(NPDR), Moderate NPDR, Severe NPDR, and Proliferative DR. Early
+detection through systematic retinal screening can prevent up to 90%
+of severe vision loss cases. Manual grading by trained specialists is
+accurate but expensive and slow, making automated image classification
+useful for large-scale screening programs.
+
+** Convolutional Neural Networks for Retinal Image Classification
+
+Convolutional neural networks (CNNs) have become the dominant approach
+for medical image classification tasks. Rather than hand-crafting
+features, CNNs learn hierarchical representations (low-level edges and
+textures in early layers, progressively more abstract features in
+deeper layers) directly from training data. For retinal imaging,
+pretrained ImageNet models (transfer learning) are effective: the
+low-level filters learned on natural images transfer well to retinal
+images, and fine-tuning on DR data allows the network to specialize to
+lesion-specific features such as microaneurysms, hard exudates, and
+hemorrhages that distinguish severity grades.  This project compares
+three architectures spanning different design philosophies: ResNet-50
+(residual connections), EfficientNet-B0 (compound scaling), and
+ViT-B/16 (pure self-attention on image patches).
+
+** Existing Model Performance
+
+The Kaggle Diabetic Retinopathy detection competitions have
+established rough baselines for this type of task. On five-class
+grading datasets similar to the one used here, top-performing single
+models typically achieve quadratic weighted kappa scores in the
+0.82--0.86 range using heavily tuned ensembles, test-time
+augmentation, and competition-grade preprocessing. More comparable
+single-model baselines reported on Kaggle notebooks for the "Diabetic
+Retinopathy Resized Arranged" dataset (224\times{}224) report
+accuracies of roughly 0.60--0.73 and kappa values around 0.50--0.58
+for standard fine-tuned CNNs, which is consistent with the results
+obtained in this project's 224px training run.
+
+* Methodology
+
+** Dataset and Class Distribution
+
+The dataset used is "Diabetic Retinopathy Resized Arranged"
+[cite:@amanneo2022dr], containing 35,126 retinal images organized into
+five class-labeled folders. The images are JPEG files at approximately
+1024\times{}768 pixel resolution. The dataset was split into 70%
+training, 15% validation, and 15% test sets using stratified sampling
+to preserve the class distribution across all splits (scikit-learn's
+=train_test_split= with =stratify=, seed 42).
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Dataset class distribution (test split, 5,269 images).}
+\begin{tabular}{clrr}
+\toprule
+Class & Grade & Test Count & Test \% \\
+\midrule
+0 & Healthy          & 3,872 & 73.5\% \\
+1 & Mild NPDR        &   366 &  6.9\% \\
+2 & Moderate NPDR    &   794 & 15.1\% \\
+3 & Severe NPDR      &   131 &  2.5\% \\
+4 & Proliferative DR &   106 &  2.0\% \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+The dataset is severely imbalanced: the Healthy class accounts for
+nearly 74% of all samples, while Severe NPDR and Proliferative DR
+together comprise only 4.5%.
+
+** Models
+
+Three ImageNet-pretrained architectures were fine-tuned for 5-class DR
+severity classification. In all cases the original classification head
+was replaced with a linear layer mapping to 5 outputs, and all
+backbone weights were unfrozen for full fine-tuning.
+
+- ResNet-50 [cite:@he2016deep]: Deep residual network with skip
+  connections. The final fully connected layer (2048 \to 5) was
+  replaced. Input: 384\times{}384.
+
+- EfficientNet-B0 [cite:@tan2019efficientnet]: Compound-scaled network
+  optimizing width, depth, and resolution simultaneously. The
+  classifier head (=Dropout(0.2)= + Linear 1280 \to 5) was replaced.
+  Input: 384\times{}384.
+
+- ViT-B/16 [cite:@dosovitskiy2021image]: Vision Transformer that
+  splits the image into 16\times{}16 pixel patches and processes them
+  with multi-head self-attention. The projection head (768 \to 5) was
+  replaced. Input: 224\times{}224 (PyTorch's ViT-B/16 implementation
+  only supports 224\times{}224 input; higher resolutions are not
+  supported without custom modifications to the model).
+
+** Preprocessing and Augmentation
+
+Images were resized from their native ~1024\times{}768 resolution to
+the model's target input resolution (384\times{}384 for ResNet-50 and
+EfficientNet-B0; 224\times{}224 for ViT-B/16) as the first
+preprocessing step.
+
+CLAHE (Contrast Limited Adaptive Histogram Equalization)
+[cite:@zuiderveld1994clahe] was applied as an optional subsequent
+preprocessing step. The transform converts the image from RGB to LAB
+color space, applies =cv2.createCLAHE= (clip limit 2.0, tile grid
+8\times{}8) to the L (lightness) channel only, and converts back to
+RGB. Operating on the lightness channel alone
+enhances local contrast in retinal structures such as vessels and
+lesions without altering hue or saturation.
+
+The training augmentation pipeline (applied after CLAHE if enabled):
+
+1. =RandomResizedCrop= to target resolution, scale 0.8--1.0
+2. =RandomHorizontalFlip= (p=0.5), =RandomVerticalFlip= (p=0.5)
+3. =RandomRotation=(\pm{}30\textdegree)
+4. =ColorJitter= (brightness 0.3, contrast 0.3, saturation 0.2, hue
+   0.02)
+5. =GaussianBlur= (kernel 3, \sigma{} 0.1--1.0)
+6. Normalize to ImageNet mean/std
+
+Validation and test images were resized to the target resolution
+deterministically (no random crop) and normalized identically.
+
+** Class Imbalance Handling
+
+Two complementary mechanisms addressed the severe class imbalance:
+
+- WeightedRandomSampler: Each training sample was assigned a weight
+  proportional to $1/\sqrt{n_c}$ where $n_c$ is the count of its
+  class. The square-root weighting provides a moderate upsampling of
+  minority classes without completely drowning the majority-class
+  signal (which full inverse-frequency weighting was found to do).
+
+- Focal Loss [cite:@lin2017focal]: The loss function
+  $\mathrm{FL}(p_t) = -(1-p_t)^\gamma \log(p_t)$ with $\gamma=2$
+  down-weights easy, well-classified examples and focuses gradient
+  updates on hard misclassifications, which helps with minority classes
+  where the model initially predicts low confidence.
+
+** Training Configuration
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Hyperparameters used for all experiments.}
+\begin{tabular}{ll}
+\toprule
+Hyperparameter & Value \\
+\midrule
+Optimizer              & AdamW \\
+Head learning rate     & 1e-4 \\
+Backbone learning rate & 1e-5 ($10\times$ lower) \\
+Weight decay           & 1e-4 \\
+Batch size             & 256 \\
+Max epochs             & 50 \\
+Early stopping         & patience 10, monitor val macro F1 \\
+LR warmup              & 3 epochs linear ($0.1\times \to 1\times$) \\
+LR schedule            & Cosine annealing after warmup \\
+Gradient clipping      & max norm 1.0 \\
+Mixed precision        & \texttt{torch.amp.autocast} + GradScaler \\
+Hardware               & $2\times$ AMD Radeon RX 7900 XTX (ROCm) \\
+Random seed            & 42 \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+A discriminative learning rate was used: the pretrained backbone was
+trained at 1e-5 while the newly initialized classification head was
+trained at the full 1e-4. This prevents the pretrained features from
+being overwritten too quickly in early epochs.
+
+** Evaluation Metrics
+
+Each trained model was evaluated on the held-out test set using:
+
+- Accuracy: fraction of correctly classified samples
+- Weighted F1: F1 averaged over classes weighted by support; reflects
+  overall performance on the imbalanced distribution
+- Macro F1: F1 averaged equally over all 5 classes; better reflects
+  performance on minority classes
+- Quadratic-weighted Cohen's Kappa: measures inter-rater agreement
+  beyond chance; the quadratic weighting penalizes predictions further
+  from the true label more heavily, which is appropriate for the
+  ordinal DR severity scale
+
+* Results
+
+** Summary of Test-Set Performance
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Test-set metrics for all six experiments at $384\times384$ (ResNet-50, EfficientNet-B0) and $224\times224$ (ViT-B/16).}
+\begin{tabular}{llccc}
+\toprule
+Model & CLAHE & Accuracy & Weighted F1 & Macro F1 \\
+\midrule
+ResNet-50       & No  & 0.80 & 0.77 & 0.51 \\
+ResNet-50       & Yes & 0.78 & 0.77 & 0.53 \\
+EfficientNet-B0 & No  & 0.80 & 0.78 & 0.52 \\
+EfficientNet-B0 & Yes & 0.78 & 0.76 & 0.51 \\
+ViT-B/16        & No  & 0.68 & 0.69 & 0.47 \\
+ViT-B/16        & Yes & 0.69 & 0.71 & 0.49 \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+** Per-Class F1 Scores
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Per-class F1-score for all six experiments.}
+\begin{tabular}{lccccc}
+\toprule
+Model / CLAHE & Healthy & Mild NPDR & Moderate NPDR & Severe NPDR & Proliferative DR \\
+\midrule
+ResNet-50 / No        & 0.90 & 0.09 & 0.57 & 0.50 & 0.52 \\
+ResNet-50 / Yes       & 0.89 & 0.17 & 0.53 & 0.45 & 0.61 \\
+EfficientNet-B0 / No  & 0.90 & 0.10 & 0.57 & 0.45 & 0.58 \\
+EfficientNet-B0 / Yes & 0.88 & 0.11 & 0.54 & 0.45 & 0.60 \\
+ViT-B/16 / No         & 0.81 & 0.14 & 0.43 & 0.43 & 0.51 \\
+ViT-B/16 / Yes        & 0.83 & 0.17 & 0.45 & 0.45 & 0.56 \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+** Effect of Input Resolution (224px vs 384px)
+
+The experiments were run twice: an initial run at 224\times{}224 and a
+subsequent run at 384\times{}384 for ResNet-50 and
+EfficientNet-B0. ViT-B/16 was kept at 224\times{}224 in both runs. The
+resolution increase produced a substantial improvement for the CNN
+models.
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Comparison of 224px (initial run) vs 384px (final run) test-set results for CNN models. ViT results shown for reference; both runs used 224px.}
+\resizebox{\linewidth}{!}{%
+\footnotesize
+\begin{tabular}{llccccccc}
+\toprule
+Model & CLAHE & Acc (224px) & W-F1 (224px) & M-F1 (224px) & Kappa (224px) & Acc (384px) & W-F1 (384px) & M-F1 (384px) \\
+\midrule
+ResNet-50       & No  & 0.61 & 0.66 & 0.46 & 0.53 & 0.80 & 0.77 & 0.51 \\
+ResNet-50       & Yes & 0.61 & 0.65 & 0.44 & 0.52 & 0.78 & 0.77 & 0.53 \\
+EfficientNet-B0 & No  & 0.61 & 0.65 & 0.46 & 0.55 & 0.80 & 0.78 & 0.52 \\
+EfficientNet-B0 & Yes & 0.62 & 0.66 & 0.45 & 0.54 & 0.78 & 0.76 & 0.51 \\
+ViT-B/16        & No  & 0.68 & 0.69 & 0.47 & 0.53 & 0.68 & 0.69 & 0.47 \\
+ViT-B/16        & Yes & 0.67 & 0.69 & 0.48 & 0.56 & 0.69 & 0.71 & 0.49 \\
+\bottomrule
+\end{tabular}%
+}
+\end{table}
+#+end_export
+
+Accuracy for ResNet-50 and EfficientNet-B0 improved by approximately
+19 percentage points (0.61 \to 0.80) with the resolution increase. The
+improvement is consistent across both CLAHE settings and is attributed
+to the finer retinal microstructure (microaneurysms, hemorrhage dots,
+exudates) that is present at 384px but lost at 224px.
+
+** Confusion Matrices
+
+*** ResNet-50 without CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/resnet50_clahe=False/confusion_matrix.png]]
+
+*** ResNet-50 with CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/resnet50_clahe=True/confusion_matrix.png]]
+
+*** EfficientNet-B0 without CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=False/confusion_matrix.png]]
+
+*** EfficientNet-B0 with CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=True/confusion_matrix.png]]
+
+*** ViT-B/16 without CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/vit_b_16_clahe=False/confusion_matrix.png]]
+
+*** ViT-B/16 with CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/vit_b_16_clahe=True/confusion_matrix.png]]
+
+** Training Curves
+
+*** ResNet-50
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/resnet50_clahe=False/training_curves.png]]
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/resnet50_clahe=True/training_curves.png]]
+
+*** EfficientNet-B0
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=False/training_curves.png]]
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=True/training_curves.png]]
+
+*** ViT-B/16
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/vit_b_16_clahe=False/training_curves.png]]
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/vit_b_16_clahe=True/training_curves.png]]
+
+* Discussion
+
+** Resolution Is the Dominant Factor for CNN Models
+
+The largest improvement was the ~19 percentage point accuracy jump for
+ResNet-50 and EfficientNet-B0 when input resolution was increased from
+224\times{}224 to 384\times{}384. Retinal images contain small
+features used for diagnosis (microaneurysms, dot-like red lesions as
+small as 10--20\mu{}m, dot hemorrhages, and hard exudates) that take
+up only a few pixels at 224px. At 384px these features are resolved
+clearly enough for the network to learn discriminative filters for
+them. The resolution improvement had no effect on ViT-B/16 because
+PyTorch's ViT-B/16 implementation only supports 224\times{}224 input
+and cannot be run at higher resolutions without custom modifications
+to the model.
+
+** CLAHE Trades Accuracy for Minority-Class Recall
+
+Applying CLAHE consistently reduced overall accuracy by approximately
+2 percentage points for the CNN models (e.g., ResNet-50: 0.80 \to
+0.78) while improving detection of minority classes. For ResNet-50,
+CLAHE nearly doubled the Mild NPDR F1-score from 0.09 to 0.17, and
+improved Proliferative DR F1 from 0.52 to 0.61. This happens because
+CLAHE enhances the local contrast of subtle lesions, making
+minority-class features more distinguishable, but the enhanced
+contrast can also introduce artifacts that disrupt features the model
+relied on for the majority Healthy class.
+
+For ViT-B/16, CLAHE produced a small but consistent improvement across
+most classes, and the Mild NPDR recall improved from 0.17 to
+0.22. This suggests that CLAHE is more beneficial for ViT, possibly
+because the transformer's attention mechanism can exploit the enhanced
+contrast across longer-range spatial dependencies.
+
+Whether CLAHE is desirable depends on the application: maximizing
+overall accuracy favors CLAHE-off, while maximizing detection of the
+higher-risk advanced DR grades favors CLAHE-on. For a clinical
+screening tool the latter is generally preferable.
+
+** Mild NPDR Remains Universally Difficult
+
+Mild NPDR (class 1) had the lowest F1-score in every experiment,
+ranging from 0.09 to 0.17. This class is defined by only
+microaneurysms with no other lesions, a very subtle change from the
+healthy retina. The confusion matrices confirm that the vast majority
+of Mild NPDR predictions are misclassified as Healthy. The visual
+difference is small, the class is heavily underrepresented (6.9% of
+the dataset), and the class boundaries are subjective even for trained
+graders. Despite weighted sampling and Focal Loss, the network does
+not reliably learn to distinguish these cases. A possible solution to
+this is to train with a classification model that supports higher
+resolutions such as the modified Hi-ResNet, where I hypothesize that
+the small lesions in class 1 get obscured or even essentially erased
+when downscaling the images during the dataset preparation for use as
+input to the models.
+
+** ViT-B/16 Underperforms at 224px
+
+ViT-B/16 achieved lower accuracy (0.68--0.69) compared to ResNet-50
+and EfficientNet-B0 (0.78--0.80). This is mainly due to the resolution
+constraint: ViT-B/16 uses 16\times{}16 pixel patches, so at 224px each
+patch covers a 16\times{}16 area, which is large enough to subsume
+entire microaneurysms. The training curves reveal severe overfitting:
+the training loss approaches zero while validation loss increases
+after ~10--15 epochs.  ViT-B/16 likely requires either a higher input
+resolution (which demands interpolated positional embeddings and
+careful fine-tuning), a larger dataset, or stronger regularization to
+generalize well on medical images.
+
+* Conclusion
+
+Pretrained CNN architectures achieved strong diabetic retinopathy
+grading performance (accuracy ~0.80, weighted F1 ~0.77--0.78) when
+fine-tuned at sufficient resolution (384\times{}384). Input resolution
+was the most impactful factor, with a ~19 percentage point accuracy
+improvement for ResNet-50 and EfficientNet-B0 when resolution was
+increased from 224 to 384 pixels. CLAHE preprocessing improved
+minority class detection at a small overall accuracy cost and is
+recommended for clinical settings where detecting advanced DR grades
+is the priority. EfficientNet-B0 achieved the best weighted F1 (0.78)
+without CLAHE, while ResNet-50 with CLAHE achieved the best macro F1
+(0.53), indicating slightly more balanced performance across
+classes. ViT-B/16 was constrained by its fixed-resolution patch
+embedding and underperformed the CNN models.
+
+Mild NPDR remains the hardest class: its visual distinction from a
+healthy retina is subtle and the class is severely
+underrepresented. Future work could address this with: dedicated data
+augmentation for lesion synthesis, higher-resolution ViT variants
+(ViT-L with interpolated positional embeddings), pre-training on
+larger retinal image datasets, or model ensembles.
+
+* Appendix
+The repository will be available at
+[[https://git.vineetk.net/eel4759_finalproject_classification/]].
+
+#+print_bibliography: