add final report

3 files changed, 514 insertions, 2 deletions

diff --git a/.gitignore b/.gitignore
index 69d6571..50bd962 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,7 +1,9 @@
 *~
-.claude/
 __pycache__/
 guix-root
 guix-root-1-link
 results/
-venv/
\ No newline at end of file
+venv/
+report.tex
+report.pdf
+report.bbl
diff --git a/references.bib b/references.bib
new file mode 100644
index 0000000..1a50b8e
--- /dev/null
+++ b/references.bib
@@ -0,0 +1,48 @@
+@inproceedings{he2016deep,
+  author    = {He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
+  title     = {Deep Residual Learning for Image Recognition},
+  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+  year      = {2016},
+  pages     = {770--778},
+}
+
+@inproceedings{tan2019efficientnet,
+  author    = {Tan, Mingxing and Le, Quoc V.},
+  title     = {{EfficientNet}: Rethinking Model Scaling for Convolutional Neural Networks},
+  booktitle = {Proceedings of the 36th International Conference on Machine Learning (ICML)},
+  year      = {2019},
+  pages     = {6105--6114},
+}
+
+@inproceedings{dosovitskiy2021image,
+  author    = {Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
+  title     = {An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
+  booktitle = {International Conference on Learning Representations (ICLR)},
+  year      = {2021},
+}
+
+@inproceedings{lin2017focal,
+  author    = {Lin, Tsung-Yi and Goyal, Priya and Girshick, Ross and He, Kaiming and Doll{\'a}r, Piotr},
+  title     = {Focal Loss for Dense Object Detection},
+  booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
+  year      = {2017},
+  pages     = {2980--2988},
+}
+
+@incollection{zuiderveld1994clahe,
+  author    = {Zuiderveld, Karel},
+  title     = {Contrast Limited Adaptive Histogram Equalization},
+  booktitle = {Graphics Gems {IV}},
+  editor    = {Heckbert, Paul S.},
+  publisher = {Academic Press},
+  year      = {1994},
+  pages     = {474--485},
+}
+
+@misc{amanneo2022dr,
+  author    = {Amanneo},
+  title     = {Diabetic Retinopathy Resized Arranged},
+  year      = {2022},
+  publisher = {Kaggle},
+  url       = {https://www.kaggle.com/datasets/amanneo/diabetic-retinopathy-resized-arranged},
+}
diff --git a/report.org b/report.org
new file mode 100644
index 0000000..044e888
--- /dev/null
+++ b/report.org
@@ -0,0 +1,462 @@
+#+title: Diabetic Retinopathy Classification -- Final Report
+#+subtitle: EEL4759: Digital Image Processing
+#+author: Vineet Kumar
+#+options: toc:nil
+#+cite_export: biblatex
+#+bibliography: references.bib
+#+latex_class_options: [12pt,letterpage]
+#+latex_header: \usepackage[margin=1in]{geometry}
+#+latex_header: \usepackage{booktabs}
+#+latex_header: \usepackage{float}
+#+latex_header: \usepackage{graphicx}
+
+* Abstract
+
+This report covers automated severity grading of diabetic retinopathy
+(DR) from retinal images using deep convolutional neural
+networks. Three pretrained architectures (ResNet-50, EfficientNet-B0,
+and ViT-B/16) were fine-tuned on the Kaggle "Diabetic Retinopathy
+Resized Arranged" dataset and evaluated on a five-class severity
+classification task (Healthy through Proliferative DR). Each model was
+trained with and without CLAHE (Contrast Limited Adaptive Histogram
+Equalization) preprocessing to assess its effect on classification
+performance. Class imbalance was addressed via sqrt-inverse-frequency
+weighted sampling and Focal Loss. At 384\times{}384 pixel resolution,
+ResNet-50 and EfficientNet-B0 achieved 0.80 overall accuracy and
+weighted F1-scores of 0.77 and 0.78 respectively. ViT-B/16 was
+constrained to 224\times{}224 and achieved 0.69 accuracy. CLAHE
+improved detection of minority classes (particularly Mild NPDR) at a
+small cost to overall accuracy. Mild NPDR remained the hardest class
+to classify across all experiments, with F1-scores no higher than 0.17
+due to its visual similarity to healthy retinal images.
+
+* Introduction
+
+** Diabetic Retinopathy
+
+Diabetic retinopathy is a progressive eye disease caused by diabetes
+that is one of the leading causes of preventable vision loss
+worldwide. It develops when high blood sugar damages the retinal blood
+vessels, causing leakage, abnormal blood vessel growth, and eventually
+retinal detachment if untreated. The disease is classified into five
+severity grades: no retinopathy (Healthy), Mild Non-Proliferative DR
+(NPDR), Moderate NPDR, Severe NPDR, and Proliferative DR. Early
+detection through systematic retinal screening can prevent up to 90%
+of severe vision loss cases. Manual grading by trained specialists is
+accurate but expensive and slow, making automated image classification
+useful for large-scale screening programs.
+
+** Convolutional Neural Networks for Retinal Image Classification
+
+Convolutional neural networks (CNNs) have become the dominant approach
+for medical image classification tasks. Rather than hand-crafting
+features, CNNs learn hierarchical representations (low-level edges and
+textures in early layers, progressively more abstract features in
+deeper layers) directly from training data. For retinal imaging,
+pretrained ImageNet models (transfer learning) are effective: the
+low-level filters learned on natural images transfer well to retinal
+images, and fine-tuning on DR data allows the network to specialize to
+lesion-specific features such as microaneurysms, hard exudates, and
+hemorrhages that distinguish severity grades.  This project compares
+three architectures spanning different design philosophies: ResNet-50
+(residual connections), EfficientNet-B0 (compound scaling), and
+ViT-B/16 (pure self-attention on image patches).
+
+** Existing Model Performance
+
+The Kaggle Diabetic Retinopathy detection competitions have
+established rough baselines for this type of task. On five-class
+grading datasets similar to the one used here, top-performing single
+models typically achieve quadratic weighted kappa scores in the
+0.82--0.86 range using heavily tuned ensembles, test-time
+augmentation, and competition-grade preprocessing. More comparable
+single-model baselines reported on Kaggle notebooks for the "Diabetic
+Retinopathy Resized Arranged" dataset (224\times{}224) report
+accuracies of roughly 0.60--0.73 and kappa values around 0.50--0.58
+for standard fine-tuned CNNs, which is consistent with the results
+obtained in this project's 224px training run.
+
+* Methodology
+
+** Dataset and Class Distribution
+
+The dataset used is "Diabetic Retinopathy Resized Arranged"
+[cite:@amanneo2022dr], containing 35,126 retinal images organized into
+five class-labeled folders. The images are JPEG files at approximately
+1024\times{}768 pixel resolution. The dataset was split into 70%
+training, 15% validation, and 15% test sets using stratified sampling
+to preserve the class distribution across all splits (scikit-learn's
+=train_test_split= with =stratify=, seed 42).
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Dataset class distribution (test split, 5,269 images).}
+\begin{tabular}{clrr}
+\toprule
+Class & Grade & Test Count & Test \% \\
+\midrule
+0 & Healthy          & 3,872 & 73.5\% \\
+1 & Mild NPDR        &   366 &  6.9\% \\
+2 & Moderate NPDR    &   794 & 15.1\% \\
+3 & Severe NPDR      &   131 &  2.5\% \\
+4 & Proliferative DR &   106 &  2.0\% \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+The dataset is severely imbalanced: the Healthy class accounts for
+nearly 74% of all samples, while Severe NPDR and Proliferative DR
+together comprise only 4.5%.
+
+** Models
+
+Three ImageNet-pretrained architectures were fine-tuned for 5-class DR
+severity classification. In all cases the original classification head
+was replaced with a linear layer mapping to 5 outputs, and all
+backbone weights were unfrozen for full fine-tuning.
+
+- ResNet-50 [cite:@he2016deep]: Deep residual network with skip
+  connections. The final fully connected layer (2048 \to 5) was
+  replaced. Input: 384\times{}384.
+
+- EfficientNet-B0 [cite:@tan2019efficientnet]: Compound-scaled network
+  optimizing width, depth, and resolution simultaneously. The
+  classifier head (=Dropout(0.2)= + Linear 1280 \to 5) was replaced.
+  Input: 384\times{}384.
+
+- ViT-B/16 [cite:@dosovitskiy2021image]: Vision Transformer that
+  splits the image into 16\times{}16 pixel patches and processes them
+  with multi-head self-attention. The projection head (768 \to 5) was
+  replaced. Input: 224\times{}224 (PyTorch's ViT-B/16 implementation
+  only supports 224\times{}224 input; higher resolutions are not
+  supported without custom modifications to the model).
+
+** Preprocessing and Augmentation
+
+Images were resized from their native ~1024\times{}768 resolution to
+the model's target input resolution (384\times{}384 for ResNet-50 and
+EfficientNet-B0; 224\times{}224 for ViT-B/16) as the first
+preprocessing step.
+
+CLAHE (Contrast Limited Adaptive Histogram Equalization)
+[cite:@zuiderveld1994clahe] was applied as an optional subsequent
+preprocessing step. The transform converts the image from RGB to LAB
+color space, applies =cv2.createCLAHE= (clip limit 2.0, tile grid
+8\times{}8) to the L (lightness) channel only, and converts back to
+RGB. Operating on the lightness channel alone
+enhances local contrast in retinal structures such as vessels and
+lesions without altering hue or saturation.
+
+The training augmentation pipeline (applied after CLAHE if enabled):
+
+1. =RandomResizedCrop= to target resolution, scale 0.8--1.0
+2. =RandomHorizontalFlip= (p=0.5), =RandomVerticalFlip= (p=0.5)
+3. =RandomRotation=(\pm{}30\textdegree)
+4. =ColorJitter= (brightness 0.3, contrast 0.3, saturation 0.2, hue
+   0.02)
+5. =GaussianBlur= (kernel 3, \sigma{} 0.1--1.0)
+6. Normalize to ImageNet mean/std
+
+Validation and test images were resized to the target resolution
+deterministically (no random crop) and normalized identically.
+
+** Class Imbalance Handling
+
+Two complementary mechanisms addressed the severe class imbalance:
+
+- WeightedRandomSampler: Each training sample was assigned a weight
+  proportional to $1/\sqrt{n_c}$ where $n_c$ is the count of its
+  class. The square-root weighting provides a moderate upsampling of
+  minority classes without completely drowning the majority-class
+  signal (which full inverse-frequency weighting was found to do).
+
+- Focal Loss [cite:@lin2017focal]: The loss function
+  $\mathrm{FL}(p_t) = -(1-p_t)^\gamma \log(p_t)$ with $\gamma=2$
+  down-weights easy, well-classified examples and focuses gradient
+  updates on hard misclassifications, which helps with minority classes
+  where the model initially predicts low confidence.
+
+** Training Configuration
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Hyperparameters used for all experiments.}
+\begin{tabular}{ll}
+\toprule
+Hyperparameter & Value \\
+\midrule
+Optimizer              & AdamW \\
+Head learning rate     & 1e-4 \\
+Backbone learning rate & 1e-5 ($10\times$ lower) \\
+Weight decay           & 1e-4 \\
+Batch size             & 256 \\
+Max epochs             & 50 \\
+Early stopping         & patience 10, monitor val macro F1 \\
+LR warmup              & 3 epochs linear ($0.1\times \to 1\times$) \\
+LR schedule            & Cosine annealing after warmup \\
+Gradient clipping      & max norm 1.0 \\
+Mixed precision        & \texttt{torch.amp.autocast} + GradScaler \\
+Hardware               & $2\times$ AMD Radeon RX 7900 XTX (ROCm) \\
+Random seed            & 42 \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+A discriminative learning rate was used: the pretrained backbone was
+trained at 1e-5 while the newly initialized classification head was
+trained at the full 1e-4. This prevents the pretrained features from
+being overwritten too quickly in early epochs.
+
+** Evaluation Metrics
+
+Each trained model was evaluated on the held-out test set using:
+
+- Accuracy: fraction of correctly classified samples
+- Weighted F1: F1 averaged over classes weighted by support; reflects
+  overall performance on the imbalanced distribution
+- Macro F1: F1 averaged equally over all 5 classes; better reflects
+  performance on minority classes
+- Quadratic-weighted Cohen's Kappa: measures inter-rater agreement
+  beyond chance; the quadratic weighting penalizes predictions further
+  from the true label more heavily, which is appropriate for the
+  ordinal DR severity scale
+
+* Results
+
+** Summary of Test-Set Performance
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Test-set metrics for all six experiments at $384\times384$ (ResNet-50, EfficientNet-B0) and $224\times224$ (ViT-B/16).}
+\begin{tabular}{llccc}
+\toprule
+Model & CLAHE & Accuracy & Weighted F1 & Macro F1 \\
+\midrule
+ResNet-50       & No  & 0.80 & 0.77 & 0.51 \\
+ResNet-50       & Yes & 0.78 & 0.77 & 0.53 \\
+EfficientNet-B0 & No  & 0.80 & 0.78 & 0.52 \\
+EfficientNet-B0 & Yes & 0.78 & 0.76 & 0.51 \\
+ViT-B/16        & No  & 0.68 & 0.69 & 0.47 \\
+ViT-B/16        & Yes & 0.69 & 0.71 & 0.49 \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+** Per-Class F1 Scores
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Per-class F1-score for all six experiments.}
+\begin{tabular}{lccccc}
+\toprule
+Model / CLAHE & Healthy & Mild NPDR & Moderate NPDR & Severe NPDR & Proliferative DR \\
+\midrule
+ResNet-50 / No        & 0.90 & 0.09 & 0.57 & 0.50 & 0.52 \\
+ResNet-50 / Yes       & 0.89 & 0.17 & 0.53 & 0.45 & 0.61 \\
+EfficientNet-B0 / No  & 0.90 & 0.10 & 0.57 & 0.45 & 0.58 \\
+EfficientNet-B0 / Yes & 0.88 & 0.11 & 0.54 & 0.45 & 0.60 \\
+ViT-B/16 / No         & 0.81 & 0.14 & 0.43 & 0.43 & 0.51 \\
+ViT-B/16 / Yes        & 0.83 & 0.17 & 0.45 & 0.45 & 0.56 \\
+\bottomrule
+\end{tabular}
+\end{table}
+#+end_export
+
+** Effect of Input Resolution (224px vs 384px)
+
+The experiments were run twice: an initial run at 224\times{}224 and a
+subsequent run at 384\times{}384 for ResNet-50 and
+EfficientNet-B0. ViT-B/16 was kept at 224\times{}224 in both runs. The
+resolution increase produced a substantial improvement for the CNN
+models.
+
+#+begin_export latex
+\begin{table}[H]
+\centering
+\caption{Comparison of 224px (initial run) vs 384px (final run) test-set results for CNN models. ViT results shown for reference; both runs used 224px.}
+\resizebox{\linewidth}{!}{%
+\footnotesize
+\begin{tabular}{llccccccc}
+\toprule
+Model & CLAHE & Acc (224px) & W-F1 (224px) & M-F1 (224px) & Kappa (224px) & Acc (384px) & W-F1 (384px) & M-F1 (384px) \\
+\midrule
+ResNet-50       & No  & 0.61 & 0.66 & 0.46 & 0.53 & 0.80 & 0.77 & 0.51 \\
+ResNet-50       & Yes & 0.61 & 0.65 & 0.44 & 0.52 & 0.78 & 0.77 & 0.53 \\
+EfficientNet-B0 & No  & 0.61 & 0.65 & 0.46 & 0.55 & 0.80 & 0.78 & 0.52 \\
+EfficientNet-B0 & Yes & 0.62 & 0.66 & 0.45 & 0.54 & 0.78 & 0.76 & 0.51 \\
+ViT-B/16        & No  & 0.68 & 0.69 & 0.47 & 0.53 & 0.68 & 0.69 & 0.47 \\
+ViT-B/16        & Yes & 0.67 & 0.69 & 0.48 & 0.56 & 0.69 & 0.71 & 0.49 \\
+\bottomrule
+\end{tabular}%
+}
+\end{table}
+#+end_export
+
+Accuracy for ResNet-50 and EfficientNet-B0 improved by approximately
+19 percentage points (0.61 \to 0.80) with the resolution increase. The
+improvement is consistent across both CLAHE settings and is attributed
+to the finer retinal microstructure (microaneurysms, hemorrhage dots,
+exudates) that is present at 384px but lost at 224px.
+
+** Confusion Matrices
+
+*** ResNet-50 without CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/resnet50_clahe=False/confusion_matrix.png]]
+
+*** ResNet-50 with CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/resnet50_clahe=True/confusion_matrix.png]]
+
+*** EfficientNet-B0 without CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=False/confusion_matrix.png]]
+
+*** EfficientNet-B0 with CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=True/confusion_matrix.png]]
+
+*** ViT-B/16 without CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/vit_b_16_clahe=False/confusion_matrix.png]]
+
+*** ViT-B/16 with CLAHE
+
+#+ATTR_LATEX: :width 0.75\textwidth :placement [H]
+[[file:results/vit_b_16_clahe=True/confusion_matrix.png]]
+
+** Training Curves
+
+*** ResNet-50
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/resnet50_clahe=False/training_curves.png]]
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/resnet50_clahe=True/training_curves.png]]
+
+*** EfficientNet-B0
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=False/training_curves.png]]
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/efficientnet_b0_clahe=True/training_curves.png]]
+
+*** ViT-B/16
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/vit_b_16_clahe=False/training_curves.png]]
+
+#+ATTR_LATEX: :width \textwidth :placement [H]
+[[file:results/vit_b_16_clahe=True/training_curves.png]]
+
+* Discussion
+
+** Resolution Is the Dominant Factor for CNN Models
+
+The largest improvement was the ~19 percentage point accuracy jump for
+ResNet-50 and EfficientNet-B0 when input resolution was increased from
+224\times{}224 to 384\times{}384. Retinal images contain small
+features used for diagnosis (microaneurysms, dot-like red lesions as
+small as 10--20\mu{}m, dot hemorrhages, and hard exudates) that take
+up only a few pixels at 224px. At 384px these features are resolved
+clearly enough for the network to learn discriminative filters for
+them. The resolution improvement had no effect on ViT-B/16 because
+PyTorch's ViT-B/16 implementation only supports 224\times{}224 input
+and cannot be run at higher resolutions without custom modifications
+to the model.
+
+** CLAHE Trades Accuracy for Minority-Class Recall
+
+Applying CLAHE consistently reduced overall accuracy by approximately
+2 percentage points for the CNN models (e.g., ResNet-50: 0.80 \to
+0.78) while improving detection of minority classes. For ResNet-50,
+CLAHE nearly doubled the Mild NPDR F1-score from 0.09 to 0.17, and
+improved Proliferative DR F1 from 0.52 to 0.61. This happens because
+CLAHE enhances the local contrast of subtle lesions, making
+minority-class features more distinguishable, but the enhanced
+contrast can also introduce artifacts that disrupt features the model
+relied on for the majority Healthy class.
+
+For ViT-B/16, CLAHE produced a small but consistent improvement across
+most classes, and the Mild NPDR recall improved from 0.17 to
+0.22. This suggests that CLAHE is more beneficial for ViT, possibly
+because the transformer's attention mechanism can exploit the enhanced
+contrast across longer-range spatial dependencies.
+
+Whether CLAHE is desirable depends on the application: maximizing
+overall accuracy favors CLAHE-off, while maximizing detection of the
+higher-risk advanced DR grades favors CLAHE-on. For a clinical
+screening tool the latter is generally preferable.
+
+** Mild NPDR Remains Universally Difficult
+
+Mild NPDR (class 1) had the lowest F1-score in every experiment,
+ranging from 0.09 to 0.17. This class is defined by only
+microaneurysms with no other lesions, a very subtle change from the
+healthy retina. The confusion matrices confirm that the vast majority
+of Mild NPDR predictions are misclassified as Healthy. The visual
+difference is small, the class is heavily underrepresented (6.9% of
+the dataset), and the class boundaries are subjective even for trained
+graders. Despite weighted sampling and Focal Loss, the network does
+not reliably learn to distinguish these cases. A possible solution to
+this is to train with a classification model that supports higher
+resolutions such as the modified Hi-ResNet, where I hypothesize that
+the small lesions in class 1 get obscured or even essentially erased
+when downscaling the images during the dataset preparation for use as
+input to the models.
+
+** ViT-B/16 Underperforms at 224px
+
+ViT-B/16 achieved lower accuracy (0.68--0.69) compared to ResNet-50
+and EfficientNet-B0 (0.78--0.80). This is mainly due to the resolution
+constraint: ViT-B/16 uses 16\times{}16 pixel patches, so at 224px each
+patch covers a 16\times{}16 area, which is large enough to subsume
+entire microaneurysms. The training curves reveal severe overfitting:
+the training loss approaches zero while validation loss increases
+after ~10--15 epochs.  ViT-B/16 likely requires either a higher input
+resolution (which demands interpolated positional embeddings and
+careful fine-tuning), a larger dataset, or stronger regularization to
+generalize well on medical images.
+
+* Conclusion
+
+Pretrained CNN architectures achieved strong diabetic retinopathy
+grading performance (accuracy ~0.80, weighted F1 ~0.77--0.78) when
+fine-tuned at sufficient resolution (384\times{}384). Input resolution
+was the most impactful factor, with a ~19 percentage point accuracy
+improvement for ResNet-50 and EfficientNet-B0 when resolution was
+increased from 224 to 384 pixels. CLAHE preprocessing improved
+minority class detection at a small overall accuracy cost and is
+recommended for clinical settings where detecting advanced DR grades
+is the priority. EfficientNet-B0 achieved the best weighted F1 (0.78)
+without CLAHE, while ResNet-50 with CLAHE achieved the best macro F1
+(0.53), indicating slightly more balanced performance across
+classes. ViT-B/16 was constrained by its fixed-resolution patch
+embedding and underperformed the CNN models.
+
+Mild NPDR remains the hardest class: its visual distinction from a
+healthy retina is subtle and the class is severely
+underrepresented. Future work could address this with: dedicated data
+augmentation for lesion synthesis, higher-resolution ViT variants
+(ViT-L with interpolated positional embeddings), pre-training on
+larger retinal image datasets, or model ensembles.
+
+* Appendix
+The repository will be available at
+[[https://git.vineetk.net/eel4759_finalproject_classification/]].
+
+#+print_bibliography: