Self-Supervised Semantic Segmentation
Learning Visual Representations Without Labels
Investigated self-supervised semantic segmentation from a single unlabeled high-resolution image. Built a CNN-based representation learning pipeline from scratch, combining contrastive learning, edge-aware preprocessing, spatial coordinates, and K-means clustering to produce meaningful segmentation masks without pretrained models or manual labels.
- Input: 1 high-resolution RGB image
- Edge extraction: Sobel filters
- Variants: Raw Sobel · CLAHE · Sharpening · CLAHE + Sharpening
- Patch size: Small local image patches
- Samples: 2,000 patches per variant
- → Generated two augmented views from each patch
- → Trained a CNN from scratch with contrastive loss
- → Pulled positive pairs together and separated negative samples
- → Contrastive learning produced meaningful patch-level representations
- → Removing the contrastive loss led to poor representations and weak clustering
- → Extracted CNN embeddings from overlapping patches
- → Appended scaled 2D spatial coordinates
- → Applied K-means clustering with K = 7
- → Mapped cluster assignments back to pixels
- → Combining visual features with spatial information produced coherent regions
- → Cluster quality depended strongly on the learned representation
- → Raw Sobel
- → CLAHE-enhanced
- → Sharpened
- → CLAHE + Sharpening
- → Raw Sobel achieved the lowest intra-cluster variance
- → Raw Sobel produced clearer segmentation boundaries
- → CLAHE + sharpening produced higher variance and more noise
- → Stronger preprocessing did not necessarily improve segmentation
What surprised me most was that stronger image preprocessing did not necessarily lead to better segmentation. Among the four variants, raw Sobel produced the lowest intra-cluster variance and clearer boundaries, while CLAHE + sharpening introduced more noise. The ablation experiments also showed how important contrastive learning was for learning useful representations. This project gave me a better understanding that segmentation quality depends not only on the clustering method, but also on the quality of the representation learned before clustering.