← Back to work
04 · Research

Self-Supervised Semantic Segmentation

Learning Visual Representations Without Labels

Self-Supervised Semantic Segmentation result figure
Overview

Investigated self-supervised semantic segmentation from a single unlabeled high-resolution image. Built a CNN-based representation learning pipeline from scratch, combining contrastive learning, edge-aware preprocessing, spatial coordinates, and K-means clustering to produce meaningful segmentation masks without pretrained models or manual labels.

Results
2,000
Patches
K=7
Clusters
4
Preprocessing Variants
1
Unlabeled Image
Process
01
Preprocess
Can edge information help identify meaningful image regions?
Setup
  • Input: 1 high-resolution RGB image
  • Edge extraction: Sobel filters
  • Variants: Raw Sobel · CLAHE · Sharpening · CLAHE + Sharpening
  • Patch size: Small local image patches
  • Samples: 2,000 patches per variant
02
Learn
Learn useful patch representations without labels.
Method:
  • → Generated two augmented views from each patch
  • → Trained a CNN from scratch with contrastive loss
  • → Pulled positive pairs together and separated negative samples
Found:
  • → Contrastive learning produced meaningful patch-level representations
  • → Removing the contrastive loss led to poor representations and weak clustering
03
Cluster
Can learned representations be converted into meaningful regions?
Method:
  • → Extracted CNN embeddings from overlapping patches
  • → Appended scaled 2D spatial coordinates
  • → Applied K-means clustering with K = 7
  • → Mapped cluster assignments back to pixels
Found:
  • → Combining visual features with spatial information produced coherent regions
  • → Cluster quality depended strongly on the learned representation
04
Compare
Which preprocessing strategy produces the most useful segmentation?
Compared:
  • → Raw Sobel
  • → CLAHE-enhanced
  • → Sharpened
  • → CLAHE + Sharpening
Found:
  • → Raw Sobel achieved the lowest intra-cluster variance
  • → Raw Sobel produced clearer segmentation boundaries
  • → CLAHE + sharpening produced higher variance and more noise
  • → Stronger preprocessing did not necessarily improve segmentation
Stack
PyTorchCNNContrastive LearningK-means
Notes

What surprised me most was that stronger image preprocessing did not necessarily lead to better segmentation. Among the four variants, raw Sobel produced the lowest intra-cluster variance and clearer boundaries, while CLAHE + sharpening introduced more noise. The ablation experiments also showed how important contrastive learning was for learning useful representations. This project gave me a better understanding that segmentation quality depends not only on the clustering method, but also on the quality of the representation learned before clustering.