An Approach to Dimensionality Reduction based on Contrastive Learning

A Preliminary Analysis

Abstract

Dimensionality Reduction (DR) is essential for filtering noise in high-dimensional data and enabling visualization, yet traditional non-linear methods often lack parametric mappings or distort global geometry. We propose a preliminary analysis of Contrastive Learning (CL) as a tool for explicit DR in supervised classification. By utilizing a siamese architecture with a SigLIP loss, we reconfigure latent spaces by attracting misclassified instances toward their correct class manifold while repelling them from the incorrect one. Preliminary experiments on X-ray images, sentiment analysis on textual customer reviews, and on a synthetic dataset for classification demonstrate that CL-based modification and CL-based reduction to just two dimensions can maintain or even improve classification accuracy compared to original high-dimensional spaces, while providing highly regular data structure improving data visualization as well.