Embedding Single-Cell Data: Approaches and Tradeoffs

The way you represent single-cell data before feeding it into any downstream model shapes what the model can and cannot learn. Embedding methods are not interchangeable. PCA, scVI, and transformer-based representations each make different assumptions about the structure of the data, and those assumptions cascade into differences in what biological variation gets preserved, what gets compressed away, and what the downstream model ends up learning.

We spend a lot of time on this at Relation Therapeutics because the choice of embedding is load-bearing for everything downstream. Getting it wrong early does not cause an obvious failure at inference time. The model trains, loss decreases, evaluations look reasonable, and the embedding-level mistake stays hidden.

What PCA Captures and What It Misses

Principal component analysis is still widely used as a first reduction step, often before UMAP visualization or clustering. It is fast, deterministic, and reproducible across platforms. For datasets where most variation is continuous and Gaussian-like, it works.

The problem is that scRNA-seq count data is not Gaussian. It is sparse, overdispersed, and the sparsity is not random. Two cells from the same cell type can show zero counts for the same gene for completely different reasons: the transcript was simply not captured during library preparation (technical dropout), or it genuinely was not expressed at the moment of lysis. PCA treats both zeros identically as low expression. When you are trying to distinguish cell subtypes that differ in subtle regulatory programs, that assumption introduces noise that does not average out.

For discovery work where rare cell populations matter, PCA has a second problem. Principal components are defined by maximum variance across the full dataset. Rare subtypes, by definition, contribute little to total variance. They get compressed into later PCs where they mix with technical noise. If a disease-relevant population constitutes two or three percent of total cells in a sample, PCA is unlikely to preserve the signal that differentiates it.

scVI: The Probabilistic Alternative

scVI (single-cell Variational Inference) addresses the distributional problem directly by modeling counts with a zero-inflated negative binomial. This is a much better fit to the actual data-generating process. The model learns latent factors that explain the observed count patterns while separating technical variation from biological variation. Because batch covariates can be passed explicitly, scVI also handles batch effects as part of the model rather than as a preprocessing afterthought.

In practice, scVI embeddings tend to separate biologically distinct populations more cleanly than PCA when the dataset contains substantial dropout and batch variation. For cross-study integration, where you are combining samples processed with different protocols, this matters considerably. Two batches of the same tissue type that cluster separately in PCA space often merge appropriately in scVI latent space once the batch covariate is included.

The cost is computational. scVI requires training a neural network, which takes longer than a matrix decomposition and introduces stochasticity from the optimization. Two runs on the same data do not produce exactly the same latent space. For production pipelines that need reproducibility guarantees, this requires locking seeds carefully and validating embedding stability across runs before promoting a model to production.

Transformer-Based Cell Representations

Pre-trained foundation model embeddings are the newest entrant. Models trained on tens of millions of cell profiles learn context-aware gene expression representations. When you pass a new cell's expression profile through the encoder, you get a representation that captures not just which genes are expressed but their co-expression patterns relative to the full training distribution.

The key property that makes these representations interesting for downstream use is generalization. A transformer encoder that has seen a broad diversity of cell states during training can produce meaningful embeddings for cell types that are rare or absent from the new dataset, provided their gene expression programs overlap with something in the training distribution.

This generalizes the scVI approach. Instead of a per-dataset latent space that needs retraining on each new study, you get a fixed representation function that can be applied to new data without retraining. For target discovery pipelines where you are regularly processing new disease datasets from different tissues and conditions, that matters practically.

The tradeoff is interpretability. A PCA loading tells you which genes contribute to a component. An scVI decoder can tell you which latent dimensions are most important for a particular gene. A transformer embedding is harder to decompose back to individual gene contributions. For regulatory work or for building explanations around a specific target hypothesis, that opacity is a real constraint that should not be glossed over.

A Concrete Case: Fibrotic Lung Data

Consider a dataset of roughly 80,000 cells from fibrotic lung tissue, captured from donors at different disease stages and processed in two batches. The goal is to identify the cell population most strongly associated with fibrotic progression markers.

With a raw PCA embedding, the two batches form partially separate clusters. Fine-grained subtypes within the fibroblast compartment, which are biologically meaningful but numerically small, merge together in PCA space. A classifier trained on PCA embeddings learns the batch-correlated structure as well as the biological structure, and predictions on a new batch show systematic bias toward one batch's cell type distribution.

With scVI, batch correction is explicit. The fibroblast subtypes resolve into distinct clusters. A classifier trained on these embeddings generalizes better across batches. The rare activated fibroblast population, which is the most biologically interesting from a target perspective, is now distinguishable from the quiescent majority.

With a pre-trained transformer embedding, the fibroblast subtypes are still well-separated without additional training on this specific dataset. However, the embedding is not tuned to the fibrotic disease context, and some of the fine-grained disease-state variation that scVI captures when trained on this dataset specifically is compressed in the general-purpose representation. Neither result is categorically better; they answer slightly different questions.

The Method Choice Is an Architectural Decision

This is the boundary we keep returning to: there is no universally optimal embedding for single-cell data. The question is what you need the embedding to do. If you need batch correction and clean separation of moderately sized cell populations from a small set of well-controlled studies, scVI gives you more control and interpretability. If you need to process many heterogeneous datasets quickly and generalize to cell types not well-represented in any single study, transformer embeddings are worth the opacity cost. If you are doing initial exploratory analysis on a well-controlled dataset and interpretability matters, PCA combined with careful normalization is still reasonable.

The mistake is treating embedding choice as a default setting. The right approach is to treat it as an architectural decision that needs to be justified against the data characteristics and the downstream task requirements. When we set up a new pipeline, the first set of questions is about the data: how many cells, how many batches, how much expected cell-type diversity, and what resolution is needed at the output. Those answers drive the embedding choice.

Evaluating Embedding Quality

Whatever embedding you choose, you need to evaluate it. Silhouette scores on known cell type labels, kBET for batch mixing, and nearest-neighbor purity for rare populations are all useful diagnostics. The failure mode to watch for is a method that optimizes aggregate clustering metrics while collapsing rare subtypes. Run your evaluations at multiple granularities of annotation and pay particular attention to what happens to small populations.

An embedding that looks clean at the coarse level (T cells separate from B cells separate from myeloid cells) but collapses at the fine level (regulatory T cells merge with exhausted T cells) will produce downstream models that miss the biology that matters most for target discovery. That failure is not visible unless you look at it explicitly, which means building the evaluation into the pipeline rather than running it occasionally.