Nicheverse#

Interpretable modeling of tissues

Every cell and niche mapped to an interpretable codebook.

381
samples: 102
datasets: 39M
cells mapped: 20
tissues: 6
platforms

A clear cell renal cell carcinoma primary tumor core read into cell-state lineages

A whole mouse pup with every cell painted by its learned cell-state code

Interpretable by construction

A discrete code for every cell and niche.

Each cell is quantized to one of 256 learned cell states and each neighborhood to one of 32 spatial niches, coupled by cross-attention so identity is always read in tissue context. The same discrete vocabulary transfers, unchanged, from one cohort and platform to the next.

256 cell-state codes 32 niche codes cross-attention coupled

How it works

Two encoders, two codebooks, one gated bridge.

A cell encoder reads each segmented cell into one of 256 cell state codes. A niche encoder reads the transcript field and the neighborhood around it (20 nearest cells within 50 µm, inverse distance weighted) into one of 32 niche codes. Gated cross attention lets the cell consult its niche before two decoders reconstruct expression (negative binomial with a detection term) and neighborhood composition (mean squared error with a Dirichlet multinomial term).

Nicheverse architecture: nuclear transcripts plus segmentation free transcript context feed a cell encoder and a cell codebook of 256 codes, radius restricted inverse distance weighted niche context feeds a niche encoder and a niche codebook of 32 codes, a gated cross attention block with 4 heads refines each cell by its niche, and cell and niche decoders reconstruct expression and neighborhood structure, giving spatially aware annotation of cell types and niches

Recovered, not supervised

An interpretable vocabulary of cell states.

Correlating the learned code embeddings blocks them into coherent lineages with no labels supplied, epithelium, stroma, endothelium, and the immune compartment separate on their own. Each code carries a stable expression signature you can read, name, and compare across tissues.

Hierarchically clustered correlation of the 256 cell-state codes, colored by dominant cell type

Explore the atlases mapped in the nicheverse

Read across 381 independent samples from 102 datasets and every accessible platform, Xenium, CosMx, MERFISH, seqFISH, RIBOmap, EEL-FISH. Every cell is painted by the lineage of the cell-state code the model assigns it.

Browse all samples

Nicheverse

A hierarchical VQ-VAE that tokenizes imaging-based spatial transcriptomics into interpretable codebooks of cell states and multicellular niches.

Hierarchical codebooks

Paired cell-state and spatial-niche codebooks coupled by cross-attention, so cell identity is read in tissue context.

Swappable components

Encoder registry: mlp_deep (default), mlp, mlp_plr, residual_mlp, transformer, cnn, fast_cnn, deep_cnn, gnn, diffusion, dit, set_transformer, perceiver_io, soft_moe, ft_transformer. Quantizer registry: vq (default), rvq, grvq, pq, qinco, rot, soft, bsq, lfq, fsq, residual_fsq.

Spatial-aware

Per-sample graphs (knn, knn_radius, radius, delaunay, alpha_complex, gabriel, rng), inverse-distance aggregation, and opt-in spatial-coherence losses.

Install

pip install nicheverse

Quickstart

import nicheverse as nv

adata = nv.read_xenium_cohort(["./run_A", "./run_B"])
mc = nv.ModelConfig(input_dim=adata.n_vars, gene_names=tuple(adata.var_names))
model, adata = nv.Trainer(nv.TrainConfig(num_epochs=300)).fit(adata, "./ckpt", model_config=mc)

annotated = nv.predict_codes(
    nv.read_xenium_cohort(["./run_C"]), "./ckpt/hierarchical_vqvae_checkpoint.pt"
)

Citation

If you use Nicheverse, please cite:

Yarlagadda DVK, Wang Z, Jiang H, Vuong L, López-Sanmiguel A, Yang C-Y, Kotecha RR, Chen Y-B, Hakimi AA, Leslie CS, Massagué J. Developmental reversion underlies resistance to immune checkpoint blockade in kidney cancer. bioRxiv 2026. doi:10.64898/2026.08.05.743137.
Yarlagadda DVK, Massagué J, Leslie C. Discrete Representation Learning for Modeling Imaging-based Spatial Transcriptomics Data. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023, pp. 3848-3857. doi:10.1109/ICCVW60793.2023.00416.