# From Genome Assemblies to Glycan Structures & Vaccine Formulations

**SeroVerse** is an end-to-end, high-performance Python ecosystem powered by a shared zero-copy Structure-of-Arrays (SoA) core (**`seroverse`**) uniting **bacterial polysaccharide locus genomics (`SeroRef`)**, **chemical glycan databases (`KBank`)**, **geno-glycomic biosynthetic pathway alignment & structure prediction (`SeroGen`)**, and **hierarchical Bayesian sero-epidemiological modelling (`SeroEpi`)**.

\>65 Mb/s Single-pass bacterial + prophage ORF calling (\~70 ms per genome)

1,530 Curated bacterial glycan graphs paired with biosynthetic gene clusters

91.4% F1 Zero-dependency 1D-CNN protein role classification across 30 roles

MCMC & SVI Hierarchical Bayesian prevalence & LOSO-validated vaccine design

---

## The Four Pillars

Built atop **[`seroverse.core`](<https://seroverse.github.io/SeroVerse/reference/core/index.md>)** and **[`seroverse.client`](<https://seroverse.github.io/SeroVerse/reference/client/index.md>)**, the SeroVerse ecosystem is organized into four specialized pillar packages. Select a pillar below to explore its documentation, interactive benchmark slide decks, API references, and live applications.

### SeroRef

1 · Genomics &amp; Loci

Genomic ORF caller, dual-head 1D-CNN protein role &amp; prophage classifier (`seroref.genes`), exact ACP-DP capsule/O-antigen locus segmenter, contig-split stitcher, and reference locus database matcher (`seroref.loci`, `seroref.db`).

Zero CLI Dependencies 99.7% 3'-Stop Recall 100% Contig-Split Stitching

[Docs &amp; Guide](<https://seroverse.github.io/SeroRef/>) [API](<https://seroverse.github.io/SeroRef/reference/>) [GitHub ↗](<https://github.com/SeroVerse/SeroRef>)

### KBank

2 · Glycomics &amp; Chemistry

Curated bacterial surface polysaccharide database (`kbank.db`), stereochemical monosaccharide &amp; substituent ontology (`kbank.sugars`), and zero-copy CSR chemical graph engine with `CSDB`, `IUPAC`, and `KEGG KCF` parsers (`kbank.glycan`, `kbank.io`).

CSDB / IUPAC / KEGG CSR Chemical Graphs SNFG Glycan Rendering

[Docs &amp; Guide](<https://seroverse.github.io/KBank/>) [Benchmarks](<https://seroverse.github.io/KBank/notebooks/>) [API](<https://seroverse.github.io/KBank/reference/>) [GitHub ↗](<https://github.com/SeroVerse/KBank>)

### SeroGen

3 · Geno-Glycomics

Geno-glycomic integration engine bridging `SeroRef` biosynthetic gene clusters and `KBank` chemical structures (`serogen.db`) via pan-graph Expectation-Maximization enzyme-to-linkage alignment, 10-head cluster classifiers, and Relational MPNN *de novo* structure prediction (`serogen.biosynthesis`).

1,530 Locus–Glycan Pairs \>92% Monosaccharide F1 Bipartite GT-Linkage EM

[Docs &amp; Guide](<https://seroverse.github.io/SeroGen/>) [Marimo Slides](<https://seroverse.github.io/SeroGen/notebooks/>) [API](<https://seroverse.github.io/SeroGen/reference/>) [GitHub ↗](<https://github.com/SeroVerse/SeroGen>)

### SeroEpi

4 · Epidemiology

`scikit-learn` and `JAX`/`NumPyro` library for genomic sero-epidemiology. Ingests Pathogenwatch cohorts and UN/World Bank demographics, fits stratified hierarchical Bayesian prevalence models, and optimizes multi-valent vaccine and phage formulations.

NUTS MCMC &amp; SVI LOSO Formulation CV PyShiny Web Dashboard

[Docs &amp; Guide](<https://seroverse.github.io/SeroEpi/>) [Live Web App ↗](<https://tomdstanton-seroepi.share.connect.posit.cloud>) [Embedded App](<https://seroverse.github.io/SeroEpi/app/>) [GitHub ↗](<https://github.com/SeroVerse/SeroEpi>)

---

## End-to-End Ecosystem Architecture

At the foundation, **`seroverse`** provides the shared Structure-of-Arrays (SoA) memory containers, SIMD sequence/clustering kernels, CSR graph solvers, and rate-limited biological repository clients consumed across all four pillars:

```
flowchart TB
    subgraph SV["<b>0. SeroVerse Foundation (seroverse.core &amp; seroverse.client)</b>"]
        direction LR
        SOA["<b>SoA &amp; Seq Engine</b><br/><i>BatchedContainer, 2-Bit DNA,<br/>Randstrobe, SW-Gotoh, MCL</i>"]
        GRP["<b>CSR Graph &amp; GNN Core</b><br/><i>Graphs, WL+Hungarian,<br/>Arborescence, RelationalMPNN</i>"]
        CLI["<b>Async API Clients</b><br/><i>NCBI, ENA, DSMZ BacDive,<br/>StrainInfo, Pathogenwatch</i>"]
    end

    subgraph SR["<b>1. SeroRef (Genomics &amp; Loci)</b>"]
        direction TB
        ORF["<b>seroref.genes</b><br/><i>29-State Viterbi ORF Caller<br/>&amp; Dual-Head 1D-CNN</i>"]
        LOC["<b>seroref.loci &amp; db</b><br/><i>ACP-DP Locus Segmenter,<br/>Stitcher &amp; LocusMatcher</i>"]
        ORF --> LOC
    end

    subgraph KB["<b>2. KBank (Glycomics)</b>"]
        direction TB
        GLY["<b>kbank.glycan &amp; sugars</b><br/><i>CSR Chemical Graphs &amp;<br/>Stereochemical Registry</i>"]
        KDB["<b>kbank.io &amp; db</b><br/><i>CSDB / IUPAC / KEGG Parsers<br/>&amp; Curated Glycan DB</i>"]
        GLY --> KDB
    end

    subgraph SG["<b>3. SeroGen (Geno-Glycomics)</b>"]
        direction TB
        SDB["<b>serogen.db</b><br/><i>Paired Locus–Glycan<br/>Reference Records</i>"]
        BIO["<b>serogen.biosynthesis</b><br/><i>BiosyntheticAssigner EM &amp;<br/>MPNN GlycanPredictor</i>"]
        SDB --> BIO
    end

    subgraph SE["<b>4. SeroEpi (Bayesian Sero-Epidemiology)</b>"]
        direction TB
        COH["<b>Surveillance Cohorts</b><br/><i>Pathogenwatch + World Bank</i>"]
        BAY["<b>Hierarchical &amp; Formulation</b><br/><i>NumPyro NUTS/SVI &amp; LOSO CV</i>"]
        COH --> BAY
    end

    SV --> SR
    SV --> KB
    SV --> SG
    SV --> SE
    LOC ==>|"Segmented Loci &amp;<br/>Orthogroups"| SDB
    KDB ==>|"Directed Glycan<br/>CSR Graphs"| SDB
    LOC ==>|"Population Serotype<br/>Assignments"| COH
    BIO ==>|"Predicted Epitopes &amp;<br/>Cross-Reactivity"| BAY
```

---

## Technical Highlights Across the Suite

- **Zero-Copy SoA Core &amp; Async Clients (`SeroVerse`)**

  ---

  Contigs, intervals, sequences, k-mer randstrobes, and CSR graphs live in contiguous NumPy Structure-of-Arrays buffers ([`seroverse.core`](<https://seroverse.github.io/SeroVerse/reference/core/index.md>)) paired with async token-bucket rate-limited clients ([`seroverse.client`](<https://seroverse.github.io/SeroVerse/reference/client/index.md>)) for NCBI, ENA, DSMZ BacDive/StrainInfo, and Pathogenwatch.
- **Single-Pass ORF Calling &amp; ACP-DP Loci (`SeroRef`)**

  ---

  `OrfPredictor` decodes bacterial and prophage genes at **\>65 Mb/s** (**99.7% 3'-stop recall**), `GeneRoleClassifier` assigns 29 biosynthetic roles + bacteriophage origin via a pure Numba 1D-CNN, and `LocusPredictor` segments and stitches capsule/O-antigen loci across contigs.
- **CSR Chemical Glycan Graphs (`KBank`)**

  ---

  `KBank` parses `CSDB`, `IUPAC`, and `KEGG KCF` representations into bit-packed CSR chemical graphs (`Glycan`, `GlycanBatch`) backed by a complete stereochemical monosaccharide and substituent registry (`kbank.sugars`).
- **Geno-Glycomic Alignment &amp; MPNN Prediction (`SeroGen`)**

  ---

  `BiosyntheticAssigner` maps glycosyltransferases, acetyltransferases, and `Wzy` polymerases onto specific monosaccharide linkages via pan-graph EM, while `GlycanPredictor` reconstructs complete repeating-unit topologies *de novo* via a Relational MPNN and constrained spanning arborescence solver.
- **Hierarchical Partial Pooling (`SeroEpi`)**

  ---

  `SeroEpi` models geographic, temporal, and clinical sampling heterogeneity using `JAX`/`NumPyro` Binomial and Beta-Binomial hierarchical estimators (`MCMC` NUTS and `SVI`), preventing small-stratum noise from distorting global antigen rankings.
- **LOSO-Validated Vaccine Formulation (`SeroEpi`)**

  ---

  The `Formulation` meta-estimator selects optimal \\(K\\)-valent capsule/O-antigen cocktails under pooled or equal-stratum weights, controls Bayesian/BH False Discovery Rates against burden thresholds, and quantifies geographic generalizability via Leave-One-Stratum-Out CV.

---

## Unified Quickstart

```python
from seroverse.client.ncbi import NCBIClient
from seroverse.core.io.fasta import GenomeAssembly
from seroverse.core.seq.clustering import Clusterer

# 1. Load a multi-contig FASTA assembly into a zero-copy 2-bit DNA SoA container
assembly = GenomeAssembly.from_fasta("klebsiella_isolate.fasta")
print(f"Loaded {len(assembly)} contigs ({assembly.total_length:,} bp)")

# 2. Cluster protein sequences using SIMD Smith-Waterman-Gotoh + sparse Markov Clustering (MCL)
clusterer = Clusterer(mode="mcl", min_identity=0.30)

# 3. Query NCBI Entrez / Datasets asynchronously with token-bucket rate limiting
async with NCBIClient() as ncbi:
    summaries = await ncbi.fetch_nuccore_summaries(["CP014225.1"])
```

```python
from seroref.loci.matcher import LocusMatcher
from seroref.loci.models import LocusDatabase
from seroref.loci.predictor import LocusPredictor
from seroverse.core.io.fasta import GenomeAssembly

# 1. Load a bacterial genome assembly (single or fragmented multi-contig FASTA)
assembly = GenomeAssembly.from_fasta("klebsiella_isolate.fasta")

# 2. Predict ORFs (bacterial + prophage), classify 29 CNN gene roles, and segment loci via ACP-DP
predictor = LocusPredictor()
loci = predictor.predict(assembly)

for locus in loci:
    print(f"{locus.archetype.name}: {locus.start}..{locus.end} ({len(locus.genes)} genes)")

# 3. Match segmented loci against a compiled reference database with bipartite synteny alignment
db = LocusDatabase.load("spneumo_cps.npz")
matcher = LocusMatcher.from_database(db)
best = matcher.match(loci)[0]
print(f"Serotype: {best.serotype} (score={best.score:.3f}, AA identity={best.mean_aa_identity:.1%})")
```

```python
from kbank.glycan import Glycan
from serogen.biosynthesis.align import BiosyntheticAssigner
from serogen.biosynthesis.predictor import GlycanPredictor
from serogen.db.records import SeroGenRecords
from seroref.loci.predictor import LocusPredictor
from seroverse.core.io.fasta import GenomeAssembly

# 1. Parse a linear CSDB/IUPAC polysaccharide structure into a directed CSR chemical graph
glycan = Glycan.from_csdb("-3)[aDGlcp(1-2)]bDGalp(1-")

# 2. Segment a capsular polysaccharide locus from raw assembly DNA via SeroRef
assembly = GenomeAssembly.from_fasta("KL106.fasta")
locus = LocusPredictor().predict(assembly)[0]

# 3. Predict the repeating-unit structure de novo or align enzymes to linkages via SeroGen
records = SeroGenRecords.load()
predictor = GlycanPredictor.from_records(records)
top_hit = predictor.predict(locus, top_k=1)[0]
assigner = BiosyntheticAssigner()
assignment = assigner.assign_record(top_hit.glycan, locus)
```

```python
from seroepi.aggregate import Aggregator
from seroepi.estimators.formulation import FormulationEstimator
from seroepi.estimators.hierarchical import HierarchicalSVIEstimator
from seroepi.examples import KlebnetNeonatalSepsis
from seroepi.plotting.choropleth import ChoroplethPlotter

# 1. Load the KlebNET Neonatal Sepsis surveillance cohort (981 K. pneumoniae isolates)
ds = KlebnetNeonatalSepsis.load_built()

# 2. Aggregate K-locus serotype counts stratified by country/site
agg = Aggregator(target=ds.var("K_locus"), strata=[ds.var("country")]).fit(ds)

# 3. Fit a NumPyro Hierarchical Bayesian estimator & 10-valent vaccine formulation
est = HierarchicalSVIEstimator().fit(agg)
vaccine = FormulationEstimator(max_valency=10).fit(est)
chart = ChoroplethPlotter().plot(est)
```

---

## Interactive Benchmarks &amp; Live Applications

| Pillar | Interactive Resource | Highlights |
| :--- | :--- | :--- |
| **SeroRef / SeroGen** | [Prophage &amp; Bacterial ORF Benchmark](<https://seroverse.github.io/SeroGen/notebooks/prophage-orfs/>) | Interactive 16:9 Marimo slide deck benchmarking `OrfPredictor` across complete bacterial chromosomes (**99.7% 3'-stop recall**) and prophages (**95.3% recall**). |
| **SeroRef / SeroGen** | [Multi-Scale 1D-CNN Role Classifier](<https://seroverse.github.io/SeroGen/notebooks/gene-classifier/>) | Optuna TPE hyperparameter tuning and layer sweep across 29 polysaccharide biosynthesis, transport, and MGE roles + phage origin head. |
| **SeroRef / SeroGen** | [*S. pneumoniae* De Novo CPS Database](<https://seroverse.github.io/SeroGen/notebooks/spneumo-serotyping/>) | Whole-genome de novo locus segmentation, hybrid MCL clustering, pan-locus graph construction, and contig-split boundary stitching. |
| **KBank / SeroGen** | [Glycan Curation, 10-Head CNNs &amp; MPNN Predictor](<https://seroverse.github.io/KBank/notebooks/>) | Curation of **1,530** bacterial surface polysaccharide structures, pan-graph EM glycosyltransferase assignment, and Relational MPNN *de novo* structure recovery. |
| **SeroEpi** | [Live PyShiny Surveillance &amp; Formulation App ↗](<https://tomdstanton-seroepi.share.connect.posit.cloud>) | Interactive cohort explorer, compositional antigen target builder, NUTS MCMC / SVI Bayesian prevalence fitting, spatial choropleths, UpSet plots, and vaccine coverage curves. |

---

## Installation

Install the core `seroverse` package or any of the pillar packages via [`uv`](<https://docs.astral.sh/uv/>) or `pip`:

```bash
# 0. Core SoA containers, SIMD sequence/graph algorithms, and async API clients
uv add "seroverse[all]"

# 1. Genomic ORF calling, 1D-CNN role classification, and de novo locus segmentation
uv add seroref

# 2. Bacterial surface glycan database and CSDB/IUPAC/KEGG chemical graph engine
uv add kbank

# 3. Geno-glycomic enzyme-to-linkage alignment and MPNN structure prediction
uv add serogen

# 4. Bayesian sero-epidemiological estimators, plotting, and interactive PyShiny dashboard
uv add "seroepi[all]"
```