# SeroGen

**De novo bacterial polysaccharide antigen locus discovery, twilight-zone CNN gene role classification, hybrid MCL orthogroup clustering, and graph-based pan-locus analysis.**

Built on zero-copy Structure-of-Arrays (SoA) containers and parallel Numba JIT kernels, **SeroGen** transforms raw bacterial or prophage genome assemblies into annotated polysaccharide biosynthetic gene clusters, orthogroup pan-graphs, and de novo reference databases in milliseconds — with zero external command-line dependencies (`minimap2`, `prodigal`, `mmseqs2`, or `diamond` are not required).

---

## Core Innovations

- **Polysaccharide-Specific CNN Classification vs. Remote Homology**

  ---

  Surface antigen machinery (`Wzx` flippases, `Wzy` polymerases, `WaaL` ligases, and glycosyltransferases) routinely diverges below `20–25%` amino-acid identity where `BLAST`/`MMseqs2` fail. `GeneRoleClassifier` combines multi-scale 1D convolutional motif filters (`DxD`, `ED`, `EX7E`, Rossmann loops) with a 20-D biophysical composition/hydropathy/charge head across **29 functional `GeneRole`s** (+ open-set `UNKNOWN`).
- **Smart Hybrid Clustering & CNN-Rescued MCL**

  ---

  `Clusterer` pairs `FNV-1a` exact deduplication, `RandstrobeIndex` strobemer seeding, `MMSEQS12` reduced-alphabet \\(12^4\\) CSR + ungapped BLOSUM62 diagonal prefiltering, and SIMD-parallel Smith-Waterman-Gotoh (`PairwiseAligner`) with **role-gated CNN active-site motif rescue** (`conv_max` cosine similarity) and compiled **sparse Markov Clustering (MCL)** (`mode="mcl"`) or greedy `CD-HIT` (`mode="cdhit"`).
- **Zero-Copy Graph Data Structures & Solvers**

  ---

  [`serogen.core.graph`](<https://seroverse.github.io/SeroGen/reference/core/index.md>) provides 32-bit bit-packed `IntNode`s (`GeneNode`, `IntergenicEdge`), zero-copy CSR/SoA `GraphBatch` containers with `.as_pan_graph()` compression, Weisfeiler-Lehman + Hungarian `GraphAligner`, Chu-Liu/Edmonds + constrained \\(K\\)-best `maximum_spanning_arborescence`, and Numba `RelationalMPNN`.
- **De Novo Fragmented &amp; MGE-Tolerant Locus Discovery**

  ---

  `LocusPredictor` and `LocusSegmenter` discover `CPS`, `O_AG`, `WTA`, `LTA`, `EPS`, `LOS_CORE`, and `PGA` loci *de novo* without a reference database — decoding frameshifted ORFs and `1-bp`/`4-bp` operon overlaps (`OrfPredictor`), bridging/healing `IS` transposase/integrase insertions (`heal_mobile_elements=True`), and stitching loci split across draft contigs (`stitch_contig_splits=True`).

---

## End-to-End Architecture

```
flowchart LR
    FA["<b>Genome Assembly</b><br/><i>FastaReader / GenomeAssembly</i>"]
    ORF["<b>29-State HMM ORF Caller</b><br/><i>OrfPredictor (Frameshifts + Phages)</i>"]
    CNN["<b>Polysaccharide Role CNN</b><br/><i>GeneRoleClassifier (29 Roles + Motifs)</i>"]
    SEG["<b>De Novo Locus Discovery</b><br/><i>LocusSegmenter (ACP-DP + Stitching)</i>"]
    MCL["<b>Hybrid MCL & Pan-Graph</b><br/><i>Clusterer + GraphBatch + LocusMatcher</i>"]

    FA --> ORF --> CNN --> SEG --> MCL
```

---

## Quickstart

```python
from serogen.core.genome import GenomeAssembly
from serogen.loci.predictor import LocusPredictor

# Load a bacterial genome assembly (single chromosome or multi-contig draft FASTA)
assembly = GenomeAssembly.from_fasta("genome.fasta")

# Discover polysaccharide loci de novo, bridging IS elements and stitching contig splits
predictor = LocusPredictor(stitch_contig_splits=True, heal_mobile_elements=False)
loci = predictor.predict(assembly)

for locus in loci:
    print(
        f"{locus.archetype.name}: {locus.start}..{locus.end} "
        f"({locus.num_genes} genes, completeness={locus.completeness:.2f}, "
        f"has_mge={locus.has_mge}, fragmented={locus.is_fragmented})"
    )
```

```python
from serogen.loci.matcher import LocusMatcher
from serogen.loci.models import GeneNode, Loci

# Build a de novo locus database from discovered loci using hybrid MCL clustering
raw_db = Loci.from_list(discovered_loci).canonicalize_strands()
matcher = LocusMatcher(raw_db, cluster_mode="mcl", min_identity=80.0, min_coverage=0.80)

# Compress the clustered Loci (GraphBatch) into a directed consensus pan-locus CSR graph
pan_adj, pan_nodes, pan_counts = matcher.targets.as_pan_graph(node_mask=GeneNode.mask("ROLE_ID", "CLUSTER_ID"))

# Match new de novo query loci against the database and inspect MatchType variant bitflags
matches = matcher.match(query_loci)
best = matches[0]
print(f"Best match: {best.target_name} (score={best.score:.3f}, flags={best.match_type!r}, is_novel={best.is_novel})")
```

```python
from serogen.core.io.fasta import FastaReader
from serogen.genes.orf.predictor import OrfPredictor

with open("CP014225.1.fasta", "rb") as fh, FastaReader(fh) as reader:
    contigs = reader.load()

# Decode bacterial ORFs, frameshifted pseudogenes, 1-bp/4-bp overlaps, and prophage ORFs in one pass
predictor = OrfPredictor(include_phage=True)
genes = predictor.predict(contigs)

print(f"Predicted {len(genes):,} ORFs ({int(genes.is_frameshift.sum())} frameshifted)")
proteins = genes.translations
```

---

## Performance Summary

| Pipeline Stage | Engine | Throughput / Latency (`4.66 Mb` *E. coli* K-12) |
| :--- | :--- | :--- |
| **2-Bit DNA Encoding** | `Alphabet.DNA.encode` | **`0.38 ms`** (`12.2 GB/s` via Numba LUT) |
| **Ragged Protein Translation** | `GeneticCode.translate_ragged` | **`0.61 ms`** (`4,968` CDSs) |
| **29-State Frameshift-Aware Bacterial ORFs** | `OrfPredictor(include_phage=False)` | **`59.6 ms`** (`78.1 Mb/s`) |
| **Single-Pass Bacterial + Prophage ORFs** | `OrfPredictor(include_phage=True)` | **`70.9 ms`** (`65.7 Mb/s`) |