Architecture - BioZig

System Architecture DATA PIPELINE

BioZig (BZ) does not act as a monolithic God object. Instead, it enforces strict separation of concerns, broken down into four distinct phases of execution. This prevents cyclic dependencies and ensures hardware-level performance.


graph TD
    classDef core fill:#111,stroke:#27272a,stroke-width:2px,color:#a1a1aa,font-family:monospace
    classDef parser fill:#111,stroke:#27272a,stroke-width:2px,color:#f7a41d,font-family:monospace
    classDef domain fill:#111,stroke:#27272a,stroke-width:2px,color:#4ade80,font-family:monospace
    classDef algo fill:#111,stroke:#27272a,stroke-width:2px,color:#60a5fa,font-family:monospace

    subgraph BZ Architecture Pipeline
        A[CORE
Arena, ThreadPool, SIMD]:::core -->|I/O Config| B(PARSER
MMap Bit-Sieve):::parser
        B -->|Raw Memory View| C{DOMAIN
SoA Layouts, DNA2}:::domain
        C -->|Type-Safe Structs| D[ALGORITHM
Compute Kernels]:::algo
        D -.->|Hardware Intrinsics| A
    end

1. CORE

Memory & Execution (Arena Allocators, SIMD, Thread Pools, and I/O handlers).

2. PARSER

Zero-Copy Ingestion (64-bit Bit-Sieve MMap parsers transforming raw bytes directly into struct slices).

3. DOMAIN

Data Representation (SoA layouts, e.g., DNA2 encoded bits, XYZ coordinate sets). Pure state, no logic.

4. ALGORITHM

Compute Kernels (Functions that take Domain structs, process via Core SIMD, and output results).

Zero-Copy Bit-Sieve Parsing ingestion/transcriptomics/mtx.zig

BZ does not parse files byte-by-byte. It reads 64 bits at a time from the memory map and uses bit-twiddling to find delimiters in O(1) cycles.


pub fn next(self: *MtxMmapIterator) ?[]const u8 {
    if (self.cursor >= self.buffer.len) return null;
    const start = self.cursor;
    while (self.cursor + 8 <= self.buffer.len) {
        // Load 64 bits directly from the mmap
        const block = std.mem.readInt(u64, self.buffer[self.cursor..self.cursor+8][0..8], .little);
        // XOR with 0x0A0A0A0A0A0A0A0A to find newlines
        const xor_mask = block ^ 0x0A0A0A0A0A0A0A0A;
        // Bit-twiddling magic: (x - 0x01...) & ~x & 0x80...
        const match_mask = (xor_mask - 0x0101010101010101) & ~xor_mask & 0x8080808080808080;
        if (match_mask != 0) {
            const offset = @ctz(match_mask) / 8;
            self.cursor += offset + 1;
            return self.buffer[start .. self.cursor - 1];
        }
        self.cursor += 8;
    }
}

MMapReader (Zero-Copy)

pub const MMapReader = struct { file: std.Io.File, mmap: std.Io.File.MemoryMap, ... }

Opens a file in read_only mode with protection .{ .read = true, .write = false }. Grants zero-copy access to data via mm.memory[0..size].

SIMD Vectorization core/simd/simd.zig

Hardware Accelerated Primitives

pub fn countChar(slice: []const u8, char: u8) usize

Vectorized char counting using @Vector(32, u8) and @select instead of byte-by-byte loops.


pub fn countChar(slice: []const u8, char: u8) usize {
    const VLen = 32;
    const V = @Vector(VLen, u8);
    const char_vec: V = @splat(char);
    // Compares chunks (vec == char_vec)
    // Converts bools to ints with @select
    // Reduces with @reduce(.Add, match_ints)
}

Molecular Domain: 2-Bit Compression molecular/dna/dna.zig

Nucleotide & DNA2

pub const Nucleotide = enum(u2) { A = 0b00, C = 0b01, G = 0b10, T = 0b11 };

DNA2 owns the memory, packing 4 bases per byte.


pub const DNA2 = struct {
    bytes: []u8,
    len: usize,
    allocator: std.mem.Allocator,
    
    pub fn get(self: DNA2, index: usize) Nucleotide {
        const byte_idx = index / 4;
        const shift = @as(u3, @truncate((index % 4) * 2));
        const bits = (self.bytes[byte_idx] >> shift) & 0b11;
        return @enumFromInt(bits);
    }
};

Structural Geometry structural/geometry.zig

Structure of Arrays (SoA)

pub const CoordinateSet = struct { x: []f64, y: []f64, z: []f64 };

By decoupling coordinates into SoA, SIMD vectors can load 8 f64 elements linearly without cache-misses from gather instructions.

Thread Pooling & Work Stealing core/concurrency/pool.zig

Lock-Free Atomics

@atomicRmw(.Add, &target, value, .SeqCst)

Instead of using std.Thread.Mutex, BZ aggregates concurrent results using hardware atomic instructions to eliminate lock contention on 50M+ element operations.