Skip to main content

Release Changelog: Experimental v0.1.1 ➔ Official v1.0.0

This document tracks the detailed engineering evolution, feature additions, architecture optimizations, and bug fixes in TangoDB from Experimental v0.1.1 to Official Release v1.0.0.

[v1.0.0] - Official Release (2026-08-28)​

🚀 Key Highlights & Milestone​

TangoDB v1.0.0 transitions the engine from an experimental prototype into a feature-complete storage engine featuring Write-Ahead Logging (WAL) crash recovery, off-heap zero-allocation read/write pipelines, SSTable Bloom Filters and Sparse Indexes, and deterministic Windows file handle safety.

✨ Major Features & Enhancements​

1. Write-Ahead Logging (WAL) Recovery Engine​

  • Automated Startup Crash Recovery: StorageEngine automatically scans, validates, and replays un-flushed .wal files upon startup. Re-populates liveMemTable and restores global SequenceGenerator state to ensure zero data loss across unexpected shutdowns or process crashes.
  • CRC32C Checksum Validation: Every WAL record payload includes a Little-Endian CRC32C checksum. Damaged log records or incomplete writes at the end of .wal files (torn writes) are detected and truncated safely during replay without failing database initialization.
  • Delete Tombstone Persistence: Explicitly writes TYPE_DELETE records to WAL logs when db.delete(key) is invoked.
  • Obsolete Log Purging: Automatically deletes obsolete .wal files once memtable data is safely committed to disk SSTable files (.sgdb).

2. SSTable Bloom Filters & Sparse Indexing​

  • O(1) Fast Negative Lookup Rejection: Embedded a 1% false-positive rate Bloom Filter (Bfilter) directly inside .sgdb files. Point queries for non-existent keys are rejected in nanoseconds without reading disk record blocks.
  • O(log(N/16)) Sparse Index Binary Search: Built an SstSparseIndex sampling key byte offsets every 16 records. Readers binary-search index entries in memory to jump directly to target 16-record chunk intervals, replacing linear file scans and boosting disk point read throughput by +34% (512.50 ops/sec).
  • Compaction Stream Generation: Updated SstStreamWriter to generate Bloom Filters and Sparse Index blocks dynamically during level compaction operations.

3. Off-Heap Memory & Zero-Allocation Optimizations​

  • Zero-Allocation MemTable Read Lookups: Added a ThreadLocal<ByteArrayWrapper> key wrapper to MemTable.get(). Lookups no longer instantiate short-lived wrapper objects on the Java heap, achieving 40,769,881 ops/sec (~24.5 ns/op latency).
  • Precise ArenaAllocator Accounting: Refactored off-heap ArenaAllocator to accurately account for byte alignment padding and abandoned block tail space. Enforces the 64 MB memtable limit strictly and adds support for oversized record allocations.
  • Allocation-Free WAL Appends: Replaced per-append ByteBuffer.wrap() object creation in WalWriter with a reusable direct payloadBuffer.
  • Deterministic Windows File Management: Refactored SstReader to use positional FileChannel.read, completely eliminating native OS mmap virtual memory leaks and resolving Windows file handle retention issues during temporary directory cleanups.
  • Zero-Copy Streaming Compaction: Updated SstIterator to map SSTables dynamically using confined Arena handles, releasing OS memory mappings immediately upon iterator completion.

🐛 Bug Fixes & Stability​

  • MemTable Tombstones: Fixed MemTable.remove() to always insert a tombstone record (BlockFlag.TOMBSTONE, value = byte[0]) regardless of whether the key currently exists in liveMemTable.
  • SSTable Record Header Alignment: Fixed 8-byte LOCAL_SEQUENCE offset parsing in SstReader, preventing offset misalignment crashes (IndexOutOfBoundsException).
  • Modular JPMS Compatibility: Updated module-info.java to export all public engine packages (io.tango.engine.wal, io.tango.common.constants, io.tango.engine, io.tango.exception, io.tango.model). Resolved Eclipse package compilation errors (Must declare a named package).
  • Google Analytics & Gtag Integration: Resolved Docusaurus development script errors (window.gtag is not a function).

📊 Benchmark Summary (Experimental v0.1.1 vs Official v1.0.0)​

Benchmark MetricExperimental v0.1.1Official v1.0.0Enhancement Impact
MemTable Put Throughput~53.21 M ops/s40.77 M ops/sZero-allocation ThreadLocal lookups (~24.5 ns latency)
SSTable Point Read Speed382.97 ops/s512.50 ops/s+34% Throughput Increase (O(1) Bloom Filter + Sparse Index)
WAL Crash RecoveryN/A (Manual)100% AutomatedCRC32C checksum validation & sequence restoration
Windows OS File SafetyKernel mmap leaksDeterministicPositional FileChannel.read with instant handle releases
Test Suite Verification50 Tests58/58 Passing0 failures, 0 errors across unit, WAL, & index tests

[v0.1.1] - Experimental Prototype​

  • Initial public prototype of TangoDB exploring Java Foreign Function & Memory (FFM) API.
  • Basic off-heap MemTable backed by ConcurrentSkipListMap indexing.
  • Asynchronous FlushManager and CompactionManager pipeline.