Warpnet Storage Layer

This document describes how WarpNet stores user data securely, how authentication integrates with the encrypted database, and how data is structured in a prefix-based flat key-value store using BadgerDB.

0. Database Directory Resolution

Warpnet determines the location of its local database based on the operating system. This ensures that data is stored in a user-accessible and system-compliant location.

Platform-Specific Roots

• windows → %LOCALAPPDATA%\warpdata

• linux → $HOME/.warpdata

• darwin → $HOME/.warpdata

• android → $HOME/.warpdata

Windows

C:\Users\{username}\AppData\Local\warpdata

Obtained from the %LOCALAPPDATA% environment variable. If it's missing, Warpnet terminates with a fatal error.

Linux/macOS/Android

/home/{username}/.warpdata

Resolved via os.UserHomeDir(). This makes the database portable per user and invisible in the file browser by default (. prefix).

Full Database Path

The application root is only the first of three segments. The final path is:

<app root>/<network>/<database dir>/<schema version>

With defaults (node.network = warpnet, database.dir = storage, schema version v0) this resolves to:

• Linux/macOS/Android: ~/.warpdata/warpnet/storage/v0

• Windows: %LOCALAPPDATA%\warpdata\warpnet\storage\v0

Running on testnet therefore uses a completely separate database directory. The v0 segment is a schema-version guard: bumping it starts a clean store instead of attempting an in-place migration. Both node.network and database.dir are configurable via flags, environment variables or the config file.

Directory Creation

Before use, WarpNet ensures the directory exists. If creation fails, the application exits with a fatal log.

1. Encrypted Embedded Database (BadgerDB)

Warpnet uses BadgerDB as its local embedded storage engine. All content is encrypted at rest and accessible only after authentication.

Encryption Model

The encryption key is derived from:

sha256(username + "@" + password)

This key is passed to Badger's WithEncryptionKey(...) and never persisted. Badger encrypts data files with AES in CTR mode.

Engine Configuration

The store is opened with SyncWrites(false), ZSTD compression, a 256 MiB index cache, a 512 MiB block cache and 4 compactors. A Badger sequence generator is registered under the internal /SEQUENCE key for monotonic ID allocation.

Access Behavior

• If credentials are wrong → Badger's ErrEncryptionKeyMismatch is caught and surfaced as wrong username or password

• If the run.lock marker file disappears from the DB folder → the process panics and exits

• First run is detected by the absence of the run.lock marker file, not by an empty-directory check

• Calling Run on an already-running store is a no-op, so a pre-seeded in-memory database is never silently reopened

2. Authentication-Integrated Cryptography

When the user logs in through AuthRepo.Authenticate(username, password), the following happens:

1. The database is decrypted and opened

2. Two secrets are derived:

   • a session token used to pair additional devices

   • a private key for node operations

Both derivations mix in the network name (warpnet or testnet). The same credentials therefore produce a different identity on each network.

Session Token Generation

The token is generated using randomness and a timestamp:

seed = username + "@" + password + "@" + network + "@" + randByte + "@" + currentTime token = base64(sha256(seed))

randByte is a single cryptographically random byte in the range [0, 127). The token is used to authorize the pairing of other devices, i.e. mobile phones.

Private Key Derivation

The private key is deterministically derived:

seed = base64(sha256(username + "@" + password + "@" + network + repeat("@", password.length))) privateKey = GenerateKeyFromSeed(seed)

The result is an Ed25519 key. This key is:

• Not stored

• Stable across logins (same credentials + same network → same key)

• Used to initialize the main node

3. Prefix-Based Key Structure

Although BadgerDB is flat, Warpnet uses structured namespace-like keys. Keys are constructed using the PrefixBuilder and form logical hierarchies.

Key Pattern

/<NAMESPACE>/<root>/<range>/<id>/<id>/...

Namespaces are uppercase and plural. The current set is: /AUTH, /USERS, /TWEETS, /TIMELINE, /CHATS, /MESSAGES, /MEDIA, /FOLLOW, /SUBSCRIPTIONS, /BLOCKS, /MUTES, /FILTERS, /BOOKMARKS, /NOTIFICATIONS, /REACTIONS, /POLLS, /DEVICES, /SETTINGS, /OUTBOX, plus /NODES and /CRDT, which back the libp2p datastore and the CRDT replication layer on the very same engine.

The <range> segment: fixed vs sortable

The range segment is either the literal fixed, the literal none, or a numeric value. Most entities are written twice:

• a fixed key — stable, addressable by ID, whose value is a pointer to the sortable key

• a sortable key — carrying a reversed timestamp, used for recency-ordered scans

Iterators refuse to scan a prefix containing fixed, and skip fixed keys encountered mid-scan, so listings never return duplicates.

Reversed Timestamp for Sorting

BadgerDB uses byte-wise lexicographical ordering, so a key can be made sorting-sensitive. Replacing the range segment with a reversed timestamp sorts items newest-first:

reversed = MaxInt64 - timestamp.UnixMilli(), zero-padded to 19 digits (%019d)

Note that the value is in milliseconds, and that real values are large — around 9223370249886775807 for present-day timestamps. The fixed width is what guarantees correct lexicographical ordering.

Sample Code

key := NewPrefixBuilder("/MESSAGES"). AddRootID(chatId). AddReversedTimestamp(time.Now()). AddParentId(messageId). Build()

Result:

/MESSAGES/chat123/9223370249886775807/message789

And its fixed counterpart:

key := NewPrefixBuilder("/MESSAGES"). AddRootID(chatId). AddRange(FixedRangeKey). AddParentId(messageId). Build()

/MESSAGES/chat123/fixed/message789

Listing and Pagination

Prefix scans are paginated: List, ReverseList and ListKeys accept a limit (default 20) and an opaque cursor, and return the literal end once the prefix is exhausted. Reads that never write should use a read-only transaction, which skips Badger's conflict tracking. Batch writes auto-commit and resume transparently when a transaction grows too large.

4. Garbage Collection and Directory Monitoring

A background process performs periodic cleanup:

• GC: RunValueLogGC(0.5) every 1 hour by default, configurable via WithIntervalGC. Each cycle loops until Badger reports ErrNoRewrite or ErrRejected, sleeping 1 second between passes. One GC pass also runs immediately at startup.

• Folder watch: every 1 second the process verifies that the run.lock marker file is still present in the DB directory. If it disappears, the process panics and exits.

Neither the collector nor the watcher runs in in-memory mode.

This ensures data safety and prevents silent resets.

4a. Time-to-Live and Retention

Not everything is kept forever. Entries can be written with a TTL and are dropped automatically by Badger:

• Notifications — 24 hours

• Paired devices — 72 hours

• Outbox messages — 7 days

• Cached foreign images and videos — 7 days

• Node blocklist entries — either a configured duration or a permanent TTL

Your own content — posts, chats, timeline, settings, bookmarks — carries no expiry.

5. First-Run Behavior

On startup, the DB checks for a run.lock marker file inside its directory. If the file is absent, the launch is treated as a first run, a new store is initialized, and the marker is written once the database opens successfully.

This is useful for:

• New user onboarding

• Portable storage resets

• Ephemeral testing environments

The marker doubles as the sentinel for the folder watcher described above. In in-memory mode no marker is written.

6. In-Memory Mode

WarpNet supports a memory-only mode via DefaultOptions().WithInMemory(true).

In this mode both the data directory and the value directory are empty, no run.lock marker is written, and neither garbage collection nor the folder watcher is started.

This is suitable for:

• Testing

• Demos

• Disposable sessions

All data is lost on shutdown.

7. No Recovery or Reset

There is no password recovery in Warpnet. All cryptographic identity and encryption is derived from login credentials.

There is also no backup or export facility: the store can only be read by a running node that holds the derived encryption key.

If you lose your password, your data is permanently lost. Your identity, on the other hand, is reproducible — the same username, password and network derive the same private key and the same node ID on any machine.

This design enforces strict data sovereignty and user ownership.