agorix
AirdropMarketsCollectionsLeaderboardActivity
Agorix · Decentralized Marketplace
Independent · Resilient · Community-Run

A cross-chain marketplace, run by the community.

Discover, list, and trade across Bitcoin, Ethereum, and Solana on an open, community-run network.

Marketplace
CollectionsMarketsLeaderboardActivity
Network
GovernanceExplorerAirdropDocs
Account
ProfileSettings
Legal
Terms of ServicePrivacy PolicyYour Privacy Choices
agorix.io — Lasting Access
CID: 0xA60R1X · v0.1
AirdropMarketsCollections
D
Protocol Docs
Node ArchitectureGateway Front DoorProtocol ExplorerMesh NetworkingHeartbeat SystemData ShardingSSR & ShortlinksLocal Node Telemetry Rollups
Docs/Network/Data Sharding
Network

Data Sharding

Rendezvous hashing for chat threads, replica count scaling, media cache lifecycle, and the critical independence of PSBT shards.

#Data Sharding

Not all data is replicated to every node. Chat messages and media cache entries are sharded across the network using rendezvous hashing to distribute storage and query load. Protocol data (listings, orders, governance) remains fully replicated.

#Chat Shard Ring

Chat threads are assigned to nodes using rendezvous hashing (also called highest random weight hashing):

text
For each node in the ring:
  score = SHA256(nodeId + ":" + threadId)

Sort all nodes by score descending.
The top R nodes are shard owners for this thread.

This approach has a critical advantage over consistent hashing: when a node joins or leaves, only the minimum number of threads need to be reassigned. There is no "virtual node" complexity.

#Replica Count

The number of replicas per thread scales with network size:

Network SizeReplica CountFormula
1-3 nodes1Every node stores everything
4-6 nodes2ceil(4/3) = 2 through ceil(6/3) = 2
7-9 nodes3ceil(N/3)
10+ nodes~33% of nodesceil(N/3)

The formula is ceil(nodeCount / 3). Each individual thread is cloned to about one third of the live mesh, while rendezvous hashing spreads ownership across the whole network as item keys change.

#Full-Node Override

When FULL_NODE_CHAT=1 is set, the shard ownership check (isShardOwner) always returns true for the local node. This means:

  • The node stores all chat threads, not just its assigned ones
  • Incoming chat messages for any thread are persisted locally
  • The node can serve queries for any thread without forwarding
  • Useful for archive nodes, search indexers, or development

The node still participates in the shard ring normally from other nodes' perspective. Other nodes do not know (or care) that this node is in full-node mode.

#Media Sharding

Media cache entries are sharded separately from chat threads. A media entry's shard assignment is determined by its own shard key, independent of the chat thread it may be associated with.

The FULL_NODE_MEDIA=1 environment variable works the same way as FULL_NODE_CHAT -- it overrides shard ownership so the node stores all media cache entries locally.

#Media Cache Lifecycle

Each media cache entry has a status that tracks its lifecycle:

StatusMeaning
PENDINGEntry created, content not yet fetched or processed
READYContent successfully cached and available for serving
ERRORFetch or processing failed (retryable)
TOO_LARGEContent exceeded size limits, will not be cached
MISSINGSource content no longer available at origin

#Media Kinds

Media entries are typed by kind:

KindDescription
IMAGEJPEG, PNG, WebP, GIF, SVG
VIDEOMP4, WebM
AUDIOMP3, WAV, OGG
HTMLEmbedded HTML content (previews, iframes)
MODEL3D model files (GLB, GLTF)
FILEGeneric file attachments

#PSBT Shard Independence

This is critical to understand: PSBT (Partially Signed Bitcoin Transaction) shares are completely independent of the chat and media sharding system.

PSBT shares are distributed using the MPC threshold scheme:

  • Share distribution is controlled by the MPC ceremony participants
  • The threshold is set at 67% (BFT standard) of participating nodes
  • Share assignment has NO relationship to SHA256(nodeId:threadId) or the shard ring
  • FULL_NODE_CHAT and FULL_NODE_MEDIA have NO effect on PSBT share storage
  • PSBT shares are stored in a separate table with their own distribution logic

Conflating PSBT shares with data shards is a common misconception. They are entirely separate systems with different distribution algorithms, different storage backends, and different security properties.

PreviousHeartbeat SystemNext SSR & Shortlinks

On this page

  • Data Sharding
  • Chat Shard Ring
  • Replica Count
  • Full-Node Override
  • Media Sharding
  • Media Cache Lifecycle
  • Media Kinds
  • PSBT Shard Independence