ThreatDB Hyper Table
High-density threat intelligence storage built for massive-scale indicators of compromise.
ThreatDB Hyper Table is Hafnova's storage and processing technology for high-volume indicators of compromise. It is designed to handle extreme-scale datasets efficiently while preserving high read performance, low latency, and strong post-processing capacity.
Built for the scale threat intelligence actually requires
As ThreatDB grew, Hafnova faced a structural problem: IoC volume increased to a level where conventional storage, indexing, and post-processing approaches became too infrastructure-heavy.
ThreatDB Hyper Table is a high-efficiency storage methodology built specifically for domains, IPs, subdomains, wildcards, ranges, and related high-volume IoCs.
Traditional approaches break at massive IOC scale
Indicators accumulate fast:
- domains
- subdomains
- wildcard patterns
- IP addresses
- IP ranges
- derived threat relationships
- post-processed intelligence artifacts
At scale, the core challenge is operational delivery speed:
- ingest it fast
- serialize and exchange it efficiently
- query it with very low latency
- post-process it continuously
- deliver results to customers at production speed
A redesigned storage architecture
ThreatDB Hyper Table is not just a bigger table or a faster index. It is a dedicated storage and access methodology engineered for high-volume threat intelligence operations.
ProtoThreat format
Alongside Hyper Table, Hafnova developed ProtoThreat, a custom format designed to reduce CPU cost for:
- serialization
- deserialization
- inter-service exchange
- transport of threat data across infrastructure components
- synchronization with satellite systems
- delivery to on-premises tooling
ProtoThreat provides a high-efficiency alternative to JSON-like payloads for very large threat-data workflows: lower CPU overhead, faster movement, stronger distributed performance, and better on-prem integration.
What it stores
Hyper Table is optimized for threat-intelligence primitives at very large scale:
Search performance
Lookup behavior is optimized for near-cache-level responsiveness across:
- exact domains
- subdomains
- wildcard matching
- IP lookups
- range queries
- large-scale post-processing access patterns
Post-processing capacity
Threat intelligence value depends on throughput after storage:
- enrich indicators
- correlate them
- post-process them
- re-score them
- redistribute them
- return them to downstream systems with low latency
Practical performance
On Apple Mac mini M4 Pro, Hafnova can:
- record 10 billion entries
- read the database at approximately 500,000 entries per second on a single core
Why it matters for customers
Hyper Table improves customer outcomes through infrastructure efficiency and delivery quality:
Key benefits
Example use cases
- Large-scale IOC ingestion
- Real-time or near-real-time lookup
- Distributed threat-data synchronization
- High-volume enrichment pipelines
- Dense on-prem intelligence delivery
Core data infrastructure for ThreatDB performance
ThreatDB Hyper Table is not a visible feature in the usual sense. It is a foundational enabling technology for performance and operational efficiency.
Combined with ProtoThreat, it provides a high-performance base for storing, moving, and serving threat intelligence at scale.
Built to make massive threat intelligence operational
ThreatDB Hyper Table is Hafnova's high-density storage and lookup architecture for massive-scale IoCs, designed for extreme performance, low-latency search, and efficient post-processing, while ProtoThreat minimizes CPU-heavy serialization across distributed and on-prem environments.