A multi-tenant strategy engine has one economic claim to make: that a thousand customers cost less than a thousand copies of the thing each of them bought. If the cost of a pass scales with the number of documents loaded, the claim is false, and it is false in the worst possible way — the marginal customer costs CPU whether or not anybody is running their document. That is not a scaling problem to postpone. It is a bug in the shape of the system.
This post is how konis_core gets from linear in documents to linear in distinct work, in three filters that compose. Everything below is compute timing measured on the engine: seconds per pass, streams per pass, cache hit rates. There are no trading figures here. Two of the three filters cannot change a row that gets written — cadence would have written bytes identical to the ones already stored, and the cache returns the same series the call would have produced. The first one does change what exists downstream, and that trade is stated where it is made.
The cross product, and the arithmetic that ended it
The original unit of work was every strategy against every permitted pair:
def streams_for(strategies, pairs, candle_type="futures"):
"""Every (strategy, pair) combination, which is the unit of shared work."""
return [
StreamSpec(strategy=strategy, pair=pair, candle_type=candle_type)
for strategy in strategies
for pair in pairs
]
This was harmless while both sides were small and fixed: seven documents over four pairs is 28 streams. It stopped being harmless the moment either side grew, and both did. Widening to 100 pairs took it to 700. The first customer strategy authored through the builder took it to 800 — because one document that nobody had yet pointed at a pair was computed against all one hundred of them.
Measured on the .33 host: 800 streams is about 17.5s of a 30s interval, and each additional customer document costs roughly another 2.1s whether or not anyone runs it. The rest follows from those two numbers:
17.5 s / 800 streams = 21.9 ms per stream
21.9 ms x 100 permitted pairs = 2.19 s per additional document (measured: ~2.1 s)
17.5 s + 5 x 2.1 s = 28.0 s of a 30 s interval
1,000 documents x 100 pairs = 100,000 streams
100,000 x 21.9 ms = 2,190 s = ~36 minutes per pass
The second line is the one that matters. The marginal cost of a document is the per-stream cost times the entire permitted universe, paid every interval, for a document with no running bot behind it. Five more documents would have consumed the interval. A thousand documents is 2,190s against a 30s loop — 73 times the interval, so buying 73x the hardware merely restores the status quo, and the next thousand documents undo it again. That is what makes it a shape problem rather than a capacity problem.
Filter one: a stream is work only if somebody is waiting for the answer
workers/stream_demand.py replaces the cross product with the set a running bot subscribes to. Nobody subscribing means nobody is waiting, and an answer nobody is waiting for is the definition of work not worth doing.
for version, pair, timeframe in subscriptions:
strategy = by_version.get(version)
if strategy is None:
report.refused_strategy.add(version)
continue
if pair not in permitted:
report.refused_pair.add(pair)
continue
if strategy.timeframe != timeframe:
report.refused_strategy.add(version)
continue
# Many bots share one stream -- that is the whole point of the matching
# plane -- so the key deduplicates them into a single unit of work.
chosen.setdefault((version, pair, timeframe),
StreamSpec(strategy=strategy, pair=pair,
candle_type=candle_type))
Three details carry weight here.
--pairs keeps its meaning and changes its job. It was the work list; it is now the permitted universe — the set of pairs this deployment will compute at all. A subscription outside it is refused here rather than served, so the flag still bounds what a host can be asked to do, and the refusal is counted rather than silent.
The key deduplicates. Many bots subscribe to the same (version, pair, timeframe); they collapse to one StreamSpec, which is the sharing the architecture exists for.
The output is sorted by strategy name then pair, deliberately. A set's iteration order would make one pass's timing incomparable with the next's, and comparing pass timings is how any of the numbers in this post exist.
Refusals are kept in a DemandReport beside the streams rather than logged from inside, so the caller decides what to say and a test can assert on the counts without reading output. A timeframe mismatch is classified as a refused strategy rather than a bad subscription, because it means the document changed underneath a subscription that was never rewritten — a real state during a rollout and a standing fault otherwise.
What this gives up, stated plainly: a stream nobody subscribes to has no rows, so a bot created mid-pass waits one interval for its first row instead of finding a window already computed. That is thirty seconds, once, against an interval that would otherwise have been spent computing for nobody.
Filter two: a stream is work only if it has a bar it has not seen
The loop runs every 30 seconds so that a 5m document does not wait five minutes to notice a bar. But the interval is the same for every stream, and a 4h document is recomputed 480 times between one bar and the next. 479 of those produce a frame identical to the one before it, byte for byte: same rows, and the store's upserts overwrite what was already there.
workers/stream_cadence.py drops those. Demand decides which streams exist; cadence decides which of those have moved.
The bar-close arithmetic
A bar's close time is arithmetic, not a guess. A bar opening at t on a P-minute timeframe covers [t, t+P) and closes at t+P. The instant now sits inside a bar that opened at floor(now / P) * P and is still forming. Therefore the newest bar that has certainly closed opens one period earlier:
newest_closed_open = floor(now / P) * P - P
Worked, for a 4h stream at 13:07:42 UTC (P = 240 min = 14,400 s):
floor(now / 14400) * 14400 -> 12:00 the bar still forming
12:00 - 4h -> 08:00 the newest bar certainly closed
The 08:00 bar closed at 12:00. The 12:00 bar closes at 16:00 and is not evidence of anything yet — which is also why the worker calls drop_forming_bar before evaluating anything: a decision made on a close that has not happened changes underneath itself when the bar completes.
def newest_closed_bar_open(timeframe: str, now: float) -> float | None:
minutes = TIMEFRAME_MINUTES.get(timeframe)
if minutes is None:
return None
period = minutes * 60
# The bar containing `now` is still forming, so the newest CLOSED one is the
# period before it.
return (now // period) * period - period
None for an unrecognised timeframe, which the caller reads as cannot reason about this, so always compute. Refusing to compute an unknown timeframe would silence a strategy over a typo.
def due(self, spec: StreamSpec) -> bool:
seen = self._computed_through.get(spec.key)
if seen is None:
return True
expected = newest_closed_bar_open(spec.timeframe, self.clock())
if expected is None:
return True
return expected > seen
Two timestamps. No event, no subscription, nothing to miss. The clock is injected so a test can stand at a chosen instant rather than sleep through a 4h boundary, and the state is one float per stream — ten thousand streams is 10k dict entries, smaller than a single candle frame, which is why this is per-process and needs no store behind it.
What it measures
Over 800 streams on a 5m/15m/1h/4h/1d mix, across one simulated hour of a 30s loop: 3,920 of 96,000 stream-passes computed — about 24x less work. A pass with nothing due costs 0.02s against 6.5s for the same streams computed cold, because a stream that is not due is never read either: the decision happens before the database is touched.
The denominator and a sanity check on the numerator:
3600 s / 30 s = 120 passes in the hour
120 x 800 = 96,000 stream-passes
3,920 / 96,000 = 4.1% -> ~24x less work
bars closing in one hour, per stream:
5m 12 15m 4 1h 1 4h 0.25 1d 0.042
an even split (160 streams per timeframe) would predict:
160 x (12 + 4 + 1 + 0.25 + 0.042) = 2,767 computations
+ 800 for the cold first pass = ~3,570
Recorded: 3,920. The module does not record the per-timeframe split of those 800 streams, so the ~350 gap is not attributable from what is stored — a heavier 5m share and the boundary retries described below both push it upward. Treat the agreement as a check on the shape, not as a derivation of the measured number.
Why it heals itself
The state kept is the open time of the newest bar the stream actually computed, which is the honest question — not when did we last run. That one distinction is what makes the whole thing self-healing:
- A late crawler write. The 4h bar lands four minutes after it closed. The pass at the boundary computes, sees the older bar as the newest one present, records that, and is due again 30 seconds later. It retries every interval until the bar arrives, rather than declaring victory and sleeping for four hours.
- A pass that raised, or found no candles. Nothing is recorded, so the stream is due again immediately. A transient fault costs one interval, never a whole timeframe.
- An edited document. The edit changes
version_id, which is part of the stream key, so the new version has no record and computes at once. - A feed that stops entirely. Its streams are never satisfied, so they stay due and resume the instant the crawler does. A stalled feed costs the pass it used to cost; it does not cost silence.
Two pieces of bookkeeping hold that up. mark is monotonic — a pass that read a shorter frame (a cache rebuild, a narrowed tail) must not move a stream backwards and re-run it forever:
current = self._computed_through.get(key)
if current is None or newest_bar_open > current:
self._computed_through[key] = newest_bar_open
And forget drops streams that are no longer demanded, called from the daemon because that is the one place that knows the full demanded set. Without it the map is append-only: every pair a customer ever tried and every version of every document they ever edited, kept for the life of a process that runs for weeks.
The case that buys nothing
A feed whose bars are not aligned to their own timeframe. A daily series opening at 12:00 rather than midnight is forever one bar short of what the arithmetic says has closed, so every pass recomputes. That degrades to the behaviour from before the module rather than to a wrong answer, which is the right way round for a failure to fall.
The one thing genuinely given up: the accidental repair of a row someone deleted by hand from the store between two bars. That now waits for the next bar instead of the next pass.
Filter three: one indicator computed once, however many documents ask for it
The namespace builder runs a document's indicators from raw OHLCV every time. That is right in isolation and wrong in aggregate: when a thousand customer documents watch BTC on 5m, rsi(close, 14) is computed a thousand times over the same thousand bars to produce the same thousand answers. Demand and cadence both remove work nobody wanted. strategy/indicator_cache.py removes work that is genuinely wanted and has already been done.
Measured over 200 builder-shaped documents — the same five indicators with different thresholds, which is what a large population of customer strategies actually looks like — across 20 pairs, 800 streams in one pass: 7.10s becomes 4.62s at a 98% hit rate.
800 streams / 20 pairs = 40 streams sharing each frame
40 streams x 5 indicators = 200 indicator lookups per frame
distinct computations per frame = 5
(200 - 5) / 200 = 97.5% (recorded: 98%)
4.62 s / 7.10 s = 0.65 -> 35% off the pass
A 98% hit rate does not buy a 98% cut, and the gap between those two numbers is the useful part: the indicator call is one component of a stream. Reading the frame, evaluating conditions and building the narrow rows are not cached, and they are most of the 4.62s that remains.
The 5 on the third line is the assumption carrying the whole estimate. These documents are builder-shaped: the thresholds that differ live in conditions, not in an indicator's parameters, so the indicator keys match and the streams share. A builder that varied period instead would vary the key, and the distinct count would rise with it. The cache does not assume similarity — it discovers it, and reports the hit rate in every pass line.
What the key has to get right, and why it can
A cache that returned one pair's RSI to another pair's stream would place real orders against fabricated rows. The key has to be exact rather than approximately right, and three properties make that provable here rather than hopeful.
1. The frame is named, not fingerprinted.
frame_id = (pair, timeframe, candle_type, len(candles),
_stamp(candles.index[0]), _stamp(candles.index[-1]))
The pair, timeframe and candle type come from the caller, so two frames of equal length with equal end timestamps — which BTC and ETH have on every single 5m bar — can never collide. The identity is fixed once when the cache is scoped to a frame, rather than passed alongside every lookup, because a lookup that could be given the wrong identity eventually would be.
2. History depth is part of the identity. rsi, and every other Wilder- or EMA-smoothed indicator, is recursive: seeded at a different depth it returns different numbers for the same timestamps. A document asking for 250 bars must not be served a series computed over 460. Frame length and first timestamp are in the key, so those two documents simply do not share — and the ones with equal warm-up, which is most of a builder's output since they inherit its defaults, do.
3. Inputs are identified by provenance, not by name. rsi_fast in one document and rsi14 in another are the same series under different labels.
for name in sorted(params):
value = params[name]
if isinstance(value, str) and value in provenance:
resolved.append((name, provenance[value]))
else:
resolved.append((name, _literal(value)))
return (fn, self.frame_id, tuple(resolved))
The runtime maintains that parallel bookkeeping as it goes — for each namespace entry an indicator could read, what produced it:
key = None if cache is None else cache.key(spec.fn, args, provenance)
result = MISS if cache is None else cache.get(key)
if result is MISS:
result = compute(spec.fn, ns, args)
...
ns[f"{spec.key}.{output_name}"] = series
if key is not None:
provenance[f"{spec.key}.{output_name}"] = (key, output_name)
Substituting each source name for the key of whatever produced it makes chained indicators dedup as readily as leaf ones, and stops two different series that happen to share a name across two documents from being confused for one.
That covers every input rather than most of them, and the reason is structural: an indicator can only reach the namespace through two doors, _series(ns, ref) and _ohlcv(ns, **overrides). In both, the name comes from a parameter or is an OHLCV column. OHLCV is pinned by the frame identity, parameters are in the key, and derived columns: are evaluated after every indicator has run, so an indicator cannot read one. The key is therefore complete, not merely careful.
Three more properties, each chosen to remove a class of mistake rather than to add a feature:
- Lifetime is one pass. Created when the pass begins, dropped when it ends. There is no invalidation to get wrong and no way for a stale series to survive a new bar. Sharing across passes would buy little — a new bar changes every series anyway — and would need exactly the invalidation this avoids.
- A ceiling, and a graceful way past it.
MAX_ENTRIES = 4096, at roughly 4 KB per entry, is tens of megabytes at the ceiling. Past it the cache keeps serving what it has, computes the rest, and counts the refusals — degrading to the previous behaviour is the correct response to a workload with no sharing in it. - Cached arrays are marked read-only. A cached series is handed to several customers' streams, so a vocabulary entry that modified its input in place would corrupt another customer's rows. None does; every entry is functional. Rather than trust that,
_freezeturns a future mistake into an exception instead of a wrong trade. It is best effort by design — anything that refuses the flag is left alone rather than rejected from the cache — a tripwire, not a correctness dependency.
How the three are wired
workers/daemon.py composes them in one order, and the order is not incidental:
def run_once(self, execution=None):
self.reload_strategies_if_changed() # a document published since last pass
self.reload_subscriptions() # the streams ARE the subscriptions now
worker = SignalWorker(candles=self.candles, store=self.store,
cadence=self._cadence,
share_indicators=self.config.share_indicators)
stats = worker.run_once(self.streams())
self.streams() runs demand, then hands the surviving key set to cadence.forget(...). Inside the worker:
cache = IndicatorCache() if self.share_indicators else None
for spec in streams:
if self.cadence is not None and not self.cadence.due(spec):
stats.skipped_not_due += 1
continue
...
if self.cadence is not None:
self.cadence.mark(spec, newest_bar)
Each filter is strictly cheaper than the work it removes — demand is set arithmetic over subscriptions, cadence is two floats, the cache is a dict lookup — and each one runs before the expense it gates. Cadence in particular runs before the candle read, which is the difference between an idle pass costing 0.02s and it costing a cheap-but-not-free 6.5s of reads. mark is called only after a computation completed, which is where the self-healing comes from. Both filters have config switches (cadence, share_indicators), kept as escape hatches rather than as options worth choosing.
One interaction is worth spelling out, because it could have gone badly. Cadence thins each pass, and a thinner pass has fewer streams sharing a frame — so cadence could have starved the cache. It does not, and the reason is in the keys. A cache entry is shared by streams with the same pair, timeframe, candle type, frame length and endpoints; due() is a function of the timeframe and the last bar computed. Streams that could share an entry are therefore streams that become due at the same instant. The cohort the cache needs is exactly the cohort cadence releases together. The exception is a stream due off-cycle — newly demanded, or retried after a failure — which computes alone and shares nothing. Correct answer, no saving.
Multiplying them out, and where the saving goes to zero
The three measurements come from three different workloads and do not compose into a single number that describes a real host. What composes is the shape:
cost per pass ~= (streams a running bot subscribes to)
x (fraction of those with a new closed bar)
x (per-stream cost, less the indicator work shared with its cohort)
Of the two factors that are constants of the workload rather than of the deployment: 0.041 x 0.65 = 0.027, roughly a 37x reduction against computing every demanded stream every pass with nothing shared. The first factor is not a constant at all. It depends on how many loaded documents have a running bot pointed at them, which is a per-deployment number the engine prints whenever it changes:
demand: %d streams (cross product would be %d, saved %d)
Each factor has a workload where it earns nothing, and they are different workloads:
- Demand saves nothing when every loaded document has a running bot on every permitted pair. Then the cross product is the demand, and the cost is back to documents times pairs.
- Cadence saves nothing when every stream has a new bar every pass — that is, when the loop interval is as long as the shortest timeframe in the fleet. At a 30s loop against the shortest timeframe the loader admits, 5m, the ceiling on savings is one computation in ten. It also buys nothing on a feed whose bars are not aligned to their own timeframe.
- The cache saves nothing when no two streams in a pass share a frame: every document watching a pair of its own. The hit rate is then zero, and the cost of having tried is one dict lookup per indicator.
That last one is the boundary of the entire argument, and it is worth being blunt about. The cache converts linear in documents into linear in distinct indicators, and when no two documents overlap, those are the same quantity. Sharing is not a property of the engine; it is a property of what customers happen to watch. What the engine can do is guarantee that work which is shared is done once, that work nobody asked for is not done at all, and that work already done is not repeated — then report, every pass, which of the three is actually paying: the demand: line whenever the answer changes, not_due and indicators in every pass line, the candle cache every twentieth pass. A saving whose size you cannot see is a saving you can never decide to remove.