Unbounded LLM Call Sites: A Budget Is Not a Ceiling
Unbounded LLM call sites are model calls whose repeat count is not in the source, so a per-call budget does not multiply into a ceiling. loop_bound_gate.py reads a Python tree with ast, offline and keyless, and answers one question per call site: BOUNDED with a dollar ceiling, UNBOUNDED with a named structural reason, or CANNOT ANALYZE.
A per-call budget is a multiplicand. The bill is that number times how many times the line runs. On the fixture below the budget stays fixed at $0.0312 in every column, and one call site spends $0.0936 on one input and $1.2480 on another. Same code. Same budget. Thirteen times the money, because I handed it thirteen times the rows.
So the question a budget never asks is the one that decides the bill: how many times can this line run? For a lot of call sites, that number is written in the code. For a lot of others it does not exist there at all, and the honest answer is a refusal rather than a comforting figure.
loop_bound_gate.py never imports or runs the code it reads.
AI disclosure. I wrote
loop_bound_gate.py,loop_bound_counter.pyandloop_bound_check.pywith AI assistance and ran them myself before publishing: offline, standard library only, no network, no keys, no funds. Every output block below is pasted from a real run on Python 3.13.5. The runner executes 29 scenarios, each three times, and byte-compares the copies: it reported29 deterministic, 0 not. Code sha256:loop_bound_gate.pyb593cc34…69c9,loop_bound_counter.pyb7d84407…3af8,loop_bound_check.py4755d232…1263,run_all.sh4b5f58ce…30d3. Where an output block is trimmed,[...]marks the lines I dropped from that same run; nothing inside a block is reworded. The fixtures underfixtures/are synthetic, written by me for this post, and I say so again where they appear. The one piece of real code here is urllib3 2.7.0, and I checked my local copy against the upstream tag byte for byte before quoting line numbers from it. The per-call budget$0.0312in the fixture comments is an input, not a price I am claiming about any vendor; put your own number there. I went looking for primary sources on the widely-quoted runaway-agent bills before writing this and could not confirm a single one, so not one of those numbers is in this post. Four bugs of my own die further down the page. Two of them I found; the third and fourth were found by a pre-publication review of this very draft, after I had already written that the tool was correct, and the fixture that caught the worst of them is that reviewer’s, not mine.
In short:
- Two printed ceilings of mine turned out to be exceedable, and neither miss was a fixed multiple. On the first, the overshoot is exactly the number of input rows:
3.00xat 3 rows,40.00xat 40,1000.00xat 1000, against the same printed$0.1560. The size of the error is the size of your input, which is another way of saying there was no ceiling. - That failure is not exotic. The call site is
for attempt in range(5), five calls, obviously bounded. The function containing it is called fromfor row in rowsin a different file. - The second one survived the first fix. A function reached from two bounded loops runs the sum of them; my code took the larger. Printed
$0.2496for 8 runs, executed 14 for$0.4368, and returned exit0. A false green in a pre-merge gate is the worst output this tool can produce, and it was mine for a day. - The fix is a refusal, not a smarter number. The gate now adds up every call edge it can resolve, inherits an unbounded caller’s reason, and where callers do not resolve at all prints
UNBOUNDED (unresolved caller). After both fixes:8 of 8ceiling checks held, against8 of 12for the one-function baseline. - The price of that refusal is measured. The one-function baseline priced
6 of 10files, the whole-tree gate prices4 of 10. Two files out of ten trade a dollar figure for a reason. - The flag is not free signal. Of
6files marked UNBOUNDED,3ran an identical number of calls on both datasets. It cries wolf on awhile n < 5loop you can bound by eye, and I left that fixture in the corpus rather than tuning it away. - Every positive result here rests on one config line. Delete
"entrypoints": ["*:main"]and the same corpus prices0call sites instead of 4 files:fixtures/repogoesBOUNDED 2toBOUNDED 0,fixtures/cleangoesBOUNDED 2toBOUNDED 0. - On real code, urllib3 2.7.0, watching
self.urlopen:6call sites,0bounded,4recursion,2unresolved caller. Retry-by-recursion, bounded at runtime by aretriesobject, not by anything in the structure. Zero dollars printed, because nobody declared a budget and the tool refuses to invent one. - My first alias rule flagged
12places in requests 2.34.2 as unanalysable. After two bug fixes:1. Both broken rules are still reachable behind flags, so12and8-against-14are numbers you can re-measure rather than take from me.
What does “bounded” mean for an LLM call site?
A repeat count is a fact about structure. for i in range(3) runs three times whatever the input is. for row in rows runs len(rows) times, and len(rows) is not in the file. Nest them and the counts multiply.
So there is a small set of shapes the gate can turn into an integer: a for over a literal list or tuple, over range() with literal integer arguments, over a module-level constant sequence that is assigned once and never mutated, and the product of nested such loops. Everything else gets a name instead of a number: a data-driven iterable, a while True, a while whose condition is assigned from the model’s own response, recursion, an unresolved caller.
The ceiling is then the declared budget times that integer. The budget comes from a # budget: 0.0312 comment on or above the call site. No comment, no dollars: the verdict is NO BUDGET and the exit code is 2. A default price would be the worst possible feature here, because a made-up number reads exactly like a measured one.
Run it in 60 seconds, no keys
python3 loop_bound_gate.py fixtures/repo --config config.json --max-spend 1.00config.json is two fields: which dotted names count as a model call, and which functions are entry points that run once per program.
{
"model_calls": ["client.messages.create"],
"entrypoints": ["*:main"]
}The entry point list looks like boilerplate. It is not. A function nobody calls inside the tree you handed over is a function whose repeat count you did not give the tool. It says so rather than assuming one.
Where are the unbounded LLM call sites in a synthetic repo?
fixtures/repo is six synthetic files I wrote for this post. Nothing in it is production code, and the dollar figures below are the declared budget times a repeat count, not anyone’s bill.
$ python3 loop_bound_gate.py fixtures/repo --config config.json --max-spend 1.00
loop_bound_gate 1.0.0 root=fixtures/repo python=3.13.5
model calls watched: client.messages.create
entry points declared: *:main
mode: whole-tree (callers resolved)
CALL SITE VERDICT CEILING / REASON
------------------------------------------------------------------------------
digest.py:13 BOUNDED $0.0936 = $0.0312 x 3 run(s)
digest.py:17 BOUNDED $0.1872 = $0.0312 x 6 run(s)
planner.py:13 UNBOUNDED while condition set by the model's own output
refiner.py:9 UNBOUNDED recursion (refine calls itself at refiner.py:11)
triage.py:12 UNBOUNDED for over a data-driven iterable
worker.py:10 UNBOUNDED for over a data-driven iterable at batch.py:12 (inside main)
------------------------------------------------------------------------------
call sites found: 6 BOUNDED 2 UNBOUNDED 4 CANNOT ANALYZE 0 NO BUDGET 0
no ceiling because: for over a data-driven iterable 2
no ceiling because: recursion 1
no ceiling because: while condition set by the model's own output 1
worst case allowed by the code: NOT FINITE (4 call site(s) have no ceiling, so there is no sum to report)
exit 1digest.py:17 sits in a for tone in ["short", "long"] nested inside a for section in SECTIONS, where SECTIONS is a three-element tuple at module level. Three times two is six, and $0.0312 x 6 is $0.1872.
planner.py:13 is the shape this whole post is about:
def main(client, data):
done = False
steps = 0
while not done:
# budget: 0.0312
resp = client.messages.create(model="m", prompt="step %d" % steps)
done = resp.stop
steps += 1
return stepsThe gate does a small taint walk here: done is assigned from a call it is watching, done appears in the loop test, so the reason is not the generic “condition not statically bounded” but the specific one. That distinction is the whole shape of an agent loop. The exit condition is an output of the thing you are paying for.
Then the counting replay, which is a separate program. It imports each fixture with a stub client that increments a counter and returns canned text, runs it under two datasets, and holds the per-call budget fixed:
$ python3 loop_bound_counter.py
dataset A = 3 rows / 1 item x 1 pass dataset B = 40 rows / 2 items x 3 passes
per-call budget held constant at $0.0312 in both columns
CALL SITE obs A spent A obs B spent B B/A
--------------------------------------------------------------------------------------------
digest.py:13+17 fixed passes 9 $0.2808 9 $0.2808 1.00x
triage.py:12 one call per row 3 $0.0936 40 $1.2480 13.33x
planner.py:13 model says when to stop 4 $0.1248 4 $0.1248 1.00x
refiner.py:9 retry by recursion 2 $0.0624 8 $0.2496 4.00x
worker.py:10 ATTACK-1 15 $0.4680 200 $6.2400 13.33x
multi.py:12 ATTACK-2 14 $0.4368 14 $0.4368 1.00x
[...]
--------------------------------------------------------------------------------------------
call sites whose observed count changed with the data: 4 of 12
total observed calls: A 186 ($5.8032) B 451 ($14.0712)triage.py:12 is the sentence at the top of this post, with its absolutes next to the ratio: 3 calls and $0.0936 on dataset A, 40 calls and $1.2480 on dataset B, and $0.0312 per call in both columns. The 13.33x in that row is 40 over 3, the two dataset sizes I picked, so read the column as a demonstration that the spend tracks the input rather than as a measurement of anything. The budget was never wrong. It was never a ceiling either.
The control matters as much. digest.py is 9 on both datasets, and on a size sweep of 1, 3, 10, 40 and 100 rows it is 9 five times out of five while triage.py tracks the row count exactly. A gate that flagged everything would be useless, and this is the run that says it does not.
ATTACK-1: how far off was my ceiling?
Here is worker.py, minus its docstring. Read it and price it.
def summarize(client, row):
for attempt in range(5):
# budget: 0.0312
client.messages.create(model="m", prompt="%s try %d" % (row, attempt))Five iterations, $0.0312 each, ceiling $0.1560. That is what my first version printed, and it is what --naive still prints on demand. The problem is in another file that contains no model call at all, which is exactly why a gate that prices a call site from its own function does not look there:
from worker import summarize
def main(client, data):
for row in data["rows"]:
summarize(client, row)loop_bound_check.py puts the printed ceiling against the dollars the replay actually spent, one row per file, one question per row: observed <= ceiling.
$ python3 loop_bound_check.py --naive
gate mode: NAIVE (call site's own function only)
per-call budget: $0.0312 datasets: A = 3 rows, B = 40 rows
FILE CEILING spent A spent B HOLDS?
--------------------------------------------------------------
digest.py $0.2808 $0.2808 $0.2808 yes
report.py $0.0936 $0.0936 $0.0936 yes
sweep.py $3.7440 $3.7440 $3.7440 yes
worker.py $0.1560 $0.4680 $6.2400 NO
overshoot: $6.2400 against a printed ceiling of $0.1560 = 40.00x
multi.py $0.0624 $0.4368 $0.4368 NO
overshoot: $0.4368 against a printed ceiling of $0.0624 = 7.00x
refiner.py UNBOUNDED $0.0624 $0.2496 n/a
exported.py $0.1248 $0.1248 $0.1248 yes
[...]
--------------------------------------------------------------
files the gate priced: 6 of 10
ceiling checks: 8 of 12 held
VERDICT: 4 printed ceiling(s) were exceeded. A ceiling that can be exceeded is not a ceiling.One line in that block is there because the same review caught me stacking the deck. refiner.py recurses into itself inside the very function the naive mode reads, and my naive mode was not checking for that at all: the recursion test lived only in the whole-tree path. So the baseline I was comparing against was doing less work than a one-function gate honestly can, and two of its four failures were mine, not the method’s. A gate reading one function can see that function call itself in three lines of code, so now it does. refiner.py reads UNBOUNDED in both modes above, and the failures that remain are worker.py and multi.py, the two files where the count genuinely lives somewhere else.
Now the part I got wrong when I first wrote this page. I led with 40.00x as though it were a finding. It is not. worker.py runs five calls per row, the naive ceiling is those five calls, so the overshoot works out to (rows x 5) / 5, which is rows and nothing else. I picked 40. Here is the same run with the row count as the only moving part:
$ python3 loop_bound_check.py --overshoot
worker.py, naive ceiling held constant at $0.1560, only the row count moves
rows obs calls spent naive ceiling overshoot
--------------------------------------------------------------
3 15 $0.4680 $0.1560 3.00x
10 50 $1.5600 $0.1560 10.00x
40 200 $6.2400 $0.1560 40.00x
100 500 $15.6000 $0.1560 100.00x
1000 5000 $156.0000 $0.1560 1000.00x
--------------------------------------------------------------
overshoot equals the row count, exactly, at every size tried.
The miss is not a constant. It is whatever the input is.That is a better result than the one I was claiming. A fixed 40x would be a bounded error you could pad around. An error equal to the input means no multiple saves you, which is the argument of this whole post arriving through the back door. A number in the headline was hiding it.
The repair to the gate is not a cleverer estimate. It is a refusal:
$ python3 loop_bound_check.py
gate mode: whole-tree (callers resolved)
per-call budget: $0.0312 datasets: A = 3 rows, B = 40 rows
FILE CEILING spent A spent B HOLDS?
--------------------------------------------------------------
digest.py $0.2808 $0.2808 $0.2808 yes
report.py $0.0936 $0.0936 $0.0936 yes
sweep.py $3.7440 $3.7440 $3.7440 yes
worker.py UNBOUNDED $0.4680 $6.2400 n/a
multi.py $0.4368 $0.4368 $0.4368 yes
[...]
--------------------------------------------------------------
files the gate priced: 4 of 10
ceiling checks: 8 of 8 held
VERDICT: every printed ceiling held on both datasets.8 of 8 held. Read that denominator carefully, because it is not the denominator of the naive 8 of 12: the checker only counts a file it was handed a price for. So 8 of 8 is the 4 files this mode still prices, twice each, and 8 of 12 is the 6 files the naive mode prices, twice each. Fewer files, not a better hit rate on the same ones. And this is a sweep, not a theorem: this corpus, these inputs, these verdicts.
Correctness cost two files out of ten their dollar figure, 6 of 10 priced down to 4 of 10. A tool that prints fewer numbers is a harder sell and a better answer.
ATTACK-2: the same bug again, in the fixed version
I would have published the section above as the end of the story. A pre-publication review of this draft went looking for a second instance of the same class and found one, in the code I had just called correct.
def helper(client, tag):
for i in range(2):
# budget: 0.0312
client.messages.create(model="m", prompt="TAG %d" % i)
def main(client, data):
for _a in range(3):
helper(client, "first")
for _b in range(4):
helper(client, "second")Nothing data-driven anywhere. Two loops, both literal, (3 x 2) + (4 x 2) = 14 calls. My program_count() walked both call edges and kept the larger instead of adding them:
$ python3 loop_bound_gate.py fixtures/multi --config config.json --max-spend 1.00 --v1-max-callers
[...]
CALL SITE VERDICT CEILING / REASON
------------------------------------------------------------------------------
multi.py:12 BOUNDED $0.2496 = $0.0312 x 8 run(s)
via helper <- main
------------------------------------------------------------------------------
call sites found: 1 BOUNDED 1 UNBOUNDED 0 CANNOT ANALYZE 0 NO BUDGET 0
worst case allowed by the code: $0.2496
exit 0$0.2496, 8 runs, exit 0. Green in CI, while the replay spends $0.4368. Same class as ATTACK-1, one layer up, and this time in the version I had already declared fixed after being burned once.
The correction is two lines, sum instead of max, and the flag above stays in so the wrong answer can be re-measured:
$ python3 loop_bound_gate.py fixtures/multi --config config.json --max-spend 1.00
[...]
CALL SITE VERDICT CEILING / REASON
------------------------------------------------------------------------------
multi.py:12 BOUNDED $0.4368 = $0.0312 x 14 run(s)
via helper <- main
------------------------------------------------------------------------------
call sites found: 1 BOUNDED 1 UNBOUNDED 0 CANNOT ANALYZE 0 NO BUDGET 0
worst case allowed by the code: $0.4368
exit 0Exactly the 14 the counter ran. What bothers me is not the bug. It is that my ceiling check reported a clean sweep before this fixture existed, because no file in my corpus had a function with two callers. The check was honest and the corpus was thin. Those two look identical from the outside, and the only thing that told them apart was somebody else writing a file I had not thought of.
Does the flag separate anything?
A gate that marks everything unbounded would post the same 8 of 8 and be worthless, for the same reason exit 0 makes a bad success denominator. So the replay checks the other direction: for each file, did the observed count actually move when the data changed?
$ python3 loop_bound_check.py --separation
gate verdict against what the replay actually did, per file
FILE VERDICT obs A obs B CHANGED?
----------------------------------------------------------
digest.py BOUNDED 9 9 no
report.py BOUNDED 3 3 no
sweep.py BOUNDED 120 120 no
worker.py UNBOUNDED 15 200 yes
multi.py BOUNDED 14 14 no
refiner.py UNBOUNDED 2 8 yes
exported.py UNBOUNDED 4 4 no
poller.py UNBOUNDED 5 5 no
triage.py UNBOUNDED 3 40 yes
planner.py UNBOUNDED 4 4 no
----------------------------------------------------------
BOUNDED files: 4, of which the observed count changed with the data: 0
UNBOUNDED files: 6, of which the observed count changed with the data: 3
A flag that changed nothing on 3 of 6 files is a real cost, not a rounding error.
A BOUNDED verdict that varied even once would have been a false ceiling: 0 of 4 did.Half the flags moved, half did not. Be careful with that ratio, though: I wrote this corpus, so 6 flagged and 4 clean is a composition I chose, not a false-alarm rate I measured. Add one more data-driven file and it reads 3 of 7. What the run does establish is that the verdict is not constant in either direction, and that no file the gate priced ever moved.
poller.py is the honest embarrassment:
def main(client, data):
n = 0
while n < 5:
# budget: 0.0312
client.messages.create(model="m", prompt="poll %d" % n)
n += 1
return nFive, obviously. The gate says while condition not statically bounded and refuses to price it. I could special-case a counter variable with a literal comparison and a matching increment, and then somebody writes n += step, and I am writing an interpreter. I left it flagged.
planner.py is a subtler entry in that column. It ran 4 calls on both datasets, but only because my stub returned stop on the same turn both times. That equality is a property of my stub, not of the code. In the source there is still no repeat count, and there would not be one on your machine either.
What happens on real code? urllib3 2.7.0
Fixtures show a tool does what its author meant. Real code decides whether the idea survives contact. I pointed the gate at urllib3 2.7.0 as installed on this machine, watching self.urlopen. My local copy is byte-identical to the upstream tag, sha256 b0616775d5d8c25c7b282e0908fd602af74d18b34af984c22437460021a3dd8f for connectionpool.py, so the line numbers below are clickable: connectionpool.py at 2.7.0.
$ python3 loop_bound_gate.py realcode/urllib3_full --config realcode_config.json
loop_bound_gate 1.0.0 root=realcode/urllib3_full python=3.13.5
model calls watched: self.urlopen
entry points declared: (none)
mode: whole-tree (callers resolved)
CALL SITE VERDICT CEILING / REASON
------------------------------------------------------------------------------
_request_methods.py:182 UNBOUNDED unresolved caller (request() is also reached from __init__.py:193, which does not resolve to a function in this tree)
_request_methods.py:278 UNBOUNDED unresolved caller (request() is also reached from __init__.py:193, which does not resolve to a function in this tree)
connectionpool.py:872 UNBOUNDED recursion (HTTPConnectionPool.urlopen calls itself at connectionpool.py:872, connectionpool.py:923, connectionpool.py:955)
connectionpool.py:923 UNBOUNDED recursion (HTTPConnectionPool.urlopen calls itself at connectionpool.py:872, connectionpool.py:923, connectionpool.py:955)
connectionpool.py:955 UNBOUNDED recursion (HTTPConnectionPool.urlopen calls itself at connectionpool.py:872, connectionpool.py:923, connectionpool.py:955)
poolmanager.py:503 UNBOUNDED recursion (PoolManager.urlopen calls itself at poolmanager.py:503)
------------------------------------------------------------------------------
call sites found: 6 BOUNDED 0 UNBOUNDED 6 CANNOT ANALYZE 0 NO BUDGET 0
no ceiling because: recursion 4
no ceiling because: unresolved caller 2
worst case allowed by the code: NOT FINITE (6 call site(s) have no ceiling, so there is no sum to report)
exit 1Four of six are return self.urlopen(...) inside urlopen itself: retry after a broken connection, redirect, retry. Mature, heavily reviewed, correct code. Its retry limit is real, and it lives in a retries object threaded through the call at runtime. It is not in the structure, so a reader of the structure cannot see it.
The other two rows are a defect in my tool, and I had them diagnosed wrong on this page until review. I wrote that request() was reached through an __init__.py re-export. It is not. urllib3/__init__.py builds a module-level instance and wraps it in an ordinary function:
_DEFAULT_POOL = PoolManager()
def request(method, url, *, ...):
return _DEFAULT_POOL.request(method, url, ...)My local_instances() only recognises x = SomeClass() written inside the function doing the calling, so a module-level instance never resolves. The name request then lands in the unresolved pile, and the rule that matches on a bare name runs before the edges that did resolve, throwing them away with it. In requests 2.34.2 that single unresolved session.request(...) in api.py discards all seven real edges from get, options, head, post, put, patch and delete. It is not a hard problem, it is a check in the wrong order, and it is on my list rather than in this release.
That is the shape of the finding I would take away from this whole exercise. “No ceiling in the source” and “no ceiling” are different sentences. urllib3 is the good case: the bound exists, it is just somewhere this tool does not look. Your agent loop may be the other case. The gate tells you which question you are now holding, not which answer.
No dollars appear in that block. Nobody wrote a # budget: comment in urllib3, and the tool will not invent one for somebody else’s code.
The other two bugs I shipped, and what they cost
My first alias rule was: if a watched name appears anywhere outside a call position, or if getattr is called on anything that prefixes a watched name, the module is unanalysable. It sounded conservative. On requests 2.34.2 with a deliberately sloppy watch list it produced this:
call sites found: 10 BOUNDED 0 UNBOUNDED 0 CANNOT ANALYZE 12 NO BUDGET 0Twelve refusals, and requests never sees an LLM. Two more separate mistakes, both mine:
self.request = None is not a binding. I was matching ast.Attribute nodes without looking at the context. An assignment target carries ast.Store; a callable being squirrelled away for later carries ast.Load. I was counting both.
getattr(self, attr, None) in __getstate__ is not dispatch. It is pickling. I now flag getattr only when the second argument is a string literal that completes a watched name; a computed attribute name gets a NOTE line that changes no verdict and no exit code.
After both fixes, same package, same sloppy config:
call sites found: 10 BOUNDED 0 UNBOUNDED 10 CANNOT ANALYZE 1 NO BUDGET 0From 12 to 1 on requests, and from 7 to 1 on urllib3 with that same watch list. Ten call sites on requests went from “I refuse” to an actual verdict, and six on urllib3. The old rule is still reachable as --v1-alias-rule, so the 12 above is a number you can re-measure rather than one you have to take from me.
The remaining 1 is my fault too, of a different kind: I put self.request in the watch list, and in requests that name is an attribute holding a Request object, not a method. Bad watch list, correct refusal.
The config line everything positive rests on
Earlier I said the entry point list is not boilerplate. Here is the price with a number on it. Delete "entrypoints": ["*:main"] and run the same corpus:
$ python3 loop_bound_gate.py fixtures/repo --config config_noentry.json --max-spend 1.00
loop_bound_gate 1.0.0 root=fixtures/repo python=3.13.5
model calls watched: client.messages.create
entry points declared: (none)
mode: whole-tree (callers resolved)
CALL SITE VERDICT CEILING / REASON
------------------------------------------------------------------------------
[...]
triage.py:12 UNBOUNDED for over a data-driven iterable
worker.py:10 UNBOUNDED for over a data-driven iterable at batch.py:12 (inside main)
------------------------------------------------------------------------------
call sites found: 6 BOUNDED 0 UNBOUNDED 6 CANNOT ANALYZE 0 NO BUDGET 0
no ceiling because: for over a data-driven iterable 2
no ceiling because: unresolved caller 2
no ceiling because: recursion 1
no ceiling because: while condition set by the model's own output 1
worst case allowed by the code: NOT FINITE (6 call site(s) have no ceiling, so there is no sum to report)
exit 1BOUNDED 2 becomes BOUNDED 0. The same thing happens to fixtures/clean, BOUNDED 2 to BOUNDED 0. Every positive result on this page, the ceilings, the 8 of 8, the separation table, exists because I told the tool which functions run once. That is correct behaviour, and it is also the honest reading of every green number here: they are conditional on a declaration I supplied.
Which made the next thing worse than it looks. Write that key as a string instead of a list and the false ceilings come straight back, in silence: list("*:main") shreds it into characters, * matches every function, and worker.py is priced at $0.1560 again. Nothing warned me. There is a type check now:
$ python3 loop_bound_gate.py fixtures/repo --config config_typo.json --max-spend 1.00
loop_bound_gate: ConfigError: 'entrypoints' must be a list, got strWhat this measures, and what it does not
It reads structure. It does not solve the halting problem and I am not claiming it finds every unbounded loop: fixtures/dynamic is one it does not find by the direct route.
HANDLERS = {}
def register(client):
HANDLERS["summarize"] = client.messages.create
def main(client, data):
register(client)
fn = HANDLERS["summarize"]
for row in data["rows"]:
fn(model="m", prompt=row)Zero direct call sites. Grep finds nothing. The replay says that file made 3 calls on dataset A and 40 on dataset B. A silent green here would be the most expensive output the tool could produce, so a module that mentions a watched call and yields zero call sites is reported, not skipped:
$ python3 loop_bound_gate.py fixtures/dynamic --config config.json
[...]
dispatch.py:11 CANNOT ANALYZE 0 direct call sites, but client.messages.create bound to a name instead of called
call sites found: 0 BOUNDED 0 UNBOUNDED 0 CANNOT ANALYZE 1 NO BUDGET 0
worst case allowed by the code: UNKNOWN (1 place(s) reach the model through something this tool will not guess at)
exit 2Exit 2, and 2 outranks 1. “I cannot answer” is a worse state for a gate than “I found something bad”, because the second one you act on.
A few more things it is not. It does not execute anything, so a bound your code enforces at runtime is invisible to it, which is exactly the urllib3 result. It does not read a bill or a trace, which is where the after-the-fact question lives: how much of a bill delta a dated export can separate. It does not stop a running loop: that is the job of a runtime spend guard, which halts an action before it fires, and which also has to keep counting to keep working. These are different questions asked with different information, and this post is not arguing about which one comes first. A guard protects the call sites you remembered to wrap. This finds the ones you did not, and the ones where no wrapping produces a ceiling because there is no repeat count to multiply.
It also does not disagree with the arithmetic in what a per-call quote leaves out or why a per-call cap is not enough. Those measure a run that happened. This asks whether the source contains an upper bound at all, which is a question you can ask of a file that has never run.
Three more limits worth knowing before you point it at a repository. One file it cannot parse ends the whole scan: the SyntaxError is caught at the top level, you get exit 2 and no results for any other file. There is no directory filter at all, so .venv/ and node_modules/ are fair game unless you hand it a narrower path. And the entry point declaration is taken on trust: an entry point is a function you promise runs once, so if you declare one that re-enters itself, directly or through a helper that calls back into it, the tool believes you and prices it at one run. I get BOUNDED $0.0312 and exit 0 on a main that recurses ten deep and spends $0.3120. The tool can see that self-edge and ought to refuse; it does not yet. All three are on the list; none is fixed here.
Reproduction: ./run_all.sh runs 29 scenarios three times each and byte-compares them. Last run: 29 deterministic, 0 not, Python 3.13.5. --selftest runs 27 assertions on money parsing, trapped rounding, config typing and each verdict class, including both halves of ATTACK-1 and both arithmetics of ATTACK-2; it reported 27 checks, 0 failed.
The part I am unsure about
The unresolved caller verdict is doing a lot of work, and right now I cannot tell how much of it is the world and how much is me. In this corpus it fires on 1 of 10 files, which tells me nothing. On urllib3 it fires on 2 of 6 sites, and both of those are the ordering defect I described above, not a property of urllib3.
That defect has to go before the question is even askable. Until the bare-name rule stops discarding edges that did resolve, and until a module-level instance resolves like a local one, any count of unresolved caller on a real repository is mostly a measurement of my bug. My whole fixture corpus is free functions with no classes at all, which is exactly the one call shape the resolver handles, and on real code full of classes it produced BOUNDED 0 of 6 and BOUNDED 0 of 16. I built the corpus that flattered the tool without noticing.
What I actually do not know sits behind that: on a service where functions are reached through a framework, a decorator or a DI container, is there any entry point declaration short enough to be worth writing? The globs are my current answer and I do not believe in them. If you have a repository shaped like that, the thing I would want to hear is not a ratio from this version of the tool, but whether declaring your real entry points is a ten-line job or a hopeless one. That is what decides whether this is a gate or a curiosity.
I publish one runnable tool per post, with the run pasted in and the failures left where they happened, including the two that a reviewer found after I had called this one correct. Follow if that is your kind of thing, and tell me in the comments how many entry points your repo would actually need declared. I read every one.