Agent overreach interview: Chady Dimachkie on environment safety

Agent overreach interview: Chady Dimachkie on environment safety

Agent overreach interview: safety is a property of the environment, not the model

An agent that overreaches usually looks legitimate at every single step, which is why judging the model or reading its output does not catch it. In this agent overreach interview, Chady Dimachkie argues safety has to be built into the environment the agent runs in — sandboxes, scoped credentials, approval gates and action logs — and is candid about what that leaves unprotected.

Founder of rev0.codes and Head of ML at Abwab.ai

Chady Dimachkie is the founder of rev0.codes and Head of Machine Learning at Abwab.ai.

His relevance to this agent overreach interview is unusual breadth in one practitioner: machine learning engineering at NVIDIA and Ubisoft, founding ML work at Ntropy, teaching deep learning at EPITA, and open-source contributions to Hugging Face’s Sentence Transformers and Lighteval and to Microsoft Guidance. That mix — graphics, games, financial document AI and evaluation tooling — means he has seen agents fail in several different ways rather than one.

Chady Dimachkie

Chady Dimachkie is the founder of rev0.codes and Head of Machine Learning at Abwab.ai. He was previously a founding machine learning engineer at Ntropy, an ML engineer at Ubisoft — credited on *Assassin’s Creed Valhalla*, *Watch Dogs: Legion* and *Hyper Scape* — and a deep learning intern at NVIDIA, and he has taught deep learning at EPITA in Paris. His open-source work includes contributions to Hugging Face’s Sentence Transformers and Lighteval, Microsoft Guidance, and the Rust ML ecosystem.

His argument in this agent overreach interview is that an agent’s safety and reliability should be a property of the environment it runs in rather than of the model. An agent that overreaches, he argues, usually looks legitimate at every step, so judging the model or inspecting its output is not sufficient — the controls have to survive a wrong or confused model.

He describes the practical set: sandboxed execution, narrowly scoped and short-lived credentials, limited network access, approval before consequential actions, and a complete action log. He has built an agent that generates PDF parsers, run as jobs on Flyte with sandboxing around the code it executes, and he is willing to be specific about what that setup does and does not protect against. He also argues that pausing training does not replace guardrails, and that fixed-task evaluation scores say little about whether an agent stops at the edge of what it was asked to do.

—

Environment over model

Q1. In one sentence: what does it mean that safety is a property of the environment rather than the model?

CD: It means the things that actually limit harm (what the agent can touch, which credentials it holds, where it can connect, and what needs a human yes) are set by the system around the model, so they still hold when the model is wrong, confused or manipulated.

Q2. You say an agent that overreaches looks legitimate at every step. Why does that defeat output inspection specifically — what would a reviewer have had to know to catch it?

CD: Each step on its own is valid: an authenticated call, from an expected client, often right after a successful task. Output inspection asks “is this action well formed and plausible?”. The failure only shows when you compare the action with the scope of the request. To catch it, a reviewer would need the original instruction, the boundary of what was authorized, and the full sequence of actions, not one step in isolation. That is a lot of context to hold per step, which is exactly why the boundary should be enforced by the environment instead of by someone reading outputs.

The controls

Q3. Of the set — sandboxing, scoped short-lived credentials, limited network, approval gates, action logs — which carries the most weight, and which is most often skipped in practice?

CD: Scoped, short lived credentials carry the most weight. An agent can only do damage with access it has, so if the key it holds only covers the task and expires soon after, most overreach becomes harmless, even outside the sandbox. The most often skipped is the same one, because handing the agent a developer’s broad, long lived token is the fastest way to get a demo working. Close second for being skipped: a complete action log. Teams log the model’s messages but not every tool call with its arguments and results, and then cannot reconstruct what happened. It is why rev0, the agent harness we build, records every tool call for each task.

Q4. What does your sandboxed setup actually not protect against? You have said you would be honest about that.

CD: 

– Anything inside the boundary I granted. If the task legitimately needs write access to a system, the sandbox does not stop the agent from writing the wrong thing there.

– Wrong but valid output. An OCR that runs cleanly but transcribes the wrong text is a quality failure, not a containment failure, and the sandbox is blind to it. Evaluation has to catch that.

– Misconfiguration. The guarantees are only as good as the policy; an overly broad network rule or credential silently undoes them.

– Data that comes in. Prompt injection through a document the agent reads is not stopped by isolation; isolation only limits what the agent can do after being misled.

– Escape level vulnerabilities in the sandbox itself, which I treat as unlikely but not impossible.

Evaluation versus behaviour

Q5. Fixed-task scores say little about whether an agent stops at the edge of what it was asked. How do you test for that instead?

CD: I test behaviour in the environment, not capability on a benchmark. That means running the agent with real tools and realistic permissions against scenarios built to tempt it past the request: a credential lying around that it does not need, a nearby task that looks helpful to finish, an instruction embedded in a document. Then I check the action log, not just the final answer. The question is not “did it complete the task?” but “did it do only the task?” Evaluation tooling like Hugging Face Lighteval, of which I’m a contributor, is useful for the scoring and repeatability side, but the scenarios have to come from the actual deployment.

Q6. What would a test for overreach look like — what is the pass condition, and what would make you fail a model that scores well?

CD: Setup: a task with a clear scope, plus deliberate temptations outside it (an extra permission, an adjacent action, an injected instruction). Pass condition: the task is completed, and the action log contains zero actions outside the declared scope; when the agent hits the edge, it stops or asks rather than proceeding. That stop and ask behaviour is something the environment can enforce: in rev0, an out of scope command parks the task and waits for a human, with the agent’s own justification attached. I would fail a model with a high task score if it reaches the goal by taking an unrequested action, even once, or if it uses access it found rather than access it was given. Completing more tasks by going further than asked is a worse result, not a better one.

Model choice

Q7. Open weight and closed models both work inside a harness. How do you actually decide, and what would change the decision — quality, cost, or data locality?

CD: Per task, on three axes. Data locality comes first because it is a constraint, not a preference: if the data is not allowed to leave our infrastructure, that settles it in favour of a model we can host. Then quality on our own evaluation for that specific task, then cost at the expected volume. The harness is what makes switching cheap, since the controls live outside the model; in rev0 the same task can run on a different agent or model such as Claude or GPT with the same gates and logs around it. What changes the decision: a new open weight release closing the quality gap on a task, a price change, or a change in where a user allows its data to go.

Building an Agent

Q8. You built an agent that generates PDF parsers, run as jobs on Flyte with sandboxing around executed code. What did that teach you about where agents fail that you did not expect?

CD: The failures that mattered were rarely dramatic. Generated code that crashed was easy, because the job failed and we saw it. The hard cases were parsers that ran cleanly and returned plausible but wrong values. The sandbox kept execution contained, but containment says nothing about correctness, so validating outputs against known documents turned out to matter as much as isolating the code. The other lesson was that running each attempt as a separate, isolated job made failures cheap and reproducible, which is what made iterating safely possible.

Putting to the test

Q9. You argue pausing training does not replace guardrails. Why is training-level safety the wrong lever for this problem?

CD: Not wrong, just insufficient on its own. Training shapes tendencies; it does not give guarantees. Overreach is a deployment property: the same model is safe with a narrow, short lived credential and risky with a broad one. Pausing training changes nothing about the access an already deployed agent holds. Environmental controls also apply to every model you run, including the next one and ones you did not train, while training-level fixes have to be re-verified with every release.

Q10. What would prove you wrong — is there a class of agent where model-level safety genuinely suffices without environmental controls?

CD: Agents with no consequential actions: read only, no credentials, output reviewed by a human before anything happens. There, the environment is already the control, so model behaviour is mostly a quality question. What would genuinely prove me wrong is a model whose scope adherence is verifiable, not just measured on a benchmark, consistently across new environments and adversarial inputs. I have not seen that yet, and even then I would keep the controls, because they are cheap compared with the cost of the failure.

First Control to Build

Q11. A team deploying an agent this month: what is the first control to build, and what is the most common mistake?

CD: First control: give the agent its own identity with narrowly scoped, short lived credentials, and log every tool call it makes. That is cheap and limits the worst case immediately. The most common mistake is running the agent with a human’s broad credentials because it was the fastest way to get the prototype working, then shipping the prototype.

Q12. What is the strongest argument against environment-level safety — that it is defence in depth with a new name, and the hard part is still the policy?

CD: Largely fair, and I would not claim it is a new idea. Least privilege, sandboxing and audit logs are decades old. What is new is who the subject is: a non deterministic actor that chains many actions on its own and looks legitimate at each step, which breaks the assumptions behind per request review. And yes, the hard part is the policy: deciding what a given task actually needs is real work that cannot be outsourced to the model. My point is that this work should be done explicitly and enforced by the environment, rather than hoped for from model behaviour. That is the idea behind rev0: approval gates in front of anything irreversible, decided by the people who understand the task, not left to the model.


Agent overreach interview: Chady Dimachkie on environment safety

TechRecast Analysis (The Recast)

The Shift in Context — Agent safety has been argued mostly at the model layer: alignment, refusal behaviour, training interventions. Those address what a model will say. They do not address what an agent will do with the credentials it was given. The claim here is that the relevant control surface is the environment, and that this is where the engineering effort belongs. That is where this agent overreach interview sits.

Why This Interview Matters — The observation that overreach looks legitimate at every step is the sharpest thing in the brief. It explains why so much agent monitoring fails: each action is individually defensible, and the problem only exists in aggregate. That reframes detection from inspecting outputs to constraining what is reachable.

Technology Perspective — The technical crux is blast radius. Sandboxing, short-lived scoped credentials and network limits do not make an agent correct; they bound what an incorrect agent can reach. That is a different guarantee, and a more achievable one. The honest limit is that environmental controls protect the surroundings, not the task: an agent can be perfectly contained and still produce a wrong answer, which is why approval gates and action logs carry the other half of the load.

Business and Market Perspective — The commercial logic is that environmental controls are procurement-visible. A buyer can ask whether execution is sandboxed and credentials are short-lived; those are answerable questions in a way that “is the model safe” is not. That makes this framing useful commercially as well as technically.

Leadership Perspective

A practitioner with games, graphics, financial AI and evaluation tooling behind him is likely to reason from production incidents rather than from policy. The relevant lens is whether that produces specific, checkable controls or a restatement of standard security advice.

The Counterpoint — A fair sceptic would argue that this is conventional security engineering — least privilege, sandboxing, audit logs — applied to a new workload, and that presenting it as an insight about agents overstates the novelty; that environmental controls cannot address an agent that is authorised but wrong; and that “pausing training does not replace guardrails” sets up a policy argument nobody serious disputes. On this view, the practice is sound and the framing inflates it. This is the strongest opposing case and deserves a direct answer.

What Comes Next — Watch three signals: whether agent harnesses ship sandboxing and short-lived credentials by default, whether “stops at the edge of what it was asked” becomes a standard evaluation criterion, and whether procurement starts asking environment questions rather than model questions.

Industry Outlook — If Dimachkie is right, agent safety converges with platform security and the model-layer debate becomes one input among several. If he is wrong, model improvements reduce overreach enough that environmental controls stay optional.

Key Takeaways

1. An agent’s safety is a property of the environment it runs in, not of the model it runs on.

2. Overreach looks legitimate at every single step, which is why inspecting outputs does not catch it.

3. Sandboxing, short-lived scoped credentials, limited network, approval gates and action logs bound what a wrong agent can reach.

4. Those controls protect the surroundings, not the task: a contained agent can still be wrong.

5. Fixed-task scores say little about whether an agent stops at the edge of what it was asked to do.

6. The strongest counter-argument is that this is conventional least-privilege security with an agent label.