Enterprise AI deployment interview: how Besk Tech’s Vladimir Beskorovainyi designs the residual first
Enterprise AI is evaluated on model accuracy and governed on model risk. In this enterprise AI deployment interview, Besk Tech’s Vladimir Beskorovainyi argues both are the wrong object. And that what actually decides whether a deployment works is the design of the residual. Moreover, what happens to the cases the system cannot settle, who receives them, how fast, and under what authority.
Company Introduction
Besk Tech is an enterprise AI company founded and led by Vladimir Beskorovainyi. Its work sits where AI meets the operational reality of large organisations. The point at which a model has to hand an unsettled case to a person. And everything that follows from that handover.
That framing is deliberate. Most enterprise AI companies sell the model; Besk Tech’s public position is that the model is the easier half, and that the design of the human and procedural path around it — who receives exceptions, how quickly, and with what authority — is what actually determines whether a deployment holds up at national scale.
Executive Introduction
Vladimir Beskorovainyi is the founder and CTO of Besk Tech. He brings seventeen years in enterprise systems, with deployments across more than fifteen industries at national scale and teams of up to 130 people. Before founding Besk Tech he spent eight years securing high-traffic production services, including security engineering for a tier-one bank. He is also a published AI researcher.
His argument is precise and deliberately counter-intuitive: enterprise AI is evaluated on model accuracy and governed on model risk, and both are the wrong object. The variable that decides whether a deployment works is the design of the residual — what happens to the cases the system cannot settle, who receives them, how fast, and under what authority. This enterprise AI deployment interview tests that claim at the level of architecture, staffing and governance rather than strategy.
The claim is backed by firsthand work. A national-scale classification system he designed for incoming citizen appeals measured around 78% accuracy in research and exceeded 95% in production — and the improvement came not from a better model but from redesigning the residual path. Readers will come away with a clearer view of why the conventional sequence of tuning the model before handling exceptions produces the opposite result, why automation quietly enriches the residual until volume-based staffing models stop matching reality, and why the same logic makes agent safety a question of reversibility rather than confidence.
The wrong object
Q1. You argue enterprise AI is evaluated on model accuracy and governed on model risk, and that both are the wrong object. In one line, what is the right object?
VB: The decision, not the model. Specifically the decision the system is permitted to settle alone, and the route taken by every case it is not.
Q2. Why does the conventional sequence — tune the model, then handle exceptions — produce the opposite result from designing the residual first?
VB: Tuning first treats exceptions as leftovers, so the exception path gets designed last, by whoever is still available, under deadline pressure. The hardest cases then arrive at the least designed part of the system. Designing the residual first forces an earlier question, which is who receives a case the model cannot settle and what they are allowed to do with it. That answer constrains how much autonomy the model may have. Reverse the order and autonomy gets decided by a threshold on a metric rather than by anyone’s actual capacity to absorb the consequences.
The residual
Q3. Define the residual concretely: what counts as a case the system cannot settle, and who should receive it?
VB: A residual case is not a low-confidence case. It is a case where one of three things is true. The input falls outside the label taxonomy, which in the appeals system meant a single appeal carrying two statutory intents. The action crosses a boundary for which the system holds no grant. Or the effect cannot be undone inside the window in which the error would be noticed. It should go to a named role with the standing to settle it, never to a shared queue, because a queue has no authority.
Q4. “Under what authority” — what does that mean in practice for the person receiving a residual case?
VB: It means the recipient can decide, not merely look. They can overturn the system without escalating to anyone. Their decision becomes the decision of record. They are accountable for the quality of their review and not for the model’s behaviour. And the interface shows them what they would need in order to notice the error. If a mistake could not have been detected from what was on the screen, the failure belongs to the design and not to the reviewer.
The 78% → 95% case
Q5. Walk us through the national-scale classification system. What specifically changed in the residual path, and why did production accuracy end up higher than research accuracy?
VB: The system I want to describe is one I designed and built. It runs on the incoming correspondence of the Russian Ministry of Construction and Housing, handling appeals from every region of the country, and at the Federal Service for Alcohol Market Regulation. By the ministry’s own figures that is over 250,000 documents and more than 2 million resolutions a year.
The classification work behind it is published as arXiv:2604.03672. Three-way classification of citizen appeals, trained on ten thousand real Russian-language appeals with fifteen hundred held out. Multilingual BERT reached 82% accuracy. The deployed model was Word2Vec with an LSTM at 78%. Individual operators scored 67% against an expert-adjudicated gold standard. End-to-end handling time fell between 53 and 56% across four appeal-length bands, from an unweighted mean of 22.5 minutes to 10.25. Inference accounts for under two seconds of that.
In The Residual Path
What changed in the residual path is that the operator verifies every prediction, so the model’s job became ordering and pre-filling rather than deciding. That is also why the four-point gap to BERT was not worth paying for. The LSTM retrained cheaply on CPU, and frequent retraining mattered more to the operator’s day than four points did.
On the comparison itself I want to be careful. The post-deployment audit found more than 95% of production classifications correct, but that was judged by operators who could already see the assigned category, which is a different and easier evaluation than a held-out test. The two numbers are not comparable and the paper says so. The real reason the system outperforms the model is that its output is model plus operator, and the operator alone measured 67%. Neither component explains the result. The arrangement does.
That Redesign Cost
Q6. What did that redesign cost — in engineering, staffing or time — and what did it replace?
VB: The environment was CPU-only, which removed the expensive options before design started. The cost sat in two places. Building the residual path as a first-class part of the product rather than an afterthought, and accepting a less accurate model in exchange for retraining that was cheap enough to run often.
What it replaced was manual triage at a mean of 22.5 minutes per appeal, performed by operators working at 67% accuracy with no systematic way to see which of their decisions had been wrong.
The enriched residual
Q7. Automation removes the easy cases first. What happens to the people handling the remainder, and why do volume-based staffing and performance models fail to detect it?
VB: The people handling the remainder end up doing harder work, item by item, while their case count falls. Volume-based staffing reads the falling count and removes headcount in proportion. Performance models measure cases closed per person, so the staff left holding the difficult residue score worse than they did before automation arrived.
Nothing raises an alarm because every number on the dashboard moves in the direction it is supposed to move. Volume down, automation rate up, cost per case down. The deterioration is in the composition of what remains, and composition is not on the dashboard.
Q8. What metric would detect an enriched residual, and why doesn’t a standard one exist?
VB: Track the residual separately from the automated path, and watch two things on it. The median handling time of cases the system did not settle, measured over time rather than as a snapshot. And the rate at which human decisions on those cases are later reversed.
A standard metric does not exist for two reasons. Reporting tools aggregate both paths into a single average because they share a queue, which hides the split by construction. And no vendor has a reason to ship this number, because it looks worse precisely when the product is working well.
Agent safety
Q9. You say the gate for autonomy is reversibility rather than confidence. How do you operationalise reversibility for an agent?
VB: Reversibility is a property of the action, not of the model, so it has to be recorded per tool rather than per agent. For every tool an agent can call, write down the window in which the effect can be undone, who can undo it, and what the undo costs. Then grant autonomy action by action against those windows.
The operational rule is short. If the undo window is shorter than your detection time, there is no such thing as reviewing that action afterwards, and it needs a human before rather than after. Alongside that, name tools by consequence rather than capability, close egress by default and list destinations explicitly, issue grants scoped to one task that expire with it, and put call and spend budgets in place so a loop stops on its own.
Actor Was Permitted
Q10. Identity tells you who acted, not what that actor was permitted to do at that minute. What architecture closes that gap — and why do agent incidents pass every control and still cannot be reconstructed afterwards?
VB: The gap closes with grants that are scoped to a task and bounded in time, and with logging that records the decision rather than the API call.
Incidents pass every control because the agent used valid credentials, so every check it met was satisfied correctly. Afterwards they cannot be reconstructed because the logs hold calls and not reasons, which leaves you able to see that a thousand records were read and unable to say why. There is a third failure that gets missed. The monitoring plane is frequently writable by the process being monitored, and anything two processes can both read and write is a channel rather than a record.
I built this position into a tool I released in October, pii-roundtrip, which masks personal data before a document is sent to an external model and restores it in the answer. The mapping key never leaves the user’s machine and the tool runs with no network at all. Its evaluation is written up in a paper submitted to PeerJ Computer Science, where the files, hashes, decision rules, run plan and stopping rule were published before the first run.
Benchmark versus production
Q11. Your arXiv preprint reports a mean F₁ of 0.997 on a deliberately synthetic benchmark. What does that figure actually tell a buyer, and what does it not?
VB: It tells a buyer one thing, which is the relative ordering of models under a stated protocol, published as arXiv:2606.02004. Character n-gram logistic regression reached 0.997 mean F₁ against 0.962 for a small CNN and 0.957 for an LSTM, and it did so far more cheaply. In that comparison the CNN cost 65 times the training time of the cheapest model and the LSTM 77 times, while the character n-gram model cost 4.2 times it. For a team choosing between those options on short noisy product names, that ordering is useful.
It tells a buyer nothing about accuracy in production, and the paper states that limitation directly. The benchmark is synthetic because the production data behind it sits under non-disclosure. Around 34% of test items appear verbatim in training, and on items unseen in training the figure is 0.996. The deterministic rule stage admits only between 74.7 and 85.9% of positives, which caps the whole pipeline’s recall well below any per-model figure. And the generator corrupts text at character level, which plausibly favours character features over the alternatives it was compared against.
0.997 is a statement about my generator. It is not a statement about anyone’s catalogue.
Putting the thesis to the test
Q12. What would prove you wrong — is there a case where model accuracy really is the right object?
VB: Yes, and it is a real category rather than a concession. When an action is fully reversible inside the detection window, when the cost of a single error is bounded and small, and when volume is high enough that no human path could exist at any realistic staffing level, the model’s accuracy genuinely is the system’s accuracy. Ranking, consumer-scale spam filtering and candidate generation for retrieval all sit there.
The argument I make applies where errors land in one person’s life rather than in an average, and where somebody has to answer for the individual case.
Q13. What is the strongest argument against designing the residual first, and how do you answer it?
VB: The strongest objection is cost and timing. Designing the residual first spends scarce early effort on the exception path of a system that may never reach production, and most projects die long before the exception path matters. That objection is fair on its own terms.
My answer is that the residual path is the part you cannot retrofit. A model can be improved after launch, continuously and cheaply. Authority cannot be granted after an incident, because by then headcount has been promised, the process has been described to an auditor, and the question of who may overturn the system has become organisational rather than technical. The expensive part was never the model.
TechRecast Analysis (The Recast)
The Shift in Context
Enterprise AI has spent its first phase optimising the model and its second writing risk frameworks around it. Both treat the model as the centre of gravity. The residual argument moves the centre to the boundary — the point where the system gives up and a human takes over — and claims that is where outcomes are actually set. This enterprise AI deployment interview sits on that boundary.
Why This Interview Matters
It is rare to hear a founder argue against the metric his own industry sells on. The claim that a 78% model beat a 95% deployment because the residual was redesigned is specific, testable, and cuts against the standard procurement logic of model accuracy. That makes it worth interrogating rather than repeating.
Technology Perspective
The technical crux is the handover. Who receives an unsettled case, how fast, and with what authority are design decisions that determine throughput, error cost and auditability. The interview should press on the mechanics: queue design, escalation latency, authority limits, and what the residual path costs to run.
Business and Market Perspective
The commercial logic is uncomfortable for vendors: if the residual decides the outcome, then buying a better model is often the wrong first purchase. That reframes enterprise AI spend toward operations and staffing — and explains why so many deployments disappoint after the pilot.
Leadership Perspective
Beskorovainyi’s path — seventeen years of enterprise systems, eight of them securing high-traffic production services including a tier-one bank — is the profile of someone who learned AI in the presence of real failure modes rather than in a lab. The relevant lens is whether that operational discipline survives contact with the commercial pressure to sell models.
The Counterpoint
A fair sceptic would argue that models are improving fast enough to shrink the residual to irrelevance, and that designing elaborate human paths is investing in a problem that will solve itself. On this view, the residual matters today but is a transitional concern. This is the strongest opposing case and deserves a direct answer.
What Comes Next
Watch three signals: whether residual design becomes a standard part of enterprise AI procurement, whether a metric for enriched residual appears, and whether agent safety standards adopt reversibility as the gate rather than confidence thresholds.
Industry Outlook
If Beskorovainyi is right, enterprise AI’s centre of gravity shifts from the model to the boundary — operations, staffing and authority design — and the accuracy leaderboard matters less than it does today. If he is wrong, models absorb the residual, and the argument becomes a footnote in the transition.
Key Takeaways
1. Model accuracy and model risk are the wrong objects; the design of the residual decides whether a deployment works.
2. A 78% model outperformed in production at over 95% because the residual path was redesigned, not the model.
3. Automation removes easy cases first, so the residual enriches — work gets harder per item while volume falls, and no standard metric catches it.
4. Volume-based staffing, training and performance models stop matching reality within months of that shift.
5. For agents, the gate for autonomy is reversibility, not confidence; identity tells you who acted, never what they were permitted to do.
6. The strongest counter-argument is that improving models will shrink the residual to irrelevance, making residual design transitional.