Learning
Events
Zone
Community
Tools
Articles/Answers/Technical Experts
Publish
Yishen AI
Published on 2026-09-29 15:55:00
FDE: Taking Enterprise AI from Demo to Production
ENTERPRISE AI · FIELD NOTES
FDEs stand between AI models and enterprises' real production systems: they go on-site with customers, understand the business and data, personally handle integration and delivery, then bring frontline experience back to the product and research teams.
In the AI era, how do FDEs deploy still-immature AI capabilities into enterprises' core business, and through evals, rules, tools, trust building, and product feedback loops, take AI from prototype to production.
The FDE role emerged at Palantir in the early 2010s. Internally, Palantir called it "Delta"; the official job title was Forward Deployed Software Engineer (FDSE). The name came from the early business development team's habit of using the NATO phonetic alphabet. Delta belonged to the business development department; product development engineers who built the platform were internally called Dev and belonged to product development department E.
Before around 2016, Palantir had more FDEs than regular software engineers. After the Foundry platform launched in 2016, many FDEs transitioned back to software engineers, bringing field experience into the core product D.
Dev is "one capability, serving multiple customers"; Delta is "one customer, calling on multiple capabilities."
Palantir's definition: Delta deploys the company's software platform to customers, with the responsibility of achieving technical outcomes for the customer; success is measured by actual impact on customer goals. When a customer project lacks a feature or hits a bug, Delta can directly modify core product code, but must coordinate with the product team; larger requests go through product roadmap review E.
The Pragmatic Engineer's description: FDEs switch between two environments: being embedded in customer teams and returning to the company's core product team. They usually need to go on-site with customers, and the work environment may be a factory floor, an Airbus final assembly line, or even an air-gapped network. The role is like a startup CTO, with a small team responsible end-to-end for a high-risk project D.
According to Colin Jarvis, head of FDE at OpenAI: The FDE's job is to "eat pain and excrete product"—absorb pain, produce product A. The team's north star metric is: "You must leave with product"—you must leave with a product C.
FDEs connect customer sites with core product teams
FDEs serve customers while also bringing field experience, code, and reusable capabilities back to the core product.
| Comparison | FDE | The other role |
|---|---|---|
| Product Development Engineer | One customer, calling on multiple capabilities; success measured by whether customer goals are achieved | One capability, serving multiple customers; responsible for a component of the platform |
| Consultant | Works with customers on long-term solutions, deploying ready-made software products, with a heavier engineering workload | Provides one-off analysis, advice, or proposals |
| Solutions Architect | Writes code directly on customer infrastructure using customer tools, facing more uncertainty | More of a consulting role, usually completing MVPs and PoCs with anonymized or offline data |
- Starting in early 2025, FDE hiring has risen noticeably, driven mainly by enterprise AI integration demand D.
- Government agencies and traditional enterprises have complex processes. Deploying engineers with an entrepreneurial mindset can often bypass organizational resistance and speed up integration.
- Colin Jarvis cites MIT research showing that 95% of enterprise AI deployments have almost no quantifiable impact on the P&L. The FDE team's positioning is to make AI actually produce business results. Here, "failure" is not technology failing to run, but a lack of quantifiable value AB.
- Once the technology is ready, dedicated trust building is still needed to truly enter production.
The complete working methodology FDEs use to drive enterprise AI adoption
From on-site problems, evals, and layered rules, to minimal end-to-end delivery and product consolidation.
- OpenAI FDEs only take on problems that can save or create tens of millions to billions of dollars in value for customers A.
- Product-hypothesis-driven: Find ideal design partners for a given capability, such as customer service or clinical trial documentation.
- Research-driven: Enter technically difficult industries like semiconductors or life sciences, even when no product direction is visible yet.
- Choose core business, not edge cases. Core business makes it easier to secure organizational investment, and results are easier to measure.
| Phase | Approach |
|---|---|
| 1. Early scoping | Spend a few days on-site with the customer, map out processes, prototype with synthetic data, and set priorities. |
| 2. Validation | Confirm whether the scope is truly the most valuable; build eval sets, scale up annotation, iterate based on evals, and finally submit a report. |
| 3. Delivery | Spend a few days per week on-site with the customer, with most of the building done at the company; the goal is to deliver "the smallest end-to-end unit." |
Customers' descriptions during scoping often don't match the real state of their data and systems. We need to quickly hit those "brick walls," then adjust the scope.
Core principle: Any LLM-powered feature is not done until there is a set of evals that can verify its effectiveness AC.
- Before large-scale development, build eval sets together with the customer's domain experts.
- Build eval sets based on "expert trajectories"—the actual sequences of actions human experts take when solving problems.
- Define core metrics and baselines at the start of a project; guard against regressions with every change during development; hand over the eval framework to the customer at delivery.
Use deterministic code wherever possible, and only use LLMs where probabilistic reasoning is genuinely valuable. Critical business rules, mathematical constraints, and validation steps must be guaranteed by deterministic code.
| Layer | Responsibilities |
|---|---|
| LLM orchestration | Synthesize data from multiple sources and decide when to combine which data sources. |
| Deterministic rules | Maintain core constraints such as minimum supplier counts and full material coverage. |
| Simulator tools | Let the LLM call the same tools human analysts use, run multiple scenarios, and present trade-offs. |
| Final gatekeeping | All recommendations still pass deterministic validation. |
| Transparency | Provide reasoning explanations, tabular data, and visualizations for easy human review. |
Start with a small N=10 test in the Playground. For example, if browser automation succeeds 7–8 times out of 10, it suggests this use case has a chance of working in production and is worth further investment B.
The first customer typically yields only about 20% reusable work; after 2–3 more customers, the reusable portion reaches about 50%, after which it is handed over to a scaling team AB.
Klarna customer service → Swarm → T-Mobile complex scenario validation → Agent SDK → Agent Kit
Recurring tech stacks include: workflow orchestration, tracing and telemetry, annotated data and eval frameworks, and runtime guardrails. What is easily underestimated is the "metadata translation layer"—it sits between raw data and business logic, enabling LLMs to truly understand and use enterprise data.
- OpenAI: Share field knowledge with the research team every two weeks; report to product leads; maintain an "FDE Field notes" channel; hold quarterly bootcamps.
- Palantir: Delta can submit features or fixes to the core product; large requests go into product roadmap review.
- Only do 0 to 1: After a project is complete, hand it over to the customer's internal team or partners, and the FDE moves on to the next hard problem.
The technical pipeline took 6–8 weeks, followed by about 4 months of pilot, feedback collection, refining evals, and building advisor trust. Final adoption by wealth advisors reached 98%, and research report usage tripled. The lesson: technical feasibility doesn't mean ready to launch; trust building must be planned as a separate workstream.
Expert trajectories
The team embedded on-site to map the value chain and found engineers spent 70%–80% of their time fixing bugs and maintaining compatibility. They forked Codex, added telemetry, and turned the roughly 20-step human debugging process into an eval set. The earliest deployed department saw a 20%–30% efficiency gain, with an overall target of 50%.
Layered rules
The LLM handles cross-system orchestration and scenario trade-offs, key constraints are validated by deterministic code, and complex optimization is delegated to a simulator. The source gives no post-launch results; the 5 optimization scenarios in the demo were called "toy examples" by Jarvis.
Productization
Klarna's 400+ customer service policies pushed the team to parameterize instructions and tools, and to build an eval set for each intent. This approach was distilled into Swarm; after further validation in T-Mobile's scenario with roughly 10x the complexity, it evolved into the Agent SDK and Agent Kit.
Field delivery
The team worked with John Deere in Iowa to provide farmers with personalized advice, helping them make better use of weeding technology and reduce pesticide spraying, completing the project before the next planting season.
Feeding back into models
The customer was initially reluctant to deploy due to insufficient model performance. The FDE built eval sets and brought the data back to the research team to improve the models. The customer ultimately became the first production deployment, and the Realtime API was improved as a result.
FDEs bridge the gap from demo to production in enterprise AI
Models are only the starting point. Data, systems, evals, security, rules, trust, and organizational collaboration together determine whether something can reach production.
| Pitfall | Description |
|---|---|
| Premature generalization | Deriving enterprise problems from existing features easily leads to high-concept solutions without a clear problem. Digging deep into one specific customer problem is actually more likely to reveal common patterns. |
| Being dragged down by services revenue | Short-term services revenue can pull the organization away from product investment. FDEs must decline high-revenue projects that don't fit the strategy. |
| Treating technical readiness as launch readiness | Morgan Stanley's technical pipeline took 6–8 weeks, and trust building took another ~4 months. |
| Letting the LLM maintain hard constraints | Key rules must be enforced by deterministic code. |
| Having the LLM solve optimization directly | The more sensible approach is to have it call tools like simulators. |
| Building complex infrastructure too early | Validate first with low-cost means like Playground. |
| One person doing two jobs | An FDE is like a consultant plus platform engineer combined, and must learn to decline low-value meetings and tasks. |
| Custom build or platform | You must weigh fast delivery against the long-term evolution of the core product. |
- Using AI to solve problems that don't need AI: complex decisions, hard-to-maintain rules, or reliance on unstructured data—at least one must apply before an LLM is worth considering.
- Mistaking "the product is bad" for "AI is bad": users need not just correct answers, but a helpful, actionable product experience.
- Starting with a complex solution: begin with the simplest approach; sometimes you don't even need an agent.
- Using multi-agent too early: first fully exploit single-agent capabilities; overlapping tools tend to cause more problems than tool count.
- Ignoring tool interfaces: tool design may matter more than prompt optimization.
- Relying entirely on LLM-as-judge: the best teams still have experts manually review production outputs every day.
- No eval system: this is the common root cause of many failed AI products.
- Model upgrades changing behavior: production environments should pin versions, and new versions should first go through shadow validation.
- Only one guardrail: implement layered defenses based on read/write, reversibility, permissions, and financial impact, and define clear conditions for human handoff.
The real dividing line: a demo proves the model "can do it"; a production system must prove it remains reliable under real data, real constraints, and real accountability.
- Be clear about whether the team serves services revenue or product growth, and decline non-strategic projects accordingly.
- Choose a core business or high-value problem.
- Compare against deterministic solutions and confirm an LLM is truly needed.
- Go on-site to map real workflows; in technically complex industries, invest more time embedded on-site.
- Anticipate that customer descriptions may not match real systems, and schedule a validation phase to hit that wall early.
- Use small tests with N=10 for low-cost feasibility validation.
- Before large-scale development, build eval sets with domain experts based on expert trajectories.
- Validate hard constraints with deterministic code, and hand optimization problems to simulators.
- Start with the simplest solution, and only add complexity when it's proven to improve results.
- Design tool interfaces to the standard of "for model use."
- Set up guardrails in layers, tier tools by risk, and specify conditions for human handoff.
- Treat trust building as a separate phase, with automation increasing gradually as trust grows.
- Have someone review production data daily, and calibrate automated evals with human judgment.
- Establish a fixed mechanism for field experience to flow back into the product.
- Plan the productization pace assuming "about 20% reuse for the first customer, about 50% after 2–3 customers."
- [A] Colin Jarvis interview original video: YouTube, 2025.
- [B] ZenML LLMOps Database: Article One, Article Two, 2025.
- [C] 不二小段, "Latest Interview with the Head of OpenAI FDE," Tencent Cloud Developer Community, 2025-11-27.
- [D] Gergely Orosz, What are Forward Deployed Engineers, and why are they so in demand?, The Pragmatic Engineer, 2025-08-12.
- [E] Palantir, Dev versus Delta, 2019-04-08.
- [F] Anthropic, Building effective agents, 2024-12-19.
- [G] OpenAI, A practical guide to building agents, 2025.
- [H] Chip Huyen, Common pitfalls when building generative AI applications, 2025-01-16.
- [I] LinkedIn Engineering, Musings on building a Generative AI product, 2024-04-25.
- [J] Eugene Yan et al., What We've Learned From A Year of Building with LLMs, 2024-06-08.
- [K] Hamel Husain, Your AI Product Needs Evals.
Note: The figures and cases in this article are presented according to the sources listed; secondary citations and data lacking production results are clearly marked.
Originality statement: This article was published by the author with authorization in the Tencent Cloud Developer Community and may not be reproduced without permission.
For infringement issues, please contact cloudcommunity@tencent.com for removal.
Table of Contents