Part III: Deployment Strategy

Chapter 9: Foundation Models — Data Strategies for Generalist Policies

Written: 2026-06-18 Last updated: 2026-07-22

Overview: The Question Before Generality

Robot foundation models promise that one policy can be reused across more tasks, environments, and embodiments. In manufacturing, however, generality is not only a model property. It also depends on who authors the scene, calibrates physical parameters, records executable actions, filters generated data, links quality outcomes to episodes, and approves updates after deployment. The question for this chapter is therefore not which model is strongest. It is which chain of data responsibilities makes a generalist policy testable in a production cell.

NVIDIA's physical-AI strategy is an important case because it places many of those responsibilities in one connected ecosystem. OpenUSD and Omniverse address scene authoring and interchange. Isaac Sim provides robot and sensor simulation. Isaac Lab orchestrates learning and evaluation. Newton extends contact and deformable physics. Cosmos models visual appearance, behavior, actions, and possible futures. GR00T is the foundation-policy layer. This is better understood as a connected data factory for different reality gaps than as one giant simulator. Integration is a capability; it is not evidence that a particular process has achieved the required accuracy, safety, or yield.

The historical arc begins with learning a task from demonstrations supplied for one robot [1]. RoboNet and Open X-Embodiment then pooled data across robots [2], [3]. Octo, OpenVLA, and π0 attempted to turn such mixtures into reusable policy priors [4], [5], [6]. The 2025–2026 frontier adds another transition: rather than collecting only real robot demonstrations, it combines explicit simulation, human video, video world models, inverse dynamics, and selective intervention to increase coverage and information density [7], [8].

After reading this chapter, you will be able to... - distinguish peer-reviewed or preprint experiments, open artifacts, official documents and videos, partner demonstrations, and independently replayed deployment evidence; - explain the separate responsibilities and non-guarantees of OpenUSD/Omniverse, Isaac Sim, Isaac Lab, Newton, Cosmos, and GR00T; - compare generalist policies by data access, embodiment, action interface, openness, evaluation, and deployment evidence; - identify where simulation and world models may reduce task-specific teleoperation and why contact, failure, and quality data remain real-cell responsibilities; and - design calibration, replay, safety, and quality authority for production model releases.

The Evidence Ladder: Do Not Promote a Demo Into a Performance Result

Comparisons of generalist models must separate peer-reviewed or preprint experiments, open artifacts, first-party reports, and results that outside teams can replay under the same conditions. OpenVLA and Octo expose substantial portions of their code, weights, and adaptation pathways, but openness does not establish performance on an industrial contact task [5], [4]. The GR00T N1 paper and released model provide inspectable evidence about an architecture and its reported evaluations; author-controlled results cannot be converted into long-duration metrics across unrelated factories [10].

This chapter uses four evidence levels. Academic evidence reports a setup, comparison, and denominator in a paper or preprint; a preprint is still labeled as such because it has not automatically passed archival peer review. Open-artifact evidence consists of code, weights, formats, and replay scripts. It improves inspectability without eliminating hidden training data or hardware differences. Official capability evidence consists of documentation, technical blogs, release announcements, and official videos. It establishes what a supplier says is available or intended, not a universal performance claim. Independent deployment evidence reports yield, stops, rework, damage, interventions, and update deltas measured by an outside operator in a real cell. The fourth level remains the least available.

Language should follow the level. An official page can support the statement that an interface is offered. A partner demonstration can support the statement that a workflow was shown in that configuration. Claims that contact accuracy improved, real data fell substantially, or production readiness was achieved require a baseline, a sample size, a task distribution, and a failure definition. A curated success clip and a customer's continuous production record are not equivalent observations.

Author-created draft positioning companies by generalist model scope and production-data ownership

Figure 9.1. An author-created editorial map using generalist model scope and production-data ownership as two reading axes. Point placement is a qualitative hypothesis, not a validated company ranking or a responsibility map of NVIDIA components. Source: author-created.

This ladder accelerates adoption instead of blocking it. Capability-only items belong in exploratory tests. Open artifacts can become internal baselines. Independently validated items can advance toward limited deployment. A team does not have to discard an interesting technology because factory evidence is not yet available, and it does not have to surrender release authority because a demonstration is compelling.

Historical Transition: From One-Task Demonstration to Cross-Robot Pretraining

Early learning from demonstration converted an operator trajectory into a mapping between state and action. It reduced manual reward design but suffered from compounding error outside the demonstrated state distribution [1]. DAgger addressed this by requesting expert labels on states induced by the learner, changing the question from how much expert data exists to where expert attention is most informative [9]. In physical systems, that idea is constrained by safety: a factory cannot deliberately visit every dangerous state merely to request a correction.

RoboNet pooled approximately fifteen million frames across seven robot platforms and several laboratories, making cross-robot video learning concrete [2]. Open X-Embodiment expanded the collaboration to 22 robots and a large set of skills and tasks [3]. These milestones also exposed a structural loss. Normalizing cameras, frequencies, joints, grippers, and coordinate frames can discard embodiment-specific contact meaning. More trajectories do not automatically create a common semantics for actions.

Octo and OpenVLA became open reference points for turning pooled robot data into adaptable policies. Octo reported pretraining on 800,000 trajectories and designed swappable observation and action heads; OpenVLA joined a vision-language backbone with robot action tokens and released an adaptation path [4], [5]. π0 used flow matching to generate continuous action chunks and targeted higher-frequency, dexterous control [6]. These works support a real achievement: one pretrained policy family can be a starting point for multiple robots and tasks. They do not establish zero-data deployment in a new production process.

Cross-embodiment pretraining broadens policy priors, but reported task success does not establish customer-specific quality, safety, or lifecycle performance. π0.5's long-horizon generalization in unseen homes and π0.7's steerable generalist evaluations are significant research evidence within their stated protocols [11], [12]. They do not include every reflective material, tolerance, worn tool, replaced sensor, PLC interaction, or shift change that defines a production cell. Broad pretraining may reduce the cost of starting from scratch; it does not replace evidence of process responsibility.

The following axes are therefore more useful than headings organized by company name.

Comparison axis Open generalist-policy path Human and multi-robot data path Vertically integrated humanoid path Manufacturing interpretation
Data access Open data plus partially private mixtures; adaptation path is inspectable Human video, multi-institution collections, or proprietary capture at scale Internal teleoperation and fleet data dominate Export and replay rights for customer-cell episodes matter more than nominal corpus size.
Embodiment Adapters span arms and grippers Common representations or common end effectors reduce mismatch Policy and one humanoid body are co-optimized Breadth across robots and precision on one body are different benefits.
Action interface Tokens, continuous actions, and chunks can be compared Human motion is retargeted or inferred by inverse dynamics Low-level controller boundaries may remain undisclosed Rate, latency, force control, and safety-controller authority must be explicit.
Openness More papers, code, and weights Scale may be visible while collection and cleaning remain partial Strong product integration with closed internal data Openness helps internal baselines; integration may reduce deployment work.
Evaluation Laboratory tasks with author-defined denominators Cross-scene, cross-robot, or human-to-robot transfer Selected product and partner demonstrations Re-evaluate all candidates with the same task and failure taxonomy.
Deployment evidence External reproduction is possible, but production records are rare Data scale rarely connects to production yield Field demonstrations are increasing, continuous metrics remain scarce A plant replay set and signed release history are the final comparison.

Openness and capability are not the same axis. A weaker open model may still give a manufacturer a valuable validation language. A stronger closed model may fail to provide adequate rights over customer data and updates. A robust strategy runs open baselines and integrated candidates against the same replay evidence instead of treating either category as inherently superior.

GEN-1's company-reported mastery, speed, and low-data adaptation claims belong in this table as official evidence for a vertically integrated candidate [28]. Its disclosed task definitions and limitations are useful market signals, but they are not independent production evidence until an outside factory replays the same criteria.

pi0 framework joining Internet and robot-data mixtures to a VLM backbone and action expert

Figure 9.2. π0 Figure 3 connects an Internet-pretrained VLM, a pretraining mixture containing self-collected robot data and OXE, and a 300M-parameter action expert that generates continuous actions for multiple robots. It is a paper architecture diagram of training inputs and generated policy actions, not measured factory-quality or safety evidence. Source: Black et al. (2024), arXiv:2410.24164, Figure 3, p. 4.

The NVIDIA Responsibility Map: Seven Contracts, Not One Stack

The NVIDIA-centered physical-AI stack assigns authoring, simulation, training, learned augmentation, foundation-policy learning, and real-cell assurance to different components [13], [14], [15]. Erasing the boundaries makes Omniverse look like a physics engine, Cosmos like a force sensor, or GR00T like a certified safety controller. Preserving them turns each layer into a contract with inputs, outputs, calibration ownership, and an evidence level.

Responsibility Main layer Inputs Outputs Gap addressed What it does not guarantee
Scene authoring and interchange OpenUSD and Omniverse libraries Geometry, materials, hierarchy, semantics, variants Composable scenes and assets Fragmentation across tools and organizations Correct real friction, tolerance, or sensor error
Robot and sensor simulation Isaac Sim Robot model, scene, sensors, physics settings State, image, and synthetic sensor data Repetition without occupying real hardware Agreement with every dynamic and failure mode
Learning and evaluation orchestration Isaac Lab Environment, observations, actions, reward, randomization Parallel training and evaluation records Cost of large repeated experiments A good reward, correct process model, or hardware safety
Contact and deformation physics Newton and supported backends CAD, mass, friction, elasticity, control Collision, contact, and deformation states Limited insertion, cable, and soft-body representation Automatic recovery of real parameters from bad inputs
Visual, behavioral, and action modeling Cosmos family Video, language, action, structural conditions Transferred video, futures, inferred actions, scores Insufficient appearance and behavior coverage Ground-truth hidden force or formal safety proof
Foundation-policy learning GR00T family and training blueprints Real and synthetic trajectories, language, robot state Robot action policy Reuse across tasks and embodiments Customer yield, lifetime, or regulatory acceptance
Deployment assurance Real-cell operating system Hardware rollouts, quality results, stops, interventions Approval, rejection, rollback, and retraining data Residual error outside every simulator Quality authority that a platform vendor can own by default

Scene Contract: OpenUSD and Omniverse

OpenUSD is fundamentally a way for tools to compose and exchange geometry, materials, variants, hierarchy, and metadata [13]. Omniverse libraries and connectors place rendering, physics, and data-access functions inside existing 3D and industrial applications [16]. This can reduce duplicate asset work, preserve design changes as scene variants, and reuse sensor placements and semantic labels across teams.

A simulation-ready asset is not synonymous with a reality-equivalent digital twin. A collision mesh can omit a manufacturing tolerance. A named material can carry the wrong friction coefficient. A semantic label can exist without a quality specification. OpenUSD defines how values travel; it does not certify that the values match the plant. Each manufacturing asset therefore needs provenance, measurement date, uncertainty, calibration target, and an owner.

This distinction matters in the Blender workflow supplied with the user brief. Bringing Omniverse libraries into a familiar authoring tool can reduce handoffs and let an agent identify missing labels or physics properties. It also creates a new review obligation. A machine-filled value is a candidate parameter, not a measured one. The correct operational benefit is faster scene preparation and reusable regression assets, not automatic physical truth.

Experiment Contract: Isaac Sim and Isaac Lab

Isaac Sim places articulations, rigid bodies, cameras, lidar, sensor models, and synthetic-data generation in a GPU-accelerated scene. A 2026 research survey of Isaac Sim highlights parallel experimentation and synthetic data while retaining open-world physics and usability as unresolved challenges [14]. That is paper-level evidence about the platform's architecture and usage patterns, not a direct experiment on every manufacturing cell. Its proximity to the supplier ecosystem should also remain visible.

Isaac Lab is not a new law of physics. It is a framework for defining environments, observations, actions, rewards, curricula, randomization, imitation learning, reinforcement learning, and evaluation [17]. Its most important manufacturing capability is not merely thousands of parallel environments. It is the ability to repeat a policy under frozen initial conditions and a frozen failure definition. The same machinery can also optimize a bad reward faster. A simulation that gives perfect object pose unavailable to the hardware, or declares success after a dropped part, can produce an impressive but unsafe policy.

An Isaac Lab result should consequently be released as an experiment bundle, not a lone policy file. The bundle includes scene version, physics backend and version, randomization distribution, observation and action schemas, reward, reset logic, failure taxonomy, seeds, and checkpoint. This makes it possible to distinguish an improvement in the policy from a change in the simulator.

Physics Contract: Newton Adds Expressivity, Not Automatic Calibration

Newton is positioned as an extensible GPU physics engine built with Warp and OpenUSD, connecting rigid-body, contact, friction, deformable, and differentiable computation to robot learning [18]. The direction is especially relevant to cable insertion, cloth or rubber deformation, torsional friction, and distributed contact surfaces that are poorly represented by simple collision models.

Greater expressivity requires more real parameters. If CAD tolerances, mass distribution, friction, elasticity, damping, motor delay, end-effector compliance, or force-sensor response are wrong, a detailed solver can be precisely wrong. Chebotar et al.[2] adapted simulation randomization using real executions for exactly this reason. A simulator is a hypothesis that must be updated with hardware evidence, not an original source that supersedes reality.

Newton's project pages and partner demonstrations establish first-party capability and integration direction. They do not independently establish long-duration sim-real correlation in contact-rich factories, drift after sensor replacement, or prediction under tool wear. A manufacturing test should compare contact onset, peak wrench, slip direction, insertion depth, and failure-type ranking under matched initial conditions. Visual polish is not a contact metric.

Contract unit Source of truth Comparative evidence retained before release
Scene and asset Measured geometry, material, tolerance, and sensor placement Versioned scene with provenance, uncertainty, and calibration target
Experiment and physics Frozen observation, action, failure definitions, and real contact Matched real-sim trajectories and mismatch records
Learned amplification Real seed data and executable actions screened through inverse dynamics Provenance for generation, filtering, pseudo-actions, and real replay

Cosmos: Separate Visual Amplification, Behavioral Amplification, and Policy

Treating Cosmos as one simulator obscures several responsibilities. Conditional transfer models can preserve depth, segmentation, edges, or keypoints while changing illumination, texture, and background. This primarily addresses the visual domain gap. It is distinct from Newton's role in computing contact and deformation.

Cosmos Predict and video world models such as DreamGen and DreamDojo generate possible futures from a current scene, action, or instruction. They can broaden behavior coverage, help rank policies, and generate synthetic trajectories [7], [8]. DreamGen creates videos of new behaviors from real seeds and attaches pseudo-actions using an inverse model. This may extend beyond the scenes for which accurate CAD assets exist. A plausible video, however, does not show that an action is executable or that the contact force is correct. The pseudo-action contains both generation error and inverse-dynamics error.

Reasoning or filtering models such as Cosmos Reason can reject obvious instruction failures, object disappearance, and penetration. They are probabilistic critics that improve a generated dataset, not formal verifiers of mechanics. They may remove visibly impossible clips without detecting a subtle overload, out-of-tolerance insertion, material damage, or sensor saturation.

Cosmos 3 introduces an omnimodal mixture-of-transformers architecture for language, image, video, audio, and action [15], [20]. Action-conditioned post-training can be described through three modes. Forward dynamics predicts future video from the current scene and an action. Inverse dynamics infers a possible action trajectory from observed video. Policy mode jointly generates an action chunk and an anticipated future. The unification is an important research direction because it reduces interfaces between a policy and a world model. It does not make an unobserved torque, pressure distribution, or tactile event uniquely identifiable from video. Multiple actions can produce visually similar outcomes.

The two NVIDIA videos supplied by the user establish first-party capability and workflow context only, unless a linked primary experiment supports a stronger statement [21], [22]. The Blender video demonstrates an authoring path in which Omniverse libraries and agent tooling add or check physics, sensor, and semantic properties before a USD handoff. It supports the possibility of reusable scene preparation; it does not show that agent-filled values match a real cell or improve real-policy performance.

The 100 teleoperated demonstrations mentioned in the Cosmos 3 action-prediction video must be read the same way. They are a tutorial configuration for a small pick-and-place adaptation in LeRobot format [21]. They are not a measured sample-efficiency law across tasks, embodiments, sensors, and target success levels. The count also excludes the upstream data used to pretrain the world model, failure collection, scene calibration, safety testing, and real-cell quality validation.

The accurate answer to whether teleoperation decreases is therefore conditional. Task-specific successful demonstrations may decrease when the pretrained visual and behavioral prior fits the task, contact is simple, and inverse-model errors can be screened by real replay. When tight tolerance, force, tactile feedback, or high failure cost dominates, failure, recovery, and quality episodes may become more valuable rather than less necessary. Human labor moves from repeating nominal trajectories toward judging exceptions and correcting failures.

The GR00T Data Factory: Mix Three Sources Without Losing Provenance

GR00T N1 is a public research reference for a humanoid foundation policy that combines higher-level visual-language processing with a lower-level action system [10]. In the broader NVIDIA strategy, the critical distinction is how data is produced. Real teleoperation, explicit physics simulation, and learned video generation enter one training pipeline with different error modes.

The first path is real seed data: teleoperation, autonomous rollouts, interventions, joint state, force, tactile signals, and quality outcomes. It is expensive but directly joins executable action to process result. The second path is physics-based amplification. Isaac Sim and Isaac Lab vary object pose, scene, robot, trajectory, lighting, and sensor noise; Newton or another backend can extend contact representation. GR00T-Mimic-style workflows are suited to generating variants of a known skill from a small number of demonstrations. The third path is learned amplification. Cosmos and GR00T-Dreams-style workflows generate new scene or behavior video, filter it, and infer pseudo-actions.

Provenance is the key property of the mixture. Every episode should state whether it is real, physics-simulated, generated from video, inferred by inverse dynamics, or corrected by a person. Without those labels, the team cannot attribute an improvement or diagnose a generated failure. It should also retain the confidence of the success label, missing sensor channels, and the mixture weight used for training.

The connected data factory competes with and complements other generalist paths. OpenVLA, Octo, and π0 provide inspectable baselines for model and action representation. Human-video paths provide scalable task semantics without occupying robots. Vertically integrated systems jointly optimize body, controller, and policy. The NVIDIA ecosystem's distinguishing scope is its attempt to connect scene, simulation, learning, world models, and policy. The benefit is lower interface cost. The risk is allowing one supplier's vocabulary to define scene truth, model performance, and deployment evidence at once.

Figure AI concept graphic comparing three Helix task-scaling curves

Figure 9.3. Figure AI's Helix announcement graphic plots conceptual task-count-versus-time curves for three scaling regimes. Because the axes provide no values, samples, or uncertainty, it is first-party workflow context—not a measured scaling law or independent factory-performance result. Source: Figure AI, 2025, Figure 1.

Counterevidence and Limits: What World Models Do Not Observe

First, inferred actions do not automatically recover force and tactile state. ForceVLA and Tactile-VLA report benefits from adding contact channels in small contact-rich task suites, exposing a missing variable in vision-centered generalist policies [23], [24]. These are also preprints with limited tasks and sensor configurations; they do not provide a production lifecycle study of contamination, replacement, or drift. Their important counterpoint is not that all VLAs fail. It is that visually similar states may require different contact responses.

Second, a synthetic trajectory count is not a count of independent useful experiences. If a generator repeats the same bias, nominal data volume grows without covering new failures. DreamGen demonstrates the promise of novel behavior generation while retaining inverse-model error, substantial generation and filtering cost, and limited complex dexterity [7]. DreamDojo opens a frontier in which broad human interaction video supports policy evaluation and planning, but rare fast events, nuanced failures, multiple views, and optimistic success prediction still require validation [8].

Third, policy-ranking correlation is different from absolute simulation accuracy. A simulator may rank policy A above B correctly while misestimating both real success rates and force traces. It can be useful for candidate selection without being qualified for product release. A photorealistic renderer may still rank contact failures incorrectly. Rendering fidelity, sensor fidelity, dynamics fidelity, and task-outcome correlation require separate measurements.

Fourth, generalist claims can hide long-horizon compounding failure. High stage success multiplied over dozens of steps can produce weak whole-task completion. Contamination, people entering the cell, mixed parts, bad resets, and network delay are rare in research data but recurring in operation. Understanding an instruction and executing safe stop, recovery, and rework are different capabilities.

Fifth, openness has limits. Released code and weights do not provide full reproduction if the original mixture, filtering decisions, compute, and failed experiments remain hidden. A closed system can still build strong deployment evidence by exporting customer-cell episodes and release history and permitting an independent audit. The sharper question is not simply open versus closed. It is who can rerun and falsify each claim.

Manufacturing Interpretation: Spend Real Data Where It Is Most Informative

The manufacturing objective is not zero real data. It is to generate cheap variation computationally and collect contact, failures, and quality outcomes selectively where only reality reveals them.

Task family What simulation or world models can amplify What remains real-cell evidence First release metrics
Structured pick-and-place Pose, lighting, background, camera view, reach path Slip, double pick, reflective or transparent parts, gripper wear Success plus drop, damage, and retry rates
Case packing and arrangement Bimanual collision, order, target layout, workspace Paper deformation, sticking, rebound, packaging variation Throughput, crushing, pose error, intervention
Insertion and fastening CAD variants, approach, force strategy candidates, tolerance distributions Friction, threads, torque-angle, tool wear, jamming Insertion depth, peak force, defect and damage rates
Cable, film, and soft goods Coarse deformation and path candidates Material elasticity, adhesion, wrinkles, slip, sensor drift Final shape, damage, and recovery success
Long-horizon work Instruction and sequence variation, scene and object diversity Exceptional transitions, wrong objects, rework, shift change End-to-end completion, stage failures, recovery time
Safety and quality Scenario enumeration and regression support Real hazards, product outcomes, stop and approval records Zero limit violations and traceable disposition

Implementation starts with synchronized logging: RGB and depth, commanded and measured joint state, wrist force and torque, gripper state, object and process identifiers, equipment state, task outcome, defect code, intervention, and cycle time. This common record lets Newton calibration, Isaac Lab observations and rewards, Cosmos post-training, GR00T adaptation, and real replay refer to the same process truth.

A small seed set can then produce a baseline policy. Instead of collecting only more successful repetitions, the manufacturer runs constrained autonomous trials and prioritizes interventions near slip, jam, wrong approach, or quality rejection. This follows DAgger's historical insight that labels at learner-induced states carry high information density [9]. Safety monitoring and residual control must still prevent deliberate visits to unacceptable states.

Simulation and generated data expand around that real seed. Each synthetic episode receives its generation route and confidence. Ablations on the real failure set determine whether it helps. If a mixture raises average success while increasing jams for one part family, the average must not hide the regression. Data weights should be adjusted by task, object family, and failure type.

The same principle changes procurement. A general model card is insufficient. The manufacturer should request the episode schema, action rate, controller boundary, safety override path, synthetic-data provenance, replay procedure, checkpoint history, and examples of explained failures. The request applies equally to open policies, an NVIDIA-centered stack, and a vertically integrated competitor.

Real-Cell Release Authority: The Last Model Is the Quality System

Independent factory-scale validation remains a separate evidence requirement from vendor demonstrations, generated trajectory counts, and model-card benchmarks [25], [12], [26]. Frontier generalist and large-data research is a strong signal of rapid progress. It often still depends on private teleoperation, synthetic data, fleet access, and in-house benchmarks. A poorly chosen post-training recipe can degrade an already strong policy. A bounded laboratory task suite or a supplier-defined mastery threshold does not establish continuous operation across factories.

Production approval needs four loops. The calibration loop compares real trajectories, contacts, and sensor responses with simulation and updates scene, material, and delay distributions. The replay loop freezes both successes and failures—slip, jam, overload, wrong object, missing sensor, human entry, and quality reject—and runs every checkpoint against them. The safety loop leaves force, speed, workspace, equipment interlock, emergency stop, and human takeover authority outside the learned policy. The quality loop joins inspection, rework, scrap, and field defects back to the episode.

Release proceeds through offline replay, simulated stress testing, shadow execution, a guarded test cell, a limited canary deployment, and expansion. Every stage has numerical pass and rollback criteria. An update that raises average success but violates peak force or increases damage on one part family must be rejected. A plant owner must be able to freeze the old checkpoint and compare it even when a supplier publishes a new model.

Data rights are part of release authority. A manufacturer must export its own video, sensor, action, and quality labels; distinguish real from synthetic episodes; and trace which checkpoint learned from which data. The supplier need not disclose every upstream pretraining asset, but the customer must be able to audit its own outcomes and update history. Without these rights, the data flywheel does not belong to the factory.

A cell evidence packet is more useful than a one-page model card. It records scene, physics, and policy versions; sensor calibration date; training provenance ratios; replay results; real-test denominator; failure confusion matrix; safety-limit violations; quality outcomes; human intervention time; approver; and rollback target. Applying the same packet to open models, the NVIDIA ecosystem, and other integrated candidates creates a stable comparison even as vendor terminology changes.

Independent validation is unresolved rather than impossible. Public research already supports bounded claims about cross-robot pretraining, synthetic augmentation, learned world models, and force-aware control. What is largely absent is a multi-site, long-duration study that exposes task denominators, interventions, maintenance, quality disposition, and update history under a shared protocol. That absence should remain explicit in technical and investment decisions instead of being filled by inference from a polished workflow.

Foundation-model deployment gate from pretraining to real-cell approval

Figure 9.4. Foundation-model deployment gate from pretraining to real-cell approval. This diagram is not a performance claim; it summarizes the operational chain that carries capability evidence through replay and quality decisions into release authority. Source: author-created SVG.

Conclusion: Expand Coverage With Compute, Close Responsibility With Reality

A foundation-model data strategy does not eliminate teleoperation. It replaces or amplifies some repetitive nominal demonstrations with open pretraining, explicit simulation, world models, and inverse dynamics, then concentrates human time on contact calibration, failure recovery, and quality judgment. OpenUSD is the scene language; Isaac Sim is the robot and sensor testbed; Isaac Lab is the learning and evaluation orchestrator; Newton is the contact representation layer; Cosmos is a visual and behavioral generator; GR00T is a policy-learning layer. None of them owns final real-cell quality authority.

The strength of an NVIDIA-centered strategy is the ability to reduce interface friction from scene to policy. Its weakness is that integration can make capability descriptions, research results, partner demonstrations, and production evidence sound like one continuous proof. The most important technology is therefore not another model. It is the operational discipline that preserves evidence level, episode provenance, calibration history, replay sets, and release authority.

Chapter 10 moves this principle into deployed data flywheels. It examines how production robotics teams that begin with narrower operating domains accumulate customer-cell logs, exceptions, remote interventions, and quality outcomes, and how ownership of those records shapes update authority. If Chapter 9 maps the data factory that builds a generalist policy, Chapter 10 maps the operating system in which that policy fails, is corrected, and earns release approval every day.

References

  1. Argall, Brenna D. et al. (2009). A Survey of Robot Learning from Demonstration. Robotics and Autonomous Systems.
  2. Dasari, Sudeep et al. (2019). RoboNet: Large-Scale Multi-Robot Learning. CoRL.
  3. Open X-Embodiment Collaboration (2023). Open X-Embodiment: Robotic Learning Datasets and RT-X Models. IEEE ICRA 2024.
  4. Ghosh, Dibya et al. (2024). Octo: An Open-Source Generalist Robot Policy. RSS 2024.
  5. Kim, Moo Jin et al. (2024). OpenVLA: An Open-Source Vision-Language-Action Model. CoRL 2024.
  6. Black, Kevin et al. (2024). π0: A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint. #2 · KO
  7. Jang, Joel et al. (2025). DreamGen: Unlocking Generalization in Robot Learning through Neural Trajectories. arXiv preprint.
  8. Gao, Shenyuan et al. (2026a). DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos. ICML 2026 Spotlight / arXiv.
  9. Ross, Stéphane et al. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS.
  10. Bjorck, Johan et al. (2025). GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv preprint.
  11. Physical Intelligence et al. (2025). π0.5: A Vision-Language-Action Model with Open-World Generalization. arXiv preprint.
  12. Physical Intelligence (2026). $π_{0.7}$: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities. arXiv preprint. #62 · KO
  13. Alliance for OpenUSD (2026). OpenUSD Specifications and Alliance Governance. Official specification and governance site.
  14. Gao, Sicong et al. (2026b). NVIDIA Isaac Sim: Enabling Scalable, GPU-Accelerated Simulation for Robotics. arXiv preprint.
  15. Aditi et al. (2026). Cosmos 3: Omnimodal World Models for Physical AI. arXiv technical report.
  16. NVIDIA Omniverse (2026). NVIDIA Agent Toolkit Expands With New Omniverse Libraries. Official announcement.
  17. Mittal, Mayank et al. (2025). Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv preprint.
  18. Newton Project (2026). Newton Physics Engine. Official project page.
  19. Chebotar, Yevgen et al. (2019). Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience. IEEE ICRA.
  20. NVIDIA Cosmos (2026). Develop Physical AI Reasoning, World, and Action Models with NVIDIA Cosmos 3. Official technical blog.
  21. NVIDIA Video (2026a). How to Post-Train NVIDIA Cosmos 3 for Robot Action Prediction. Official tutorial video.
  22. NVIDIA Video (2026b). Bringing Agent-Ready Simulation Into Blender. Official workflow video.
  23. Yu, Jiawen et al. (2025). ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation. arXiv preprint.
  24. Huang, Jialei et al. (2025). Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization. arXiv preprint.
  25. Kim, Dongyoung et al. (2026). RLDX-1 Technical Report. arXiv preprint. #67 · KO
  26. Zheng, Ruijie et al. (2026). EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data. arXiv preprint.
  27. Figure AI (2025). Helix: A Vision-Language-Action Model for Generalist Humanoid Control. Official company research post.
  28. Generalist Team (2026). GEN-1: Scaling Embodied Foundation Models to Mastery. Official company research post. #24 · KO
  29. Chelsea Finn et al. (2016). Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization. International Conference on Machine Learning.
  30. Michael Janner et al. (2019). When to Trust Your Model: Model-Based Policy Optimization. NeurIPS.
  31. Oliver Kroemer et al. (2021). A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. JMLR.
  32. Fanbo Xiang et al. (2020). SAPIEN: A SimulAted Part-based Interactive ENvironment. CVPR.
  33. Mohit Shridhar et al. (2022). CLIPort: What and Where Pathways for Robotic Manipulation. CoRL.
  34. Songming Liu et al. (2024). RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation. arXiv primary preprint (2024-10-10).
  35. Pertsch, Karl et al. (2024). Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models. Primary publication.