Chapter 9: Foundation Models — Data Strategies for Generalist Policies
Overview: The Question Before Generality
Robot foundation models promise that one policy can be reused across more tasks, environments, and embodiments. In manufacturing, however, generality is not only a model property. It also depends on who authors the scene, calibrates physical parameters, records executable actions, filters generated data, links quality outcomes to episodes, and approves updates after deployment. The question for this chapter is therefore not which model is strongest. It is which chain of data responsibilities makes a generalist policy testable in a production cell.
NVIDIA's physical-AI strategy is an important case because it places many of those responsibilities in one connected ecosystem. OpenUSD and Omniverse address scene authoring and interchange. Isaac Sim provides robot and sensor simulation. Isaac Lab orchestrates learning and evaluation. Newton extends contact and deformable physics. Cosmos models visual appearance, behavior, actions, and possible futures. GR00T is the foundation-policy layer. This is better understood as a connected data factory for different reality gaps than as one giant simulator. Integration is a capability; it is not evidence that a particular process has achieved the required accuracy, safety, or yield.
The historical arc begins with learning a task from demonstrations supplied for one robot [1]. RoboNet and Open X-Embodiment then pooled data across robots [2], [3]. Octo, OpenVLA, and π0 attempted to turn such mixtures into reusable policy priors [4], [5], [6]. The 2025–2026 frontier adds another transition: rather than collecting only real robot demonstrations, it combines explicit simulation, human video, video world models, inverse dynamics, and selective intervention to increase coverage and information density [7], [8].
After reading this chapter, you will be able to... - distinguish peer-reviewed or preprint experiments, open artifacts, official documents and videos, partner demonstrations, and independently replayed deployment evidence; - explain the separate responsibilities and non-guarantees of OpenUSD/Omniverse, Isaac Sim, Isaac Lab, Newton, Cosmos, and GR00T; - compare generalist policies by data access, embodiment, action interface, openness, evaluation, and deployment evidence; - identify where simulation and world models may reduce task-specific teleoperation and why contact, failure, and quality data remain real-cell responsibilities; and - design calibration, replay, safety, and quality authority for production model releases.
The Evidence Ladder: Do Not Promote a Demo Into a Performance Result
Comparisons of generalist models must separate peer-reviewed or preprint experiments, open artifacts, first-party reports, and results that outside teams can replay under the same conditions. OpenVLA and Octo expose substantial portions of their code, weights, and adaptation pathways, but openness does not establish performance on an industrial contact task [5], [4]. The GR00T N1 paper and released model provide inspectable evidence about an architecture and its reported evaluations; author-controlled results cannot be converted into long-duration metrics across unrelated factories [10].
This chapter uses four evidence levels. Academic evidence reports a setup, comparison, and denominator in a paper or preprint; a preprint is still labeled as such because it has not automatically passed archival peer review. Open-artifact evidence consists of code, weights, formats, and replay scripts. It improves inspectability without eliminating hidden training data or hardware differences. Official capability evidence consists of documentation, technical blogs, release announcements, and official videos. It establishes what a supplier says is available or intended, not a universal performance claim. Independent deployment evidence reports yield, stops, rework, damage, interventions, and update deltas measured by an outside operator in a real cell. The fourth level remains the least available.
Language should follow the level. An official page can support the statement that an interface is offered. A partner demonstration can support the statement that a workflow was shown in that configuration. Claims that contact accuracy improved, real data fell substantially, or production readiness was achieved require a baseline, a sample size, a task distribution, and a failure definition. A curated success clip and a customer's continuous production record are not equivalent observations.
Figure 9.1. An author-created editorial map using generalist model scope and production-data ownership as two reading axes. Point placement is a qualitative hypothesis, not a validated company ranking or a responsibility map of NVIDIA components. Source: author-created.
This ladder accelerates adoption instead of blocking it. Capability-only items belong in exploratory tests. Open artifacts can become internal baselines. Independently validated items can advance toward limited deployment. A team does not have to discard an interesting technology because factory evidence is not yet available, and it does not have to surrender release authority because a demonstration is compelling.
Historical Transition: From One-Task Demonstration to Cross-Robot Pretraining
Early learning from demonstration converted an operator trajectory into a mapping between state and action. It reduced manual reward design but suffered from compounding error outside the demonstrated state distribution [1]. DAgger addressed this by requesting expert labels on states induced by the learner, changing the question from how much expert data exists to where expert attention is most informative [9]. In physical systems, that idea is constrained by safety: a factory cannot deliberately visit every dangerous state merely to request a correction.
RoboNet pooled approximately fifteen million frames across seven robot platforms and several laboratories, making cross-robot video learning concrete [2]. Open X-Embodiment expanded the collaboration to 22 robots and a large set of skills and tasks [3]. These milestones also exposed a structural loss. Normalizing cameras, frequencies, joints, grippers, and coordinate frames can discard embodiment-specific contact meaning. More trajectories do not automatically create a common semantics for actions.
Octo and OpenVLA became open reference points for turning pooled robot data into adaptable policies. Octo reported pretraining on 800,000 trajectories and designed swappable observation and action heads; OpenVLA joined a vision-language backbone with robot action tokens and released an adaptation path [4], [5]. π0 used flow matching to generate continuous action chunks and targeted higher-frequency, dexterous control [6]. These works support a real achievement: one pretrained policy family can be a starting point for multiple robots and tasks. They do not establish zero-data deployment in a new production process.
Cross-embodiment pretraining broadens policy priors, but reported task success does not establish customer-specific quality, safety, or lifecycle performance. π0.5's long-horizon generalization in unseen homes and π0.7's steerable generalist evaluations are significant research evidence within their stated protocols [11], [12]. They do not include every reflective material, tolerance, worn tool, replaced sensor, PLC interaction, or shift change that defines a production cell. Broad pretraining may reduce the cost of starting from scratch; it does not replace evidence of process responsibility.
The following axes are therefore more useful than headings organized by company name.
| Comparison axis | Open generalist-policy path | Human and multi-robot data path | Vertically integrated humanoid path | Manufacturing interpretation |
|---|---|---|---|---|
| Data access | Open data plus partially private mixtures; adaptation path is inspectable | Human video, multi-institution collections, or proprietary capture at scale | Internal teleoperation and fleet data dominate | Export and replay rights for customer-cell episodes matter more than nominal corpus size. |
| Embodiment | Adapters span arms and grippers | Common representations or common end effectors reduce mismatch | Policy and one humanoid body are co-optimized | Breadth across robots and precision on one body are different benefits. |
| Action interface | Tokens, continuous actions, and chunks can be compared | Human motion is retargeted or inferred by inverse dynamics | Low-level controller boundaries may remain undisclosed | Rate, latency, force control, and safety-controller authority must be explicit. |
| Openness | More papers, code, and weights | Scale may be visible while collection and cleaning remain partial | Strong product integration with closed internal data | Openness helps internal baselines; integration may reduce deployment work. |
| Evaluation | Laboratory tasks with author-defined denominators | Cross-scene, cross-robot, or human-to-robot transfer | Selected product and partner demonstrations | Re-evaluate all candidates with the same task and failure taxonomy. |
| Deployment evidence | External reproduction is possible, but production records are rare | Data scale rarely connects to production yield | Field demonstrations are increasing, continuous metrics remain scarce | A plant replay set and signed release history are the final comparison. |
Openness and capability are not the same axis. A weaker open model may still give a manufacturer a valuable validation language. A stronger closed model may fail to provide adequate rights over customer data and updates. A robust strategy runs open baselines and integrated candidates against the same replay evidence instead of treating either category as inherently superior.
GEN-1's company-reported mastery, speed, and low-data adaptation claims belong in this table as official evidence for a vertically integrated candidate [28]. Its disclosed task definitions and limitations are useful market signals, but they are not independent production evidence until an outside factory replays the same criteria.
Figure 9.2. π0 Figure 3 connects an Internet-pretrained VLM, a pretraining mixture containing self-collected robot data and OXE, and a 300M-parameter action expert that generates continuous actions for multiple robots. It is a paper architecture diagram of training inputs and generated policy actions, not measured factory-quality or safety evidence. Source: Black et al. (2024), arXiv:2410.24164, Figure 3, p. 4.
The NVIDIA Responsibility Map: Seven Contracts, Not One Stack
The NVIDIA-centered physical-AI stack assigns authoring, simulation, training, learned augmentation, foundation-policy learning, and real-cell assurance to different components [13], [14], [15]. Erasing the boundaries makes Omniverse look like a physics engine, Cosmos like a force sensor, or GR00T like a certified safety controller. Preserving them turns each layer into a contract with inputs, outputs, calibration ownership, and an evidence level.
| Responsibility | Main layer | Inputs | Outputs | Gap addressed | What it does not guarantee |
|---|---|---|---|---|---|
| Scene authoring and interchange | OpenUSD and Omniverse libraries | Geometry, materials, hierarchy, semantics, variants | Composable scenes and assets | Fragmentation across tools and organizations | Correct real friction, tolerance, or sensor error |
| Robot and sensor simulation | Isaac Sim | Robot model, scene, sensors, physics settings | State, image, and synthetic sensor data | Repetition without occupying real hardware | Agreement with every dynamic and failure mode |
| Learning and evaluation orchestration | Isaac Lab | Environment, observations, actions, reward, randomization | Parallel training and evaluation records | Cost of large repeated experiments | A good reward, correct process model, or hardware safety |
| Contact and deformation physics | Newton and supported backends | CAD, mass, friction, elasticity, control | Collision, contact, and deformation states | Limited insertion, cable, and soft-body representation | Automatic recovery of real parameters from bad inputs |
| Visual, behavioral, and action modeling | Cosmos family | Video, language, action, structural conditions | Transferred video, futures, inferred actions, scores | Insufficient appearance and behavior coverage | Ground-truth hidden force or formal safety proof |
| Foundation-policy learning | GR00T family and training blueprints | Real and synthetic trajectories, language, robot state | Robot action policy | Reuse across tasks and embodiments | Customer yield, lifetime, or regulatory acceptance |
| Deployment assurance | Real-cell operating system | Hardware rollouts, quality results, stops, interventions | Approval, rejection, rollback, and retraining data | Residual error outside every simulator | Quality authority that a platform vendor can own by default |
Scene Contract: OpenUSD and Omniverse
OpenUSD is fundamentally a way for tools to compose and exchange geometry, materials, variants, hierarchy, and metadata [13]. Omniverse libraries and connectors place rendering, physics, and data-access functions inside existing 3D and industrial applications [16]. This can reduce duplicate asset work, preserve design changes as scene variants, and reuse sensor placements and semantic labels across teams.
A simulation-ready asset is not synonymous with a reality-equivalent digital twin. A collision mesh can omit a manufacturing tolerance. A named material can carry the wrong friction coefficient. A semantic label can exist without a quality specification. OpenUSD defines how values travel; it does not certify that the values match the plant. Each manufacturing asset therefore needs provenance, measurement date, uncertainty, calibration target, and an owner.
This distinction matters in the Blender workflow supplied with the user brief. Bringing Omniverse libraries into a familiar authoring tool can reduce handoffs and let an agent identify missing labels or physics properties. It also creates a new review obligation. A machine-filled value is a candidate parameter, not a measured one. The correct operational benefit is faster scene preparation and reusable regression assets, not automatic physical truth.
Experiment Contract: Isaac Sim and Isaac Lab
Isaac Sim places articulations, rigid bodies, cameras, lidar, sensor models, and synthetic-data generation in a GPU-accelerated scene. A 2026 research survey of Isaac Sim highlights parallel experimentation and synthetic data while retaining open-world physics and usability as unresolved challenges [14]. That is paper-level evidence about the platform's architecture and usage patterns, not a direct experiment on every manufacturing cell. Its proximity to the supplier ecosystem should also remain visible.
Isaac Lab is not a new law of physics. It is a framework for defining environments, observations, actions, rewards, curricula, randomization, imitation learning, reinforcement learning, and evaluation [17]. Its most important manufacturing capability is not merely thousands of parallel environments. It is the ability to repeat a policy under frozen initial conditions and a frozen failure definition. The same machinery can also optimize a bad reward faster. A simulation that gives perfect object pose unavailable to the hardware, or declares success after a dropped part, can produce an impressive but unsafe policy.
An Isaac Lab result should consequently be released as an experiment bundle, not a lone policy file. The bundle includes scene version, physics backend and version, randomization distribution, observation and action schemas, reward, reset logic, failure taxonomy, seeds, and checkpoint. This makes it possible to distinguish an improvement in the policy from a change in the simulator.
Physics Contract: Newton Adds Expressivity, Not Automatic Calibration
Newton is positioned as an extensible GPU physics engine built with Warp and OpenUSD, connecting rigid-body, contact, friction, deformable, and differentiable computation to robot learning [18]. The direction is especially relevant to cable insertion, cloth or rubber deformation, torsional friction, and distributed contact surfaces that are poorly represented by simple collision models.
Greater expressivity requires more real parameters. If CAD tolerances, mass distribution, friction, elasticity, damping, motor delay, end-effector compliance, or force-sensor response are wrong, a detailed solver can be precisely wrong. Chebotar et al.[2] adapted simulation randomization using real executions for exactly this reason. A simulator is a hypothesis that must be updated with hardware evidence, not an original source that supersedes reality.
Newton's project pages and partner demonstrations establish first-party capability and integration direction. They do not independently establish long-duration sim-real correlation in contact-rich factories, drift after sensor replacement, or prediction under tool wear. A manufacturing test should compare contact onset, peak wrench, slip direction, insertion depth, and failure-type ranking under matched initial conditions. Visual polish is not a contact metric.
| Contract unit | Source of truth | Comparative evidence retained before release |
|---|---|---|
| Scene and asset | Measured geometry, material, tolerance, and sensor placement | Versioned scene with provenance, uncertainty, and calibration target |
| Experiment and physics | Frozen observation, action, failure definitions, and real contact | Matched real-sim trajectories and mismatch records |
| Learned amplification | Real seed data and executable actions screened through inverse dynamics | Provenance for generation, filtering, pseudo-actions, and real replay |
Cosmos: Separate Visual Amplification, Behavioral Amplification, and Policy
Treating Cosmos as one simulator obscures several responsibilities. Conditional transfer models can preserve depth, segmentation, edges, or keypoints while changing illumination, texture, and background. This primarily addresses the visual domain gap. It is distinct from Newton's role in computing contact and deformation.
Cosmos Predict and video world models such as DreamGen and DreamDojo generate possible futures from a current scene, action, or instruction. They can broaden behavior coverage, help rank policies, and generate synthetic trajectories [7], [8]. DreamGen creates videos of new behaviors from real seeds and attaches pseudo-actions using an inverse model. This may extend beyond the scenes for which accurate CAD assets exist. A plausible video, however, does not show that an action is executable or that the contact force is correct. The pseudo-action contains both generation error and inverse-dynamics error.
Reasoning or filtering models such as Cosmos Reason can reject obvious instruction failures, object disappearance, and penetration. They are probabilistic critics that improve a generated dataset, not formal verifiers of mechanics. They may remove visibly impossible clips without detecting a subtle overload, out-of-tolerance insertion, material damage, or sensor saturation.
Cosmos 3 introduces an omnimodal mixture-of-transformers architecture for language, image, video, audio, and action [15], [20]. Action-conditioned post-training can be described through three modes. Forward dynamics predicts future video from the current scene and an action. Inverse dynamics infers a possible action trajectory from observed video. Policy mode jointly generates an action chunk and an anticipated future. The unification is an important research direction because it reduces interfaces between a policy and a world model. It does not make an unobserved torque, pressure distribution, or tactile event uniquely identifiable from video. Multiple actions can produce visually similar outcomes.
The two NVIDIA videos supplied by the user establish first-party capability and workflow context only, unless a linked primary experiment supports a stronger statement [21], [22]. The Blender video demonstrates an authoring path in which Omniverse libraries and agent tooling add or check physics, sensor, and semantic properties before a USD handoff. It supports the possibility of reusable scene preparation; it does not show that agent-filled values match a real cell or improve real-policy performance.
The 100 teleoperated demonstrations mentioned in the Cosmos 3 action-prediction video must be read the same way. They are a tutorial configuration for a small pick-and-place adaptation in LeRobot format [21]. They are not a measured sample-efficiency law across tasks, embodiments, sensors, and target success levels. The count also excludes the upstream data used to pretrain the world model, failure collection, scene calibration, safety testing, and real-cell quality validation.
The accurate answer to whether teleoperation decreases is therefore conditional. Task-specific successful demonstrations may decrease when the pretrained visual and behavioral prior fits the task, contact is simple, and inverse-model errors can be screened by real replay. When tight tolerance, force, tactile feedback, or high failure cost dominates, failure, recovery, and quality episodes may become more valuable rather than less necessary. Human labor moves from repeating nominal trajectories toward judging exceptions and correcting failures.
The GR00T Data Factory: Mix Three Sources Without Losing Provenance
GR00T N1 is a public research reference for a humanoid foundation policy that combines higher-level visual-language processing with a lower-level action system [10]. In the broader NVIDIA strategy, the critical distinction is how data is produced. Real teleoperation, explicit physics simulation, and learned video generation enter one training pipeline with different error modes.
The first path is real seed data: teleoperation, autonomous rollouts, interventions, joint state, force, tactile signals, and quality outcomes. It is expensive but directly joins executable action to process result. The second path is physics-based amplification. Isaac Sim and Isaac Lab vary object pose, scene, robot, trajectory, lighting, and sensor noise; Newton or another backend can extend contact representation. GR00T-Mimic-style workflows are suited to generating variants of a known skill from a small number of demonstrations. The third path is learned amplification. Cosmos and GR00T-Dreams-style workflows generate new scene or behavior video, filter it, and infer pseudo-actions.
Provenance is the key property of the mixture. Every episode should state whether it is real, physics-simulated, generated from video, inferred by inverse dynamics, or corrected by a person. Without those labels, the team cannot attribute an improvement or diagnose a generated failure. It should also retain the confidence of the success label, missing sensor channels, and the mixture weight used for training.
The connected data factory competes with and complements other generalist paths. OpenVLA, Octo, and π0 provide inspectable baselines for model and action representation. Human-video paths provide scalable task semantics without occupying robots. Vertically integrated systems jointly optimize body, controller, and policy. The NVIDIA ecosystem's distinguishing scope is its attempt to connect scene, simulation, learning, world models, and policy. The benefit is lower interface cost. The risk is allowing one supplier's vocabulary to define scene truth, model performance, and deployment evidence at once.
Figure 9.3. Figure AI's Helix announcement graphic plots conceptual task-count-versus-time curves for three scaling regimes. Because the axes provide no values, samples, or uncertainty, it is first-party workflow context—not a measured scaling law or independent factory-performance result. Source: Figure AI, 2025, Figure 1.
Counterevidence and Limits: What World Models Do Not Observe
First, inferred actions do not automatically recover force and tactile state. ForceVLA and Tactile-VLA report benefits from adding contact channels in small contact-rich task suites, exposing a missing variable in vision-centered generalist policies [23], [24]. These are also preprints with limited tasks and sensor configurations; they do not provide a production lifecycle study of contamination, replacement, or drift. Their important counterpoint is not that all VLAs fail. It is that visually similar states may require different contact responses.
Second, a synthetic trajectory count is not a count of independent useful experiences. If a generator repeats the same bias, nominal data volume grows without covering new failures. DreamGen demonstrates the promise of novel behavior generation while retaining inverse-model error, substantial generation and filtering cost, and limited complex dexterity [7]. DreamDojo opens a frontier in which broad human interaction video supports policy evaluation and planning, but rare fast events, nuanced failures, multiple views, and optimistic success prediction still require validation [8].
Third, policy-ranking correlation is different from absolute simulation accuracy. A simulator may rank policy A above B correctly while misestimating both real success rates and force traces. It can be useful for candidate selection without being qualified for product release. A photorealistic renderer may still rank contact failures incorrectly. Rendering fidelity, sensor fidelity, dynamics fidelity, and task-outcome correlation require separate measurements.
Fourth, generalist claims can hide long-horizon compounding failure. High stage success multiplied over dozens of steps can produce weak whole-task completion. Contamination, people entering the cell, mixed parts, bad resets, and network delay are rare in research data but recurring in operation. Understanding an instruction and executing safe stop, recovery, and rework are different capabilities.
Fifth, openness has limits. Released code and weights do not provide full reproduction if the original mixture, filtering decisions, compute, and failed experiments remain hidden. A closed system can still build strong deployment evidence by exporting customer-cell episodes and release history and permitting an independent audit. The sharper question is not simply open versus closed. It is who can rerun and falsify each claim.
Manufacturing Interpretation: Spend Real Data Where It Is Most Informative
The manufacturing objective is not zero real data. It is to generate cheap variation computationally and collect contact, failures, and quality outcomes selectively where only reality reveals them.
| Task family | What simulation or world models can amplify | What remains real-cell evidence | First release metrics |
|---|---|---|---|
| Structured pick-and-place | Pose, lighting, background, camera view, reach path | Slip, double pick, reflective or transparent parts, gripper wear | Success plus drop, damage, and retry rates |
| Case packing and arrangement | Bimanual collision, order, target layout, workspace | Paper deformation, sticking, rebound, packaging variation | Throughput, crushing, pose error, intervention |
| Insertion and fastening | CAD variants, approach, force strategy candidates, tolerance distributions | Friction, threads, torque-angle, tool wear, jamming | Insertion depth, peak force, defect and damage rates |
| Cable, film, and soft goods | Coarse deformation and path candidates | Material elasticity, adhesion, wrinkles, slip, sensor drift | Final shape, damage, and recovery success |
| Long-horizon work | Instruction and sequence variation, scene and object diversity | Exceptional transitions, wrong objects, rework, shift change | End-to-end completion, stage failures, recovery time |
| Safety and quality | Scenario enumeration and regression support | Real hazards, product outcomes, stop and approval records | Zero limit violations and traceable disposition |
Implementation starts with synchronized logging: RGB and depth, commanded and measured joint state, wrist force and torque, gripper state, object and process identifiers, equipment state, task outcome, defect code, intervention, and cycle time. This common record lets Newton calibration, Isaac Lab observations and rewards, Cosmos post-training, GR00T adaptation, and real replay refer to the same process truth.
A small seed set can then produce a baseline policy. Instead of collecting only more successful repetitions, the manufacturer runs constrained autonomous trials and prioritizes interventions near slip, jam, wrong approach, or quality rejection. This follows DAgger's historical insight that labels at learner-induced states carry high information density [9]. Safety monitoring and residual control must still prevent deliberate visits to unacceptable states.
Simulation and generated data expand around that real seed. Each synthetic episode receives its generation route and confidence. Ablations on the real failure set determine whether it helps. If a mixture raises average success while increasing jams for one part family, the average must not hide the regression. Data weights should be adjusted by task, object family, and failure type.
The same principle changes procurement. A general model card is insufficient. The manufacturer should request the episode schema, action rate, controller boundary, safety override path, synthetic-data provenance, replay procedure, checkpoint history, and examples of explained failures. The request applies equally to open policies, an NVIDIA-centered stack, and a vertically integrated competitor.
Conclusion: Expand Coverage With Compute, Close Responsibility With Reality
A foundation-model data strategy does not eliminate teleoperation. It replaces or amplifies some repetitive nominal demonstrations with open pretraining, explicit simulation, world models, and inverse dynamics, then concentrates human time on contact calibration, failure recovery, and quality judgment. OpenUSD is the scene language; Isaac Sim is the robot and sensor testbed; Isaac Lab is the learning and evaluation orchestrator; Newton is the contact representation layer; Cosmos is a visual and behavioral generator; GR00T is a policy-learning layer. None of them owns final real-cell quality authority.
The strength of an NVIDIA-centered strategy is the ability to reduce interface friction from scene to policy. Its weakness is that integration can make capability descriptions, research results, partner demonstrations, and production evidence sound like one continuous proof. The most important technology is therefore not another model. It is the operational discipline that preserves evidence level, episode provenance, calibration history, replay sets, and release authority.
Chapter 10 moves this principle into deployed data flywheels. It examines how production robotics teams that begin with narrower operating domains accumulate customer-cell logs, exceptions, remote interventions, and quality outcomes, and how ownership of those records shapes update authority. If Chapter 9 maps the data factory that builds a generalist policy, Chapter 10 maps the operating system in which that policy fails, is corrected, and earns release approval every day.
References
- Argall, Brenna D. et al. (2009). A Survey of Robot Learning from Demonstration. Robotics and Autonomous Systems.
- Dasari, Sudeep et al. (2019). RoboNet: Large-Scale Multi-Robot Learning. CoRL.
- Open X-Embodiment Collaboration (2023). Open X-Embodiment: Robotic Learning Datasets and RT-X Models. IEEE ICRA 2024.
- Ghosh, Dibya et al. (2024). Octo: An Open-Source Generalist Robot Policy. RSS 2024.
- Kim, Moo Jin et al. (2024). OpenVLA: An Open-Source Vision-Language-Action Model. CoRL 2024.
- Black, Kevin et al. (2024). π0: A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint. #2 · KO
- Jang, Joel et al. (2025). DreamGen: Unlocking Generalization in Robot Learning through Neural Trajectories. arXiv preprint.
- Gao, Shenyuan et al. (2026a). DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos. ICML 2026 Spotlight / arXiv.
- Ross, Stéphane et al. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS.
- Bjorck, Johan et al. (2025). GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv preprint.
- Physical Intelligence et al. (2025). π0.5: A Vision-Language-Action Model with Open-World Generalization. arXiv preprint.
- Physical Intelligence (2026). $π_{0.7}$: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities. arXiv preprint. #62 · KO
- Alliance for OpenUSD (2026). OpenUSD Specifications and Alliance Governance. Official specification and governance site.
- Gao, Sicong et al. (2026b). NVIDIA Isaac Sim: Enabling Scalable, GPU-Accelerated Simulation for Robotics. arXiv preprint.
- Aditi et al. (2026). Cosmos 3: Omnimodal World Models for Physical AI. arXiv technical report.
- NVIDIA Omniverse (2026). NVIDIA Agent Toolkit Expands With New Omniverse Libraries. Official announcement.
- Mittal, Mayank et al. (2025). Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv preprint.
- Newton Project (2026). Newton Physics Engine. Official project page.
- Chebotar, Yevgen et al. (2019). Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience. IEEE ICRA.
- NVIDIA Cosmos (2026). Develop Physical AI Reasoning, World, and Action Models with NVIDIA Cosmos 3. Official technical blog.
- NVIDIA Video (2026a). How to Post-Train NVIDIA Cosmos 3 for Robot Action Prediction. Official tutorial video.
- NVIDIA Video (2026b). Bringing Agent-Ready Simulation Into Blender. Official workflow video.
- Yu, Jiawen et al. (2025). ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation. arXiv preprint.
- Huang, Jialei et al. (2025). Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization. arXiv preprint.
- Kim, Dongyoung et al. (2026). RLDX-1 Technical Report. arXiv preprint. #67 · KO
- Zheng, Ruijie et al. (2026). EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data. arXiv preprint.
- Figure AI (2025). Helix: A Vision-Language-Action Model for Generalist Humanoid Control. Official company research post.
- Generalist Team (2026). GEN-1: Scaling Embodied Foundation Models to Mastery. Official company research post. #24 · KO
- Chelsea Finn et al. (2016). Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization. International Conference on Machine Learning.
- Michael Janner et al. (2019). When to Trust Your Model: Model-Based Policy Optimization. NeurIPS.
- Oliver Kroemer et al. (2021). A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. JMLR.
- Fanbo Xiang et al. (2020). SAPIEN: A SimulAted Part-based Interactive ENvironment. CVPR.
- Mohit Shridhar et al. (2022). CLIPort: What and Where Pathways for Robotic Manipulation. CoRL.
- Songming Liu et al. (2024). RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation. arXiv primary preprint (2024-10-10).
- Pertsch, Karl et al. (2024). Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models. Primary publication.