NeRD：一种改进机器人模拟的神经网络方法

Modern robotics requires more than what classical analytic dynamics provides because of simplified contacts, omitted kinematic loops, and non-differentiable models. Neural Robot Dynamics (NeRD) tackles these hurdles by:

Using expressive, differentiable models that predict stable states over long horizons.Capturing complex contact-rich physics.Generalizing across tasks, environments, and controllers, narrowing the sim-to-real gap. Fine-tuning on real data.

Unlike task-specific neural simulators, NeRD serves as a drop-in backend within physics engines like Newton, enabling teams to reuse existing policy-learning environments by simply switching the physics solver. This hybrid of analytical modules with robot-centric neural modeling paves the way for robots whose dynamics continually improve through both simulation and real-world experience.

In this post, we explore how NeRD overcomes longstanding simulation challenges, providing the foundation for modern robotics in physics engines like Newton

What is NeRD?

NeRD is a neural simulation framework. NeRD models are a learned embodiment of specific dynamics models that can predict future states of articulated rigid bodies (e.g., robots with multiple joints) in contact with the environment.

Once trained, NeRD models can:

Provide stable and accurate predictions over hundreds to thousands of simulation steps.Generalize to different tasks, environments, and low-level controllers for a particular robot. Be fine-tuned from real-world data to bridge sim-to-real gaps.

NeRD models may be trained on data from any simulator. Once trained, they can be deployed as drop-in replacements for analytic solvers such as those found in modular frameworks like Newton. This enables users to reuse existing policy-learning environments and activate NeRD as a new physics backend through a single-line switch.

Start using NeRD in Newton. View our research on arXiv or explore our project page.

Vision for the future of robotic simulation

As robotic technologies advance, we envision a lifecycle where each robot is equipped with a neural dynamics model pretrained from analytical simulations. Such a neural dynamics model can be continuously fine-tuned as the robot interacts with the real world, enabling it to account for wear-and-tear of the robot and environmental changes.

The neural dynamics model of the robot can be embedded into a hybrid simulation system, where neural dynamics simulate the robot while analytical dynamics are used for other parts of the scene (e.g., obstacles). These continuously-improved neural robot dynamics provide a better replica of real-world dynamics for facilitating the learning of versatile robotic skills in a digital twin powered by this continuously updated simulator.

*Figure 1. An envisioned lifecycle of a robot in the future*

How does neural robot dynamics work?

NeRD is characterized by two key innovations that achieve generalizability and long-horizon prediction accuracy—a hybrid prediction framework and robot-centric input parameterization. NeRD models replace the time integration (solver) portion of a traditional simulator. In frameworks like Newton, where collision detection is decoupled from the solver, we can combine analytic collision detection in conjunction with our learned model.

This hybrid framework enables NeRD to leverage intermediate simulation quantities (i.e., robot state, contact information, and joint-space torques) to describe the full simulation state, providing necessary information to evolve the robot dynamics regardless of the applications (e.g., tasks, scenes, and controllers). This is in contrast with previous approaches that only take robot state and task-specific actions as inputs, thus overfitting to the tasks used for training.

Second, NeRD uses a robot-centric parameterization of inputs to enable the learned dynamics model to spatially generalize. Specifically, the robot state and contact-related quantities are transformed into the robot’s base frame before they are passed as input to the NeRD model, as shown in Figure 2(c).

Such a robot-centric state representation enables NeRD to perform reliable predictions at unseen spatial robot locations encountered during robot motion, enhancing the long-horizon accuracy of the model.

*Figure 4. Comparison of the analytical simulator and NeRD on a double pendulum with various configurations of the ground plane*

Learning robotic policies exclusively in a NeRD-integrated simulator

NeRD’s efficiency and generalizability across tasks, controllers, and space enable large-scale robotic policy learning for diverse downstream tasks. We pre-train a NeRD model for an ANYmal robot and then train a forward-walking policy and a sideways-walking policy using the PPO reinforcement-learning algorithm inside the NeRD-integrated simulator, without access to the ground-truth analytical simulator.

The learned policies can then be transferred in zero-shot to the ground-truth analytical simulator with minimal performance loss (<0.1% error in accumulated reward for 1000-step trajectories). Figures 5 and 6 show a side-by-side visualization of NeRD-trained policies executed in both the NeRD-integrated simulator and the ground-truth analytical simulator.

*Figure 5. Comparison of an analytical simulator and a NeRD model for an ANYmal robot with an RL policy for forward walking at 1 m/s*

*Figure 6. Comparison of an analytical simulator and a NeRD model for an ANYmal robot with an RL policy for sideways walking at 1 m/s*

Zero-shot sim-to-real transfer

The accuracy of the NeRD model was also validated on a 7-DoF Franka robot arm, where we performed zero-shot sim-to-real transfer for a go-to-pose (reach) policy trained exclusively in the NeRD-integrated simulator (Figure 7).

Fine-tuning NeRD models from real-world data

Inherent differentiability of the NeRD models enables them to be fine-tuned rapidly from real-world data. We fine-tune a pre-trained NeRD model for a cube-tossing task using a real-world cube-tossing dataset. The fine-tuned NeRD model significantly improves ‌dynamics accuracy compared to the analytical simulator (shown in Figure 8)

*Figure 8. Fine-tuning a NeRD model on real-world data better matches real-world cube-tossing dynamics. The light-green cubic frames illustrate the real-world cube trajectory*

Summary

Neural Robot Dynamics (NeRD) is a neural-network-based robotic simulation framework designed to accurately predict the dynamics of complex, articulated robots over long periods. Unlike traditional robotic simulators that use simplified models and struggle with modern robot complexities, NeRD learns robot-specific dynamics directly from data, enabling stable, generalizable, and precise simulations.

A single trained NeRD model generalizes to diverse tasks, environments, and controllers for a given robot and can be fine-tuned with real-world data to reduce the simulation-to-reality gap, making it a highly adaptable and advanced solution for robotic simulation.

Future directions

Developing effective neural simulators for modeling complex real-world robot dynamics is an active area of research. To achieve generalizable and fine-tunable neural dynamics models for robotics, this research can be extended in several exciting directions:

Robots with more complex structures and higher degrees of freedom

Learning a neural simulator for more complicated robots (e.g., humanoid robots) can significantly improve simulation efficiency and accelerate downstream applications (e.g., learning whole-body controllers for humanoids).

Fine-tuning from partially-observable real-world data

Real-world robot data is often only partially observable due to sensor limitations. For example, contact points may not be precisely known. Investigating methods to fine-tune pre-trained NeRD models from partially observable real-world data can improve the accuracy of predicting real-world dynamics, thereby better bridging sim-to-real gaps.

Simulating robotic manipulation

Our development of the NeRD framework has thus far focused primarily on locomotion tasks. Supporting the simulation of manipulation tasks is a natural extension of this work that can further broaden its applicability.