Assured Autonomy in Unstructured Environment
An important challenge for navigating robots in outdoor unstructured environments (e.g., off-road terrains) is the stochastic modeling of unexpected robot behaviors. Basically, the robots operating in real-world complex environments need to reason about the long-term results of their physical interactions with the environment, but due to the high complexity of the real world, it is generally impossible to predict future events in an accurate manner. For example, the effect of uneven road conditions or various disturbances on the robot's motion is hard to model (or learn from data) precisely. It is even more challenging to model the interaction between the robot and the environment, especially when the environment is dynamic. Other representative scenarios include drones flying with strong winds or submarines moving under ocean currents, where both air and water flows vary significantly in both space and time. Thus, it is necessary for the robots to consider these epistemic uncertainties caused by a lack of precise modeling of the environment while making decisions.
Due to the uncertainty, safety naturally becomes a concern in many real world applications. For example,
successful mobile navigation tasks, such as last-mile delivery and autonomous driving, require robots to navigate safely and efficiently in complex, cluttered environments. Achieving this is challenging due to mismatches between the actual vehicle dynamics and the approximated motion planning models, as well as uncertainties during execution. This necessitates the use of feedback-based policies, rather than relying on a single pre-planned trajectory.
Markov Decision Processes (MDPs) have been widely adopted to address motion planning and control under uncertainty, especially in complex environments with disturbances. The solution to these problems is a closed-loop policy that maximizes a long-term goal and satisfies the safety constraints under a probabilistic interaction model between the robot and the environment. However, extending MDPs to continuous states in real-world applications requires approximating the value function or policy. Common approaches use continuous function approximations like neural networks or kernel functions to fit continuous states within the discrete MDP framework. While effective, these approximations tend to worsen policy optimality and increase collision risks, as the true optimal value function can have sharp gradients or discontinuities near non-navigable boundaries. Unfortunately, these methods often blur the distinction between safe (navigable) and unsafe (non-navigable) spaces.
We have developed a principled framework for representing the value function in continuous MDPs that guarantees boundary conditions. We model the safety-constrained decision problem using a variant of the continuous-state MDP called the Diffusion MDP (DMDP), which is characterized by a partial differential equation (PDE). This formulation allows us to incorporate environmental geometries and task constraints directly into the boundary conditions, coupled with the PDE. By embedding safety and goal constraints within the value function, we ensure the solution satisfies these critical boundaries. We then use the Finite Element Method (FEM) to approximate the value function and solve the PDE numerically. FEM discretizes the state space into a mesh of discrete elements, with each element representing local basis functions. These basis functions are highly sensitive to local changes in state, making them particularly effective for accurately modeling safety-critical boundaries. Even with simple basis functions, FEM provides a high-quality approximation that captures boundary conditions far better than existing methods.
Left: autonomous navigation in environments with elevation and obstacles.
In our recent results, we have tested in simulation with Mars surface and used a rover simulator to simulate the 2.5 D motion planning for slow-speedcomplex terrain traversability assessment. Left: Mars rover needs to make motion decisions in the navigable space with spatially varying terrestrial characteristics (cliffs, valleys, ridges, etc). This is different from the simplified and structured environments where there are only two types of representations, i.e., either obstacle-occupied or obstacle-free. Middle and Right: our preliminary result with three rovers exploring on NASA's Mars terrain model.
YouTube link:
YouTube link: