Autonomous Shaping of Latent-Spaces from Reduced PDEs for Physical Neural Networks

Numerical simulations using partial differential equations (PDEs) are a central tool for a wide variety of scientific and engineering applications. Due to their challenging nature, many numerical methods rely on a reduced representation of degrees of freedom and adopt an efficient solver that solves the PDEs in the reduced space. In general, however, it is extremely challenging to faithfully preserve the correct solutions over long timespans with reduced representations. This problem is particularly pronounced for solutions with large amounts of small scale features. To address this, data-driven methods can learn to restore the details as required for accurate solutions of the underlying PDE problem. This paper studies the training of deep neural network models that autonomously interact with a PDE solver to achieve the desired solutions. In contrast to previous work, we do not constrain the PDE solver but instead give the neural network complete freedom to shape the PDE solutions as degrees of freedom of a latent space. Surprisingly, this autonomy allows the neural network to discover new physical dynamics that allow for better performance in the given learning objectives. We showcase that this approach allows the trained encoder to transform accurate solutions into abstract yet physical reduced representations, which are significantly different from conventional down-sampling results. Moreover, we demonstrate that our decoder outperforms models trained with different methodologies in terms of restoration accuracy.