FResNet++: Augmenting deep residual surrogates with Fourier operators for accelerated PDE based flow and transport simulations

Accurate numerical modeling of multi-phase flow and transport mechanisms is essential to study varied, complex physical phenomena. State-of-the-art complete physics-based solvers suffer from many computational challenges. High-fidelity data-driven surrogate models that solve the governing PDEs have the potential to optimize the time to solution and increase confidence in critical business and engineering decisions through better quantification of solution statistics. We leverage the recently proposed Fourier neural operators with quasi-linear time complexity to capture the spectral information from feature maps. Embedding Fourier layers within the residual blocks result in a highly effective structure that, while achieving competitive accuracy, also enables efficient training of deeper networks with a drastically reduced number of trainable parameters. FResNet++ also uses squeeze and excitation blocks, Atrous Spatial Pyramidal Pooling (ASPP), and attention blocks to increase its sensitivity to the relevant features, capture multi-scale information, and simulate 2D flow properties over long time horizons. Results show a speedup of at least six orders of magnitude compared to conventional numerical simulators. We demonstrate the ability of FResNet++ to generalize over multiple high-dimensional input parameter spaces with variable controls that model the complex interplay of gravity, capillary, and viscous forces. Our experiments validate that Fourier neural operators have the potential to be applied in various CNN-based architectures and can effectively substitute for repetitive physics-based forward simulations in the context of scenario testing and solving inverse problems.