Talks

Transformers have been a major breakthrough in Natural Language Processing (NLP) due to their ability to capture long-range dependencies through self-attention. However, the (self-)attention mechanism suffers from massive memory consumption, especially for tasks with large context windows and high resolution data, such as climate data.

In this talk, I will present a transformer model that uses multiple icosahedral grids to enable large (physical) context windows and high resolutions for various climate-related modelling tasks.

As the model is still under development, the presentation will focus on its technical foundations and properties such as resolution independence and multi-scale output in the context of climate data.

Zoomlink:
https://tuhh.zoom.us/j/81920578609?pwd=TjBmYldRdXVDT1VkamZmc1BOajREZz09

Spectral deferred correction (SDC) is a time-stepping method where fully implicit Runge-Kutta methods (RKM) are solved iteratively. The method is only marginally more complicated to implement than the more ubiquitous diagonally implicit RKM, and it is often simpler for obtaining high-order solutions. We present numerical experiments that show SDC to be a modern and HPC capable method with various advantages over other RKM, including efficient time-parallelisation extensions. To this end, we present adaptive step size selection algorithms for SDC and demonstrate that they boost computational efficiency and resilience against soft faults at the same time. Then, we show that the parallel-in-time algorithm diagonal SDC can be used to extend strong-scaling capabilities beyond the saturation point of space-only scaling. This enables our space-time parallel Python code for the Gray-Scott equation to scale to the entirety of the JUWELS booster machine.

Zoomlink:
https://tuhh.zoom.us/j/81920578609?pwd=TjBmYldRdXVDT1VkamZmc1BOajREZz09

Zoomlink:
https://tuhh.zoom.us/j/88999585223?pwd=YsLVQkzLogvKJt6NNFN2x6OypSvbc9.1

Fluid flow problems can be modeled by the Navier-Stokes or Oseen equations. Their discretization results in saddle point problems. These systems of equations are typically of large scale and thus need to be solved iteratively. Standard (block-) preconditioning techniques for saddle point problems rely on an approximation of the Schurcomplement. Such an approximation can be obtained by a hierarchical matrix (H-Matrix) LU factorization for which the Schur complement is computed explicitly.

We present two strategies to improve the preconditioner set-up time. The first is a problem-dependent construction of the hierarchical block structure for the involved sparse matrices. These block structures are obtained from a partitioning of the velocity index set based on the connection with the pressure index set and results in a sparser block structure of the off-diagonal blocks of the saddle point system matrix.The second strategy are different approaches to the H-matrix multiplication which an important part of the H-LU factorzation and is used directly for the computation of the Schur complement. We briefly describe two variants introduced in [1] and [2] and examine their effectiveness for our problem with results from numerical experiments.

[1] S. Börm. “Hierarchical matrix arithmetic with accumulated updates”. In: Comput. Vis. Sci. 20.3-6 (2019), pp. 71–84. issn: 1432-9360. doi: 10 . 1007 /s00791 - 019 - 00311-3. url: https://doi.org/10.1007/s00791-019-00311-3.

[2] J. Dölz, H. Harbrecht, and M. D. Multerer. “On the best approximation of the hierarchical matrix product”. In: SIAM J. Matrix Anal. Appl. 40.1 (2019), pp. 147–174. issn: 0895-4798. doi: 10.1137/18M1189373. url: https://doi.org/10.1137/18M1189373.

Zoomlink:
https://tuhh.zoom.us/j/81920578609?pwd=TjBmYldRdXVDT1VkamZmc1BOajREZz09