Talks

This presentation explores the intricacies of training large language models (LLM) on High-Performance Computing (HPC) systems, unveiling the key components, challenges, and optimizations involved in handling the computational demands of state-of-the-art language models. Delving into the nuances of model architecture, data preprocessing, and hyperparameter tuning, a comprehensive understanding of parallelization strategies, scalability challenges, and resource allocation will be given. Additionally, the talk touches on the implications for research, highlighting potential progress and future applications of LLMs.

Zoomlink:
https://tuhh.zoom.us/j/81920578609?pwd=TjBmYldRdXVDT1VkamZmc1BOajREZz09

With exascale-capable supercomputers already on the horizon, reducing communication operations in orthogonalization kernels like QR factorization has become even more imperative. Low-synchronization Gram-Schmidt methods, first introduced in Swirydowicz et al. (Numer. Lin. Alg. Appl. 28(2):e2343, 2020), have been shown to improve the scalability of the Arnoldi method in high-performance, distributed computing. Block versions of low-synchronization Gram-Schmidt show further potential for speeding up algorithms, as column-batching allows for maximizing cache usage with matrix-matrix operations. We will examine how low-synchronization block Gram-Schmidt variants can be transformed into block Arnoldi variants for use in standard Krylov subspace methods like block generalized minimal residual methods (BGMRES). We also demonstrate how an adaptive restarting heuristic can handle instabilities that arise with the increasing condition number of the Krylov basis. The performance, accuracy, and stability of these methods are assessed via a flexible comparison tool written in MATLAB.

Zoomlink:
https://tuhh.zoom.us/j/81920578609?pwd=TjBmYldRdXVDT1VkamZmc1BOajREZz09

Zoomlink:
https://tuhh.zoom.us/j/81325639377?pwd=emRwaU9KOXhseStxUEU2M2NFS0Qwdz09

Neural networks trained with machine learning techniques are currently attracting great attention as nonlinear approximation methods to solve forward and inverse problems involving high-dimensional partial differential equations (PDEs). In a recent paper, Neural Galerkin scheme has been proposed to solve PDEs by means of deep learning. In this approach, the deep learning process generates the training data samples with an active learning process for the numerical approximation. We apply this approach in this talk to tackle a parameter estimation problem and propose an algorithm based on Neural Galerkin scheme to estimate a scalar coefficient involved in a non-linear PDE problem. We provide numerical results with Korteweg-de Vries (KdV) equation in one dimension.

Zoomlink:
https://tuhh.zoom.us/j/81920578609?pwd=TjBmYldRdXVDT1VkamZmc1BOajREZz09

We discuss approximation methods for linear equations in sequence spaces. When cutting out a finite matrix from an infinite dimensional operator, a choice of boundary conditions has to be made. Choosing zero boundary conditions leads to the classical finite section method, for which conditions for the applicability are known. We derive similar conditions for the applicability for the choice of periodic boundary conditions.
As an important tool, we demonstrate a way to approximate spectral quantities of an infinite dimensional operator with the help of finitely supported vectors.
Moreover, we investigate discrete Schrödinger operators and find conditions for the applicability of the finite section method.

This talk gives an overview of the results obtained in my PhD under the supervision of Prof. Dr. Marko Lindner.

Zoom link: https://tuhh.zoom.us/j/8757671580?pwd=ZjgyYURxYWxrQmJjaUVtTE5uTnBHUT09

tba