Jesus Labarta
Title: Towards throughput oriented hybrid programming
Abstract: The talk will present a personal vision of how hybrid programs with MPI+OpenMP should address the challenge of maximizing the efficiency obtained from the current and foreseable large scale machines.
In a context of huge complexity of the infrastructure (system software and hardware layers) and variability in resource performance and workload demands, we advocate for malleability and dynamic resource management as fundamental criteria to minimize resource stalls. The productivity challenge is how to write/refactor programs with the least effort and ensuring durability and maintainability for the resulting code. From our point of view, proper programming model mechanism are key for this but it also requires abstract identification of fundamental behavioral issues exposed by a program and deep observation of "microscopic" effects that impact the macroscopic behavior. We are facing a joint challenge between environment (compiler, runtime, OS, architecture) developers and even more important programmer best practices.
I will discuss on the interaction between MPI and OpenMP, analyze dominant practices in the way we today write MPI and hybrid programs, and show effects on performance that appear under these practices at even relatively low core counts. I will argue how I believe programming practices should be adapted to allow for intelligent runtimes to achieve good performance. Measurements and examples of practical examples will be used to sustain the observations.
The fundamental message is that a throughput oriented vision and a hybrid MPI+OpenMP task based approach has the potential to minimize the amount of information the use has to specify in terms of configuration parameters for a run while ensuring that the performance is essentially related to the amount of resources available in a dynamic context.
Bio: Jesus Labarta is full professor on Computer Architecture at the Technical University of Catalonia (UPC) since 1990. Since 2005 he is responsible of the Computer Science Research Department within the Barcelona Supercomputing Center (BSC). His major directions of current work relate to performance analysis tools, programming models and resource management. His team distributes the Open Source BSC tools (Paraver and Dimemas) and performs research on increasing the intelligence embedded in the performance analysis tools. He is involved in the development of the OmpSs programming model and its different implementations for SMP, GPUs and cluster platforms.
Bronis R. de Supinski
Title: Sierra, CORAL and OpenMP
Abstract: Lawrence Livermore National Laboratory (LLNL) has a long history of leadership in large-scale computing. Our next platform, Sierra, a heterogeneous system that is part of a Collaboration between Oak Ridge, Argonne and Lawrence Livermore National Laboratories (CORAL) and delivered through a partnership with IBM, NVIDIA and Mellanox, continues that tradition. The system is now fully installed and will soon be accepted. This talk will provide a detailed look at the design of Sierra as well as early application and benchmark results on it. It will also discuss the new features in OpenMP 5.0 that are most expected to impact its use.
Bio: Bronis R. de Supinski is the Chief Technology Officer (CTO) for Livermore Computing (LC) at Lawrence Livermore National Laboratory (LLNL). In this role, he is responsible for formulating LLNL's large-scale computing strategy and overseeing its implementation. His position requires frequent interaction with high performance computing (HPC) leaders and he oversees several collaborations with the HPC industry as well as academia.
Prior to becoming CTO for LC, Bronis led several research projects in LLNL's Center for Applied Scientific Computing. Most recently, he led the Exascale Computing Technologies (ExaCT) project and co-led the Advanced Scientific Computing (ASC) program's Application Development Environment and Performance Team (ADEPT). ADEPT is responsible for the development environment, including compilers, tools and run time systems, on LLNL's large-scale systems. ExaCT explored several critical directions related to programming models, algorithms, performance, code correctness and resilience for future large scale systems. He currently continues his interests in these topics, particularly programming models, and serves as the Chair of the OpenMP Language Committee.
Bronis earned his Ph.D. in Computer Science from the University of Virginia in 1998 and he joined LLNL in July 1998. In addition to his work with LLNL, Bronis is also a Professor of Exascale Computing at Queen's University of Belfast and an Adjunct Associate Professor in the Department of Computer Science and Engineering at Texas A&M University. Throughout his career, Bronis has won several awards, including the prestigious Gordon Bell Prize in 2005 and 2006, as well as an R&D 100 for his leadership of a team that developed a novel scalable debugging tool. He is a member of the ACM and the IEEE Computer Society.
Martin Schulz
Title: A Case for More Adaptivity in MPI
Abstract: Current HPC environments and applications are rather rigid and inflexible, and MPI’s inability to efficiently support malleability, i.e., the ability to grow and shrink the computational resources associated with a job at runtime, is a significant part of the problem. Future generations of HPC systems, however, require a more flexible approach, e.g., to support a greater level of fault tolerance, to adjust to changing levels of available resources, or to match more complex workflows. This will also require MPI to change. In this talk I will discuss the challenges facing MPI in these scenarios as well as several approaches that are first steps towards supporting malleability in MPI. They will open the door for MPI to both support a new generation of applications as well as to provide more flexible runtime support for higher level programming models.
Bio: Martin Schulz is a Full Professor and Chair for Computer Architecture and Parallel Systems at the Technische Universität München (TUM), which he joined in 2017. Prior to that, he held positions at the Center for Applied Scientific Computing (CASC) at Lawrence Livermore National Laboratory (LLNL) and Cornell University. He earned his Doctorate in Computer Science in 2001 from TUM and a Master of Science in Computer Science from UIUC. Martin has published over 200 peer-reviewed papers and currently serves as the chair of the MPI Forum, the standardization body for the Message Passing Interface. His research interests include parallel and distributed architectures and applications; performance monitoring, modeling and analysis; memory system optimization; parallel programming paradigms; tool support for parallel programming; power-aware parallel computing; and fault tolerance at the application and system level. Martin was a recipient of the IEEE/ACM Gordon Bell Award in 2006 and an R&D 100 award in 2011.