In Situ Visualization and Analysis Services for Extreme Scale Supercomputers, David Pugmire, Oak Ridge National Laboratory and University of Tennessee

David Pugmire
July 8, 2016 –
14:00 to 15:00
Access Grid Room (room 277)
Oxford e-Research Centre, 7 Keble Road, Oxford

The scientific data that are being generated today, and in the near future, will quickly outpace our ability to process and understand it. The data generated are growing in multiple ways, including the size of the data, the rate at which it arrives, and the varying types of data. Additionally, data are available from multiple sources, including for example, computational simulations and sensor data extracted from experiments. In such situations, workflows for the movement and management of data, as well as the analysis and visualization while the data are in transit, become even more critical, and increasingly challenging.

At the same time, revolutionary changes are emerging in the architectures of supercomputers. These changes include tremendous increases in concurrency on each node, as well as a deep hierarchy of memory. These architectures are a challenge for currently running HPC codes, as well as visualization and analysis algorithms. We are, along with collaborators at other national laboratories, and universities, researching and developing a cohesive framework that will enable scientists to manage the growing data problem, and to enable them to extract knowledge and insight. Central to this work is a “Service Oriented Architectures” (SOA) framework that provides a clear separation from the data producers and consumers.

Our framework is based on the ADIOS middleware layer, that manages the data as they are produced, and orchestrates movement through the deep memory hierarchy. The data consumers, in this case visualization and analysis services built from tools like VisIt, ParaView, VTK-m, and pbdR, will interact with the middleware layer for access to data, and for coordination to optimize access across the deep memory hierarchy. These services, which will be self-consistent, interoperable operations will enable scientists to construct multiple complex workflows.

This talk will present the preliminary research on the development of this framework and the associated services as well as the initial results obtained working with several large-scale simulation codes running on the Titan supercomputer at ORNL. And finally, a detailed discussion on the future research plans and directions that will enable scientific insight on exascale computing systems.

Profile

I am a Senior Staff Scientist in the Scientific Data Group, Computer Science and Mathematics Division at the Oak Ridge National Laboratory. I also have a Joint Faculty Professor appointment in the Electrical Engineering and Computer Science Department at the University of Tennessee. I am interested in scientific visualization and high performance computing.

Seminar – no booking required – open to all – coffee and cakes