The 12 KAUST Core Labs were founded to provide technical expertise and advanced equipment across all scientific fields. As part of this work, their staff develop and optimize processes that accelerate research. The latest example, inshimtu, comes from the Visualization Core Lab (KVL), which was reported at the 2023 International Conference on High Performance Computing. Inshimtu is a pseudo in situ visualization system for data scientists, computer scientists and other scientists handling enormous quantities of data.

Supercomputing is changing the world. The ability to predict all sorts of complex events, like rising sea levels, the spread of disease, and the economic reverberations of bank failures, has enabled better preventive actions and responses to global disruptions. To come to a solution, however, supercomputers must consume massive amounts of energy and store massive amounts of data, actions that have high cost. Scientists use in situ visualization to determine if this cost is justified.

In situ visualization provides a snapshot of a simulation that is running on a supercomputer. This gives researchers the chance to verify that the simulation is operating properly before executing to completion, much like checking a map to confirm you are on the right road before arriving at your final destination. However, the implementation of in situ simulations is demanding, so much so that scientists are hesitant to use it. In response, the KVL created Inshimtu.

The biggest challenge for many scientists adopting in situ visualization is effort. The time and energy saved by interrupting an erroneous simulation may be outweighed by the time and energy needed to execute the in situ visualization.

"Simulation scientists spend a lot of time and effort writing their code. They don’t want to change something not knowing if it’s beneficial or not," explains Mr. Thomas Theussel, a visualization scientist at the KVL and an author of the study, about the reluctance to apply in situ visualization.

Inshimtu was designed to simplify the implementation. Using ParaView Catalyst, a standard open-source toolkit used for in situ research, inshimtu provides a visualization using existing simulation output files without changes to the simulation code.

Simulations are used to determine how a complex system behaves in different theoretical circumstances. While in situ visualization aims to reduce the memory demands and time consumed by the experiment, inshimtu does neither. Rather, using inshimtu, scientists can judge the simulation to decide if implementing a full in situ visualization into their code is worthwhile.

"Inshimtu is not a solution. It is a stepping stone. It shows what in situ can do,” said Dr. James Kress, another member of the KVL and author of the paper. It is not designed to save cost or time, but it demonstrates the benefits of in situ visualization to help scientists decide if the effort is justified."

He added that for many closed source codes, which is common with commercial products, in situ visualization is not possible. Inshimtu provides a simplistic simulation for such cases.

As well as explaining the principles of inshimtu and its adoption, the study provides a case study of a weather model run on the KAUST supercomputer Shaheen II.

KVL has made inshimtu open source to encourage more scientists to adopt in situ visualization into their simulation workflow. Contact the Core Labs Labs to learn more about inshimtu.