​-By the KAUST Supercomputing Core Lab team; Nicholas Demille, KAUST News; and Andrea Hulsbosch, KAUST Core Labs

As the largest peninsula on Earth, the Arabian Peninsula measures over 3.2 million square kilometers. For decades, scientists have sought to map the geologic formations of the area and its surrounding seas for the purposes of resource extraction to within one meter. Thanks to world-record simulations run on the KAUST supercomputer Shaheen, that goal may be within reach.

Shaheen recently completed a set of ultra-resolution subsurface mapping and characterization simulations for the Geophysics Technology Division at the Saudi Aramco EXPEC Advanced Research Center. For the first time, scientists produced a 3D image of the subsurface geologic layers to a 7.5 meter resolution.

"Unlocking ultra-resolution subsurface mapping and characterization at production scale is a major milestone for exploration," said Ali A. Al-Meshari, manager of EXPEC ARC. "This is a major step toward seamless integration of geophysical, geological and engineering models."

To put the team's accomplishment in perspective, it allows scientists to see underground features half the length of a semi-trailer or lorry; one-third the length of a bowling alley; or about three times longer than a sedan car.

KAUST hardware and Aramco software

Aramco scientists used integrated GeoDRIVE (Geophysical solution for Dynamic Real-time Image and Velocity Estimation) software to successfully produce a 3D image of subsurface geologic layers at a record resolution of 7.5 meters at production scale. Previous simulations had a resolution of tens of meters.

"The vast computational cost of accurate seismic simulations created a gap between theoretical advancements in the academic world and practical industrial applications," said Ali Momin, GeoDRIVE project lead. "This provided us with an unique opportunity to build an innovative solution that can satisfy our current needs as well as the future needs of researchers and professionals."

(From left to right:) Samuel Kortas, Maciej Olchowik, Saber Feki and Bilel Hadri from the KAUST Supercomputing Core Laboratory are pictured here with their TOP500 certificate. The list ranks the 500 most powerful computer systems. File photo.

​

Shaheen Cray XC-40 supercomputer

Supercomputers consist of tens of thousands of CPUs, memory chips, kilometers of networking cables, thousands of disks and hundreds of applications and libraries. Shaheen, which is located in the KAUST Supercomputing Core Laboratory (KSL), is one of the largest pure CPU systems in the world with a total of 12,348 Intel Haswell CPUs, 790 TB of memory and 17.6 PB of storage. It is currently the largest supercomputer in the Middle East.

Campaigns like the one conducted by the Aramco team require upward of 54,000 simulations using more than 100,000 cores, or around half of the total capacity of Shaheen. Large scale campaigns—referred to as hero runs—require careful optimization of the system.

Supercomputing expertise

One key element of a successful hero run is the readiness of the system. A single underperforming component can jeopardize the performance of the entire run. Maciej Olchowik, the systems administrator team lead, and Bilel Hadri, a computational scientist in the KSL, checked the health of all Shaheen components using a special test suite they developed before handing over the system to Aramco scientists.

KSL scientists also helped to optimize the simulations so that Aramco scientists can explore high-fidelity resolutions and more diverse scenarios.

"My mission is to help scientists use our resource in the most efficient way," said KSL computational scientist Samuel Kortas. "That is why I developed the software tool Decimate—it makes complex things as simple as possible for end users."

With the Decimate tool, Aramco scientists were able to test all the possible combinations of cache usage patterns by exploring large numbers of parameter values. The optimum pattern found with the tool was used for the final production run.

Racking up supercomputing records

This is the third world record resulting from the productive collaboration between Saudi Aramco and the KAUST Supercomputing Core Lab. The teams previously collaborated to conduct the largest-ever scale reservoir simulation and the largest-ever scale engineering simulation.

"The KAUST Core Lab maintains world-class state-of-the-art facilities, and I feel that Shaheen, as a national asset, should be further utilized to meet the demands of organizations around the Kingdom," said Justin Mynar, KAUST associate vice president for research and executive director of the KAUST Core Labs.

Related stories:

• Shaheen computes a world record
• ANSYS, Saudi Aramco and KAUST achieve a new supercomputing milestone
• Lee named to top HPCwire list