research-article

Open access

Scaling the “Memory Wall” for Multi-Dimensional Seismic Processing with Algebraic Compression on Cerebras CS-2 Systems

Authors:

Leighton Wilson,

Mathias Jacquelin,

David Elliot KeyesAuthors Info & Claims

SC '23: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 6, Pages 1 - 12

https://doi.org/10.1145/3581784.3627042

Published: 11 November 2023 Publication History

Abstract

We exploit the high memory bandwidth of AI-customized Cerebras CS-2 systems for seismic processing. By leveraging low-rank matrix approximation, we fit memory-hungry seismic applications onto memory-austere SRAM wafer-scale hardware, thus addressing a challenge arising in many wave-equation-based algorithms that rely on Multi-Dimensional Convolution (MDC) operators. Exploiting sparsity inherent in seismic data in the frequency domain, we implement embarrassingly parallel tile low-rank matrix-vector multiplications (TLR-MVM), which account for most of the elapsed time in MDC operations, to successfully solve the Multi-Dimensional Deconvolution (MDD) inverse problem. By reducing memory footprint along with arithmetic complexity, we fit a standard seismic benchmark dataset into the small local memories of Cerebras processing elements. Deploying TLR-MVM execution onto 48 CS-2 systems in support of MDD gives a sustained memory bandwidth of 92.58PB/s on 35, 784, 000 processing elements, a significant milestone that highlights the capabilities of AI-customized architectures to enable a new generation of seismic algorithms that will empower multiple technologies of our low-carbon future.

References

[1]

Sameh Abdulah, Hatem Ltaief, Ying Sun, Marc G. Genton, and David E. Keyes. 2018. Parallel Approximation of the Maximum Likelihood Estimation for the Prediction of Large-Scale Geostatistics Simulations. In 2018 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, Belfast, UK, 98--108.

[2]

Aria Abubakar, Mette Juncker Brædstrup, Haibin Di, Alberto Troya Diaz, Steve Freeman, Simon Hviid, Knud Ho Karkov, Sachin Kriplani, Sunil Manikani, Gwenaëlle Salun, and Tao Zhao. 2021. Deep learning applications for wind farms site characterization and monitoring. 3009--3013. arXiv:https://library.seg.org/doi/pdf/10.1190/segam2021-3583026.1

[3]

Michael Aftosmis, Marsha Berger, and Scott Murman. 2004. Applications of spacefilling-curves to Cartesian methods for CFD. In 42nd AIAA Aerospace Sciences Meeting and Exhibit. AIAA, 1232.

[4]

K. Akbudak, H. Ltaief, A. Mikhalev, and D. Keyes. 2017. Tile Low Rank Cholesky Factorization for Climate/Weather Modeling Applications on Manycore Architectures. In 32nd International Conference on High Performance, Frankfurt, Germany. Springer, 22--40.

Digital Library

[5]

N. Al-Harthi, R. Alomairy, K. Akbudak, R. Chen, H. Ltaief, H. Bagci, and D. Keyes. 2020. Solving Acoustic Boundary Integral Equations Using High Performance Tile Low-Rank LU Factorization. In 35th International Conference on High Performance, Frankfurt, Germany. Springer, 209--229.

Digital Library

[6]

Rabab Alomairy, Wael Bader, Hatem Ltaief, Youssef Mesri, and David Keyes. 2022. High-Performance 3D Unstructured Mesh Deformation Using Rank Structured Matrix Computations. ACM Trans. Parallel Comput. 9, 1, Article 4 (mar 2022), 23 pages.

Digital Library

[7]

Sivaram Ambikasaran, Daniel Foreman-Mackey, Leslie Greengard, David W Hogg, and Michael O'Neil. 2015. Fast direct methods for Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 2 (2015), 252--265.

Digital Library

[8]

Patrick Amestoy, Cleve Ashcraft, Olivier Boiteau, Alfredo Buttari, Jean-Yves L'Excellent, and Clément Weisbecker. 2015. Improving Multifrontal Methods By Means Of Block Low-Rank Representations. SIAM Journal on Scientific Computing 37, 3 (2015), A1451--A1474.

Digital Library

[9]

Amirhossein Aminfar, Sivaram Ambikasaran, and Eric Darve. 2016. A fast block low-rank dense solver with applications to finite element matrices. J. Comput. Phys. 304 (2016), 170--188.

Digital Library

[10]

F. Aminzadeh, J. Brac, and T. Kunz. 1997. 3D Salt and Overthrust models. In SEG/EAGE Modeling Series, SEG Book Series Tulsa, Oklahoma, Vol. 1.

[11]

L. Amundsen. 2001. Elimination of Free-surface Related Multiples Without Need of a Source Wavelet. Geophysics 66 (2001), 327--341.

[12]

D. Boiero and C. Bagaini. 2020. Up-down deconvolution in complex geological scenarios. Online EAGE Conference and Exhibition, Extended Abstracts (2020).

[13]

Steffen Börm. 2010. Efficient Numerical Methods for Non-Local Operators: H2-matrix Compression, Algorithms and Analysis. Vol. 14. European Mathematical Society.

[14]

Kenneth Bredesen, Esben Dalgaard, Anders Mathiesen, Rasmus Rasmussen, and Niels Balling. 2020. Seismic characterization of geothermal sedimentary reservoirs: A field example from the Copenhagen area, Denmark. Interpretation 8 (2020). Issue 2.

[15]

Qinglei Cao, Sameh Abdulah, Rabab Alomairy, Yu Pei, Pratik Nag, George Bosilca, Jack Dongarra, Marc G. Genton, David E. Keyes, Hatem Ltaief, and Ying Sun. 2022. Reshaping Geostatistical Modeling and Prediction for Extreme-Scale Environmental Applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Dallas, Texas). Association for Computing Machinery, 13--24. Gordon Bell Prize Finalist.

Digital Library

[16]

T. F. Chan. 1987. Rank Revealing QR Factorizations. Linear Algebra and Its Applications 88/89 (1987), 67--82.

[17]

Eduardo Corona, Per-Gunnar Martinsson, and Denis Zorin. 2015. An O(N) Direct Solver For Integral Equations On The Plane. Applied and Computational Harmonic Analysis 38, 2 (2015), 284--317.

[18]

G. H. Golub and C. F. Van. 2012. Matrix Computations. Vol. 3. Third Edition, Johns Hopkins University Press 2012.

[19]

SA Goreinov, EE Tyrtyshnikov, and A Yu Yeremin. 1997. Matrix-Free Iterative Solution Strategies For Large Dense Linear Systems. Numerical Linear Algebra with Applications 4, 4 (1997), 273--294.

[20]

Wolfgang Hackbusch. 1999. A Sparse Matrix Arithmetic Based on H-matrices. Part I: Introduction to H-Matrices. Computing 62, 2 (1999), 89--108.

Digital Library

[21]

N. Halko, P. G. Martinsson, and J. A. Tropp. 2011. Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions. SIAM Rev. 53, 2 (2011), 217--288. arXiv:https://doi.org/10.1137/090771806

Digital Library

[22]

Y. Hong, H. Ltaief, M. Ravasi, L. Gatineau, and D.E. Keyes. 2021. Accelerating Seismic Redatuming Using Tile Low-Rank Approximations on NEC SX-Aurora TSUBASA. Supercomputing Frontiers and Innovations 8 (2021). Issue 2.

[23]

Y. Hong, H. Ltaief, M. Ravasi, D.E. Keyes, and D. Vargas. 2022. Large-scale Marchenko imaging with distance-aware matrix reordering, tile low-rank compression, and mixed-precision computations. SEG Technical Abstract (2022).

[24]

Yuxi Hong, Hatem Ltaief, Matteo Ravasi, and David E. Keyes. 2022. HPC Seismic Redatuming by Inversion with Algebraic Compression and Multiple Precisions. Submitted to International Journal of HPC Applications (2022). http://hdl.handle.net/10754/685271

[25]

Mathias Jacquelin, Mauricio Araya-Polo, and Jie Meng. 2022. Scalable Distributed High-Order Stencil Computations. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (Dallas, Texas) (SC '22). IEEE Press, Article 30, 13 pages.

Digital Library

[26]

H. Jakubowicz. 1998. Wave equation prediction and suppression of interbed multiples. Society of Exploration Geophysicists (1998).

[27]

Bander Jumah and Felix J. Herrmann. 2012. Dimensionality-reduced Estimation of Primaries by Sparse Inversion. SEG Technical Program Extended Abstracts (2012), 3520--3525.

[28]

Ronald Kriemann. 2013. H-LU Factorization on Many-Core Systems. Computing and Visualization in Science 16, 3 (June 2013), 105--117.

Digital Library

[29]

Lin Lin, Jianfeng Lu, and Lexing Ying. 2011. Fast construction of hierarchical matrix representation from matrix-vector multiplication. J. Comput. Phys. 230, 10 (2011), 4071--4087.

Digital Library

[30]

G. A. Lopez and D.J. Verschuur. 2015. 3D Focal Closed-Loop SRME For Shallow Water. Geophysical Journal International 203 (2015), 792--813.

[31]

Hatem Ltaief, Jesse Cranney, Damien Gratadour, Yuxi Hong, Laurent Gatineau, and David Keyes. 2021. Meeting the Real-Time Challenges of Ground-Based Telescopes Using Low-Rank Matrix Computations. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (St. Louis, Missouri). Association for Computing Machinery, New York, NY, USA, Article 29, 16 pages.

Digital Library

[32]

H. Ltaief, Y. Hong, A. Dabah, R. Alomairy, S. Abdulah, C. Goreczny, P. Gepner, M. Ravasi, D. Gratadour, and D. Keyes. 2023. Steering Customized AI Architectures for HPC Scientific Applications. In International Supercomputing Conference, Vol. 13948. Springer Lecture Notes in Computer Science (LNCS), 125--143. https://link.springer.com/chapter/10.1007/978-3-031-32041-5_7

[33]

D. Lumley. 2019. The Role of Geophysics in Carbon Capture and Storage. In Geophysics and Geosequestration. Cambridge: Cambridge University Press.

[34]

Christopher Paige and Michael A. Saunders. 1982. LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares. ACM Trans. Math. Software 8 (1982). Issue 1.

[35]

M. Ravasi, T. Selvan, and N. Luiken. 2022. Stochastic multi-dimensional deconvolution. IEEE Transactions on Geoscience and Remote Sensing 60 (2022).

[36]

Matteo Ravasi and Ivan Vasconcelos. 2021. An Open-source Framework for the Implementation of Large-scale Integral Operators with Flexible, Modern HPC Solutions - Enabling 3D Marchenko Imaging by Least Squares Inversion. Geophysics 86 (2021), WC177--WC194.

[37]

M. Ravasi, I. Vasconcelos, A. Curtis, and A. Kritski. 2015. Multi-dimensional free-surface multiple elimination and source deblending of Volve OBC data. 77th Conference and Exhibition, EAGE, Extended Abstracts (2015).

[38]

François-Henry Rouet, Xiaoye S Li, Pieter Ghysels, and Artem Napov. 2016. A Distributed-memory Package for Dense Hierarchically Semi-separable Matrix Computations Using Randomization. ACM Transactions on Mathematical Software (TOMS) 42, 4 (2016), 27.

Digital Library

[39]

H Sagan. 1994. Space-Filling Curves. Springer.

[40]

Justin Selig. 2022. The Cerebras Software Development Kit: A Technical Overview. Technical Report. Cerebras.

[41]

J. van der Neut and F. Herrmann. 2013. Interferometric redatuming by sparse inversion. Geophysical Journal International 192 (2013), 666--670.

[42]

G. J. van Groenestijn and D. J. Verschuur. 2009. Estimating Primaries by Sparse Inversion and Application to Near-offset Data Reconstruction. Geophysics 74 (2009), 1MJ-Z54. Issue 3.

[43]

D. Vargas, I. Vasconcelos, M. Ravasi, and N. Luiken. 2021. Time-domain multidimensional deconvolution: A physically reliable and stable preconditioned implementation. Remote Sensing 13 (2021). Issue 18.

[44]

I. Vasconcelos, M. Ravasi, A. Kritski, J. van der Neut, and T. Cui. 2017. Local, reservoir-only reflection and transmission responses by target-enclosing extended imaging. SEG Technical Program Expanded Abstracts (2017), 5289--5293.

[45]

D. J. Verschuur. 1992. Surface-related Multiple Elimination in Terms of Huygens Sources. Journal of Seismic Exploration 1 (1992), 49--59.

[46]

K. Wapenaar, J. Thorbecke, J. van der Neut, F. Broggini, E. Slob, and R. Snieder. 2014. Marchenko Imaging. Geophysics 79 (2014), WA39--WA57. Issue 3.

[47]

K. Wapenaar, J. van der Neut, E. Ruigrok, D. Draganov, J. Hunziker, E. Slob, J. Thorbecke, and R. Snieder. 2011. Seismic interferometry by crosscorrelation and by multidimensional deconvolution: A systematic comparison. Geophysical Journal International 185 (2011), 1335--1364.

[48]

William Wulf and Sally McKee. 1995. Hitting the Memory Wall: Implications of the Obvious. ACM SIGARCH Computer Architecture News 23 (1995), 20--24.

Digital Library

[49]

Kezhong Zhao, M.N. Vouvakis, and Jin-Fa Lee. 2005. The adaptive cross approximation algorithm for accelerated method of moments computations of EMC problems. IEEE Transactions on Electromagnetic Compatibility 47, 4 (2005), 763--773.

Cited By

Hong YLtaief HRavasi MKeyes D(2024)High performance computing seismic redatuming by inversion with algebraic compression and multiple precisionsInternational Journal of High Performance Computing Applications10.1177/1094342023122619038:3(225-244)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1177/10943420231226190
Luczynski PGianinazzi LIff PWilson LDe Sensi DHoefler TMencagli GDazzi PLowenthal DBadia R(2024)Near-Optimal Wafer-Scale ReduceProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658693(334-347)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658693
Song SHuang YJiang PYu XZheng WDi SCao QFeng YXie ZCappello FMencagli GDazzi PLowenthal DBadia R(2024)CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658691(309-321)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658691

Recommendations

Exploring Processing In-Memory for Different Technologies
GLSVLSI '19: Proceedings of the 2019 Great Lakes Symposium on VLSI

The recent emergence of IoT has led to a substantial increase in the amount of data processed. Today, a large number of applications are data intensive, involving massive data transfers between processing core and memory. These transfers act as a ...
Cache Design with Domain Wall Memory
Domain wall memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with very high density and ...
Scaling the "memory wall"
ICCAD '12: Proceedings of the International Conference on Computer-Aided Design

DRAM has been the technology for computer main memory since Intel released the first commercial DRAM chip (i1103) in 1970. As technology scales and demand for memory performance, it seems DRAM is facing several challenges. Many other memory technologies ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '23: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2023

1428 pages

ISBN:9798400701092

DOI:10.1145/3581784

Chair:
Dorian Arnold,
Program Chair:
Rosa M Badia,
Program Co-chair:
Kathryn Mohror

Copyright © 2023 Owner/Author(s).

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2023

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

King Abdullah University of Science and Technology

Conference

SC '23

Sponsor:

SIGHPC

SC '23: International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2023

CO, Denver, USA

Acceptance Rates

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
668
Total Downloads

Downloads (Last 12 months)668
Downloads (Last 6 weeks)78

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hong YLtaief HRavasi MKeyes D(2024)High performance computing seismic redatuming by inversion with algebraic compression and multiple precisionsInternational Journal of High Performance Computing Applications10.1177/1094342023122619038:3(225-244)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1177/10943420231226190
Luczynski PGianinazzi LIff PWilson LDe Sensi DHoefler TMencagli GDazzi PLowenthal DBadia R(2024)Near-Optimal Wafer-Scale ReduceProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658693(334-347)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658693
Song SHuang YJiang PYu XZheng WDi SCao QFeng YXie ZCappello FMencagli GDazzi PLowenthal DBadia R(2024)CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658691(309-321)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658691

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents