research-article

Open access

Accelerating applications using edge tensor processing units

Authors:

Kuan-Chieh Hsu,

Hung-Wei TsengAuthors Info & Claims

SC '21: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 56, Pages 1 - 14

https://doi.org/10.1145/3458817.3476177

Published: 13 November 2021 Publication History

PDF eReader

Abstract

Neural network (NN) accelerators have been integrated into a wide-spectrum of computer systems to accommodate the rapidly growing demands for artificial intelligence (AI) and machine learning (ML) applications. NN accelerators share the idea of providing native hardware support for operations on multidimensional tensor data. Therefore, NN accelerators are theoretically tensor processors that can improve system performance for any problem that uses tensors as inputs/outputs. Unfortunately, commercially available NN accelerators only expose computation capabilities through AI/ML-specific interfaces. Furthermore, NN accelerators reveal very few hardware design details, so applications cannot easily leverage the tensor operations NN accelerators provide.

This paper introduces General-Purpose Computing on Tensor Processing Units (GPTPU), an open-source, open-architecture framework that allows the developer and research communities to discover opportunities that NN accelerators enable for applications. GPTPU includes a powerful programming interface with efficient runtime system-level support---similar to that of CUDA/OpenCL in GPGPU computing---to bridge the gap between application demands and mismatched hardware/software interfaces.

We built GPTPU machine uses Edge Tensor Processing Units (Edge TPUs), which are widely available and representative of many commercial NN accelerators. We identified several novel use cases and revisited the algorithms. By leveraging the underlying Edge TPUs to perform tensor-algorithm-based compute kernels, our results reveal that GPTPU can achieve a 2.46× speedup over high-end CPUs and reduce energy consumption by 40%.

Supplementary Material

MP4 File (Accelerating Applications using Edge Tensor Processing Units.mp4.mp4)

Presentation video

Download
198.29 MB

References

[1]

Google LLC, "Coral M.2 accelerator datasheet." https://coral.withgoogle.com/static/files/Coral-M2-datasheet.pdf, 2019.

Abstract

Supplementary Material

References

Cited By

Index Terms

Recommendations

Accelerating a hydrological uncertainty ensemble model using graphics processing units (GPUs)

Accelerating statistical static timing analysis using graphics processing units

Accelerating cardiac excitation spread simulations using graphics processing units

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Check for updates

Badges

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations