From the course: Rust LLMOps

Unlock this course with a free trial

Join today to access over 23,100 courses taught by industry experts.

VS Code SSH to AWS accelerated

VS Code SSH to AWS accelerated - Rust Tutorial

From the course: Rust LLMOps

Start my 1-month free trial Buy for my team

VS Code SSH to AWS accelerated

“

- [Instructor] Rust is a great place to start for doing high-performance, modern LLMOps because Rust is one of the fastest languages in the world, it doesn't have legacy issues with a scripting language and it can do inference with just one command using cargo. One problem though is where do you run this? If you need to have a very powerful GPU, this can be an issue because not everybody has a powerful GPU. Well, one of the ways you can do this is via remote development and using VS Code. So what you would do in this scenario here is first go to AWS, set up an EC2 instance. You'd have to pick the right one, then go ahead and install VS Code and the remote SSH extension. Next step, SSH into EC2. Then install Rust via the rustup command, just a one liner. Then do a git clone for the Candle repo and you're ready to run inference. Really, it's that easy, and you can do all of this actually via VS Code locally. So let's go ahead and do a demo. So one of the things that's a little bit…

Contents