From the course: UX for AI: Design Practices for AI Developers

Unlock the full course today

Join today to access over 23,100 courses taught by industry experts.

Semantic coding for cost

Semantic coding for cost - Microsoft Copilot Tutorial

From the course: UX for AI: Design Practices for AI Developers

Start my 1-month free trial Buy for my team

Semantic coding for cost

“

- [Instructor] Okay, we're back in the code, and we're worried about cost, and cost correlates to a lot of things in the AI world, especially in the model usage because it costs tokens to use them. You may be using an open source modeling locally, so you're like, ah, no problem. But it is these incredibly powerful foundation models that you really like using, and they cost per token. So you're sitting there, maybe you're by yourself. You don't have like an architect to work with you. That's most people, right? And wouldn't you like to have help? So help me deal with the cost. I was sort of hoping someone out there will help you. Engineering can be lonely, and this is a fundamental issue with LLMs cost. Okay, so there's a bunch of tricks. We call them tricks still, but they are harder knowledge. Caching is a proven means to reducing costs. Therefore, we want to cache the completions, ie GenAI, I ranked 'em myself.…

Contents