FastAPI Endpoint for Sparrow LLM Agent

FastAPI Endpoint for Sparrow LLM Agent. I show how FastAPI endpoint is used in Sparrow to run LLM agent functionality from API client. 

 

FastAPI File Upload and Temporary Directory for Stateless API

I explain how to handle file upload with FastAPI and how to process the file by using Python temporary directory. Files placed into temporary directory are automatically removed once request completes, this is very convenient for stateless API. 

 

FastAPI and LlamaIndex RAG: Creating Efficient APIs

FastAPI works great with LlamaIndex RAG. In this video, I show how to build a POST endpoint to execute inference requests for LlamaIndex. RAG implementation is done as part of Sparrow data extraction solution. I show how FastAPI can handle multiple concurrent requests to initiate RAG pipeline. I'm using Ollama to execute LLM calls as part of the pipeline. Ollama processes requests sequentially. It means Ollama will process API requests in the queue order. Hopefully, in the future, Ollama will support concurrent requests. 

 

How to Host FastAPI from Your Computer with ngrok

With ngrok you can host your FastAPI app from your computer. This can be a handy and cheaper option for some projects. In this video, I explain my experience running FastAPI apps from my very own Cloud with ngrok :)

 

ChatGPT Plugin Backend with FastAPI

This tutorial explains how to integrate FastAPI backend with ChatGPT plugin implemented in Python. Backend stores data from ChatGPT in MongoDB to be persistent and available across sessions. 

 

FastAPI, Pydantic and MongoDB for Beginners

I show how to initialize a connection to MongoDB from FastAPI endpoint with a startup event. Before pushing it to MongoDB collection, new record validation is done with Pydantic. I like the flexibility of MongoDB Motor async library. It helps to implement seamless communication from FastAPI to MongoDB.

 

PaddleOCR as a Service with FastAPI

PaddleOCR is a great tool to extract text data from docs, and it can group related words into a sentence. Such functionality can simplify extracted data analysis. In this video, I explain how to run it as a service with FastAPI in Python. 

 

Speeding Up FastAPI App with Background Tasks

FastAPI runs background tasks in a parallel thread, which prevents blocking app endpoints when a long task executes. I explain it in this video and show the benefit of running time-consuming operations in background tasks.

 

Optimizing FastAPI for Concurrent Users when Running Hugging Face ML Models

To serve multiple concurrent users accessing FastAPI endpoint running Hugging Face API, you must start the FastAPI app with several workers. It will ensure current user requests will not be blocked if another request is already running. I show and describe it in this video. 

 

Optimizing ML Model Loading Time Using LRU Cache in FastAPI

Are you facing challenges with the time it takes to load large ML models in your backend API? This video presents a practical solution: utilizing LRU cache with properly annotated functions. Implementing this approach will make your model cached in memory, eliminating the need for disk reads on subsequent calls. Enhance the efficiency and performance of your ML workflow by incorporating LRU cache techniques. Join us to learn more about this valuable strategy! 

 

Deploying FastAPI Applications to Hugging Face Spaces

In this video, I demonstrate how to deploy a FastAPI backend API to Hugging Face Spaces using Docker. I cover creating a Dockerfile, setting up secrets for FastAPI, and deploying the application on the platform.

     

Build a Structured API with FastAPI

Learn how to create a structured API using FastAPI. In this tutorial, we explore the benefits of FastAPI, its powerful code structuring capabilities, and how it connects services within Sparrow to build scalable and efficient applications. Unleash the true potential of FastAPI and enhance your app development process! 

 

Building Bigger Applications with FastAPI

I share a few tips and tricks on how to build a clean and readable REST API for larger apps with FastAPI framework. You will learn how to split endpoints into separate Python scripts and assign tags with prefixes for all endpoints from the given router at once.

 

Scale FastAPI on Kubernetes Pod

This video shows how to scale FastAPI REST endpoint running on Kubernetes Pod.

 

BIY Workflow with FastAPI, Python and Skipper

Build It Yourself. In this video, I explain how you can build workflow running with FastAPI REST generic endpoints. The best thing about it, Skipper architecture is modular and workflow runs in a separate Docker container. This means you can replace it with your own implementation if needed. I explain how workflow call is integrated into FastAPI logic and how the call is made to get the queue name from the workflow. Using this queue name and RabbitMQ message broker, event-based communication runs between containers.

 

MLOps: Extend Skipper ML Services

The goal of this video is to explain Skipper from MLOps user perspective, different blocks of Skipper and how they fit together. I show how a sample set of ML services works and how you could replace it or add your own service. Skipper engine is implemented with Python, but you could add service container implemented in any language. All runs on Kubernetes.

 

FastAPI on Kubernetes with NGINX Ingress

A simple tutorial about a complex thing - how to expose FastAPI app to the world on Kubernets with NGINX Ingress Controller. I explain the structure of Kubernetes Pod for FastAPI along with Kubernetes service. I show how FastAPI properties should be set to be accessible through Ingress path definition. You will learn how to check the log for NGINX Ingress Controller and FastAPI Pod.

 

Routing Traffic Between FastAPI Pods in Kubernetes

This is a quick tutorial to show how to route traffic between Kubernetes Pods. Both Pods are running FastAPI endpoints. I show how to create Deployment and Service elements for Kubernetes Pod, and how to refer to that service from another Pod to execute HTTP call.

 

FastAPI Running on Kubernetes Pod

Step-by-step tutorial where I explain and show how to run FastAPI app on Kubernetes Pod. I keep it simple. I explain when it makes sense to use multiple containers in a single Pod and when you should put containers into different Pods.

 

Serving ML Model with Docker, RabbitMQ, FastAPI and Nginx

In this tutorial I explain how to serve ML model using such tools as Docker, RabbitMQ, FastAPI and Nginx. The solution is based on our open-source product Katana ML Skipper (or just Skipper). It allows running ML workflow using a group of microservices. It is not limited to ML, you can run any workload using Skipper and plugin your own services. You can reach out to me if you got any questions.