OpenAI API

- For AI systems to be truly useful in a development setting, they have to be fully configurable and provide basic features like state management and dynamic controls. But when OpenAI first released their API, developers had to do all this manually. So when they built custom agents, they had to declare system assistant and user messages with every request and manage their state by storing each message response pair in some form of database, and then pass that to store data back into the API with every message. It was labor intensive and expensive and quite messy. In response to this, OpenAI has now introduced the Assistants API. This new API is the developer version of ChatGPT's Custom GPTs, and with it, you can create custom assistants with their own instructions, their own tools, and their own knowledge retrieval and spin up individual stateful chats under each assistant without having to store the exchange on your end. This makes developing custom agents with API significantly easier and opens the door to far more rapid and advanced AI development. You can create and configure assistants programmatically through the API and also through the OpenAI Playground, and you can change them at any time through both of these modalities. Now, assistants work by tying three distinct components together to create a seamless user experience. First, there's the assistant itself. It has a unique ID and you can create as many assistants as you like. Each assistant holds the main system instruction or prompt, configuration of tools, including custom function calls, the code interpreter, and content retrieval, and the ability to upload knowledge. So, documents to be used by the assistants for reference. Next, there are threads. Each thread is a separate continuous chat with an assistant, with its own unique ID, and you can have as many threads as you want with any one of your assistants. The thread stores the chat history, and this is what makes each chat stateful. And this means you can have multiple parallel threads and jump back and forth between them based on their ID without losing the thread. I mean, losing the state of each separate chat. And finally, there are runs. Each run contains a message or prompt sent to the API and the response from the AI. Think of a run as a prompt-response pair. The interesting thing about runs is they can have multiple steps, each of which is executed separately. So for example, when you send a message, the assistant may trigger the code interpreter tool and a custom function call for responding, totaling three steps. This means a single prompt can do multiple things at once, even fire off sequences of functions and tool calls. Now, one important thing to note here is the way assistants preserve states and context for each of these chats is by passing the entire thread back in with each run and then adding the new message and response to that thread. So as you go back and forth with the assistant, the token count for each run will grow as long as the conversation keeps growing and can end up consuming a lot of tokens. Bottom line, with assistants, you can now programmatically spin up custom assistants in the GPT environment and set them up to do whatever you want and integrate them in any way you want with your own applications.