Schillace's laws of semantic AI

- So if you're like me, last year, OpenAI beta was a hobby. That was my hobby. And until I joined Sam's team where it's become a full-time, full-on passion, and when the ChatGPT sonic boom, I've come to realize how the lessons that Sam has inside his GPT mind are extremely useful. So that's why the Schillace Laws to me are like where I go to, to kind of orient myself, and one of them is, "Ask smart to get smart." Can you talk about that Sam? - Yeah, let's talk about this. So yeah, you have to think about these models as programming tools in very particular ways. So one way is we get a little bit fooled because it seems smart and it is smart, it knows a lot of stuff, it seems smart. But it it doesn't really have the same energy that a person does. Like if you ask a person something, the person wants to show off and so they'll tell you everything they know about the subject. But the model doesn't really have that. So if you ask a simple question, you'll get a simple answer back. If you ask a more complicated question, you will get a more complicated question back. So often when you're enacting with the model when you're building a prompt or saying things into it, you have to think really hard about like how to structure your question so that you get the answer back that you need. Well, one thing we do sometimes is we ask like to have it return things in XML format or to give it a list or to be elaborate in certain ways. And like you need to give the model that kind of instruction to really get back out what you want. You can't just say like, give me this answer and have it be a short question, because you'll get a dumb answer back. It works in many different contexts. - As someone who is trained as a hardware engineer I love this Schillace Law about text being the universal wire protocol. - Yeah. - Talk about that. - This is really interesting. So you know, we're doing a bunch of code that is using these large language models and you know, we're kind of sending stuff in and out. And right now they're mostly texts. They'll eventually become multimodal and stuff like that. Well one of the things we figured out is you can sort of hand stuff around. You can write a prompt that is a planner, that has a bunch of pieces that need to go get executed. And you can start to think about, well like some of those pieces might get executed by other kernels somewhere else, right? Like I might build a kernel into a native app or into the desktop or into a browser and I want to hand part of the task over. Okay, so that's serialization. Well how do we do that, right? Well normally you would build some complicated serialization protocol 'cause we need to communicate across these services or functions. But we don't need to do that anymore. You can actually do that communication just with natural language. So text is the serialization format. You can just talk to different, and this happens too, if you're just doing a program that's multi-step, doesn't have to be multiple kernels. If you want to pass things from one prompt, to another from one skill to another, the right way to do it is often just natural language or just some natural text. And you can often just get away with describing things. The models are very good at picking the order back up out of that unstructured chaos. So text is like this kind of wire protocol now for sending things back and forth. You don't really need to do as much work to build these rigid near interchange formats. It's really interesting. - One of my favorite things that happens in the developer channel for a Semantic Kernel is when some appears with another Schillace Law. Can you talk about that one? - Yeah, this is an important one actually because we're still in the early innings of this thing, right? So we're still in the stages where the base models are getting better. In fact, often the next generation of the base model will be better than the last generation fine-tuned was. So like the models are continually getting better. So there's this real temptation to like, if you're close to a problem, to write a bunch of code to kind of get the rest of the way there, 'cause we're just used to writing code, like we're used to writing in programming languages. But if the model can do it at all, even a little bit, you should let the model do it, because the model will get better over time but the code won't. The code is not, you know, the code is the code. You might go back and rewrite it, but the code is otherwise going to be static unless you go back to it. So as long as you can get the model to do something like a plan or like a formatting or like a parsing or anything like that, it's probably better to have the model do that part of the program, if it can do it reliably, 'cause it will only get better at doing that. And there are things like planning that are just barely in the capability of the current models and they don't seem super practical, but once you start to break the problems down and get them to do that, they do work pretty well. - And the Schillace Laws are available in our knowledge base, open source. And before we go Sam, I know this idea of hybrid, cyborg, native code and semantic code coming together for those who know the, we're from the same generation, the Bionic Man (laughs), the Six Million Dollar Man, that era seems to be here in code-- but also. - Sure. - On the developer side too. They've become bionic. - They've become bionic. But yeah, well the interesting things you need to think about a little bit is that you have two different kinds of code that you're writing now. You have the code where we're used to, the imperative code written in a programming language and you have these prompt natural language skills that you're writing. That's the cyborg thing, right? Like that's the machine and the kind of more organic and just like any system like that, the joint is the hard part, right? Joining them together is kind of the hard part. So you know, each one does its piece pretty well. Each one does the other's piece pretty badly. Crossing that boundary is kind of hard. So you might have something on the AI side that's getting some stuff that's generated. You got to hand it back over to the programming language side to parse that out. So that can be hard, 'cause the AI doesn't necessarily generate it, you know, perfectly reliably and things like that. So thinking carefully about how you cross back and forth across that boundary, how you do loops where the AI is involved or things like that is really interesting and important to do. I'll add one bonus law to this conversation, which I think is real important, which is, and well it's a kind of a whole general principle but it's sort of like the easiest way to say it is, "Easy for you, "is easy for it," and vice versa, "Hard for you is hard for it." So like often one of the best ways that we have to think about how you build a program using a large language model is to think about how you would do it as a person. So for example, like sometimes you ask questions where it has to really reason through the answer. And these models are pretty good at doing that. They'll like reason out. They'll work through stuff out loud. But as a programmer you might just want the answer, right? Like you just want the number at the end of it or something like that. You don't need all this text, 'cause that's going to gum up your programming, the natural language side. You just want to get the string that can be turned into a number that could be run through the next step of the program. So it's tempting to like kind of squeeze and cram on this thing to try to get a prompt that perfectly just answers the question. But that doesn't really work that well, 'cause it wouldn't work that well for you as a person. You can't kind of do it all in one shot either. What is easy is to ask a question, get a big answer and then if I handed that to you, and if you weren't an expert in that domain and I said look at this conversation between these two people, the question and the answer, and to pick the answer out. That's much easier to do, even if you don't know how to solve the problem. You can say, well like I'm reading this and there's the answer and I'll pick it back out. And it turns out that's easy for the model too. So like easy for you is easy for it, hard for you is hard for it. So you have to really think about like, how would I do this as a person? How would my brain do this? And then think about how you map that onto the the model itself, 'cause it's often the right way to think through a programming problem. That might be one of the most important principles we've found yet. - Schillace Laws available in the knowledge base. Thank you for tuning in and we are-- - Semantic Kernel.