From the course: Generative AI Skills for Creative Content: Opportunities, Issues, and Ethics

Image-to-image generative AI

- Let's now explore image to image generative AI. That simply means that new images are generated from existing images. These AI models are typically designed using a machine learning model that includes two neural networks, a generator and a discriminator. The generator, which is trained on a data set of real images, creates the imagery. The discriminator is trained on the same data set but is designed to determine whether or not the images are real. Over time, both the generator and the discriminator improve at their respective jobs until the image generator is able to create new imagery with exceptional accuracy. Image to image generative AI models are often harder to train than text to image AI. That's because they need to learn the relationship between images and their core corresponding visual traits, like shape, color, texture, lighting, and more. Contrast that with text to image AI, which focuses mainly on learning relationships between text and images. In general, this means that image to image generative AI creates more realistic imagery consistent with the input image. There are many different use cases for this type of technology. Like text image AI, it can be used to create art and other original imagery but it's also an effective tool to create a wide variety of interesting face filters like changing someone's age or generating a collection of professional headshots or profile photos or interesting avatars. You can also create different lighting scenarios for an image. You can restore, up res, and enhance low res or noisy images or add color to black and white photos. You can transfer the style from a specific painting or art movement from any time in history or transform a basic sketch into a complete, synthesized image. There are also ways to combine text to image and image to image AI. Most text to image AI programs let you upload an original image and then use text to alter it however you like. As you can imagine, because you're working with your own assets, image to image AI is often most useful when you want to garner specific results. Maybe you need a specific color palette or composition or you want to add AI generated elements to an existing image. That's contrasted with text to image AI, where you start with a blank slate and see what the AI comes up with. Here's one way of looking at it. When you provide some type of starter image, it's a bit like sending a rough sketch to a designer and telling them, here's what I have in mind. Now, do a refined version. When you're starting from scratch with just a text prompt, it's a little like saying, here's what I have in mind, see what you can come up with. That isn't a perfect comparison of course but you get the idea. Both aspects of AI powered image generation are useful and both can be great tools for brainstorming, ideation and creative inspiration.

Contents