From the course: Generative AI: Working with Large Language Models
Unlock the full course today
Join today to access over 23,100 courses taught by industry experts.
Gopher
From the course: Generative AI: Working with Large Language Models
Gopher
- [Instructor] The DeepMind research team released Gopher in January of 2022. They released six flavors of the model ranging from 44 million parameters to 280 billion parameters. And they also put together a diverse dataset called MassiveText and then they tested the model on 152 different tasks. Now in the next few minutes, we'll look at each of these tasks in a little more detail. So let's take a look at the architecture first. And you can see it's similar to GPT-3, where you're just using the decoder portion of the transformer. And in their paper, the DeepMind team presented results for the six models with the smallest model being 44 million, all the way to 280 billion parameters. Now, the reason the model sizes increase is because you can see that we're increasing the number of layers, the number of self attention heads, and so on as we move down the table. Now let's take a look at what data Gopher was trained on.…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
GPT-34m 32s
-
(Locked)
GPT-3 use cases5m 27s
-
(Locked)
Challenges and shortcomings of GPT-34m 17s
-
(Locked)
GLaM3m 6s
-
(Locked)
Megatron-Turing NLG Model1m 59s
-
(Locked)
Gopher5m 23s
-
(Locked)
Scaling laws3m 14s
-
(Locked)
Chinchilla7m 53s
-
(Locked)
BIG-bench4m 24s
-
(Locked)
PaLM5m 49s
-
(Locked)
OPT and BLOOM2m 51s
-
-