Build a Large Language Model (From Scratch)

The book is amazing. Much better than I expected. I was minimally familiar with neural networking techniques (finished 6-months course on Coursera, and by now have forgotten most of it). So, I had a vague idea about forward and backward propagation, remembered such terms as dropout, normalization etc. without actually remembering their meaning. From the Andrew Ng course I remembered the term "transformer" (since he had a few good introductory explanations of it), but by now I completely forgot how it works. My knowledge of Python was very limited (and mostly forgotten). I knew nothing about PyTorch. When I saw the references to the book on Facebook, I decided that it might be helpful for me to recall these concepts, and especially interesting was to learn the concept of transformers and self-attention which I knew belong to the foundation of modern LLMs. The book exceeded my expectations. It is written in an excellent methodical style. Introduces concepts one by one, helps experimenting with them in the real code. It provided an excellent introduction to PyTorch (in Appendix A, which the author recommended to consume before reading the rest of the book). The introduction is short, not overwhelming the reader with millions potential concepts of the huge ecosystem of Python and PyTorch, and still sufficient for productive consuming the entire book that uses both. All the concepts are defined in easy-to-consume steps, leading eventually to a complete overall understanding of GPT model. I am not naive to think that I can develop LLMs by myself now, but I definitely got more than expected. And enjoyed the material a lot. I did not use the code from GitHub (by the book's reference). Instead, I meticulously re-entered all the examples from the book's text into several Jupyter Notebooks in VSCode. This way I moved a bit slower but understood material better. Even found a few minor (typo-level) issues in the code. I am working on an ordinary Surface Book (no GPU), and all examples work instantaneously so far (obviously, it will change when I come to training). I am now in the position after chapter 4: Built the untrained GPT model and cannot wait when I will start training and using it. Highly recommend the book to everyone who wants to make their hands "dirty" with the AI.

This book is an absolute masterpiece. The writer knows how to present complex concepts in simple, absorbable ways. From concepts to labs/demoes, he makes you feel like you’re sitting in an ivy league class. The companion YouTube channel is the icing on the cake. I highly recommend this for anyone interested in learning the fundamentals of ML

The best book I have ever read on generative AI and machine learning. It is essential reading for anyone interested in working in the AI field. Even the appendix is highly valuable and well worth reading.

The best LLM book that I have read so far! Crystal clear.

Good book to get a up close look at LLM development in a rudimentary level. Glosses over some sections so you may need to supplement with some searching but you can make a starter chat bot with what's in book.

This is my first book review. I feel I have to write this to rebut one of the one-star reviews. I have finished the first five chapters of the book, and I absolutely loved it. The book is clearly written and easy to understand. The figures, which the one-star review dismissed as high-school textbook-level, are actually very helpful and serve as roadmaps showing where we are in the book. The book explains concepts that are needed to understand the code and then walks readers line-by-line through the source code towards building an LLM. I now have an in-and-out understanding of what embedding is, how the attention mechanism is implemented, what the GPT block looks like, and how to train a model. The companion notebooks are also helpful. The code works out of the box. Although I still have two chapters on fine-tuning, I already have a thorough understanding of how an LLM works from the first five chapters, and I can't wait to optimize the code and train it with larger datasets.

I was able to instantly take what I learned from this book and find out why past AI projects failed and how to apply it the the project I'm currently working. Even with up to date college classes, the subject area is so vast.

This book is so-so. I wouldn't buy it again. I wanted to learn how the llm works and how the embedding algorithms are designed. Alternatively, he could have discussed the training algorithm of the llm and how the weighting matrices are determined. Alternatively, he could have discussed how the math by setting the vector spaces relate meanings to words so that an llm can convert that into something intelligible as a response. None of this was done. He presents code for llm and uses python libraries. However, it is a black box. All the discussion varies from 2 extremes of high level generalities and then specific lingo and code for particular abstractions. However, virtually nothing is made concrete. Of course some will disagree, but if I knew how llm's worked, I wouldn't write this book. If I don't understand the details of llm functions and code design with llm, this book wouldn't help much. That being said, if you want some code snippets, you will find some useful ones here.

A great book to start and understand AI.

Für LLM-Einsteiger gibt es kein besseres Buch!

Poor printing quality: paper is so thin so one can see letters from back side while reading front side. Also for some reason main cover is not alligned with the rest of the book. Overall impression like it was printed at home

İçerik çok güzel ama ben basım kağıdını beğenmedim

Zeer goed om zelf je eerste LLM te leren maken.