index.md

layout	default
title	GenAI Handbook

William Brown
@willccbb | willcb.com
v0.1 (June 5, 2024)

{% include intro.md %}

Section I: Foundations of Sequential Prediction

Goal: Recap machine learning basics + survey (non-DL) methods for tasks under the umbrella of “sequential prediction”.

{% include section1.md %}

Section II: Neural Sequential Prediction

Goal: Survey deep learning methods + applications to sequential and language modeling, up to basic Transformers.

{% include section2.md %}

Section III: Foundations for Modern Language Modeling

Goal: Survey central topics related to training LLMs, with an emphasis on conceptual primitives.

{% include section3.md %}

Section IV: Finetuning Methods for LLMs

Goal: Survey techniques used for improving and "aligning" the quality of LLM outputs after pretraining.

{% include section4.md %}

Section V: LLM Evaluations and Applications

Goal: Survey how LLMs are used and evaluated in practice, beyond just "chatbots".

{% include section5.md %}

Section VI: Performance Optimizations for Efficient Inference

Goal: Survey architecture choices and lower-level techniques for improving resource utilization (time, compute, memory).

{% include section6.md %}

Section VII: Sub-Quadratic Context Scaling

Goal: Survey approaches for avoiding the "quadratic scaling problem" faced by self-attention in Transformers.

{% include section7.md %}

Section VIII: Generative Modeling Beyond Sequences

Goal: Survey topics building towards generation of non-sequential content like images, from GANs to diffusion models.

{% include section8.md %}

Section IX: Multimodal Models

Goal: Survey how models can use multiple modalities of input and output (text, audio, images) simultaneously.

{% include section9.md %}

{% include credits.md %}