NLP Course, 2026

Current curriculum covers topic from basic NLP techinques to the most modern ones, that may be helpful for custom training of LLMs:

• NLP Basics: tokenization, text preprocessing, text representations
• Text & Language Models: embeddings, n-gram models, RNNs, LSTMs, seq2seq, attention
• Transformers & LLMs: Transformer, pre-training (MLM/CLM), prompting, fine-tuning, PEFT
• Scaling & Optimization: : distributed training, MoE, KV-cache, Flash Attention, efficient inference, quantization
• Retrieval & Agents: Information Retrieval, RAG, agent-based systems
• Post-training: alignment, RLHF, DPO

Course Staff

• German Gritsai @grgera
• Anastasiia Vozniuk @natriistorm
• Ildar Khabutdinov @depinwhite

Materials

Week #	Date	Topic	Lecture	Seminar	Additional	Recording
1	February 10	Intro to NLP & Tokenization	slides	ipynb	materials	TBA
2	February 17	Feature Extraction and Word Representations	slides	ipynb	materials	TBA
3	February 24	Language Modeling, Seq2Seq, Attention	slides	-	materials	TBA
4	March 3	Transfer Learning, BERT-like, LLMs	slides	ipynb	materials	YouTube
5	March 10	Large Language Models Pre-Training	slides	-	materials	YouTube
6	March 17	Modern LLMs evolution beyond the Transformer	slides	ipynb	materials	YouTube
7	March 31	Training Large Language Models	slides	-	materials	TBA

Homeworks

Task #	Release	Deadline	Inside	Materials
1	March 10	March 19 - 23:59	Explore NLP pipeline	ipynb

Game Rules

Final mark = 0.3 × (oral answer grade) + 0.7 × (average score for practical assignments)

Both oral exam and homeworks are blocking parts, you need to pass both parts to pass the course.

Prerequisities

• Probability Theory + Statistics
• Machine Learning
• Python Python guide
• Basic knowledge on NLP

We expect students to know basics of Natural Language Processing, as the course focuses on more advanced topics. When you unsure about the basics, we recommned to read these lectures / materials:

1. Course from Lena Voita
2. Speech and Language Processing by Jurafsky and Martin
3. Stanford CS 224n
4. Great blog on Transformer & BERT