Can a language model read behavior the way a good recommender engineer reads a session — not as anonymous IDs, but as clues about taste, intent, and the next plausible choice? A six-part arc, from prompt-based tests to a repo-backed cross-encoder.
Traditional recommenders compress users and items into learned vectors and measure distance between them. This project explores a different interface: give a language model a user's recent behavior, the candidate item, and sometimes attributes or reasoning traces — then ask whether it can infer what the user would actually like.
The bet is that preference prediction can benefit from language-model reasoning: plot knowledge, genre associations, taste explanations, cold-start behavior, and the ability to read natural-language metadata that an ID-based model never sees. The old RecSys stack — collaborative filtering, matrix factorization, sequential models — is kept as the control group throughout.
Not anonymous IDs, but clues about taste, intent, genre memory, and the next plausible choice.
The basic unit is a user's last ten movies and a single candidate film: does it match the inferred taste profile? Built on MovieLens 1M (a million ratings, six thousand users), the task is framed as a binary like/dislike call, with SVD matrix factorization and the sequential transformer SASRec as baselines.
Zero-shot and few-shot prompting struggled with precision; classical baselines had the opposite failure. But a LoRA fine-tune of a small Llama model overtook both — and it got there on roughly 6,400 examples where SASRec needed 900,000. The pre-trained world knowledge buys an enormous head start in data efficiency.
Sweeping Qwen2.5 from 0.5B to 14B parameters showed real but flattening gains — diminishing returns past about 7B, and the 1.5B model sometimes beating larger siblings. Under a short training budget, the bigger models learned the task while the smallest hovered near random.
Adding user attributes — age, gender, occupation — improved results even in cold-start setups where there was no history to read at all. The honest conclusion: fine-tuned LLMs can match or beat specialized recommenders, but the win has to be weighed against compute and inference latency.
Next, GPT-4o generated short reasoning traces — under fifty tokens explaining why a user would like or dislike a film from the genre, theme, and style of their history. Training a 1.5B model to emit both an explanation and a verdict matched or edged past the baselines on a tenth of the engagement data, and pointed toward more transparent recommendations.
Then the reinforcement-learning route: GRPO, with no synthetic traces, scoring groups of responses relative to their average. The model gradually taught itself the <reasoning> / <answer> format on its own — but the result was the genuine surprise of the series.
SFT on well-curated reasoning data turned out to be a robust baseline — beating pure RL, despite the theory.
A detour into generative recommendation: instead of retrieving and ranking from a fixed catalog, model the user–item interaction as a sequence and have the model produce the next item directly. A LoRA-tuned Qwen2.5-7B was trained to predict the next movie from the last ten, evaluated top-1 against sampled negatives.
It worked as a proof of concept — clear signs of learning, with the caveats stated plainly: single-epoch training, thin evaluation, no strong baseline yet. The interesting shift is conceptual: recommendation as generation rather than selection.
The arc lands on a bridge between worlds. Dual-encoders embed the user and the item separately and compare the two vectors; a cross-encoder feeds the user history and candidate through a single model together — joined by a [CAND] token — so the model can inspect their interaction directly rather than after the fact.
Qwen2.5-7B produces the representation, an MLP head scores it as a five-way rating prediction. Title alone was weak; titles plus genre and rating moved the needle sharply, reaching an F1 around 0.75. It's the natural home for everything the series learned — and it's the one piece with a first-party repo behind it.
LLMs bring semantic understanding, but recommendation still punishes latency, calibration errors, popularity bias, and weak evaluation. The project lives in that tension — it doesn't pretend language models simply replace ranking systems.