<- AI Systems Dossier
ENTRY NO. 03EdTech / Professional DevelopmentLive deliveryAdaptive Learning / RAGLIVE

Nullfall

Identity · Adaptive learning engine

AI-assisted adaptive learning platform for ML/AI engineers — RAG-grounded explanations, competency progression, spaced repetition

Adaptive Learning / RAG
Problem

Fixed course sequences delivered identically to every learner — no personalisation.

System Flow
7 stages · click to inspect
->
->
->
->
->
->
Click any stage to inspect
Signature Module · Adaptive learning engine

Inspect adaptive engine

Adaptive Learning / RAG

Replay how learner state, competency gaps, and spaced repetition determine the next learning action.

Adaptive Path Trace · Prototype● live
ASSESSCAT diagnostic · θ = 1.24 (above avg)
Converged in 17 items · SE < 0.3
GAPS├─ Transformer attention ········ 0.42
└─ Positional encoding ·········· 0.61
↓ RAG explanation · 768-dim · top-k 8
QUIZ3 items · judge-validated · ~9% rejection
Mastery check: ✗ attention gate
MICROTargeted microdrill · 40 XP · 4 min
SRSNext review: +1d · ease factor 2.5

Business Impact

Outcomes
Before

Curriculum path: Fixed sequence, identical for all learners

->
After

Curriculum path: Personalised adaptive path per placement result

Learners receive paths matched to their demonstrated competency with RAG-grounded explanations and targeted microdrills when mastery checks fail. Explicit XP mechanics (500 XP/level, 40 XP/microdrill, 50 XP/quest) create visible progress milestones. The target metric is a 68% improvement in quiz completion rates vs a fixed-path baseline — not yet measured at scale.

68%
Quiz completion increase
500 XP
Per level milestone
50 XP
Per quest completed

Engineering Evaluation

Nightly eval
68%
Quiz Completion Rate Improvement
84%
Assessment Reliability (Cronbach α)
91%
Quiz Generation Pass Rate
86%
RAG Explanation Grounding
Headline business result68%
Quiz completion increase

Why This Is Hard

4 engineering challenges
Challenge · 01

Cold-start ability estimation

IRT CAT must converge on a reliable ability estimate in 15–25 items. Each item is selected to maximise Fisher information at the current θ estimate, making the diagnostic efficient. Learners who skip the diagnostic start at θ=0 (population mean) and the estimate updates rapidly from the first 5 interaction events.

Challenge · 02

Knowledge graph sparsity in advanced subgraphs

some advanced competency nodes have few explicitly defined prerequisite edges, creating sequencing ambiguity. Resolved by inferring implicit co-prerequisites from learner performance patterns across the cohort: nodes that are consistently failed together are treated as implicit prerequisites.

Challenge · 03

LLM quiz quality control

initial GPT-4o quiz generation had a ~12% ambiguity or answer-leakage rate without a validation layer. A GPT-4o-mini judge with a structured rubric reduced this to ~9%. Persistent failures on the same objective fall back to a curated question pool.

Challenge · 04

XP calibration

the XP system required calibration so that one level (500 XP) corresponds to approximately 8–12 hours of engaged learning. Calibrated via content time estimates (microdrills ~4 min, quests ~25 min) and adjusted against a pilot cohort to match realistic learner pacing.

Engineering Depth

5 topics · click to expand

Course material is chunked into 1,200-character sentence-boundary units and embedded at 768 dimensions. At explanation time: the learner's detected misconception + learning objective is embedded, top-k 8 chunks retrieved from FAISS, passed to GPT-4o with a grounding requirement. The system prompt explicitly states that every claim must be supported by the retrieved context. Grounding rate: ~86% (RAGAS faithfulness, internal eval on 100 explanation samples — prototype benchmark).

Key numbers
768-dimEmbedding dimension
1,200Avg chunk size (chars)
Built withAdaptive LearningRAGIRTSpaced RepetitionCompetency Systems