20 Nov 2026
Instructor-designed AI raised exam scores in a 19-university trial. After errors, Socratic practice beat better explanations
LDPro teamSTAFF
Welcome
Welcome to edition 4 of the LDPro Newsletter. At LDPro we think learning should be AI-driven, procedural, and always evolving, with people still in charge of quality. This week's research makes that concrete in two ways: who gets to design the AI, and what should happen after a learner makes a mistake. Every claim below links to its source, so you can check the detail yourself.
Research: learning evidence
Instructor-designed AI improved exam scores in a 19-university trial
A new arXiv preprint (17 Nov 2026) reports a pre-registered randomised controlled trial across 7,200 students in 55 applied STEM classes at 19 universities, over three semesters (spring 2025 to spring 2026). Instructors designed their own AI tools — 33 types, including Socratic coaching agents, practice tutors, simulations and role-play — with technical support from the research team. Classes were block-randomised to deploy immediately or after a one-term delay. Learning was measured on AI-proof, typically proctored in-person exams, using a difference-in-differences comparison with each class's own history.
The student-level effect was 0.32 SD (SE = 0.13; P = 0.019; 95% CI [0.06, 0.59]).
The class-level effect was 0.26 SD (SE = 0.12; P = 0.038; 95% CI [0.02, 0.51]).
The learning analysis used 44 class-terms and 4,768 students, after excluding classes that allowed AI on exams or that had major instructor or curriculum changes.
Engagement and course-evaluation satisfaction did not change. The estimates were small, negative and not statistically significant. Students who used the tools often rated the experience very positively, which did not match the flat official satisfaction scores.
Why it matters: this is the strongest evidence so far that AI can raise learning when frontline instructors or designers shape the tools for a specific course, rather than deploying a generic chatbot. It also reinforces a point from edition 3: don't use satisfaction or engagement as your main pilot metric, and test on assessments the AI cannot help with.
Caveats: this is a preprint and hasn't been peer reviewed. The classes were primarily applied STEM, so effects in other domains are untested. Few classes were observed on a second offering, so persistence is unknown. The trial was not powered to say which of the 33 designs worked best. Instructors had research-team technical support, which a typical L&D team may not.
Try this: give a named instructor or designer the ability to shape and revise the AI for a specific course or programme. Keep the primary outcome an AI-free assessment.
After errors, Socratic tutoring helped adults correct mistakes — personalized explanations did not
A second arXiv preprint (submitted 15 Nov 2026) reports a preregistered online experiment with 400 adults practising how to set up constraint equations. After each error, participants got the correct answer, a generic explanation, a personalized LLM explanation of their error, or a Socratic tutor that guided them through scaffolded practice.
The Socratic tutor helped participants resolve 64% of their errors (131 of 206). They scored 0.24 SD (9 percentage points) higher on practice-aligned posttest questions than the explanation-only groups (p = .048).
That advantage did not appear on a pretest-aligned question or a harder challenge question.
There was no reliable benefit of personalized explanations over generic ones, and neither type of explanation reliably beat simply seeing the correct answer.
Socratic sessions took nearly twice as long (13.3 minutes vs about 7 minutes) but were rated as more interesting, less frustrating and less costly.
Why it matters: this is adult evidence, close to workplace learning. It suggests the learner-facing value is in helping people correct errors themselves, not in writing a more precise explanation of the mistake.
Caveats: this is a preprint and hasn't been peer reviewed. It covers one task, one session, and adults recruited online. The Socratic condition bundled several features (retry, scaffolding, personalized feedback and tone), so the design cannot isolate which one drove the effect. The authors note the time cost is real, and a time-equated comparison with more practice was not run. The 0.24 SD result was just at the conventional significance threshold, and it was contrary to the authors' preregistered prediction of no benefit.
Try this: after an error, have the AI scaffold the learner back to a correct attempt rather than only explaining the mistake. Measure near transfer, and don't assume a personalized explanation is better than a good generic one.
Models
SenseNova Loop-9B: a compact Apache 2.0 model, with vendor-reported scores close to a 27B baseline
SenseTime released the official SenseNova-Loop-9B weights and code on 17 Nov 2026 (announcement). The Hugging Face model card marks the licence as Apache 2.0. Official AWQ 4-bit and GGUF builds were added on 18 Nov.
The model has about 9B parameters, reuses middle layers (32 distinct layers, 56 executions per pass), accepts text and images, and supports a 512K context. SenseTime says it is competitive with Qwen3.8-27B on some of its own agent and data-analysis scores (for example 41.36 vs 44.60 on NL2Repo, and 65.31 vs 65.01 on AIDABench). The card says 4-bit weights with an 8-bit KV cache need about 15 GiB at a 256K context. There are still no official Qwen4 open weights.
Why it matters: if the claims hold, a 9B Apache 2.0 model could be a cheaper fine-tune and inference baseline than a 27B dense model. It is not a Qwen4 substitute.
Caveats: the benchmarks are SenseTime's own, and they cover coding, agents and data analysis, not instructional design. The looped architecture uses custom code, so standard fine-tuning stacks may not support it out of the box. Memory use and quality have not been independently verified here.
Try this: if you run your own instructional-design evaluation set, add Loop-9B as a cost comparison after you confirm that your fine-tuning and serving tools can load a looped model with custom code. Keep a known baseline such as Qwen3.8-27B until official Qwen4 open weights and a licence file appear.
The takeaway for L&D teams
Two studies this week point the same way: AI can raise learning when a named instructor or designer shapes it for a specific course, and when it helps people correct their own errors rather than only explaining them. Measure that on assessments the AI cannot help with, and don't treat satisfaction as a substitute. That's the kind of always-evolving, human-overseen learning we're building at LDPro.
Sources are linked inline. Preprints and vendor-reported figures are labelled as such. The LDPro Newsletter is a digest, not an endorsement of any model or vendor.
Questions about this?
Comments are off while the community is read-only. Ask us and we reply within 48 hours.
Ask us