Get Started
Home
Topics
Search
Library
Evaluation · LLM Training · Jul 23, 2026

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

Source: research paper via Hugging Face Daily Papers
Current K–12 evals only check if a model answers exam questions, never whether it understands how the curriculum is organized. K12-KGraph extracts a textbook knowledge graph and derives SFT pairs where each sample encodes a labeled edge — 2,300 samples beat generic instruction sets 60x larger, with cross-subject transfer.
TL;DR
K12-KGraph extracts a curriculum-aligned knowledge graph from official Chinese K–12 textbooks, then uses the same graph to build both a benchmark (K12-Bench) probing prerequisites, taxonomies, and figure grounding, and an Supervised Fine-Tuning corpus (K12-Train) whose 2,300 samples beat eight general instruction datasets at matched budget.
Why It Matters
Suppose you’re shipping an AI tutor. Your model can solve a linear-equations exam question, but when a 7th-grader asks “what do I need to know first?”, it can’t reliably name arithmetic operations as the prerequisite or point to Chapter 3 where the topic actually appears. Existing K–12 evaluations like C-Eval, CMMLU, GaokaoBench, and EduEval only check whether the model answers exam questions. They never test whether it understands how the curriculum is organized. And because no training set explicitly teaches this structure, instruction-tuning on general corpora like OpenHermes or Tulu-3 doesn’t fix it either. This paper argues that structured curriculum knowledge is the missing piece for educational AI, and that a knowledge graph is the natural way to both measure it and teach it.
How It Works
The authors build one knowledge graph and derive everything from it. The graph has nine node types (Book, Chapter, Section, Concept, Skill, Experiment, Exercise, Figure, VisualElement) and fourteen edge types covering taxonomy, prerequisites, associations, experiment-verifies-concept links, exercise-tests-concept links, and how textbook figures ground concepts. Extraction runs a five-stage pipeline: MinerU OCRs the PDFs into Markdown, a splitter creates per-section files, GPT-5.2 emits nodes and edges as JSON per section with evidence citations, a hierarchical merge deduplicates across books, and a cycle-check validates that taxonomic and prerequisite subgraphs are proper DAGs. Twelve subject-qualified annotators then verify every triple, reaching Fleiss’ Fleiss’ kappa of 0.84 overall.
From that single graph they derive two things. K12-Bench takes graph neighborhoods and instantiates them as multi-select questions across five task families: Ground (exercise↔concept), Prereq (prerequisite closure and direct successors), Neighbor (related concepts via is_a or relates_to), Evidence (experiments that verify concepts), and Locate (where a topic first appears, chapter ordering). Distractors are sampled from structurally nearby but non-answer nodes (2-hop neighbors, is_a siblings), then filtered by a pedagogical LLM check. K12-Train turns the same graph into Supervised Fine-Tuning pairs: node-grounded QA teaches content (definitions, procedures, experiment conclusions), edge-grounded QA teaches structure with relation-specific templates like “Why must one learn A before B?” for prerequisites, and deterministic templates fill in exercise-assessment edges without any LLM call.
for node in graph.nodes: qa = llm_prompt(template_for(node.type), node.properties) add_to_train(qa, modality="text") for edge in graph.edges: if edge.type in DETERMINISTIC: # tests_concept, tests_skill qa = fill_template(edge) else: qa = llm_prompt(template_for(edge.type), edge.src, edge.dst) add_to_train(qa, modality="mm" if edge.uses_figure else "text")
Core Insight
The prevailing recipe for domain-adapting an LLM is to scrape or synthesize more Q&A pairs and fine-tune. This paper shows the opposite. The unit of supervision that transfers isn’t the fact, it’s the labeled relation between facts. Because every training pair traces back to a specific graph edge with an explicit relation type, the model learns how curriculum knowledge is organized rather than memorizing more content. The load-bearing evidence is not the headline benchmark lift, it’s cross-subject transfer: K12-Train contains zero humanities content yet still improves Chinese and history scores.
What They Found
The load-bearing finding is that on K12-Bench, even Gemini-3-Flash only reaches 57% exact match and Gemma-4-31B-IT reaches 46%, while Meta-LLaMA-3-8B-Instruct at 7.2% is essentially indistinguishable from random guessing at 6.7%. Prereq and Neighbor are the hardest categories across every model. This confirms current LLMs have no reliable parametric model of curriculum structure, even when they ace the exam-answering benchmarks.
The fine-tuning results then show the graph fixes it, sample-efficiently:
•
Text-only, matched 2,300-sample Supervised Fine-Tuning budget on Qwen3-4B-Base: K12-Train-Text scores 1009.96 on GaokaoBench vs 985.91 for the strongest baseline (DataFlow), and 66.76 on EduEval vs 66.70 for WizardLM. Same pattern on Llama3.1-8B-Base: 625.49 vs 593.08.
•
Cross-subject transfer: despite training only on math/physics/chemistry/biology, K12-Train-Text wins the Chinese (120.18) and Humanities Math (132.00) subject scores too. Content memorization can’t explain this.
•
Multimodal: K12-Train-Full (7,335 samples) beats WizardLM (142,759 samples) on Gaokao-MM, MDK12-Bench, and K12Vista. It’s also the only fine-tuned config that improves the base Qwen3.5-2B on K12Vista (79.95 vs 79.72), where all other datasets cause capability regression.
•
Text and multimodal supervision are complementary: K12-Train-Full consistently beats K12-Train-Text and K12-Train-MM alone.
What’s Useful
Reach for this when you’re building a tutoring product, curriculum planner, or study-path recommender for K–12, especially in the Chinese People’s Education Press curriculum. The graph itself is the interesting artifact. You can query prerequisite chains directly instead of asking an LLM to guess them, use the figure↔concept edges to retrieve textbook diagrams for a concept, or ground Retrieval-Augmented Generation over authoritative curriculum structure rather than scraped web text. If you’re doing domain Supervised Fine-Tuning more generally, the recipe transfers: extract a graph, template QA pairs per edge type, and expect better sample efficiency than adding more general instruction data.
The authors say they release the graph, benchmark, training data, and full construction pipeline, but the paper text doesn’t include a repository URL. The Chinese textbook PDFs used as input come from a public GitHub repository referenced in the appendix.
Takeaway
When domain-adapting an LLM, label the relation, not just the fact. A knowledge graph forces every training pair to carry an explicit relation type (prerequisite, taxonomy, verification), and that structural labeling is what lets a few thousand samples outperform tens of thousands of generic instruction pairs. The trick only works when your domain actually has recoverable structure. Standardized curricula, API docs, and legal statutes qualify. Open-ended creative domains do not.
Caveats
•
The graph is extracted by GPT-5.2 and verified by human annotators for a specific curriculum (Chinese People’s Education Press). Reproducing this for another curriculum requires both the extractor budget and the domain-qualified annotator pool, and the paper’s quality guarantees don’t automatically carry over.
•
The benchmark tests parametric curriculum knowledge with no graph context in the prompt. A retrieval-augmented system with access to the graph at inference time would likely score very differently, and the paper doesn’t run that comparison.
•
Gains on EduEval are small (K12-Train-Text 66.76 vs WizardLM 66.70 on Qwen3-4B-Base). The strong story is really GaokaoBench and cross-subject transfer. If your downstream eval looks more like EduEval’s mixed capability dimensions, expect modest lifts.
Topics
Don't miss new content
Log in to follow topics and personalize your feed.
By content type
Research Paper171 episodes
AI171 episodes