Clonal Evolution

Yuan et al. (2026) — Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains

Properties1
tagsmethods, medium-evidence

10 min read

Knowledge-to-Verification (K2V) — Yuan et al. (2026)

Bibliographic Reference

Yuan, Z., Wang, Z., Hu, F., Chen, Z., Li, J., Li, G., Ying, J., Kong, H., Zhang, S., & Dong, N. (2026). Knowledge-to-Verification: Exploring RLVR for LLMs in knowledge-intensive domains. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026). https://doi.org/10.18653/v1/2026.acl-long.1891. Code: https://github.com/SeedScientist/K2V

Core Argument

Reinforcement learning with verifiable rewards (RLVR) has driven recent advances in LLM reasoning for mathematics and coding, but cannot be applied to knowledge-intensive domains (agriculture, law, medicine) because answers take the form of open-ended text that cannot be automatically validated, and high-quality verifiable data is scarce. K2V solves both problems: (1) fill-blank style verification constructs a knowledge graph from unstructured corpora, samples quintuples of connected triples, masks one entity, and converts the result into verifiable QA pairs — the masked entity’s name is the ground truth, verifiable by a rule-based checker; (2) checklist-style verification decomposes the intractable “is this reasoning correct?” into N binary-verifiable sub-questions, each answerable by a judge LLM, producing a dense pass-rate reward; (3) an answer-gated reward mechanism awards the reasoning reward only when the final answer is correct, anchoring logical consistency to factual accuracy and preventing reward hacking. The principle: structured knowledge is easier to verify than unstructured knowledge.

Methods

K2V operates in three stages:

Stage 1 — Fill-blank verification (data synthesis). An LLM performs named entity recognition (NER) and relation extraction (RE) on unstructured domain corpora via GraphGen (Chen et al., 2025b), constructing a KG of (head, relation, tail) triples. Quintuples — two linked triples sharing an entity, formally (e₁, r₁, e₂, r₂, e₃) — are sampled from the KG. One entity is randomly masked, and another LLM converts the masked quintuple into a natural-language fill-blank question where the masked entity’s name is the verifiable ground truth. The process requires no human annotation or seed datasets and scales to arbitrary corpora.

Stage 2 — Checklist-style verification (reasoning audit). For each QA pair, a set of general criteria G = {g₁, g₂, …, g_N} characterizing high-quality reasoning (derived from AP Course and Exam Description rubrics) is instantiated into a question-specific checklist C[x] = {c₁, c₂, …, c_k} by an LLM. A judge LLM evaluates the policy model’s reasoning process z against each criterion c_i, outputting binary scores v_i ∈ {0, 1}. The pass rate p = (1/k) Σ v_i serves as a dense reward signal. This decomposes the intractable reasoning verification task into k binary-verifiable subtasks.

Stage 3 — Answer-gated reward. Total reward R_total = R_format + R_answer + R_reason, where R_reason = 1(ŷ = y) · p. The reasoning reward equals the checklist pass rate only if the predicted answer matches the ground truth; otherwise zero. This prevents the model from learning to produce plausible-sounding reasoning that leads to wrong answers.

Evaluation. Tested on agriculture (RiceCorpus, 37.64 MB), law (DISC-Law-SFT, 20.69 MB), and medicine (shibing624-medical-pretrain, 23.51 MB) using Qwen2.5-3B/7B and Llama3.2-3B/3.1-8B backbones against four baselines (Liquid, Genie, SDR, BDS). Domain performance measured on SeedBench, CMMLU, MMLU (agriculture), LawBench (law), and MedQA (medicine). General capability retention measured on BBH, MATH-500, GSM8K, AIME2024, GPQA-Diamond. Ablation studies on all three components.

Key Findings

  1. K2V outperforms all baselines across all three knowledge-intensive domains. On Qwen2.5-7B, K2V achieves domain-average accuracies of 78.09% (agriculture), 68.47% (law), and 83.00% (medicine), outperforming the best baseline (SDR) by 3.2, 3.0, and 3.5 percentage points respectively. Results hold across both Qwen and Llama backbones and both 3B and 7B/8B scales (Table 1).

  2. K2V improves reasoning without compromising general capabilities. K2V-3B-Qwen matches or exceeds the Qwen2.5-3B-Instruct baseline on BBH (42.64 vs. 37.21), MATH-500 (65.20 vs. 66.40 — slight regression), GSM8K (84.17 vs. 85.05 — slight regression), AIME2024 (5.83 vs. 5.00), and GPQA-Diamond (32.45 vs. 31.94). This is achieved without SFT on any general or mathematical data — K2V applies RL directly to the base backbone (Table 2). Results at the 7B scale were not verifiable from the PDF extraction due to table garbling.

  3. All three components are necessary; checklist verification contributes most. Ablation studies (Table 3): removing checklist-style verification causes the largest performance drop, confirming that reasoning process verification is the primary driver of improvement. Removing fill-blank verification (replacing with unverifiable open-ended QA) degrades performance by reducing the quality of the reward signal. Removing the answer gate (awarding reasoning reward regardless of answer correctness) enables reward hacking — the model learns to produce checklist-satisfying reasoning that doesn’t lead to correct answers.

  4. Fill-blank verification converts unverifiable data into verifiable data. The law domain experiment is particularly instructive: K2V takes DISC-Law-SFT, a dataset of unverifiable QA pairs, concatenates questions and answers into unstructured text, constructs a KG from it, and synthesizes new verifiable QA pairs. This demonstrates that even corpora without ground-truth labels can be converted into verifiable training data through KG-based restructuring — a principle directly applicable to scientific literature.

  5. The checklist approach is question-specific and generalizes. A unique checklist is synthesized for each QA pair by instantiating general criteria with question-specific content. Quality assessment (Appendix F) confirms that synthesized checklists are relevant and discriminative. The general criteria are domain-agnostic (derived from educational assessment rubrics); domain specificity enters through the instantiation step where the synthesizer LLM incorporates domain context from the QA pair.

Concepts Introduced or Used

  • Fill-blank style verification: Converting KG completion tasks into natural-language fill-in-the-blank questions where the masked entity name serves as automatically verifiable ground truth. Operates on quintuples (two linked triples) rather than individual triples to provide sufficient context for complex reasoning.
  • Checklist-style verification: Decomposing the intractable task of verifying an LLM’s reasoning process into N binary-verifiable subtasks (criteria), each answered yes/no by a judge LLM. The pass rate across criteria serves as a dense reward signal.
  • Answer-gated reward mechanism: A reward design where the reasoning quality reward is awarded only when the final answer is correct. Prevents reward hacking — the model cannot earn high rewards for plausible reasoning that leads to wrong answers.
  • Quintuple: A knowledge graph subgraph consisting of two triples sharing an entity: (e₁, r₁, e₂, r₂, e₃). Provides richer context than single triples for QA synthesis.
  • RLVR (Reinforcement Learning with Verifiable Rewards): A training paradigm where model outputs are compared against automatically verifiable ground truth, enabling self-exploration without human feedback. Previously confined to mathematics and coding; K2V extends it to knowledge-intensive domains.
  • GraphGen: LLM-driven KG construction pipeline: NER → RE → entity linking → KG assembly (Chen et al., 2025b). Used by K2V as the first stage of data synthesis.

Entities Referenced

  • K2V (Knowledge-to-Verification): The proposed framework. Code available at github.com/SeedScientist/K2V.
  • GraphGen: KG construction module (Chen et al., 2025b). Performs NER, RE, and entity linking via LLM prompting.
  • Qwen2.5 (3B, 7B) and Llama3 (3.2-3B, 3.1-8B): Backbone LLMs used for evaluation.
  • RiceCorpus: Agricultural domain corpus (37.64 MB) used for agriculture experiments.
  • DISC-Law-SFT: Legal domain dataset of unverifiable QA pairs, converted to verifiable data by K2V.
  • shibing624-medical-pretrain: Chinese medical corpus from online encyclopedias and textbooks.
  • SeedBench, CMMLU, MMLU, LawBench, MedQA: Domain-specific evaluation benchmarks.
  • BBH, MATH-500, GSM8K, AIME2024, GPQA-Diamond: General capability benchmarks.

Limitations

  • Model scale. Experiments limited to 3B–8B parameter models due to computational constraints. Scalability to 70B+ models unverified. The authors acknowledge this as the primary limitation.
  • Domain scope. Tested on agriculture, law, and medicine — domains with relatively stable factual knowledge. Applicability to rapidly evolving scientific domains (where facts are contested and the knowledge frontier moves quickly) is untested.
  • KG construction quality unevaluated. The GraphGen step (NER+RE via LLM prompting) is assumed correct. Hallucinated triples in the KG would produce QA pairs with incorrect ground truth, undermining the entire verification pipeline. No evaluation of KG precision/recall is provided.
  • LLM-judge regress. The checklist judge is itself an LLM. If the judge shares blind spots with the policy model, it may approve flawed reasoning. No calibration of judge competence against human experts is provided.
  • Static KG assumption. K2V constructs a KG once from a static corpus. It does not address continuous KG updating as new documents are added — a critical requirement for scientific knowledge management where the literature grows continuously.
  • Quintuple decomposition may lose context. Complex scientific claims often involve n-ary relations that don’t decompose cleanly into binary triples. Forcing such claims into the quintuple format may lose qualifying conditions, uncertainty quantification, or contextual dependencies.

Relevance to Clonal Evolution

K2V, like Bian (2025), contains zero biology. It earns its place in the corpus as the verification layer in the wiki’s four-layer knowledge infrastructure stack. Its relevance is architectural, not domain.

K2V completes the stack

The wiki’s knowledge infrastructure now has a four-layer architecture (see docs/superpowers/specs/2026-07-11-kg-construction-stack.md):

K2V fills the stack’s most critical gap: automated verification. Our wiki has manual verification (Phase 2 source verification bridge, L1/L2/L3 audits). K2V provides the architectural blueprint for automating parts of this — particularly claim verification against source text and reasoning quality audit.

Structural parallels with existing wiki infrastructure

K2V componentWiki analogueStatus
KG construction from corporaIngest pipeline (Phase 1): subagents extract entities, relations, and claims from PDFs into structured source summariesManual, ~1 paper per subagent
Fill-blank QA synthesisSource verification bridge (Phase 2): key claims cross-referenced against extracted PDF textManual, one agent per source. 28 sources unverified
Checklist-style verificationL1/L2/L3 audit criteria: wikilinks resolve, frontmatter valid, claims traceable to sources, no contradiction driftL1 automated (lint); L2/L3 manual
Answer-gated rewardVerification bridge logic: claim verified only if matching passage found in source textSame principle, manually applied
General criteria → question-specific checklistAudit criteria → page-specific verification questionsNot implemented — audits use generic criteria

What K2V enables for this wiki

  1. Automated claim verification at scale. K2V’s fill-blank approach, applied to our wiki: construct a KG from verified source summaries (entities = genes, drugs, cell types; relations = “deaminates”, “upregulated in”, “confers resistance to”), synthesize fill-blank QA pairs, verify new claims by checking whether the LLM can correctly predict the masked entity given the surrounding context. A claim that contradicts the KG would produce systematically wrong predictions — automatically detectable.

  2. Checklist-based audit automation. Our L2/L3 audits currently use generic criteria (“trace every claim to its source”). K2V’s checklist approach would enable question-specific criteria: for a claim about APOBEC3B nuclear localization, the checklist could include “Does the reasoning cite the primary evidence (Burns et al., 2013, immunofluorescence) rather than a review?”, “Does it distinguish APOBEC3B from APOBEC3A localization?”, “Does it acknowledge that nuclear localization was demonstrated in cell lines, not primary tissue?”

  3. Wiki self-consistency testing. The most ambitious application: treat the entire wiki as a KG, synthesize QA pairs from it, and test whether new ingestions are consistent with existing knowledge. A new paper claiming “APOBEC3G is the primary mutator in breast cancer” would fail fill-blank verification against a KG where the established answer is APOBEC3B/APOBEC3A — flagging a potential contradiction for human review. This operationalizes the contradiction registry as an automated test.

  4. The answer-gated principle for synthesis. K2V’s answer-gated reward — “reasoning credit only if the answer is correct” — is a principle our concept synthesis already follows implicitly: we don’t publish synthesis until sources are verified. Making this explicit as a design principle would strengthen the ingest gate.

Limitations of K2V for our domain

  • The KG construction problem. K2V assumes GraphGen produces correct KGs. In clonal evolution, where facts are contested and papers disagree, KG construction IS the hard problem — not a solved preprocessing step. Our manual source summaries with human-reviewed claims are higher quality than any automated KG construction, but slower.
  • Static KG vs. evolving knowledge. K2V’s KG is built once. Our wiki grows continuously. The framework would need extension to support incremental KG updating and contradiction detection when new triples conflict with existing ones.
  • Domain checklist criteria needed. K2V’s general criteria come from educational assessment rubrics. For clonal evolution, we’d need to develop domain-specific criteria: evidence hierarchy awareness, distinction between correlation and causation, detection limit acknowledgment, primary-vs-review source discrimination, uncertainty quantification.

Graph View

Backlinks