Research · Measurement Science

Method vs. Mastery

Course grades increasingly measure compliance and executive function; standardized assessment measures knowledge. At population scale, the two diverge — and the gap falls hardest on neurodivergent students.

Gate: PassedCRDC 2017–18 universePeer-reviewed anchor

The finding

Two measures of the same student frequently disagree: the teacher-assigned course grade and the standardized assessment of mastery. The divergence is not random noise. Factor-analytic work on nationally-representative student samples shows that teacher-assigned grades load on a distinct “Success at School Factor” — conscientiousness, engagement, and executive function — that is separate from, and only modestly correlated with, tested achievement (Bowers 2011; Willingham, Pollack & Lewis 2002; Brookhart et al. 2016). In plain terms: the grade and the mastery measure are measuring different things, and the gap is largest for students whose executive function is impaired.

Why it matters. Where executive function is not the competency being taught, a completion-weighted grade records the disability, not the learning — and that is exactly the student profile an AI-era economy most needs (see the Deep-Thinker Thesis).

Population evidence — CRDC 2017–18

If grading penalizes an executive-function/behavior profile, the institutional signature should appear in exclusionary discipline. It does. Using the full Civil Rights Data Collection universe (US Department of Education, Office for Civil Rights; all 52 reporting jurisdictions), students with disabilities (IDEA + Section 504) are 15.9% of enrollment (8.11M of 50.9M students) yet are over-represented in every exclusionary-discipline and physical-intervention outcome. Risk ratio = the rate for students with disabilities ÷ the rate for non-disabled students.

OutcomeSWD share of outcomeRisk ratio
Out-of-school suspension29.1%2.17×
In-school suspension25.5%1.81×
Expulsion26.1%1.87×
Referral to law enforcement30.8%2.35×
School-related arrest30.9%2.36×
Seclusion78.5%19.3×
Physical restraint81.5%23.3×
Any restraint or seclusion79.4%20.4×
Corporal punishment19.5%1.28×

Corporal punishment shows the smallest disparity (near parity) — reported here for honesty; not every measure is extreme. Per-jurisdiction figures (all 52) are computed and available to research partners.

Method

Full CRDC universe via the Urban Institute Education Data API, page-paginated across all jurisdictions; sex-total and race-total rows only (no double-count); CRDC negative suppression-codes excluded; students with disabilities = IDEA + Section 504. These figures reproduce the published OCR/GAO topline — students with disabilities suspended at roughly twice the rate of peers, and roughly four-in-five of all restraint/seclusion (cf. GAO-19-243). Source data and the per-jurisdiction breakdown are reproducible from the published method.

Achievement context — SEDA

The Stanford Education Data Archive establishes that academic achievement is measurable at population scale and structurally unequal: state-pooled, economically-disadvantaged students score roughly 2.2 grade-levels below non-disadvantaged peers in math, with comparable gaps by group (SEDA v4.1, grade-cohort-standardized). SEDA carries achievement, not course grades and not a disability subgroup — so it supports the “mastery is measurable; the mis-measurement is the problem” context, while the grade-vs-mastery divergence itself rests on the peer-reviewed factor-analytic literature above.

The constructive answer

The fix is not lower standards — it is honest measurement: score demonstrated mastery directly, and report work-habits separately. Trellison's mastery-based math program (algebra through calculus, validated by a deterministic computer-algebra equivalence checker) is the working proof that rigor can be measured engagingly without softening the bar.

References & gate status

Gate: PASSED for publication. Each population claim is either computed from federal universe data (CRDC) or supported by peer-reviewed, nationally-representative research (Bowers 2011; SEDA). Per the Trellison standard: per-subgroup, falsifiable, disconfirmation reported.