AI in education touches the most protected data any institution holds: student records governed by FERPA, minors' information, and disability accommodations under the ADA. A governance failure is not a fine, it is a breach of trust with families, students, and regulators. This playbook sets the guardrails schools and universities need before scaling AI: FERPA-compliant data handling, academic integrity policy, bias and equity auditing, accessibility compliance, and model transparency. It defines who approves what, how student consent works, and how to keep AI outputs explainable and contestable when a recommendation affects a learner's path.
Education AI carries the highest-trust data and the youngest users
Student records fall under FERPA, and roughly 95 percent of K-12 students are minors whose data enjoys additional protections under COPPA when they are under 13. More than 7.5 million students in US higher education report a disability, so accessibility under Section 504 and the ADA is not optional. Academic integrity concerns surged after generative AI became widely available, with the majority of faculty reporting confirmed or suspected AI-assisted misconduct in 2024. Meanwhile bias in predictive models has real stakes: a flawed retention or admissions model can systematically disadvantage low-income or first-generation students.
Governance is the license to operate. Without documented FERPA data flows, a clear academic integrity stance, equity auditing, and accessible outputs, an institution cannot scale AI safely, and a single incident can trigger a Department of Education inquiry, a civil rights complaint, or a collapse of parental trust. The goal is not to slow AI down but to make its outputs traceable, contestable, and fair before they touch a student's record.
Governance in education also has to survive scrutiny from more constituencies than most sectors face. A retention model that quietly deprioritizes a subgroup can draw a complaint under Title VI or the Office for Civil Rights, a chatbot that mishandles a disclosure of self-harm can create a duty-of-care problem, and a tool that trains on student essays can trigger both FERPA and intellectual-property disputes over who owns the work. Parents, unions, faculty senates, accreditors, and state education agencies all have standing to object. The practical answer is to make governance a named function with an accountable owner, not a policy document that no one enforces, and to require that every AI use case clears a documented checklist before it reaches a single student.
Six governance domains every education AI deployment must clear
Map each AI use case against these domains and require a documented owner and control for each before production. Any domain left without a named owner is a gap that will surface at the worst possible moment, during an incident or an audit rather than a planning review.
| Domain | Core requirement | Control and owner |
|---|---|---|
| FERPA and student privacy | Records disclosed only under a valid exception or consent | Data processing agreement, vendor review; registrar or DPO |
| Minors and COPPA | Verifiable parental consent for under-13 data use | Consent workflow, age gating; district counsel |
| Academic integrity | Clear policy on permitted AI use per assignment | Syllabus disclosure, detection review; provost or dean |
| Bias and equity | Models audited for disparate impact by subgroup | Fairness audit each term; institutional research |
| Accessibility (ADA and 504) | AI outputs usable with assistive technology | WCAG conformance test; disability services |
| Model transparency | Explainable reasoning behind consequential outputs | Provenance log, contest path; AI governance board |
Stand up governance before you scale, not after an incident
- Require a signed data processing agreement and FERPA review for every AI vendor that touches student records, with data minimization and deletion terms written in.
- Publish an academic integrity policy that states, assignment by assignment, what AI use is permitted, and disclose it in every syllabus.
- Run a disparate-impact audit each term on any model that influences admissions, retention flags, or advising, broken out by income, race, and first-generation status.
- Test every student-facing AI output against WCAG accessibility standards with real assistive technology before launch.
- Attach provenance to every consequential recommendation, source data, model, and assumptions, and give students a documented path to contest it.
How governance fails in schools and universities
- Signing up for free AI tools that ingest student data with no data processing agreement, silently violating FERPA.
- Relying on AI detection tools as proof of misconduct despite their well-documented false-positive rates against non-native English writers.
- Deploying predictive models without a subgroup fairness audit, then discovering they flag first-generation students at biased rates.
- Shipping AI chat and dashboards that screen readers cannot navigate, excluding students with disabilities and breaching the ADA.
Prove governance is working
- Share of AI tools in production covered by a signed, FERPA-compliant data processing agreement.
- Disparate-impact ratio for each consequential model, tracked by student subgroup each term.
- Accessibility conformance pass rate for student-facing AI surfaces against WCAG standards.
- Number of AI-driven decisions with complete provenance and an available contest path.
Frequently asked questions
Can free AI tools be used with student data?
Not without a signed data processing agreement that meets FERPA and, for under-13 students, COPPA. Many free consumer tools ingest inputs for training, which can constitute an unlawful disclosure of protected student records.
Are AI detection tools reliable for academic integrity?
No single detector should be treated as proof. They carry documented false-positive rates, especially against non-native English writers. Use them as one signal, pair them with clear syllabus policy, and keep a human review and appeal process.
How do we keep predictive models fair?
Run a disparate-impact audit each term on any model that influences admissions, retention, or advising, broken out by income, race, and first-generation status, and treat every flag as a prompt for a human advisor rather than an automated decision.
Related reading
Go deeper on this sector and topic.