A practical guide for teachers, principals, and district leaders — built around one question: what is the AI actually being used for? Globally informed; this is guidance, not legal advice.
The first question about a classroom AI tool isn't "Is it allowed?" — it's "What is the AI being used for?" In plain terms: the risk is low when a teacher uses AI to draft a generic worksheet. It is much higher when AI scores student work, recommends interventions, flags cheating, or influences discipline.
A teacher using AI to draft a generic lesson plan is not doing the same thing as a student-facing tutor — and both are very different from a system that scores essays, recommends interventions, or flags misconduct. Sorting a tool into the right bucket is the first move. And regulators are starting to treat the high-stakes buckets differently: under the EU AI Act, certain education uses — admissions, evaluating learning outcomes, assessing the appropriate level of education, and monitoring during tests — are placed in a high-risk category with heavier obligations.
The AI use-case risk ladder
A simple way to triage any tool before adopting it — four levels, from lowest risk to the ones that need the most scrutiny:
Lower risk — Teacher productivity. Use AI for generic plans, examples, and rubric drafts. Do not include identifiable student data. The teacher reviews, edits, and owns the output.
Medium risk — Student-work support. Use AI for anonymized feedback drafts or resource suggestions. Requires privacy review, anonymization, human checking, and no final automated decision.
High risk — Assessment and decisions. Scoring, grading, ranking, placement, discipline. Requires formal approval, accuracy/bias testing, documented purpose, human accountability, and an appeal path.
Highest scrutiny — Monitoring and integrity. Proctoring, cheating detection, behavior or attention flags. Requires transparency, false-positive review, due process, and a strict necessity test.
What teachers can safely use AI for
The lowest-risk uses involve no identifiable student data and keep the teacher firmly in control of the output. UK government guidance and UNESCO's global guidance both warn schools not to enter personal or sensitive student data into general-purpose AI tools unless they have appropriate safeguards, approvals, and a clear data-protection basis.
Draft a generic lesson plan aligned to curriculum standards.
Generate practice questions or worked examples.
Rewrite instructions at a simpler reading level.
Brainstorm classroom activities.
Draft a rubric the teacher then edits and owns.
Three quick examples: Safer — "Create five practice questions on photosynthesis for Grade 7." Riskier (needs care) — "Here is Maria's science assignment — write feedback based on her IEP." Don't, without approval — "Grade these essays and rank the students."
What teachers should be careful with
These uses can be valuable but need privacy review, anonymization, and human checking. In the EU and UK, putting student personal data into an AI tool also requires a clear data-protection basis.
Uploading student writing for feedback drafts — even with names removed, writing can be identifying.
Asking AI to personalize work for a named student, or a student with a disability or specific learning need.
Summarizing class performance or recommending practice groups.
Drafting parent communications about a specific child.
Grading, feedback, and recommendations
This is where the accountability burden is highest — and where AI should assist, not decide. AI can help draft feedback, but it shouldn't be the final grader unless the tool has been formally approved, validated for your student population, and kept under human accountability. Don't upload identifiable student work into unapproved consumer chatbots. And watch for bias: AI-text detectors have been shown to misclassify non-native English writers' work as AI-generated far more often than native writers', so check outputs for unfair impact on multilingual students, students with disabilities, and students who write in non-standard ways.
The human-oversight rule: AI can assist a teacher, but it should not replace the accountable adult. Any grade, placement, intervention, discipline, or high-stakes recommendation should remain a human decision — with the AI's role documented and reviewable.
The vendor checklist (for the teacher bringing a tool forward)
Before you adopt any AI tool, get clear answers to these questions and bring them to your district's privacy officer or approval process:
Training: "Do you use identifiable student inputs to train or improve your models?" For most K–12 uses the answer you want is no, in writing. If you're unsure which one a vendor means, treat it as identifiable.
Retention: "How long is data kept, and can we set or shorten that?"
Deletion: "Can we require deletion of all student data on request and at contract end?"
Sub-processors: "Who else processes the data — which model and cloud providers?"
Purpose limits: "Is the data used only to provide this service — never for advertising, profiling, or sale?"
Age gating: "Is the tool appropriate for our students' ages, and how do you handle under-13 users?"
The agreement: "Will you sign our district's Data Privacy Agreement?" A refusal is a red flag.
Status: "Is the tool on an approved/vetted list, or independently privacy-evaluated?"
What principals and district leaders should do
Procurement is where the real protection happens. Frameworks like the NIST AI Risk Management Framework offer governance language for this. Before a tool reaches classrooms, a leader or district team should be able to answer:
What exact educational purpose does this tool serve, and is it teacher-, student-, or decision-facing?
What student data does it collect, process, infer, or store — and does it train models on that data (can that be disabled, contractually and technically)?
Where is data stored, and which country's laws apply? Who are the sub-processors?
Can parents and schools request access, correction, deletion, or export?
Does the tool score, rank, profile, recommend, or monitor students — and is a human required before any grade, intervention, placement, or discipline?
Has it been tested for accuracy, bias, accessibility, age-appropriateness, and safety — and is there a pilot, opt-out path, parent-communication plan, and review date?
Will the vendor sign the district's DPA (or local equivalent), and is the tool on an approved list?
A global legal and policy snapshot
The names of the laws differ by country, but the core questions are the same: what student data is collected, who controls it, whether it trains models, whether the AI makes or influences decisions about students, and whether a human stays accountable. This is not a substitute for local legal advice — use it to know which office or framework to ask about.
For U.S. schools specifically: FERPA permits sharing records with a vendor only under the "school official" exception (institutional service, the school's direct control, use only for the authorized purpose); COPPA requires verifiable parental consent for under-13s, which a school may authorize only for educational, non-commercial use; California's SOPIPA bars using K–12 student data for advertising, profiling, or sale; and the 2023 FTC action against Edmodo shows what happens when a vendor pushes its consent duties onto schools.
Getting started: three moves for school and district leaders
If you lead a school or district and do nothing else this month, take these three steps to get a handle on AI already in your classrooms:
Inventory every AI tool currently used by teachers or students.
Sort each tool into the risk ladder: productivity, learning, decision support, or monitoring.
Pause any tool that grades, ranks, profiles, monitors, or recommends action about students until it has had a formal review.
📎 Free download: "The Classroom AI Vendor Checklist" — the risk ladder plus the vendor and leadership questions, ready to bring to your district's approval process.
What parents can ask
Parents can ask their child's school four simple things: which AI tools are used, what student data those tools process, who approved them, and whether AI affects grades, recommendations, or discipline. Parents also retain legal rights to their child's school records — under FERPA in the U.S., and under data-protection law in the UK and EU.
Sources
UNESCO — Guidance for generative AI in education and research (2023). https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research
EU AI Act — high-risk classification of certain education uses. https://artificialintelligenceact.eu/
U.K. Department for Education — generative AI in education guidance. https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education
NIST — AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework
U.S. Dept. of Education — FERPA. https://studentprivacy.ed.gov/
U.S. FTC — COPPA guidance for schools. https://www.ftc.gov/business-guidance/privacy-security/childrens-privacy
U.S. FTC — action against Edmodo (2023). https://www.ftc.gov/legal-library/browse/cases-proceedings/edmodo-llc
This is general guidance for educators and school leaders, not legal advice — confirm specifics with your district or data-protection lead. Disclosure: Hossein works in AI and builds AI-related products. AI by Age takes no AI-vendor sponsorships. Full disclosure on our About page.